Accessibility, extensibility, and accuracy in mobile device interaction

Abstract

Direct touch is the major interaction model of modern mobile touchscreen devices. Although it is praised for its intuitiveness and input speed, its suffers from three major problems, which are inaccessibility, inextensibility, and inaccuracy, due to the limited control space of touch interaction. In this thesis, we present three contributions, each addressing one of the three major problems: 1. For the problem of inaccessibility, which refers to the inability of interacting in the distant out-of-reach screen area when using one-handed or two-handed thumb interactions, we present BezelCursor. It is a novel one-handed thumb interaction technique that improves screen accessibility in the context of target acquisition. It combines bezel-initiated interaction and pointing gesture to control a screen cursor. Unlike the existing techniques, it requires no explicit mode-switching to invoke and can be smoothly used together with commonly adopted interaction styles such as direct touch and dragging. Our user studies show that BezelCursor requires less grip adjustments, and is more accurate or faster than the state-of-theart techniques when using a fixed secure grip. 2. For the problem of inextensibility, which refers to the limited number of supported interaction methods, we motivate mobile interface designers to use bezelinitiated gestures for implicit mode-switching to support multiple touch interaction techniques in a limited space by presenting findings of an exploration of the design space of bezel-initiated gestures. It consists of a guessability study, which elicits end-user touch gestures on mobile devices. While previous studies give full freedom to the participants for gesture design, we apply a constraint that gestures must be initiated from the bezel, to ensure they are compatible with the mainstream gestures like dragging and pitching, and are highly detectable. We analyze the consensus among our participants, and reveal a set of user-defined bezel-initiated gestures including a number of novel designs. This serves as an important foundation in designing and customizing bezel-initiated gestures for extending existing interaction techniques. 3. For the problem of inaccuracy, which refers to the offset between userintended touch position and the system-detected touch position, we present the idea of simultaneous co-refinement of multiple touch paths. We identify that inaccurate touch input is particularly problematic when drawing in a limited control space. It is because the canvas cannot be zoomed in too much or else the user will lose the overview of the drawing. We utilize the co-refinement idea to develop an assistive image tracing system, EZ-Sketching, to automatically refine sketch lines roughly traced over single images based on the image features being traced. Unlike existing edge snapping methods, which aim at optimizing individual strokes, we show that a co-refinement of multiple nearby strokes results in drawings that are more accurate. Our user studies confirm that the refined sketches show closer resemblance to the traced images and are often aesthetically more pleasing.

Date of Award	15 Jul 2015
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Hongbo FU (Supervisor) & Jeffrey SHAW (Co-supervisor)