Design of Sketch-based Image Search UI for Finger Gesture

With the progress of sensor technology, many new input methods using pattern recognition for touch and speech are incorporated into various computer devices. Although several GUI (Graphical User Interface) elements such as textbox, button, slider, selector, radio button, and checkbox are provided in HTML form for most Web browsers, no GUI elements are designed for gesture-based input. Using gesture as input would enable a user to operate GUI elements easily interacting with the Web browser. As a case example in this study, we design a sketch-based image search user interface for finger gesture on a Web browser. In particular, we focus on incorporating gestures of "grab" and "pinch" to Web GUI, and discuss the usability of our approach based on comparison with the conventional input approaches using keyboard, mouse, and touchscreen.