Defended 28 avril 2017

Abstract

Due to the popularization of the touch screen devices, nowadays people are used to conduct the human-computer interactions with touch gestures. However, limited by current studies, users can use only simple multi-touch gestures to execute only simple manipulations such as rotation, translation, scaling, with most of time one user even if adapted devices are now available. The work reported here concerns the expanding usage of multi-touch gestures, that make them available at the same time for more shortcut commands (called indirect commands, as copy, past, …), more manipulation commands (called direct commands like zoom or rotation) and in the context of multiple users on the same screen.
For this purpose, we analyze the shape of the gesture’s motion trajectories and the temporal and spatial relations between trajectories in order to characterize a multi-touch gesture. We propose a graph modeling to characterize these motion features and develop a graph based analysis and recognition system. To resolve the conflict between interface manipulation and shortcut command inputs, we study and validate an early recognition strategy for multi-touch gesture. We built a reject option based multi-classifier early recognition system to recognize multi-touch gestures in early stage.
To set-up, train and validate our systems, we built MTGSet, a multi-touch gesture dataset formed by 7938 gestures from 41 different classes collected in isolated contexts and MUMTDB a dataset of gestures collected in a real multi-user usage case of diagram drawing. The experimental results prove that our system can well recognize multi-touch gestures in these different situations.

Examinators

• Mme Nicole Vincent, Professeur des universités, Université Paris Descartes, Rapporteur
• Mme Véronique Eglin, Professeur des universités, INSA de Lyon, Rapporteur
• M. Hubert Cardot, Professeur des universités, Université François-Rabelais de Tours, Examinateur
• M. Harold Mouchère, Maître de Conférences, Université de Nantes, co-encadrant de thèse
• M. Eric Anquetil, Professeur des universités, INSA de Rennes, directeur de thèse
• M. Chritian Viard-Gaudin, Professeur des universités, University of Nantes, directeur de thèse