Pointing gestures accelerate collaborative problem-solving on tangible user interfaces

Authors

Anastasiou D., Maquil V.

Reference

Journal on Multimodal User Interfaces, vol. 19, n° 1, pp. 75-92, 2025

Description

Collaboration is included in the Learning and Innovation skills of the 21st Century. Collaborative problem-solving represents the interaction of two dimensions i) complex problem-solving and ii) collaboration. Technology-based assessment of collaborative problem-solving should focus on both dimensions. We ran user studies with 66 participants at three secondary schools in Luxembourg and Belgium to observe the gestural user behaviour of triads while solving a collaborative problem on a tangible user interface (TUI). Social interactions and embodiment by using gestures are important collaboration channels. Our main objective is the relation between the usage of gestures with collaboration and complex problem-solving performance. Our apparatus to test collaborative problem-solving is a tangible tabletop and a micro-world about power plants. We analysed the videos manually and found correlations between gestures, complex problem-solving performance, and user experience. The results showed that pointing gestures and adaptors significantly correlate with response time of problem-solving. Our results on user experience showed that the use of a TUI was regarded as a novel and straightforward solution that many people could learn to use very quickly. We suggest gesture performance to be considered as one indicator of collaboration.

Link

doi:10.1007/s12193-024-00448-7