YouMove: Enhancing Movement Training with an Augmented Reality Mirror

Fraser Anderson, Tovi Grossman, Justin Matejka, George Fitzmaurice

January 2013 · Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST)

Abstract

YouMove is a novel system that allows users to record and learn physical movement sequences. The recording system is designed to be simple, allowing anyone to create and share training content. The training system uses recorded data to train the user using a large-scale augmented reality mirror. The system trains the user through a series of stages that gradually reduce the user's reliance on guidance and feedback. This paper discusses the design and implementation of YouMove and its interactive mirror. We also present a user study in which YouMove was shown to improve learning and short-term retention by a factor of 2 compared to a traditional video demonstration.

DOI PDF

Figures

Figure 1: YouMove allows users to record and learn physical movement sequences. An augmented reality mirror provides graphic overlays for guidance and feedback. Note that for this photo the virtual vi

Figure 2: Overview of system design showing projector, layered screen, dynamic lighting, Kinect and user location.

Figure 3: The editing interface allows authors to specify keyframes and global movement parameters. Each keyframe specifies the important joints for that moment, and can be associated with a recording

Figure 4: The movement gallery allows users to change profiles, query by example, and select a movement.

Figure 5: Left) Stage selection interface allowing users to begin one of the unlocked stages. Right) Demonstration stage.

Figure 6: The posture guide requires trainees to maintain a stable posture, matching the position of the trainer. Errors in joint position are indicated by red circles. The callout (top right of figur

Figure 7: The movement guide encourages the trainee to match the trainer’s movements at full speed, using green ribbons (inset) to cue upcoming movements.

Figure 8: The Post-Stage Feedback screen, showing trainees their overall score, average position (skeleton), and video for each keyframe.

Figure 9: Performance on each trial, for each of the conditions, averaged over all 8 subjects.

Figure 10: Pre-test, training, and post-test scores for the YouMove and Video conditions.

Figure 11: Frequency of use for each guide type for all participants during training with the YouMove system.

BibTeX

@inproceedings{10.1145/2501988.2502045,
 abstract = {YouMove is a novel system that allows users to record and learn physical movement sequences. The recording system is designed to be simple, allowing anyone to create and share training content. The training system uses recorded data to train the user using a large-scale augmented reality mirror. The system trains the user through a series of stages that gradually reduce the user's reliance on guidance and feedback. This paper discusses the design and implementation of YouMove and its interactive mirror. We also present a user study in which YouMove was shown to improve learning and short-term retention by a factor of 2 compared to a traditional video demonstration.},
 address = {New York, NY, USA},
 author = {Anderson, Fraser and Grossman, Tovi and Matejka, Justin and Fitzmaurice, George},
 booktitle = {Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology},
 doi = {10.1145/2501988.2502045},
 isbn = {9781450322683},
 keywords = {guidance., full body, motor learning, learning, augmented reality, 3d, training, movement},
 location = {St. Andrews, Scotland, United Kingdom},
 numpages = {10},
 pages = {311–320},
 publisher = {Association for Computing Machinery},
 series = {UIST '13},
 title = {YouMove: Enhancing Movement Training with an Augmented Reality Mirror},
 url = {https://doi.org/10.1145/2501988.2502045},
 year = {2013}
}