Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture

Rebecca Krosnick, Fraser Anderson, Justin Matejka, Steve Oney, Walter S. Lasecki, Tovi Grossman, George Fitzmaurice

January 2021 · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI)

Abstract

When users complete tasks on the computer, the knowledge they leverage and their intent is often lost because it is tedious or challenging to capture. This makes it harder to understand why a colleague designed a component a certain way or to remember requirements for software you wrote a year ago. We introduce think-aloud computing, a novel application of the think-aloud protocol where computer users are encouraged to speak while working to capture rich knowledge with relatively low effort. Through a formative study we find people shared information about design intent, work processes, problems encountered, to-do items, and other useful information. We developed a prototype that supports think-aloud computing by prompting users to speak and contextualizing speech with labels and application context. Our evaluation shows more subtle design decisions and process explanations were captured in think-aloud than via traditional documentation. Participants reported that think-aloud required similar effort as traditional documentation.

DOI PDF

Figures

Figure 1: Participants’ ratings of how useful the information they shared was, by domain and condition. The 7-point Likert scale results (between 1-Not Useful, and 7-Extremely Useful) have been aggregated into three categories: Not Useful (1, 2), Moderately Useful (3, 4, 5), and Very Useful (6, 7).

Figure 2: Participants’ ratings of how confident they were that they captured all the important information to share. The 7- point Likert scale results (between 1-Very Not Confident, and 7-Very Confident) have been aggregated into three categories: Not Confident (1, 2, 3), Neutral (4), and Confident (5, 6, 7).

Figure 3: Top 20 categories by count (i.e., all categories with 5 or more speech items), ordered by concurrent to retrospective occurrence ratio. See full data in supplementary material.

Figure 4: Several possible designs for the user interface of a think-aloud computing system.

Figure 5: Visual widget that allows people to reflect on the amount they have spoken about different categories of in- formation. Wedges fill up as that element is spoken about, and subtle prompts encourage the user to speak about a topic. The annotations indicate which words will ensure au- tomatic classification into those categories.

Figure 6: Live archive window, where users can review recorded speech, screen recording, and meta-data from past sessions, as well as the current capture session.

Figure 7: System overview of the implemented think-aloud computing system

Figure 8: Ratings of efort it took to document knowledge, for think-aloud and traditional conditions. The 7-point Likert scale results (between 1-Very Low Efort, and 7-Very High Efort) have been aggre

BibTeX

@inproceedings{10.1145/3411764.3445066,
 abstract = {When users complete tasks on the computer, the knowledge they leverage and their intent is often lost because it is tedious or challenging to capture. This makes it harder to understand why a colleague designed a component a certain way or to remember requirements for software you wrote a year ago. We introduce think-aloud computing, a novel application of the think-aloud protocol where computer users are encouraged to speak while working to capture rich knowledge with relatively low effort. Through a formative study we find people shared information about design intent, work processes, problems encountered, to-do items, and other useful information. We developed a prototype that supports think-aloud computing by prompting users to speak and contextualizing speech with labels and application context. Our evaluation shows more subtle design decisions and process explanations were captured in think-aloud than via traditional documentation. Participants reported that think-aloud required similar effort as traditional documentation.},
 address = {New York, NY, USA},
 articleno = {199},
 author = {Krosnick, Rebecca and Anderson, Fraser and Matejka, Justin and Oney, Steve and S. Lasecki, Walter and Grossman, Tovi and Fitzmaurice, George},
 booktitle = {Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems},
 doi = {10.1145/3411764.3445066},
 isbn = {9781450380966},
 keywords = {knowledge capture, Think-aloud, process, documentation},
 location = {Yokohama, Japan},
 numpages = {13},
 publisher = {Association for Computing Machinery},
 series = {CHI '21},
 title = {Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture},
 url = {https://doi.org/10.1145/3411764.3445066},
 year = {2021}
}