Lost in Translation: The Value of Verbalizations in Interpreting 3D Computer-Aided Design Workflows

Kathy Cheng, Jo Vermeulen, George Fitzmaurice, Justin Matejka

January 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI)

Abstract

AI assistants are transforming creative and knowledge domains, holding similar promise for mechanical design via 3D CAD software. Yet, current AI assistance for CAD relies on geometry or command history, lacking rich design intent. We investigate think-aloud computing as a lightweight approach to capture designers’ spoken intent and inform how future AI assistance could leverage this to provide in-situ feedback. Through a three-part study with 10 designers and 10 experts, we (1) recorded designers’ think-aloud verbalizations during 3D modelling, (2) compared expert feedback with and without think-aloud recordings, and (3) interviewed the original designers to evaluate feedback quality. Findings show that verbalizations surface rationale, future actions, and challenges — insights absent from geometric and command data — that enable feedback attuned to designers’ goals. By harnessing think-aloud data, we uncover when to intervene, what to prompt, and characteristics of effective feedback, paving the way for context-aware AI assistance for CAD.

DOI PDF

Figures

Figure 1: Overview of our three-part user study. Part 1is a 45-minute think-aloud study with 10 designers completing a CAD modelling task. Part 2 is a 90-minute interview with 10 experts reviewing and providing feedback on the designers’ recordings under two audio conditions (Think-Aloud and No- Think-Aloud). Part 3 is a 60-minute follow-up interview with the original designers, where they rated the expert feedback.

Figure 2: CAD models that designers worked on in the Tereate task (top) and Teontinue task (bottom).

Figure 3: Visualization of designers’ verbalizations during the 30-minute modelling session. The timeline plot (left) shows the distribution of verbalization categories over time for each designer, grouped by task Tcreate (top) and Teontinue (bottom). The stacked bar chart (right) aggregates the total utterances per category, with white numbers indicating the number of utterances in the most frequent category for each designer.

Figure 4: Percentage of feedback rated as relevant (top) and helpful (bottom) across four feedback topics: software, work- flow, product, and designer. Feedback was collected under two task conditions: Tcreate (circles), Teontinue (squares); and two audio conditions: Think-Aloud (arrowheads), No-Think- Aloud (circle or square). The arrow points in the direction from No-Think-Aloud to Think-Aloud. Colours indicate which condition was rated higher: Think-Aloud (teal), No- Think-Aloud (red). Sample sizes for each task and feedback topic are also included on the right.

Figure 5: Designer participants’ ratings of the expert feed- back sessions on relevance, helpfulness, actionability, and potential impact on their design process.

Figure 6: Designer participants’ preferred feedback session. The feedback session from the Think-Aloud expert was the preferred session most often.

Figure 7: Designer participants’ attitudes toward think-aloud computing, measured immediately after the design task and after receiving expert feedback (shown in slopegraphs). Teal arrows indicate increases in agreement, red arrows indicate decreases, and grey arrows indicate no change. Jitter has been added to reduce overlap between lines. Each arrow is labelled with the designer ID (D1-D10).

Figure 8: Framework illustrating how designers’ verbalizations are valuable for interpreting a designer’s CAD work. The nested blue ovals represent what the designer knows (larger oval) and what the designer says (smaller oval). The nested yellow ovals represent what an agent needs to know about the designer’s process and rationale (larger oval) and what an agent asks for to build understanding in order to provide feedback for the designer (smaller oval). These oval boundaries represent the findings from our study, while each shaded intersection highlights an opportunity that we identify and discuss for future think-aloud computing systems (A-F, discussed in Sections 5.1.1- 5.1.6). The blank regions (what the designer knows but does not say + what an agent needs to know but does not ask for) are also valid opportunities, but out of the scope of this work.

BibTeX

@inproceedings{10.1145/3772318.3791084,
author = {Cheng, Kathy and Vermeulen, Jo and Fitzmaurice, George and Matejka, Justin},
title = {Lost in Translation: The Value of Verbalizations in Interpreting 3D Computer-Aided Design Workflows},
year = {2026},
isbn = {9798400722783},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3772318.3791084},
doi = {10.1145/3772318.3791084},
abstract = {AI assistants are transforming creative and knowledge domains, holding similar promise for mechanical design via 3D CAD software. Yet, current AI assistance for CAD relies on geometry or command history, lacking rich design intent. We investigate think-aloud computing as a lightweight approach to capture designers’ spoken intent and inform how future AI assistance could leverage this to provide in-situ feedback. Through a three-part study with 10 designers and 10 experts, we (1) recorded designers’ think-aloud verbalizations during 3D modelling, (2) compared expert feedback with and without think-aloud recordings, and (3) interviewed the original designers to evaluate feedback quality. Findings show that verbalizations surface rationale, future actions, and challenges — insights absent from geometric and command data — that enable feedback attuned to designers’ goals. By harnessing think-aloud data, we uncover when to intervene, what to prompt, and characteristics of effective feedback, paving the way for context-aware AI assistance for CAD.},
booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
articleno = {1422},
numpages = {22},
keywords = {Think-Aloud Computing, Product Design, Human-AI Collaboration, Creativity Support Tools, Design Intent},
location = {
},
series = {CHI '26}
}