Justin Matejka
k j

Deploying CommunityCommands: A Software Command Recommender System Case Study

Wei Li, Justin Matejka, Tovi Grossman, George Fitzmaurice
January 2015 · AI Magazine (AI Magazine)
Deploying CommunityCommands: A Software Command Recommender System Case Study

Abstract

In 2009 we presented the idea of using collaborative filtering within a complex software application to help users learn new and relevant commands (Matejka et al. 2009). This project continued to evolve and we explored the design space of a contextual software command recommender system and completed a six‐week user study (Li et al. 2011). We then expanded the scope of our project by implementing CommunityCommands, a fully functional and deployable recommender system. CommunityCommands was a publically available plug‐in for Autodesk's flagship software application AutoCAD. During a one‐year period, the recommender system was used by more than 1100 users. In this article, we discuss how our practical system architecture was designed to leverage Autodesk's existing customer involvement program (CIP) data to deliver in‐product contextual recommendations to end users. We also present our system usage data and payoff, and provide an in‐depth discussion of the challenges and design issues associated with developing and deploying the software command recommender system. Our work sets important groundwork for the future development of recommender systems within the domain of end user software learning assistance.

DOI PDF

Figures

Figure 1. The Number of Commands in AutoCAD.
Figure 2. CIP Enrollment Interface in AutoCAD 2012.
Figure 3. System Architecture.
Figure 4. Design Space of Command Recommendation Systems.
Figure 5. Example User History of a Typical User Showing Distinct Sessions of Activity with Breaks in Activity in Between.
Figure 6. The Recommendation Process with the Two Decision Points of Using User Histories Broken Up into Sessions or Users.
Figure 7. k-Tail Evaluation of a Command Sequence.
Figure 8. Offline Evaluation of Four Approaches in Figure 5.
Figure 9. New Command Adoption Rates Based on 27 Users.
Figure 10. Recommender Training Phase UI.
Figure 11. Recommender Plug-In Palette Is Opened in AutoCAD.
Figure 12. Recommended and Recently Used Commands.
Figure 13. Recommended Command Adoption.
Figure 14. Total Adopted Useful Recommendations Over Deployed Time.
Figure 15. Legend of Software Usage DNA Diagram.
Figure 16. Software Usage DNA Diagrams from the 17 Most Active Users.

BibTeX

@article{10.1609/aimag.v36i3.2600,
  author = {Li, Wei and Matejka, Justin and Grossman, Tovi and Fitzmaurice, George},
  title = {Deploying CommunityCommands: A Software Command Recommender System Case Study},
  year = {2015},
  issue_date = {Fall 2015},
  publisher = {John Wiley \& Sons, Inc.},
  address = {USA},
  volume = {36},
  number = {3},
  issn = {0738-4602},
  url = {https://doi.org/10.1609/aimag.v36i3.2600},
  doi = {10.1609/aimag.v36i3.2600},
  abstract = {In 2009 we presented the idea of using collaborative filtering within a complex software application to help users learn new and relevant commands (Matejka et al. 2009). This project continued to evolve and we explored the design space of a contextual software command recommender system and completed a six‐week user study (Li et al. 2011). We then expanded the scope of our project by implementing CommunityCommands, a fully functional and deployable recommender system. CommunityCommands was a publically available plug‐in for Autodesk's flagship software application AutoCAD. During a one‐year period, the recommender system was used by more than 1100 users. In this article, we discuss how our practical system architecture was designed to leverage Autodesk's existing customer involvement program (CIP) data to deliver in‐product contextual recommendations to end users. We also present our system usage data and payoff, and provide an in‐depth discussion of the challenges and design issues associated with developing and deploying the software command recommender system. Our work sets important groundwork for the future development of recommender systems within the domain of end user software learning assistance.},
  journal = {AI Mag.},
  month = {sep,},
  pages = {19–34},
  numpages = {16},
}