Justin Matejka
k j

Interaction Techniques for Comparing Video

Adam Baker, Carl Gutwin, Justin Matejka, Ian Stavness
January 2024 · Proceedings of the 50th Graphics Interface Conference (GI)
Interaction Techniques for Comparing Video

Abstract

Comparison is a well-studied task in visual analytics, but there is still little support for comparison of temporal streams such as video. There are a wide range of tasks that involve video comparison, but there are very few systems or techniques to support this kind of analysis. To help address this problem, we have developed new interaction techniques that explicitly support video comparison. We provide techniques for equalizing the reference frame of videos to be compared, juxtaposition techniques for enhancing side-by-side and small-multiples comparisons, superposition techniques for comparing overlaid videos, explicit-encoding techniques that visualize differences between extracted points, and temporal-to-linear techniques that translate between a temporal sequence of frames and a 1D timeline. We built a demonstration system with five different datasets, and evaluated our interaction techniques in two ways: an analysis of steps to show their efficiency, and a preliminary user study to explore learnability, utility, and usability.

DOI PDF

Figures

Figure 1: Web-based demonstration application. (A) Title (B) Video selection dropdown (C) Global system options (D)Individual video options (E) Workspace with 12 plant videos (F) Global timeline widge
Figure 2: Coincident-points technique. Videos aremisaligned (a), so points are placed to create an alignmenttransformation and equalize the videos (b).
Figure 3: Endpoint adjustment: time-lapse videos of twoCanola varieties, with endpoints adjusted to show flowering.
Figure 4: Small-multiples technique showing animations ofsea ice changes, 1979-2022; each video is made up of 365satellite images.
Figure 5: Small-multiples technique with hidden borders.
Figure 6: Shadow markers to compare different pitches fromthe same pitcher. A marker (yellow circle) placed in the leftvideo is replicated in the right video.
Figure 7: Superposition video of two pitches from the samepitcher. (Ball brightness has been increased for visibility).
Figure 8: Window-blind overlay showing a horizontal slider,used to compare summer/winter traffic levels.
Figure 9: See-through lens showing a specific region in theunderlying video at full opacity.
Figure 10: Overlay with image filters, showing one RGB plant image and one segmented image. Image source.
Figure 11: Explicit encodings displaying predicted changesto a pollution spill. Difference lines and areas represent therelationship between predicted and observed point data.
Figure 12: Annotations. Four overlaid pitch videos, withtimeline annotations for beginning of the wind-up (red),release of the ball (yellow), catch by catcher (cyan), and callfrom the umpire (magenta)
Figure 13: Timeline measurement tool, measuring from thestart to end of the flowering cycle (marked withannotations) in Canola.

BibTeX

@inproceedings{10.1145/3670947.3670948,
  author = {Baker, Adam and Gutwin, Carl and Matejka, Justin and Stavness, Ian},
  title = {Interaction Techniques for Comparing Video},
  year = {2024},
  isbn = {9798400718281},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3670947.3670948},
  doi = {10.1145/3670947.3670948},
  abstract = {Comparison is a well-studied task in visual analytics, but there is still little support for comparison of temporal streams such as video. There are a wide range of tasks that involve video comparison, but there are very few systems or techniques to support this kind of analysis. To help address this problem, we have developed new interaction techniques that explicitly support video comparison. We provide techniques for equalizing the reference frame of videos to be compared, juxtaposition techniques for enhancing side-by-side and small-multiples comparisons, superposition techniques for comparing overlaid videos, explicit-encoding techniques that visualize differences between extracted points, and temporal-to-linear techniques that translate between a temporal sequence of frames and a 1D timeline. We built a demonstration system with five different datasets, and evaluated our interaction techniques in two ways: an analysis of steps to show their efficiency, and a preliminary user study to explore learnability, utility, and usability.},
  booktitle = {Proceedings of the 50th Graphics Interface Conference},
  articleno = {5},
  numpages = {13},
  keywords = {Visual comparison, interaction techniques, time-lapse, video},
  location = {Halifax, NS, Canada},
  series = {GI '24},
}