Gemma3 vision-language models can label UI snapshot test failure causes with 84% recall on a small synthetic iOS dataset, but prompt-based selective ignore is unreliable.
Read It, Don't Watch It: Captioning Bug Recordings Automatically
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Screen recordings of mobile applications are easy to capture and include a wealth of information, making them a popular mechanism for users to inform developers of the problems encountered in the bug reports. However, watching the bug recordings and efficiently understanding the semantics of user actions can be time-consuming and tedious for developers. Inspired by the conception of the video subtitle in movie industry, we present a lightweight approach CAPdroid to caption bug recordings automatically. CAPdroid is a purely image-based and non-intrusive approach by using image processing and convolutional deep learning models to segment bug recordings, infer user action attributes, and generate subtitle descriptions. The automated experiments demonstrate the good performance of CAPdroid in inferring user actions from the recordings, and a user study confirms the usefulness of our generated step descriptions in assisting developers with bug replay.
citation-role summary
background 1
citation-polarity summary
fields
cs.SE 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
LLMShot: Reducing snapshot testing maintenance via LLMs
Gemma3 vision-language models can label UI snapshot test failure causes with 84% recall on a small synthetic iOS dataset, but prompt-based selective ignore is unreliable.