A self-critique prompting loop outperformed retrieval-augmented and decomposed prompting for zero-shot generation of high-fidelity GUI prototypes, based on over 3,000 crowdworker ratings.
VINS: Visual Search for Mobile User Interface Design
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Searching for relative mobile user interface (UI) design examples can aid interface designers in gaining inspiration and comparing design alternatives. However, finding such design examples is challenging, especially as current search systems rely on only text-based queries and do not consider the UI structure and content into account. This paper introduces VINS, a visual search framework, that takes as input a UI image (wireframe, high-fidelity) and retrieves visually similar design examples. We first survey interface designers to better understand their example finding process. We then develop a large-scale UI dataset that provides an accurate specification of the interface's view hierarchy (i.e., all the UI components and their specific location). By utilizing this dataset, we propose an object-detection based image retrieval framework that models the UI context and hierarchical structure. The framework achieves a mean Average Precision of 76.39\% for the UI detection and high performance in querying similar UI designs.
citation-role summary
background 1
citation-polarity summary
fields
cs.SE 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Zero-Shot Prompting Approaches for LLM-based Graphical User Interface Generation
A self-critique prompting loop outperformed retrieval-augmented and decomposed prompting for zero-shot generation of high-fidelity GUI prototypes, based on over 3,000 crowdworker ratings.