A system that turns mid-air sketches plus voice into textured 3D meshes in XR by chaining ControlNet image generation with convolutional mesh reconstruction.
Fine-Grained VR Sketching: Dataset and Insights
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We present the first fine-grained dataset of 1,497 3D VR sketch and 3D shape pairs of a chair category with large shapes diversity. Our dataset supports the recent trend in the sketch community on fine-grained data analysis, and extends it to an actively developing 3D domain. We argue for the most convenient sketching scenario where the sketch consists of sparse lines and does not require any sketching skills, prior training or time-consuming accurate drawing. We then, for the first time, study the scenario of fine-grained 3D VR sketch to 3D shape retrieval, as a novel VR sketching application and a proving ground to drive out generic insights to inform future research. By experimenting with carefully selected combinations of design factors on this new problem, we draw important conclusions to help follow-on work. We hope our dataset will enable other novel applications, especially those that require a fine-grained angle such as fine-grained 3D shape reconstruction. The dataset is available at tinyurl.com/VRSketch3DV21.
citation-role summary
citation-polarity summary
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
MS2Mesh-XR: Multi-modal Sketch-to-Mesh Generation in XR Environments
A system that turns mid-air sketches plus voice into textured 3D meshes in XR by chaining ControlNet image generation with convolutional mesh reconstruction.