pith. sign in

arxiv: 2402.06795 · v2 · pith:XI4TVIM7new · submitted 2024-02-09 · 💻 cs.GR · cs.HC

Squidgets: Sketch-based Widget Design for Scene Manipulation

Joonho Kim , Fanny Chevalier , Karan Singh This is my paper

Pith reviewed 2026-05-24 04:03 UTC · model grok-4.3

classification 💻 cs.GR cs.HC
keywords squidgetssketch-based UIscene manipulationdirect manipulationstroke matching3D animationMaya implementationuser study
0
0 comments X

The pith

Squidgets interpret sketched strokes over scenes as direct controls to adjust associated parameters.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes squidgets as a sketch-based UI framework that automatically interprets user strokes to execute scene changes. This approach rests on the idea that curves abstracting scene elements act as natural handles for parameter manipulation, with users able to add custom curves for new associations. Implementation inside Maya demonstrates stroke input for both 2D and 3D scenes, supported by a controlled experiment on object translation and deformation plus an informal study on squidget creation. A sympathetic reader would care because people already sketch to indicate desired changes, so turning those strokes into working controls could remove steps in traditional interfaces.

Core claim

Squidgets are sketch-widgets that interpret partially matched strokes against scene curves to select and control associated parameters, enabling direct manipulation. Additional user-defined curves can author custom handles tied to scene attributes. The framework is realized in Maya for 2D/3D stroke input on 2D/3D scenes, with evaluation through a controlled experiment on translation and deformation tasks and a broader informal study.

What carries the argument

Squidgets: sketch-widgets that use partial stroke matching to scene curves as handles for selecting and controlling parameters.

If this is right

  • Custom handles can be authored by defining additional curves associated with specific scene attributes.
  • The same stroke-matching mechanism supports both 2D and 3D input for manipulating corresponding scenes.
  • Selection and interactive control occur after a stroke is partially matched to a squidget curve.
  • Evaluation shows the approach works for object translation and deformation tasks inside an existing animation system.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Squidgets could reduce reliance on menu-driven widget selection in professional animation pipelines.
  • The partial-matching technique might transfer to sketch interfaces in 2D image or video editing tools.
  • If curve abstraction proves robust across varied scenes, similar handles could apply to real-time simulation parameters.

Load-bearing premise

Curves resulting from visually abstracting scene elements provide natural handles for the direct manipulation of scene parameters.

What would settle it

A controlled comparison in Maya where participants using squidgets complete translation and deformation tasks with lower accuracy or speed than with standard widgets would falsify the claim of effective direct manipulation.

Figures

Figures reproduced from arXiv: 2402.06795 by Fanny Chevalier, Joonho Kim, Karan Singh.

Figure 1
Figure 1. Figure 1: Humans naturally sketch strokes (cyan) over graphical scenes to communicate desired scene changes [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Implicit Squidgets comprise a set of perceived curves (2D or 3D) that are already scene curves or [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Explicit squidgets comprise a set of curves (2D or 3D) that are explicitly authored in ( [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: All perceived scene curves are potential squidgets including: real curves (left) that are part of the scene’s construction (green), or explicitly authored squidgets (white); and virtual curves (center) imagined by a user (highlighted in red). A user draw stroke (cyan) can manipulate scene attributes (right), such that their associated squidget curve matches the user stroke. Implicit Squidgets: We refer to … view at source ↗
Figure 5
Figure 5. Figure 5: Explicitly created squidget curves can be manifest on surfaces (a), canvases in space (b-c), or using VR [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Explicitly authored curves can be associated with discrete configurations of face attributes to capture [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The user stroke is matched for similarity against all scene squidgets. The cuboid contour best matches [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: (a) An implicit stroke is drawn to match the contour of the cloud. (b) The cloud resizes to fit along the [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: A self-authored, continuous squidget with four keyed attribute states that translates and rotates a [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Multiple squidgets control different parameters of a light cone on one canvas: (a) light visibility, (b) [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Within a VR environment, curves can be drawn to control an wood-chopper (a-b). 3D curves drawn [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Squidgets can pose combinations of individual FACs control handles to drive complex facial pose [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Squidgets can control character rigs to key multiple poses (a-c) for a jumping motion (d-g). [PITH_FULL_IMAGE:figures/full_fig_p013_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Using previously hand-authored squidgets to move the boat and moon, we can create a parent [PITH_FULL_IMAGE:figures/full_fig_p013_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Study scenarios with various tasks: arranging object using implicit squidgets (a), creating squidgets [PITH_FULL_IMAGE:figures/full_fig_p014_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Post-study survey questions and results from six participants. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Examples of self-authored widgets from user-study participants: squidget controlling other squidgets [PITH_FULL_IMAGE:figures/full_fig_p016_17.png] view at source ↗
read the original abstract

People naturally sketch strokes over graphical scenes to convey scene changes. We propose automatically interpreting these strokes to execute scene changes with squidgets (sketch-widgets), a novel sketch-based UI framework for direct scene manipulation. Squidgets are motivated by the observation that curves resulting from visually abstracting scene elements provide natural handles for the direct manipulation of scene parameters. Additional curves can be defined by users to author custom handles associated with scene attributes. Users manipulate a scene by simply drawing strokes, partially matched against scene curves to select a squidget and interactively control associated parameters. We present an implementation of squidgets within the 3D animation system Maya, showing 2D/3D stroke input to manipulate 2D/3D scenes. We report on a controlled experiment evaluating squidgets on 2D object translation and deformation tasks, and a broader informal study on squidget creation and manipulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes squidgets, a sketch-based UI framework for direct scene manipulation in which user strokes are partially matched against curves (either automatically derived from scene elements or user-authored) to select and control associated parameters. It describes an implementation in Maya supporting 2D/3D input for 2D/3D scenes, a controlled experiment on 2D translation and deformation tasks, and an informal study on squidget creation and manipulation.

Significance. If the partial-matching mechanism proves reliable across varied scenes and stroke styles, the framework could provide a practical, natural interface for animation and modeling workflows that reduces reliance on traditional widgets. The controlled experiment supplies quantitative evidence for the 2D case, and the Maya implementation demonstrates feasibility within an existing production tool.

major comments (2)
  1. [Evaluation and Implementation sections] The description of the partial curve matching algorithm (mentioned in the abstract and evaluation) provides no technical details on the matching procedure, similarity metric, threshold selection, or handling of ambiguous or noisy strokes, which is load-bearing for the central claim that strokes can be automatically interpreted to execute scene changes.
  2. [Controlled experiment description] The controlled experiment reports results on 2D tasks but supplies no information on participant numbers, exact task protocol, how matching errors were logged or recovered, or the baseline interface used for comparison, undermining assessment of the claimed performance advantage.
minor comments (2)
  1. [Abstract] The abstract states that 'curves resulting from visually abstracting scene elements provide natural handles' without clarifying how the abstraction step is performed or what constitutes a valid abstraction.
  2. [Framework overview] Notation for squidget parameters and stroke-to-curve association is introduced informally; a short table or diagram defining the core data structures would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed review and constructive comments. We address each of the major comments below and will make revisions to the manuscript to incorporate the suggested improvements.

read point-by-point responses

  1. Referee: [Evaluation and Implementation sections] The description of the partial curve matching algorithm (mentioned in the abstract and evaluation) provides no technical details on the matching procedure, similarity metric, threshold selection, or handling of ambiguous or noisy strokes, which is load-bearing for the central claim that strokes can be automatically interpreted to execute scene changes.

    Authors: We agree with this assessment. The current version of the manuscript does not provide sufficient technical details on the partial curve matching algorithm. We will revise the Implementation section to include a thorough description of the matching procedure, the similarity metric employed, how thresholds are selected, and approaches for managing ambiguous or noisy strokes. This will strengthen the central claim by making the technical foundation explicit. revision: yes

  2. Referee: [Controlled experiment description] The controlled experiment reports results on 2D tasks but supplies no information on participant numbers, exact task protocol, how matching errors were logged or recovered, or the baseline interface used for comparison, undermining assessment of the claimed performance advantage.

    Authors: We acknowledge that additional details are necessary for a complete evaluation of the experiment. In the revised manuscript, we will expand the description of the controlled experiment to include the number of participants, the precise task protocol, the methods for logging and recovering from matching errors, and clarification on the baseline interface used for comparison. This will allow readers to better assess the performance advantages claimed. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is a system contribution proposing squidgets, a sketch-based UI framework for direct scene manipulation. It describes motivation from visual abstraction of scene elements, implementation in Maya, partial curve matching for stroke interpretation, a controlled 2D experiment, and an informal study. No equations, derivations, predictions, fitted parameters, or first-principles results are present. No self-citations are load-bearing for any central claim, and the work does not reduce any result to its own inputs by construction. The central claim is the proposal and empirical evaluation of the framework itself.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that abstracting scene elements into curves creates usable manipulation handles, with squidgets as the core invented concept; no free parameters or formal axioms.

axioms (1)
  • domain assumption Curves resulting from visually abstracting scene elements provide natural handles for the direct manipulation of scene parameters.
    Stated as the motivation for squidgets in the abstract.
invented entities (1)
  • squidgets no independent evidence
    purpose: Sketch-widgets that interpret strokes for direct scene parameter control
    New framework and term introduced to describe the proposed UI elements and interaction.

pith-pipeline@v0.9.0 · 5675 in / 1084 out tokens · 32342 ms · 2026-05-24T04:03:12.626943+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    Crossy: a crossing-based drawing application

    Georg Apitz and Fran c ois Guimbreti \` e re. Crossy: a crossing-based drawing application. In Steven Feiner and James A. Landay, editors, Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, Santa Fe, NM, USA, October 24-27, 2004 , pages 3--12. ACM , 2004

    work page 2004

  2. [2]

    Inverse kinematics techniques in computer graphics: A survey

    Andreas Aristidou, Joan Lasenby, Yiorgos Chrysanthou, and Ariel Shamir. Inverse kinematics techniques in computer graphics: A survey. Computer Graphics Forum , 37:35--58, 09 2018

    work page 2018

  3. [3]

    Bae, Ravin Balakrishnan, and Karan Singh

    S.H. Bae, Ravin Balakrishnan, and Karan Singh. ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models . In Proc. User Interface Software and Technology , 2008

    work page 2008

  4. [4]

    Line drawings from 3d models: A tutorial

    Pierre B\' e nard and Aaron Hertzmann. Line drawings from 3d models: A tutorial. Found. Trends. Comput. Graph. Vis. , 11(1–2):1–159, sep 2019

    work page 2019

  5. [5]

    Mouse, tactile, and tangible input for 3d manipulation

    Lonni Besan c on, Paul Issartel, Mehdi Ammi, and Tobias Isenberg. Mouse, tactile, and tangible input for 3d manipulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems , CHI '17, page 4727–4740, New York, NY, USA, 2017. Association for Computing Machinery

    work page 2017

  6. [6]

    E. A. Bier. Skitters and jacks: Interactive 3d positioning tools. In Proc. I3D 86 , pages 183--196, 1986

    work page 1986

  7. [7]

    E. A. Bier. Snap-dragging in three dimensions. In Proc. I3D 1990 , pages 193--204, 1990

    work page 1990

  8. [8]

    Bowman, Ernst Kruijff, Joseph J

    Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola, and Ivan Poupyrev. 3D User Interfaces: Theory and Practice . Addison Wesley Longman Publishing Co., Inc., USA, 2004

    work page 2004

  9. [9]

    Byungkuk Choi, Roger Blanco i Ribera, J. P. Lewis, Yeongho Seol, Seokpyo Hong, Haegwang Eom, Sunjin Jung, and Junyong Noh. Sketchimo: Sketch-based motion editing for articulated characters. ACM Trans. Graph. , 35(4), jul 2016

    work page 2016

  10. [10]

    Active contours without edges

    Tony F Chan and Luminita A Vese. Active contours without edges. IEEE Transactions on image processing , 10(2):266--277, 2001

    work page 2001

  11. [11]

    The mental canvas: A tool for conceptual architectural design and analysis

    Julie Dorsey, Songhua Xu, Gabe Smedresman, Holly Rushmeier, and Leonard McMillan. The mental canvas: A tool for conceptual architectural design and analysis. In Proceedings of the 15th Pacific Conference on Computer Graphics and Applications , PG '07, page 201–210, USA, 2007. IEEE Computer Society

    work page 2007

  12. [12]

    The line of action: An intuitive interface for expressive character posing

    Martin Guay, Marie-Paule Cani, and R\' e mi Ronfard. The line of action: An intuitive interface for expressive character posing. ACM Trans. Graph. , 32(6), nov 2013

    work page 2013

  13. [13]

    Non-photorealistic rendering

    Bruce Gooch and Amy Gooch. Non-photorealistic rendering . CRC Press, 2001

    work page 2001

  14. [14]

    Introduction to 3d non-photorealistic rendering: Silhouettes and outlines

    Aaron Hertzmann. Introduction to 3d non-photorealistic rendering: Silhouettes and outlines. Non-Photorealistic Rendering. SIGGRAPH , 99(1), 1999

    work page 1999

  15. [15]

    Igarashi, T

    T. Igarashi, T. Moscovich, and J. F. Hughes. Spatial keyframing for performance-driven animation. In ACM SIGGRAPH 2007 Courses , SIGGRAPH '07, page 25–es, New York, NY, USA, 2007. Association for Computing Machinery

    work page 2007

  16. [16]

    Teddy: A sketching interface for 3d freeform design

    Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. Teddy: A sketching interface for 3d freeform design. SIGGRAPH '99, pages 409--416, New York, NY, USA, 1999. ACM Press/Addison-Wesley Publishing Co

    work page 1999

  17. [17]

    Computational support for sketching in design: A review

    Gabe Johnson, Mark Gross, Jason Hong, and Ellen Do. Computational support for sketching in design: A review. Foundations and Trends in Human-Computer Interaction , 2:1--93, 01 2009

    work page 2009

  18. [18]

    Optimizing ui layouts for deformable face-rig manipulation

    Joonho Kim and Karan Singh. Optimizing ui layouts for deformable face-rig manipulation. ACM Trans. Graph. , 40(4), jul 2021

    work page 2021

  19. [19]

    Direct manipulation blendshapes

    John P Lewis and Ken-ichi Anjyo. Direct manipulation blendshapes. IEEE Computer Graphics and Applications , 30(4):42--50, 2010

    work page 2010

  20. [20]

    Yunjin Lee, Lee Markosian, Seungyong Lee, and John F. Hughes. Line drawings via abstracted shading. ACM Trans. Graph. , 26(3):18, 2007

    work page 2007

  21. [21]

    Dag amendment for inverse control of parametric shapes

    Elie Michel and Tamy Boubekeur. Dag amendment for inverse control of parametric shapes. ACM Transactions on Graphics , 40(4):173:1--173:14, 2021

    work page 2021

  22. [22]

    Mendes, F

    D. Mendes, F. M. Caputo, A. Giachetti, A. Ferreira, and J. Jorge. A survey on 3d virtual object manipulation: From the desktop to immersive virtual environments. Computer Graphics Forum , 38(1):21--45, 2019

    work page 2019

  23. [23]

    Meshless deformations based on shape matching

    Matthias M \"u ller, Bruno Heidelberger, Matthias Teschner, and Markus Gross. Meshless deformations based on shape matching. ACM transactions on graphics (TOG) , 24(3):471--478, 2005

    work page 2005

  24. [24]

    A sketch-based interface for detail-preserving mesh editing

    Andrew Nealen, Olga Sorkine, Marc Alexa, and Daniel Cohen-Or. A sketch-based interface for detail-preserving mesh editing. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) , 24(3):1142--1147, 2005

    work page 2005

  25. [25]

    Juxtaform: Interactive visual summarization for exploratory shape design

    Karran Pandey, Fanny Chevalier, and Karan Singh. Juxtaform: Interactive visual summarization for exploratory shape design. ACM Trans. Graph. , 42(4), jul 2023

    work page 2023

  26. [26]

    Wires: A geometric deformation technique

    Karan Singh and Eugene Fiume. Wires: A geometric deformation technique. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques , SIGGRAPH '98, 1998

    work page 1998

  27. [27]

    Sketching and Composing Widgets for 3D Manipulation

    Ryan Schmidt, Karan Singh, and Ravin Balakrishnan. Sketching and Composing Widgets for 3D Manipulation . Computer Graphics Forum , 27(2):301--310, 2008

    work page 2008

  28. [28]

    Sutherland

    Ivan E. Sutherland. Sketch pad a man-machine graphical communication system. In Proceedings of the SHARE Design Automation Workshop , DAC '64, pages 6.329--6.346, New York, NY, USA, 1964. ACM

    work page 1964

  29. [29]

    The visual perception of 3d shape

    James T Todd. The visual perception of 3d shape. Trends in cognitive sciences , 8(3):115--121, 2004

    work page 2004

  30. [30]

    Shortstraw: A simple and effective corner finder for polylines

    Aaron Wolin, Brian Eoff, and Tracy Hammond. Shortstraw: A simple and effective corner finder for polylines. SBIM , 8:33--40, 2008

    work page 2008

  31. [31]

    Wobbrock, Andrew D

    Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. Gestures without libraries, toolkits or training: A \ 1 recognizer for user interface prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology , UIST '07, page 159–168, New York, NY, USA, 2007. Association for Computing Machinery

    work page 2007

  32. [32]

    Object-oriented drawing

    Haijun Xia, Bruno Araujo, Tovi Grossman, and Daniel Wigdor. Object-oriented drawing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems , CHI '16, page 4610–4621, New York, NY, USA, 2016. Association for Computing Machinery

    work page 2016

  33. [33]

    True2form: 3d curve networks from 2d sketches via selective regularization

    Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. True2form: 3d curve networks from 2d sketches via selective regularization. ACM Transactions on Graphics , 33(4), 2014

    work page 2014

  34. [34]

    R. C. Zeleznik, K. P. Herndon, and J. F. Hughes. Sketch: An interface for sketching 3d scenes. In Proc. SIGGRAPH '96 , pages 163--170, 1996

    work page 1996