Pith. sign in

REVIEW 3 major objections 5 minor 67 references

Code Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Free-form sketches drawn directly on code and console output can guide an AI model to make iterative code edits, according to a three-stage design study with 18 programmers.

desk verdict Code shaping is a real, novel HCI contribution—an exploratory design study of sketch-on-code editing with LLMs—but the spatial advantage over text prompts is asserted, not measured. read the letter →

arxiv 2502.03719 v2 pith:5HCER4PM submitted 2025-02-06 cs.HC

Ryan Yen , Jian Zhao , Daniel Vogel This is my paper · ORCID
classification cs.HC
keywords codeshapingsketch-basedinteractionAI-assistededitingfree-formannotationlargelanguagemodelsmultimodaldesignstudyhuman-AIcollaboration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Code shaping is the proposal that programmers can edit code the way they think on a whiteboard: by drawing arrows, circles, cross-outs, pseudocode, and small diagrams directly on top of the code and its console output, then letting a multimodal AI model turn those marks into actual code changes. The paper reports a three-stage design study with 18 programmers that develops this idea into a working prototype and, more importantly, into an empirical account of how people sketch for code edits: what kinds of marks they make, how they recover when the AI misreads them, and which interface mechanisms keep the loop fluent. A sympathetic reader should care because programming interaction is still almost entirely text-based; if the claim holds, sketching becomes a viable complementary input modality for AI-assisted editing rather than a separate planning activity. The paper is careful not to claim code shaping beats typing, only that it is a viable alternative demonstrated through two realistic use cases.

What carries the argument

The central object is an invisible sketch canvas overlaid on the code editor and the console panel, so that ink and code share one visual space. The prototype renders the combined view into an SVG image, adds a grid for locating annotations, greys out the code to make ink prominent, and sends the image together with version history to a multimodal AI model that proposes edits; a diff algorithm then applies only the changed sections. Around this core, the final design adds an always-on feedforward interpretation that triggers about 500 milliseconds after the pen stops, showing the model's reading of the sketch, the inferred editing action, and the affected code through gutter markers and transient highlights, plus gestures such as check and cross marks to accept or reject edits.

What would settle it

A direct test would be a controlled comparison in which the same edit tasks are completed through code shaping and through typed prompts or direct editing, with independent scoring of whether committed edits match the intended change; the central claim fails if sketch-mediated edits are substantially less accurate or slower across typical programmers. A cheaper check is to measure the model's sketch-to-edit accuracy on a held-out set of the captured sketches from the paper's studies, since the paper reports iteration rates but never isolates model interpretation error from user error.

Watch

Extended reading notes

Core claim

The paper's core claim is that code shaping is a viable interaction paradigm: rather than switching to a text prompt or making edits by hand, a programmer draws arrows, circles, cross-outs, pseudocode, or flowcharts directly on the code and on console or graphical output, and a multimodal large language model turns those marks into staged code edits. Evidence comes from three successive design studies, each with six programmers, in which the prototype matured from a basic generate button to an interface with always-on feedforward interpretation, gutter glyphs, and check-and-cross gestures. The studies produced a taxonomy of sketch roles (command, parameter, target), a catalogue of six AI interpretation error types and the repair strategies programmers use, and design principles for reconciling the sketch canvas, the code editor, and the AI model. The paper explicitly does not claim code shaping is superior to typing; it claims it is a usable alternative that lets programmers express and refine edits iteratively through free-form sketching, and it demonstrates the claim with two multi-file use-case scenarios.

Load-bearing premise

The whole approach leans on the assumption that an AI model can reliably read hand-drawn marks placed on top of code well enough to produce correct edits, and that the 18 convenience-sampled programmers who already use ChatGPT or Copilot regularly stand in for programmers generally.

Editorial extensions

If this is right

  • Programmers can express an intended edit without translating it into natural language: a circle plus an arrow plus the word 'def' is enough to insert a function that plots data.
  • Iterative refinement is part of the workflow rather than a failure: 23.2% of sketches in stage two required a second pass, and participants repaired errors by redrawing, adding code references, or rewriting pseudocode.
  • Always-on feedforward interpretation, showing the model's reading of text, the inferred action, and the affected code before committing, increased commit frequency by 32.4% in stage three relative to stage two.
  • Making the editor reachable through the canvas, with gestures for selection, undo, and accept or reject actions, reduces context switching and lets programmers keep a spatial mental model of the code.
  • The mechanism generalizes in principle beyond Python, and the two use cases show tablet-based solo programming and whiteboard-based pair review, although the empirical evidence is limited to small Python codebases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's Section 9 explicitly restricts the evidence to small Python codebases and states that multi-file editing and dependency propagation are untested; scaling code shaping to large repositories would likely require retrieval-augmented generation and static analysis, which is an inference beyond the reported data.
  • Because participants reported a shift from linear to spatial thinking and often resketched even correct edits, a plausible but untested extension is that code shaping is most valuable in planning-heavy activities like refactoring and code review rather than routine line-level edits.
  • Versioning the sketches instead of deleting them on commit could turn the annotations into executable design documentation; the paper mentions this only as future work.
  • The reported 23.2% iteration rate is not decomposed into model-interpretation error versus sketch ambiguity, so a direct measurement of model interpretation accuracy on the captured sketches would be a natural next experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces 'code shaping,' an interaction paradigm in which programmers edit code by sketching free-form annotations (arrows, circles, pseudocode, and text) directly on top of the code and console output. The authors built a prototype that overlays an ink canvas on an editor, sends the composed SVG to GPT-4o, and applies the generated edits as a diff. They report three sequential design studies with six different programmers each (18 total), analyzing sketch types, AI interpretation errors, repair strategies, and interaction flows. The paper concludes with design implications, two usage scenarios, and a set of limitations. The stated contribution is not superiority over typed prompts but the establishment of a viable alternative interaction modality.

Significance. If the result holds, the paper makes a useful contribution to HCI by demonstrating that free-form sketching on and around code can be a practical interface for iterative AI-driven code editing. The three-stage design process, the qualitative coding of sketch behaviors, and the openly available stage-three prototype are assets that could seed further research on multimodal programming interfaces. The paper is careful in several places to limit its claims to viability rather than superiority, and its limitations section is candid about codebase size and language coverage. However, the central novelty hinges on the spatial placement of ink being functionally meaningful, and that load-bearing point is not yet supported by the evidence presented.

major comments (3)
  1. [Section 8.3 and Table 1] The paper does not isolate the contribution of spatial sketch placement from the textual content of the annotations, even though spatial placement is central to the claimed novelty. Section 8.3 explicitly states that sketches were not compared with textual prompts, and Table 1 shows that 34 of 74 coded annotations were written text or code/pseudocode (11 text + 23 code), with another 31 annotation symbols and only 9 freeform sketches. As a result, the observed successful edits could plausibly be driven largely by OCR-accessible text and code syntax plus the existing code context, rather than by the geometric arrangement of strokes. This is load-bearing for the claim that 'free-form sketch annotations directly on top of the code' is a new interaction paradigm. I ask the authors to either add an ablation (e.g., feeding the model the same annotations as typed text, or masking stroke positions while preserving ink content) and report how the generated edits change, or substantially reframe the contribution as the viability of free-form ink on the code surface without asserting that spatial placement is the operative mechanism.
  2. [Section 6.3.2 and Table 4] The cross-stage comparisons are confounded by the between-subjects design. Each of the three stages used a different set of six participants, yet the text attributes design improvements to the interface changes: for example, 'participants pressed the Commit button 32.4% more frequently in the third stage compared to that in the second' and Table 4 reports p-values such as UMUX-LITE stage 1 vs 3 p=0.031. Because the same tasks were reused across stages and the participant pools differ in AI-tool familiarity and other characteristics, these differences cannot be causally attributed to the design iterations. The paper should present these comparisons explicitly as exploratory, acknowledge the between-subjects confounds, and avoid causal language, or provide a within-subjects follow-up. This issue undermines the strength of the 'design implications' contribution as currently worded.
  3. [Section 4.2] The description of task pre-testing is ambiguous and directly relevant to the spatial-ablation concern. The paper states that 'All tasks were pre-tested to ensure that GPT-4o could not immediately generate the correct code,' but it does not specify what input the model received in the pre-test. If the baseline was only the starter code and the plain task description, then the finding merely shows that the base prompt is insufficient; it does not show that a typed instruction conveying the same semantic content as the sketch would also fail. Please specify the exact pre-test prompt and, where possible, run a comparison of the model's output with a text-only instruction that carries the same content as the sketches.
minor comments (5)
  1. [Table 3 and Section 6.3.3] The notation 'Cross (/reve) drawing' appears to be a typographical error; likely 'Cross (×)' or 'Cross (X)' is intended, and the same '//reve' artifact appears in Section 6.3.3. Please correct these occurrences.
  2. [Section 4.4.1] The reported average of 3 sketch interactions per subtask with SD=4.0 suggests a strongly skewed distribution; reporting the median and range would be more informative for readers.
  3. [Appendix A, Table 4] The pairwise comparisons in Table 4 use n=6 per group and no correction for multiple comparisons. Since the paper's main contributions are qualitative, please either add an explicit caveat that these are exploratory statistics or remove the inferential tests altogether.
  4. [Section 6.1.2] The sentence about repositioning 'the interpretation text from the upper right to the lower right' would be clearer if it specified the reference frame (e.g., the upper-right corner of the canvas or screen).
  5. [Section 9] The limitations section is thorough, but it should explicitly mention the absence of a direct comparison to text prompts as a threat to the novelty claim, rather than leaving that caveat to the discussion in Section 8.3.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper is an empirical design study whose claims rest on observed participant behavior, not on derivation or fitted inputs.

full rationale

This paper contains no derivation chain that reduces to its own inputs. The central claim—that free-form sketch annotations on and around code can viably communicate edits to an AI model—is supported by three staged user studies with 18 programmers, analyzed through inductive thematic coding, system logs, interviews, and quantitative comparisons across iterations. No parameter is fitted and then renamed as a prediction; no result is defined into existence; and no uniqueness theorem is imported from the authors' prior work. The only overlapping-author reference that appears in the bibliography (CoLadder, [69]) is not used as load-bearing evidence for code shaping's viability. The paper explicitly states in Section 8.3 that it did not compare sketches with textual prompts, but that acknowledged limitation concerns the strength of the spatial-contribution claim, not circular reasoning. Likewise, the limitations in Section 9—small Python codebases, convenience sampling, and dependence on GPT-4o interpretation—are validity threats, not circular steps. The statistical results in Appendix A compare successive design iterations and are empirical observations rather than predictions generated from fitted constants. The paper also explicitly disclaims the stronger claim that code shaping is superior to typing, further reducing any sense that the conclusion is forced by the setup. No quoted reduction of Eq. X to Eq. Y, no fitted-input-called-prediction, and no self-citation chain can be exhibited; accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

Code shaping is an empirical design study rather than a derivation, so there are no fitted numeric constants or invented entities. The central claims rest on three domain assumptions: that GPT-4o can interpret mixed sketch content, that the small convenience samples represent the intended users, and that the pre-tested Python tasks capture general code-shaping behavior.

assumptions (3)
  • domain assumption GPT-4o can interpret overlaid sketches as code edit intents.
    The prototype relies on this for every edit; the paper reports observed outcomes but no independent accuracy metric (Sections 4.1, 5.1, 6.1).
  • domain assumption Convenience-sampled participants with frequent AI-assistant use represent broader programmers.
    Recruitment in Sections 4.2, 5.2, and 6.2 used screening for Python experience and ChatGPT/Copilot use; Section 9 limits generalizability to Python and small codebases.
  • domain assumption Pre-tested Python tasks capture representative code-shaping behavior.
    Tasks span three programming paradigms but were selected so GPT-4o could not immediately solve them; transfer to other languages and larger codebases is untested (Section 4.2, Section 9).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Code Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching." pith.science (2026). https://pith.science/paper/5HCER4PM

@misc{pith2026250203719,
  author       = {Pith},
  title        = {Pith review of: Code Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5HCER4PM}},
  note         = {Machine review of arXiv:2502.03719}
}
read the original abstract

We introduce the concept of code shaping, an interaction paradigm for editing code using free-form sketch annotations directly on top of the code and console output. To evaluate this concept, we conducted a three-stage design study with 18 different programmers to investigate how sketches can communicate intended code edits to an AI model for interpretation and execution. The results show how different sketches are used, the strategies programmers employ during iterative interactions with AI interpretations, and interaction design principles that support the reconciliation between the code editor and sketches. Finally, we demonstrate the practical application of the code shaping concept with two use case scenarios, illustrating design implications from the study.

Figures

Figures reproduced from arXiv: 2502.03719 by the authors.

Figure 1
Figure 1. Code shaping usage example: (a) a programmer draws an arrow from a few lines of code defining data attributes to a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Interface design from the first stage. (a) Pen tool with color options for code annotation; (b) Canvas tools including [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The classification of sketched annotations from [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Sketches from stage one showing how participants employ arrows ( [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The interface design from the second stage. (a) Command brushes used to steer AI model’s interpretation, including [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The interface design from the third stage. (a) programmers can use one finger tap and drag to select items on canvas; [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 41 canonical work pages

  1. [2]

    Ian Arawjo. 2020. To Write Code: The Cultural Fabrication of Programming Notation and Practice. InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20) . Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376731

    arXiv 2020

  2. [4]

    Stephen C Arnold, Leo Mark, and John Goldthwaite. 2000. Programming by voice, VocalProgramming. In Proceedings of the fourth international ACM conference on Assistive technologies. 149–155

    work page 2000

  3. [5]

    James, and Nadia Polikarpova

    Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2022. Grounded Copilot: How Programmers Interact with Code-Generating Models. (June 2022). http://arxiv.org/abs/2206.15000 arXiv: 2206.15000

    arXiv 2022

  4. [6]

    Michel Beaudouin-Lafon, Susanne Bødker, and Wendy E. Mackay. 2021. Genera- tive Theories of Interaction. ACM Trans. Comput.-Hum. Interact. 28, 6, Article 45 (nov 2021), 54 pages. https://doi.org/10.1145/3468505

    doi:10.1145/3468505 2021

  5. [7]

    Tony Beltramelli. 2018. pix2code: Generating Code from a Graphical User Interface Screenshot. In Proceedings of the ACM SIGCHI Symposium on Engi- neering Interactive Computing Systems (Paris, France) (EICS ’18). Association for Computing Machinery, New York, NY, USA, Article 3, 6 pages. https: //doi.org/10.1145/3220134.3220135

    arXiv 2018

  6. [8]

    Daniel Bobrow et al . 1964. Natural language input for a computer problem solving system. (1964)

    work page 1964

  7. [9]

    Xiaofan Chen and Beryl Plimmer. 2007. CodeAnnotator: digital ink annotation within Eclipse. In Proceedings of the 19th Australasian Conference on Computer- Human Interaction: Entertaining User Interfaces (Adelaide, Australia) (OZCHI ’07). Association for Computing Machinery, New York, NY, USA, 211–214. https: //doi.org/10.1145/1324892.1324935

    work page arXiv 2007

  8. [10]

    Mauro Cherubini, Gina Venolia, Rob DeLine, and Amy J. Ko. 2007. Let’s go to the whiteboard: how and why software developers use drawings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’07) . Association for Computing Machinery, New York, NY, USA, 557–566. https: //doi.org/10.1145/1240624.1240714

    arXiv 2007

Show all 67 references
  1. [11]

    Alain Desilets. 2001. VoiceGrip: a tool for programming-by-voice. International Journal of Speech Technology 4 (2001), 103–116

    2001

  2. [12]

    EPICLab. 2023. Synectic Integrated Development Environment . https://epiclab. github.io/synectic/ A novel IDE supporting a human-centered approach to programming by enhancing context-specific functionality and supporting pro- grammers’ cognitive processes

    2023

  3. [13]

    Dedre Gentner. 1988. Metaphor as Structure Mapping: The Relational Shift.Child Development 59, 1 (1988), 47–59. http://www.jstor.org/stable/1130388

    1988

  4. [14]

    V Goel. 1995. Sketches of thought

    1995

  5. [15]

    Thomas R. G. Green and Marian Petre. 1996. Usability analysis of visual pro- gramming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages & Computing 7, 2 (1996), 131–174

    1996

  6. [16]

    Gross and Ellen Yi-Luen Do

    Mark D. Gross and Ellen Yi-Luen Do. 1996. Ambiguous intentions: a paper-like interface for creative design. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology (Seattle, Washington, USA) (UIST ’96). Association for Computing Machinery, New ...

    1996

  7. [17]

    Tracy Hammond and Randall Davis. 2007. LADDER, a sketching language for user interface developers. InACM SIGGRAPH 2007 Courses (San Diego, California) (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 35–es. https://doi.org/10.1145/1281500.1281546

    2007

  8. [18]

    Juris Hartmanis. 1994. Turing award lecture on computational complexity and the nature of computer science. Commun. ACM 37, 10 (1994), 37–43

    1994

  9. [19]

    Catrinel Haught and Philip N Johnson-Laird. 2003. Creativity and constraints: The production of novel sentences. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 25

    2003

  10. [20]

    Kasper Hornbæk and Antti Oulasvirta. 2017. What Is Interaction?. InProceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5040–5052. https://doi.org/10.1145/3025453.3025765

    2017

  11. [21]

    Joshua Horowitz and Jeffrey Heer. 2023. Live, Rich, and Composable: Qualities for Programming Beyond Static Text. arXiv preprint arXiv:2303.06777 (2023)

    2023 arXiv

  12. [23]

    Donald Ervin Knuth. 1984. Literate programming. The computer journal 27, 2 (1984), 97–111

    1984

  13. [24]

    Microsoft AI Lab. 2018. Sketch2Code: Transform Hand-drawn Designs into HTML Code using AI. https://github.com/microsoft/ailab/tree/master/ Sketch2Code. Accessed: 2024-09-02

    2018

  14. [25]

    Landay and B.A

    J.A. Landay and B.A. Myers. 2001. Sketching interfaces: toward more human interface design. Computer 34, 3 (March 2001), 56–64. https://doi.org/10.1109/2. 910894

    2001 doi

  15. [26]

    James A Landay and Brad A Myers. 1995. Interactive sketching for the early stages of user interface design. In Proceedings of the SIGCHI conference on Human factors in computing systems . 43–50

    1995

  16. [27]

    LaToza, Gina Venolia, and Robert DeLine

    Thomas D. LaToza, Gina Venolia, and Robert DeLine. 2006. Maintaining mental models: a study of developer work habits. In Proceedings of the 28th international conference on Software engineering (ICSE ’06) . Association for Computing Ma- chinery, New York, NY, USA, 492–501. htt...

    2006

  17. [28]

    Chuanjun Li, Timothy S Miller, Robert C Zeleznik, and Joseph J LaViola Jr. 2008. AlgoSketch: Algorithm Sketching and Interactive Computation. SBIM 8 (2008), 175–182

    2008

  18. [29]

    Leonhard Lichtschlag and Jan Borchers. 2010. CodeGraffiti: communication by sketching for pair programmers. In Adjunct Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (New York, New York, USA) (UIST ’10). Association for Computing Machine...

    2010

  19. [30]

    Leonhard Lichtschlag, Lukas Spychalski, and Jan Bochers. 2014. CodeGraffiti: Using hand-drawn sketches connected to code bases in navigation tasks. In 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) . IEEE, 65–68

    2014

  20. [31]

    Newman, Jason I

    James Lin, Mark W. Newman, Jason I. Hong, and James A. Landay. 2000. DENIM: finding a tighter fit between tools and practice for Web site design. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, The Netherlands) (CHI ’00). Association fo...

    2000

  21. [32]

    James Lindenbaum, Szymon Kaliski, and Joshua Horowitz. 2022. Inkbase: Pro- grammable Ink. https://www.inkandswitch.com/inkbase/ Accessed: 2024-12-07

    2022

  22. [33]

    What It Wants Me To Say

    Michael Xieyang Liu, Advait Sarkar, Carina Negreanu, Benjamin Zorn, Jack Williams, Neil Toronto, and Andrew D. Gordon. 2023. “What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code- Generating Large Language Models. In Proceedings of the 2...

    2023

  23. [34]

    Walid Maalej, Rebecca Tiarks, Tobias Roehm, and Rainer Koschke. 2014. On the Comprehension of Program Comprehension. ACM Transactions on Software Engineering and Methodology 23, 4 (Sept. 2014), 31:1–31:37. https://doi.org/10. 1145/2622669

    2014

  24. [35]

    LaToza, Marian Petre, and André van der Hoek

    Nicolas Mangano, Thomas D. LaToza, Marian Petre, and André van der Hoek

  25. [36]

    McGuffin and Christopher P

    Michael J. McGuffin and Christopher P. Fuhrman. 2020. Categories and Com- pleteness of Visual Programming and Direct Manipulation. In Proceedings of the 2020 International Conference on Advanced Visual Interfaces (Salerno, Italy) (A VI ’20). Association for Computing Machinery...

    2020

  26. [37]

    Hussein Mozannar, Gagan Bansal, Adam Fourney, and Eric Horvitz. 2022. Read- ing Between the Lines: Modeling User Behavior and Costs in AI-Assisted Pro- gramming. (Oct. 2022). http://arxiv.org/abs/2210.14306 arXiv: 2210.14306

    2022 arXiv

  27. [38]

    Emerson Murphy-Hill, Moin Ayazifar, and Andrew P Black. 2011. Restructuring software with gestures. In 2011 ieee symposium on visual languages and human- centric computing (vl/hcc). IEEE, 165–172

    2011

  28. [39]

    B. A. Myers. 1986. Visual programming, programming by example, and program visualization: a taxonomy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, Massachusetts, USA)(CHI ’86). Association for Computing Machinery, New York, NY, USA, 59...

    1986

  29. [40]

    Eugene W Myers. 1986. An O (ND) difference algorithm and its variations. Algorithmica 1, 1 (1986), 251–266

    1986

  30. [41]

    Mark Noone and Aidan Mooney. 2018. Visual and textual programming lan- guages: a systematic review of the literature. Journal of Computers in Education 5 (2018), 149–174

    2018

  31. [42]

    Anna Offenwanger, Theophanis Tsandilas, and Fanny Chevalier. 2024. Data- Garden: Formalizing Personal Sketches into Structured Visualization Templates. IEEE Transactions on Visualization and Computer Graphics (2024)

    2024

  32. [44]

    Chris Parnin and Robert DeLine. 2010. Evaluating cues for resuming interrupted programming tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10) . Association for Computing Machinery, New York, NY, USA, 93–102. https://doi.org/10.1145/1...

    2010

  33. [45]

    Chris Parnin and Carsten Gorg. 2006. Building usage contexts during program comprehension. In 14th IEEE International Conference on Program Comprehension (ICPC’06). IEEE, 13–22

    2006

  34. [46]

    Chris Parnin, Carsten Görg, and Spencer Rugaber. 2010. CodePad: interactive spaces for maintaining concentration in programming environments. In Proceed- ings of the 5th International Symposium on Software Visualization (Salt Lake City, Utah, USA) (SOFTVIS ’10). Association fo...

    2010

  35. [47]

    Plimmer, J

    B. Plimmer, J. Grundy, J. Hosking, and R. Priest. 2006. Inking in the IDE: Experi- ences with Pen-based Design and Annotatio. In Visual Languages and Human- Centric Computing (VL/HCC’06). 111–115. https://doi.org/10.1109/VLHCC.2006. 28

    2006 doi

  36. [48]

    Josh Pollock, Ian Arawjo, Caroline Berger, and Arvind Satyanarayan. 2024. De- signing for Semi-Formal Programming with Foundation Models. In PLATEAU: 14th Annual Workshop at the Intersection of PL and HCI

    2024

  37. [49]

    Felix Raab, Christian Wolff, and Florian Echtler. 2013. RefactorPad: editing source code on touchscreens. In Proceedings of the 5th ACM SIGCHI symposium on Engi- neering interactive computing systems (EICS ’13) . Association for Computing Ma- chinery, New York, NY, USA, 223–22...

    2013

  38. [50]

    Miguel A Renom, Baptiste Caramiaux, and Michel Beaudouin-Lafon. 2022. Ex- ploring technical reasoning in digital tool use. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems . 1–17

    2022

  39. [51]

    Steve Ruiz and tldraw contributors. 2024. tldraw. https://github.com/tldraw/ tldraw. Accessed: 2024-08-26

    2024

  40. [52]

    Russell, Mark J

    Daniel M. Russell, Mark J. Stefik, Peter Pirolli, and Stuart K. Card. 1993. The cost structure of sensemaking. In Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems (CHI ’93) . Association for Computing Machinery, New York, NY, USA, 26...

    1993

  41. [53]

    Sigurdur Gauti Samuelsson and Matthias Book. 2020. Eliciting Sketched Expres- sions of Command Intentions in an IDE. Proceedings of the ACM on Human- Computer Interaction 4, ISS (2020), 1–25

    2020

  42. [54]

    Sigurdur Gauti Samuelsson and Matthias Book. 2023. Towards a Visual Language for Sketched Expression of Software IDE Commands. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) . 115–123. https: //doi.org/10.1109/VL-HCC57772.2023.00021 ISSN: 1943-6106

    2023

  43. [55]

    Nazmus Saquib, Rubaiat Habib Kazi, Li-yi Wei, Gloria Mark, and Deb Roy. 2021. Constructing Embodied Algebra by Sketching. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York,...

    2021

  44. [56]

    Margaret-Anne Storey, Jody Ryall, Janice Singer, Del Myers, Li-Te Cheng, and Michael Muller. 2009. How Software Developers Use Tagging to Support Re- minding and Refinding. IEEE Transactions on Software Engineering 35, 4 (2009), 470–483. https://doi.org/10.1109/TSE.2009.15

    2009 doi

  45. [57]

    Sangho Suh, Martinet Lee, Gracie Xia, and Edith law. 2020. Coding Strip: A Peda- gogical Tool for Teaching and Learning Programming Concepts through Comics. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–10. https://doi.org/10.1109/VL/HCC500...

    2020

  46. [58]

    Sangho Suh, Jian Zhao, and Edith Law. 2022. CodeToon: Story Ideation, Auto Comic Generation, and Structure Mapping for Code-Driven Storytelling. In Pro- ceedings of the 35th Annual ACM Symposium on User Interface Software and Tech- nology (Bend, OR, USA) (UIST ’22). Associatio...

    2022

  47. [59]

    Sarah Suleri, Vinoth Pandian Sermuga Pandian, Svetlana Shishkovets, and Matthias Jarke. 2019. Eve: A Sketch-based Software Prototyping Workbench. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Comput- ing Systems (Glasgow, Scotland Uk) (CHI EA ’19) . Asso...

    2019

  48. [60]

    Craig Sutherland. 2017. An Investigation into Freeform, Dynamic, Digi- tal Ink Annotation for Program Code . Thesis. ResearchSpace@Auckland. https://researchspace.auckland.ac.nz/handle/2292/31647 Accepted: 2017-01- 23T03:21:24Z

    2017

  49. [61]

    Craig J Sutherland, Andrew Luxton-Reilly, and Beryl Plimmer. 2015. An ob- servational study of how experienced programmers annotate program code. In Human-Computer Interaction–INTERACT 2015: 15th IFIP TC 13 International Conference, Bamberg, Germany, September 14-18, 2015, Pro...

    2015

  50. [62]

    Nikolai Tillmann, Michal Moskal, Jonathan De Halleux, Manuel Fahndrich, and Sebastian Burckhardt. 2012. TouchDevelop: app development on mobile de- vices. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering . 1–2

    2012

  51. [63]

    Barbara Tversky. 2002. What do sketches say about thinking. In 2002 AAAI Spring Symposium, Sketch Understanding Workshop, Stanford University, AAAI Technical Report SS-02-08, Vol. 148. 151

    2002

  52. [64]

    Jagoda Walny, Sheelagh Carpendale, Nathalie Henry Riche, Gina Venolia, and Philip Fawcett. 2011. Visual Thinking In Action: Visualizations As Used On Whiteboards. IEEE Transactions on Visualization and Computer Graphics 17, 12 (Dec 2011), 2508–2517. https://doi.org/10.1109/TVC...

    2011 doi

  53. [65]

    Joseph Weizenbaum. 1966. ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 1 (1966), 36–45

    1966

  54. [66]

    Wobbrock, Andrew D

    Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. InPro- ceedings of the 20th Annual ACM Symposium on User Interface Software and Tech- nology (Newport, Rhode Island, USA) (U...

    2007

  55. [67]

    Haijun Xia, Nathalie Henry Riche, Fanny Chevalier, Bruno De Araujo, and Daniel Wigdor. 2018. DataInk: Direct and creative data-oriented drawing. InProceedings of the 2018 CHI Conference on Human Factors in Computing Systems . 1–13

    2018

  56. [68]

    Haijun Xia, Ken Hinckley, Michel Pahud, Xiao Tu, and Bill Buxton. 2017. Writ- Large: Ink Unleashed by Unified Scope, Action, & Zoom.. In CHI, Vol. 17. 3227– 3240

    2017

  57. [69]

    Ryan Yen, Jiawen Stefanie Zhu, Sangho Suh, Haijun Xia, and Jian Zhao. 2024. CoLadder: Manipulating Code Generation via Multi-Level Blocks. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (Pittsburgh, PA, USA) (UIST ’24). Association fo...

    2024

  58. [70]

    Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. Repocoder: Repository-level code completion through iterative retrieval and generation. arXiv preprint arXiv:2303.12570 (2023). Yen et al. A STATISTICAL RESULT...

    2023 arXiv

  59. [2015]

    IEEE Transactions on Software Engineering 41, 2 (2015), 135–156

    How Software Designers Interact with Sketches at the Whiteboard. IEEE Transactions on Software Engineering 41, 2 (2015), 135–156. https://doi.org/10. 1109/TSE.2014.2362924

    2015

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.