Pith. sign in

REVIEW 3 major objections 4 minor 38 references

Codetations: Intelligent, Persistent Notes and UIs for Programs and Other Documents

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Codetations argues that document-external, editor-hosted annotations keep code context alive and that relevant annotation context materially improves LLM program repair.

desk verdict A solid systems paper whose central LLM-repair claim leaks the answer in its own appendix. read the letter →

arxiv 2504.18702 v1 pith:JN4V3IZ7 submitted 2025-04-25 cs.SE cs.HC

Edward Misback , Erik Vank , Zachary Tatlock , Steven Tanimoto This is my paper · ORCID
classification cs.SEcs.HC
keywords documentannotationliveprogrammingvisualend-userlanguagemodelscodegenerationprogramrepairsemanticanchoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Codetations is a VSCode extension that attaches rich, interactive annotations to selected spans of code without modifying the source file. The paper's central claim is that these document-external notes solve two problems at once: developers get a clutter-free place to keep design rationale, examples, and debug data, and LLMs get the external context they need to repair unfamiliar code. Its controlled experiments, reported in Section 8.1.3, showed that GPT-4o solved repair tasks reliably only when the relevant slice of annotation context was present, and produced confident wrong fixes without it. A nine-developer user study found consistent interest in the non-intrusive, self-updating annotation model. If the claim holds, annotation layers that belong to the editor rather than the file become a cheap way to improve both human and machine understanding of code.

What carries the argument

The central machinery is the split between anchor records and document content. Anchors live in a separate annotation data file, so nothing is inserted into the source; online edits move anchors through low-latency editor heuristics, while offline changes trigger LLM-based semantic reattachment, using the method the paper cites as [25]. On top of that, annotation types act as living UIs: they receive the document text, their own data, and an API to call the editor's language model, so an annotation can regenerate its contents or run code when the anchored span changes. External storage, hybrid anchoring, and LLM-equipped annotation types together are what allow notes to survive code evolution and to act on their surroundings.

What would settle it

Run the paper's two repair prompts (the GitHub API authentication bug and the PN532 NFC bug) on a current sample of LLMs with no context and with randomized distractor sets; if a substantial fraction of models solve them without context, or if relevant hints do not beat distractors alone, the claim that relevant annotation context is necessary for reliable repair collapses.

Watch

Extended reading notes

Core claim

Codetations treats context as a first-class, editor-hosted layer: an annotation record stores an ID, start and end positions in a document version, a way to obtain that version, an annotation type, and a data field, all kept in a JSON file outside the source. While the user types, anchor positions move with edit heuristics; when a file changes offline, the system asks permission and re-attaches anchors semantically using an LLM. Annotation types are React components with an API for reading and writing document text, reading and writing their own data, and calling the editor's language model, which is what lets notes regenerate their content and re-execute as code changes. The paper reports that a single prompt with API examples was enough to generate new annotation types, including the Show Debugged Example annotation that writes, runs, and displays test code for the anchored region. The empirical anchor is a small controlled comparison: for two GPT-4o-generated debugging problems, one a GitHub API authentication bug and one an NFC/UART hardware bug, the model misdiagnosed without context and diagnosed correctly when relevant annotation-style context was included alongside distractors; the paper concludes that curated local context still materially boosts state-of-the-art models and can guard against confident error.

Load-bearing premise

The evaluation assumes that the two debugging problems generated by GPT-4o have repairs latent in the model, so that failing without context is a fair measure of context's value, and that the hand-picked hints embedded in the distractor materials represent real annotation content.

Editorial extensions

If this is right

  • Relevant, curated context is not optional for current LLM program repair: in the paper's two tasks, GPT-4o repaired the bugs only when the right hint was present, even though the knowledge was latent in the model.
  • Persistent annotation layers can be built without polluting source files, because anchor records live outside the document; this lets comments, diagrams, and debugging widgets scale without bloating code.
  • LLM-generated annotation types are practical: a single prompt plus API examples produced a working Show Debugged Example annotation, suggesting the ecosystem can grow quickly.
  • Irrelevant context is dangerous: when the model was given only unrelated documentation, it confidently produced incorrect diagnoses, so systems need context filtering and provenance tracking, not just retrieval.
  • Editor-hosted annotations lower the barrier to keeping context: study participants understood and valued the system once it lived in their IDE, and specifically valued robust re-anchoring and live widgets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the two-case result generalizes, the bottleneck shifts from model capability to context curation: benchmarks for code repair should report which external hints were supplied, and annotation systems could standardize that reporting.
  • A testable extension would automate note generation from git history, runtime traces, or review comments, then measure whether the LLM-repair benefit persists for automatically produced notes rather than hand-picked hints.
  • Because unrelated context actively misled the model, one could build a relevance-scoring layer that ranks candidate annotations for each repair task; the paper leaves that design open.
  • The same anchoring machinery could attach notes to non-code documents such as specs and configuration files, and the LLM-repair result suggests a direct test for whether such notes help LLM agents in other structured-text domains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents Codetations, a VSCode extension that lets users attach persistent, interactive annotations to spans of code without modifying the source file. The system uses a hybrid edit-tracking/LLM-based anchoring method to keep annotations attached as the document evolves, and exposes an API that lets annotation types read/write document text, call the editor's LLM API, and host interactive UI widgets. The authors report a qualitative need-finding and prototype evaluation with nine experienced programmers, describe several annotation types they generated in minutes with LLM prompts, and present two worked examples in which GPT-4o is said to repair code bugs only when supplied with context that mimics Codetations-style annotations. The paper concludes that external annotations can improve LLM program repair and motivate building persistent annotation layers into IDEs.

Significance. The system design and qualitative user study are the main strengths of the paper. The idea of document-external, semantically anchored, interactive annotations is well-motivated by prior work, and the demonstration that new annotation types can be generated by prompting an LLM with API examples is a credible and useful contribution. The anchoring approach builds on the authors' prior Misback et al. work and is appropriately situated relative to Catseye, Sodalite, and other annotation systems. The user study (N=9) is modest but honestly reported, and the qualitative findings about documentation fragmentation, maintenance burden, and users' desire for non-intrusive rich annotations are plausible and useful for future tool builders. However, the paper's central empirical claim about LLM repair is not supported by the evidence as presented: the with-context condition in both worked examples includes the full corrected code, so the reported behavior is consistent with copying the patch rather than reasoning from annotation context, and the evaluation rests on only two examples with no repeated trials or statistics.

major comments (3)
  1. [Section 7.1 and Appendix D] The 'with context' condition in both worked examples embeds the complete corrected code. In D.1, the context bullet 'Code before and after debugging' contains the full 'Debugged version' of list_repos with the authentication header; in D.2, the 'Code comparison' block contains the complete HSU-mode initialization sequence. The reported with-context responses ('adds boilerplate to provide the authentication', 'switch the module to HSU mode') are therefore consistent with copying or lightly adapting the supplied patch rather than with reasoning from annotation context. The comparison is uninterpretable as evidence that Codetations-style external notes improve LLM repair. The authors must either withhold the code-after blocks from the prompt and re-run the comparison, or add an ablation in which the context contains only the relevant hint bullet (e.g., the authentication requirement or the HSU-mode note) without the patch.
  2. [Sections 7 and 8.1.3] The paper's central RQ3 claim is stated in Section 8.1.3 as 'Our controlled experiments demonstrate that GPT-4o solved repair tasks reliably only when the relevant slice of annotation context was present.' Section 7, however, describes the evaluation as 'we also evaluated a few examples.' With only two examples, no repeated trials, no variation in distractor sets, and no reported prompt/context transcripts, the term 'controlled experiments' and the adverb 'reliably' overstate the evidence. Either report a systematic evaluation (e.g., multiple trials per condition, multiple models, ablation of context components) or explicitly present the two examples as illustrative anecdotes and temper the abstract and conclusion accordingly.
  3. [Section 7] The method for guaranteeing that the bug information is 'latent in the model' is to generate the problems with GPT-4o without tool calling. This assumes that a model that can generate a problem statement can also solve the repair task from a zero-context prompt, which is not verified. The repair evaluation then uses GPT-4o again, so the zero-context failure may reflect prompt underspecification or the model's inability to retrieve the latent knowledge on demand, rather than the absence of contextual annotations. Add a sanity check: run each problem with the explicit bug explanation (or with only the relevant hint bullet) to confirm the model can solve it when given the key information, and also test at least one other model family to rule out model-specific effects.
minor comments (4)
  1. [Abstract and Section 7] The phrase 'interactively-collected data' in the abstract is not operationalized; the appendix shows hand-assembled context bullets, not data collected through the Codetations UI. Clarify what the annotations actually contained and how they were produced.
  2. [Appendix D] The exact prompts and the full context strings shown to the LLM are not reported; include a transcript or the exact template so the conditions can be reproduced.
  3. [Section 8.3] The limitations paragraph discusses the user study but omits the limitations of the LLM evaluation (n=2, code-after confound, single model family); add a sentence acknowledging the anecdotal nature of the repair examples.
  4. [Section 6.1 and Figure 1] The 'Show Debugged Example' annotation is said to execute via a 'webview interpreter'; specify whether this is a JavaScript interpreter embedded in the webview and whether it can execute code of languages other than JavaScript.

Circularity Check

1 steps flagged · score 6.0 of 10

LLM repair evaluation supplies the fix inside the "context," making the with-context result circular.

  1. other [Section 7, Appendix D.1/D.2, and Section 8.1.3]
    "Appendix D.1: "• Code before and after debugging: ... # Debugged version: def list_repos(username, token=None): ... headers['Authorization'] = ..." ; D.2: "• Code comparison: ... # With HSU mode enabled: nfc.reset() ..." ; 8.1.3: "Our controlled experiments demonstrate that GPT-4o solved repair tasks reliably only when the relevant slice of annotation context was present.""

    The independent variable (the 'relevant slice of annotation context') is operationalized in Appendix D as a block containing the complete, corrected code: for the GitHub API task, the 'Debugged version' adds the authentication header; for the NFC task, the 'With HSU mode enabled' code performs the mode switch. The measured outcome is that GPT-4o 'add[s] boilerplate to provide the authentication' and 'switch[es] the module to HSU mode'—exactly the code provided in the context. The success of the with-context condition is therefore entailed by the input: the model need only copy or lightly adapt the supplied patch. The conclusion that notes improve LLM repair is not an inference from external context to a fix; the fix is the context.

full rationale

The system design and user study are not circular: Codetations is implemented and demonstrated, and the anchoring method reuses the authors' prior Misback et al. [25] approach, but that self-citation is supporting rather than load-bearing for the paper's main claims. The LLM evaluation, however, is circular in its current form. Section 7 defines the task as repairing code with context that contains 'a hint at the true explanation mixed into a set of extraneous information,' but Appendix D operationalizes the context as including the full corrected implementation. Thus the paper's headline empirical claim—'GPT-4o solved repair tasks reliably only when the relevant slice of annotation context was present'—is true by construction: the relevant slice is the patch. This does not show that external, Codetations-style notes improve LLM repair; it shows that supplying the answer improves performance. The paper candidly notes in Section 7 that 'we evaluated a few examples,' and Appendix M contains only two cases, so the overstatement in Section 8.1.3 amplifies the circularity. Apart from this evaluation step, no equation-level circularity, fitted-parameter renaming, or imported uniqueness theorem is present. Score 6 reflects a partial, but real, circularity in the central empirical demonstration; the system-building and user-study contributions are independent.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This is a systems/HCI paper, so there are no mathematical free parameters fitted to data. The load-bearing assumptions are about the reliability of the reused LLM anchoring method, the representativeness of the nine-participant study, the latent-knowledge guarantee for LLM-generated debugging problems, and the availability of the VSCode language model API. The paper's own limitations section acknowledges the sample and effort assumptions.

assumptions (4)
  • domain assumption The LLM-based position update method of Misback et al. [25] is reliable enough to keep annotations attached to the correct semantic entities during offline edits.
    Codetations adopts this method without re-validating it; Appendix F records several failures during the study.
  • domain assumption The nine participants, drawn mainly from research environments, represent the range of developer needs and reactions relevant to the claims.
    Section 8.3 explicitly concedes the sample is small and convenience-based; industry teams may differ.
  • domain assumption Debugging problems generated by GPT-4o without tool calling are guaranteed to have solutions latent in that model.
    Section 7 relies on this guarantee to make the zero-context condition a meaningful baseline.
  • ad hoc to paper Supplying a hint about the true bug among distractor facts mimics realistic annotation context well enough to support the LLM repair claim.
    The context in Appendix D is hand-built by the authors; no study measured how often real Codetations annotations contain such hints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Codetations: Intelligent, Persistent Notes and UIs for Programs and Other Documents." pith.science (2026). https://pith.science/paper/JN4V3IZ7

@misc{pith2026250418702,
  author       = {Pith},
  title        = {Pith review of: Codetations: Intelligent, Persistent Notes and UIs for Programs and Other Documents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JN4V3IZ7}},
  note         = {Machine review of arXiv:2504.18702}
}
read the original abstract

Software developers maintain extensive mental models of code they produce and its context, often relying on memory to retrieve or reconstruct design decisions, edge cases, and debugging experiences. These missing links and data obstruct both developers and, more recently, large language models (LLMs) working with unfamiliar code. We present Codetations, a system that helps developers contextualize documents with rich notes and tools. Unlike previous approaches, notes in Codetations stay outside the document to prevent code clutter, attaching to spans in the document using a hybrid edit-tracking/LLM-based method. Their content is dynamic, interactive, and synchronized with code changes. A worked example shows that relevant notes with interactively-collected data improve LLM performance during code repair. In our user evaluation, developers praised these properties and saw significant potential in annotation types that we generated with an LLM in just a few minutes.

Figures

Figures reproduced from arXiv: 2504.18702 by the authors.

Figure 1
Figure 1. Codetations lets users attach rich, living annotations to any text file. Here, (1) the behavior of highlighted code is illustrated by (2) a “Show Debugged Example” annotation through (2A) user-customizable debugging instructions, (2B) LLM-generated test code, and (2C) live execution results. We added this annotation type to our system in just a few minutes using the prompts shown in Appendix C. Unlike traditional co… view at source ↗
Figure 2
Figure 2. (right) Codetations connects the external Odyssey floating-point workbench, hosted on the web or in an editor, to (left) source code. An Analyze Floating-point Expression annotation uses an LLM to translate the Python expression shown on the left, math.log(x + math.sqrt(x*x + 1)), into log(x + sqrt(x*x + 1)), an expression in Odyssey’s input language. This operation works for all input and output languages the LLM u… view at source ↗
Figure 3
Figure 3. The LM Unit Test annotation type lets users ask yes-or-no questions about the code and suggests changes if the answer is “no.” Suggestions can be applied to the buffer. The question stays with the code and can be asked again after the code is updated. This annotation type was generated in a few minutes by an LLM (see Appendix C). sections for specific properties, including summaries or ex￾planations. Our LM Unit Tes… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The document-external annotations enabled by our system could facilitate the separation of documents into layers that help with specific tasks, such as instrumenting code, without cluttering the main document. the issue is resolved, without leaving residual code in the…
Figure 5
Figure 5. Figure 5: An early prototype of our system was fully editor￾independent and ran in a browser. It featured (1) a document view to allow users to select annotation targets, (2) an annota￾tion view with (user-irrelevant) configuration and document attachment information, and (3) a …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 33 canonical work pages

  1. [1]

    [n. d.]. Cursor – rules. https://docs.cursor.com/context/rules

    work page

  2. [2]

    Maristella Agosti, Giorgetta Bonfiglio-Dosio, and Nicola Ferro. 2007. A historical and contemporary study on annotations to derive key features for systems design. International Journal on Digital Libraries 8 (2007), 1–19

    work page 2007

  3. [3]

    Maristella Agosti and Nicola Ferro. 2007. A formal model of anno- tations of digital content. ACM Transactions on Information Systems (TOIS) 26, 1 (2007), 3–es

    work page 2007

  4. [4]

    Andrew W Appel. 2011. Verified Software Toolchain: (Invited Talk). In European Symposium on Programming . Springer, 1–17

    work page 2011

  5. [5]

    Antranig Basman, Clayton Lewis, and Colin Clark. 2018. The open authorial principle: supporting networks of authors in creating ex- ternalisable designs. In Proceedings of the 2018 ACM SIGPLAN Inter- national Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software. 29–43

    work page 2018

  6. [6]

    Christian Bird, Denae Ford, Thomas Zimmermann, Nicole Forsgren, Eirini Kalliamvakou, Travis Lowdermilk, and Idan Gazit. 2022. Taking Flight with Copilot: Early insights and opportunities of AI-powered pair-programming tools. Queue 20, 6 (2022), 35–57

    2022

  7. [7]

    AJ Bernheim Brush, David Bargeron, Anoop Gupta, and Jonathan J Cadiz. 2001. Robust annotation positioning in digital documents. In Proceedings of the SIGCHI conference on Human factors in computing systems. 285–292

    work page 2001

  8. [8]

    Michael L Collard, Jonathan I Maletic, and Andrian Marcus. 2002. Supporting document and data views of source code. In Proceedings of the 2002 ACM symposium on Document engineering . 34–41

    work page 2002

Show all 38 references
  1. [9]

    Ekwa Duala-Ekoko and Martin P Robillard. 2010. Clone region de- scriptors: Representing and tracking duplication in source code. ACM Transactions on Software Engineering and Methodology (TOSEM) 20, 1 (2010), 1–31

    2010

  2. [10]

    Kasra Ferdowsi, Michael B James, Nadia Polikarpova, Sorin Lerner, et al. 2023. Live Exploration of AI-Generated Programs. arXiv preprint arXiv:2306.09541 (2023)

    2023 arXiv

  3. [11]

    Camille Gobert and Michel Beaudouin-Lafon. 2023. Lorgnette: Cre- ating Malleable Code Projections. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . 1–16

    2023

  4. [12]

    Devamardeep Hayatpur, Brian Hempel, Kathy Chen, William Duan, Philip Guo, and Haijun Xia. 2024. Taking ascii drawings seriously: How programmers diagram code. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems . 1–16

    2024

  5. [13]

    Brian Hempel, Justin Lubin, and Ravi Chugh. 2019. Sketch-n-sketch: Output-directed programming for svg. In Proceedings of the 32nd An- nual ACM Symposium on User Interface Software and Technology . 281– 292

    2019

  6. [14]

    Joshua Horowitz and Jeffrey Heer. 2023. Engraft: An API for Live, Rich, and Composable Programming. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . 1–18

    2023

  7. [15]

    Joshua Horowitz and Jeffrey Heer. 2023. Live, Rich, and Compos- able: Qualities for Programming Beyond Static Text. arXiv preprint arXiv:2303.06777 (2023)

    2023 arXiv

  8. [16]

    Amber Horvath, Andrew Macvean, and Brad A Myers. 2023. Support for Long-Form Documentation Authoring and Maintenance. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 109–114

    2023

  9. [17]

    Amber Horvath, Andrew Macvean, and Brad A Myers. 2024. Meta- Manager: A Tool for Collecting and Exploring Meta Information about Code. In Proceedings of the CHI Conference on Human Factors in Com- puting Systems. 1–17

    2024

  10. [18]

    Amber Horvath, Brad Myers, Andrew Macvean, and Imtiaz Rahman

  11. [19]

    Donald Ervin Knuth. 1984. Literate programming. The computer journal 27, 2 (1984), 97–111

    1984

  12. [20]

    Sam Lau, Ian Drosos, Julia M Markel, and Philip J Guo. 2020. The design space of computational notebooks: An analysis of 60 systems in academia and industry. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) . IEEE, 1–11

    2020

  13. [21]

    Yongkun Liu, Jiachi Chen, Tingting Bi, John Grundy, Yanlin Wang, Ting Chen, Yutian Tang, and Zibin Zheng. 2024. An Empirical Study on Low Code Programming using Traditional vs Large Language Model Support. arXiv:2402.01156 [cs.SE]

    2024

  14. [22]

    Ariana Martino, Michael Iannelli, and Coleen Truong. 2023. Knowledge injection to counter large language model (LLM) hallucination. In European Semantic Web Conference. Springer, 182–185

    2023

  15. [23]

    Microsoft. 2024. Visual Studio Code Language Model API. https: //code.visualstudio.com/api/extension-guides/language-model First released in the July 2024 (version 1.92) release of Visual Studio Code

    2024

  16. [24]

    Edward Misback, Caleb C Chan, Brett Saiki, Eunice Jun, Zachary Tat- lock, and Pavel Panchekha. 2023. Odyssey: An interactive workbench for expert-driven floating-point expression rewriting. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–15

    2023

  17. [25]

    Tanimoto

    Edward Misback, Zachary Tatlock, and Steven L. Tanimoto. 2024. Magic Markup: Maintaining Document-External Markup with an LLM. ArXiv abs/2403.03481 (2024). https://api.semanticscholar.org/ CorpusID:268253244

    2024 arXiv

  18. [26]

    David Rauch, Patrick Rein, Stefan Ramson, Jens Lincke, and Robert Hirschfeld. 2019. Babylonian-style Programming: Design and Imple- mentation of an Integration of Live Examples into General-purpose Source Code. arXiv preprint arXiv:1902.00549 (2019)

    2019 arXiv

  19. [27]

    Steven P. Reiss. 1990. Connecting tools using message passing in the Field environment. IEEE software 7, 4 (1990), 57–66

    1990

  20. [28]

    Steven P Reiss. 2008. Tracking source locations. In Proceedings of the 30th international conference on Software engineering . 11–20

    2008

  21. [29]

    Steven L Tanimoto. 2013. A perspective on the evolution of live programming. In 2013 1st International Workshop on Live Programming (LIVE). IEEE, 31–34

    2013

  22. [30]

    Tim Teitelbaum and Thomas Reps. 1981. The Cornell program synthe- sizer: a syntax-directed programming environment. Commun. ACM 24, 9 (1981), 563–573

    1981

  23. [31]

    Update this component to include a field that allows a user to adjust what is debugged with a natural language prompt

    Moritz Wittenhagen, Christian Cherek, and Jan Borchers. 2016. Chron- icler: Interactive exploration of source code history. In Proceedings of the 2016 CHI conference on human factors in computing systems . 3522–3532. A Basic Annotation Affordances A.1 Connecting the buffer and...

    2016

  24. [33]

    All round 2 participants quickly confirmed at this stage that they understood what the annotations meant and how they worked

    An HTML file pre-annotated with a basic comment annotation type and another type that rendered a pre- view of a section of the HTML. All round 2 participants quickly confirmed at this stage that they understood what the annotations meant and how they worked

  25. [34]

    Another HTML file, where all round 2 participants quickly showed that they could add an annotation to the file without interviewer help

  26. [35]

    A JavaScript quicksort algorithm annotated with an an- imated GIF visualizing quicksort; no task was assigned for this example

  27. [36]

    The extension’s on- and offline annotation position updating capabilities were also demonstrated at this point

    A JavaScript regular expression (shown in Figure 1 on the left); participants were asked to add and use a Show Debugged Example annotation (see Section 6.1) to validate the regex. The extension’s on- and offline annotation position updating capabilities were also demonstrated ...

  28. [37]

    Participants were asked to analyze the er- ror and find a better expression using anAnalyze Float- ing Point Expression annotation type (Section 6.2.1)

    A Python program including an expression with floating- point error. Participants were asked to analyze the er- ror and find a better expression using anAnalyze Float- ing Point Expression annotation type (Section 6.2.1)

  29. [38]

    future of programming

    A JavaScript loop body with an incorrect comment; participants were asked to add a unit test for the com- ment (see Section 6.2.2). K Post-demo Surveys Participants were asked the following questions in our post- demo survey. The questions mainly served as a starting point for...

    2018

  30. [2022]

    In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology

    Using Annotations for Sensemaking About Code. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–16

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.