Pith. sign in

REVIEW 1 major objections 1 minor 45 references

ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review

T0 review · 1 major / 1 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ReVoicer lets reviewers speak their in-the-moment reactions to a paper; an LLM cleans, tags, and anchors each comment, checks the notes against a venue rubric, and drafts a review in the reviewer's own voice.

desk verdict A well-specified prototype paper that is honest about being unevaluated; the central risk of LLM cleanup distorting reviewer substance is real and the authors name it themselves. read the letter →

arxiv 2608.00299 v2 pith:I5AN6EEE submitted 2026-07-31 cs.HC

Matt Gottsacker , Ahinya Alwin , Hiroshi Furuya , Robert W. Lindeman , Gerd Bruder , Gregory F. Welch This is my paper
classification cs.HC
keywords peerreviewvoiceannotationlargelanguagemodelsactivereadinghuman-AIinteractiondraftingrubriccoveragespeechtranscription
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ReVoicer is a prototype desktop system that lets peer reviewers converse with a paper as they read it. The reviewer highlights a passage and speaks a raw, train-of-thought reaction; a large language model (LLM) rewrites that transcript into a concise note, tags it by comment type, and anchors it to the passage, using the highlighted text and surrounding prose to resolve ambiguous spoken references. Once reading is done, the system compares the accumulated notes against a venue-specific rubric and shows which items remain uncovered, then drafts a review composed only from the reviewer's comments and overall reflection, styled according to a guide distilled from the reviewer's past reviews. The paper's central claim is that every judgment stays with the reviewer, with the LLM confined to cleaning, classifying, organizing, and formatting. This is a design-and-implementation paper; the planned evaluations with academic reviewers are not yet reported.

What carries the argument

The load-bearing mechanism is the cleanup-and-drafting pipeline. Each spoken note goes to an LLM together with the highlighted passage, the last 1,500 characters of the previous page, the first 3,000 characters of the current page, and the reviewer's reference library, so phrases like "this claim" or "the Smith paper" can be resolved against actual prose and citations. A second prompt distills past reviews into an editable style guide covering voice, tone, structure, length, and formatting. The coverage check labels each rubric item as covered, partial, or missing with the supporting note ids, and the review draft is grounded in the notes, reflections, rubric, and style guide, with the raw transcripts included so later conversational comments that amend earlier ones can be merged. A separate Q&A prompt is limited to factual lookup with verbatim quotes and page numbers, explicitly barred from offering critiques.

What would settle it

Take real reviewing sessions, keep the raw speech transcripts, and have independent raters extract the substantive critique points from each transcript, then from the cleaned notes and the final draft; if any substantial fraction of the transcript's points is missing from the cleaned version and the loss is not caught by the reviewer, the central claim that the reviewer remains the only source of judgment is undercut.

Watch

Extended reading notes

Core claim

The paper starts from the observation that reviewing feels like an internal dialogue with the text, and that existing workflows interrupt that dialogue with copying, note-polishing, and section tracking, so reviewers record fewer critiques. ReVoicer's contribution is a two-phase workflow that keeps the reviewer in the conversation: in phase one, highlight-and-speak input with LLM cleanup captures critiques at the speed of thought and lets the reviewer ask factual questions grounded in the paper's own text; in phase two, rubric parsing and coverage checking reveal gaps, and a style guide distilled from the reviewer's past reviews shapes the final draft. The system prompt for cleanup instructs the LLM to preserve all of the reviewer's intellectual content, and the review-generation prompt states that the reviewer's comments are the only source of critique content and that no acceptance recommendation should be given unless the reviewer supplied one. The paper does not claim to have measured these effects yet; it offers the design as a response to concerns about fully automated AI reviewers.

Load-bearing premise

Everything rests on the assumption that the LLM's cleanup and drafting keep every substantive point, critique, and judgment the reviewer actually made, without adding, dropping, or distorting any of them.

Editorial extensions

If this is right

  • If cleanup preserves substance, reviewers can speak rough, rambling comments throughout reading and still end up with precise, well-structured notes.
  • If the style-guide drafting works as intended, final reviews should read like the reviewer's own writing, preserving authorship and accountability.
  • Rubric coverage checking should make venue-specific dimensions more consistent across reviews, while leaving the reviewer to fill gaps.
  • Voice input and in-paper anchoring should reduce time spent copying passages and polishing prose, freeing time for reflection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the sharpest test is content loss: ask raters to mark substantive points in raw transcripts and check whether cleaned notes and drafts preserve all of them; the paper lists this as planned work, not a result.
  • The design implies a comparison no one has run yet: voice capture versus typed annotation for the number, specificity, and constructiveness of critiques, independent of time savings.
  • The same pipeline could transfer to code review, grant panels, and essay grading, as the paper notes; those settings share the dependency on preserving the reviewer's intent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper presents ReVoicer, a desktop application that lets peer reviewers speak or type 'train-of-thought' comments on highlighted passages while reading a PDF. An LLM cleans each raw transcript into a concise, tagged, and passage-anchored note; the system also supports document Q&A, a rubric coverage check, and the generation of a draft review that follows a style guide distilled from the reviewer's past reviews. The paper describes the design rationale, four design considerations, implementation details, and full LLM prompts, but reports no user study or empirical evaluation; it explicitly defers evaluation to future work.

Significance. If the system realizes its stated design goals, it could reduce the mechanical friction of review writing while keeping the reviewer's voice and intellectual ownership, which would be a valuable complement to fully automated review systems. The paper's strengths are its detailed interaction design, its transparent reporting of open questions, and its public release of the prototype and prompts. However, the central claim that 'the reviewer is the only source of judgment' (DC2) is unverified, and several design decisions delegate substantive editorial judgments to the LLM; the paper does not yet provide evidence that the final review faithfully reflects reviewer intent.

major comments (1)
  1. [Section 5; absence of evaluation] The paper claims concrete benefits—reduced friction, keeping the reviewer 'in the conversation,' and a review 'composed from the reviewer's own comments'—without reporting any user evaluation or even a formative pilot. Section 5 lists planned surveys, think-aloud sessions, and expert rating, but no results are included. Since the system's entire rationale rests on the claim that reviewers can produce better reviews with less effort, the absence of evaluation leaves the central claims as hypotheses. I recommend that the manuscript add a small empirical study (even a qualitative usability test with 3–5 reviewers) or explicitly reframe all benefit statements as intended effects pending future work.
minor comments (1)
  1. [Appendix A.2] The prompt's instruction to use 'other' when the section does not fit is clear, but the formatting of that line is inconsistent with the surrounding prompt text and could be tidy.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning: the paper is a system design/implementation report with no fitted parameters, predictions, or derivation chains that reduce to their own inputs.

full rationale

ReVoicer is a prototype description and design-rationale paper, not an empirical or theoretical derivation. Its claims are about what the system does (cleans transcripts, tags comments, checks rubric coverage, drafts a review from a style guide distilled from the reviewer's past reviews), and these are implementation features rather than predictions or fitted results. The style guide is distilled from the reviewer's own reviews, but this is a user-personalization mechanism, not a validation loop: the paper makes no claim that this distillation validates the system's effectiveness. The central value proposition that 'the reviewer is the only source of judgment' (DC2, Sec. 3.1) is a design constraint expressed in prompts (Secs. A.2, A.7) and not a measured outcome; the paper explicitly defers evaluation to future work (Sec. 5) and flags the open question of what is lost during cleanup. Thus there are no equations, no fitted parameters, no imported uniqueness theorems, and no self-citation chain carrying the argument. The paper's own limitation statements are appropriately identified as open evaluation questions, not as circular derivations. The honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The system introduces no free parameters or invented entities. Its central claims rest on unverified assumptions about LLM fidelity and style transfer, all acknowledged as future work by the authors.

assumptions (4)
  • domain assumption LLM cleanup preserves all of the reviewer's intellectual content and does not add, remove, or alter substantive points.
    This is a design requirement in the note cleanup prompt (Appendix A.2) and the review generation guidelines (A.7), but no evaluation is provided to verify it.
  • domain assumption The reviewer's past reviews are sufficient to distill an accurate writing style guide, and using it yields drafts the reviewer would endorse.
    Style guide generation (A.10) assumes the example reviews are representative and that the LLM captures the style; this is unverified.
  • domain assumption Reviewing is an active dialogue with the paper and voice capture reduces friction without harming the reviewer's reflection process.
    Design basis in Section 3.1 is based on the first author's reflection and informal conversations, not empirical data.
  • domain assumption The rubric coverage check and rubric extraction reliably represent the venue's expectations and correctly assess annotation coverage.
    Rubric extraction (A.5) and coverage check (A.6) rely on LLM judgment; no accuracy evaluation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review." pith.science (2026). https://pith.science/paper/I5AN6EEE

@misc{pith2026260800299,
  author       = {Pith},
  title        = {Pith review of: ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5AN6EEE}},
  note         = {Machine review of arXiv:2608.00299}
}
read the original abstract

We present ReVoicer, a prototype system that supports peer reviewers by letting them converse with a paper as they read it. The reviewer highlights a passage and speaks (or types) a train-of-thought comment. A large language model then cleans the comment using the surrounding prose as context, tags it by comment type, and anchors it to the passage. After the reviewer finishes reading, ReVoicer checks the accumulated notes against a venue-specific rubric and reports coverage gaps to assist with further reflection. Then ReVoicer drafts a review composed from the reviewer's comments, written to a style guide distilled from the reviewer's past reviews. We describe the system's design rationale and implementation, and we outline plans for future evaluations.

Figures

Figures reproduced from arXiv: 2608.00299 by the authors.

Figure 1
Figure 1. Annotated screenshot of ReVoicer. Left: the PDF viewer with color-coded highlights and the voice recording overlay (1), [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The ReVoicer pipeline. While reading (Phase 1), the reviewer highlights a passage and speaks or types a reaction; the LLM cleans and tags it using the passage, surrounding prose, and the reference library as context, and the same input flow supports questions answered from the full document text. When reading is done (Phase 2), the reviewer records an overall reflection on the whole paper, and the notes are checked … view at source ↗

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 24 canonical work pages

  1. [1]

    August, L

    T. August, L. L. Wang, J. Bragg, M. A. Hearst, A. Head, and K. Lo. Paper Plain: Making medical research papers approachable to healthcare consumers with natural language processing.ACM Trans. Comput.-Hum. Interact., 30(5), Sept. 2023. doi: 10.1145/3589955 2

    doi:10.1145/3589955 2023

  2. [2]

    Baumann, J

    J. Baumann, J. Pei, S. Koyejo, and D. Hovy. Stop automating peer review without rigorous evaluation. InProceedings of the 43rd Inter- national Conference on Machine Learning (ICML), vol. 306 ofPro- ceedings of Machine Learning Research. PMLR, 2026. doi: 10.48550/ arXiv.2605.03202 2

    arXiv doi:10.48550/arxiv.2605.03202 2026

  3. [3]

    Beygelzimer, Y

    A. Beygelzimer, Y . N. Dauphin, P. Liang, and J. Wortman Vaughan. Has the machine learning review process become more arbitrary as the field has grown? The NeurIPS 2021 consistency experiment, 2023. doi: 10.48550/arXiv.2306.03262 1

    arXiv doi:10.48550/arxiv.2306.03262 2021

  4. [4]

    Biswas, S

    J. Biswas, S. Schoepp, G. Vasan, A. Opipari, A. Zhang, Z. Hu, S. Joseph, M. Lease, J. J. Li, P. Stone, K. L. Wagstaff, M. E. Tay- lor, and O. C. Jenkins. AI-assisted peer review at scale: The AAAI-26 AI review pilot, 2026. doi: 10.48550/arXiv.2604.13940 1, 2

    arXiv doi:10.48550/arxiv.2604.13940 2026

  5. [5]

    Bornmann, R

    L. Bornmann, R. Haunschild, and R. Mutz. Growth rates of modern science: a latent piecewise growth curve approach to model publi- cation numbers from established and new literature databases.Hu- manities and Social Sciences Communications, 8(224), 2021. doi: 10 .1057/s41599-021-00903-w 1

    work page 2021

  6. [6]

    Cortes and N

    C. Cortes and N. D. Lawrence. Inconsistency in conference peer re- view: Revisiting the 2014 NeurIPS experiment, 2021. doi: 10.48550/ arXiv.2109.09774 1

    arXiv doi:10.48550/arxiv.2109.09774 2014

  7. [7]

    Devanga, G

    A. Devanga, G. Bruder, D. Zielasko, A. Giovannelli, Z. Choudhary, H. Furuya, M. Gottsacker, R. W. Lindeman, and G. Welch. Two last human sections? A case study of largely AI-generated paper writing in empirical XR research. InAlt’ISMAR: Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (IS- MAR), 2026. 1

    work page 2026

  8. [8]

    Draxler, A

    F. Draxler, A. Werner, F. Lehmann, M. Hoppe, A. Schmidt, D. Buschek, and R. Welsch. The AI ghostwriter effect: When users do not perceive ownership of AI-generated text but self-declare as au- thors.ACM Trans. Comput.-Hum. Interact., 31(2), Feb. 2024. doi: 10 .1145/3637875 1, 4

    work page 2024

Show all 45 references
  1. [9]

    R. Fok, L. Soldaini, C. Trier, E. Bransom, K. MacMillan, E. Cheng, H. Kambhamettu, J. Bragg, K. Lo, M. A. Hearst, A. Head, and D. S. Weld. Accelerating scientific paper skimming with augmented intelli- gence through customizable faceted highlights.ACM Trans. Interact. Intell. ...

    2024 doi

  2. [10]

    C. W. Fox, A. Y . K. Albert, and T. H. Vines. Recruitment of reviewers is becoming harder at some journals: a test of the influence of reviewer fatigue at six journals in ecology and evolution.Research Integrity and Peer Review, 2(3), 2017. doi: 10.1186/s41073-017-0027-x 1

    2017 doi

  3. [11]

    A. Head, K. Lo, D. Kang, R. Fok, S. Skjonsberg, D. S. Weld, and M. A. Hearst. Augmenting scientific papers with just-in-time, position-sensitive definitions of terms and symbols. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21. Associati...

    2021

  4. [12]

    Hinckley, X

    K. Hinckley, X. Bi, M. Pahud, and B. Buxton. Informal information gathering techniques for active reading. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, p. 1893–1896. Association for Computing Machinery, New York, NY , USA, 2012. doi: 1...

    2012

  5. [13]

    Hosseini and S

    M. Hosseini and S. P. J. M. Horbach. Fighting reviewer fatigue or am- plifying bias? Considerations and recommendations for use of Chat- GPT and other large language models in scholarly peer review.Re- search Integrity and Peer Review, 8(1):4, May 2023. doi: 10.1186/ s41073-02...

    2023

  6. [14]

    F. Huq, A. Samee, D. C.-E. Lin, A. X. Tang, and J. P. Bigham. NoTee- line: Supporting real-time, personalized notetaking with llm-enhanced micronotes. InProceedings of the 30th International Conference on Intelligent User Interfaces, IUI ’25, p. 1064–1081. Association for Comp...

    2025

  7. [15]

    CHI 2025 papers track: Post- PC outcomes report.https://chi2025.acm.org/ chi-2025-papers-track-post-pc-outcomes-report/, 2025

    Johannes Zagermann. CHI 2025 papers track: Post- PC outcomes report.https://chi2025.acm.org/ chi-2025-papers-track-post-pc-outcomes-report/, 2025. 1

    2025

  8. [16]

    H. Kang, J. C. Chang, Y . Kim, and A. Kittur. Threddy: An interactive system for personalized thread-based exploration and organization of scientific literature. InProceedings of the 35th Annual ACM Sympo- sium on User Interface Software and Technology, UIST ’22. Associ- ation...

    2022

  9. [17]

    H. B. Kang, T. Wu, J. C. Chang, and A. Kittur. Synergi: A mixed- initiative system for scholarly synthesis and sensemaking. InProceed- ings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23. Association for Computing Machinery, New York, NY ,...

    2023

  10. [18]

    J. Kim, Y . Lee, and S. Lee. Position: The AI conference peer review crisis demands author feedback and reviewer rewards. InProceedings of the 42nd International Conference on Machine Learning (ICML), vol. 267 ofPMLR, 2025. doi: 10.48550/arXiv.2505.04966 1

    2025 doi arXiv

  11. [19]

    Liang, Z

    W. Liang, Z. Izzo, Y . Zhang, H. Lepp, H. Cao, X. Zhao, L. Chen, H. Ye, S. Liu, Z. Huang, D. Mcfarland, and J. Y . Zou. Monitoring AI- modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews. In R. Salakhutdinov, Z. Kolter, K. Heller, A. ...

    2024

  12. [20]

    Liang, Y

    W. Liang, Y . Zhang, H. Cao, B. Wang, D. Y . Ding, X. Yang, K. V odra- halli, S. He, D. S. Smith, Y . Yin, D. A. McFarland, and J. Zou. Can large language models provide useful feedback on research papers? a large-scale empirical analysis.NEJM AI, 1(8):39, 2024. doi: 10. 1056/...

    2024

  13. [21]

    S. Lin, J. Warner, J. Zamfirescu-Pereira, M. G. Lee, S. Jain, S. Cai, P. Lertvittayakumjorn, M. X. Huang, S. Zhai, B. Hartmann, and C. Liu. Rambler: Supporting writing with speech via llm-assisted gist manipulation. InProceedings of the 2024 CHI Conference on Hu- man Factors i...

    2024 doi

  14. [22]

    K. Lo, J. C. Chang, A. Head, J. Bragg, A. X. Zhang, C. Trier, C. Anas- tasiades, T. August, R. Authur, D. Bragg, et al. The semantic reader project: Augmenting scholarly documents through AI-powered inter- active reading interfaces. 2023. doi: 10.48550/arXiv.2303.14334 2

    2023 doi arXiv

  15. [23]

    NeurIPS 2026 AI-assisted review- ing experiment.https://neurips.cc/Conferences/2026/ ai-reviewing-experiment, 2026

    NeurIPS 2026 Organizers. NeurIPS 2026 AI-assisted review- ing experiment.https://neurips.cc/Conferences/2026/ ai-reviewing-experiment, 2026. 1, 2

    2026

  16. [24]

    Russo, M

    G. Russo, M. Horta Ribeiro, T. R. Davidson, V . Veselovsky, and R. West. The AI review lottery: Widespread AI-assisted peer reviews boost paper scores and acceptance rates.Proc. ACM Hum.-Comput. Interact., 9(7), Oct. 2025. doi: 10.1145/3757667 1

    2025 doi

  17. [25]

    B. N. Schilit, G. Golovchinsky, and M. N. Price. Beyond paper: sup- porting active reading with free form digital ink annotations. InPro- ceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’98, p. 249–256. ACM Press/Addison-Wesley Publish- ing Co., US...

    1998

  18. [26]

    L. A. Schintler, C. L. McNeely, and J. Witte. A critical examination of the ethics of AI-mediated peer review. 2023. doi: 10.48550/arXiv. 2309.12356 1, 2

    2023 doi arXiv

  19. [27]

    N. B. Shah. Challenges, experiments, and computational solutions in peer review.Communications of the ACM, 65(6):76–87, 2022. doi: 10 .1145/3528086 1

    2022

  20. [28]

    L. Sun, S. Tao, J. Hu, and S. P. Dow. MetaWriter: Exploring the potential and perils of AI writing support in scientific peer review. Proc. ACM Hum.-Comput. Interact., 8(CSCW1), Apr. 2024. doi: 10. 1145/3637371 1

    2024

  21. [29]

    Y . Tanaka. Revised CHI 2026 papers desk reject process.https://chi2026.acm.org/2025/08/08/ revised-chi-2026-papers-desk-reject-process/, 2025. 1

    2026

  22. [30]

    C. S. Tashman and W. K. Edwards. LiquidText: a flexible, multitouch environment to support active reading. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, p. 3285–3294. Association for Computing Machinery, New York, NY , USA, 2011. doi: 1...

    2011

  23. [31]

    Thakkar, M

    N. Thakkar, M. Yuksekgonul, J. Silberg, A. Garg, N. Peng, F. Sha, R. Yu, C. V ondrick, and J. Zou. A large-scale randomized study of large language model feedback in peer review.Nature Machine Intel- ligence, 8(3):326–336, Mar. 2026. doi: 10.1038/s42256-026-01188-x 1, 2

    2026 doi

  24. [32]

    Q. Wang, Q. Zeng, L. Huang, K. Knight, H. Ji, and N. F. Rajani. ReviewRobot: Explainable paper review generation based on knowl- edge synthesis. InProceedings of the 13th International Conference on Natural Language Generation, pp. 384–397. Association for Com- putational Ling...

    2020 doi

  25. [33]

    cri- tique: Identifying a weakness, flaw, or disagreement

    Z. Zhuang, J. Chen, H. Xu, Y . Jiang, and J. Lin. Large language models for automated scholarly paper review: A survey.Information Fusion, 124:103332, 2025. doi: 10.1016/j.inffus.2025.103332 2 A APPENDIX: LLM PROMPTS This appendix includes the system prompts for the main LLM- ...

    2025

  26. [35]

    Do not add your own analysis

    Rewrite their annotation as a clear, concise, well−structured comment that PRESERVES ALL of their intellectual content, insights, questions, and critiques. Do not add your own analysis. Do not remove any substantive points they made. Just clean up the delivery . When the trans...

  27. [36]

    Use multiple tags ONLY when the comment genuinely spans categories (e.g

    Classify the comment with one or more tags from this list. Use multiple tags ONLY when the comment genuinely spans categories (e.g. a strength that also leads to a suggestion). Most comments need just one tag. {TAG LIST}

  28. [37]

    I have highlighted the following passage for context:

    Identify which section of the paper the highlighted passage is in. Use one of: {SECTION LIST} Use ”other” if it doesn’t fit. Infer from page context and content. Output ONLY valid JSON with exactly three fields: {”comment”: ”the cleaned annotation”, ”tags”: [”tag1”, ”tag2”], ”...

  29. [38]

    Identify 3–10 distinct sections the reviewer is expected to address

  30. [39]

    Examples: ”Novelty”, ”Soundness of Methods”, ”Clarity of Writing”

    For each section, write: − ”section”: a short, plain−language label (1–4 words, Title Case). Examples: ”Novelty”, ”Soundness of Methods”, ”Clarity of Writing”. − ”description”: a concise one−to−two−sentence explanation of what the reviewer should comment on for that section. U...

  31. [40]

    Skip meta−content (submission instructions, scoring scales, formatting rules) that the reviewer themselves does not need to write about

  32. [41]

    If the input is too short or vague to extract anything meaningful, return an empty array

    Do not invent sections that are not implied by the input. If the input is too short or vague to extract anything meaningful, return an empty array. Return ONLY valid JSON with this exact shape: {”items”:[{”section”:”...”,”description”:”...”},{”section”:”...”,”description ”:”.....

  33. [42]

    Use short titles (e.g

    Extract 3–8 of the most important components the rubric requires the review to address. Use short titles (e.g. ”Novelty”, ”Soundness of methods”)

  34. [43]

    − ”partial”: annotations touch on it but the coverage is shallow or one− sided

    For each component, decide how well the reviewer’s annotations address it: − ”covered”: one or more annotations clearly engage with this component. − ”partial”: annotations touch on it but the coverage is shallow or one− sided. − ”missing”: no annotation addresses this compone...

  35. [44]

    For each component, list the ids of the annotations that support your verdict (empty array if status is ”missing”)

  36. [45]

    Note context

    For each component, write a one−sentence gap summary describing what the reviewer should still address. For ”covered”, a brief affirmation is fine. Return a JSON object with this exact structure: {”components”:[{”title”:”...”,”description”:”...”,”status”:”covered”|” partial”|”...

  37. [2021]

    doi: 10.1145/3411764.3445648 2

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.