Pith. sign in

REVIEW 5 major objections 4 minor 24 references

A new framework checks whether a paper's claims are backed by its own methods, and its reviews align with human reviewers on novelty concerns.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 10:30 UTC pith:QOOSTOER

load-bearing objection A reasonable proof-of-concept for intra-paper claim verification, but the abstract's headline statistics come from the SBERT similarity analysis, not the human evaluation, and the evaluation is too small and under-controlled to support the strong wording. the 5 major comments →

arxiv 2607.26066 v1 pith:QOOSTOER submitted 2026-06-21 cs.CL cs.AIcs.DL

Do Methods Support the Claims? Intra-Paper Verification for Peer Review

Ranjitha Shivaprasad Ballakuraya , Arash Mahyari , Ashok Srinivasan This is my paper
classification cs.CL cs.AIcs.DL
keywords intra-paper claim verificationLLM-assisted peer reviewnovelty assessmentclaim substantiationsemantic similarityhuman-LLM alignmentscientific review automationmethodological evidence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces a task called intra-paper claim verification: checking whether the novelty claims a paper makes in its introduction are actually substantiated by the methods described later in the same paper. The authors build a four-stage pipeline that extracts claims, derives four evaluation categories from human reviewer comments, links each claim to methodological evidence, and generates structured review assessments. Human evaluation on a balanced set of accepted and rejected papers shows the generated reviews align with human reviewer concerns, most strongly for novelty-related issues, with a large effect size. The work matters because current automated novelty assessment only compares a paper against prior literature, missing the internal mismatch between claimed contributions and how they are actually realized.

Core claim

The central discovery is that an LLM-driven framework can identify whether a paper's stated novelty claims are backed by its own methodological evidence, and that the resulting assessments overlap with human reviewer concerns. In human evaluation, the mean alignment score was 3.29 on a five-point scale, and the novelty category showed statistically significant semantic correspondence (SBERT similarity 0.628, p<0.001, Cohen's d=1.17). Additionally, BERTScore perfectly discriminated reviews generated for the correct paper from reviews generated for a different paper, indicating the framework captures paper-specific concerns.

What carries the argument

The key mechanism is claim–method substantiation: each novelty claim extracted from the introduction is paired with explicit methodological evidence (algorithms, architectures, training strategies, implementation details) found in the methods section. Claims that trace to such evidence are marked substantiated; those without are flagged as unsupported. This claim-to-evidence linking, guided by reviewer-derived evaluation categories (novelty, methodology, clarity, other), drives structured review generation and enables direct comparison with human reviews.

Load-bearing premise

The evaluation categories are derived from the same corpus of papers that the twenty test papers are sampled from, so the apparent alignment between generated and human reviews may be inflated by this overlap.

What would settle it

Run the same alignment study with evaluation categories derived from a disjoint corpus (e.g., from a different conference or from meta-reviews only), and show that the novelty category no longer exceeds the 0.50 baseline with large effect size.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the framework works, reviewing systems can flag papers whose promises are not internally supported, before even consulting external literature.
  • It can complement literature-based novelty detection by adding an internal consistency check, reducing both false 'novel' and false 'incremental' judgments.
  • The structured categories allow for scalable semantic comparison via embeddings, suggesting automated evaluation of review alignment is feasible.
  • The success on novelty and weakness on clarity and other categories shows which reviewer concerns are easier to formalize, guiding future work.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The evaluation categories are induced from the same corpus used to sample the 20 test papers, so the measured alignment, especially for novelty, may be optimistically biased; a clean test requires categories derived from disjoint data.
  • The framework's focus on the introduction-methods link could be extended to other sections (e.g., results vs. claims) and other claim types (e.g., reproducibility, significance).
  • The perfectly separating BERTScore result may be partly an artifact of the small sample and the fake reviews being about different topics; larger-scale tests with more similar papers would be informative.
  • If alignment is confirmed across venues, the framework could be used to train novice reviewers by exposing them to structured claim-evidence reasoning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes 'intra-paper claim verification,' a framework that checks whether a paper's stated novelty claims are supported by its own methodology, rather than by comparison to external literature. The pipeline extracts novelty claims from the Introduction, derives four evaluation categories (Novelty, Methodology, Clarity, Other) from human reviews of 182 ICLR 2025 papers, retrieves methodological evidence for each claim, and generates structured LLM reviews using GPT-4o. The framework is evaluated on 20 papers (10 accepted, 10 rejected) by four scientist evaluators who rate alignment between LLM-generated and human reviews on a 5-point scale, plus automated SBERT and BERTScore similarity analyses. The paper claims that human evaluation demonstrates significant alignment, particularly for novelty-related issues (p<0.001, Cohen's d=1.17), and that BERTScore distinguishes corresponding from non-corresponding review pairs.

Significance. If the central empirical claim were sound, the paper would introduce a genuinely useful complementary task for LLM-assisted peer review: checking internal claim–method consistency rather than only external novelty. The framework is well-motivated and the pipeline is described in sufficient detail to be reproduced; the authors also make code, prompts, and materials public, which is a clear strength. However, the reported statistical evidence supporting the headline claim is misattributed: the p<0.001 and d=1.17 come from an SBERT similarity comparison against an arbitrary threshold, not from the human evaluation. The human evaluation itself is small (20 papers, 4 evaluators), shows only moderate absolute alignment (mean 3.29/5), reveals significant disagreement between evaluator groups, and lacks a comparison against a meaningful null or baseline. The inductive derivation of evaluation categories from the same corpus used for evaluation raises a circularity concern that is acknowledged but not addressed. These issues do not necessarily invalidate the framework, but they do mean that the paper's strongest claim is currently unsupported and the evaluation design needs substantial revi

major comments (5)
  1. [Section IV-A, Section V-A] The abstract and Section I state that 'Human evaluation demonstrates significant alignment between framework-generated assessments and human reviewer concerns, particularly for novelty-related issues (p < 0.001, Cohen's d = 1.17).' These statistics are not from the human evaluation. Table III reports only means, standard deviations, and a group comparison for the human ratings; no test against a null hypothesis or a control condition is run on the 5-point human scores. The p < 0.001 and d = 1.17 values in Table V come from a one-sided Wilcoxon test on SBERT cosine similarities for the Novelty category against an ad hoc threshold of 0.50. This is a direct misattribution of the statistical support for the paper's central claim. The authors should either (a) report an appropriate inferential analysis of the human ratings (e.g., comparing real vs. fake reviews on the ordinal scale, or testin
  2. [Section III Stage 2, Section IV-A, Section VII] The human evaluation as reported cannot support the claim of 'significant alignment' even descriptively. The mean rating is 3.29 on a 5-point scale where 3 means 'captures roughly half of the concerns' and 4 means 'most of the concerns with minor gaps.' Without a null distribution or a control condition, a mean of 3.29 is uninterpretable as evidence for alignment. Moreover, the two evaluator groups differ significantly (3.75 vs. 2.83, paired t = -5.40, p < 0.001), which indicates substantial rater subjectivity. No inter-rater reliability statistic (e.g., ICC or weighted kappa) is reported. The authors should provide per-item agreement measures and a comparison of real vs. fake reviews on the human ratings, which the current design already collects but does not analyze.
  3. [Section V-C, Figure 8] The evaluation categories are derived from the same 182-paper corpus from which the 20 evaluation papers are sampled. The framework's output is structured by these categories, and the human evaluators are asked to rate alignment within those same categories. This overlap can inflate alignment scores because the category structure was designed to match the human reviews in the evaluation set. The paper acknowledges this in Section VII, but no sensitivity analysis is provided. A minimal fix would be to derive categories from a disjoint subset of the corpus (e.g., 162 papers) and evaluate on the held-out 20 papers, or to run the category-induction procedure multiple times with different random splits and report stability. As written, the significant Novelty result in Table V could be partly an artifact of this circularity.
  4. [Section V-B, Tables IV-V] The 100% BERTScore discrimination between 'real' and 'fake' review pairs is expected and not informative about the framework's quality. The fake reviews are generated from entirely different papers, so the document-level semantic similarity between a human review and a mismatched framework review is almost guaranteed to be lower than for a matched pair. This experiment does not compare against any baseline, such as a generic LLM review generated without the intra-paper verification pipeline, or a review generated from the same paper by a different method. The claim that 'BERTScore may capture review-specific semantic correspondence' is therefore overstated. Please add a meaningful control condition, or reinterpret the result as a sanity check rather than evidence for the framework's validity.
  5. [Section V-B, Tables IV-V] The SBERT analysis uses a threshold of 0.50 as a 'baseline' without justification. This threshold is arbitrary; cosine similarities depend on the embedding model and text lengths. The one-sided Wilcoxon test against 0.50 is not a test against a meaningful null hypothesis about alignment with human reviewer concerns. The only significant category is Novelty (p < 0.001, d = 1.17), while Methodology (p = 0.351), Clarity (p = 0.877), and Other (p = 0.981) are not significant. This pattern is consistent with the possibility that the framework captures novelty concerns but not other types of reviewer concerns; the paper's general conclusion should be limited accordingly.
minor comments (4)
  1. [Section V-B, Figure 7] Typo: 'human reviews-derived evaluation categories' should be 'human-review-derived evaluation categories.' Similarly, 'human reviewed summary' in Section IV-A should be 'human review summary.'
  2. [References] Figure 7 would benefit from error bars or boxplot details, as the text reports standard deviations but the figure shows only distributions. Also, Table V has an extra vertical bar in the header ('One-Sided Wilcoxon t-test p') that should be cleaned.
  3. [Section IV-A] Reference [7] is incomplete: it lists a journal name but no article title or authors. Reference [8] (OpenReview) and [4] are informal online sources; please provide access dates or version identifiers for reproducibility.
  4. [Section IV-A] The paper says '20 papers were randomly selected' but does not specify the random seed or whether the selection was stratified beyond acceptance status. Please report the random selection procedure or make the list of selected papers explicit in the public repository so the evaluation can be reproduced.

Circularity Check

1 steps flagged

Partial circularity: the evaluation categories are derived from the same review corpus used as ground truth and then structure both compared outputs, inflating the headline Novelty statistic; the abstract additionally misattributes the SBERT p-value to the human evaluation.

specific steps
  1. self definitional [Section III Stage 2; Section IV-B; Section V-B; Section VII]
    "Human reviews collected from OpenReview.net for 182 ICLR 2025 papers are analyzed to identify recurring patterns of reviewer concerns ... Examination of these recurring themes reveals four dominant categories of reviewer concerns: Novelty Issue, Methodological Issue, Clarity Issue, and Other Issues. ... Human and framework-generated review summaries are first organized into the four evaluation categories derived in Section III. ... the evaluation categories were derived from reviewer feedback within this corpus and may not fully reflect evaluation practices in other research communities or pub"

    The same 182-paper review corpus supplies both the category taxonomy and the 20-paper evaluation subset. GPT-4o is prompted to generate reviews under these derived categories, while human reviews are relabeled into the same categories. The only statistically significant alignment result (Novelty vs the 0.50 baseline, Wilcoxon p<0.001, d=1.17) is therefore computed on a shared template fit to the gold-standard reviews. The measured novelty alignment is at least partly self-referential rather than an independent test of whether the framework captures human concerns.

full rationale

Most of the pipeline is not circular: claim extraction, claim-method evidence retrieval, and structured review generation are implemented independently of the alignment labels, and the human evaluator study (mean 3.29, median 3.50; evaluator-group difference) is a genuine if modest measurement. The BERTScore real-vs-fake discrimination (100%) is a proper control and is not contaminated by the category overlap. However, the paper's headline quantitative support (Abstract: 'Human evaluation demonstrates significant alignment ... (p <0.001, Cohen's d=1.17)') is not actually derived from the human evaluation; Section V-B reports that this p-value and effect size come from a one-sided Wilcoxon test on SBERT cosine similarities for the Novelty category. That category was inductively derived from the same 182-review corpus that contains the 20 evaluation papers, and the same category template was used to structure both the LLM-generated and human summaries. The author-acknowledged limitation in Section VII confirms the same-corpus derivation. This category leakage makes the flagship novelty statistic partially constructed by the evaluation setup, but the human ratings and BERTScore discrimination provide independent supporting content; no fitted parameter is simply renamed as a prediction, and no self-citation chain is load-bearing. Overall this is a moderate partial circularity, not a total reduction.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 0 invented entities

The framework relies on the assumptions that claims are in the Introduction, evidence in the Methods, and that human OpenReview comments are a gold standard. The main circularity concern is that the evaluation categories are derived from the same corpus as the evaluation subset. The only hand-set numeric constants are the SBERT baseline threshold of 0.50 and the choice of 4 categories. No new physical or conceptual entities are postulated.

free parameters (2)
  • SBERT similarity baseline threshold = 0.50
    A fixed threshold chosen for the one-sided Wilcoxon test; it is arbitrary and not derived from the data.
  • Number of evaluation categories = 4
    Determined by GPT-4o thematic analysis; the choice affects the structure of both the generated reviews and the human-review comparison.
axioms (6)
  • domain assumption Novelty claims can be reliably extracted from the Introduction using a fixed LLM prompting strategy.
    Stage 1 relies on GPT-4o with a prompt adapted from [10] to identify claims; if extraction is noisy, downstream verification is compromised.
  • domain assumption The Methods section contains the methodological evidence needed to substantiate novelty claims.
    Stage 3 retrieves evidence only from the Methods section; claims supported by experiments, related work, or appendices could be misclassified.
  • domain assumption OpenReview human reviews for ICLR 2025 are a valid gold standard for reviewer concerns.
    Stage 2 uses 786 reviews from OpenReview as ground truth; reviewer comments are noisy and may not represent all valid scientific criticism.
  • domain assumption The four inductively derived categories are sufficient and transferable across the evaluation subset.
    Derived from the same 182-paper corpus; the 20 evaluation papers are a subset, risking leakage. The paper's own limitations section acknowledges this.
  • domain assumption GPT-4o produces sufficiently reliable assessment for the claim-method substantiation analysis.
    The entire pipeline depends on GPT-4o's generation; no independent verification of the LLM's claim-method matching is provided.
  • domain assumption Human evaluator ratings on the five-point ordinal scale are a valid measure of review alignment.
    Section IV.A uses four scientists' ratings; inter-rater reliability is not reported and the scale's interpretation may vary.

pith-pipeline@v1.3.0-alltime-deepseek · 10437 in / 11877 out tokens · 96974 ms · 2026-08-02T10:30:55.834196+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Do Methods Support the Claims? Intra-Paper Verification for Peer Review." pith.science (2026). https://pith.science/paper/QOOSTOER

@misc{pith2026260726066,
  author       = {Pith},
  title        = {Pith review of: Do Methods Support the Claims? Intra-Paper Verification for Peer Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QOOSTOER}},
  note         = {Machine review of arXiv:2607.26066}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The growing volume of scientific submissions has motivated interest in using large language models (LLMs) to assist peer review. Existing automated novelty assessment approaches typically compare a paper's claimed contributions against prior literature, implicitly assuming that these contributions are accurately realized in the work itself. Human reviewers, however, frequently challenge novelty claims not because similar ideas already exist, but because the methodological evidence presented in the paper does not adequately support them. This internal mismatch between claimed contributions and methodological realization is rarely examined by current LLM-based review systems. To address this gap, we introduce intra-paper claim verification, a framework that evaluates whether novelty claims articulated in a paper are substantiated by the methods used to realize them. The framework employs an LLM to extract novelty claims from the introduction, retrieve claim-relevant methodological evidence, and assess whether the methods substantiate the stated contributions. Assessment is guided by reviewer-inspired evaluation criteria derived inductively from human peer reviews collected from 182 ICLR 2025 papers. These criteria capture recurring reviewer concerns related to novelty, methodology, clarity, and other issues and are used to generate structured reviewer-style assessments of claim substantiation. We evaluate the framework by comparing LLM-generated review comments against human reviewer concerns on a balanced subset of accepted and rejected papers. Human evaluation demonstrates significant alignment between framework-generated assessments and human reviewer concerns, particularly for novelty-related issues. BERTScore further distinguishes corresponding human-LLM review pairs from mismatched controls, indicating that the framework captures concerns consistent with human reviewer observations.

Figures

Figures reproduced from arXiv: 2607.26066 by Arash Mahyari, Ashok Srinivasan, Ranjitha Shivaprasad Ballakuraya.

Figure 1
Figure 1. Figure 1: Proposed framework for intra-paper claim verification and LLM peer review benchmarking. Academic papers are converted from PDF to structured [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Workflow for PDF-to-Markdown conversion and novelty claim [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. Figure 5: Structured review generation and summarization workflow [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Category-level SBERT similarity distributions across novelty, method [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of BERTScore similarity between human reviews and cor [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 6 linked inside Pith

  1. [1]

    The strain on scientific publishing,

    M. A. Hanson, P. G. Barreiro, P. Crosetto, and D. Brockington, “The strain on scientific publishing,”Quantitative Science Studies, vol. 5, no. 4, pp. 823–843, 2024

    2024

  2. [2]

    The present and future of peer review: Ideas, interventions, and evidence,

    B. Aczel, A.-S. Barwich, A. B. Diekman, A. Fishbach, R. L. Goldstone, P. Gomez, O. E. Gundersen, P. T. von Hippel, A. O. Holcombe, S. Lewandowskyet al., “The present and future of peer review: Ideas, interventions, and evidence,”Proceedings of the National Academy of Sciences, vol. 122, no. 5, p. e2401232121, 2025

    2025

  3. [3]

    Peer review of scientific studies: Problems and potential solutions,

    S. Gupta and A. Sarkar, “Peer review of scientific studies: Problems and potential solutions,” vol. 17, no. 10, 2025

    2025

  4. [4]

    The neurips experiment,

    N. D. Lawrence, “The neurips experiment,” https://inverseprobability. com/talks/notes/the-neurips-experiment-snsf.html, 2022, accessed: 2025-05-23

    2022

  5. [5]

    Agentreview: Exploring peer review dynamics with llm agents,

    Y . Jin, Q. Zhao, Y . Wang, H. Chen, K. Zhu, Y . Xiao, and J. Wang, “Agentreview: Exploring peer review dynamics with llm agents,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 1208–1226

    2024

  6. [6]

    Ai-assisted peer review at scale: The aaai-26 ai review pilot,

    J. Biswas, S. Schoepp, G. Vasan, A. Opipari, A. Zhang, Z. Hu, S. Joseph, M. Lease, J. J. Li, P. Stoneet al., “Ai-assisted peer review at scale: The aaai-26 ai review pilot,”arXiv preprint arXiv:2604.13940, 2026. [7]Nature Computational Science, 2026. [Online]. Available: https: //www.nature.com/articles/s43588-026-00989-9

    Pith/arXiv arXiv 2026

  7. [8]

    (2026) OpenReview: A Venue for Open Peer Review

    OpenReview. (2026) OpenReview: A Venue for Open Peer Review. [Online]. Available: https://openreview.net

    2026

  8. [9]

    Why ai can’t be trusted to write scientific reviews,

    R. Sarkar, “Why ai can’t be trusted to write scientific reviews,”Nature, vol. 653, no. 8116, pp. 983–983, 2026

    2026

  9. [10]

    Beyond “not novel enough

    O. M. Afzal, P. Nakov, T. Hope, and I. Gurevych, “Beyond “not novel enough”: Enriching scholarly critique with LLM-assisted feedback,” inProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), V . Demberg, K. Inui, and L. Marquez, Eds. Rabat, Morocco: Association for Computationa...

    2026

  10. [11]

    Schol- areval: Research idea evaluation grounded in literature,

    H. N. Moussa, P. Q. Da Silva, D. Adu-Ampratwum, A. East, Z. Lu, N. Puccetti, M. Xue, H. Sun, B. P. Majumder, and S. Kumar, “Schol- areval: Research idea evaluation grounded in literature,”arXiv preprint arXiv:2510.16234, 2025

    arXiv 2025

  11. [12]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radfordet al., “Gpt-4o system card,”arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  12. [13]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” inProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP- IJCNLP), 2019, pp. 3982–3992

    2019

  13. [14]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,”arXiv preprint arXiv:1904.09675, 2019

    Pith/arXiv arXiv 1904

  14. [15]

    Reviewer2: Optimiz- ing review generation through prompt generation,

    Z. Gao, K. Brantley, and T. Joachims, “Reviewer2: Optimiz- ing review generation through prompt generation,”arXiv preprint arXiv:2402.10886, 2024

    Pith/arXiv arXiv 2024

  15. [16]

    Openreviewer: A specialized large language model for generating critical scientific paper reviews,

    M. Idahl and Z. Ahmadi, “Openreviewer: A specialized large language model for generating critical scientific paper reviews,” inProceedings of the 2025 Conference of the Nations of the Americas Chapter of the As- sociation for Computational Linguistics: Human Language Technologies (System Demonstrations), 2025, pp. 550–562

    2025

  16. [17]

    Marg: Multi-agent review generation for scientific papers,

    M. D’Arcy, T. Hope, L. Birnbaum, and D. Downey, “Marg: Multi-agent review generation for scientific papers,”arXiv preprint arXiv:2401.04259, 2024

    Pith/arXiv arXiv 2024

  17. [18]

    Reviewrobot: Explainable paper review generation based on knowledge synthesis,

    Q. Wang, Q. Zeng, L. Huang, K. Knight, H. Ji, and N. F. Rajani, “Reviewrobot: Explainable paper review generation based on knowledge synthesis,” inProceedings of the 13th International Conference on Natural Language Generation, 2020, pp. 384–397

    2020

  18. [19]

    Mamorx: Multi-agent multi-modal scientific review generation with external knowledge,

    P. Taechoyotin, G. Wang, T. Zeng, B. Sides, and D. Acuna, “Mamorx: Multi-agent multi-modal scientific review generation with external knowledge,” inNeurips 2024 Workshop Foundation Models for Science: Progress, Opportunities, and Challenges, 2024

    2024

  19. [20]

    Autorev: Multi-modal graph retrieval for automated peer- review generation,

    M. P. Chitale, K. M. Shetye, H. Gupta, M. Chaudhary, M. Shrivastava, and V . Varma, “Autorev: Multi-modal graph retrieval for automated peer- review generation,”arXiv preprint arXiv:2505.14376, 2025

    arXiv 2025

  20. [21]

    Sc4anm: Identifying optimal section combinations for automated novelty prediction in academic papers,

    W. Wu, C. Zhang, T. Bao, and Y . Zhao, “Sc4anm: Identifying optimal section combinations for automated novelty prediction in academic papers,”Expert Systems with Applications, vol. 273, p. 126778, 2025

    2025

  21. [22]

    Literature-grounded novelty assessment of scientific ideas,

    S. Shahid, M. Radensky, R. Fok, P. Siangliulue, D. S. Weld, and T. Hope, “Literature-grounded novelty assessment of scientific ideas,” in Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), 2025, pp. 96–113

    2025

  22. [23]

    Opennovelty: An llm-powered agentic system for verifiable scholarly novelty assessment,

    M. Zhang, K. Tan, Y . Huang, Y . Shen, C. Ma, L. Ju, X. Zhang, Y . Wang, W. Jing, J. Denget al., “Opennovelty: An llm-powered agentic system for verifiable scholarly novelty assessment,”arXiv preprint arXiv:2601.01576, 2026

    arXiv 2026

  23. [24]

    GraphMind: Interactive novelty assessment system for accelerating scientific discovery,

    I. L. da Silva, H. Yan, L. Gui, and Y . He, “GraphMind: Interactive novelty assessment system for accelerating scientific discovery,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, I. Habernal, P. Schulam, and J. Tiedemann, Eds. Suzhou, China: Association for Computational Linguistics, Nov...

    2025

  24. [25]

    Mineru: An open-source solution for precise document content extraction,

    B. Wang, C. Xu, X. Zhao, L. Ouyang, F. Wu, Z. Zhao, R. Xu, K. Liu, Y . Qu, F. Shanget al., “Mineru: An open-source solution for precise document content extraction,”arXiv preprint arXiv:2409.18839, 2024

    Pith/arXiv arXiv 2024