Pith. sign in

REVIEW 3 major objections 5 minor 51 references

ResearchPulse claims that research evolution can be tracked automatically by a modular pipeline of small 7B-parameter agents that extract motivation–method chains and experimental trends, outperforming direct prompting of GPT-4o on its own

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 10:55 UTC pith:JIFJW5LB

load-bearing objection Solid benchmark idea, untenable 7B-superiority claim: GPT-4o is inside the system, the references, and the judge. the 3 major comments →

arxiv 2509.03565 v1 pith:JIFJW5LB submitted 2025-09-03 cs.CL cs.MM

ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference

Qi Chen , Jingxuan Wei , Zhuoya Yao , Haiguang Wang , Gaowei Wu , Bihui Yu , Siyuan Li , Cheng Tan This is my paper
classification cs.CL cs.MM
keywords multi-document scientific inferencemethod trackingexperimental analysisagent-based frameworkmind map generationline chart generationcitation-aware benchmarkresearch trajectory visualization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Understanding how a research field develops usually means reading many papers and working out who built on whom and which methods won. This paper argues that this can be turned into a machine task—multi-document scientific inference—in which related papers are mined for their motivation, method, and experimental results, aligned by publication time, and rendered as a mind map of the method lineage and a line chart of benchmark scores over time. To test this, the authors built ResearchPulse, a pipeline of coordinated agents (a planner, a mind-map agent, and a line-chart agent) all based on 7-billion-parameter language models, plus a benchmark of 100 citation-aware paper clusters. Their headline result is that this small, specialized pipeline beats strong baselines such as GPT-4o in semantic alignment, structural consistency, and visual fidelity. If correct, the work suggests that structured research tracking—a task usually done by expert readers—can be automated at modest model scale and made visually checkable.

Core claim

This paper formalizes multi-document scientific inference: extracting and aligning motivation, methodology, and experimental results across thematically related papers to reconstruct research development chains. It presents ResearchPulse, a pipeline of three coordinated agents built on 7-billion-parameter language models: a Plan Agent that routes instructions, a Mmap-Agent that extracts per-paper (motivation, method) tuples and sorts them by publication time into a mind map, and a Lchart-Agent that extracts experimental tables, model names, metrics, and baseline years, aligns them over time, and auto-generates Python to plot benchmark trajectories. The paper also contributes ResearchPulse-Be

What carries the argument

The carrying mechanism is the split of scientific reading into two structural extractions, each owned by a specialized fine-tuned agent. The Mmap-Agent reduces each paper to a (motivation, method) tuple drawn from abstract and introduction, then sorts tuples by publication timestamp to form a research chain rendered as a hierarchical mind map. The Lchart-Agent reduces each paper to its main experimental table, model names, metric values, and baseline years, resolves the years through citations, aligns values across papers, and generates executable Python that plots metric-over-time line charts. A Plan Agent classifies the user instruction and routes parsed document sections to the right agen

Load-bearing premise

The load-bearing premise is that the benchmark's clusters and reference outputs are reliable ground truth: clusters were assembled by citation-graph and embedding similarity, and the reference mind maps and charts were drafted by large models and then 'human validated' without reported annotator counts, agreement rates, or correction statistics, so all system comparisons inherit whatever errors those references contain.

What would settle it

Have domain experts independently rebuild the reference outputs for a random sample of ResearchPulse-Bench clusters straight from the source PDFs, with no model-assisted pre-extraction, then recompute the Mmap-Agent and Lchart-Agent scores against these human-built references. The central claim stands or falls on whether the 7B pipeline still beats direct GPT-4o prompting on the clean targets; the paper's own error analysis in Section 5.3, which documents factual deviations and missing extractions in the systems' outputs, indicates the references must be verified before the deltas are trusted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Research-trend tracking becomes a local, small-model task: a lab can run the full method-tracking and experimental-analysis pipeline on 7-billion-parameter models rather than depending on a frontier closed API.
  • Method evolution is delivered as a time-ordered mind map whose branches can be traced to specific papers, giving readers a checkable alternative to prose surveys.
  • Heterogeneous experimental tables are normalized into metric-over-time line charts, so performance comparisons across papers with different table formats become direct and visual.
  • The benchmark's 1,958 method-tracking and 1,550 experimental-analysis training samples provide supervised signal for training future extraction agents.
  • Ablations show the advantage comes from the modular design—the feedback module and the output compiler—not from model scale alone, so pipeline structure is a design variable worth optimizing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same motivation–method–result chain appears in any cumulative science, so the pipeline should transfer to fields like biomedicine or systems research once citation-aware clusters and experimental tables are available; the clustering step is already domain-agnostic.
  • Inference: scoring line charts with image-quality metrics (FID, SSIM) rewards visual similarity rather than numerical correctness, so a natural follow-up metric would measure whether each plotted value matches the source table.
  • Inference: the mind-map output suggests a new evaluation axis the paper does not measure—traceability, i.e., whether every node can be attributed to a specific source paper, which is what makes the artifact verifiable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces multi-document scientific inference as a task, presents ResearchPulse-Bench, a citation-aware benchmark of 100 paper clusters with reference mind maps and line charts, and proposes ResearchPulse, a three-agent system (Plan Agent, Mmap-Agent, Lchart-Agent) for method tracking and experimental analysis. The main empirical claim is that ResearchPulse, despite using 7B-scale agents, outperforms strong baselines such as GPT-4o on semantic alignment, structural consistency, and visual fidelity (Tables 3a and 4a). A careful reading shows, however, that the system as reported is incompletely specified: the method section never defines the 'GPT-4o module' or 'Compiler module' that appear in the ablations, the Plan Agent is based on Qwen-72B, and the evaluation pipeline uses GPT-4o as reference generator, judge, baseline, and internal component. These issues undermine the reproducibility of the central claim.

Significance. The task formulation and the benchmark resource could be useful to the community if the data and annotations are released with proper validation. The idea of temporally aligning motivation–method pairs and experimental results across related papers is a reasonable extension of multi-document summarization. The modular agent design is also interesting. However, the paper's headline contribution—that 7B-scale agents consistently outperform GPT-4o—is not established by the reported experiments because the architecture is not fully specified and the evaluation is circular. The reported ablations actually show that removing the undefined GPT-4o component erases or reverses the claimed advantage (Table 3b). For the paper to be accepted, the authors would need to define all components, remove GPT-4o from the evaluation loop, and provide independent human validation of the benchmark.

major comments (3)
  1. [3.2; 5.1; Tables 3b/4b] The method section defines only FMmap and FLchart (Eqs. 2 and 5), yet the ablations in Tables 3b and 4b remove a 'GPT-4o module' and a 'Compiler module' that are never defined in the architecture. Additionally, §5 states the Plan Agent uses Qwen-72B, so the claim of a '7B-scale' system is already imprecise at the system level. This makes the headline comparison unfalsifiable as reported: the reader cannot tell whether the 7B fine-tuned agents, the GPT-4o component, or the compiler drive the gains. Table 3b is direct evidence of the problem: removing 'GPT-4o' drops METEOR from 46.14 to 41.78, below the GPT-4o baseline's 42.66 in Table 3a, and removing both modules drops BERTScore F1 to 82.83. The abstract's claim is therefore contradicted by the reported system composition unless the GPT-4o module is fully specified and shown not to be doing the generation.
  2. [4; 4.2; Figure 3] The evaluation is circular. Figure 3 shows GPT-4o/DeepSeek-R1 generating the benchmark reference summaries and code, Section 4.2 defines GPT-Score as a rating produced by GPT-4o, Table 3a includes GPT-4o as a baseline, and Table 3b indicates GPT-4o is also an internal module of the proposed system. Thus GPT-4o serves simultaneously as reference creator, quality judge, baseline, and part of the tested system. The reported deltas between Mmap-Agent and GPT-4o are not independent. A concrete remedy is to evaluate on a held-out set with human-annotated references, report annotator counts and agreement, and ablate the proposed system with all GPT-4o components removed.
  3. [4; 5.3] The benchmark's ground truth is load-bearing, but its validation is only described as 'human inspection' without annotator counts, inter-annotator agreement, or correction rates. Section 5.3's own error analysis documents factual deviations, misattributed innovations, missing extractions, and partial references in model outputs. If the reference outputs contain such errors, all system comparisons in Tables 3 and 4 are against noisy targets. The authors should report a reliability analysis of the human validation step and, ideally, release the annotation and correction records so that the benchmark's validity can be assessed.
minor comments (5)
  1. [5] The text says 'four core agents: Plan Agent, Mmap-Agent, and Lchart-Agent' but only three agents are listed. The count should be corrected.
  2. [Figure 2] Figure 2 contains a large block of Chinese text and the caption is duplicated in Chinese. This should be translated and the figure cleaned for a general readership.
  3. [Table 3a] The main-results text says Mmap-Agent 'leads in METEOR (46.14)', but InternLM3-8B-Instruct also reports 46.14 in the same table. The tie should be acknowledged, or the table corrected.
  4. [References] References [46] and [47] are both listed as the Qwen technical report, but the text cites [47] as the Qwen2.5-Coder technical report. The citation should be checked and corrected.
  5. [Table 2] The cluster distribution table is ambiguous: it reports 80/20 train/test clusters but also totals of 100, and the 'Minimum/Maximum/Average Papers' columns mix train, test, and total. Clarify whether the same clusters are used for both tasks and how the splits are drawn.

Circularity Check

3 steps flagged

Claimed 7B-only superiority is self-referential: the '7B' agents contain an undeclared GPT-4o module, and GPT-4o also authors the benchmark references and acts as the GPT-Score judge.

specific steps
  1. self definitional [Abstract; Section 3.2.2 Eq. (2); Section 5.1 Ablation Study (Table 3b)]
    "As illustrated in Table 3, the removal of the GPT-4o module resulted in notable declines in fluency and creativity, with BERTScore F1 dropping to 86.91."

    The abstract claims 'our system, despite using 7B-scale agents, consistently outperforms strong baselines like GPT-4o'. Table 3(a) labels 'Mmap-Agent 7B' as beating GPT-4o, but the ablation shows this 7B agent contains a 'GPT-4o module'. Section 3.2.2 defines Mmap-Agent only as FMmap, a fine-tuned LLM (Eq. 2), with no GPT-4o module in the method. Removing GPT-4o drops METEOR to 41.78, below the GPT-4o baseline's 42.66. The compared system is therefore 7B-extractor + GPT-4o + Compiler versus GPT-4o; GPT-4o is on both sides, so the claimed '7B-scale' outperformance reduces by construction to a GPT-4o-augmented pipeline beating GPT-4o.

  2. self definitional [Section 3.2.3 Eq. (5); Section 5.2 Ablation Study (Table 4b)]
    "As shown in Table 4, removing the GPT-4o module resulted in a decrease in IS to 2.32 and an increase in FID to 7.13, indicating the module’s crucial role in maintaining high-quality image generation."

    Table 4(a) reports 'Lchart-Agent 7B' with Pass@1 97.50 and FID 6.73, outperforming GPT-4o (96.25, 7.15). The method defines Lchart-Agent as FLchart (Eq. 5) and says the chart is 'Python code automatically generated by the agent' (Eq. 7). The ablation removes a 'GPT-4o module' and a 'Compiler module': removing GPT-4o raises FID to 7.13, and removing the Compiler drops Pass@1 to 90.63 and FID to 8.02. Thus the visual/code-generation advantage is not produced by the 7B Lchart model alone; it depends on GPT-4o and an external compiler. The comparison reduces by construction to a GPT-4o-containing system versus the GPT-4o baseline.

  3. other [Section 4 (Data Processing); Section 4.2 (Evaluation Metrics)]
    "Within each cluster, we use DeepSeek-R1 or GPT-4o [44] to extract fine-grained elements including motivations, methods, and experimental results."

    The benchmark's reference outputs are generated by GPT-4o (or DeepSeek-R1), and the qualitative GPT-Score is 'generated by GPT-4o' (Section 4.2). GPT-4o is thus simultaneously the reference-label generator, the judge, the baseline in Tables 3(a)/4(a), and—per the ablations—an internal module of the proposed agents. Human validation is asserted but no annotator counts, agreement, or correction rates are reported, so the loop is not independently broken. The reported 'outperforms GPT-4o' is therefore a self-referential evaluation: GPT-4o-containing agents are compared against GPT-4o-authored references and scored by GPT-4o.

full rationale

The central derivation chain is the experimental claim that a 7B-scale agent pipeline outperforms closed-source GPT-4o. Inspecting the method and ablation sections shows this claim does not follow from the described 7B components alone. Mmap-Agent and Lchart-Agent are defined in Section 3.2 as fine-tuned 7B LLMs (Eqs. 2 and 5), but Section 5's ablations explicitly remove a 'GPT-4o module' and a 'Compiler module' from both agents, and removing GPT-4o erases or reverses the margin over GPT-4o. The compared system is therefore 7B + GPT-4o + Compiler, making the headline comparison self-referential. Compounding this, the benchmark references and GPT-Score are produced by GPT-4o (Section 4 / 4.2), so GPT-4o is inside the proposed system, inside the gold labels, and inside the evaluation metric. This is not a harmless self-citation or an independent external benchmark; no load-bearing self-citation or uniqueness theorem is involved. The circularity is in the evaluation/comparison: the result reduces by construction to a GPT-4o-augmented system beating a GPT-4o baseline on GPT-4o-authored references judged by GPT-4o. Human validation is mentioned but not quantified, so it does not establish independence. Because there is a genuine fine-tuned 7B component and a newly released dataset, the paper is not entirely vacuous, but the central superiority claim as reported is not supported. Score 7 reflects partial circularity by construction, without going to 8-10.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 4 invented entities

The central system comparison rests on: (1) benchmark cluster validity built by citation graph plus K-Means, (2) ground-truth references generated by GPT-4o/DeepSeek-R1 under lightly documented human validation, (3) GPT-4o as judge, component, and baseline, and (4) automatic timestamp and citation-year resolution. The fine-tuned model weights, the in-pipeline GPT-4o module, and the Compiler module are fitted or external components that are not released, so the net contribution is the benchmark plus a partially specified pipeline.

free parameters (4)
  • Mmap-Agent fine-tuned weights (Qwen2.5-7B) = not released
    Fine-tuned for 4 epochs on model-assisted, human-validated extraction data; Table 3 method-tracking results depend on these weights.
  • Lchart-Agent fine-tuned weights (Qwen2.5-Coder-7B) = not released
    Fine-tuned for 4 epochs on aligned tables and metric annotations; Table 4 experimental-analysis results depend on these weights.
  • GPT-4o module (internal pipeline component) = proprietary external model
    Tables 3b/4b 'w/o GPT-4o' ablations degrade results (METEOR 46.14 to 41.78; FID 6.73 to 7.13); the module is never defined in Section 3.
  • Compiler module (code execution tool) = unspecified
    Tables 3b/4b 'w/o Compiler' lowers BERTScore F1 to 83.49 and Pass@1 to 90.63; not defined in Section 3.
axioms (4)
  • domain assumption Citation-aware K-Means clusters from arXiv/OpenReview papers are semantically and temporally coherent research lineages
    Section 4 Data Collection: clusters built by forward/backward citation retrieval plus Sentence-BERT K-Means; every benchmark instance and every system comparison inherits this structure.
  • domain assumption LLM-generated extractions that pass human inspection are factually correct ground truth
    Section 4 Data Processing/Human Inspection: references produced by DeepSeek-R1 or GPT-4o are 'rigorously' human-validated, but no annotator counts, agreement statistics, or correction rates are reported.
  • ad hoc to paper GPT-4o's five-dimension rating is a valid unbiased proxy for human quality judgment
    Section 4.2 GPT-Score; the same model is an internal pipeline component and a baseline, so the judge is not independent of the systems it rates.
  • domain assumption Publication timestamps and cited baseline years are correctly resolved for temporal alignment
    Sections 3.2.2/3.2.3: Mchain is sorted by t_i and Echain aligns metric values by citation-resolved baseline years Y_i; wrong years would reorder the chains and charts.
invented entities (4)
  • ResearchPulse-Bench independent evidence
    purpose: Annotated benchmark of 100 citation-aware paper clusters with reference mind maps and line charts for two tasks
    Public Hugging Face URL is an external artifact checkable against source papers, though I could not verify that the URL resolves in this review.
  • Mmap-Agent no independent evidence
    purpose: Extracts temporally aligned motivation-method tuples and renders mind maps
    No released weights or code; its reported behavior is only observable through the paper's own tables.
  • Lchart-Agent no independent evidence
    purpose: Extracts experimental tables, model names, metrics, baseline years and generates line-chart Python code
    No released weights or code; behavior only observable through the paper's own tables.
  • GPT-Score no independent evidence
    purpose: GPT-4o-generated 0-1 ratings along five dimensions used as the qualitative evaluation metric
    'Human-aligned' is asserted, but no human-correlation study, annotator set, or agreement statistic is reported.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 14633 in / 22567 out tokens · 203417 ms · 2026-08-05T10:55:49.013683+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference." pith.science (2026). https://pith.science/paper/JIFJW5LB

@misc{pith2026250903565,
  author       = {Pith},
  title        = {Pith review of: ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JIFJW5LB}},
  note         = {Machine review of arXiv:2509.03565}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Understanding how scientific ideas evolve requires more than summarizing individual papers-it demands structured, cross-document reasoning over thematically related research. In this work, we formalize multi-document scientific inference, a new task that extracts and aligns motivation, methodology, and experimental results across related papers to reconstruct research development chains. This task introduces key challenges, including temporally aligning loosely structured methods and standardizing heterogeneous experimental tables. We present ResearchPulse, an agent-based framework that integrates instruction planning, scientific content extraction, and structured visualization. It consists of three coordinated agents: a Plan Agent for task decomposition, a Mmap-Agent that constructs motivation-method mind maps, and a Lchart-Agent that synthesizes experimental line charts. To support this task, we introduce ResearchPulse-Bench, a citation-aware benchmark of annotated paper clusters. Experiments show that our system, despite using 7B-scale agents, consistently outperforms strong baselines like GPT-4o in semantic alignment, structural consistency, and visual fidelity. The dataset are available in https://huggingface.co/datasets/ResearchPulse/ResearchPulse-Bench.

Figures

Figures reproduced from arXiv: 2509.03565 by Bihui Yu, Cheng Tan, Gaowei Wu, Haiguang Wang, Jingxuan Wei, Qi Chen, Siyuan Li, Zhuoya Yao.

Figure 1
Figure 1. Figure 1: ResearchPulse transforms scientific papers into mind maps and line charts for structured research tracking. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The ResearchPulse pipeline, consisting of three main agents: Plan Agent, Mmap-Agent, and Lchart-Agent. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The construction pipeline of ResearchPulse-Bench. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Representative examples from Method-Tracking (top) and Experimental-Analysis (bottom) tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The error examples of Mmap-Agent and Lchart [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 34 canonical work pages · 3 internal anchors

  1. [1]

    Cvt-slr: Contrastive visual-textual transformation for sign language recognition with variational alignment

    Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, and Stan Z Li. Cvt-slr: Contrastive visual-textual transformation for sign language recognition with variational alignment. In CVPR, pages 23141–23150, 2023

    2023

  2. [2]

    Simvpv2: Towards simple yet powerful spatiotemporal predictive learning.IEEE Transactions on Multimedia, 2025

    Cheng Tan, Zhangyang Gao, Siyuan Li, and Stan Z Li. Simvpv2: Towards simple yet powerful spatiotemporal predictive learning.IEEE Transactions on Multimedia, 2025

    work page 2025

  3. [3]

    Openstl: A comprehensive benchmark of spatio- temporal predictive learning

    Cheng Tan, Siyuan Li, Zhangyang Gao, Wenfei Guan, Zedong Wang, Zicheng Liu, Lirong Wu, and Stan Z Li. Openstl: A comprehensive benchmark of spatio- temporal predictive learning. Advances in Neural Information Processing Systems , 36:69819–69831, 2023

    work page 2023

  4. [4]

    Peek across: Improving multi-document modeling via cross-document question- answering

    Avi Caciularu, Matthew Peters, Jacob Goldberger, Ido Dagan, and Arman Cohan. Peek across: Improving multi-document modeling via cross-document question- answering. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers) , pages 1970–1...

    work page 1970

  5. [5]

    Compressed heterogeneous graph for abstractive multi-document summarization

    Miao Li, Jianzhong Qi, and Jey Han Lau. Compressed heterogeneous graph for abstractive multi-document summarization. In AAAI, volume 37, pages 13085– 13093, 2023

    work page 2023

  6. [6]

    Structvrm: Aligning multimodal reasoning with structured and verifiable reward models, 2025

    Xiangxiang Zhang, Jingxuan Wei, Donghong Zhong, Qi Chen, Caijun Jia, Cheng Tan, Jinming Gu, Xiaobo Qin, Zhiping Liu, Liang Hu, Tong Sun, Yuchen Wu, Zewei Sun, Chenwei Lou, Hua Zheng, Tianyang Zhan, Changbao Wang, Shuangzhi Wu, Zefa Lin, Chang Guo, Sihang Yuan, Riwei Chen, Shixiong Zhao, Yingping Zhang, Gaowei Wu, Bihui Yu, Jiahui Wu, Zhehui Zhao, Qianqian...

    work page 2025

  7. [7]

    Text summarization with pretrained encoders

    Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. In EMNLP-IJCNLP, pages 3730–3740, 2019

    work page 2019

  8. [8]

    Long-span summarization via local attention and content selection

    Potsawee Manakul and Mark Gales. Long-span summarization via local attention and content selection. In ACL, pages 6026–6041, 2021

    work page 2021

  9. [9]

    Hegel: Hypergraph transformer for long document summarization

    Haopeng Zhang, Xiao Liu, and Jiawei Zhang. Hegel: Hypergraph transformer for long document summarization. In EMNLP, pages 10167–10176, 2022

    work page 2022

  10. [10]

    Hierarchical attention graph for scientific document summarization in global and local level

    Chenlong Zhao, Xiwen Zhou, Xiaopeng Xie, and Yong Zhang. Hierarchical attention graph for scientific document summarization in global and local level. In NAACL, pages 714–726, 2024

    work page 2024

  11. [11]

    Generating a Structured Summary of Numerous Academic Papers: Dataset and Method

    Shuaiqi Liu, Jiannong Cao, Ruosong Yang, and Zhiyuan Wen. Generating a structured summary of numerous academic papers: Dataset and method. arXiv preprint arXiv:2302.04580, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  12. [12]

    Chime: Llm-assisted hierarchical organization of scientific studies for literature review support

    Chao-Chun Hsu, Erin Bransom, Jenna Sparks, Bailey Kuehl, Chenhao Tan, David Wadden, Lucy Lu Wang, and Aakanksha Naik. Chime: Llm-assisted hierarchical organization of scientific studies for literature review support. In ACL, pages 118–132, 2024

    work page 2024

  13. [13]

    Autosurvey: Large language models can automatically write surveys

    Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Qingsong Wen, Wei Ye, et al. Autosurvey: Large language models can automatically write surveys. NeurIPS, 37:115119–115145, 2024

    work page 2024

  14. [14]

    Surveyx: Academic survey automation via large language models

    Xun Liang, Jiawei Yang, Yezhaohui Wang, Chen Tang, Zifan Zheng, Simin Niu, Shichao Song, Hanyu Wang, Bo Tang, Feiyu Xiong, et al. Surveyx: Academic survey automation via large language models. arXiv preprint arXiv:2502.14776, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback

    Jiakang Yuan, Xiangchao Yan, Botian Shi, Tao Chen, Wanli Ouyang, Bo Zhang, Lei Bai, Yu Qiao, and Bowen Zhou. Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback. arXiv preprint arXiv:2501.03916, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Re- searchagent: Iterative research idea generation over scientific literature with large language models

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Re- searchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Agentic reasoning: Reasoning llms with tools for the deep research

    Junde Wu, Jiayuan Zhu, and Yuyuan Liu. Agentic reasoning: Reasoning llms with tools for the deep research. arXiv preprint arXiv:2502.04644, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Sketchagent: Generating structured diagrams from hand-drawn sketches

    Cheng Tan, Qi Chen, Jingxuan Wei, Gaowei Wu, Zhangyang Gao, Siyuan Li, Bihui Yu, Ruifeng Guo, and Stan Z Li. Sketchagent: Generating structured diagrams from hand-drawn sketches. arXiv preprint arXiv:2508.01237, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    From words to structured visuals: A benchmark and framework for text-to-diagram generation and editing

    Jingxuan Wei, Cheng Tan, Qi Chen, Gaowei Wu, Siyuan Li, Zhangyang Gao, Linzhuang Sun, Bihui Yu, and Ruifeng Guo. From words to structured visuals: A benchmark and framework for text-to-diagram generation and editing. In CVPR, pages 13315–13325, 2025

    work page 2025

  20. [20]

    Deepresearcher: Scaling deep research via reinforcement learning in real-world environments

    Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments. arXiv preprint arXiv:2504.03160, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Textrank: Bringing order into text

    Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In EMNLP, pages 404–411, 2004

    work page 2004

  22. [22]

    Lexrank: Graph-based lexical centrality as salience in text summarization

    Günes Erkan and Dragomir R Radev. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research , 22:457– 479, 2004

    work page 2004

  23. [23]

    An empirical survey on long document summarization: Datasets, models, and metrics

    Huan Yee Koh, Jiaxin Ju, Ming Liu, and Shirui Pan. An empirical survey on long document summarization: Datasets, models, and metrics. ACM computing surveys, 55(8):1–35, 2022

    work page 2022

  24. [24]

    Extractive summarization using extended textrank algorithm

    Ansh N Vora, Rinit Mayur Jain, Aastha Sanjeev Shah, and Sheetal Sonawane. Extractive summarization using extended textrank algorithm. In Proceedings of the 21st International Conference on Natural Language Processing (ICON) , pages 462–471, 2024

    work page 2024

  25. [25]

    Dependency- based discourse parser for single-document summarization

    Yasuhisa Yoshida, Jun Suzuki, Tsutomu Hirao, and Masaaki Nagata. Dependency- based discourse parser for single-document summarization. In EMNLP, pages 1834–1839, 2014

    work page 2014

  26. [26]

    Learning-based single- document summarization with compression and anaphoricity constraints

    Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. Learning-based single- document summarization with compression and anaphoricity constraints. In ACL, pages 1998–2008, 2016

    work page 1998

  27. [27]

    Single-document abstractive text summarization: A systematic literature review

    Abishek Rao, Shivani Aithal, and Sanjay Singh. Single-document abstractive text summarization: A systematic literature review. ACM Computing Surveys, 57(3):1–37, 2024

    work page 2024

  28. [28]

    Booookscore: A systematic exploration of book-length summarization in the era of llms

    Yapei Chang, Kyle Lo, Tanya Goyal, and Mohit Iyyer. Booookscore: A systematic exploration of book-length summarization in the era of llms. arXiv preprint arXiv:2310.00785, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Long document summarization with top-down and bottom-up inference

    Bo Pang, Erik Nijkamp, Wojciech Kryściński, Silvio Savarese, Yingbo Zhou, and Caiming Xiong. Long document summarization with top-down and bottom-up inference. In EACL, pages 1267–1284, 2023

    work page 2023

  30. [30]

    Hi-Map: Hierarchical Factorized Radiance Field for High-Fidelity Monocular Dense Mapping

    Tongyan Hua, Haotian Bai, Zidong Cao, Ming Liu, Dacheng Tao, and Lin Wang. Hi-map: Hierarchical factorized radiance field for high-fidelity monocular dense mapping. arXiv preprint arXiv:2401.03203, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    SKT5SciSumm -- Revisiting Extractive-Generative Approach for Multi-Document Scientific Summarization

    Huy Quoc To, Ming Liu, Guangyan Huang, Hung-Nghiep Tran, Andr’e Greiner- Petter, Felix Beierle, and Akiko Aizawa. Skt5scisumm–revisiting extractive- generative approach for multi-document scientific summarization. arXiv preprint arXiv:2402.17311, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  32. [32]

    Improving multi-document summarization through referenced flexible extraction with credit-awareness

    Yun-Zhu Song, Yi-Syuan Chen, and Hong-Han Shuai. Improving multi-document summarization through referenced flexible extraction with credit-awareness. In ACL, pages 1667–1681, 2022

    work page 2022

  33. [33]

    Super- vising the centroid baseline for extractive multi-document summarization

    Simão Gonçalves, Gonçalo Correia, Diogo Pernes, and Afonso Mendes. Super- vising the centroid baseline for extractive multi-document summarization. In EMNLP, page 87, 2023

    work page 2023

  34. [34]

    Enhancing multi-document summarization with cross-document graph- based information extraction

    Zixuan Zhang, Heba Elfardy, Markus Dreyer, Kevin Small, Heng Ji, and Mohit Bansal. Enhancing multi-document summarization with cross-document graph- based information extraction. In EACL, pages 1696–1707, 2023

    work page 2023

  35. [35]

    Disentangling specificity for abstractive multi-document summarization

    Congbo Ma, Wei Emma Zhang, Hu Wang, Haojie Zhuang, and Mingyu Guo. Disentangling specificity for abstractive multi-document summarization. In IJCNN, pages 1–8. IEEE, 2024

    work page 2024

  36. [36]

    Embrace divergence for richer insights: A multi-document summarization benchmark and a case study on summarizing diverse information from news articles

    Kung-Hsiang Huang, Philippe Laban, Alexander Richard Fabbri, Prafulla Kumar Choubey, Shafiq Joty, Caiming Xiong, and Chien-Sheng Wu. Embrace divergence for richer insights: A multi-document summarization benchmark and a case study on summarizing diverse information from news articles. In NAACL, pages 570–593, 2024

    work page 2024

  37. [37]

    From single to multi: How llms hallucinate in multi- document summarization

    Catarina G Belem, Pouya Pezeskhpour, Hayate Iso, Seiji Maekawa, Nikita Bhutani, and Estevam Hruschka. From single to multi: How llms hallucinate in multi- document summarization. arXiv preprint arXiv:2410.13961, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Assisting in writing wikipedia-like articles from scratch with large language models

    Yijia Shao, Yucheng Jiang, Theodore Kanell, Peter Xu, Omar Khattab, and Monica Lam. Assisting in writing wikipedia-like articles from scratch with large language models. In NAACL, pages 6252–6278, 2024

    work page 2024

  39. [39]

    Chatcite: Llm agent with human workflow guidance for comparative literature summary

    Yutong Li, Lu Chen, Aiwei Liu, Kai Yu, and Lijie Wen. Chatcite: Llm agent with human workflow guidance for comparative literature summary. In COLING, pages 3613–3630, 2025

    work page 2025

  40. [40]

    Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    R1-searcher: Incentivizing the search capability in llms via reinforcement learning

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Gpt-4o: Advancing multimodal ai with real-time audio, vision, and text integration

    OpenAI. Gpt-4o: Advancing multimodal ai with real-time audio, vision, and text integration. https://openai.com/index/hello-gpt-4o/, 2024. Technical Report

    work page 2024

  45. [45]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024

    Pith/arXiv arXiv 2024

  46. [47]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. MM ’25, October 27–31, 2025, Dublin, Ireland Qi Chen et al

    Pith/arXiv arXiv 2023

  47. [48]

    Internlm2 technical report

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprint arXiv:2403.17297, 2024

    Pith/arXiv arXiv 2024

  48. [49]

    Llama 3: Open foundation for generative ai

    Meta AI. Llama 3: Open foundation for generative ai. https://github.com/meta- llama/llama3, 2024. 8B parameter instruct-tuned model

    work page 2024

  49. [50]

    Codellama: Open foundation models for code

    Meta AI. Codellama: Open foundation models for code. https://ai.meta. com/research/publications/code-llama-open-foundation-models-for-code/, 2023. Code-specific adaptation of Llama 2

    work page 2023

  50. [51]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. https://www. anthropic.com/news/claude-3-family, 2024. Technical Report

    work page 2024

  51. [52]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023