Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review

Iryna Gurevych; Qian Ruan

REVIEW 3 major objections 5 minor 2 cited by

Rebuttal writing should be modeled as author-in-the-loop generation: authors supply their planned revisions and response strategy, and systems are evaluated on whether the response reflects that input and answers the reviewer.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 09:37 UTC pith:S2BDKR6Q

load-bearing objection Solid resource paper with a real dataset and evaluation suite, but the headline claim that author input helps is undercut by a temporal leak: the model is fed the exact post-rebuttal revised sentences, not the author's plans. the 3 major comments →

arxiv 2602.11173 v3 pith:S2BDKR6Q submitted 2026-01-19 cs.CL

Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review

Qian Ruan , Iryna Gurevych This is my paper

classification cs.CL

keywords author response generationpeer reviewrebuttalcontrollable text generationLLM evaluationscientific revisionauthor-in-the-loopdataset

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Author response generation has so far been modeled as a generic text-generation problem that takes only reviewer comments as input. This paper claims that rebuttals are fundamentally author-in-the-loop: authors hold expertise, author-only information, and response strategies, and an assistive system should take those signals as explicit, controllable inputs. To make that claim testable, the paper builds three things: a dataset of review–response–revision triplets where post-hoc paper revisions stand in for author signals; a generation framework that accepts author inputs, response plans, and length limits, and refines drafts using automated evaluation feedback; and a 20-plus-metric evaluation suite measuring discourse, controllability, input utilization, and response quality. Experiments across five large language models and nine settings report that author input improves response quality, richer input improves factual grounding but dilutes coverage of the author's core edits, controls trade off against quality, and evaluation-guided refinement improves quality. A sympathetic reader would care because this reframes what an AI rebuttal assistant should be: not a reviewer-comment paraphraser, but a controller for the author's own expertise and intent.

Core claim

The central discovery is a formulation and an empirical result. Rebuttal generation, the paper argues, should be organized around explicit author input: edits the author plans to make, paragraph context, and strategic choices about response actions and length. The paper constructs Re3Align, the first large-scale dataset of aligned review–response–revision triplets—15,521 triplets from 3,394 papers with 439,798 sentence-level edit annotations—where sentence-level edits between original and revised papers proxy author expertise and intent. Using this, it tests REspGen, a modular generation framework, and REspEval, an evaluation suite spanning input utilization (factual grounding, input coverag

What carries the argument

Re3Align is the central object: a dataset of aligned review–response–revision triplets in which sentence-level edits extracted between paper versions serve as proxies for author expertise and intent, linked to specific reviewer concerns through two-way classifiers and similarity matching. REspGen is the generation framework: configurable author input (edit strings with or without paragraph context, plus optional retrieval of paper paragraphs), itemized response planning with 16 response-action labels, length constraints, and iterative refinement driven by REspEval. REspEval operationalizes the dependent variables—generation factuality precision, input coverage recall, length and plan control

Load-bearing premise

The argument collapses if the extracted revisions do not actually stand in for what the author knew and planned while writing the rebuttal—the paper's own human validation of edit-to-review linking reaches only 0.71 precision on one of its two data sources, so some 'author input' may be text that is lexically similar but not the author's intended response signal.

What would settle it

Collect review–response pairs from a venue where authors record their intended revisions at rebuttal time, before revisions exist, and compare REspGen run with those prospective author inputs against REspGen run with post-hoc extracted edits. If the quality advantage of author input disappears under prospective inputs, the proxy assumption fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

If rebuttal writing is author-in-the-loop, then review-only generation is an incomplete task formulation; systems that omit author input will miss information only authors have.
Adding author input—planned revision text—improves targeting, specificity, and convincingness of generated responses across five large language models, often above human baselines.
Richer input, such as paragraph context and retrieved paper sections, improves factual grounding but lowers coverage of the author's core edits, a dilution effect that response systems must manage.
Imposing length limits lowers response quality; response-plan control alone preserves quality better; combined multi-attribute control remains difficult, with length as the primary bottleneck.
Evaluation-guided refinement using REspEval feedback produces significant quality gains, especially for initially weak responses, while largely maintaining controllability.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The paper's proxy assumption is the point to stress-test: if post-hoc revisions do not faithfully capture what authors knew at rebuttal time, the reported quality gains may reflect extra relevant text rather than true author intent; a live rebuttal study collecting author input during the actual review process would settle this.
The controllability–quality trade-off suggests a practical design rule: author-in-the-loop rebuttal assistants should let authors choose how much context to reveal, since richer context can actively hurt focus; an interface that lets authors select individual edits rather than whole paragraphs could preserve grounding without dilution.
Because REspEval decomposes quality into targeting, specificity, and convincingness plus input-utilization metrics, it enables a new kind of diagnostic for other collaborative writing tasks beyond peer-review rebuttals, such as grant revisions or paper revisions.
The refinement loop is an instance of generator–evaluator iteration; a natural extension is to make refinement online and interactive, with the author choosing among refinements rather than accepting a single revised draft.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid resource paper with a real dataset and evaluation suite, but the headline claim that author input helps is undercut by a temporal leak: the model is fed the exact post-rebuttal revised sentences, not the author's plans.

read the letter

First thing to know: this is a resource paper, and judged as one it's solid. The Re3Align dataset—15k aligned review-response-revision triplets—is a real step up from prior ARG datasets, and the 20+ metric evaluation suite, with human validation on a subset, is genuinely useful. The authors also release their generation and evaluation tools. If you work on peer-review NLP, this is worth having on your shelf.

But the headline empirical claim—that author input improves response quality—doesn't hold up as stated. The stress-test concern is valid: the "author input" in the experiments is the exact final revised sentence from the accepted paper, supplied to the LLM as an edit string. At rebuttal time, the human author did not have that sentence; they had a plan, maybe a rough note, not polished wording. So the model is effectively receiving the answer key. It can quote or paraphrase the sentence that was written weeks after the response. The comparison against human responses is therefore stacked. This isn't the same as the "revisions proxy intent" approximation, which is at least a defensible coarse proxy; this is "revisions as oracle text." The RQ2 conclusion in §6.2.1—that human baselines are surpassed—does not establish that author-in-the-loop works in practice.

Also worth noting: the 48 EMNLP cases are deliberately selected for explicit score increases, so the human baseline is strong; and the refinement loop is scored by the same GPT-5-based judge that gives the feedback, so part of the reported gain is fitting the judge. The PeerJ triplet alignment precision is 0.71, which is decent for a large-scale resource but means a chunk of the "aligned edits" may be noise.

None of this kills the paper as a framework and dataset contribution. The resource is new, large, and documented. But the empirical claims should be reframed as illustrative of the framework, not as evidence that the author-in-the-loop formulation works as intended. To make the case, the authors would need to collect actual author plans (or at least blur the final text into rough notes), or run a study with a true held-out before/after design.

My recommendation: send to peer review. It deserves a serious referee to push on the design, but it should be conditionally accepted as a resource paper, with the understanding that the main empirical claim needs to be qualified.

Referee Report

3 major / 5 minor

Summary. The paper reformulates author response generation (ARG) as an author-in-the-loop task, where the author's expertise and intent are provided as explicit inputs alongside review comments. It contributes (i) Re3Align, a dataset of review–response–revision triplets with 15,521 aligned instances; (ii) REspGen, a modular generation framework supporting author input, response-plan and length control, and evaluation-guided refinement; and (iii) REspEval, a multi-dimensional evaluation suite with 20+ metrics. Experiments with five LLMs across nine settings report that author input improves quality, richer input improves factual grounding but dilutes focus, controls create quality trade-offs, and refinement yields statistically significant gains. The paper also releases dataset and tools.

Significance. The released dataset and evaluation suite are potentially valuable resources for a neglected task, and the paper is ambitious in formalizing author input, controllability, and evaluation for ARG. The human-validation studies for REspEval (agreement ratings >4.17/5, Krippendorff's α=0.81–0.89) and the fine-grained alignment effort are creditable; the resource release is a concrete contribution. However, the central empirical claim—that author input, as simulated, improves response quality—rests on a proxy that may not capture author intent at response time. The temporal leak in the EMNLP24 experiments and the moderate alignment precision on PeerJ are load-bearing issues that need to be addressed before the author-in-the-loop conclusion can be accepted.

major comments (3)

[§4.2, §6.1, §6.2.1] Temporal leak in the author-input simulation. §4.2 states that author input is simulated via aligned sentence-level edits [ealign], supplied as an edited-sentence string or with paragraph context. For the EMNLP24 experiments (§6.1), these edits are extracted from the post-rebuttal revised paper, while the generation prompt explicitly says the response is written 'during the rebuttal phase, before submitting any revisions' (Table 11). The model is therefore given the exact final wording that was written after the human response, whereas the human baseline was written before that wording existed. The RQ2 conclusion in §6.2.1 ('all models surpass them in most author-input settings') is thus a comparison between humans without the answer key and models with the answer key. This does not establish that author intent available at response time improves quality. A concrete fix is to re-run the
[§4.3, §6.2.4, Table 14] Evaluation-guided refinement is scored by the same judge that produces the feedback. §4.3 feeds REspEval's justifications and suggestions back into the generator, and §6.2.4 reports 'statistically significant gains' in Table 14 measured with the same GPT-5-based REspEval scores (§5.4). This creates a potential circularity: the model may be fitting the judge rather than producing genuinely better responses. The human validation in §C.2 was performed on static responses and pairwise comparisons, not on original-vs-refined pairs, so it does not establish that refinement-induced improvements correspond to human-perceived quality. I recommend adding a human pairwise evaluation on a random subset of Setting 6/7 vs. Setting 8/9 outputs, or at least reporting human agreement on win/loss for refinement pairs.
[§3.3, §1] Proxy validity and alignment precision. The paper's load-bearing premise, stated in §1, is that revisions proxy author expertise and intent at response time. But even perfectly aligned post-hoc edits are not necessarily what the author planned during rebuttal; for EMNLP24 the revised text is temporally future (Major 1). In addition, §3.3 reports human-validated alignment precision of only 0.71 for PeerJ and 0.86 for EMNLP24, with errors caused by aggregation of lexically or semantically similar but irrelevant edits. With 29% false positives in PeerJ, the 'author input' condition contains substantial unrelated text, which may explain quality gains as extra context rather than author intent. I recommend reporting results on a high-precision subset (e.g., edits agreed by both CE and AE components), and ablating by the number of supplied edits to separate 'more text' from 'author intent'.

minor comments (5)

[Limitations] The Limitations section does not mention the temporal-leak or proxy-validity concerns raised above. A dedicated paragraph acknowledging these threats and the steps taken to mitigate them is needed.
[§4.1] Length-control simulation uses n+50 where n is the human response length. This is an oracle bound that leaks information about the human response; please discuss this choice and its effect on the controllability-quality trade-off (RQ6).
[Table 3 / Table 12] Several cells contain '/' without explanation (e.g., lenC/planC in settings where the control is not applied). Please add a table note clarifying that these are not-applicable settings. The green highlighting may also be difficult to distinguish in grayscale.
[§5.2] The order-fidelity definition OF(m)=LCS(s,s*)/|s| is underspecified regarding ties and unmatched actions at the beginning/end. A brief example would improve clarity.
[Table 14] The 'eD' column is not defined in the caption or text; please define the edit-distance measure used.

Circularity Check

2 steps flagged

Author-input experiments supply post-hoc final revision text as 'author input,' and refinement gains are scored by the same GPT-5 judge that issued the feedback; both load-bearing claims are partly self-confirming.

specific steps

other [§4.2 (Input Component Configuration) with §6.2.1 (Why Author Input Matters)]
"We simulate author input through aligned sentence-level edits [ealign], where each edit is supplied as either (i) an edited-sentence string (S), simulating rough revision ideas not yet anchored to a specific location, or (ii) the string with its paragraph context and section title... Together, these results demonstrate that author input is both necessary and effective for ARG, motivating our author-in-the-loop framework."

The 'author input' is not a plan available during rebuttal writing; it is the exact sentence from the revised, accepted paper that was written after the human response. The RQ2 result that author input improves quality is therefore partly an artifact of giving the generator the answer key: the model can quote or paraphrase the final revision, while the human baseline was produced before that text existed. The central author-in-the-loop claim is thus supported by a temporal leak rather than by evidence that author expertise/intent as available at response time is beneficial.
fitted input called prediction [§4.3 (Evaluation-guided Refinement) + §5.4 (Response Quality Evaluation) + §6.2.4 (Refinement Effectiveness)]
"REspEval then evaluates this draft and returns evaluation metrics, justifications and refinement suggestions... are fed back into REspGen to produce a refined response. ... To address RQ7 ... Table 14 ... shows refinement produces statistically significant gains across all LLMs, settings, and quality metrics, confirming the effectiveness of evaluation-guided refinement."

The refinement loop is closed: the GPT-5-based REspEval generates the justifications and suggestions, the generator is instructed to follow them, and the same GPT-5-based REspEval then scores the refined response. The reported improvements are therefore partly the result of the response conforming to the judge's own stated preferences, not an independent measure of quality. Human validation of REspEval on static responses mitigates but does not remove the circularity, since the validation does not cover the judge-guided refinement loop itself.

full rationale

The paper makes several genuinely independent contributions: Re3Align is a large curated resource with human-validated triplet precision, and REspEval is validated against experienced human annotators (agreement >4.17/5; Krippendorff's alpha 0.81-0.89). These elements give the dataset and evaluation tools real value and prevent a fully circular verdict. However, two load-bearing empirical claims are partially self-confirming. First, the central 'author input improves ARG' result operationalizes author input as the final revised sentences from the accepted paper, which are not available at rebuttal time; supplying them is closer to providing oracle text than to capturing author intent. Second, the 'evaluation-guided refinement improves responses' result uses REspEval both as the source of the feedback and as the metric that measures the improvement, so the loop can inflate gains by fitting the judge. Because the paper's headline conclusions depend on these two demonstrations, the derivation chain is only partially independent of its own inputs. Self-citations to Ruan et al. (2024a,b) for revision analysis and alignment classifiers are real, externally evaluated tools (reported F1/accuracy), so they are not themselves the circular step.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical or theoretical entities are introduced; the contributions are datasets, frameworks, and metrics. The empirical conclusions depend on the domain assumptions above and on pipeline thresholds tuned in small pilot studies.

free parameters (4)

t0 and t1 quotation-match thresholds = 85, 85
Used in §A.2 review-response matching; selected from a 20-example pilot study.
CEsim/AEsim similarity thresholds = fuzzy ≥60, SBERT ≥20, bigram ≥10
Used in §A.3 edit alignment; optimized in a pilot to ensure precision.
length-control simulated bound = n + 50 words
Set to human response length plus 50 (§4.1); a simulation of author choice, not a measured author preference.
v1 retrieval top-k = 5
Number of retrieved paragraphs used as paper context (§B.2); chosen by default.

axioms (5)

domain assumption Post-hoc revisions proxy author signals at response time.
Central premise in §1 and §3; if false, the simulated author input is not author intent.
domain assumption The source corpora provide complete Re3 records (original, revised, reviews, responses).
Required to build triplets in §3.1; incomplete records would bias alignment.
domain assumption GPT-5-based quality scoring is a valid proxy for human judgments of targeting/specificity/convincingness.
REspEval uses GPT-5 as judge (§5.4); human validation supports but does not guarantee transfer.
domain assumption DISAPERE response-action taxonomy applies to the merged EMNLP/PeerJ data.
Response planning and discourse metrics rely on this taxonomy (§4.1, §5.1); it was derived from a different corpus.
domain assumption Atomic-fact decomposition via GPT-5 correctly captures factual support and input coverage.
GFP and ICR metrics in §5.3 rest on this decomposition; errors would bias input-utilization findings.

pith-pipeline@v1.3.0-alltime-deepseek · 35213 in / 12747 out tokens · 124237 ms · 2026-08-03T09:37:47.820311+00:00 · methodology

0 comments

read the original abstract

Author response (rebuttal) writing is a critical stage of scientific peer review that demands substantial author effort. In practice, authors possess domain expertise, author-only information, and response strategies - concrete forms of author expertise and intent - and seek NLP assistance that integrates these signals into author response generation (ARG). Yet this author-in-the-loop paradigm lacks formal NLP formulation and systematic study: no dataset provides fine-grained author signals, existing ARG work lacks author inputs and controls, and no evaluation measures response reflection of author signals and effectiveness in addressing reviewer concerns. To fill these gaps, we introduce (i) Re3Align, the first large-scale dataset of aligned review-response-revision triplets, where revisions proxy author signals; (ii) REspGen, an author-in-the-loop ARG framework supporting flexible author input, multi-attribute control, and evaluation-guided refinement; and (iii) REspEval, a comprehensive evaluation suite with 20+ metrics spanning input utilization, controllability, response quality, and discourse. Experiments with SOTA LLMs demonstrate the benefits of author input and evaluation-guided refinement, the impact of input specificity on response quality, and controllability-quality trade-offs. We release our dataset, generation and evaluation tools.

Figures

Figures reproduced from arXiv: 2602.11173 by Iryna Gurevych, Qian Ruan.

**Figure 1.** Figure 1: In this work, we contribute (1) REspGen, an author-in-the-loop ARG framework that integrates explicit author input (d), controllable planning and length (b–c), and additional paper context (e); (2) Re3Align, the first large-scale review–response–revision triplets dataset for modeling author signals; and (3) REspEval, a comprehensive response evaluation framework with over 20 metrics spanning four dimensio… view at source ↗

**Figure 2.** Figure 2: Frameworks: REspGen & REspEval. ple is shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: Changes in Specificity after refinement across five LLMs. Colors indicate increase (green), no change (yellow), or decrease (red); the first bar shows overall proportions, followed by distributions by initial score. control. For RQ6: How do different controls affect response quality?, we compare Settings 5–7 against the unconstrained Setting 4. Length control alone (Setting 5) substantially degrades respo… view at source ↗

**Figure 4.** Figure 4: An illustrative example of segment-level review-response pair matching. Given a reviewer–author [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

**Figure 5.** Figure 5: An illustrative example of an aligned sentence-level edit. Given original paper [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗

**Figure 6.** Figure 6: Optimized prompt to itemize review segments and label response actions. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗

**Figure 7.** Figure 7: An illustrative example of annotated review items, response action labels, and the stance classes. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗

**Figure 8.** Figure 8: Optimized prompt to evaluate response quality in terms of targeting (directness), specificity, and [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗

**Figure 9.** Figure 9: An illustrative output example of response quality evaluation, including scores, justifications and [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗

**Figure 10.** Figure 10: Robustness verification. Mean scores and corresponding standard deviation error bars are presented for [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗

**Figure 11.** Figure 11: Annotation interface used in the human studies. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗

**Figure 12.** Figure 12: Changes in Convincingness after refinement across five LLMs. Colors indicate increase (green), no change (yellow), or decrease (red); the first bar shows overall proportions, followed by distributions by initial score [PITH_FULL_IMAGE:figures/full_fig_p025_12.png] view at source ↗

**Figure 13.** Figure 13: Tone–stance profiles of author responses generated by five LLMs (a–e) under nine settings (1–9), together [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗

**Figure 14.** Figure 14: Distributions of stance types across relative positions in author responses generated by five LLMs (a–e) [PITH_FULL_IMAGE:figures/full_fig_p027_14.png] view at source ↗

**Figure 15.** Figure 15: Case Study 1: Richer Inputs Degrade Response Quality Across Targeting, Specificity, and Convincingness. [PITH_FULL_IMAGE:figures/full_fig_p028_15.png] view at source ↗

**Figure 16.** Figure 16: Case Study 2: Richer Inputs Reduce Specificity and ICR. In Author Input, green indicates edit sentences [PITH_FULL_IMAGE:figures/full_fig_p029_16.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

AI for Auto-Research: Roadmap & User Guide
cs.AI 2026-05 unverdicted novelty 4.0

The paper delivers a stage-by-stage roadmap for AI in research, showing reliable assistance in retrieval and tool tasks but fragility in novelty and judgment, advocating human-governed collaboration.
AI for Auto-Research: Roadmap & User Guide
cs.AI 2026-05 conditional novelty 4.0

AI can generate research artifacts faster than it can verify them, so across all eight lifecycle stages the credible deployment mode is human-governed collaboration rather than full autonomy.

Reference graph

Works this paper leans on

8 extracted references · 5 linked inside Pith · cited by 1 Pith paper

[1]

Script is a structure that describes an appropriate sequence of events in a particular context (Schank and Abelson, 1975;Abelson, 2014)

Introduction ... Script is a structure that describes an appropriate sequence of events in a particular context (Schank and Abelson, 1975;Abelson, 2014). In daily routines, individuals often rely on meticulously outlined steps to realize their objectives. For instance, Figure 1 illustrates the process of opening a can with a spoon. Recentstudies have appl...

1975
[2]

Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych

Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning.Preprint, arXiv:2501.12948. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2023. NLPeer: A unified resource for the computational study of peer review. InProceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

Pith/arXiv arXiv 2023
[4]

Peer review as a multi-turn and long-context dialogue with role-based interactions.Preprint, arXiv:2406.05688. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esi...

Pith/arXiv arXiv 2023
[5]

\n”+x t+1 i ). For author response–edit alignment (AE), we anal- ogously consider all pairs (s1 =a k nq, s2 =x t j + “\n

Re 2: A consistency-ensured dataset for full- stage peer review and multi-turn rebuttal discussions. Preprint, arXiv:2505.07920. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models.ACM Computing Surveys, 56(3). A Dataset Construction A.1 Prepro...

arXiv 2023
[7]

questions

Introduction ... Script is a structure that describes an appropriate sequence of events in a particular context (Schank and Abelson, 1975;Abelson, 2014). In daily routines, individuals often rely on meticulously outlined steps to realize their objectives. For instance, Figure 1 illustrates the process of opening a can with a spoon. Recentstudies have appl...

Pith/arXiv arXiv 1975
[8]

- Critical details are missing: execu- tion environment is missing, no document descrip- tion, no sections/tables/figures or configurations [question_1]

FOLIO also contains examples with binary and ternary relations. We provide some example sentences with their corresponding FOL formulas here. The Turing Award has been awarded to Donald Knuth, Marvin Minsky, Richard Hamming, and John McCarthy. AwardedTo(turingAward, donaldKnuth) AwardedTo(turingAward, marvinMinsky) AwardedTo(turingAward, richardHamming) A...

1977
[2024]

Haoyuan Li, Somnath Basu Roy Chowdhury, and Snigdha Chaturvedi

Llms-as-judges: A comprehensive sur- vey on llm-based evaluation methods.Preprint, arXiv:2412.05579. Haoyuan Li, Somnath Basu Roy Chowdhury, and Snigdha Chaturvedi. 2023. Aspect-aware unsuper- vised extractive opinion summarization. InFindings of the Association for Computational Linguistics: ACL 2023, pages 12662–12678, Toronto, Canada. Association for C...

Pith/arXiv arXiv 2023
[2025]

Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

Phi-4-reasoning technical report.Preprint, arXiv:2504.21318. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3- embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation. InFindings of the Asso- ciation for Computational Linguistics: ACL 2024, pages 2318–233...

Pith/arXiv arXiv 2024

[1] [1]

Script is a structure that describes an appropriate sequence of events in a particular context (Schank and Abelson, 1975;Abelson, 2014)

Introduction ... Script is a structure that describes an appropriate sequence of events in a particular context (Schank and Abelson, 1975;Abelson, 2014). In daily routines, individuals often rely on meticulously outlined steps to realize their objectives. For instance, Figure 1 illustrates the process of opening a can with a spoon. Recentstudies have appl...

1975

[2] [2]

Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych

Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning.Preprint, arXiv:2501.12948. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2023. NLPeer: A unified resource for the computational study of peer review. InProceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

Pith/arXiv arXiv 2023

[3] [4]

Peer review as a multi-turn and long-context dialogue with role-based interactions.Preprint, arXiv:2406.05688. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esi...

Pith/arXiv arXiv 2023

[4] [5]

\n”+x t+1 i ). For author response–edit alignment (AE), we anal- ogously consider all pairs (s1 =a k nq, s2 =x t j + “\n

Re 2: A consistency-ensured dataset for full- stage peer review and multi-turn rebuttal discussions. Preprint, arXiv:2505.07920. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models.ACM Computing Surveys, 56(3). A Dataset Construction A.1 Prepro...

arXiv 2023

[5] [7]

questions

Introduction ... Script is a structure that describes an appropriate sequence of events in a particular context (Schank and Abelson, 1975;Abelson, 2014). In daily routines, individuals often rely on meticulously outlined steps to realize their objectives. For instance, Figure 1 illustrates the process of opening a can with a spoon. Recentstudies have appl...

Pith/arXiv arXiv 1975

[6] [8]

- Critical details are missing: execu- tion environment is missing, no document descrip- tion, no sections/tables/figures or configurations [question_1]

FOLIO also contains examples with binary and ternary relations. We provide some example sentences with their corresponding FOL formulas here. The Turing Award has been awarded to Donald Knuth, Marvin Minsky, Richard Hamming, and John McCarthy. AwardedTo(turingAward, donaldKnuth) AwardedTo(turingAward, marvinMinsky) AwardedTo(turingAward, richardHamming) A...

1977

[7] [2024]

Haoyuan Li, Somnath Basu Roy Chowdhury, and Snigdha Chaturvedi

Llms-as-judges: A comprehensive sur- vey on llm-based evaluation methods.Preprint, arXiv:2412.05579. Haoyuan Li, Somnath Basu Roy Chowdhury, and Snigdha Chaturvedi. 2023. Aspect-aware unsuper- vised extractive opinion summarization. InFindings of the Association for Computational Linguistics: ACL 2023, pages 12662–12678, Toronto, Canada. Association for C...

Pith/arXiv arXiv 2023

[8] [2025]

Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

Phi-4-reasoning technical report.Preprint, arXiv:2504.21318. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3- embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation. InFindings of the Asso- ciation for Computational Linguistics: ACL 2024, pages 2318–233...

Pith/arXiv arXiv 2024