AUEB-Archimedes at RIRAG-2025: Is obligation concatenation really all you need?

Ioannis Chasandras; Ion Androutsopoulos; Odysseas S. Chlapanis

REVIEW 3 major objections 4 minor 17 references

AUEB-Archimedes at RIRAG-2025: Is obligation concatenation really all you need?

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By feeding RePASs its own extracted obligation sentences back as answers, the NOC system scores 0.947, above human experts, despite copying retrieved text.

desk verdict This shared task paper earns its place with a simple, reproducible demonstration that RePASs can be gamed by concatenating its own extracted obligations; the VRR system is a solid engineering contribution but its score is partly fitted to the metric it evaluates with. read the letter →

arxiv 2412.11567 v1 pith:6NDL5OEV submitted 2024-12-16 cs.CL

Ioannis Chasandras , Odysseas S. Chlapanis , Ion Androutsopoulos This is my paper · ORCID

classification cs.CL

keywords RePASsretrieval-augmentedgenerationanswerregulatoryNLPobligationextractionadversarialevaluationrankfusionreference-free

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish two things about RIRAG-2025, a shared task on answering regulatory questions from retrieved passages. First, a system called NOC can achieve a near-perfect RePASs score (0.947), higher than human experts, simply by concatenating obligation sentences extracted from the retrieved passages and outputting them verbatim as the answer. Second, a more principled system called VRR, which generates multiple candidate answers, selects the best one with RePASs, and then iteratively removes contradictions and inserts missing obligations, produces coherent answers with a more plausible score (0.639). The authors use these results to argue that reference-free model-based metrics like RePASs are vulnerable to adversarial exploitation and that super-human scores on such metrics do not necessarily indicate useful answers.

What carries the argument

The central mechanism is the obligation-extraction and entailment pipeline inside RePASs: a fine-tuned LegalBERT model identifies important 'obligation' sentences in the retrieved passages, and an NLI model measures whether each answer sentence entails or contradicts those obligation sentences. NOC exploits this by using the exact same extracted obligations as the answer, so every answer sentence trivially entails the obligations it was extracted from, yielding near-perfect scores. VRR uses the same RePASs pipeline as a verifier: it selects the best among several candidate answers and then iteratively refines the chosen answer by adding missing obligations and deleting sentences that cause contradictions.

What would settle it

Run a human usefulness study where annotators compare NOC's verbatim answers with human-written answers on the same regulatory questions; if NOC answers are judged as useful as human answers, the claim that RePASs is deceived by verbatim copying would collapse. Alternatively, modify RePASs to remove or ignore its obligation-extraction component and re-run NOC: a score close to baseline would confirm the metric, not the answer content, was the source of the 0.947.

Watch

Extended reading notes

Core claim

The central discovery is that RePASs, the task's reference-free evaluation metric, can be tricked by exploiting its own obligation-extraction component. NOC takes the obligation sentences that RePASs extracts from the retrieved passages and returns them verbatim as the generated answer, achieving a RePASs score of 0.947, above the reported human expert score of 0.859. The paper also shows that when answers are rewritten to be readable and directly responsive, as in the LOC system, the score drops below the baseline, demonstrating that RePASs is sensitive to answer style and rewards verbatim copying. The VRR system instead uses RePASs as a verifier: it generates several LLM answers, picks the one with the highest RePASs score, then alternates between removing contradictory sentences and adding uncovered obligations, reaching 0.639 while keeping answers coherent.

Load-bearing premise

The reported gains assume the public test set used to tune retrieval and preprocessing hyperparameters is representative of the hidden test set, since the choices were made by trying a few combinations without cross-validation or confidence intervals.

Editorial extensions

If this is right

Reference-free model-based metrics like RePASs can be gamed by returning extracted sentences verbatim, so leaderboard scores based on such metrics should be treated with caution unless answers are checked for readability and directness.
Super-human scores in RAG evaluation may indicate metric exploitation rather than genuinely better answers, a concern the paper explicitly raises about the top-scoring systems.
Generating readable answers while maximizing a model-based metric is hard: rewriting obligations (LOC) hurts the score, while iterative verification and refinement (VRR) improves it substantially over the baseline.
Combining BM25 with two domain-specific neural retrievers and a reranker improves passage retrieval over the single models and the baseline fusion approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A similar concatenation attack may transfer to other reference-free RAG metrics that reward entailment, lexical overlap, or extracted key sentences; a natural test is to apply NOC-style answers to such metrics and see whether scores inflate.
The hidden-test-set hyperparameters were chosen by trying a few combinations on the public test set, so the relative gains of VRR and the retrieval pipeline may not generalize if the two test sets differ in question or passage distribution.
The authors' suspicion that other super-human systems also trick RePASs could be tested directly by checking whether their top-scoring answers are largely verbatim excerpts from the retrieved passages.
If future RAG evaluations pair reference-free metrics with a readability or directness check, the incentive to use verbatim-extraction exploits like NOC would be reduced.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing worth knowing about this paper is that it gives a working, reproducible demonstration that RePASs—a reference-free RAG metric—can be trivially gamed. The NOC system concatenates obligations extracted by RePASs's own LegalBERT component, scores 0.947, above the human experts' 0.859, and the answers are verbatim excerpts from retrieved passages. That is a real result, and it lands. The paper is honest about the exploit, and the evidence is direct. Code is public.

VRR is a reasonable engineering contribution: generate several candidate answers, pick the one with the best RePASs, then iteratively remove contradictions and insert missing obligations. The ablations show each stage helps. For a shared task system description, the analysis is above average.

The main soft spot is circularity. VRR uses RePASs as the selection signal, as the refinement reward, and then as the evaluation metric. The reported 0.639 is therefore to some degree a fitted value, not an independent assessment. The paper acknowledges this in the Limitations section, noting that VRR needs an accurate verifier, but it never tests whether RePASs is accurate enough on this task. The NOC result cuts both ways: if RePASs can be gamed, then optimizing it does not guarantee semantic quality. The claim that VRR produces 'readable, coherent answers' has no human evaluation behind it. Also, hyperparameters (fusion weights, thresholds, rerank N) were chosen on the public test set without cross-validation; the hidden test results assume those choices generalize.

None of this kills the paper. The NOC demonstration is the payload, and it holds. The VRR part is a system description, not a new evaluation methodology, and its limitations are stated. A reader who cares about metric robustness in RAG will get real value here, especially the clear contrast between the gamed near-perfect score and the 'plausible but still optimized' score of VRR.

I would send this to peer review. A serious referee should ask for an independent check of VRR's coherence claim and for a robustness analysis around the public-test tuning, but the central cautionary result deserves to be in the literature. The paper is a good citizen: it shows a flaw in a metric that shared task organizers and metric designers should take seriously.

Referee Report

3 major / 4 minor

Summary. The paper describes three answer-generation systems submitted to the RIRAG-2025 shared task: NOC, which concatenates obligation sentences extracted by the RePASs metric's own LegalBERT component and achieves a RePASs score of 0.947, surpassing human experts; LOC, which rewrites obligations with an LLM and performs below baseline; and VRR, which selects among LLM-generated candidates using RePASs and then iteratively removes contradictions and inserts missing obligations, achieving 0.639 on the hidden test set. The retrieval backbone combines rank fusion of BM25, voyage-law-2, and voyage-finance-2 with a reranker. The paper's stated goal is to show that RePASs can be adversarially gamed and to propose a more plausible non-adversarial system.

Significance. The NOC result is a valuable, reproducible demonstration that the RePASs metric is gameable by a trivial concatenation of extracted obligations; the released code strengthens this contribution. However, the central positive claim—that VRR produces readable, coherent answers that score well—is not independently substantiated: VRR optimizes the same RePASs metric that is used for evaluation, and the paper's own NOC attack shows that RePASs does not track answer quality. The paper is transparent about this in its Limitations section but does not provide the human evaluation or independent metric needed to support the coherence claim.

major comments (3)

[Sections 4.5.1, 4.5.2, Table 5; Limitations] VRR's reported RePASs score (0.639 in Table 5) is a fitted value rather than an independent assessment of answer quality. The verification step (Section 4.5.1) selects the candidate answer with the highest RePASs, and the refinement step (Section 4.5.2) removes sentences with above-average contradiction scores and inserts obligations to increase obligation coverage, both measured using the same NLI and obligation-extraction components that constitute RePASs. The paper then reports RePASs as the evaluation metric. The Limitations section correctly notes that VRR requires an accurate verifier 'such as RePASs,' but it never validates whether RePASs is accurate enough on this task. Given the NOC attack in Section 4.3 shows that RePASs can be trivially deceived by verbatim obligation concatenation, the claim that VRR generates 'readable, coherent answers' (Abstract, Section 5) is unsupported without a human evaluation or an independent, non-gameable metric.
[Sections 3.3 and 4.2] Several hyperparameters are selected by trying a few combinations on the public test set: the rank-fusion weights a=0.25 and b=0.2 (Table 2), the filtering threshold 0.90 and max drop 0.10 (Table 3), and the reranking top-N=50 (Figure 1). The hidden test set results in Table 5 are the only reported evaluation, and they assume these public-test choices generalize. The paper provides no cross-validation on the development set or confidence intervals to assess the risk of overfitting to the public test set. If the public test set is not representative of the hidden test set, the relative gains of the proposed systems may not transfer.
[Tables 5, 6, and 7] The reported VRR RePASs values are inconsistent across tables. Table 5 lists the leaderboard score as 0.639 and the authors' own recomputation as 0.646 (in brackets), Table 6 reports a final VRR RePASs of 0.646, and Table 7 reports 0.647. The paper attributes differences to 'using different GPUs,' but it is unclear whether Tables 6 and 7 are computed on the public test set or the hidden test set. Please clarify the evaluation split for each table and reconcile the final VRR score.

minor comments (4)

[Section 4.4] The retry limit K=3 for LOC is introduced without a sensitivity analysis or justification; a brief ablation or rationale would help the reader understand its impact.
[Section 3.1] The phrase 'three of the best 3 text embedding models' is grammatically awkward; consider rewriting as 'three of the best text embedding models.'
[Abstract and Section 1] The abstract describes the NOC score as 'dubiously high' while Section 1 calls it a 'perfect score'; these characterizations are consistent in spirit but could be unified to avoid confusion.
[Appendix B] The prompts are said to have been improved using GPT-4o and selected according to the authors' opinion on 2-3 sample questions; this selection procedure is informal and should be mentioned in the main text or accompanied by a note on its potential bias.

Circularity Check

1 steps flagged · score 6.0 of 10

VRR's reported RePASs is the same metric it optimizes in verification and refinement, so its 'coherent, readable' claim is not independently established.

fitted input called prediction [Section 4.5.1-4.5.2, 4.6, Table 5, Conclusion]
"In the verification step, we obtain N alternative answers from the LLM (using all the extracted obligations and the question as input) and evaluate them using RePASs. We choose the alternative answer with the best RePASs score."

The system's answer is explicitly chosen to maximize RePASs, and the refinement step removes sentences with above-average RePASs contradiction and inserts missing obligations to raise RePASs obligation coverage. Table 5 then reports RePASs (0.639) as the measure of VRR's quality, and the Conclusion calls the answers 'coherent'. The reported metric is exactly the objective being optimized, so the score is partly a fitted value rather than an independent assessment. The NOC result (0.947 via verbatim obligation concatenation) shows RePASs can be inflated by construction, so the VRR RePASs gain does not by itself support the coherence claim without an independent human or non-RePASs evaluation.

full rationale

The paper's central non-adversarial contribution (VRR) selects candidate answers by RePASs and iteratively edits them to reduce RePASs contradiction and increase RePASs obligation coverage, then reports the hidden-test RePASs as evidence of answer quality. That is an optimization of the evaluation metric, not an independent derivation; the NOC system demonstrates the metric is easily gameable, so VRR's RePASs gain does not by itself establish 'readable, coherent answers'. The paper is transparent about the NOC exploit and about the verifier-dependence of VRR, and the retrieval and LOC components are not circular, but the VRR evaluation-as-evidence is partially circular. Score 6 reflects a central claim whose main quantitative support is the same quantity the system was built to maximize; it does not reach 8-10 because the paper is honest about the limitation and the hidden test set is an external benchmark.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central evaluation relies on RePASs as ground truth, which the paper itself shows is gameable; VRR further optimizes this metric, so the reported scores are not independent evidence of answer quality.

free parameters (6)

fusion weights a, b = a=0.25, b=0.2
Selected by trying a few combinations on the public test set (Table 2).
filtering threshold = 0.90
Selected from a small grid on the public test set (Table 3).
max drop = 0.10
Selected with threshold on the public test set (Table 3).
reranking top-N = 50
Chosen as best Recall@10 on the public test set (Figure 1).
LOC retry limit K = 3
Used for LOC obligation rewriting (Section 4.4).
VRR candidate count N = not reported
Number of alternative answers in the verification step is not stated (Section 4.5.1).

assumptions (3)

domain assumption RePASs faithfully measures answer quality
The paper's evaluations and the task's leaderboard treat RePASs as the ground truth; the paper itself demonstrates this assumption is questionable (Sections 2, 4.6).
domain assumption Obligation extraction by LegalBERT is reliable
Both RePASs scoring and the NOC attack depend on the same LegalBERT model reliably identifying 'obligation' sentences (Sections 2, 4.1).
domain assumption The public test set is representative of the hidden test set
All hyperparameters were tuned on the public test set, and the hidden test set is used only once (Sections 3.2, 4.2).

how reviews work

0 comments

Cite this review

Pith. "Pith review of AUEB-Archimedes at RIRAG-2025: Is obligation concatenation really all you need?." pith.science (2026). https://pith.science/paper/6NDL5OEV

@misc{pith2026241211567,
  author       = {Pith},
  title        = {Pith review of: AUEB-Archimedes at RIRAG-2025: Is obligation concatenation really all you need?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6NDL5OEV}},
  note         = {Machine review of arXiv:2412.11567}
}

read the original abstract

This paper presents the systems we developed for RIRAG-2025, a shared task that requires answering regulatory questions by retrieving relevant passages. The generated answers are evaluated using RePASs, a reference-free and model-based metric. Our systems use a combination of three retrieval models and a reranker. We show that by exploiting a neural component of RePASs that extracts important sentences ('obligations') from the retrieved passages, we achieve a dubiously high score (0.947), even though the answers are directly extracted from the retrieved passages and are not actually generated answers. We then show that by selecting the answer with the best RePASs among a few generated alternatives and then iteratively refining this answer by reducing contradictions and covering more obligations, we can generate readable, coherent answers that achieve a more plausible and relatively high score (0.639).

Figures

Figures reproduced from arXiv: 2412.11567 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages

[1]

Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.261 LEGAL - BERT : The muppets straight out of law school . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898--2904, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.findings-emnlp.261 2020
[2]

Catalina Goanta, Nikolaos Aletras, Ilias Chalkidis, Sofia Ranchord \'a s, and Gerasimos Spanakis. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.539 Regulation and NLP ( R eg NLP ): Taming large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8712--8724, Singapore. Association for Compu...

doi:10.18653/v1/2023.emnlp-main.539 2023
[3]

Tuba Gokhan, Kexin Wang, Iryna Gurevych, and Ted Briscoe. 2024. https://arxiv.org/abs/2409.05677 Regnlp in action: Facilitating compliance through automated information retrieval and answer generation . Preprint, arXiv:2409.05677

arXiv 2024
[4]

Yichen Huang and Timothy Baldwin. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.340 Robustness tests for automatic machine translation metrics with adversarial attacks . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5126--5135, Singapore. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-emnlp.340 2023
[5]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Sy...

work page 2020
[6]

Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.500 BERT - ATTACK : Adversarial attack against BERT using BERT . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193--6202, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.emnlp-main.500 2020
[7]

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2024. Self-refine: iterative refinement with self-feedback. In Proceedings of the 37th International ...

work page 2024
[8]

Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, P...

arXiv 2022

Show all 17 references

[9]

Dennis, and Andre Freitas

Xin Quan, Marco Valentino, Louise A. Dennis, and Andre Freitas. 2024. https://aclanthology.org/2024.emnlp-main.172 Verification and refinement of natural language explanations through LLM -symbolic theorem proving . In Proceedings of the 2024 Conference on Empirical Methods in...

2024
[10]

Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford

Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at trec-3. In Proceedings of The Third Text REtrieval Conference, TREC 1994, pages 109--126, Gaithersburg, Maryland, USA

1994
[11]

Ante Wang, Linfeng Song, Ye Tian, Baolin Peng, Lifeng Jin, Haitao Mi, Jinsong Su, and Dong Yu. 2024. https://aclanthology.org/2024.findings-emnlp.349 Self-consistency boosts calibration for math reasoning . In Findings of the Association for Computational Linguistics: EMNLP 20...

2024
[12]

Shuai Wang, Shengyao Zhuang, and Guido Zuccon. 2021. https://doi.org/10.1145/3471158.3472233 Bert-based dense retrievers require interpolation with bm25 for effective passage retrieval . In Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Ret...

2021
[13]

Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conferenc...

2023
[14]

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. https://openreview.net/forum?id=CfXh93NDgH Wizard LM : Empowering large pre-trained language models to follow complex instructions . In The Twelfth Internatio...

2024
[15]

Griffiths, Yuan Cao, and Karthik Narasimhan

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: deliberate problem solving with large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems,...

2024
[16]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[17]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.261 LEGAL - BERT : The muppets straight out of law school . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898--2904, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.findings-emnlp.261 2020

[2] [2]

Catalina Goanta, Nikolaos Aletras, Ilias Chalkidis, Sofia Ranchord \'a s, and Gerasimos Spanakis. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.539 Regulation and NLP ( R eg NLP ): Taming large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8712--8724, Singapore. Association for Compu...

doi:10.18653/v1/2023.emnlp-main.539 2023

[3] [3]

Tuba Gokhan, Kexin Wang, Iryna Gurevych, and Ted Briscoe. 2024. https://arxiv.org/abs/2409.05677 Regnlp in action: Facilitating compliance through automated information retrieval and answer generation . Preprint, arXiv:2409.05677

arXiv 2024

[4] [4]

Yichen Huang and Timothy Baldwin. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.340 Robustness tests for automatic machine translation metrics with adversarial attacks . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5126--5135, Singapore. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-emnlp.340 2023

[5] [5]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Sy...

work page 2020

[6] [6]

Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.500 BERT - ATTACK : Adversarial attack against BERT using BERT . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193--6202, Online. Association for Computational Linguistics

doi:10.18653/v1/2020.emnlp-main.500 2020

[7] [7]

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2024. Self-refine: iterative refinement with self-feedback. In Proceedings of the 37th International ...

work page 2024

[8] [8]

Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, P...

arXiv 2022