ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation

Peter Devine

REVIEW 3 major objections 5 minor 49 references

ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ALoFTRAG shows a RAG model can fine-tune itself on its own synthetic Q&A, lifting citation accuracy 8.3% and answer accuracy 3.0%.

desk verdict Useful self-training RAG method with broad multilingual eval, but the reported gains are at risk from train/eval overlap: synthetic questions come from the same passages as the gold test questions, and no dedup is reported. read the letter →

arxiv 2501.11929 v1 pith:JRWCGQ2K submitted 2025-01-21 cs.LG

Peter Devine This is my paper · ORCID

classification cs.LG

keywords retrievalaugmentedgenerationsyntheticquestionansweringdataLoRAfine-tuningself-improvementmultilingualcitationaccuracydomainadaptationlocalLLM

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ALoFTRAG tests a simple loop: a RAG system's own local LLM writes questions and answers from the domain texts it will later be queried on, keeps only the examples that pass its own rating prompts, and then LoRA fine-tunes itself on those examples in a cite-first format. The paper claims this raises citation accuracy by 8.3% and answer accuracy by 3.0% on average over the base RAG model across 20 question-answering datasets spanning 26 languages. If true, the value is practical: a domain-specific RAG system can be improved on consumer-grade hardware, without manual labels, without a larger teacher model, and without sending private data to a cloud service. The paper also finds the optional Q&A filtering step (Step 3) hurts performance, so the released configuration omits it, and that gains grow as more retrieved contexts are supplied.

What carries the argument

The load-bearing object is the self-generated training set: for each reference text, the base LLM produces one self-contained question and answer, rates and thresholds both text and Q&A quality with 0-10 prompts (threshold 8), and the dense retriever supplies n-1 hard-negative texts so each training example presents n shuffled contexts, the correct one flagged by ordinal. LoRA fine-tuning then teaches the model to output the correct document ordinal before the answer, a cite-then-answer format that the paper argues adds explainability and acts as a curriculum. Hard negatives and the use of n=10 contexts are what make the trained model cope with realistic RAG setups; ablations varying n show the advantage over the base model widens as distractors increase.

What would settle it

Have bilingual annotators score a stratified random sample of base versus ALoFTRAG answers for each of the 26 languages and compare their agreement with GPT-4o; the answer-accuracy claim fails if the human-measured gain is not consistently positive or if GPT-4o's judgments disagree with humans more in low-resource languages.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a base RAG model (Qwen2-7B-Instruct) can improve its own domain-specific retrieval-augmented performance by generating synthetic Q&A from unlabeled reference texts, filtering low-rated texts and Q&As with its own zero-temperature ratings, augmenting contexts with hard negatives selected by a dense embedding model (BGE-M3), and LoRA fine-tuning for one epoch on the resulting cite-then-answer conversations. Across 20 datasets in 26 languages, every ALoFTRAG configuration achieved higher citation and answer accuracy than the base model on almost all datasets; the full pipeline averaged +8.3% citation accuracy and +3.0% answer accuracy. The paper reports that removing the Q&A filtering step improves both metrics on average, while removing the reference-text filtering step lowers answer accuracy but raises citation accuracy, and that ALoFTRAG reduces cases where the model cites the wrong text yet gives the right answer.

Load-bearing premise

The 3.0% average answer-accuracy improvement rests entirely on GPT-4o's TRUE/FALSE judgments, with no human validation or per-language bias check; if that judge is unreliable across languages, the answer gain may not be real.

Editorial extensions

If this is right

Every ALoFTRAG variant beat the base model on citation and answer accuracy for almost all 20 datasets; the all-steps model averaged 77.7% citation accuracy versus 69.4% base, and 78.0% answer accuracy versus 75.0% base.
Dropping the Q&A filtering step (Step 3) raised both metrics on average, so the default released configuration skips it; a cheaper pipeline can outperform the full one.
ALoFTRAG's answer-accuracy gain is larger on 'hard' questions (where the correct text is not among the top-10 retrieved), suggesting the largest benefits appear exactly when retrieval is imperfect.
The accuracy gap between ALoFTRAG and base grows with more contexts (2, 5, 10), so production RAG systems that feed many chunks may see larger gains than this benchmark average.
Training reduces wrong-citation-but-right-answer cases from 19.7% to 10.1% (all steps), making the system more auditable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because Step 3 filtering was harmful, the paper's own evidence suggests that in this self-training loop volume of generated data matters more than its self-rated quality; a testable extension would vary the filtering threshold or generate multiple Q&As per text to see whether more data continues to help.
The hard-negative mechanism points to a natural extension the authors leave for future work: using the same generated triplets to fine-tune the retriever, not just the generator, which could compound the citation gains if the retriever learns which distractors are confusable.
The privacy claim depends on keeping generation, training, and inference local; the paper itself uses a cloud judge (GPT-4o) only for evaluation, so a fully offline deployment would need a local or human evaluation scheme.
If the distractor-count trend extrapolates to the 50-100 chunk RAG systems cited in the paper, ALoFTRAG may matter most for large-context production systems, but training with n=10 contexts may not transfer to substantially larger n without retraining.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear Colleague,

Quick take: this is a genuinely practical framework for adapting a local LLM to a RAG domain without labels, and the breadth of evaluation is impressive. But the headline numbers should not be taken at face value. The most serious problem, which the stress-test note flags correctly, is that the synthetic training questions are generated from the same reference texts that are later used as gold contexts in evaluation, and there is no deduplication or split reported.

Here is the mechanism. Section 3.2 generates one Q&A per reference text from the unlabelled corpus. Section 4.2-4.3 evaluates on the original gold questions using those same reference texts. For reading-comprehension benchmarks like SQuAD, MLQA, and XQuAD, the gold questions were written from those passages. When you prompt an LLM to write a self-contained question about a passage, you will often get something close to the gold question. If a near-duplicate appears in training, the LoRA-tuned model can answer it by pattern-matching rather than by reasoning over the provided contexts. That would inflate both the citation accuracy (+8.3%) and the answer accuracy (+3.0%). The paper does not address this in the limitations section.

This is more load-bearing than the GPT-4o judge issue. The judge affects only the answer metric, and even then only if the judge is biased. The overlap affects both metrics and is a direct consequence of the experimental design.

What the paper does well: the combination of self-instruction, hard-negative sampling, and LoRA for citation training is a sensible and, as far as I know, new assembly of known components. The authors ship code and report ablations, and they are honest that their QA-filtering step (Step 3) hurts performance and that they disable it by default. That is the kind of transparency I like to see.

Other soft spots, in decreasing order: no comparison to RAFT, the closest prior method; the answer metric relies on an unvalidated GPT-4o judge with no human agreement; evaluation always injects the correct context, which they acknowledge but means the numbers are optimistic for real RAG; and the paper does not discuss the large reference-accuracy drop on KenSwQuAD (54.6% to 32.1%) or PublicHealth QA.

Bottom line: this paper deserves a serious referee, but it needs revision. The authors should rerun with deduplicated training questions (or report overlap statistics) and re-evaluate; add a RAFT baseline; and validate the judge on a small human sample. If the gains survive the dedup test, it is a solid contribution. If they do not, the framework may still be useful but the claims would need scaling back.

I would bring it to a reading group as a case study in evaluation pitfalls, and I would probably not cite it in its current form.

Best, [Your name]

Referee Report

3 major / 5 minor

Summary. The paper proposes ALoFTRAG, a self-training framework for retrieval-augmented generation. Using a single local instruction-tuned LLM (Qwen2-7B-Instruct) and a multilingual dense retriever (BGE-M3), the method filters reference texts, generates one synthetic question-answer pair per text, optionally filters those pairs, mines hard-negative contexts, and LoRA fine-tunes the LLM to output the correct reference ordinal followed by the answer. The authors evaluate on 20 QA datasets spanning 26 languages, comparing the base RAG model with the ALoFTRAG model and with ablations that remove the text-filtering or QA-filtering steps. They report an average improvement of 8.3 percentage points in citation accuracy and 3.0 percentage points in answer accuracy, with paired t-tests claimed to be significant at p<0.05. The paper also reports hard/easy question splits, context-count scaling, and a discussion of limitations.

Significance. If the reported gains are real, ALoFTRAG is a practically valuable contribution: it requires no manual labels, no larger teacher model, and can run on consumer-grade hardware, all while preserving data locality. The breadth of the evaluation across 26 languages and 20 datasets is a genuine strength, as is the release of code and results, which supports reproducibility. However, the central empirical claim currently rests on two unaddressed risks: possible overlap between synthetic training questions and gold test questions, and an answer-accuracy metric that depends entirely on an unvalidated GPT-4o judge. The ablations and the honest limitations section are useful, but the most serious confound is not mentioned there. The contribution is empirical rather than theoretical, and the practical significance will depend on whether the evaluation can be made clean.

major comments (3)

[§3.2 and §4.2] The synthetic training questions are generated from the same reference texts that define the evaluation gold questions, and the paper reports no split or deduplication between the synthetic training set and the gold test questions. For the reading-comprehension datasets in Table 1 (SQuAD-style, XQuAD, MLQA, DRCD, etc.), the gold questions were authored from exactly these passages; a synthetic question that is identical or a near-paraphrase of a gold question gives the fine-tuned model the correct answer and the correct reference at test time. Because §4.3 always injects the correct reference text into the context list, this can directly inflate the reported +8.3% citation and +3.0% answer gains relative to the base model, which must reason from the context. Section 8 lists limitations but does not address this confound. The authors should quantify the overlap (e.g., string/embedding similarity) and re-evaluate after removing near-duplicate synthetic questions, or use held-out passages for evaluation.
[§4.3, Listing 6] Answer accuracy is measured solely by GPT-4o TRUE/FALSE judgments with no human validation, no agreement metric, and no per-language or per-dataset analysis. In a study spanning 26 languages, including low-resource languages such as Kirundi and Swahili, judge reliability cannot be assumed. If GPT-4o is biased toward fluent or verbose outputs, or less reliable in lower-resource languages, the +3.0% answer-accuracy claim may be an artifact of the judge rather than a real improvement. The authors should validate the judge on a human-annotated subset, report per-language agreement, and ideally include human evaluation for at least a subset of datasets.
[§5, Table 2] The average gains in Table 2 conceal large per-dataset drops that are not discussed: KenSwQuAD reference accuracy falls from 54.6% (base) to 32.1% (All Steps), and PublicHealth QA from 82.7% to 68.8%, with several answer-accuracy drops in Table 4 (e.g., chaii-1 Hindi, TyDi English). The claim in §5 that 'every ALoFTRAG implementation achieves higher citation accuracy ... across almost all datasets' is too strong without a discussion of these failures, and the paper gives no confidence intervals or variance measures for the dataset-level means. The paired t-test across 20 dataset-level scores would be more informative if accompanied by per-dataset uncertainty and an analysis of failure cases.

minor comments (5)

[§4.3] The term 'hard questions' should be defined more carefully in the main text: these are questions whose gold reference is not in the top-10 retrieved contexts before the forced swap, not necessarily questions that are semantically difficult. The current wording is understandable but easy to misinterpret.
[Appendix A and Table 1] For CalmQA and PublicHealth QA, the gold answer is also used as the reference text; this should be stated in the main text next to Table 1, because it changes the meaning of 'citation accuracy' for those datasets.
[§2] The sentence beginning 'However, these approach does not seek to improve the accuracy the actual RAG system' contains a grammatical error and should be rewritten.
[Appendix C, Listing 7] Several hyperparameters (lora_rank=64, lora_alpha=32, learning_rate=2e-4) are reported without justification or sensitivity analysis; an ablation of at least the LoRA rank or learning rate would strengthen the practical claims.
[Figure 2] The x-axis label 'Number of contexts given' is clear from the caption, but the caption could state explicitly that the correct context is always included, since this is essential for interpreting the '2 contexts' condition as an oracle setting rather than a realistic retrieval setting.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: ALoFTRAG's reported gains are empirical results against external gold labels, not consequences of its own definitions.

full rationale

The paper makes no formal derivation; its claims are empirical. ALoFTRAG uses the same base LLM to generate and filter synthetic training data, and it then LoRA fine-tunes on that data, but the reported citation and answer accuracies are measured against the original gold-label questions and answers of 20 public QA benchmarks. No parameter is fitted to those gold answers, and no equation defines the measured accuracy in terms of the training inputs. The self-training loop is the method under test, not a hidden equivalence: the evaluation compares the fine-tuned model with the base model on the same external gold labels, and the ablations (e.g., removing Step 3, which is found detrimental in most cases) show that the authors do not assume the self-filtering loop is beneficial. The skeptical concern that synthetic training questions may overlap with gold questions is a possible data-contamination risk, but the paper does not report deduplication and the circularity analysis cannot infer near-duplicate generation without evidence; no specific reduction by construction is exhibited. The framework also does not rely on load-bearing self-citations: prior work is cited for standard techniques such as LoRA, Self-Instruct, and RAFT, and no uniqueness theorem or ansatz is imported from the authors' own earlier papers to force the choice of ALoFTRAG. Overall, the central claim is self-contained as an empirical comparison against external benchmarks, so there is no significant circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The framework depends on hand-set thresholds (8 for text and Q&A filtering), a fixed context count (10), and standard LoRA hyperparameters that are not tuned. The strongest auxiliary assumptions are that the local model can generate useful self-training data across 26 languages and that GPT-4o provides an unbiased answer judge. No new physical or conceptual entities are introduced.

free parameters (7)

text_filter_threshold = 8
Hand-set threshold in Step 1; chosen after observing that 57% of CaLMQA Chinese texts were filtered at 8 while only 6.9% average across subsets. Not optimized per dataset.
qa_filter_threshold = 8
Hand-set threshold for both question and answer ratings in Step 3; ablations show this step is often detrimental, indicating the threshold is not tuned to maximize final accuracy.
num_contexts = 10
Number of reference texts presented as context during training and evaluation; set to the maximum feasible under the 20k token limit, not selected by performance.
lora_rank = 64
LoRA rank hyperparameter from Appendix C; standard for Qwen2-7B, not tuned in this paper.
lora_alpha = 32
LoRA scaling hyperparameter from Appendix C; not tuned.
learning_rate = 0.0002
Axolotl training learning rate from Appendix C; not tuned.
num_epochs = 1
Single epoch to limit overfitting; acknowledged in Limitations as a naive choice.

assumptions (4)

domain assumption The base LLM used for generation and filtering can produce fluent, answerable questions and correct answers from reference texts across 26 languages.
ALoFTRAG's training signal depends on the quality of self-generated Q&A; the paper excludes low-resource scripts like Amharic where generation failed (Limitations section).
domain assumption Qwen2-7B-Instruct has not been trained on the test datasets, so fine-tuning gains reflect adaptation rather than retrieval of memorized answers.
The authors only assert that CalmQA and M2QA postdate Qwen2's release; other datasets may overlap with training data, meaning some improvement could be re-memorization.
domain assumption GPT-4o judge provides a correct and language-fair evaluation of answer correctness.
Answer accuracy is computed solely from GPT-4o TRUE/FALSE ratings (Listing 6); no human agreement study is reported.
domain assumption Fine-tuning on synthetic RAG data transfers to gold-standard questions.
The paper evaluates on gold questions but trains on generated questions; if the generated questions are systematically easier or different, the reported improvement may not generalize.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/JRWCGQ2K

@misc{pith2026250111929,
  author       = {Pith},
  title        = {Pith review of: ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JRWCGQ2K}},
  note         = {Machine review of arXiv:2501.11929}
}

read the original abstract

Retrieval Augmented Generation (RAG) systems have been shown to improve the accuracy of Large Language Model (LLM) outputs. However, these models can often achieve low accuracy when applied to new data domains. We introduce the Automatic Local Fine Tuning of Retrieval Augmented Generation models (ALoFTRAG) framework, designed to improve the accuracy of RAG systems on a given domain by training LLMs without manually labeled data or using larger teacher models. By generating and filtering synthetic training data and performing LoRA fine-tuning, ALoFTRAG improves citation and answer accuracy across 20 datasets in 26 languages by, on average, 8.3% and 3.0% respectively. Our results demonstrate that ALoFTRAG offers a practical, cost-effective, and data-secure solution for improving RAG accuracy, making it particularly applicable to sensitive domains such as healthcare and finance.

Figures

Figures reproduced from arXiv: 2501.11929 by the authors.

**Figure 2.** Plots of answer and reference accuracy varied over number of chunks. Note that the correct context was [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 13 canonical work pages

[1]

Shane Arora, Marzena Karpinska, Hung-Ting Chen, Ipsita Bhattacharjee, Mohit Iyyer, and Eunsol Choi. 2024. Calmqa: Exploring culturally specific long-form question answering across 23 languages. arXiv preprint arXiv:2406.17761

arXiv 2024
[2]

Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2019. https://arxiv.org/abs/1910.11856 On the cross-lingual transferability of monolingual representations . CoRR, abs/1910.11856

arXiv 2019
[3]

Mohammad Yasin Ayoubi, Sajjad & Davoodeh. 2021. Persianqa: a dataset for persian question answering. https://github.com/SajjjadAyobi/PersianQA

work page 2021
[4]

Yoshua Bengio, J \'e r \^o me Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41--48

2009
[5]

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://arxiv.org/abs/2402.03216 Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . Preprint, arXiv:2402.03216

arXiv 2024
[6]

Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W Cohen. 2022. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. arXiv preprint arXiv:2210.02928

arXiv 2022
[7]

Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. Tydi qa: A benchmark for information-seeking question answering in ty pologically di verse languages. Transactions of the Association for Computational Linguistics, 8:454--470

2020
[8]

Pavel Efimov, Andrey Chertok, Leonid Boytsov, and Pavel Braslavski. 2020. https://doi.org/10.1007/978-3-030-58219-7_1 Sberquad -- russian reading comprehension dataset: Description and analysis . In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 3--15. Springer International Publishing

doi:10.1007/978-3-030-58219-7_1 2020

Show all 49 references

[9]

Leon Engl \"a nder, Hannah Sterz, Clifton Poth, Jonas Pfeiffer, Ilia Kuznetsov, and Iryna Gurevych. 2024. https://arxiv.org/abs/2407.01091 M2qa: Multi-domain multilingual question answering . arXiv preprint

2024 arXiv
[10]

Masoomali Fatehkia, Ji Kim Lucas, and Sanjay Chawla. 2024. T-rag: lessons from the llm trenches. arXiv preprint arXiv:2402.07483

2024 arXiv
[11]

Paulo Finardi, Leonardo Avila, Rodrigo Castaldoni, Pedro Gengo, Celio Larcher, Marcos Piau, Pablo Costa, and Vinicius Carid \'a . 2024. The chronicles of rag: The retriever, the chunk and the generator. arXiv preprint arXiv:2401.07883

2024 arXiv
[12]

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997

2023 arXiv
[13]

Asier Gutiérrez-Fandiño, Jordi Armengol-Estapé, Marc Pàmies, Joan Llop-Palao, Joaquin Silveira-Ocampo, Casimiro Pio Carrino, Carme Armentano-Oller, Carlos Rodriguez-Penagos, Aitor Gonzalez-Agirre, and Marta Villegas. 2022. http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/a...

2022
[14]

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929--3938. PMLR

2020
[15]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

2015 arXiv
[16]

Daniel Hl \'a dek, J \'a n Sta s , Jozef Juh \'a r, and Tom \'a s Koct \'u r. 2023. Slovak dataset for multilingual question answering. IEEE Access, 11:32869--32881

2023
[17]

Addison Howard, Deepak Nathani, Divy Thakkar, Julia Elliott, Partha Talukdar, and Phil Culliton. 2021. https://kaggle.com/competitions/chaii-hindi-and-tamil-question-answering chaii - hindi and tamil question answering

2021
[18]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

2021 arXiv
[19]

Simon Hughes, Minseok Bae, and Miaoran Li. 2023. https://github.com/vectara/hallucination-leaderboard Vectara hallucination leaderboard

2023
[20]

Tom \'a s Ko c isk \`y , Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, G \'a bor Melis, and Edward Grefenstette. 2018. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317--328

2018
[21]

Kentaro Kurihara, Daisuke Kawahara, and Tomohide Shibata. 2022. Jglue: Japanese general language understanding evaluation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2957--2966

2022
[22]

Gonzalez, Hao Zhang, and Ion Stoica

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

2023
[23]

Patrick Lewis, Barlas O g uz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2019. Mlqa: Evaluating cross-lingual extractive question answering. arXiv preprint arXiv:1910.07475

2019 arXiv
[24]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

2020
[25]

Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. 2023. Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.06259

2023 arXiv
[26]

Seungyoung Lim, Myungji Kim, and Jooyoul Lee. 2019. Korquad1. 0: Korean qa dataset for machine reading comprehension. arXiv preprint arXiv:1909.07005

2019 arXiv
[27]

Xing Han Lu. 2023. https://www.kaggle.com/datasets/xhlulu/covidqa Covid-qa . Accessed: 2024-08-19

2023
[28]

Hussein Mozannar, Elie Maamary, Karl El Hajal, and Hazem Hajj. 2019. https://doi.org/10.18653/v1/W19-4612 Neural A rabic question answering . In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 108--118, Florence, Italy. Association for Computationa...

2019 doi
[29]

Timo Möller, Julian Risch, and Malte Pietsch. 2021. https://arxiv.org/abs/2104.12741 Germanquad and germandpr: Improving non-english question answering and passage retrieval . Preprint, arXiv:2104.12741

2021 arXiv
[30]

OpenAI. 2024. https://openai.com/index/hello-gpt-4o/ Hello GPT-4o

2024
[31]

Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. 2023. Language model self-improvement by reinforcement learning contemplation. arXiv preprint arXiv:2305.14483

2023 arXiv
[32]

Mehmet Ali Peker. 2020. Tquad. https://github.com/TQuad/turkish-nlp-qa-dataset

2020
[33]

Paulo Pirozelli, Marcos M Jos \'e , Igor Silveira, Fl \'a vio Nakasato, Sarajane M Peres, Anarosa AF Brand \ a o, Anna HR Costa, and Fabio G Cozman. 2024. Benchmarks for pir \'a 2.0, a reading comprehension dataset about the ocean, the brazilian coast, and climate change. Data...

2024
[34]

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316--1331

2023
[35]

Amanda Ross and Victor L. Willson. 2017. https://doi.org/10.1007/978-94-6351-086-8_4 Paired Samples T-Test , pages 17--19. SensePublishers, Rotterdam

2017 doi
[36]

Chih Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng, and Sam Tsai. 2018. Drcd: A chinese machine reading comprehension dataset. arXiv preprint arXiv:1806.00920

2018 arXiv
[37]

Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567

2021 arXiv
[38]

Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering. Transactions of the Association for Comp...

2023
[39]

Tilahun Abedissa Taffa, Ricardo Usbeck, and Yaregal Assabie. 2024. Low resource question answering: An amharic benchmarking dataset. In Proceedings of the Fifth Workshop on Resources for African Indigenous Languages@ LREC-COLING 2024, pages 124--132

2024
[40]

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560

2022 arXiv
[41]

Barack W Wanjawa, Lilian DA Wanzare, Florence Indede, Owen McOnyango, Lawrence Muchemi, and Edward Ombui. 2023. Kenswquad—a question answering dataset for swahili low-resource language. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4):1--20

2023
[42]

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024 a . Qwen2 technical report. arXiv preprint arXiv:2407.10671

2024 arXiv
[43]

Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, et al. 2024 b . Crag--comprehensive rag benchmark. arXiv preprint arXiv:2406.04744

2024 arXiv
[44]

Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476--15488

2022
[45]

Shenglai Zeng, Jiankun Zhang, Pengfei He, Jie Ren, Tianqi Zheng, Hanqing Lu, Han Xu, Hui Liu, Yue Xing, and Jiliang Tang. 2024. Mitigating the privacy issues in retrieval-augmented generation (rag) via pure synthetic data. arXiv preprint arXiv:2406.14773

2024 arXiv
[46]

Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E Gonzalez. 2024. Raft: Adapting language model to domain specific rag. arXiv preprint arXiv:2403.10131

2024 arXiv
[47]

Kunlun Zhu, Yifan Luo, Dingling Xu, Ruobing Wang, Shi Yu, Shuo Wang, Yukun Yan, Zhenghao Liu, Xu Han, Zhiyuan Liu, et al. 2024. Rageval: Scenario specific rag evaluation dataset generation framework. arXiv preprint arXiv:2408.01262

2024 arXiv
[48]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[49]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Shane Arora, Marzena Karpinska, Hung-Ting Chen, Ipsita Bhattacharjee, Mohit Iyyer, and Eunsol Choi. 2024. Calmqa: Exploring culturally specific long-form question answering across 23 languages. arXiv preprint arXiv:2406.17761

arXiv 2024

[2] [2]

Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2019. https://arxiv.org/abs/1910.11856 On the cross-lingual transferability of monolingual representations . CoRR, abs/1910.11856

arXiv 2019

[3] [3]

Mohammad Yasin Ayoubi, Sajjad & Davoodeh. 2021. Persianqa: a dataset for persian question answering. https://github.com/SajjjadAyobi/PersianQA

work page 2021

[4] [4]

Yoshua Bengio, J \'e r \^o me Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41--48

2009

[5] [5]

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://arxiv.org/abs/2402.03216 Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . Preprint, arXiv:2402.03216

arXiv 2024

[6] [6]

Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W Cohen. 2022. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. arXiv preprint arXiv:2210.02928

arXiv 2022

[7] [7]

Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. Tydi qa: A benchmark for information-seeking question answering in ty pologically di verse languages. Transactions of the Association for Computational Linguistics, 8:454--470

2020

[8] [8]

Pavel Efimov, Andrey Chertok, Leonid Boytsov, and Pavel Braslavski. 2020. https://doi.org/10.1007/978-3-030-58219-7_1 Sberquad -- russian reading comprehension dataset: Description and analysis . In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 3--15. Springer International Publishing

doi:10.1007/978-3-030-58219-7_1 2020