{"id":"2cce66c1-4ab2-4f50-80e9-86ad897e6a0e","arxiv_id":"2502.09073","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"AL4RAG uses a retrieval-aware similarity metric to select annotation-worthy RAG conversation records, yielding DPO-trained models that reject hallucination-prone queries and preserve answer quality.","lead":"This paper proposes AL4RAG, an active learning method that picks conversation records to annotate and then fine-tunes a RAG model to refuse hallucination-prone questions while answering easy ones accurately. It introduces a similarity metric, ras, that measures query and reference distances separately, and reports gains over baselines on the RAGTruth benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Rejection Rate is undefined in §6.1 and the preference pairs in §4.2 share the same fixed refusal string; the reported ras gains in Table 1 could depend on that unstated metric.","rationale":"The reader's weakest assumption identifies the same issue: the binary hallucination labels, the fixed refusal template, and the undefined Rejection Rate. I agree with that reading. This is load-bearing because the paper's only quantitative evidence for ras is Tables 1-2 and Figure 2. ROUGE-L and BERTScore are standard and well-defined, but RR is not, and the rejection half of the central claim is therefore unverifiable as written. The mechanical preference construction in §4.2 uses one fixed refusal string as both DPO target and the obvious candidate for an exact-match RR; if RR is exact match, models that learn to emit the template on inputs resembling hallucinated training samples will score high. That might be the intended behavior, but it does not by itself show that ras's diversity score causes the improvement, because the paper does not report the hallucination class balance of each selected set or false-refusal rates. The absence of code and data makes these gaps unresolvable from the preprint. These are addressable rather than fatal problems, so the reader's CONDITIONAL verdict is appropriate and I would not change it.","tokens_in":14736,"tokens_out":11776,"duration_ms":123744,"concrete_test":"Release the selection and DPO pipeline, then rerun Table 1 at 12.5%, 25%, and 50% with Rejection Rate computed three ways on held-out hallucination-prone queries: exact match to the fixed refusal template; containment of 'beyond my ability' or 'cannot answer'; and independent LLM judgment. Report refusal rate on answerable queries under each definition. If the AL4RAGras margin over IDDS is not preserved across definitions, or if the gain comes with a large false-refusal rate, the advantage is an artifact of the unstated metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The rejection half of the central claim rests on Rejection Rate (RR), but §6.1 only says 'we use Rejection Rate (RR) for evaluation' without giving a formula, matching rule, or threshold. Meanwhile §4.2 constructs every DPO preference pair from a binary hallucination label and one fixed refusal string ('Sorry, this question is beyond my ability.'): for h=1 the chosen response is that string, and for h=0 it is the rejected response. The headline comparison in Table 1 (AL4RAGras RR 22.65 vs IDDS 18.66 at 12.5% data) therefore depends on how RR is computed. If RR is exact match to the same template used as the DPO target, the gap may partly measure how strongly each selected subset teaches the model to copy the template on hallucination-like inputs, not how well ras captures diversity. If RR is substring- or judge-based, the ranking could change. Section 6.1 also says hallucination labels were taken from RAGTruth and mapped to preferred/non-preferred answers, rather than obtained through the human-annotation procedure described in §4.2, so the 'human preference dataset' framing is not what the experiments implement. The paper does not report false-refusal rates on answerable queries, so the RR gain cannot be separated from a general refusal bias.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AL4RAG, an active learning method for curating RAG conversation records for DPO fine-tuning. It introduces retrieval-augmented similarity (ras) (Eq. 5), which measures sample distances via separate query/reference/answer attributes and a minimum operation, and selects diverse samples under an annotation budget. The selected samples are converted into preference pairs using hallucination labels and a fixed refusal response, and the model is fine-tuned with DPO. Experiments on a RAGTruth-derived dataset report consistent gains in Rejection Rate and stability metrics over random, entropy, coreset, BLEUVar, and IDDS baselines.","tokens_in":14966,"tokens_out":6951,"duration_ms":63442,"significance":"The paper addresses a real bottleneck: annotation cost for building preference data in RAG. The ras distance is a sensible idea and the ablations in Table 3 support the role of query/reference separation. The direction of the reported gains is consistent across Tables 1 and 2, and the authors provide implementation details and explicit hyperparameter values. However, the evaluation is compromised by an undefined central metric and by a preference-construction pipeline that is described as human annotation but implemented with externally supplied hallucination labels and a fixed template. These issues must be resolved before the empirical claims can be interpreted. If resolved and the gains survive significance testing, the contribution would be a useful practical tool for budget-limited RAG alignment.","major_comments":[{"comment":"The central rejection metric, Rejection Rate (RR), is never defined. The paper reports RR in Table 1 and Figure 2 without specifying the formula, matching rule, or threshold; if RR counts exact string matches to the fixed refusal template used to construct DPO targets in §4.2, the AL4RAGras gains could largely reflect how strongly each selected subset teaches the model to copy that template. Please provide the exact definition and also report the false-refusal rate on answerable queries.","section":"§6.1, Rejection Rate metric"},{"comment":"The preference dataset is not human preference data as described. §4.2 says annotators assess hallucinations and generate label h, but §6.1 says hallucination labels were taken from RAGTruth and mapped to preferred/non-preferred answers. Moreover, every h=1 sample uses the same fixed refusal string as chosen and every h=0 sample uses the same string as rejected, so the preference set is constructed by a fixed template rather than by paired human judgments. The contributions claiming a human preference dataset, including the 'first human preference dataset tailored to the RAG scenario', are therefore not supported as written. Please clarify which labels came from humans and which were derived, and report the annotation details.","section":"§4.2 vs. §6.1"},{"comment":"Results are reported as means over five runs with no standard deviations or significance tests, so the claimed 'consistent outperformance' cannot be separated from run-to-run noise. This matters because stability differences are small; for example, Table 2 shows several baselines within 20-21 ROUGE-L at 12.5%, yet no variance information is given. Please provide per-run values, error bars, and a paired significance test across the five runs.","section":"Tables 1 and 2"},{"comment":"The ras similarity uses an unmotivated 'min' over prompt similarity and the average of query/reference similarities, and the IDDS score depends on λ in Eq. (3), with neither subject to a sensitivity analysis. Since the core claim is that ras improves diversity selection, please provide ablations on the fusion function (min vs. max vs. average) and a λ sweep; otherwise the observed gains could be due to the specific hyperparameter choice rather than the principle of independent attribute measurement.","section":"Eq. (5) and Eq. (3)"},{"comment":"The DPO experiments use an SFT model 'fine-tuned on the training set', i.e., on the full data, while the active learning budget is applied only to the DPO preference pairs. This makes the comparison an evaluation of data selection for DPO given a fully supervised SFT stage, not the annotation-budget scenario described in the abstract and introduction. Please state whether the SFT model is trained on all data or only on the AL-selected subset, and if the former, discuss how this affects the budget interpretation.","section":"§6.1, Model paragraph"},{"comment":"The reference list includes Xu et al. (2024b), 'ActiveRAG: Revealing the Treasures of Knowledge via Active Learning', but the text never cites or discusses it. Because the abstract and §1 claim the first AL framework for RAG, this omission is load-bearing for the novelty claim. Please discuss this work explicitly and position the contribution relative to it.","section":"References / novelty"}],"minor_comments":[{"comment":"'top %k samples' should read 'top k% samples'.","section":"§4.1, step (5)"},{"comment":"The word 'referneces' is a typo for 'references'.","section":"§5.1"},{"comment":"The left panel's y-axis label 'Rejection' should be 'Rejection Rate' for consistency with the text and tables.","section":"Figure 2"},{"comment":"The description of the RAGTruth expansion should state the exact filtering criteria and per-task counts; as written, it is unclear how the approximately 3,000 entries were obtained.","section":"§6.1"},{"comment":"Several bibliography entries (e.g., Wang et al., 2024a,b; Xu et al., 2024a) are not cited in the text; please either cite them in context or remove them from the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's strongest result is the consistent improvement of AL4RAGras over baselines, but I cannot recommend acceptance before the Rejection Rate is defined and the preference-construction pipeline is reconciled with the claimed human annotation. The novelty claim of being 'first' for AL in RAG also requires a direct discussion of the cited but uncited ActiveRAG work. Please also ensure the SFT protocol is described precisely; if the SFT model uses the full training set, the annotation-budget argument is considerably weaker."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a workmanlike extension of IDDS-style diversity-based active learning to RAG conversation records. The genuinely new piece is the ras similarity: take the minimum of prompt similarity and the average of query/reference similarities, then feed that into the IDDS score. That is a simple, sensible fix for the fact that prompt similarity is dominated by long retrieved passages. The ablations do some work: dropping the query term (-q) hurts rejection, adding the answer term (+a) hurts stability, which is consistent with their story. The preference construction from single-response conversations, by pairing a fixed refusal string against the original answer depending on a hallucination label, is also a practical trick worth stealing.\n\nThe headline numbers are consistent. AL4RAGras beats seven baselines across three data proportions on both rejection and stability metrics. The stability gains on ROUGE-L/BERTScore are on answerable queries, which reduces the worry that the method just teaches the model to refuse everything.\n\nNow the soft spots, in order of seriousness. First, Rejection Rate is never defined in §6.1. No formula, no matching rule, no threshold. The stress-test note is right that this matters: since the same fixed refusal string is used as the DPO target for hallucinated samples, if RR is exact- or substring-match against that template, part of your 22.65 vs 18.66 gap might be how strongly each selected subset teaches template copying, not diversity. The paper does not report false-refusal rates on answerable queries, so the cleanup is cheap: define the metric, and report recall and precision of refusal behavior. Second, the paper says \"human annotation\" and \"first human preference dataset\" in §4.2 and the contributions, but §6.1 says labels came from RAGTruth. There is no new human annotation beyond what RAGTruth already had. That framing should be corrected. Third, \"first AL framework for RAG\" is contradicted by the paper's own citation of ActiveRAG (Xu et al., 2024b), which is exactly that.\n\nThe lack of error bars, significance tests, and released code/data is a real limitation, but this is an arXiv preprint; for a workshop or a revision, these are addressable. The citation pattern looks normal; the self-citations are all the authors' prior federated-learning work, unrelated to the method, and the relevant AL/RAG work is cited.\n\nBottom line: the central argument holds up directionally, and the ras idea is worth engaging with. The paper deserves a serious referee. I'd want the metric defined, the framing fixed, and code/data released before I'd trust the exact numbers, but this is a legitimate conditional, not a reject.","headline":"A useful ras similarity variant that improves diversity-based active learning for RAG, but the undefined Rejection Rate and over-claimed 'first' framing need fixing before the headline numbers can be trusted.","tokens_in":15534,"tokens_out":2778,"would_cite":true,"duration_ms":26397,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AL4RAG claims that a diversity-based active learner using retrieval-augmented similarity (ras) selects RAG conversation records whose annotation yields models that refuse hallucination-prone queries and answer the rest more steadily.","keywords":["active learning","retrieval-augmented generation","preference optimization","DPO","hallucination refusal","sample diversity","similarity metric","RAG conversation records"],"falsifier":"Re-run the 12.5%-budget comparison with Rejection Rate defined and measured against human judgments of whether the model should have refused, using independently re-annotated hallucination labels; if ras-selected DPO no longer beats the IDDS baseline, the reported advantage rests on the label and refusal pipeline rather than on the similarity measure.","tokens_in":14498,"feed_emoji":"🎯","tokens_out":5281,"duration_ms":44677,"temperature":0.7,"pith_summary":"The paper's central claim is that the annotation bottleneck for teaching retrieval-augmented generation (RAG) models when to refuse and when to answer can be attacked with active learning tailored to RAG's three-part conversation structure. It proposes AL4RAG, which selects a diverse subset of unlabeled query-reference-response records for human labeling, then converts those labels into a preference dataset and fine-tunes the model with DPO. The key to the selection is a new similarity measure, ras, that compares queries and retrieved references separately and takes the minimum with the full-prompt similarity, so that short user queries are not drowned out by long reference texts. On an expanded RAGTruth benchmark, models trained on data selected by ras reach a rejection rate of 22.65 with 12.5% of the data, higher than the 18.66 of the strongest IDDS baseline, and also improve answer stability (ROUGE-L 22.23 vs 20.96). A sympathetic reading is that budget-limited preference-data creation for RAG can be materially improved by better sample-distance measurement.","feed_headline":"With 12.5% of data, a smart pick beats full-data RAG training","feed_subtitle":"A new similarity score ras selects conversation records that teach RAG models when to refuse and when to answer.","key_machinery":"The load-bearing object is retrieval-augmented similarity (ras), defined for two samples $x$ and $y$ as $\\operatorname{ras}(x,y) = \\min(\\cos(p_x,p_y), \\tfrac{1}{2}(\\cos(q_x,q_y)+\\cos(r_x,r_y)))$, where $q$ is the user query, $r$ is the concatenated retrieved reference, and $p$ is the prompt combining $q$ and $r$. The min with the prompt similarity prevents long references from inflating apparent similarity, while the separate query-reference term keeps the user's intent influential. This similarity feeds an IDDS-style scoring loop that prefers samples far from the already-selected set and close to the remaining unlabeled pool, producing the subset that is annotated and turned into DPO preferences.","core_discovery":"On the paper's own terms, the discovery is that diversity-based active learning for RAG fails when sample similarity is computed on the user query alone or on the whole prompt, because retrieved references are long and dominate the distance; a similarity that separates the query and reference fields, averages those similarities, and caps the result by the prompt similarity selects more informative records. Given the same annotation budget, DPO training on the ras-selected preference set outperforms random, entropy, coreset, BLEU-variance, and three IDDS variants on both refusal of hallucination-prone queries and stability of correct answers, and a quarter of the ras-selected data beats full-data DPO on rejection and stability. The paper further claims that this is the first active-learning framework for RAG and the first human preference dataset built for RAG-style refusal and answer decisions.","pith_inferences":["The min-of-prompt and mean-of-parts construction suggests a general principle for structured records: when fields have very different lengths, aggregate field-level similarities and guard against dominance by a single long field; the same trick could transfer to other multi-field inputs such as code-and-doc or table-and-question pairs.","The observed decline of ras's advantage as the budget grows hints that diversity selection is most valuable when the annotation budget is tight; one testable extension is an adaptive budget rule that switches from diversity sampling to another criterion once the marginal gain flattens.","Because the preference set teaches refusal through a fixed template, a natural next step not pursued in the paper is measuring whether the learned refusal generalizes to new phrasing of hallucination-prone questions or to out-of-distribution queries.","Implicit in the paper is the idea that hallucination labels themselves are the supervision; if those labels are noisy, the same pipeline could be re-run with labels from a different source to see whether ras still selects better records."],"forward_implications":["With 12.5% of records selected by ras, DPO reaches a rejection rate of 22.65 and ROUGE-L of 22.23, beating the best IDDS variant (18.66 and 20.96) and full-data DPO, so annotation budgets can be cut to a small fraction of the available pool.","Selecting samples before annotation is enough; all selected records are labeled at the end, so annotators do not wait between rounds.","The refusal preference is constructed from single-answer records by pairing the original answer with a fixed refusal template according to hallucination labels, making single-response RAG logs usable for DPO.","Ablation results show that dropping the query term or adding the answer term hurts rejection or stability, identifying which part of the record the diversity score must preserve.","TF-IDF vectorization outperforms Sentence-BERT and a 1.5B embedding model for this selection, so surface-form diversity rather than deep semantics drives the reported gains."],"supporting_citations":[{"why":"Supplies the RAGTruth corpus of queries, references, responses, and hallucination labels that the paper expands into its preference dataset.","marker":"(Niu et al., 2023)"},{"why":"Provides the IDDS diversity scoring framework and the summarization active-learning baseline that ras is inserted into and compared against.","marker":"(Tsvigun et al., 2022)"},{"why":"Supplies the DPO objective used to fine-tune the model on the constructed preference pairs.","marker":"(Rafailov et al., 2024)"},{"why":"Establishes similarity sampling for large-model selection, the diversity-based approach the paper extends and compares with.","marker":"(Margatina et al., 2023)"},{"why":"Defines the Llama-2-7B-chat base model that is SFT-initialized and DPO-trained throughout the experiments.","marker":"(Touvron et al., 2023)"}],"fun_headline_variants":["Active learning on conversations lets RAG do more with 12.5% data","RAG refusal trained on 12.5% active-learned conversations wins","New ras distance makes active learning pick better RAG samples","12.5% of well-chosen conversation data beats full RAG training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that the RAGTruth hallucination labels, converted into preference pairs through a fixed refusal template, are correct targets for DPO, and that the unstated Rejection Rate metric measures what the model learned about when to refuse.","fun_headline_variants_meta":{"raw":{"variants":["Active learning on conversations lets RAG do more with 12.5% data","RAG refusal trained on 12.5% active-learned conversations wins","New ras distance makes active learning pick better RAG samples","12.5% of well-chosen conversation data beats full RAG training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000821,"raw_usage":{"total_tokens":3576,"prompt_tokens":914,"completion_tokens":2662,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":2582}},"tokens_in":530,"tokens_out":2662,"duration_ms":17002,"temperature":1.0,"reasoning_tokens":2582,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T22:42:18.059479+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 12.5%-budget comparison with Rejection Rate defined and measured against human judgments of whether the model should have refused, using independently re-annotated hallucination labels; if ras-selected DPO no longer beats the IDDS baseline, the reported advantage rests on the label and refusal pipeline rather than on the similarity measure.","supporting_citations":[{"cited_title":"Active learning for abstractive text summarization","cited_arxiv_id":null,"evidence_quote":"Provides the IDDS diversity scoring framework and the summarization active-learning baseline that ras is inserted into and compared against."},{"cited_title":"Active learning principles for in-context learning with large language models","cited_arxiv_id":null,"evidence_quote":"Establishes similarity sampling for large-model selection, the diversity-based approach the paper extends and compares with."}],"review_version":1}