{"id":"3beeb3dc-7ef7-47b4-92d8-bc3cc7e96366","arxiv_id":"2412.14922","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A noise-robust SFT framework that detects noisy responses via multi-expert LLM consensus, relabels them with context-enhanced reasoning, and filters low-confidence samples, improving LLM performance on five benchmarks.","lead":"RobustFT detects and corrects noisy labels in supervised fine-tuning data for large language models, improving downstream accuracy under 30% to 70% noise. The method combines multi-expert LLM consensus checking, context-based relabeling, and confidence filtering.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Eq. 3 Checker partition is never evaluated against the known synthetic noise mask, so the claimed detection-and-correction mechanism is unverified.","rationale":"The reader's weakest_assumption identifies exactly the concern I would raise: the Checker in Eq. 3 is the load-bearing component, and its accuracy is never measured. The synthetic noise setup makes detector evaluation trivial, so the absence of precision/recall is a conspicuous evidence gap. The strongest claim—that RobustFT reliably detects and corrects noisy labels—cannot be separated from Checker quality. I also note the secondary issues the reader mentions (the FPB 70% discrepancy between Table 1 and Table 2, missing error bars, underspecified prompts), but those do not change the central concern. The end-task improvements are plausible and the code/data are promised, so the appropriate verdict remains conditional: acceptance should require reporting Checker precision/recall against the injected noise mask and resolving the table inconsistency. My read does not move the reader's CONDITIONAL verdict, so I mark it UNCHANGED.","tokens_in":12970,"tokens_out":5200,"duration_ms":50224,"concrete_test":"Run RobustFT's noise-detection stage (Eq. 3) on MMLU and ARC at 30%, 50%, and 70% synthetic noise using the released code, and compare the Checker's ri decisions against the known noise mask; report precision, recall, and F1 for Dclean. If the Checker F1 is low while final accuracy remains high, the observed gains cannot be attributed to the claimed detection mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RobustFT's central mechanism is the Eq. 3 Checker, which splits the training set into Dclean and Dnoise based on agreement among the original label yi, the base prediction yhat_i, and the reasoning-enhanced prediction yhat^reas_i. Everything downstream depends on this partition: context retrieval (Eq. 5), relabeling via the Review Agent (Eq. 6), and entropy-based selection (Eq. 8) operate only on Dnoise, while Dclean is passed into the final fine-tuning set unmodified. Because the noise in Section 4 is synthetically injected, the ground-truth clean/noisy mask is known; the paper nevertheless reports no precision, recall, or confusion matrix for this binary split. This gap is load-bearing for two reasons. First, if the Checker has low precision, clean samples are relabeled or discarded; if it has low recall, noisy labels survive untouched. Second, the agreement heuristic is biased toward the model's prior: a noisy label that happens to match the base or reasoning output is marked clean, while a correct label that disagrees with the base model is sent for relabeling and may be dropped by the entropy filter. End-task gains could then reflect aggressive filtering or reinforcement of model priors rather than accurate noise correction. The paper's claim that RobustFT detects and corrects noisy labels therefore rests on an unmeasured component. A direct evaluation of the Checker against the injected noise mask would settle whether the partition is actually separating clean from noisy samples.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RobustFT, a two-stage framework for robust supervised fine-tuning of LLMs under noisy response data. In the noise-detection stage, the base LLM generates a prediction for each training sample, a reasoning-enhanced LLM generates a second prediction via iterative reasoning and reflection, and a 'Checker' partitions the data into clean and noisy sets based on agreement among the original label and the two model predictions. In the denoising stage, noisy samples are relabeled using retrieved context from the clean set plus a Review Agent, and an entropy-based selection step retains the most confident relabeled samples. The final fine-tuning set is the union of the clean set and the selected relabeled set. The method is evaluated on five benchmarks (MMLU, ARC, PubMedQA, DROP, FPB) at 30%, 50%, and 70% injected noise, using three base LLMs, with ablations and sensitivity analyses.","tokens_in":13252,"tokens_out":3683,"duration_ms":30269,"significance":"The problem addressed is practically important: noisy training responses are common in real-world data collection, and the paper demonstrates large end-task degradations from noise and consistent improvements of RobustFT over vanilla SFT across datasets, noise levels, and model families. Strengths include the breadth of the experimental study (three models, five datasets, three noise rates), the ablation study showing that each framework component contributes, the public release of code and data, and the clear framing of a self-contained pipeline without external models. If the detection-and-denosing mechanism is actually identifying and correcting the injected noise, this is a useful contribution to noise-robust LLM adaptation. The main caveat, discussed below, is that the central Checker component is both under-specified and never directly evaluated against the known noise mask, leaving the claimed mechanism unverified.","major_comments":[{"comment":"The Checker is never concretely defined. Equation (3) states ri = Checker(yi, ŷi, ŷ_i^reas) ∈ {0,1}, but the text gives no operational definition of the consistency metric, no description of the prompt or rule used to combine the three inputs, and no formal meaning of the threshold θ that appears in §4.1.3. Without this specification the method is not reproducible, and the claim that this 'Checker mechanism' performs noise detection cannot be tested.","section":"§3.2, Eq. (3)"},{"comment":"The Encoder used for query retrieval is not specified. Equation (4) introduces hi = Encoder(qi) ∈ R^d, but the paper does not state what model or embedding is used, how d is chosen, or whether the encoder is trained or frozen. Since the quality of the retrieved context in Eq. (5) directly depends on the encoder's semantic similarity, this missing detail is load-bearing for the denoising mechanism.","section":"§3.3, Eq. (4)"},{"comment":"The central noise-detection mechanism is never evaluated against the known synthetic noise mask. The experiments inject noise into training responses, so the ground-truth clean/noisy status of every sample is known; yet the paper reports no precision, recall, or confusion matrix for the Checker's partition into D_clean and D_noise. This omission is load-bearing because if the Checker has low precision, clean samples are unnecessarily relabeled or discarded, and if it has low recall, noisy labels pass through to fine-tuning. End-task improvements could therefore arise from aggressive filtering or from reinforcing the base model's prior rather than from accurate noise correction.","section":"§4, 'Main Result' and §3.2"},{"comment":"The reported results for Llama3.1-8B on FPB at 70% noise are inconsistent: Table 1 lists the RobustFT result as 76.2, while Table 2 lists the same condition as 73.2. Since these tables are both used to support the central claim of consistent improvement, the discrepancy must be reconciled (or explained, e.g., by different evaluation splits or seeds).","section":"Table 1 vs. Table 2"},{"comment":"The abstract and conclusion claim 'significant improvements,' but the main tables report only point estimates with no error bars, significance tests, or multiple-seed training runs. The stability analysis in §4.3.5 and Figure 6 covers only MMLU and ARC and measures inference variability under rephrased instructions, not variance of the fine-tuning procedure. Please provide variance estimates for at least the main comparisons, or soften the statistical claim.","section":"§4.2, Tables 1 and 2"}],"minor_comments":[{"comment":"The sentence 'We set then = 4and θ = 50%' contains a typo; it should read 'We set n = 4 and θ = 50%.'","section":"§4.1.3"},{"comment":"The DROP dataset is used in the experiments but no reference is provided for it; please add a citation for DROP.","section":"§4.1.1"},{"comment":"The quantity in Eq. (7) is the average negative log-likelihood per token (which becomes perplexity when exponentiated), not the Shannon entropy of the token distribution; please rename or clarify the terminology to avoid confusion.","section":"§3.4, Eq. (7)"},{"comment":"The ablation variants (w/oSelection, w/oChecker, w/oReviewer, w/oCER, w/oREL) are not fully defined in the main text; please specify what each variant removes and how the removed component is replaced.","section":"§4.3.1"},{"comment":"The baselines SelfLabel, SelfSelect, and SelfRAG are only mentioned by name in the main text with implementation details deferred to the appendix; please add one-sentence descriptions in the main text so that a reader can understand the comparisons without consulting the appendix.","section":"§4.1.2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and the experimental scope is broad, but the central Checker component is both under-specified and unevaluated against the known noise mask. This is a load-bearing gap that can be fixed within the manuscript's scope by adding a precise Checker definition, reporting precision/recall of the detection, and reconciling the Table 1/Table 2 discrepancy. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the takeaway: RobustFT is a practical, well-tested pipeline for fine-tuning LLMs on noisy response data, and the gains it reports are consistent across three models and five datasets. But the central claim—that it detects and corrects noisy labels—rests on a noise-detection step that is never directly evaluated, and there's an inconsistency in the reported numbers that needs resolving.\n\nWhat's actually new: the combination of multi-expert consensus (base prediction, reasoning-enhanced prediction, original label) to flag noisy samples, followed by retrieval-augmented relabeling and entropy-based selection, applied to SFT. The components are individually known, but the integrated framework is new. The ablations show each piece contributes, and the improvements over vanilla SFT at high noise rates are large (e.g., 81% relative on MMLU at 70% noise). That's worth taking seriously.\n\nWhat's soft: First, the Checker in Eq. 3 partitions the data into clean and noisy, and everything downstream depends on that partition. Noise is synthetically injected, so the ground-truth mask is known. The paper never reports precision, recall, or a confusion matrix for this binary split. That's a load-bearing omission. If the Checker is biased—say, it marks noisy labels that agree with the model as clean, or sends correct labels that disagree with the model for relabeling—the end-task gains might come from filtering or prior reinforcement, not from actual noise correction. A direct evaluation against the injected mask would settle it.\n\nSecond, Table 1 gives FPB at 70% noise as 76.2 for Llama-3.1-8B, but Table 2 gives 73.2 for the same model and setting. One of these is wrong, and that makes me want to verify the other numbers. Third, there are no error bars or significance tests on the main tables. The stability analysis in Fig. 6 is about instruction rephrasing, not the core results. Fourth, Eq. 7 is labeled entropy but is actually per-token negative log-likelihood (log perplexity). Minor, but it should be fixed.\n\nThe circularity concern is not a real problem here—this is a self-training-style pipeline, and that's standard. The missing Checker evaluation is the real issue.\n\nWho's this for? Anyone fine-tuning LLMs on imperfect data, and researchers working on data cleaning for SFT. It's a useful paper with a clear contribution. It deserves a serious referee, but the revision should be conditional on evaluating the Checker against the known noise mask, fixing the table inconsistency, and adding some measure of variance.\n\nMy recommendation: send it to review, but insist on those additions.","headline":"A plausible and broadly effective noisy-SFT pipeline, but the load-bearing noise-detection step is never evaluated against the known noise mask, and the numbers need cleaning up.","tokens_in":13795,"tokens_out":3164,"would_cite":true,"duration_ms":23307,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RobustFT claims that a multi-model consistency check plus context-based relabeling can detect and fix noisy responses in supervised fine-tuning data, keeping LLM downstream performance high even at 70% noise.","keywords":["noisy labels","supervised fine-tuning","large language models","noise detection","data relabeling","multi-expert consistency","entropy-based data selection","context-enhanced retrieval"],"falsifier":"Inject noise with known ground truth into the five datasets, run only the Checker, and compare its clean/noisy flags with the true injected noise bit; if the Checker misses most noisy samples or flags many clean ones, the reported fine-tuning gains cannot be attributed to accurate noise detection.","tokens_in":12766,"feed_emoji":"🧠","tokens_out":11644,"duration_ms":80429,"temperature":0.7,"pith_summary":"Supervised fine-tuning (SFT) of large language models is only as good as its training responses, and collected data inevitably contains wrong answers. This paper proposes RobustFT, a framework that tries to make SFT noise-robust by detecting and relabeling noisy responses before training. It claims that a consensus check among the base LLM, a reasoning-refined LLM, and the original label separates clean from noisy samples, and that relabeling with retrieved clean context plus review restores most of the lost accuracy. Across five benchmarks and three open LLMs, RobustFT reports higher downstream accuracy than vanilla inference or standard SFT at 30%, 50%, and 70% synthetic noise. If the claim holds, practitioners can fine-tune on imperfect real-world collections without sacrificing reliability.","feed_headline":"RobustFT keeps LLM fine-tuning accurate even with 70% noisy responses","feed_subtitle":"A consensus check plus context-based relabeling lets downstream models beat plain SFT on five benchmarks.","key_machinery":"The load-bearing object is the Checker consistency rule of Eq. (3), which maps the triple of original label $y_i$, base prediction $\\hat{y}_i$, and reasoning-enhanced prediction $\\hat{y}^{\\mathrm{reas}}_i$ to a binary reliability flag $r_i \\in \\{0,1\\}$; it is what partitions the dataset into $D_{\\mathrm{clean}}$ and $D_{\\mathrm{noise}}$. A second mechanism is the entropy-based selection of Eq. (8), which computes the per-token negative log-likelihood of context-enhanced responses and keeps only the top-$\\beta$ most confident repaired samples. Supporting machinery includes the reasoning-enhanced LLM that alternates reasoning and reflection, and the Review Agent that synthesizes context-enhanced and reasoning-enhanced candidate responses into a repaired label. Together these components define the pipeline that converts a noisy task dataset into the filtered fine-tuning set $D_{\\mathrm{ft}}$.","core_discovery":"The paper's central claim is that noisy response data, the kind that inevitably accumulates in real-world SFT collections, can be corrected through a fully self-contained detection-and-denoising loop, without external clean data. The detector is a consistency check: a sample is labeled clean only if the original answer, the base LLM's direct prediction, and the prediction from an iterative reasoning-reflection LLM all agree; any disagreement routes the sample to a denoising stage. Denoising first retrieves the most similar clean samples as context, generates a context-enhanced candidate, and then has a Review Agent synthesize a repaired label from that candidate and the reasoning-enhanced prediction. Finally, response entropy ranks the repaired samples and only the most confident fraction is kept. The reported result is that this pipeline outperforms vanilla models, standard SFT, and several noisy-label baselines across MMLU, ARC, PubMedQA, Drop, and FPB under 30–70% injected noise on Llama-3.2-3B, Llama-3.1-8B, and Gemma2-9B.","pith_inferences":["(Editorial) The Checker's agreement rule is never validated against ground-truth noise, so a natural test is to inject known noise, run only the detection stage, and measure how often the clean/noisy split matches the injected labels; that would separate detection quality from relabeling quality.","(Editorial) The entropy gate keeps only the most confident half of repaired samples, indicating a deliberate trade-off of data quantity for confidence; the same gate could be reused as a generic quality filter for self-training or preference-tuning datasets.","(Editorial) Because the final model is trained partly on the base model's own predictions, some of the gain over vanilla could come from self-training regularization rather than from correcting the specific noisy answers; a variant that relabels all samples without the Checker partition would expose how much of the gain is detection-driven.","(Editorial) The cross-model results suggest data repair matters more than scale for noisy SFT, a hypothesis that could be stress-tested by running RobustFT on a broader family of base models and on natural, rather than synthetically injected, noise."],"forward_implications":["Fine-tuning can tolerate noisy collection pipelines: at 70% injected noise, RobustFT-trained Llama-3.1-8B reaches 67.6 on MMLU and 84.1 on ARC, above the vanilla model's 65.3 and 82.7.","No external clean dataset is needed; the framework is self-contained, so it applies wherever only noisy downstream data is available.","Smaller models gain the most from denoising: Llama-3.2-3B moves from 38.3 to 57.9 on MMLU at 70% noise, closing much of the gap to larger models.","Domain-specific fine-tuning becomes viable on noisy data, including finance (FPB) and biomedical QA (PubMedQA), where base models often lack strong priors.","The ablations imply all three stages—consistency detection, context-enhanced relabeling, and entropy selection—are needed for the full gain; removing any one lowers accuracy."],"supporting_citations":[{"why":"MMLU is the primary benchmark; its degradation numbers motivate the entire problem.","marker":"Hendrycks et al., 2020"},{"why":"LoRA is the fine-tuning method used to train every model in the experiments.","marker":"Hu et al., 2021"},{"why":"NoiseAL is the strongest noisy-label learning baseline RobustFT is compared against and must outperform.","marker":"Yuan et al., 2024"},{"why":"Llama-3.1-8B and Llama-3.2-3B are two of the three base LLMs used across all main results.","marker":"Dubey et al., 2024"},{"why":"Gemma2-9B is the third backbone and the basis for the cross-architecture comparison.","marker":"Team et al., 2024"},{"why":"SelfRAG is the retrieval-augmented inference baseline that RobustFT's context-enhanced relabeling is compared with.","marker":"Lewis et al., 2020"},{"why":"ARC supplies the reasoning benchmark used in the ablation study and in the main accuracy comparisons.","marker":"Clark et al., 2018"},{"why":"This prior noise-robust fine-tuning method represents the discrete-label denoising approaches the paper argues are inadequate for open-ended generation.","marker":"Wang et al., 2023a"}],"fun_headline_variants":["Consensus check fixes noisy SFT data, beats baselines at 70% noise","Noise-proof SFT: multi-expert consensus and relabeling wins","70% noisy responses? RobustFT still fine-tunes better","Consensus-based denoising makes SFT robust to noisy labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that agreement among the original label, the base model's direct answer, and its reasoning-refined answer reliably separates clean from noisy samples, but the paper never reports how often that consensus judgment is actually correct.","fun_headline_variants_meta":{"raw":{"variants":["Consensus check fixes noisy SFT data, beats baselines at 70% noise","Noise-proof SFT: multi-expert consensus and relabeling wins","70% noisy responses? RobustFT still fine-tunes better","Consensus-based denoising makes SFT robust to noisy labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0011,"raw_usage":{"total_tokens":4590,"prompt_tokens":950,"completion_tokens":3640,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":3560}},"tokens_in":566,"tokens_out":3640,"duration_ms":21641,"temperature":1.0,"reasoning_tokens":3560,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:47:07.647590+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inject noise with known ground truth into the five datasets, run only the Checker, and compare its clean/noisy flags with the true injected noise bit; if the Checker misses most noisy samples or flags many clean ones, the reported fine-tuning gains cannot be attributed to accurate noise detection.","supporting_citations":[],"review_version":1}