{"id":"401438db-7296-42e3-a32a-1af73f026dc5","arxiv_id":"2502.06806","paper_version":4,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Plugin trains a small autoregressive model to multiply its softmax into a frozen LLM's softmax, adapting the output distribution to a target domain using only logits and limited data.","lead":"The paper proposes adapting closed-source large language models by reweighting their token probabilities with a small reweighting model trained on task-specific data, using only logits. It argues that logit access would unlock domain-specific generation without model weights, and presents experiments on four datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theory in §3 and the algorithm in Eq (4) are mathematically different: a row-stochastic diagonal transition matrix is the identity, so the label-noise derivation cannot support the product reweighting method as claimed.","rationale":"The paper's strongest claim—that logit reweighting alone is theoretically sufficient for task adaptation (Abstract, §1)—requires the method in §4 to be the estimator corresponding to the label-noise correction derived in §3. It is not. Section 3 derives a linear forward correction, T_t^T b_t, for a row-stochastic T_t. Section 3.1 restricts T_t to diagonal, which forces T_t = I and makes the correction vacuous; the accompanying description of 'class-independent noise' is also inconsistent with a diagonal matrix. The actual algorithm (Eq 4) multiplies probabilities elementwise and renormalizes, an operation that cannot be obtained from any non-identity row-stochastic transition matrix. No lemma connects this product reweighting to estimating T_t, and Theorem 1 is stated entirely in terms of a squared-loss surrogate for a parameter θ* in a generic transition function, without reference to T_t, b_t, or p_i. Thus the theoretical guarantee does not bear on the implemented objective. The empirical comparisons are extensive and the wrapper experiment is informative, but they cannot substitute for the missing link between the theory and the method. Secondary issues—the placeholder 'this https URL' for code and the apparent copied GPT2-XL zero-shot row between Table 2 and Table 3—reinforce the impression of an unpolished manuscript, though they are not the basis of this verdict. The load-bearing concern is the theory–algorithm gap, which is precisely the reader's weakest assumption. I agree with that identification and do not recommend changing the REJECT verdict; a revised paper could plausibly reframe Plugin as a heuristic density-ratio reweighting method, but the current central claim is unsupported.","tokens_in":40053,"tokens_out":6100,"duration_ms":64741,"concrete_test":"Analytically settle the derivation gap on a 3-token example. Let the base distribution be b = (0.6, 0.3, 0.1) and the target distribution be p = (0.1, 0.2, 0.7). Under the paper's diagonal restriction, any row-stochastic T is the identity, so the forward correction in Eq (3) gives T^T b = b, which cannot move toward p. In contrast, the Plugin update (Eq 4) can reach p by setting r ∝ p/b, but this is a density-ratio reweighting and is not expressible as T^T b for any row-stochastic T. This two-step check confirms that Eq (4) is not a special case of Eq (3), so Theorem 1, which bounds θ* estimation under a squared-loss surrogate, does not apply to the actual algorithm.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that training the Plugin reweighting network r_t by cross-entropy on the normalized product p_i = b_i ⊙ r_i / ||b_i ⊙ r_i||_1 (Eq 4) is equivalent to estimating and correcting a label-noise transition matrix T_t. Section 3 derives a forward correction using a row-stochastic T_t: Eq (3) replaces b with T_t^T b. Section 3.1 then restricts T_t to be diagonal. But a row-stochastic diagonal matrix is the identity, so T_t^T b = b and the correction is vacuous; the text also misdescribes a diagonal matrix as 'class-independent noise,' which would require off-diagonal mass. The implemented operation (Eq 4) is a normalized elementwise product, which is not of the form T_t^T b for any non-identity row-stochastic T_t. The paper never proves that minimizing cross-entropy on Eq (4) estimates T_t or applies its inverse. Theorem 1, moreover, analyzes estimation of θ* under Assumption 5.2, where the loss is upper bounded by a squared loss; its statement never references T_t, b_t, or p_i. Even a correct Theorem 1 would therefore not imply that Plugin's output distribution matches the target. This is not a presentation issue: it removes the theoretical support for 'reweighting logits alone is sufficient,' leaving the method as a heuristic density-ratio reweighting whose empirical gains remain plausible but unbacked by the claimed guarantee.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Plugin, an inference-time adaptation method for closed-source LLMs that have logit access. A small autoregressive reweighting network r_t is trained on task-specific data, and generation uses the normalized elementwise product p_i = b_i ⊙ r_i / ||b_i ⊙ r_i||_1 of the base model's softmax b_i and the reweighting output r_i. The paper frames this as a label-noise correction problem: the base model is treated as a noisy label predictor, a transition matrix T_t is introduced, and the authors claim that estimating T_t and reweighting logits alone is sufficient for task adaptation. The theoretical section gives a finite-time bound for an M-estimator of a parameter θ* under a squared-loss surrogate, and the experiments compare Plugin with zero-shot prompting, in-context learning, a newly trained model, weighted combination, and TempNet on four datasets and three base models, with additional ablations, a wrapper experiment, human evaluation, and a LoRA comparison.","tokens_in":40345,"tokens_out":7566,"duration_ms":76769,"significance":"If the theoretical claim were correct, the paper would make a practically valuable contribution: logit access plus a small task set would allow closed-model adaptation without weights, gradients, or training data. The empirical package is a genuine strength: four datasets, three base models, seven metrics, five seeds, ablations on base-model quality and reweighting architecture, a distribution-shift analysis, a human evaluation, and a comparison with LoRA, with code availability stated. Unfortunately, the central theoretical bridge between the label-noise derivation and the implemented update in Eq. (4) is missing, so the paper currently establishes only that a product-of-softmax reweighting heuristic performs well in the studied settings; it does not establish the promised label-noise consistency guarantee. The claimed theoretical novelty is the main advertised contribution, and it is disconnected from the algorithm actually evaluated.","major_comments":[{"comment":"The derivation from the forward-corrected loss to the implemented reweighting is invalid. The matrix T_t in Eq. (3) is row-stochastic; restricting it to be diagonal forces T_t = I, so Eq. (3) reduces to the uncorrected loss. The text's statement that a diagonal transition matrix models \"class-independent label noise\" is also incorrect, because class-independent noise has nonzero off-diagonal probabilities. The normalized elementwise product p_i = b_i ⊙ r_i / ||b_i ⊙ r_i||_1 is nonlinear in b_i and is not of the form T_t^T b_i for any row-stochastic T_t; no argument is given that minimizing cross-entropy on Eq. (4) estimates T_t or applies its inverse. This missing step is load-bearing for the abstract's claim that reweighting logits alone is sufficient.","section":"Sec. 3.1, Eq. (4)"},{"comment":"The theorem analyzes a different objective from the one used by Plugin. Assumption 5.2 asserts that the clipped cross-entropy loss is bounded by C|V|^2 (Y_t − f_I(θ*))^2, and the text then explicitly switches to \"this squared loss\" as ℓ_t(θ). Theorem 1 is therefore a finite-time M-estimation bound for squared-loss minimization; its statement never refers to b_t, r_t, p_t, or T_t, and it does not bound any divergence between Plugin's output distribution and the target distribution. An upper bound on cross-entropy by a squared loss does not imply that minimizing the squared loss minimizes the cross-entropy actually used in Eq. (5).","section":"Sec. 5, Theorem 1 and Assumption 5.2"},{"comment":"The consistency claim is largely assumed rather than derived. The theorem's assumptions already include the true parameter θ*, the transition function f_I, and a squared-loss bound; under those assumptions the proof is a standard M-estimation convergence argument. It does not establish that Plugin's autoregressive reweighting estimates the transition matrix of a label-noise process, because the mapping from the reweighting network r_t to the transition function f_I is never specified. The sentence after the theorem, claiming that accurate θ* estimation implies accurate estimation of T_t(θ*|F^{t−1}), therefore has no supporting statement in the theorem.","section":"Sec. 5, Theorem 1"}],"minor_comments":[{"comment":"The GPT2-XL zero-shot, ICL-1, and ICL-3 rows in Table 3 appear identical to the corresponding GPT2-XL entries in Table 2; this looks like a copy-paste error and should be corrected or explained.","section":"Table 3"},{"comment":"The GPT2-M ICL-1 row reports BLEU 0.0088 with standard deviation 0.054, which is not plausible for five seeds; several other ICL standard deviations in the same table appear to be an order of magnitude too large.","section":"Table 4"},{"comment":"The code link appears as the placeholder \"this https URL\" rather than an actual URL; the final version should include the link.","section":"Abstract"},{"comment":"The GPT-4o evaluation reports only aggregate percentages without specifying the number of generated samples, decoding parameters, or prompt repetitions; please add protocol details so the 17.99% versus 71.34% claim is reproducible.","section":"Sec. 7.4"},{"comment":"The index j is overloaded in Eq. (5): it is used both as a vocabulary coordinate and as the actual token position; please clarify that e_j denotes the one-hot vector of the token at position i.","section":"Algorithm 1 / Eq. (5)"}],"recommendation":"reject","confidential_remarks":"The empirical study is solid and the method may well be useful as a reweighting heuristic, but the theoretical framing is central to the abstract and to the contributions list. Because the label-noise derivation cannot produce the diagonal elementwise-product update in Eq. (4), and because Theorem 1 analyzes a squared-loss objective that the algorithm does not use, the promised theoretical guarantee is not established. I do not see a path to repair this within the scope of a revision; the authors would need either a genuinely new derivation connecting Eq. (4) to transition-matrix estimation or a repositioning of the paper as an empirical study without the consistency theorem."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Name],\n\nQuick take: the empirical package is worth a look; the theory is not. The method—train a small autoregressive rewriter on a few hundred target examples and multiply its softmax into the frozen base model's softmax—is simple and the experiments are unusually thorough. Gains over WeightedComb, TempNet, ICL, and even a white-box LoRA baseline are consistent across four datasets and three base models, and the BDPL wrapper result is a nice practical data point. The distribution-shift analysis with fine-tuned base models is a good idea, even if the GPT-4o eval is rough. Practitioners who want logit-only adaptation should read the experimental sections.\n\nThe soft spots are real, though. Section 3 connects the method to label-noise correction via a row-stochastic transition matrix T_t, and Eq (3) is the standard forward correction T^T b. Then Section 3.1 restricts T_t to be diagonal. A row-stochastic diagonal matrix is the identity, so that correction is vacuous—it cannot produce the upweighting in Eq (4). The implemented operation is a normalized elementwise product, which is not T^T b for any non-identity row-stochastic T. The paper never proves the converse or connects the diagonal restriction back to Eq (4). Theorem 1 is a finite-time M-estimation bound for a squared-loss surrogate under Assumption 5.2; its statement never mentions T_t, b_t, or the product distribution, so even if the theorem is correct, it does not establish that Plugin matches the target distribution. This is not a cosmetic gap; the central theoretical claim is unsupported as written.\n\nThere is also a minor but definite tabulation error: Table 3's GPT2-XL zero-shot and ICL rows are identical to Web NLG's rows in Table 2, which suggests a copy-paste mistake. The code URL is a placeholder, so reproducibility claims can't be checked. Citations look fine; DExperts and Patrini are the right anchors.\n\nNet: treat this as an empirical paper with a flawed theoretical framing. The method is well-tested and the practical message—logit access enables useful adaptation—survives. A revised version that either fixes the theory or drops the label-noise claim would be worth citing. I'd send it to review, but I'd insist the theory be rewritten or removed before publication.\n\nWarmly, [Your name]","headline":"Genuinely useful empirical recipe, but the theory section doesn't match the algorithm and should not be taken at face value.","tokens_in":40923,"tokens_out":3806,"would_cite":false,"duration_ms":32609,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that aligning a black-box LLM with task-specific data reduces to label noise correction, and that reweighting logits alone is sufficient for adaptation.","keywords":["black-box LLM adaptation","token-level probability reweighting","logit access","label noise correction","autoregressive transition matrix","closed-source models","distribution shift","Plugin model"],"falsifier":"On a synthetic corpus where token A is systematically confused with token B in the base model, Plugin can only up-weight or down-weight A and B separately rather than swap their probabilities; if the target distribution requires that swap, Plugin's output cannot match it, and the diagonal assumption is directly falsified.","tokens_in":1818,"feed_emoji":"🎯","tokens_out":2688,"duration_ms":94440,"temperature":0.7,"pith_summary":"The paper aims to show that a closed large language model can be adapted to a task-specific distribution using token logits plus a small corpus of target examples, without touching weights or gradients. It reinterprets next-token prediction as classification under label noise: the closed model's broad-corpus probabilities act as noisy labels, and the target data act as clean labels. The proposed Plugin model is a small autoregressive reweighting network trained so that the final token distribution is the normalized element-wise product of the base model's probabilities and the reweighting vector; the paper claims this is sufficient to align generation with the task. The main theoretical result is a finite-time consistency bound showing that the estimated autoregressive transition matrix converges to its true value, and experiments across four datasets and three base models report consistent gains over prompting and logit-scaling baselines. If the claim is right, developers would not need fine-tuning APIs or exposed weights to customize closed models, only logit access.","feed_headline":"Logits alone can steer a closed LLM to a new task","feed_subtitle":"Rather than prompt-tuning, the paper trains a small reweighting network on the model's token probabilities, with no weight access","key_machinery":"The machinery is the Plugin reweighting equation and its training objective. A small autoregressive model produces a per-token probability vector $r_t$; the combined distribution is the normalized element-wise product $p_t = b_t \\odot r_t / \\|b_t \\odot r_t\\|_1$, where $b_t$ is the frozen black-box model's softmax. The reweighting network is trained by sequence-level cross-entropy on target data with only its own parameters updated, and the theory treats $r_t$ as the diagonal of a class-conditional label-noise transition matrix $T_t$ that maps noisy (base-model) labels back to clean (target) labels. Theorem 1 gives the finite-time concentration argument that this estimated transition converges, which is the formal load-bearing step of the paper.","core_discovery":"On the paper's own terms, the central discovery is that reweighting logits alone is sufficient for task adaptation. The paper casts the black-box model's per-token distribution as the noisy version of a target distribution and proposes to estimate an autoregressive transition vector from task-specific data. At inference, the Plugin model combines the frozen base probabilities $b_t$ with the reweighting model's output $r_t$ as $p_t = b_t \\odot r_t / \\|b_t \\odot r_t\\|_1$ and greedily decodes from $p_t$. Theorem 1 is stated as a high-probability bound on $L_t(\\hat{\\theta}_t) - L_t(\\theta^*)$, showing that as the number of training tokens grows, the reweighting model's parameter estimate approaches the parameter governing the true autoregressive transition; this is the paper's basis for claiming the Plugin distribution aligns with the target. The empirical section supports the claim with improvements across BLEU, ROUGE, METEOR, and CIDEr on E2E NLG, Web NLG, CommonGen, and an Adidas product-description dataset, including settings where the base model has been biased by a distribution-shifted training set.","pith_inferences":["A consequence the paper leaves implicit is that logit access could become a practical middle ground for closed-model deployment: vendors would expose an inference-only probability endpoint while keeping weights private, and the same reweighting recipe would transfer across tasks.","The diagonal restriction implies Plugin can only scale each token's probability, not move probability mass between tokens; tasks that require restructuring syntax or executing multi-step reasoning are therefore a likely ceiling, which matches the paper's own limitation statement.","A testable extension is to truncate the logit support to the top-k tokens and rerun Plugin: the degradation curve as k shrinks would quantify exactly how much access to full logits buys, and would tell vendors whether top-k APIs suffice.","One can also test the equivalence at the heart of the method by using a synthetic vocabulary with a known off-diagonal noise matrix; if the normalized elementwise product cannot represent that inverse transition, the consistency theorem's assumptions are not satisfied by the deployed algorithm."],"forward_implications":["A developer with logit access and a small amount of task-specific text can adapt a closed model by training only a small reweighting network, with no weights, gradients, or vendor training data.","Plugin can wrap any prompt-based method: applying it on top of black-box discrete prompt learning improves that baseline on every dataset tested in the paper.","The reported gap between Plugin and a white-box fine-tuning method is small on several metrics, suggesting that logit reweighting recovers most of the observable task adaptation in low-data conditions.","Initializing the reweighting network with a small pretrained language model improves results, and a single transformer layer is reported to be near-optimal, so the overhead beyond the base model is modest in the paper's settings.","In distribution-shift evaluations, Plugin shifts generation toward the held-out target concept, such as raising Person-related Web NLG sentences from 17.99% to 71.34% and Woman-related CommonGen sentences from 10.37% to 31.92%."],"supporting_citations":[{"why":"Supplies the class-conditional label-noise transition matrix and the forward loss-correction result that the paper extends to autoregressive next-token prediction.","marker":"Patrini et al., 2017"},{"why":"Provides the earlier noisy-label training-with-corruption-matrix result that motivates the loss-correction step in Section 3.","marker":"Sukhbaatar et al., 2015"},{"why":"Offers the label-shift-with-black-box-predictors perspective that the paper uses to frame distribution shift as label noise.","marker":"Lipton et al., 2018"},{"why":"Gives the single-pass empirical-risk-minimization convergence technique whose martingale argument the paper adapts to its autoregressive losses.","marker":"Frostig et al., 2015"},{"why":"Supplies the active regression concentration lemmas that the proof of Theorem 1 builds on.","marker":"Mukherjee et al., 2022"},{"why":"Provides the martingale tail inequality used to derive the high-probability concentration bounds in the proof.","marker":"Hsu et al., 2012"},{"why":"The black-box discrete prompt learning baseline that Plugin is shown to improve on in the wrapper experiment.","marker":"Diao et al., 2022"},{"why":"The recent temperature-scaling baseline that Plugin is compared against in the generation experiments.","marker":"Qiu et al., 2024"},{"why":"The probability-combining WeightedComb/DExperts baseline that the paper modifies and compares with.","marker":"Liu et al., 2021"}],"fun_headline_variants":["Logit reweighting alone adapts closed LLMs to new tasks","Token probability tweaks steer closed LLMs, no weight access","Closed LLM adaptation via pure logit reweighting","Reweight logits to repurpose closed LLMs, no prompts needed"],"cache_read_input_tokens":42880,"weakest_assumption_plain":"The load-bearing premise is that a small network trained to reweight each token's probability independently is actually estimating the label-noise transition that separates the closed model's distribution from the target distribution; the paper asserts this equivalence when it restricts the transition to diagonal form but does not prove the converse.","fun_headline_variants_meta":{"raw":{"variants":["Logit reweighting alone adapts closed LLMs to new tasks","Token probability tweaks steer closed LLMs, no weight access","Closed LLM adaptation via pure logit reweighting","Reweight logits to repurpose closed LLMs, no prompts needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000509,"raw_usage":{"total_tokens":2498,"prompt_tokens":981,"completion_tokens":1517,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":1443}},"tokens_in":597,"tokens_out":1517,"duration_ms":10167,"temperature":1.0,"reasoning_tokens":1443,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T14:11:32.976586+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a synthetic corpus where token A is systematically confused with token B in the base model, Plugin can only up-weight or down-weight A and B separately rather than swap their probabilities; if the target distribution requires that swap, Plugin's output cannot match it, and the diagonal assumption is directly falsified.","supporting_citations":[{"cited_title":"Training convolutional networks with noisy labels","cited_arxiv_id":null,"evidence_quote":"Provides the earlier noisy-label training-with-corruption-matrix result that motivates the loss-correction step in Section 3."},{"cited_title":"M., and Sidford, A","cited_arxiv_id":null,"evidence_quote":"Gives the single-pass empirical-risk-minimization convergence technique whose martingale argument the paper adapts to its autoregressive losses."},{"cited_title":"S., and Nowak, R","cited_arxiv_id":null,"evidence_quote":"Supplies the active regression concentration lemmas that the proof of Theorem 1 builds on."},{"cited_title":"A tail inequality for quadratic forms of subgaussian random vectors","cited_arxiv_id":null,"evidence_quote":"Provides the martingale tail inequality used to derive the high-probability concentration bounds in the proof."},{"cited_title":"To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO","cited_arxiv_id":"2404.04575","evidence_quote":"The recent temperature-scaling baseline that Plugin is compared against in the generation experiments."},{"cited_title":"A., and Choi, Y","cited_arxiv_id":null,"evidence_quote":"The probability-combining WeightedComb/DExperts baseline that the paper modifies and compares with."}],"review_version":1}