Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

SLOT: Sample-specific Language Model Optimization at Test-time

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Optimizing a tiny additive vector on the prompt's own text at test time improves LLM instruction following and reasoning accuracy, with reported gains of up to 8.6 points on GSM8K.

desk verdict A cheap test-time adaptation idea with a plausible mechanism, but the evidence is under-powered and the abstract's headline number doesn't match the tables. read the letter →

arxiv 2505.12392 v2 pith:LNTGKHUI submitted 2025-05-18 cs.CL cs.AIcs.LG

Yang Hu , Xingyu Zhang , Xueji Fang , Zhiyang Chen , Xiao Wang , Huatian Zhang , Guojun Qi This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords test-timeadaptationlanguagemodeloptimizationsample-specificparameterspromptcross-entropylosslogitmodulationreasoningbenchmarksparameter-efficientinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SLOT claims that a language model can be made more accurate on a single question by spending a few gradient steps at inference time on the question's own text, without retraining or fine-tuning. The adaptation is a small per-sample vector $\delta$ added to the final hidden layer before the output head, and the only supervision is the standard next-token cross-entropy loss on the prompt itself. The paper reports consistent gains across several open models and reasoning benchmarks, including an 8.6-point improvement for Qwen2.5-7B on GSM8K and a state-of-the-art 68.69% for DeepSeek-R1-Distill-Llama-70B on GPQA Diamond. If the link between prompt likelihood and answer correctness holds, this gives a cheap, universal way to sharpen instruction following.

What carries the argument

The load-bearing object is the sample-specific additive parameter $\delta\in\mathbb{R}^{1\times d}$, broadcast over the sequence and added to the cached last-layer hidden features before the LM head. Its effect is equivalent to an additive shift $W_{\mathrm{LM}}\delta$ on the vocabulary logits, named the Logit Modulation Vector. The two-stage scheme—optimize $\delta$ on the prompt for $T$ steps, then reuse the fixed $\delta$ during autoregressive generation—keeps the extra cost small, and zero initialization preserves the original model at step zero. The gradient path is short because only the final linear head participates in the update.

What would settle it

Run SLOT on prompts whose words are randomly shuffled but whose answers are unchanged; if accuracy still rises, the effect is not specific to understanding the instruction, and if prompt loss decreases while answer accuracy falls, the proxy is broken.

Watch

Extended reading notes

Core claim

The central claim is that optimizing a zero-initialized additive vector on the final hidden features, $H' = H + \delta$, with a few AdamW steps against the prompt's own cross-entropy loss, biases subsequent generation toward the intended answer. Because $\delta$ touches only the layer just before the language-model head, the per-sample optimization reuses cached features and costs only the head computation, so the method is nearly free at generation time. The paper interprets the resulting logit shift, $W_{\mathrm{LM}}\delta$, as a Logit Modulation Vector and observes that it raises tokens like 'think' and 'reasoning' while suppressing numbers and the end-of-text token, which the authors read as encouraging deeper reasoning before answering. Empirically, SLOT improves both answer accuracy and format compliance on GSM8K, AIME24, Math500, GPQA Diamond, C-Eval, and HumanEval across Qwen, Llama, and DeepSeek-R1 models.

Load-bearing premise

The method assumes that lowering the language-modeling loss on the prompt text itself makes the model more likely to produce the correct held-out answer, and this proxy link is empirical rather than derived.

Editorial extensions

If this is right

  • Any transformer with an exposed pre-softmax layer can carry SLOT as a drop-in inference wrapper; no training data, labels, or model access beyond the final hidden states is needed.
  • The overhead is bounded by the head dimension and prompt length, not by model size: the reported wall-clock increase is about 7.9% with five optimization steps on GSM8K.
  • Instructions with strict output formats should become easier to satisfy, since format accuracy rises alongside answer accuracy as $T$ increases.
  • The reported negative entries (e.g., Qwen2.5-14B on Math500 and DeepSeek-R1-Distill-Qwen-32B on GPQA Diamond) are part of the paper's own data, so the benefit is not guaranteed across all models and datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The prompt-loss proxy is likely to be least reliable when the prompt is already highly predictable or when the correct answer requires information the prompt does not contain; testing SLOT on trivia or open-ended generation would separate format alignment from factual gains.
  • The Logit Modulation Vector analysis suggests SLOT is effectively a learned decoding bias; a cheaper or more controllable variant might compute the bias analytically or tune it per task rather than per sample.
  • Negative results in Table 2 hint that the method can overfit to prompt surface form; comparing SLOT against prompt paraphrases or shuffled token orders would reveal whether it captures instruction semantics or just local token statistics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SLOT (Sample-specific Language Model Optimization at Test-time), a method that, for each input prompt, performs a few gradient steps to optimize a small per-sample vector delta that is added to the final hidden layer before the language-model head. The optimization minimizes the cross-entropy loss on the prompt tokens only, after which the optimized delta is reused during generation. The authors argue that this test-time adaptation better aligns the model with each individual instruction. They evaluate SLOT across several model families (Qwen, Llama, DeepSeek-R1-Distill) on benchmarks including GSM8K, AIME24, Math500, GPQA Diamond, HumanEval, and C-Eval, and report accuracy gains in many cases, with the largest claimed gains being an 8.6% improvement on GSM8K for Qwen2.5-7B and a 68.69% accuracy on GPQA Diamond for DeepSeek-R1-Distill-Llama-70B. They also provide an ablation over the number of optimization steps and learning rate, an inference-time cost analysis, a logit-modulation analysis, and a code snippet.

Significance. If the claimed effects are real, SLOT would be an appealingly simple and parameter-efficient test-time adaptation technique: it adds only a single vector of dimension d per sample, avoids backpropagation through the full model by caching last-layer features, and is reported to improve accuracy across diverse benchmarks and model sizes. The paper also has strengths in breadth (many model-benchmark pairs are reported) and in shipping a concrete algorithm description with a core implementation snippet. However, the significance is substantially constrained by the current evidence: the headline result does not appear in the main results table, no error bars or significance tests are provided, default hyperparameters are inconsistent with the paper's own ablation, and the core causal mechanism (prompt-loss minimization improves answer accuracy) is not validated beyond anecdotal examples. As presented, the paper is best viewed as an initial exploration rather than a fully supported claim of a new state-of-the-art test-time adaptation method.

major comments (5)
  1. [Section 1 and Table 1] The headline claim in the abstract and introduction of an 8.6% GSM8K gain for Qwen2.5-7B (57.54% to 66.19%) does not appear in Table 1, which reports Qwen-7B improving from 51.2% to 54.2% on GSM8K. The paper must either reconcile this discrepancy (e.g., by clarifying that Qwen-7B is a different model) or remove the abstract claim, because the central advertised result is not reproducible from the presented tables.
  2. [Section 4.6, Table 4] The main experiments use T=3 and eta=0.01, yet the hyperparameter ablation on AIME-24 shows that this configuration achieves only 26.67% accuracy, identical to the baseline, while T=4, eta=0.05 achieves 40.00%. The authors need to justify why they selected the default hyperparameters in Section 4.1 rather than the ablation-optimal ones, and to report results with the optimal settings; otherwise the reported gains may reflect an under-tuned configuration.
  3. [Section 3.2, Eq. (4) and Section 4.3] The central assumption that minimizing the prompt-only cross-entropy loss improves accuracy on the held-out answer is not validated. The analysis in Figure 3, which shows that reasoning-related tokens are increased and end-of-text is suppressed, is a direct consequence of optimizing the prompt loss on a template that contains the tokens 'think', 'answer', and '<|endoftext|>'; it does not provide independent evidence of deeper reasoning. A controlled test, such as correlating per-sample prompt-loss decrease with answer correctness, or ablating the objective to exclude format tokens, is needed to support the causal claim.
  4. [Tables 2 and 3; Checklist item 7] The paper reports numerous small positive and negative deltas (e.g., DeepSeek-R1-Distill-Qwen-32B Math500 -0.60 and GPQA -1.01, Llama-3.1-8B Math500 -0.60, Qwen2.5-Math-1.5B GPQA -0.51) without error bars, confidence intervals, or statistical significance tests, and the authors explicitly state in the checklist that such reporting is not provided. On small benchmarks like AIME24, where one question corresponds to several percentage points, these differences may be within sampling noise. The authors should report multiple runs with variance estimates and, where possible, significance tests so that the reader can assess which improvements are reliable.
  5. [Section 4.3] The experiments do not include any same-compute test-time baselines, such as continuous prompt tuning with a comparable number of gradient steps, LoRA-style adaptation at test time, or even simple logit-adjustment heuristics. Without such comparisons, the claim that SLOT is a competitive test-time technique relative to existing efficient adaptation approaches is not established; the current comparison is only against the unmodified base model.
minor comments (5)
  1. [Algorithm 1] The pseudocode in Algorithm 1 recomputes the hidden features H inside each optimization iteration, which contradicts the text in Section 3.3 and the appendix that state the features are cached and reused; the pseudocode should be updated to reflect the caching implementation.
  2. [Table 1] There is a typo in the table header: 'A VERAGE' should be 'AVERAGE'.
  3. [Abstract and Section 4.3] The claim that 68.69% on GPQA Diamond is 'SOTA' for 70B-level open-source models is not accompanied by a citation or a leaderboard reference; please provide the source of this comparison.
  4. [Appendix A] The implementation in Listing 1 uses an os.environ variable as a global flag to switch between prompt-stage and generation-stage behavior; this approach is fragile and not thread-safe, and the released code should use a cleaner mechanism if the authors intend the code to be widely used.
  5. [NeurIPS Checklist, item 2] The checklist states that the method has no obvious limitations, yet Table 2 contains several negative results and Table 4 shows sensitivity to hyperparameters; the authors should provide a limitations discussion that addresses these observations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the optimized δ is fitted to prompt cross-entropy, while all reported results are held-out answer accuracies on external benchmarks.

full rationale

SLOT's method is not circular. The sample-specific vector δ is optimized by Eq. (4) against the cross-entropy loss on the prompt tokens, and then applied at generation time via Eq. (2)-(3); the paper's headline quantities are held-out answer accuracies (Tables 1-2, GSM8K, AIME24, Math500, GPQA Diamond, HumanEval, C-Eval). There is no equation or reduction in which the predicted answer accuracy is constructed from Eq. (4), nor is any fitted parameter renamed as a prediction: the optimized δ is not fit to answer labels or to the evaluation metric. The LMV analysis (Eq. (6), Fig. 3) is a post-hoc reading of what the optimized logits do (increase 'think'/'reasoning' tokens, suppress EOS); it is a consequence of the optimization target and is presented as observation, not as independent evidence, so it does not create a definitional loop. The paper cites no prior work by the same authors and invokes no uniqueness theorem; therefore no self-citation chain is load-bearing. Remaining concerns are correctness/robustness, not circularity: the prompt-CE proxy is empirically assumed (Sec. 3.2), several entries in Table 2 are negative, the abstract's 8.6% GSM8K gain for Qwen2.5-7B does not reappear in Table 1's Qwen-7B row, the default (T=3, η=0.01) is not the best configuration in the paper's own Table 4 ablation, no error bars are given, and the NeurIPS checklist asserts 'no obvious limitations' without support. None of these make the central claim equivalent to its inputs, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The central claim rests on an empirical proxy (prompt loss improves answer accuracy) and a representational assumption (a single additive vector can steer generation). The hyperparameters T and eta are chosen by hand and are not the optimal ones according to the paper's own ablation. No new physical entities are introduced; LMV is a definitional tool.

free parameters (4)
  • T (optimization steps) = 3 (main experiments)
    Set to 3 in Section 4.1; ablation in Section 4.6 suggests T=4 or T=5 performs better on AIME-24 for one model.
  • learning rate eta = 0.01
    Set to 0.01 in Section 4.1; ablation shows eta=0.05 gives higher accuracy on AIME-24.
  • weight decay = 1e-8
    AdamW hyperparameter fixed in Section 4.1; not explored.
  • AdamW epsilon = 1e-5
    Fixed in Section 4.1; not explored.
assumptions (3)
  • domain assumption Prompt cross-entropy loss is a valid proxy for answer accuracy
    Section 3.2 optimizes delta to minimize Eq. (4) on the prompt; no analysis links this to answer correctness.
  • domain assumption A single additive vector to final hidden states can meaningfully steer generation
    Section 3.2 defines H'=H+delta; Figure 3 provides qualitative LMV evidence but no quantitative guarantee.
  • domain assumption The benchmark prompts are representative of prompts that benefit from adaptation
    Negative results in Table 2 indicate the benefits do not hold across all settings.
invented entities (1)
  • Logit Modulation Vector (LMV)
    purpose: Interpret the effect of delta as an additive logit shift
    Defined in Section 3.3 as W_LM * delta; it is a notational device with no external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SLOT: Sample-specific Language Model Optimization at Test-time." pith.science (2026). https://pith.science/paper/LNTGKHUI

@misc{pith2026250512392,
  author       = {Pith},
  title        = {Pith review of: SLOT: Sample-specific Language Model Optimization at Test-time},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LNTGKHUI}},
  note         = {Machine review of arXiv:2505.12392}
}
read the original abstract

We propose SLOT (Sample-specific Language Model Optimization at Test-time), a novel and parameter-efficient test-time inference approach that enhances a language model's ability to more accurately respond to individual prompts. Existing Large Language Models (LLMs) often struggle with complex instructions, leading to poor performances on those not well represented among general samples. To address this, SLOT conducts few optimization steps at test-time to update a light-weight sample-specific parameter vector. It is added to the final hidden layer before the output head, and enables efficient adaptation by caching the last layer features during per-sample optimization. By minimizing the cross-entropy loss on the input prompt only, SLOT helps the model better aligned with and follow each given instruction. In experiments, we demonstrate that our method outperforms the compared models across multiple benchmarks and LLMs. For example, Qwen2.5-7B with SLOT achieves an accuracy gain of 8.6% on GSM8K from 57.54% to 66.19%, while DeepSeek-R1-Distill-Llama-70B with SLOT achieves a SOTA accuracy of 68.69% on GPQA among 70B-level models. Our code is available at https://github.com/maple-research-lab/SLOT.

Figures

Figures reproduced from arXiv: 2505.12392 by the authors.

Figure 1
Figure 1. SLOT significantly boosts Qwen-2.5-7B’s GSM8K format and answer accuracy at test [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. SLOT pipeline during inference. The process consists of two stages: (1) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. About the most increased and decreased tokens in Qwen-2.5-7B on GSM8K with [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Samples of the comparison between with SLOT and without SLOT in GSM8K. We mark [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Self-Reflective Generation at Test Time

    cs.CL 2025-10 conditional novelty 5.0 of 10

    SRGen improves LLM math reasoning by detecting high-entropy tokens and injecting a small corrected vector into the hidden state at those points during decoding, without training.

  2. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

Reference graph

Works this paper leans on

42 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    The surprising effectiveness of test-time training for few-shot learning

    Ekin Aky ¨urek, Mehul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob Andreas. The surprising effectiveness of test-time training for few-shot learning. arXiv preprint arXiv:2411.07279, 2024

    arXiv 2024

  2. [2]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  3. [3]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    arXiv 2021

  4. [4]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  5. [5]

    Learning how hard to think: Input-adaptive allocation of lm computation.arXiv preprint arXiv:2410.04707, 2024

    Mehul Damani, Idan Shenfeld, Andi Peng, Andreea Bobu, and Jacob Andreas. Learning how hard to think: Input-adaptive allocation of lm computation.arXiv preprint arXiv:2410.04707, 2024

    arXiv 2024

  6. [6]

    Test-time training can close the natural distribution shift performance gap in deep learning based compressed sensing

    Mohammad Zalbagi Darestani, Jiayu Liu, and Reinhard Heckel. Test-time training can close the natural distribution shift performance gap in deep learning based compressed sensing. In International conference on machine learning, pages 4754–4776. PMLR, 2022

    work page 2022

  7. [7]

    Test-time training with masked autoencoders.Advances in Neural Information Processing Systems, 35:29374–29385, 2022

    Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros. Test-time training with masked autoencoders.Advances in Neural Information Processing Systems, 35:29374–29385, 2022

    2022

  8. [8]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

Show all 42 references
  1. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  2. [10]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  3. [11]

    Parameter-efficient transfer learning for nlp

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. InInternational conference on machine learning, pages 2790–2799. PMLR, 2019

    2019

  4. [12]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  5. [13]

    C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models.Advances in Neural Information Processing Systems, 36:62991–63010, 2023

    Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, et al. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models.Advances in Neural Information Processing Systems, 36:62...

    2023

  6. [14]

    Test-time com- puting: from system-1 thinking to system-2 thinking.arXiv preprint arXiv:2501.02497, 2025

    Yixin Ji, Juntao Li, Hai Ye, Kaixin Wu, Jia Xu, Linjian Mo, and Min Zhang. Test-time com- puting: from system-1 thinking to system-2 thinking.arXiv preprint arXiv:2501.02497, 2025. 10

    2025 arXiv

  7. [15]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Sym- posium on Operating System...

    2023

  8. [16]

    The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021

    2021 arXiv

  9. [17]

    Prefix-tuning: Optimizing continuous prompts for generation

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021

    2021 arXiv

  10. [18]

    A survey on llm test-time compute via search: Tasks, llm profiling, search algo- rithms, and relevant frameworks.arXiv preprint arXiv:2501.10069, 2025

    Xinzhe Li. A survey on llm test-time compute via search: Tasks, llm profiling, search algo- rithms, and relevant frameworks.arXiv preprint arXiv:2501.10069, 2025

    2025 arXiv

  11. [19]

    Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling

    Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling. arXiv preprint arXiv:2502.06703, 2025

    2025 arXiv

  12. [20]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks.arXiv preprint arXiv:2110.07602, 2021

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks.arXiv preprint arXiv:2110.07602, 2021

    2021 arXiv

  13. [21]

    Ttt++: When does self-supervised test-time training fail or thrive?Advances in Neural Information Processing Systems, 34:21808–21820, 2021

    Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Ttt++: When does self-supervised test-time training fail or thrive?Advances in Neural Information Processing Systems, 34:21808–21820, 2021

    2021

  14. [22]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  15. [23]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 1:3, 2020

    Ben Mann, N Ryder, M Subbiah, J Kaplan, P Dhariwal, A Neelakantan, P Shyam, G Sas- try, A Askell, S Agarwal, et al. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 1:3, 2020

    2005 arXiv

  16. [24]

    Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation.arXiv preprint arXiv:2410.02725, 2024

    Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation.arXiv preprint arXiv:2410.02725, 2024

    2024 arXiv

  17. [25]

    Rethinking the role of demonstrations: What makes in-context learning work?arXiv preprint arXiv:2202.12837, 2022

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work?arXiv preprint arXiv:2202.12837, 2022

    2022 arXiv

  18. [26]

    Improving black-box robustness with in-context rewriting

    Kyle O’Brien, Nathan Ng, Isha Puri, Jorge Mendez, Hamid Palangi, Yoon Kim, Marzyeh Ghassemi, and Thomas Hartvigsen. Improving black-box robustness with in-context rewriting. arXiv preprint arXiv:2402.08225, 2024

    2024 arXiv

  19. [27]

    Tttflow: Unsupervised test-time training with normalizing flow

    David Osowiechi, Gustavo A Vargas Hakim, Mehrdad Noori, Milad Cheraghalikhani, Ismail Ben Ayed, and Christian Desrosiers. Tttflow: Unsupervised test-time training with normalizing flow. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2...

    2023

  20. [28]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling, 2024

    2024

  21. [29]

    Towards real-world test-time adaptation: Tri-net self-training with balanced normalization

    Yongyi Su, Xun Xu, and Kui Jia. Towards real-world test-time adaptation: Tri-net self-training with balanced normalization. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15126–15135, 2024

    2024

  22. [30]

    Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  23. [31]

    Learning to (learn at test time).arXiv preprint arXiv:2310.13807, 2023

    Yu Sun, Xinhao Li, Karan Dalal, Chloe Hsu, Sanmi Koyejo, Carlos Guestrin, Xiaolong Wang, Tatsunori Hashimoto, and Xinlei Chen. Learning to (learn at test time).arXiv preprint arXiv:2310.13807, 2023. 11

    2023 arXiv

  24. [32]

    Test- time training with self-supervision for generalization under distribution shifts

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test- time training with self-supervision for generalization under distribution shifts. InInternational conference on machine learning, pages 9229–9248. PMLR, 2020

    2020

  25. [33]

    Tent: Fully test-time adaptation by entropy minimization.arXiv preprint arXiv:2006.10726, 2020

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization.arXiv preprint arXiv:2006.10726, 2020

    2006 arXiv

  26. [34]

    Self-consistency improves chain of thought reasoning in lan- guage models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in lan- guage models.arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  27. [35]

    Beyond model adaptation at test time: A survey.arXiv preprint arXiv:2411.03687, 2024

    Zehao Xiao and Cees GM Snoek. Beyond model adaptation at test time: A survey.arXiv preprint arXiv:2411.03687, 2024

    2024 arXiv

  28. [36]

    Stta: enhanced text classification via selective test-time augmentation.PeerJ Computer Science, 9:e1757, 2023

    Haoyu Xiong, Xinchun Zhang, Leixin Yang, Yu Xiang, and Yaping Zhang. Stta: enhanced text classification via selective test-time augmentation.PeerJ Computer Science, 9:e1757, 2023

    2023

  29. [37]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  30. [38]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement.arXiv preprint arXiv:2409.12122, 2024

    2024 arXiv

  31. [39]

    Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809–11822, 2023

    2023

  32. [40]

    Benchmarking reasoning robustness in large language models

    Tong Yu, Yongcheng Jing, Xikun Zhang, Wentao Jiang, Wenjie Wu, Yingjie Wang, Wenbin Hu, Bo Du, and Dacheng Tao. Benchmarking reasoning robustness in large language models. arXiv preprint arXiv:2503.04550, 2025

    2025 arXiv

  33. [41]

    On pitfalls of test-time adaptation

    Hao Zhao, Yuejiang Liu, Alexandre Alahi, and Tao Lin. On pitfalls of test-time adaptation. arXiv preprint arXiv:2306.03536, 2023

    2023 arXiv

  34. [42]

    p r o m p t _ o n l y

    Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084, 2025. 12 A Source Code Implementation We present the core part of our source code impl...

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.