{"id":"1e8cd058-09c1-47ea-b236-5f0d356cc44d","arxiv_id":"2508.09016","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey that catalogs and categorizes training-free LLM alignment methods into pre-decoding, in-decoding, and post-decoding, with a limited experimental comparison on one model.","lead":"This paper is a survey of 'training-free' alignment methods for large language models, which change prompts, decoding, or generated text instead of fine-tuning weights. It organizes dozens of existing techniques into three stages and adds a small experiment comparing a few methods on one model.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's 'FT' comparator SafeDecoding is itself a decoding-time, non-fine-tuning method, so the headline TF-vs-FT comparison lacks a genuine fine-tuned baseline.","rationale":"I read the paper as a survey whose central empirical pivot is the quantitative comparison in Section 3.4 and Table 1. The abstract makes a strong comparative claim, and the authors explicitly invoke the table as evidence. The weakest point is not merely the small scale or the single model; it is the identity of the comparator. SafeDecoding is not a fine-tuning method—it is an inference-time decoding adjustment. Therefore Table 1 cannot distinguish TF from FT, and the claim that TF matches or exceeds FT is not established by the reported experiment. This is an internal inconsistency with the cited work, not merely a disagreement with the field's consensus. The taxonomy and literature organization are useful, and a corrected comparison with an actual fine-tuned method might support a weaker version of the claim. However, as written, the central claim rests on a mislabeled baseline, so I recommend rejecting the current version while noting that a revision replacing the baseline and qualifying the claim could be acceptable.","tokens_in":19554,"tokens_out":3676,"duration_ms":43211,"concrete_test":"Inspect SafeDecoding's official code and original paper to confirm whether any model parameters are updated during alignment. Then re-run Table 1 on llama2-7b-chat with at least one genuine parameter-updating baseline (e.g., SFT or DPO trained from llama2-7b) using the same Wang et al. (2024a) safety classifier and refusal-keyword metric. If SafeDecoding involves no training, or if the genuine FT baseline changes the ranking, the headline comparison is unsupported as written.","verdict_should_be":"REJECT","load_bearing_attack":"Section 3.4 (Table 1) labels SafeDecoding (Xu et al., 2024b) as 'FT Alignment' and uses it as the sole fine-tuning representative to support the abstract's claim that TF alignment methods 'can match or even exceed the safety and helpfulness performance of FT alignment methods.' SafeDecoding is a safety-aware decoding method: it modifies token probabilities at inference time using a safety-critical token set and an expert model, without updating any model parameters. It is a training-free (or at least non-fine-tuning) method. Thus Table 1 does not compare TF methods against a fine-tuned baseline; it compares three TF methods against a fourth TF method mislabeled as FT. Moreover, the base model llama2-7b-chat is itself already RLHF-fine-tuned, so even the 'Defaults' row is an FT-aligned model. This mislabeling is load-bearing because every explicit statement that TF matches/exceeds FT—abstract, §2.2, §3.4, §A.2—rests on this single table. The finding may still be true, but Table 1 as reported cannot support it. The reader's identified weakness (single model, three benchmarks) is secondary; even expanding experiments would not fix the absence of a genuine FT comparator.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper surveys training-free (TF) alignment methods for large language models and multimodal large language models. It proposes a taxonomy of pre-decoding, in-decoding, and post-decoding interventions, reviews representative methods in each stage, discusses their mechanisms and limitations, and reports a small comparative experiment on llama2-7b-chat that is intended to show that TF alignment can match or exceed fine-tuning (FT) alignment in safety and helpfulness. The paper concludes with open challenges and future research directions, including general-capability preservation, inference overhead, generalization, controllability, and multimodal output alignment.","tokens_in":19888,"tokens_out":4266,"duration_ms":45815,"significance":"The survey addresses a timely and rapidly growing area and provides a useful organizing framework. The pre-/in-/post-decoding taxonomy is intuitive, and the coverage spans both unimodal LLMs and MLLMs, with a broad set of cited methods. The appendix comparison of methods along accessibility, storage, efficiency, and generalization dimensions (Table 2) is practical and valuable. If the quantitative claim were properly supported, it would be important for practitioners choosing alignment strategies under resource or access constraints. However, the current experimental support for the headline claim is not valid, and the survey's descriptive content is stronger than its comparative conclusion.","major_comments":[{"comment":"The 'FT Alignment' comparator SafeDecoding (Xu et al., 2024b) is not a fine-tuning method. Its published mechanism is safety-aware decoding: at inference time it uses an expert model to identify safety-critical token sets and adjusts token probabilities, without updating any model parameters. Labeling it 'FT Alignment' in Table 1 and using it as the sole fine-tuning representative invalidates the conclusion that TF methods 'can match or even exceed' FT methods. The table actually compares four TF/decoding-time methods. In addition, the 'Defaults' row is llama2-7b-chat, which is itself an RLHF-fine-tuned model, so no row represents a genuinely fine-tuned baseline versus an unfine-tuned one. The same unsupported inference is repeated in §2.2 and §A.2 ('FT alignment methods cause the most severe knowledge impairment'). This is load-bearing: the paper's headline comparative claim rests on th","section":"§3.4, Table 1; abstract; §2.2; §A.2"},{"comment":"Even setting the misclassification aside, the quantitative evidence is a single model (llama2-7b-chat), three benchmarks, no error bars or significance tests, and a safety classifier borrowed from the SafeEdit paper (Wang et al., 2024a). The Limitations acknowledge the single-model scope, but the abstract and §2.2 state the match/exceed conclusion without that caveat. Please either present Table 1 explicitly as an illustrative case study or add the necessary scope conditions and uncertainty quantification before making a general comparative claim.","section":"§3.4 and Limitations"},{"comment":"The survey's use of 'training-free' is broader than the phrase 'no training overheads' in §2.2. For instance, Aligner (Ji et al., 2024a) trains a separate correction model, and CA VGAN (Li et al., 2025c) trains a GAN on internal representations. These are listed as TF alignment because the target LLM is not fine-tuned. This definition is defensible, but the paper should state it explicitly; otherwise the abstract's 'without heavily retraining LLMs' and the §2.2 claim of 'no training overheads' are misleading. This is not fatal, but it affects the taxonomy's clarity.","section":"§3.3, Figure 1, §2.2"}],"minor_comments":[{"comment":"The sentence 'For FT alignment methods, we select SafeDecoding...' is contradicted by the cited paper's own description; this should be corrected regardless of the experimental outcome.","section":"§3.4"},{"comment":"The evaluation description says 'If responses to safety questions ... contain more refusal-related keywords', but TruthfulQA is described as 753 benign questions. Please adjust the wording to avoid implying that TruthfulQA contains safety questions.","section":"§A.1"},{"comment":"The subsection title 'TF Alignment for Uni-Modal Model' appears to be a typo; the content discusses extending alignment to models with multimodal output. Consider renaming to clarify the intended scope.","section":"§4.2"},{"comment":"The legend uses the symbol ' and %' which likely lost the checkmark/cross glyphs in typesetting. Please ensure the table symbols are rendered consistently and explained.","section":"Table 2"},{"comment":"The statement that 'there will not be any negative social impacts' is too absolute for a paper discussing safety methods; a more measured phrasing would be appropriate.","section":"Ethics Statement"}],"recommendation":"major_revision","confidential_remarks":"The skeptical concern about SafeDecoding is correct and directly affects the central comparative claim. The survey's taxonomic and descriptive content is worthwhile, but the headline 'TF matches/exceeds FT' statement should not stand without a genuine FT baseline. If the authors either add such a baseline or explicitly reframe Table 1 as an illustrative comparison among TF methods, the paper could become publishable. I would also encourage the editors to check whether the paper's 'first systematic review' claim is accurate relative to closely related concurrent surveys."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe stress-test note is correct, and it matters more than the single-model issue: Table 1 labels SafeDecoding as 'FT Alignment,' but SafeDecoding is a decoding-time method that modifies token probabilities without any parameter update. So the table actually compares four training-free (or at least non-fine-tuning) methods, and the 'Defaults' row is llama2-7b-chat, which is itself RLHF-fine-tuned. The abstract's central claim—that TF alignment can match or exceed FT alignment—is not supported by this table. That's a load-bearing flaw in the paper's top-line message, not a cosmetic one.\n\nThat said, the survey itself is worth engaging with. The three-stage taxonomy (pre-decoding, in-decoding, post-decoding) is a sensible and useful way to organize a scattered literature, and the per-stage discussion of mechanisms and limitations is informative. The appendix comparison (Table 2) gives practitioners a practical way to think about trade-offs. The authors do state in the Limitations that base models must be 'sufficiently capable' and that experiments are limited to one model, which is honest as far as it goes. But the mislabeling of SafeDecoding is the kind of error a careful referee should catch.\n\nOther soft spots are mostly minor. The 'training-free' label is stretched: some methods use a trained detector or a distilled guard model, so the category boundary is fuzzy. The evaluation of helpfulness relies on a Benign Refusing Rate keyword heuristic, which is a rough proxy. No error bars, one model, three datasets. None of these are fatal for a survey, but they matter because the headline comparison is presented as quantitative evidence.\n\nWho is this for? Someone new to inference-time alignment who wants a structured map and a curated reference list will get real value. A researcher working on safety alignment might also find the taxonomy a convenient starting point. The paper is not a rigorous empirical study, and it should not be cited as one.\n\nMy recommendation: send it to peer review, but require the authors to either add a genuine fine-tuned comparator (e.g., a model that actually underwent parameter updates for safety) or drop the 'match or exceed FT' claim from the abstract and Section 3.4, presenting Table 1 as an illustrative comparison of inference-time methods. Also ask them to tighten the definition of 'training-free.' With those changes, it becomes a solid reference survey.","headline":"Useful survey with a sensible taxonomy, but the headline 'TF beats FT' claim is unsupported because Table 1's FT baseline is itself training-free.","tokens_in":20318,"tokens_out":2275,"would_cite":true,"duration_ms":20809,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training-free alignment matches fine-tuned results, survey argues","keywords":["training-free alignment","large language models","in-context learning","decoding-time alignment","safe decoding","LLM safety","multimodal LLM alignment","survey"],"falsifier":"Run the same three training-free methods (URIAL, SCANS, RA-LLM) against SafeDecoding across several additional open-weight models such as Llama-3-8B and Mistral-7B, evaluating with human annotators and multiple safety classifiers; if fine-tuning wins on most tasks and models, the paper's central claim would fail.","tokens_in":1198,"feed_emoji":"🛡️","tokens_out":2108,"duration_ms":45771,"temperature":0.7,"pith_summary":"This survey argues that large language models can be aligned with human values without fine-tuning, by changing the prompt, steering the decoding process, or filtering the final output. It organizes the rapidly growing field of training-free alignment methods into three stages - pre-decoding, in-decoding, and post-decoding - and claims that such methods can match or even exceed the safety and helpfulness of fine-tuned alignment. If true, this matters because it would allow users of closed-source or resource-constrained models to align them cheaply, without retraining or parameter access. The paper also catalogs the main weaknesses of each stage, such as generalization limits and added latency, and proposes an agenda for making training-free alignment more robust and controllable.","feed_headline":"Training-free alignment matches fine-tuned results, survey argues","feed_subtitle":"Prompt tweaks, decoding steering, and output filtering rival fine-tuning on safety and helpfulness in benchmark tests.","key_machinery":"The organizing device is a three-stage taxonomy of the generation pipeline: pre-decoding interventions (prompt engineering, in-context examples, input detectors), in-decoding adjustments (modifying hidden states, computing logits differences, reward-guided search), and post-decoding refinements (self-examination, filtering, and correction of complete outputs). This taxonomy is the paper's framework for comparing methods, and the quantitative comparison in Table 1 grounds the claim that training-free alignment can match fine-tuning.","core_discovery":"The paper's central claim is that training-free alignment - methods that modify prompts, steer decoding, or filter outputs rather than updating weights - can match or even exceed the safety and helpfulness performance of fine-tuning. The evidence is a single quantitative comparison on llama2-7b-chat: SCANS, a decoding-time activation-steering method, outperforms the fine-tuned SafeDecoding on SafeEdit and TruthfulQA and matches it on AdvBench, while the post-decoding method RA-LLM matches or exceeds the fine-tuned baseline on safety metrics. The authors read this as evidence that training-free alignment is a viable supplement to fine-tuning, especially in closed-source, low-resource, or know","pith_inferences":["One consequence the authors leave implicit is that if training-free alignment genuinely matches fine-tuning on safety and helpfulness, the expensive fine-tuning step in alignment pipelines could shrink to a knowledge-plus-capability stage, with safety handled at inference time.","The three-stage taxonomy suggests that future work might treat alignment as a search problem over the intervention space - which stage, which steering strength, which filter - rather than as a fixed training recipe.","A testable extension would be to compose methods across stages, since the survey evaluates each stage in isolation; prompt-level defense plus decoding-time steering plus post-hoc filtering might outperform any single method.","The paper's reliance on a single safety classifier suggests results could shift if the evaluation metric changes, so a robustness check with multiple classifiers and human judgments would be a direct next step."],"forward_implications":["Users of closed-source or proprietary models can align them through prompts, decoding steering, and output filtering - no parameter access needed.","Because training-free methods avoid weight updates, they tend to preserve pretrained knowledge better than fine-tuning, which the paper shows through lower benign refusal rates on TruthfulQA.","The taxonomy implies alignment can be treated as a modular intervention at any pipeline stage, letting practitioners trade off safety, helpfulness, latency, and model-access requirements.","Pre-decoding and post-decoding methods work in black-box settings but face generalization limits and latency; in-decoding methods are stronger but require internal access - so no single training-free method fits all scenarios."],"supporting_citations":[{"why":"URIAL shows that three in-context examples and one system prompt can align a base LLM to a level comparable to fine-tuned models; the survey's central claim builds directly on this.","marker":"Lin et al. 2023"},{"why":"LIMA provides the empirical premise that alignment tuning is superficial, motivating the idea that training-free interventions may suffice.","marker":"Zhou et al. 2023"},{"why":"SafeDecoding is the fine-tuning-based baseline in the quantitative comparison used to argue training-free methods can match or exceed fine-tuning.","marker":"Xu et al. 2024b"},{"why":"SCANS is the in-decoding training-free method in the comparison and the source of the AdvBench and TruthfulQA evaluation setup.","marker":"Cao et al. 2025"},{"why":"RA-LLM is the post-decoding training-free method in the comparison, providing evidence for the post-decoding stage's competitiveness.","marker":"Cao et al. 2024"},{"why":"Supplies the SafeEdit dataset and the safety judgment classifier used to measure Defense Success Rate in the quantitative analysis.","marker":"Wang et al. 2024a"}],"fun_headline_variants":["Training-free alignment rivals fine-tuning, survey shows","Survey: prompt and decoding tweaks match fine-tuning","No fine-tuning needed? Survey says alignment can match","Training-free methods match fine-tuning for LLM alignment","Survey finds training-free alignment equals fine-tuning"],"cache_read_input_tokens":22272,"weakest_assumption_plain":"The central comparison rests on one model (llama2-7b-chat), three benchmarks, and one safety classifier, and the paper generalizes from that single table to the broad claim that training-free alignment can match or exceed fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["Training-free alignment rivals fine-tuning, survey shows","Survey: prompt and decoding tweaks match fine-tuning","No fine-tuning needed? Survey says alignment can match","Training-free methods match fine-tuning for LLM alignment","Survey finds training-free alignment equals fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1172,"prompt_tokens":725,"completion_tokens":447,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":388}},"tokens_in":469,"tokens_out":447,"duration_ms":5145,"temperature":1.0,"reasoning_tokens":388,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:15:03.216419+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same three training-free methods (URIAL, SCANS, RA-LLM) against SafeDecoding across several additional open-weight models such as Llama-3-8B and Mistral-7B, evaluating with human annotators and multiple safety classifiers; if fine-tuning wins on most tasks and models, the paper's central claim would fail.","supporting_citations":[],"review_version":1}