{"id":"c92e6fe3-c337-4b62-b83e-093aaadc0c4c","arxiv_id":"2412.00054","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Task vectors can be reduced to binary masks, signs, and a scalar per task while preserving or improving merging performance at 1-3% storage.","lead":"This paper shows that task vectors, the difference between fine-tuned and pretrained weights, can be compressed to binary switches and still merge models effectively. The authors report matching or improving on full-precision merging while using only 1-3% of the storage.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'pulse-like' claim is confounded: P-Discard and Bin-Discard change the task-vector norm, and the controls do not separate magnitude selection from norm reduction.","rationale":"The reader identified the weakest assumption as the global quantile threshold and the sign-only directional approximation. My concern is more specific and precedes that assumption: the controlled experiments that establish the 'pulse-like' phenomenon do not hold the task-vector norm fixed, so the observed gains may be a norm-regularization artifact rather than evidence that low-magnitude coordinates are redundant. This matters because the entire method is motivated by the causal claim that only high-magnitude parameters contribute positively. If a random mask matched to the same norm reproduces the improvement, then magnitude-based selection is not the operative mechanism. However, the practical binary-switch proposal could still work even if the stated mechanism is wrong, and the empirical tables show strong relative performance. The absence of code and error bars already justifies a conditional verdict; the missing norm-matched control adds a concrete, checkable condition rather than grounds for rejection. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":16134,"tokens_out":8708,"duration_ms":87405,"concrete_test":"Re-run the §3.2 protocol at α=0.2,...,0.8 with two additional arms: (A) P-Discard rescaled so the fine-tuned model remains at the original distance from θ, i.e., λ=||τ_i||_2/||τ_i^α||_2; and (B) randomly discarding the same number of coordinates, then rescaling survivors to match P-Discard's norm. Apply the same test on both single-task performance and the Eq. 4 merging scheme. If (A) removes the reported gain or (B) reproduces it, the pulse-like causal claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanistic claim—that only above-threshold task-vector parameters contribute positively—rests on Fig. 3 and Table 1. But the 'Discard Low' arm uses P-Discard (Eq. 3) without rescaling, so its task vector has a smaller L2 norm than the original; DARE, the 'zero control', rescales by 1/(1−α) and keeps the original norm. Thus the comparison is not norm-matched: the improved fine-tuning performance could come from shrinking the delta toward the pretrained model (a regularization effect), not from removing low-magnitude coordinates. Bin-Discard (Eq. 5) scales to the P-Discarded norm, so it inherits the same confound. As a result, the paper has not shown that the surviving high-magnitude coordinates are the causal carriers of task information, which is the stated basis for both T-Switch and the storage-accuracy trade-off.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims that task vectors exhibit a pulse-like characteristic: only parameters whose magnitude exceeds a per-task quantile threshold contribute positively, and discarding the rest while binarizing the survivors to their signs times a scalar preserves or even improves fine-tuning and merging performance. This observation motivates T-Switch, a decomposition of each task vector into a binary activation mask, a binary sign vector, and a scalar scaling knob, and Auto-Switch, a training-free variant that weights these switches by nearest-neighbor retrieval on a small query set. The authors evaluate on eight vision tasks with CLIP ViT-B/32 and ViT-L/14, eight language tasks with RoBERTa, and LoRA fine-tuned settings, reporting state-of-the-art merging accuracy with storage of 1-3% of full-precision task vectors.","tokens_in":16332,"tokens_out":9011,"duration_ms":82318,"significance":"If the pulse-like property is real, it would provide a simple and powerful mechanistic account of task-vector redundancy and would make binary task-vector storage an attractive practical option for multi-task deployment. The paper's decomposition is conceptually clean, Auto-Switch is genuinely training-free, and the reported storage reductions are large and easy to verify from the given storage numbers. These are real strengths. However, the central mechanistic claim is not yet isolated from a norm-reduction confound, the main tables lack any uncertainty quantification, and T-Switch as defined in Eq. (7) does not actually combine task information, which undermines the framing as a merging method. The empirical results are suggestive but need additional controls and reframing before the paper's conclusions are supported.","major_comments":[{"comment":"The comparison between P-Discard and DARE is not norm-matched. P-Discard (Eq. 3) returns tau_i * g_m(tau_i) without rescaling, so its L2 norm shrinks as alpha grows, whereas the DARE control rescales the remaining entries by 1/(1-alpha) and approximately preserves the original task-vector norm. The observed advantage of 'Discard Low' over DARE, and the continued improvement as alpha increases, can therefore be explained by a shrinkage/regularization effect toward the pretrained weights rather than by the removal of low-magnitude coordinates. Bin-Discard (Eq. 5) inherits the same confound because it rescales to the P-Discarded norm. Please add norm-matched controls, such as random discarding with rescaling to exactly the P-Discarded norm and P-Discard rescaled back to the original norm, and report the per-alpha curves separately for magnitude selection and norm reduction before claiming the pulse-like mechanism.","section":"Section 3.2, Fig. 3, Table 1, Eq. (3)"},{"comment":"All reported numbers appear to be single runs, with no error bars, confidence intervals, or significance tests. Claims such as T-Switch outperforming EMR-Merging by 2.24 points on ViT-B/32 (Table 2) and by 0.0415 on RoBERTa (Table 4) are central to the conclusions and could easily lie within fine-tuning run-to-run variation. Please report means and standard deviations over multiple fine-tune/merge seeds for the main comparisons, and use a statistical test where feasible.","section":"Tables 2-4 and Table 6"},{"comment":"The main results fix the discard ratio at alpha=0.5 for all discarding methods, but this value is selected after sweeping alpha from 0.0 to 0.9 on the same evaluation benchmarks. The choice is therefore informed by the test set, and the reported improvements over baselines are not an independent evaluation at a pre-specified operating point. Please either select alpha on a held-out validation split or report the comparison to baselines across the full alpha range so that the sensitivity of the conclusions to this hyperparameter is visible.","section":"Section 4.1 and Fig. 7"},{"comment":"With U being a vector of all ones, Eq. (7) reduces to theta_i = theta + lambda_i * S_A^i * S_P^i, which is a binary decompression of a single task vector. No information from multiple tasks is combined, so T-Switch does not realize the multi-task merging objective in Eq. (1); given a task identity, it simply reconstructs an approximation of that task's fine-tuned model. The comparisons of T-Switch to static merging methods (Task-Arithmetic, TIES, DARE) and to dynamic merging methods in Tables 2-4 are therefore not comparisons of merging algorithms. Auto-Switch (Eq. 8) does combine switches across tasks; please reposition T-Switch as a per-task compression component or an ablation, or define a merging operation in Eq. (7) that actually combines task information.","section":"Section 3.4, Eq. (7)"}],"minor_comments":[{"comment":"There is a typo in the text: 'obervation' should be 'observation'.","section":"Section 3.2"},{"comment":"The relationship between gamma_u, gamma_l, and the discard ratio alpha is described only in prose; it would be clearer to define gamma_u and gamma_l directly as alpha-quantiles in Eq. (2), or to state the dependency explicitly.","section":"Section 3.2, Eqs. (2)-(3)"},{"comment":"The notation f_ex used in the query-set construction is undefined; please specify which layer's feature outputs are used and how they are pooled.","section":"Section 3.4"},{"comment":"The text says the query set is built from 'directly averaged task vectors according to the scheme in equation 4', but Eq. (4) is a norm-weighted sum, not a simple average. Please clarify the exact construction.","section":"Section 3.4"},{"comment":"The supplementary section title contains a typo: 'Exprimental Details' should be 'Experimental Details'.","section":"Section 6, Table 5"},{"comment":"No code or data release is mentioned, which hinders reproducibility of the reported merging results. Please provide code or at least detailed per-dataset fine-tuning and evaluation protocols.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper has a nice empirical idea and the Auto-Switch formulation is interesting, but the T-Switch method as written is not a merging method, and the mechanistic claim rests on a norm confound. I would encourage the authors to resubmit after adding norm-matched controls, uncertainty quantification, and a clearer separation between per-task compression and actual multi-task merging."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the practical method is strong, the storage results are impressive, but the central causal claim has a norm confound that the paper never controls for. I'd send it to review, but the authors need to fix the controls before I'd accept it.\n\nWhat's actually new: T-Switch decomposes a task vector into a binary mask, a binary sign, and a scalar, and shows that this binarized representation matches or beats full-precision merging on 8 vision and 8 language tasks at 1–3% of the storage. That is a concrete, useful step beyond DARE and TIES. Auto-Switch's training-free retrieval is a clean alternative to a learned router, and the experiments span two CLIP sizes, RoBERTa, and LoRA. The paper is easy to follow, the α-ablation is informative, and the method is simple enough to reproduce.\n\nThe soft spot is the pulse-like claim. The comparison behind it is not norm-matched. P-Discard (Eq. 3) drops low-magnitude parameters without rescaling, so the task-vector norm shrinks. DARE, the zero control, rescales by 1/(1−α) and keeps the norm. In merging, Eq. 4 applies a global scaling factor, but that factor cancels the common scaling across tasks, so P-Discard effectively moves the merged model less from the pretrained weights. The improvement could be a regularization effect of shrinking the delta, not evidence that low-magnitude coordinates are noise. Bin-Discard (Eq. 5) inherits the same issue because it only rescales to the P-Discarded norm. The \"Discard High\" arm shows that not all shrinkage helps, but it does not separate magnitude selection from norm reduction. The fix is straightforward: add a P-Discard-plus-rescale control (discard low, then rescale to the original norm) and a norm-matched random discard. If the improvement survives, the pulse story holds; if not, the binary switch method may still be valuable, but the headline insight needs to be downgraded.\n\nMinor issues: no error bars or significance tests, single runs, α=0.5 selected on the same benchmarks it is evaluated on, and no code or data released. These are all fixable.\n\nBottom line: worth a serious referee. The practical method is strong and the storage result matters regardless of the causal story. The paper as written over-claims the pulse-like mechanism, and that needs to be addressed before acceptance.","headline":"A genuinely useful binary task-vector merging method with real storage savings, but the paper's headline 'pulse-like' mechanistic claim is not yet supported because the key control is confounded with norm shrinkage.","tokens_in":16857,"tokens_out":3508,"would_cite":true,"duration_ms":33689,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Binarizing task vectors matches full-precision merging while cutting storage to 1-3%.","keywords":["model merging","task vectors","binarization","parameter pruning","multi-task learning","storage efficiency","dynamic merging","pulse activation"],"falsifier":"Find one task vector where systematically discarding the smallest-magnitude entries, say at alpha = 0.5, produces a clear accuracy drop relative to the full task vector, or where the optimal alpha for a single layer is far outside the paper's global optimum. Alternatively, construct two task vectors whose low-magnitude entries are essential for one task, and show that binarizing both degrades merged accuracy below full-precision merging.","tokens_in":15957,"feed_emoji":"⚡","tokens_out":4022,"duration_ms":32772,"temperature":0.7,"pith_summary":"The paper claims that task vectors, the parameter differences between a fine-tuned model and its pretrained base, are dominated by a small set of large-magnitude entries and that the rest are noise. It proposes discarding those small entries, binarizing the survivors to their signs, and rescaling by a single scalar, which it calls a task switch. The claim is that this binary approximation keeps or improves both single-task fine-tuning and multi-task merging accuracy, while cutting per-task storage to 1-3% of full precision. If true, multi-task deployment could store many specialized models as a few bits per parameter and combine them at inference without retraining.","feed_headline":"Binary task vectors match full-precision merging at 1-3% storage","feed_subtitle":"Signs plus a scalar capture task knowledge; discarded small parameters cut conflicts.","key_machinery":"The load-bearing object is the pulse activation function $g_m(\\tau)$, which keeps task-vector entries outside symmetric upper and lower quantile thresholds and zeros the rest. P-Discard applies this mask; Bin-Discard then replaces surviving entries with their signs $g_b(\\tau)$ and rescales by the ratio of the $\\ell^2$ norms of the masked full-precision vector and the masked sign vector. The product is a task switch with three parts: an activation mask $S_A$, a polarity sign $S_P$, and a scalar knob $\\lambda$, combined dynamically at inference by Eq. 7 (or Eq. 8 for Auto-Switch). The mechanism works because the paper's experiments find that low-magnitude parameters are redundant noise whose removal reduces inter-task conflict.","core_discovery":"The central discovery is that task vectors exhibit a pulse-like characteristic: parameters whose magnitude exceeds a per-task quantile threshold carry the useful signal, while parameters below that threshold actively hurt fine-tuning and merging performance. The paper demonstrates through controlled experiments that discarding the low-magnitude entries (P-Discard) improves average accuracy, and that replacing the surviving values with their signs times a scalar ratio (Bin-Discard) retains nearly all of that gain. From this it builds Task Switch (T-Switch), which stores each task as a binary activation mask, a binary sign vector, and one scalar knob, and Auto-Switch, which retrieves and combines these switches at inference from a small query set. The paper reports that T-Switch and Auto-Switch outperform prior merging methods on eight vision and eight language tasks while using 1-3% of the storage of full-precision task vectors.","pith_inferences":["The pulse claim is about coordinate-wise magnitude, not task geometry; a testable extension is whether the optimal discard threshold varies by layer or task, and whether an adaptive per-layer alpha would outperform a single global ratio.","If the sign-and-scalar approximation preserves enough signal, it suggests that fine-tuning knowledge is largely directional: what matters is which parameters increase or decrease and by roughly how much, not the precise magnitudes.","A natural stress test is to merge tasks with heavily overlapping or adversarial task vectors, where the assumption that low-magnitude parameters are pure noise may break down.","The storage accounting (1-3%) assumes the mask and signs are stored efficiently; the paper reports example sizes but a reader should verify whether the mask is stored uncompressed or bit-packed."],"forward_implications":["If the pulse assumption holds, any model-merging pipeline can store each task as roughly 1-3% of its original parameter bits, consisting of a mask, signs, and one scalar per task.","Discarding small-magnitude entries can push merged accuracy above the average of the individually fine-tuned models, as observed in the vision experiments.","The same binarization applies to LoRA-style low-rank task vectors, suggesting that parameter-efficient fine-tuning results can be stored and merged this way too.","Auto-Switch shows that task selection can be performed without a learned router, using nearest-neighbor retrieval on a small label-free query set."],"supporting_citations":[{"why":"Supplies the random drop-and-scale baseline (DARE) that the pulse discard experiments are contrasted against and motivates the redundancy claim.","marker":"[52]"},{"why":"Defines task vectors as fine-tuned minus pretrained weights, the object the paper binarizes.","marker":"[16]"},{"why":"Shows parameter interference in merging and provides the sign-masking idea that T-Switch's polarity switch builds on.","marker":"[50]"},{"why":"Represents the full-precision dynamic merging baseline whose storage cost motivates binarization.","marker":"[27]"},{"why":"The strongest dynamic merging baseline on the same benchmarks; T-Switch compares directly against it.","marker":"[15]"},{"why":"Provides the CLIP vision backbones used in the controlled experiments and vision merging benchmarks.","marker":"[34]"},{"why":"Adaptive merging baseline that requires training; Auto-Switch contrasts its training-free retrieval against it.","marker":"[51]"}],"fun_headline_variants":["Binarized task vectors: 1-3% storage, same or better merging","Pulse-like task vectors: keep signs, drop small weights","Binarized task switches cut storage to 1-3% without loss","Auto-Switch: training-free merging from binarized task vectors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a single global discard ratio alpha, applied as symmetric per-task quantile thresholds, cleanly separates informative from noisy coordinates in every layer of every task vector; if the informative set is task-, layer-, or data-dependent, the switch discards the wrong parameters and the storage-accuracy trade-off collapses.","fun_headline_variants_meta":{"raw":{"variants":["Binarized task vectors: 1-3% storage, same or better merging","Pulse-like task vectors: keep signs, drop small weights","Binarized task switches cut storage to 1-3% without loss","Auto-Switch: training-free merging from binarized task vectors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1313,"prompt_tokens":976,"completion_tokens":337,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":257}},"tokens_in":592,"tokens_out":337,"duration_ms":3740,"temperature":1.0,"reasoning_tokens":257,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:48:31.207004+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find one task vector where systematically discarding the smallest-magnitude entries, say at alpha = 0.5, produces a clear accuracy drop relative to the full task vector, or where the optimal alpha for a single layer is far outside the paper's global optimum. Alternatively, construct two task vectors whose low-magnitude entries are essential for one task, and show that binarizing both degrades merged accuracy below full-precision merging.","supporting_citations":[{"cited_title":"Language models are super mario: Absorbing abilities from homologous models as a free lunch","cited_arxiv_id":null,"evidence_quote":"Supplies the random drop-and-scale baseline (DARE) that the pulse discard experiments are contrasted against and motivates the redundancy claim."},{"cited_title":"Editing models with task arithmetic","cited_arxiv_id":null,"evidence_quote":"Defines task vectors as fine-tuned minus pretrained weights, the object the paper binarizes."},{"cited_title":"Raf- fel, and Mohit Bansal","cited_arxiv_id":null,"evidence_quote":"Shows parameter interference in merging and provides the sign-masking idea that T-Switch's polarity switch builds on."},{"cited_title":"Twin-merging: Dynamic integration of modular expertise in model merging","cited_arxiv_id":null,"evidence_quote":"Represents the full-precision dynamic merging baseline whose storage cost motivates binarization."},{"cited_title":"Emr-merging: Tuning-free high- performance model merging","cited_arxiv_id":null,"evidence_quote":"The strongest dynamic merging baseline on the same benchmarks; T-Switch compares directly against it."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Provides the CLIP vision backbones used in the controlled experiments and vision merging benchmarks."},{"cited_title":"Adamerging: Adap- tive model merging for multi-task learning","cited_arxiv_id":null,"evidence_quote":"Adaptive merging baseline that requires training; Auto-Switch contrasts its training-free retrieval against it."}],"review_version":1}