{"id":"46983b7d-1b44-44d6-8cd6-8d554b122b05","arxiv_id":"2607.16637","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"TopoTuner freezes attention projections ranked by topological drift and transfers the profile across tasks, beating LoRA in 7/9 settings.","lead":"TopoTuner is a method for fine-tuning large language models that uses shape analysis of weight matrices to decide which attention projections to freeze. It claims these 'freezing profiles' learned on one task (GSM8K) transfer to other tasks, beating LoRA in 7 of 9 settings while training far fewer parameters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Topology-specific attribution is untested: no transferred Eltwise/random V-O freeze baseline, and perfect-knowledge TDA vs Eltwise is near-tied.","rationale":"The reader's weakest assumption identifies the same concern: no control isolates the topological signal. This is the most load-bearing issue because the paper's novelty and reusable-profile contribution depend on persistence rankings being informative for transfer. Without a transferred Eltwise or random freeze baseline, the 7/9 advantage over LoRA is compatible with a much simpler explanation: TDA-High3 is simply one way to select a small V/O trainable subset, and any such subset might perform similarly. The perfect-knowledge tables already show TDA and Eltwise are near-tied, which makes the missing transferred Eltwise control especially damaging. The theorem's unverified assumptions are a secondary concern because the theorem is only a sufficient condition; if the empirical control were run and favored TDA, the theorem's weakness would not overturn the central claim. The proposed test directly settles the attribution question. Since the reader already issued a CONDITIONAL verdict focused on this gap, no verdict adjustment is needed.","tokens_in":40640,"tokens_out":2122,"duration_ms":25171,"concrete_test":"Run transferred-knowledge freezing with Eltwise-High3 and random-High3 (3 V + 3 O matrices, same total trainable budget, MLP frozen) using profiles estimated from QA:GSM8K, across all 3 models and 3 target datasets in Table 2. If Eltwise-High3 or random freeze accuracy is within +/-1 point of TDA-High3 for most settings, the topological ranking is not load-bearing; if TDA-High3 clearly beats both, the topology-specific transfer claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is not merely that freezing a small V/O subset beats LoRA; it is that the persistence-diagram ranking identifies reusable adaptation channels. In the transferred-knowledge experiments (Table 2), TopoTuner is represented only by TDA-High3, and no transferred Eltwise-High3 or random-equivalent-budget freezing baseline is reported. The perfect-knowledge diagnostic Tables 9-11 show TDA and Eltwise essentially tied: LLaMA best 61.17 (TDA-High6) vs 61.17 (Eltwise-Low9); Qwen 87.62 (TDA-High3) vs 87.43 (Eltwise-High9); Mistral 54.31 (TDA-Low6) vs 53.96 (Eltwise-High6). If any fixed 6-matrix V/O freeze transfers comparably, the observed gains could come entirely from the small trainable V/O subset, not from the topological ranking. The paper itself concedes 'neither signal alone induces a universally dominant ordering' (Appendix O) and supports TDA's added value only through ranking geometry, not through transferred accuracy. Theorem 5.1 is a sufficient condition whose epsilon-gradient-closeness assumption is unverified, so it cannot rescue the attribution. The load-bearing empirical premise—that the persistence ranking, rather than the trainable subset, drives transfer performance—is therefore unsupported by the current experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces TopoTuner, a method that ranks attention projection matrices by topological drift, defined as the Wasserstein distance between persistence diagrams of pretrained and fine-tuned row clouds. A freezing profile learned on QA:GSM8K is transferred to SA:SST-2, SA:IMDB, and QA:MMLU across LLaMA-3.1-8B, Mistral-7B-v0.3, and Qwen3-8B-Base, with the claim that TopoTuner is competitive with full fine-tuning, outperforms LoRA in 7 of 9 settings, and reduces training time by 20.4% relative to full fine-tuning. The paper also proposes a topology-based early-stopping score and evaluates forgetting on eleven datasets.","tokens_in":40997,"tokens_out":5086,"duration_ms":53091,"significance":"If the central attribution were established, this would be a novel and useful PEFT contribution: matrix-level topological drift would identify reusable adaptation channels, and freezing profiles learned on one task could transfer across tasks. The paper is unusually thorough in its appendix documentation, includes open-source code, reports wall-clock times, and evaluates catastrophic forgetting beyond the main tasks. However, the load-bearing claim that the topology-based ranking, rather than the small trainable V/O subset, drives transfer performance is not yet supported. The perfect-knowledge tables show TDA and Eltwise are nearly tied, and the transferred experiments include no non-topological selective-freezing baseline. The abstract also contains a factual inconsistency about the fraction of trained parameters.","major_comments":[{"comment":"The transferred-knowledge experiment reports only TDA-High3; no transferred Eltwise-High3 or random equivalent-budget V/O freezing baseline is provided. The perfect-knowledge tables show TDA and Eltwise are near-tied: LLaMA 61.17 (TDA-High6) vs 61.17 (Eltwise-Low9), Qwen 87.62 (TDA-High3) vs 87.43 (Eltwise-High9), Mistral 54.31 (TDA-Low6) vs 53.96 (Eltwise-High6). Since the central claim is that the persistence ranking identifies reusable adaptation channels, the observed gains in Table 2 could be explained by the small trainable V/O subset rather than by the topological ordering. Please add transferred Eltwise and random freezing baselines with the same budget to Table 2, and report the comparison in the main text.","section":"§6.1, Table 2; Appendix Tables 9–11"},{"comment":"The abstract states that TopoTuner trains 'only 1–2% of the model parameters,' but Table 2 reports Train.% = 7.57–8.45 for TopoTuner. The 1–2% figure corresponds to Upd.% (0.68–2.98), which is a different quantity. This is a factual inconsistency in the headline claim and must be corrected, along with the related sentence 'which can change up to 39.57% of the projection parameters,' which is unclear and appears to refer to a LoRA mean relative change in Table 7, not to TopoTuner.","section":"Abstract; Table 2"},{"comment":"The freezing budget b=3 and the early-stopping thresholds η appear to be selected after observing target-task outcomes; only the TDA-High3 budget is shown in the main Table 2, while larger and low-drift budgets are relegated to the appendix. This post-hoc selection inflates the '7 out of 9' claim. Please report all budgets in the main comparison or provide a pre-specified selection rule, and discuss the sensitivity of the main result to the budget choice.","section":"§6.1, Table 2; Appendix Tables 9–11"},{"comment":"The transfer guarantee assumes source and target gradient fields are ϵ-close on a ball around initialization and that the row-cloud metric is stable under row perturbations; neither assumption is verified experimentally. As a sufficient condition, the theorem does not by itself establish that the GSM8K-learned profile transfers. Please add an empirical check of gradient closeness between the prior and target tasks (e.g., gradient cosine/distance), or soften the theoretical framing so that the transfer claim rests on the experiments rather than on an unverified assumption.","section":"Theorem 5.1; Appendix Q"},{"comment":"The abstract and introduction describe TopoTuner as 'competitive with full fine-tuning,' but Table 2 shows substantial gaps in some settings, e.g., Mistral-7B-v0.3 on IMDB: TopoTuner 79.76 vs full fine-tuning 83.10; Qwen3-8B-Base on MMLU: 72.83 vs 73.22. The paper should characterize the settings where TopoTuner trails full fine-tuning and avoid an unconditional competitiveness claim.","section":"Table 2"}],"minor_comments":[{"comment":"The appendix states that 'neither signal alone induces a universally dominant ordering' for TDA vs Eltwise. This concession should be reflected in the main text, where the ordering is described as decisive for transfer.","section":"Appendix O"},{"comment":"The LoRA Train.% entries are marked '100*' with a footnote about merging. Clarify whether this column reports trainable parameters before merging or affected parameters after merging; the current notation is confusing when compared with TopoTuner's Train.%.","section":"Table 2"},{"comment":"The profile cards use 'ETA' to denote training time. Consider renaming to 'Time' or 'Wall-clock time' to avoid ambiguity with estimated time of arrival.","section":"Figure 7 and profile cards"},{"comment":"Low-drift transferred results are omitted from the main text. A sentence summarizing their performance would help readers assess how sensitive the method is to the High/Low choice.","section":"§6.1"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong empirical base and a clear presentation, but the novelty claim currently rests on an incomplete attribution experiment. Adding transferred Eltwise and random freezing baselines is essential; without them, the 7/9 result may be a property of the small trainable subset rather than of topological ranking. The abstract mismatch on trainable parameters should also be fixed. I believe the paper is salvageable within its scope and recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline result is real: a freezing profile learned on GSM8K transfers to three other tasks, and TopoTuner beats LoRA in 7 of 9 model-dataset settings while updating far fewer parameters. That is worth taking seriously. Using Wasserstein distances between persistence diagrams to rank attention projections for freezing is genuinely new, and the transfer experiment is properly source-to-target, not circular. The paper also ships code and reports forgetting behavior, which is more than many PEFT papers do.\n\nThe soft spots are real too, and they touch the central claim. The abstract says \"training only 1–2% of the model parameters,\" but Table 2 reports Train.% between 7.6 and 8.5. Maybe they meant updated parameters, but that is not what they wrote. \"Competitive with full fine-tuning\" also fails for LLaMA and Mistral on IMDB, where TopoTuner scores 83.6 and 79.8 versus 90.7 and 83.1 for full fine-tuning. That is not competitive.\n\nThe bigger issue is attribution. The paper claims the persistence ranking identifies reusable adaptation channels, but it never runs a transferred magnitude-based or random freezing baseline. The perfect-knowledge tables show TDA and Eltwise are near-tied: 61.17 vs 61.17 on LLaMA, 87.62 vs 87.43 on Qwen, 54.31 vs 53.96 on Mistral. Without a transferred Eltwise or random baseline, the observed gains over LoRA could come simply from freezing a small V/O subset. Theorem 5.1 is a sufficient condition with unverified gradient-closeness assumptions, so it does not rescue the attribution. The paper even concedes that \"neither signal alone induces a universally dominant ordering\" in Appendix O.\n\nThe choice of High-3 as the representative budget also looks post hoc, and the early-stopping thresholds are tuned after seeing outcomes. None of this is fatal: the transfer result is still useful even if the topological ranking is not uniquely responsible. But the paper's current text overclaims.\n\nWho should read it: anyone working on parameter-efficient fine-tuning or on applying TDA to model internals. It deserves a serious referee, but the referee should send it back with a request for the missing baselines and a corrected abstract. I would not desk-reject it, and I would not accept it as is.\n\nMy recommendation: send it to peer review, with the expectation of major revision. The core idea is worth engaging with; the evidence is not yet there for the strong topological interpretation.","headline":"A clever TDA-based freezing method with a real transfer result, but the paper doesn't prove the topology matters — and its abstract overstates efficiency.","tokens_in":41486,"tokens_out":2828,"would_cite":true,"duration_ms":31561,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Topology of attention matrices reveals which weights a language model can leave frozen during fine-tuning.","keywords":["topological data analysis","persistent homology","parameter-efficient fine-tuning","selective freezing","large language models","attention projections","transfer learning","early stopping"],"falsifier":"Run the same transfer experiment with a magnitude-based freezing profile learned from GSM8K and applied to SST-2, IMDB, and MMLU. If the magnitude-based transferred profile achieves accuracy within the same range as TopoTuner while using the same small parameter budget, the topological signal is not necessary. Conversely, if the magnitude-based profile loses significantly on transfer, topology adds measurable value.","tokens_in":40523,"feed_emoji":"🧊","tokens_out":5309,"duration_ms":46938,"temperature":0.7,"pith_summary":"The paper argues that fine-tuning a large language model changes its attention projection matrices in a structured, topology-detectable way. Treating each projection matrix as a point cloud of its rows, it computes persistence diagrams and measures the Wasserstein distance between pretrained and fine-tuned diagrams. This 'topological drift' ranks matrices from most to least changed. The paper claims that freezing the highest-drift matrices, using a profile learned on one reasoning task, transfers to other tasks, letting models be fine-tuned with only 1–2% of parameters updated while matching or beating LoRA in 7 of 9 model–dataset settings and cutting training time. It also proposes stopping fine-tuning when the drift between epochs falls below a threshold.","feed_headline":"Freezing 2% of LLM weights still beats LoRA on 7 of 9 settings","feed_subtitle":"Topological drift in attention matrices reveals which projections to freeze, cutting training time by 20% while preserving accuracy.","key_machinery":"The central object is the persistence diagram of an attention projection matrix's row cloud, built with a Vietoris–Rips filtration on cosine distances. The key identity is the p-Wasserstein distance between the pretrained and fine-tuned diagrams, which defines topological distance. Topological drift is the normalized difference of these distances between consecutive epochs. The stability proposition (interleaving of filtrations) and the reuse theorem (gradient-field closeness implies ranking preservation) are the theoretical supports.","core_discovery":"For a fixed pretrained model, the same attention projection matrices tend to absorb the largest topological change across different fine-tuning tasks. TopoTuner measures this change by treating each K/Q/V/O projection matrix as a row cloud, building a Vietoris–Rips filtration, and computing the Wasserstein distance between the persistence diagrams (in homological dimension 0) of the pretrained and fine-tuned matrices. The resulting drift score ranks matrices; freezing the highest-drift matrices on a target task preserves accuracy while updating only about 1–2% of model parameters. The profile learned on GSM8K transfers to sentiment analysis and other QA sets. The paper also shows that epoch-","pith_inferences":["The near-tie with entrywise-magnitude selection in the paper's perfect-knowledge tables suggests the main driver may be the small trainable subset rather than the persistence ranking; a transferred magnitude-based baseline would separate these hypotheses.","Testing the same transferred profile on very different task families, such as code generation or instruction following, would probe the limits of the claimed 'reusable adaptation routes'.","The early-stopping rule could be combined with other parameter-efficient methods to reduce training further, though drift computation adds offline overhead that must be amortized.","If topological drift largely tracks gradient magnitude or parameter norm, simpler importance scores might replicate the results; comparing drift ranks against gradient-based importance would settle this."],"forward_implications":["If the transfer claim holds, model cards can ship with a reusable freezing profile—a small set of K/Q/V/O projection IDs to freeze—so future tasks skip per-task selection searches.","Updating only 1–2% of parameters for competitive accuracy reduces compute and memory footprint, making fine-tuning viable on smaller hardware.","Topological drift can serve as an early-stopping criterion, cutting the epoch budget by roughly half to two-thirds without sacrificing final accuracy.","Because drift profiles are model-dependent, practitioners must learn a profile per architecture; there is no universal 'always freeze V/O' rule."],"fun_headline_variants":["Topology-guided freezing: 2% params, beats LoRA in 7/9","LLM finetuning: freeze attention matrices that drift most, save time","Which LLM weights to freeze? Topological change answers","Freeze high-drift attention weights, beat LoRA with 1-2% params","TopoTuner: reuse source freezing profile across tasks, beat LoRA"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The benefit comes specifically from the persistence-based ranking rather than from simply training a small subset of matrices; the paper's own perfect-knowledge experiments show topology-based and magnitude-based selectors are nearly tied, and no transferred magnitude baseline is reported.","fun_headline_variants_meta":{"raw":{"variants":["Topology-guided freezing: 2% params, beats LoRA in 7/9","LLM finetuning: freeze attention matrices that drift most, save time","Which LLM weights to freeze? Topological change answers","Freeze high-drift attention weights, beat LoRA with 1-2% params","TopoTuner: reuse source freezing profile across tasks, beat LoRA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001439,"raw_usage":{"total_tokens":5664,"prompt_tokens":799,"completion_tokens":4865,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":4763}},"tokens_in":543,"tokens_out":4865,"duration_ms":33399,"temperature":1.0,"reasoning_tokens":4763,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T04:12:46.353589+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same transfer experiment with a magnitude-based freezing profile learned from GSM8K and applied to SST-2, IMDB, and MMLU. If the magnitude-based transferred profile achieves accuracy within the same range as TopoTuner while using the same small parameter budget, the topological signal is not necessary. Conversely, if the magnitude-based profile loses significantly on transfer, topology adds measurable value.","supporting_citations":[],"review_version":2}