Pith. sign in

REVIEW 5 major objections 4 minor 42 references

Topology of attention matrices reveals which weights a language model can leave frozen during fine-tuning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 04:12 UTC pith:Y4KGV2YY

load-bearing objection A clever TDA-based freezing method with a real transfer result, but the paper doesn't prove the topology matters — and its abstract overstates efficiency. the 5 major comments →

arxiv 2607.16637 v2 pith:Y4KGV2YY submitted 2026-07-18 cs.AI

TopoTuner: Topological Finetuning of Large Language Models

Abdulkadir Erol , Yash Mahajan , Vepaul Hariprashad , Baha Rababah , Santu Karmaker , Cuneyt G. Akcora , Mubarak Shah This is my paper
classification cs.AI
keywords topological data analysispersistent homologyparameter-efficient fine-tuningselective freezinglarge language modelsattention projectionstransfer learningearly stopping
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that fine-tuning a large language model changes its attention projection matrices in a structured, topology-detectable way. Treating each projection matrix as a point cloud of its rows, it computes persistence diagrams and measures the Wasserstein distance between pretrained and fine-tuned diagrams. This 'topological drift' ranks matrices from most to least changed. The paper claims that freezing the highest-drift matrices, using a profile learned on one reasoning task, transfers to other tasks, letting models be fine-tuned with only 1–2% of parameters updated while matching or beating LoRA in 7 of 9 model–dataset settings and cutting training time. It also proposes stopping fine-tuning when the drift between epochs falls below a threshold.

Core claim

For a fixed pretrained model, the same attention projection matrices tend to absorb the largest topological change across different fine-tuning tasks. TopoTuner measures this change by treating each K/Q/V/O projection matrix as a row cloud, building a Vietoris–Rips filtration, and computing the Wasserstein distance between the persistence diagrams (in homological dimension 0) of the pretrained and fine-tuned matrices. The resulting drift score ranks matrices; freezing the highest-drift matrices on a target task preserves accuracy while updating only about 1–2% of model parameters. The profile learned on GSM8K transfers to sentiment analysis and other QA sets. The paper also shows that epoch-

What carries the argument

The central object is the persistence diagram of an attention projection matrix's row cloud, built with a Vietoris–Rips filtration on cosine distances. The key identity is the p-Wasserstein distance between the pretrained and fine-tuned diagrams, which defines topological distance. Topological drift is the normalized difference of these distances between consecutive epochs. The stability proposition (interleaving of filtrations) and the reuse theorem (gradient-field closeness implies ranking preservation) are the theoretical supports.

Load-bearing premise

The benefit comes specifically from the persistence-based ranking rather than from simply training a small subset of matrices; the paper's own perfect-knowledge experiments show topology-based and magnitude-based selectors are nearly tied, and no transferred magnitude baseline is reported.

What would settle it

Run the same transfer experiment with a magnitude-based freezing profile learned from GSM8K and applied to SST-2, IMDB, and MMLU. If the magnitude-based transferred profile achieves accuracy within the same range as TopoTuner while using the same small parameter budget, the topological signal is not necessary. Conversely, if the magnitude-based profile loses significantly on transfer, topology adds measurable value.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the transfer claim holds, model cards can ship with a reusable freezing profile—a small set of K/Q/V/O projection IDs to freeze—so future tasks skip per-task selection searches.
  • Updating only 1–2% of parameters for competitive accuracy reduces compute and memory footprint, making fine-tuning viable on smaller hardware.
  • Topological drift can serve as an early-stopping criterion, cutting the epoch budget by roughly half to two-thirds without sacrificing final accuracy.
  • Because drift profiles are model-dependent, practitioners must learn a profile per architecture; there is no universal 'always freeze V/O' rule.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The near-tie with entrywise-magnitude selection in the paper's perfect-knowledge tables suggests the main driver may be the small trainable subset rather than the persistence ranking; a transferred magnitude-based baseline would separate these hypotheses.
  • Testing the same transferred profile on very different task families, such as code generation or instruction following, would probe the limits of the claimed 'reusable adaptation routes'.
  • The early-stopping rule could be combined with other parameter-efficient methods to reduce training further, though drift computation adds offline overhead that must be amortized.
  • If topological drift largely tracks gradient magnitude or parameter norm, simpler importance scores might replicate the results; comparing drift ranks against gradient-based importance would settle this.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces TopoTuner, a method that ranks attention projection matrices by topological drift, defined as the Wasserstein distance between persistence diagrams of pretrained and fine-tuned row clouds. A freezing profile learned on QA:GSM8K is transferred to SA:SST-2, SA:IMDB, and QA:MMLU across LLaMA-3.1-8B, Mistral-7B-v0.3, and Qwen3-8B-Base, with the claim that TopoTuner is competitive with full fine-tuning, outperforms LoRA in 7 of 9 settings, and reduces training time by 20.4% relative to full fine-tuning. The paper also proposes a topology-based early-stopping score and evaluates forgetting on eleven datasets.

Significance. If the central attribution were established, this would be a novel and useful PEFT contribution: matrix-level topological drift would identify reusable adaptation channels, and freezing profiles learned on one task could transfer across tasks. The paper is unusually thorough in its appendix documentation, includes open-source code, reports wall-clock times, and evaluates catastrophic forgetting beyond the main tasks. However, the load-bearing claim that the topology-based ranking, rather than the small trainable V/O subset, drives transfer performance is not yet supported. The perfect-knowledge tables show TDA and Eltwise are nearly tied, and the transferred experiments include no non-topological selective-freezing baseline. The abstract also contains a factual inconsistency about the fraction of trained parameters.

major comments (5)
  1. [§6.1, Table 2; Appendix Tables 9–11] The transferred-knowledge experiment reports only TDA-High3; no transferred Eltwise-High3 or random equivalent-budget V/O freezing baseline is provided. The perfect-knowledge tables show TDA and Eltwise are near-tied: LLaMA 61.17 (TDA-High6) vs 61.17 (Eltwise-Low9), Qwen 87.62 (TDA-High3) vs 87.43 (Eltwise-High9), Mistral 54.31 (TDA-Low6) vs 53.96 (Eltwise-High6). Since the central claim is that the persistence ranking identifies reusable adaptation channels, the observed gains in Table 2 could be explained by the small trainable V/O subset rather than by the topological ordering. Please add transferred Eltwise and random freezing baselines with the same budget to Table 2, and report the comparison in the main text.
  2. [Abstract; Table 2] The abstract states that TopoTuner trains 'only 1–2% of the model parameters,' but Table 2 reports Train.% = 7.57–8.45 for TopoTuner. The 1–2% figure corresponds to Upd.% (0.68–2.98), which is a different quantity. This is a factual inconsistency in the headline claim and must be corrected, along with the related sentence 'which can change up to 39.57% of the projection parameters,' which is unclear and appears to refer to a LoRA mean relative change in Table 7, not to TopoTuner.
  3. [§6.1, Table 2; Appendix Tables 9–11] The freezing budget b=3 and the early-stopping thresholds η appear to be selected after observing target-task outcomes; only the TDA-High3 budget is shown in the main Table 2, while larger and low-drift budgets are relegated to the appendix. This post-hoc selection inflates the '7 out of 9' claim. Please report all budgets in the main comparison or provide a pre-specified selection rule, and discuss the sensitivity of the main result to the budget choice.
  4. [Theorem 5.1; Appendix Q] The transfer guarantee assumes source and target gradient fields are ϵ-close on a ball around initialization and that the row-cloud metric is stable under row perturbations; neither assumption is verified experimentally. As a sufficient condition, the theorem does not by itself establish that the GSM8K-learned profile transfers. Please add an empirical check of gradient closeness between the prior and target tasks (e.g., gradient cosine/distance), or soften the theoretical framing so that the transfer claim rests on the experiments rather than on an unverified assumption.
  5. [Table 2] The abstract and introduction describe TopoTuner as 'competitive with full fine-tuning,' but Table 2 shows substantial gaps in some settings, e.g., Mistral-7B-v0.3 on IMDB: TopoTuner 79.76 vs full fine-tuning 83.10; Qwen3-8B-Base on MMLU: 72.83 vs 73.22. The paper should characterize the settings where TopoTuner trails full fine-tuning and avoid an unconditional competitiveness claim.
minor comments (4)
  1. [Appendix O] The appendix states that 'neither signal alone induces a universally dominant ordering' for TDA vs Eltwise. This concession should be reflected in the main text, where the ordering is described as decisive for transfer.
  2. [Table 2] The LoRA Train.% entries are marked '100*' with a footnote about merging. Clarify whether this column reports trainable parameters before merging or affected parameters after merging; the current notation is confusing when compared with TopoTuner's Train.%.
  3. [Figure 7 and profile cards] The profile cards use 'ETA' to denote training time. Consider renaming to 'Time' or 'Wall-clock time' to avoid ambiguity with estimated time of arrival.
  4. [§6.1] Low-drift transferred results are omitted from the main text. A sentence summarizing their performance would help readers assess how sensitive the method is to the High/Low choice.

Circularity Check

0 steps flagged

No circularity: transferred freezing profiles are computed on a source task and evaluated on independent target tasks; missing controls are evidence gaps, not circularity.

full rationale

The headline transfer experiment is genuinely source-to-target: the freezing profile is computed from full fine-tuning on QA:GSM8K (Definitions 4.1-4.2 and Section 6.1) and then applied to out-of-domain datasets (SA:SST-2, SA:IMDB, QA:MMLU) with task accuracy as an independent evaluation. The TDA scores are not defined in terms of target-task outcomes, and the target datasets are not used to construct the frozen set. The perfect-knowledge tables in Appendix J are explicitly labeled diagnostic ('mainly for diagnostics'), so they are not presented as held-out predictions. Theorem 5.1 is a conditional continuity result: assuming epsilon-close gradient fields and stable row metrics, it derives bounded perturbation of Wasserstein scores; it does not assume the target ranking, and its proof rests on standard persistence stability [38], not on author-specific claims. The cited prior work by co-authors (e.g., [1]) is background and is not load-bearing. The main limitations - no transferred Eltwise/random-equivalent-budget baseline, near-tied perfect-knowledge TDA vs Eltwise, post-hoc selection of b=3 and eta, and the paper's own concession that 'neither signal alone induces a universally dominant ordering' (Appendix O) - are evidence/validity concerns about topology-specific attribution, but no equation or fitted parameter is renamed as a prediction. Hence no circular step can be exhibited.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The method imports TDA as a tool and does not introduce new physical entities. The central free choices are the freezing budget, the stopping threshold, and the metric/homological dimension. The theoretical transfer result rests on an unverified gradient-closeness assumption, so the empirical transfer claim is carrying the argument.

free parameters (3)
  • Freezing budget b = 3 (High-3) = 3 V-layers + 3 O-layers per profile
    The main transfer table reports only TDA-High3, selected after observing that it is the smallest high-drift budget that improves over LoRA in 7/9 settings. No principled criterion for choosing b is given.
  • Early-stopping threshold eta = 0.15-0.25 depending on model and dataset
    In Figure 6 and Appendix M, eta is chosen per model and target dataset to stop at epochs 2-3; these values are fit to the runs that generate the claimed time savings.
  • Homological dimension k and row metric = k=0, cosine distance
    All experiments use H0 persistence with cosine distance. The paper does not report sensitivity to this choice, so the method's behavior may depend on these fixed choices.
axioms (5)
  • standard math Vietoris-Rips persistent homology and the stability theorem for persistence diagrams
    Used to define topological distance and drift; the proof of Proposition 4.3 relies on standard interleaving stability from [38].
  • ad hoc to paper The cosine row-cloud metric is stable under row perturbations with implied Lipschitz constant 1
    The proof of Theorem Q.2 asserts norm-difference of metrics is bounded by 2R_t from row closeness. This is postulated as 'stable under row perturbations' but is not established for cosine distance on arbitrary unnormalized rows.
  • ad hoc to paper Source and target gradient fields remain epsilon-close on a ball around initialization
    This is the key premise of Theorem 5.1's transfer guarantee. It is never measured or verified; it is a sufficient condition assumed to hold.
  • domain assumption Fine-tuning trajectories stay within a ball and objectives are L-Lipschitz
    Standard regularity assumptions in Theorem 5.1; plausible for small learning rates but not checked empirically.
  • domain assumption Freezing high-drift matrices acts as a structural regularizer and reduces overfitting
    Used to explain why high-drift freezing is preferable to low-drift freezing; supported only by the Appendix P gap analysis on one model.

pith-pipeline@v1.3.0-alltime-deepseek · 40328 in / 13995 out tokens · 146329 ms · 2026-08-04T04:12:46.353589+00:00 · methodology

0 comments
read the original abstract

Full fine-tuning remains a strong way to adapt pretrained LLMs, but it updates all weights and can be expensive. LoRA reduces the number of trainable parameters, but it does not directly answer which pretrained components should be trained and which can be frozen during adaptation. We introduce TopoTuner, a topology-guided fine-tuning framework for selective freezing of attention projection matrices. \method treats each projection matrix as a row cloud and uses Wasserstein distances between persistence diagrams to measure how its topology changes during fine-tuning. TopoTuner learns a reusable freezing profile from a source dataset and transfers it to efficiently fine-tune models on out-of-domain datasets, evaluating whether task-specific topological drift generalizes across question answering and sentiment analysis tasks. Across LLaMA-3.1-8B, Mistral-7B-v0.3, and Qwen3-8B-Base, TopoTuner is competitive with full fine-tuning while training only 1-2\% of the model parameters, and outperforms LoRA in 7 out of 9 model-dataset settings, which can change up to 39.57\% of the projection parameters. Along with minimized updates, TopoTuner reduces training time by 20.4\% relative to full fine-tuning and 5.5\% relative to LoRA on average. TopoTuner opens a new direction for reusable freezing profiles, where fine-tuning behavior learned on one dataset can be shared across multiple tasks.

Figures

Figures reproduced from arXiv: 2607.16637 by Abdulkadir Erol, Baha Rababah, Cuneyt G. Akcora, Mubarak Shah, Santu Karmaker, Vepaul Hariprashad, Yash Mahajan.

Figure 1
Figure 1. Figure 1: LLaMA O-projection change under high-drift Wasserstein freezing (3 to 15 layers). As more matrices are frozen during training, the remaining trainable matrices absorb updates in a more concentrated manner; selective freezing redistributes adaptation across layers. reusable freezing profiles, where fine-tuning behavior learned on one dataset can be shared across multiple tasks. 1 Introduction Fine-tuning [1… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of TOPOTUNER. TOPOTUNER uses a prior fine-tuning task to compute the topological drift of attention projection matrices. The computed topological drift is used to (A) build reusable freezing profiles for selective fine-tuning on new datasets and tasks, and to (B) track structural change across epochs for early stopping. Freezing profiles are provided in Appendix B. • We introduce TOPOTUNER, a topo… view at source ↗
Figure 3
Figure 3. Figure 3: Epoch-6 Wasserstein [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Evolution of Wasserstein H0 distances D (t) i over layers for the V projection of LLaMA fully fine-tuned on QA:MMLU. Each line is a dis￾tance to the pretrained model [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Epoch-to-epoch topological drift bars for the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Topology-based stopping epochs under full fine-tuning. The plots show selected stopping epochs across η thresholds for SA:SST-2 across the three model families. Time efficiency. On QA:GSM8K, persistence computa￾tion takes approximately 2.1 min for LLaMA, 2.4 min for Qwen, and 2.2 min for Mistral per epoch; the subsequent Wasserstein computation is shorter, taking approximately 1.8 min, 2.1 min, and 2.0 min… view at source ↗
Figure 7
Figure 7. Figure 7: Reusable freez￾ing profile for LLaMA-3.1-8B learned from QA:GSM8K. Our results and descriptive analysis in the Appendix Section E sug￾gest that fine-tuning changes LLMs in a structured but highly con￾strained manner. Entrywise weight movements are small, and most parameters remain close to their pretrained values. This makes purely magnitude-based descriptions incomplete: the important question is not only… view at source ↗
Figure 8
Figure 8. Figure 8: Reusable Freezing Profiles. We define a reusable freezing profile learned from a prior￾task fine-tuning run. Each model card summarizes the model, prior task, candidate matrix set, freezing strategy, selected frozen IDs, and outcome on target datasets. We report three profiles used in our experiments: TDA-High3-LLaMA, TDA-High3-Qwen, and TDA-High3-Mistral. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Epoch-6 Eltwise distances for the V and O projection matrices under H0. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Epoch-6 Wasserstein distances for the V and O projection matrices under H0. H Freezing Pseudocode Algorithms 1–3 summarize the full TOPOTUNER selection pipeline. Algorithm 1 computes projection-specific layer orderings using Wasserstein topological drift, Algorithm 2 converts these orderings into frozen K/Q/V /O layer sets under the Low or High regime, and Algorithm 3 applies the resulting sets by freezin… view at source ↗
Figure 11
Figure 11. Figure 11: Topological drift of the V and O projection on QA:GSM8K under H0. The top row shows epoch-to-epoch drift bars, and the bottom row shows Wasserstein distance curves across layers. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Topological drift of the V and O projection on QA:MMLU under H0. The top row shows epoch-to-epoch drift bars, and the bottom row shows Wasserstein distance curves across layers. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Topological drift of the V projection on SA:IMDB under H0. The top row shows epoch￾to-epoch drift bars, and the bottom row shows Wasserstein distance curves across layers. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Topological drift of the V and O projection on SA:SST-2 under H0. The top row shows epoch-to-epoch drift bars, and the bottom row shows Wasserstein distance curves across layers. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Accuracies on each epoch for all the models across datasets [PITH_FULL_IMAGE:figures/full_fig_p027_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Topology-based stopping epochs under full fine-tuning. The plots show selected stopping [PITH_FULL_IMAGE:figures/full_fig_p028_16.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 10 linked inside Pith

  1. [1]

    Llm post-training: A deep dive into reasoning large language models.arXiv preprint arXiv:2502.21321, 2025

    Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. Llm post-training: A deep dive into reasoning large language models.arXiv preprint arXiv:2502.21321, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026

    Lingling Xu, Haoran Xie, S Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026

    2026

  3. [3]

    A survey of large language model agents for question answering.arXiv preprint arXiv:2503.19213, 2025

    Murong Yue. A survey of large language model agents for question answering.arXiv preprint arXiv:2503.19213, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    An empirical study of catastrophic forgetting in large language models during continual fine-tuning.IEEE Trans- actions on Audio, Speech and Language Processing, 2025

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning.IEEE Trans- actions on Audio, Speech and Language Processing, 2025

    2025

  5. [5]

    Be confident: Uncovering overfitting in mllm multi-task tuning

    Wenke Huang, Jian Liang, Guancheng Wan, Didi Zhu, He Li, Jiawei Shao, Mang Ye, Bo Du, and Dacheng Tao. Be confident: Uncovering overfitting in mllm multi-task tuning. InForty- second International Conference on Machine Learning, 2025

    2025

  6. [6]

    Targeted vaccine: Safety alignment for large language models against harmful fine-tuning via layer-wise perturbation.IEEE Transactions on Information Forensics and Security, 2025

    Guozhi Liu, Weiwei Lin, Qi Mu, Tiansheng Huang, Ruichao Mo, Yuren Tao, and Li Shen. Targeted vaccine: Safety alignment for large language models against harmful fine-tuning via layer-wise perturbation.IEEE Transactions on Information Forensics and Security, 2025

    2025

  7. [7]

    Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    2022

  8. [8]

    Beyond single-task: Robust multi-task length generalization for llms.arXiv preprint arXiv:2502.11525, 2025

    Yi Hu, Shijia Kang, Haotong Yang, Haotian Xu, and Muhan Zhang. Beyond single-task: Robust multi-task length generalization for llms.arXiv preprint arXiv:2502.11525, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Shuttleworth, Jacob Andreas, Antonio Torralba, and Pratyusha Sharma

    Reece S. Shuttleworth, Jacob Andreas, Antonio Torralba, and Pratyusha Sharma. LoRA vs full fine-tuning: An illusion of equivalence. InProceedings of the Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 10

    2025

  10. [10]

    Topology and data.Bulletin of the American Mathematical Society, 46(2): 255–308, 2009

    Gunnar Carlsson. Topology and data.Bulletin of the American Mathematical Society, 46(2): 255–308, 2009

    2009

  11. [11]

    An introduction to topological data analysis: fundamen- tal and practical aspects for data scientists.Frontiers in Artificial Intelligence, 4, 2021

    Fr ´ed´eric Chazal and Bertrand Michel. An introduction to topological data analysis: fundamen- tal and practical aspects for data scientists.Frontiers in Artificial Intelligence, 4, 2021

    2021

  12. [12]

    Characterizing and measuring the similarity of neural networks with persistent homology

    David P ´erez-Fern´andez, Asier Guti ´errez-Fandi˜no, Jordi Armengol-Estap ´e, and Marta Ville- gas. Characterizing and measuring the similarity of neural networks with persistent homology. arXiv preprint arXiv:2101.07752, 2021

    Pith/arXiv arXiv 2021

  13. [13]

    Experimental observations of the topology of convolutional neural network activations

    Emilie Purvine, Davis Brown, Brett Jefferson, Cliff Joslyn, Brenda Praggastis, Archit Rathore, Madelyn Shapiro, Bei Wang, and Youjia Zhou. Experimental observations of the topology of convolutional neural network activations. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9470–9479, 2023

    2023

  14. [14]

    Compressible dynamics in deep over- parameterized low-rank learning & adaptation

    Can Yaras, Peng Wang, Laura Balzano, and Qing Qu. Compressible dynamics in deep over- parameterized low-rank learning & adaptation. InForty-first International Conference on Ma- chine Learning, 2026

    2026

  15. [15]

    Spurious forgetting in continual learn- ing of language models

    Junhao Zheng, Xidi Cai, Shengjie Qiu, and Qianli Ma. Spurious forgetting in continual learn- ing of language models. InThe Thirteenth International Conference on Learning Representa- tions, 2025

    2025

  16. [16]

    Dropbp: Accelerating fine-tuning of large language models by drop- ping backward propagation.Advances in Neural Information Processing Systems, 37:20170– 20197, 2024

    Sunghyeon Woo, Baeseong Park, Byeongwook Kim, Minjung Jo, Se Jung Kwon, Dongsuk Jeon, and Dongsoo Lee. Dropbp: Accelerating fine-tuning of large language models by drop- ping backward propagation.Advances in Neural Information Processing Systems, 37:20170– 20197, 2024

    2024

  17. [17]

    A survey of topological machine learning methods.Frontiers in Artificial Intelligence, 4:52, 2021

    Felix Hensel, Michael Moor, and Bastian Rieck. A survey of topological machine learning methods.Frontiers in Artificial Intelligence, 4:52, 2021

    2021

  18. [18]

    A theoretical framework for llm fine-tuning using early stopping for non-random initialization.arXiv preprint arXiv:2602.13942, 2026

    Zexuan Sun and Garvesh Raskutti. A theoretical framework for llm fine-tuning using early stopping for non-random initialization.arXiv preprint arXiv:2602.13942, 2026

    arXiv 2026

  19. [19]

    Christopher Tralie, Nathaniel Saul, and Rann Bar-On. Ripser. py: A lean persistent homology library for python.Journal of Open Source Software, 3(29):925, 2018

    2018

  20. [20]

    The gudhi library: Simplicial complexes and persistent homology

    Cl ´ement Maria, Jean-Daniel Boissonnat, Marc Glisse, and Mariette Yvinec. The gudhi library: Simplicial complexes and persistent homology. InInternational congress on mathematical software, pages 167–174. Springer, 2014

    2014

  21. [21]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  22. [22]

    Measuring massive multitask language understanding.Proceedings of the International Conference on Learning Representations (ICLR), 2021

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  23. [23]

    Aligning ai with shared human values.Proceedings of the International Conference on Learning Representations (ICLR), 2021

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning ai with shared human values.Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  24. [24]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y . Ng, and Christo- pher Potts. Learning word vectors for sentiment analysis. InProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon, USA, June 2011. Association for Computational Linguis- tics...

    2011

  25. [25]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a senti- ment treebank. InProceedings of the 2013 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1631–1642, Seattle, Washington, USA, October 2013. Association fo...

    2013

  26. [26]

    Cohen, Ruslan Salakhut- dinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhut- dinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2018

    2018

  27. [27]

    SQuAD: 100,000+ ques- tions for machine comprehension of text

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ ques- tions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras, editors, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas, November 2016. Association for Computational Linguisti...

    doi:10.18653/v1/d16-1264 2016

  28. [28]

    Cohen, and Mirella Lapata

    Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.ArXiv, abs/1808.08745, 2018

    Pith/arXiv arXiv 2018

  29. [29]

    Liu, and Christopher D

    Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer-generator networks. InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073–1083, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10.18653/v1/P17-1099. URL h...

    doi:10.18653/v1/p17-1099 2017

  30. [30]

    Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023. URLhttps://www.databricks.com/blog/2023/ 04/12/dolly-first-open-commercially-viable-instruction-tuned-llm

    2023

  31. [31]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model.https://github.com/tatsu-lab/stanford_alpaca, 2023

    2023

  32. [32]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    2021

  33. [33]

    Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    Pith/arXiv arXiv 2021

  34. [34]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L´elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timoth´ee Lacroix, and William El Sayed. Mistral 7b.arXiv preprint arXiv:...

    Pith/arXiv arXiv 2023

  36. [36]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025. URLhttps://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  37. [37]

    Spectrum: Targeted training on signal to noise ratio.https://github.com/ QuixiAI/spectrum, 2024

    QuixiAI. Spectrum: Targeted training on signal to noise ratio.https://github.com/ QuixiAI/spectrum, 2024. GitHub repository

    2024

  38. [38]

    Springer, 2016

    Fr ´ed´eric Chazal, Vin de Silva, Marc Glisse, and Steve Oudot.The Structure and Stability of Persistence Modules. Springer, 2016. 12

    2016

  39. [39]

    Intrinsic dimensionality explains the effectiveness of language model fine-tuning

    Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. InProceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 7319–7328, 2021

    2021

  40. [40]

    Bitfit: Simple parameter-efficient fine- tuning for transformer-based masked language-models

    Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine- tuning for transformer-based masked language-models. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1–9, 2022

    2022

  41. [41]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Ha- jishirzi, and Ali Farhadi. Editing models with task arithmetic. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  42. [42]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without in- creasing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo- Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without in- creasing inference time. InInternational conference on machine learning, pages 23965–23998...

    arXiv 2022