Pith. sign in

REVIEW 5 major objections 6 minor 40 references

The paper claims that with architecture, adapter capacity, data, and seeds fixed, switching the training objective from empirical risk minimization to meta-learning for domain generalization reorganizes the loss-relevant geometry of LoRA ad

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:03 UTC pith:HJ7EO43B

load-bearing objection Useful descriptive diagnostic for comparing training objectives in LoRA adapters, with a very consistent q/k-concentrate/out-distribute pattern; the 'loss-relevant' label rests on an unvalidated diagonal Fisher, and the paper overpromises a correlation analysis it never delivers. the 5 major comments →

arxiv 2607.22010 v1 pith:HJ7EO43B submitted 2026-07-24 eess.AS

How Meta-Learning Shapes LoRA Adapter Geometry in Speech Deepfake Detection

Ivan Kukanov , Janne Laakkonen , Ville Hautam\"aki This is my paper
classification eess.AS
keywords meta-learningdomain generalizationlow-rank adaptation (LoRA)speech deepfake detectionempirical Fishereffective rankadapter geometryFisher-weighted importance
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Speech deepfake detection must generalize to generators never seen in training. Prior work showed that meta-learning for domain generalization (MLDG) outperforms standard empirical risk minimization (ERM) when both train low-rank (LoRA) adapters on the same frozen speech encoder. This paper asks what the objective changes inside the adapter. Weighting each adapter coordinate by its empirical-Fisher loss sensitivity, the authors find that MLDG concentrates the loss-relevant update in the query and key attention projections and distributes it in the output projection, with the same pattern on all six evaluation corpora and all five seeds, and strongest in the upper layers. The pattern also survives merging the low-rank factors, so it is a property of the effective update, not the factorization. The paper's central claim is that the MLDG gain is a geometric reorganization of capacity, not just a better error rate.

Core claim

The central claim is that the training objective, not the adapter's capacity, leaves a measurable, projection-specific fingerprint in a finished LoRA adapter. Using the diagonal empirical Fisher to weight the squared displacement of each LoRA coordinate, the paper computes the effective rank of the resulting importance distribution separately for the query, key, value, and output projections of every attention layer. MLDG lowers the effective rank of the query projection by 19.4% and the key projection by 19.3% relative to ERM, meaning the loss-relevant update is carried by fewer coordinates, while raising it by 29.4% for the output projection, meaning the update spreads across more coordina

What carries the argument

The engine of the diagnostic is the Fisher-weighted importance pi_p = F_p (Delta phi_p)^2 / sum_q F_q (Delta phi_q)^2, where F is the diagonal empirical Fisher computed on the finished adapter over a balanced probe set, and Delta phi is the displacement of the LoRA parameters from initialization. This separates where the adapter moved (Delta phi^2) from where those moves matter to the loss (F). The concentration of the importance distribution is summarized by the effective rank RankME_F, the exponential of the entropy; a lower value means fewer coordinates carry the loss-relevant update. A second, data-free check uses the stable rank of the merged update Delta W = BA, which is invariant to t

Load-bearing premise

The load-bearing premise is that the diagonal empirical Fisher, computed on the finished adapter, is an adequate proxy for how much the loss depends on each LoRA coordinate; if that curvature estimate misses cross-coordinate structure, the reported concentration pattern could be an artifact of the approximation.

What would settle it

Recompute the effective-rank diagnostics on at least one checkpoint using a full empirical Fisher or K-FAC curvature estimate instead of the diagonal; if the query/key concentration and output dispersion disappear, the central claim fails. Alternatively, a causal intervention, such as erasing the low-importance query/key directions from an ERM adapter and observing no out-of-domain improvement, would undercut the interpretation.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, the ERM-to-MLDG out-of-distribution gap is explained by a reallocation of loss-relevant adapter capacity, not by more parameters.
  • Fisher-weighted effective rank can serve as a descriptive signature that distinguishes training objectives on the same architecture and data.
  • The depth-resolved pattern, with query/key concentration sharpening in upper layers and output-projection distribution in lower layers, gives a testable account of where domain-invariant attention routing is learned.
  • The perturbation-sensitivity result implies the MLDG adapter sits in a flatter, less locally brittle region of parameter space, which can be checked in other domains.
  • The diagnostic transfers to any pair of objectives and any low-rank adaptation parameterization, inside or outside speech.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One could test causality directly: take an ERM adapter, project its query/key updates onto the top Fisher-weighted directions, or prune the low-importance directions, and see whether out-of-domain accuracy rises toward MLDG levels.
  • The same Fisher-weighted-rank comparison could be run with a full Fisher or K-FAC curvature estimate on one checkpoint; if the query/key concentration and output dispersion persist, the diagonal approximation is vindicated, and if not, the paper's central readout is an artifact of the coordinate basis.
  • If the pattern is causal, it suggests a cheap training-free selection rule: among candidate adapters, prefer ones with low query/key and high output effective rank on a small probe set.
  • The geometric signature might also appear in other generalization-enhancing objectives such as adversarial training or consistency regularization, linking the finding to a broader program of characterizing robust solutions by their geometry.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a descriptive diagnostic for comparing how different training objectives shape LoRA adapters. Holding architecture, rank, data, and seeds fixed, it weights the LoRA displacement Δφ by a diagonal empirical Fisher estimate and summarizes the resulting importance distribution with an effective rank (RankME_F). Applied to ERM- vs MLDG-trained adapters in a frozen Wav2Vec 2.0–AASIST speech deepfake detector, the diagnostic reports that MLDG concentrates the Fisher-weighted update in the query and key projections (effective rank −19.4% and −19.3%) and distributes it in the output projection (+29.4%), with 30/30 dataset-seed consistency for q/k/out, stronger q/k concentration in upper layers, and a parallel stable-rank pattern in the merged update ΔW. A perturbation experiment on one corpus shows the MLDG adapter's score function is less locally sensitive. The paper also includes a speculative mechanistic section (V.D) and a discussion that correctly labels the causal link as unestablished.

Significance. If the central diagnostic is accepted, the paper offers a useful descriptive tool for characterizing training objectives through solution geometry rather than error rates alone. Its strengths are the controlled comparison (only the objective varies), the unusually high consistency (five seeds × six corpora = 30/30 cells for the main q/k/out effects), the explicit decomposition of raw update magnitude versus Fisher-weighted importance, and the merged-update check that guards against factor-coordinate artifacts. The authors are also appropriately cautious in the Discussion about causal interpretation. However, the central 'loss-relevant' claim rests entirely on an unvalidated diagonal empirical Fisher approximation, and the paper promises but never delivers a correlation with out-of-domain metrics. These issues make the current evidence insufficient to support the full narrative, though they are addressable within the manuscript's scope.

major comments (5)
  1. [§III-B, Eqs. (5)–(6)] The central claim that MLDG concentrates the *loss-relevant* update in q/k depends entirely on the diagonal empirical Fisher F̂_p as a proxy for loss sensitivity. The paper cites [22] on limitations of the empirical Fisher but provides no validation that the π-ranking corresponds to actual loss changes. Because φ0 (B=0) is not a stationary point, the quadratic approximation drops the first-order term; because only the diagonal is kept, cross-coordinate curvature is ignored. The consistency counts (30/30) show the approximation is stable across datasets/seeds, not that it measures loss relevance. Please add a direct check—e.g., compare the loss change under perturbations along top-π coordinates vs random/bottom-π coordinates, or compare against a block-diagonal/K-FAC estimate. Without this, the q/k concentration could be an artifact of the importance estimator.
  2. [§III-C and §V] Section III-C promises: 'We report ... and then correlate these diagnostics with out-of-domain detection metrics.' No such correlation appears in the results. Since the motivation is the MLDG OOD gain, the absence of any quantitative link between the geometry contrast and OOD accuracy leaves the claimed 'why MLDG generalizes better' narrative unsupported. Add the correlation (e.g., per-seed RankME_F vs OOD EER/ACC, with confidence bounds) or explicitly remove the promise and label the generalization link as conjecture.
  3. [§V.D] This section asserts a mechanistic explanation: 'That trade ... is the mechanistic reason MLDG transfers better.' No causal experiment supports this; the Discussion (Sec. VI) correctly says the causal link is not established. As written, V.D overstates and conflicts with the paper's own caveat. Reframe as hypotheses and, if possible, test by intervening (e.g., training with a regularizer that encourages q/k concentration or out-projection spread and measuring OOD performance).
  4. [§V.B, Fig. 3] The stable-rank result on the merged update ΔW is a raw-magnitude measure; it contains no Fisher weighting. It therefore corroborates that the projection-wise reorganization is not an artifact of the A/B factorization, but it does not corroborate the 'loss-relevant' characterization. The text should not present Fig. 3 as independent evidence for the Fisher-weighted claim; only for the magnitude geometry. Note also that Fig. 2 shows the magnitude-only measure moves in the opposite direction on q/k, which underscores that the two measures are distinct.
  5. [§V.A, Tables I–III] The paper reports no confidence intervals or significance tests. The phrase 'no significant global difference' in V.A is unsupported without a test. Sign-consistency counts are useful but do not convey effect-size uncertainty. Provide variability (e.g., bootstrap CIs over the 30 cells, or per-seed distributions) at least for the aggregate q/k/out effects and for the depth rows in Table II.
minor comments (6)
  1. [§III.A, Eq. (4)] The paper says the experiments use first-order MLDG, but Eq. (4) is written as the full objective. Clarify whether the gradient of L_te with respect to φ is stopped at the inner-update argument, as in first-order MAML.
  2. [Tables I–III] Table headers contain typos and unclear notation: 'Seeds agree COUNTS SIGN-CONSISTENT SEEDS' and 'Consistency IS THE DESCRIPTIVE SEED×DATASET COUNT.' Also define what 'joint' means in Table II (all projections combined? a specific concatenation?).
  3. [§III.C] The protocol lists 'Fisher alignment, domain-wise Fisher stability, and output-space effective rank' but only RankME_F is defined and used. Either define these terms or remove them from the protocol description.
  4. [Eq. (9)] The log-offset ε in RankME_F is not given a numerical value. Report ε and, ideally, a brief sensitivity check (e.g., ε ∈ {1e-12, 1e-8, 1e-4}) since the effective rank is computed on normalized importances.
  5. [§IV.A] The analysis uses checkpoints from [1] but no code or data-release link is provided. If available, include a reproducibility statement; if not, state the computational budget and expected runtime so the reader can gauge reproducibility.
  6. [Fig. 5] The x-axis label 'Layer' appears twice, and the panel layout is hard to read because all six corpora are overlaid. Consider faceting by corpus or using a small-multiple layout with shared y-limits.

Circularity Check

0 steps flagged

No significant circularity: the geometry diagnostic is applied post hoc to frozen checkpoints and no parameter is fitted to produce the reported pattern.

full rationale

The paper's central claim is descriptive: after fixing architecture, rank, data, and seeds, ERM and MLDG LoRA adapters differ in the distribution of a Fisher-weighted update measure. The measure is defined in Eqs. (5)-(6) and computed post hoc on frozen checkpoints taken from the authors' prior work [1]. No parameter is fitted to data to produce the q/k-concentration pattern, and no 'prediction' is derived from a fitted value: the effective-rank statistic (Eq. 9) is applied to the same π distribution on both objectives, and the comparison is observational. The merged-update stable-rank result (Fig. 3) is independent of both the Fisher estimate and the LoRA factor coordinates, providing a check that the projection-level reorganization is not an artifact of the factorization. The only notable limitation is that the diagonal empirical Fisher (Eq. 5) is an approximation of loss curvature and is not validated against the full Fisher or K-FAC; however, the authors explicitly describe it as an approximation and cite [22], and this is a validity/robustness concern, not a circular reduction. Self-citation to [1] supplies the checkpoints and the established OOD gain, but the geometry claim is not derived from that gain nor from any uniqueness theorem; it is an independent, applied measurement. Therefore no circular step is exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central claim rests entirely on the descriptive Fisher-weighted effective rank and on checkpoints from the authors' own prior work. No new physical or mathematical entities are postulated. The main upstream choices are LoRA rank/scaling and the empirical Fisher approximation, neither of which is validated in this paper.

free parameters (4)
  • LoRA rank r = 16
    Fixed adapter capacity carried from [1]; no rank sweep, so the pattern's rank-dependence is untested.
  • LoRA scaling alpha_LoRA = 2
    Fixed from [1]; scales the merged update and could influence the measured geometry.
  • Empirical Fisher probe set size = 1000 bonafide + 1000 spoof per corpus
    Chosen for class balance; not swept, affects variance and quality of the diagonal Fisher estimate.
  • Effective-rank log offset epsilon = unspecified small constant
    Introduced in Eq. (9) to avoid log(0); its value is not reported and could affect RankME_F.
axioms (6)
  • domain assumption Empirical Fisher diagonal approximates loss curvature over LoRA parameters
    Eq. (5); acknowledged as an approximation, with [22] noting limitations, but no validation against full Fisher or other curvature estimates.
  • domain assumption Importance pi = F DeltaPhi^2, normalized, captures loss-relevant update concentration
    Eq. (6); central to interpretation; no alternative weighting is compared.
  • domain assumption Frozen Wav2Vec2-AASIST with LoRA rank 16 is a representative testbed
    Sec IV.A; the fixed testbed is inherited from [1].
  • domain assumption Checkpoints from [1] were trained correctly under ERM and MLDG
    Self-cited prior work; this paper does not retrain or independently verify those checkpoints.
  • standard math Effective rank (entropy of the normalized importance distribution) is a meaningful concentration measure
    Eq. (9); standard entropy-based effective rank from [11], [12], [28]-[31].
  • domain assumption Six evaluation corpora and five seeds are representative of out-of-distribution shift
    Sec IV.C and V; no power analysis, and consistency is reported descriptively.

pith-pipeline@v1.3.0-alltime-deepseek · 10678 in / 12804 out tokens · 134060 ms · 2026-08-01T06:03:53.223610+00:00 · methodology

0 comments
read the original abstract

Meta-learning for domain generalization (MLDG) improves out-of-distribution speech deepfake detection over empirical risk minimization (ERM) when both objectives train low-rank adapters on the same frozen self-supervised speech model. Because the architecture and adapter capacity are held fixed, this gap points to differences in how the training objective shapes the adapter, yet the field characterizes objectives through error rates rather than through the geometry of the solution they reach. We introduce a descriptive diagnostic for this question: holding architecture, rank, data, and seeds fixed and varying only the objective, we use the empirical Fisher on the finished adapter to compare the geometry that ERM and MLDG leave behind. We characterize each adapter with effective-rank diagnostics that separate where the adapter changes from where those changes matter to the loss, resolved by projection and by depth. Applied to ERM and MLDG, the diagnostic shows that the objective does not reshape all adapter projections alike: the loss-relevant update concentrates in the query and key projections while becoming more distributed in the output projection, consistently across six corpora and most strongly in the upper layers. The same contrast appears in the merged update independently of the low-rank factorization, indicating that it reflects the geometry of the effective update rather than the parameterization. These results show that the gap between ERM and MLDG is not only a difference in error rate, but a difference in how loss-relevant capacity is organized inside the adapter, and that loss-aware adapter geometry is a way to see it.

Figures

Figures reproduced from arXiv: 2607.22010 by Ivan Kukanov, Janne Laakkonen, Ville Hautam\"aki.

Figure 1
Figure 1. Figure 1: ERM- vs. MLDG-trained LoRA adapters in a frozen Wav2Vec [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Per-module effective-rank change under MLDG versus ERM, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Stable rank of the merged update ∆W = αAB, 100 × (MLDG − ERM)/ERM. This measure uses neither the Fisher nor any data and is invariant to the LoRA factor coordinates; values are means over five seeds. B. Decomposing the Fisher-Weighted Effect The per-module RankMEF contrast combines two factors: how much the adapter moved and how much the loss depends on where it moved. To separate them, we recomputed the e… view at source ↗
Figure 4
Figure 4. Figure 4: LoRA perturbation sensitivity on In-The-Wild. Random perturbations [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effect of MLDG vs ERM training, measured using RankME [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 15 linked inside Pith

  1. [1]

    Generalizable speech deepfake detection via meta-learned lora,

    J. Laakkonen, I. Kukanov, and V . Hautam ¨aki, “Generalizable speech deepfake detection via meta-learned lora,” inICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2026, pp. 19 092–19 096

    2026

  2. [2]

    Does Audio Deepfake Detection Generalize?

    N. M ¨uller, P. Czempin, F. Diekmannet al., “Does Audio Deepfake Detection Generalize?” inInterspeech 2022, 2022, pp. 2783–2787

    2022

  3. [3]

    Domain generalization: A survey,

    K. Zhou, Z. Liu, Y . Qiaoet al., “Domain generalization: A survey,”

  4. [4]

    Learning to learn: Introduction and overview,

    S. Thrun and L. Pratt, “Learning to learn: Introduction and overview,” inLearning to learn. Springer, 1998, pp. 3–17

    1998

  5. [5]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” inInternational conference on machine learning. PMLR, 2017, pp. 1126–1135

    2017

  6. [6]

    Meta-learning approaches for improving detection of unseen speech deepfakes,

    I. Kukanov, J. Laakkonen, T. Kinnunenet al., “Meta-learning approaches for improving detection of unseen speech deepfakes,” in2024 IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 1173–1178

    2024

  7. [7]

    Learning to generalize: Meta-learning for domain generalization,

    D. Li, Y . Yang, Y . Songet al., “Learning to generalize: Meta-learning for domain generalization,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/11596

    2018

  8. [8]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Walliset al., “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Representations, 2022, iCLR 2022 Poster. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  9. [9]

    Principles of risk minimization for learning theory,

    V . N. Vapnik, “Principles of risk minimization for learning theory,” inNeural Information Processing Systems, 1991. [Online]. Available: https://api.semanticscholar.org/CorpusID:15348764

    1991

  10. [10]

    New insights and perspectives on the natural gradient method,

    J. Martens, “New insights and perspectives on the natural gradient method,”Journal of Machine Learning Research, vol. 21, no. 146, pp. 1–76, 2020

    2020

  11. [11]

    The effective rank: A measure of effective dimensionality,

    O. Roy and M. Vetterli, “The effective rank: A measure of effective dimensionality,” inEUSIPCO. IEEE, 2007, pp. 606–610

    2007

  12. [12]

    Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank,

    Q. Garrido, R. Balestriero, L. Najmanet al., “Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank,” inICML. PMLR, 2023, pp. 10 929–10 974

    2023

  13. [13]

    Intrinsic dimensionality explains the effectiveness of language model fine-tuning,

    A. Aghajanyan, S. Gupta, and L. Zettlemoyer, “Intrinsic dimensionality explains the effectiveness of language model fine-tuning,” inACL- IJCNLP, 2021, pp. 7319–7328

    2021

  14. [14]

    Adaptive budget allocation for parameter-efficient fine-tuning,

    Q. Zhang, M. Chen, A. Bukharinet al., “Adaptive budget allocation for parameter-efficient fine-tuning,” inICLR, 2023

    2023

  15. [15]

    Pissa: Principal singular values and singular vectors adaptation of large language models,

    F. Meng, Z. Wang, and M. Zhang, “Pissa: Principal singular values and singular vectors adaptation of large language models,”NeurIPS, vol. 37, pp. 121 038–121 072, 2024

    2024

  16. [16]

    Lora vs full fine-tuning: An illusion of equivalence,

    R. Shuttleworth, J. Andreas, A. Torralbaet al., “Lora vs full fine-tuning: An illusion of equivalence,”NeurIPS, vol. 38, pp. 174 627–174 662, 2026

    2026

  17. [17]

    Asymmetry in low-rank adapters of foundation models,

    J. Zhu, K. Greenewald, K. Nadjahiet al., “Asymmetry in low-rank adapters of foundation models,” inICML, 2024

    2024

  18. [18]

    Lora learns less and forgets less,

    D. Biderman, J. Portes, J. J. G. Ortizet al., “Lora learns less and forgets less,”arXiv preprint arXiv:2405.09673, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Riemannian preconditioned lora for fine- tuning foundation models,

    F. Zhang and M. Pilanci, “Riemannian preconditioned lora for fine- tuning foundation models,”arXiv preprint arXiv:2402.02347, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Learning in the fisher sub- space: A guided initialization for lora fine-tuning,

    Z.-Q. Feng, Y .-J. Lin, and H.-Y . Kao, “Learning in the fisher sub- space: A guided initialization for lora fine-tuning,”arXiv preprint arXiv:2605.01046, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    Curvature-guided lora: Steering in the pretrained ntk subspace,

    F. Zheng and A. Prouti `ere, “Curvature-guided lora: Steering in the pretrained ntk subspace,”arXiv preprint arXiv:2603.29824, 2026

    Pith/arXiv arXiv 2026

  22. [22]

    Limitations of the empirical fisher approximation for natural gradient descent,

    F. Kunstner, P. Hennig, and L. Balles, “Limitations of the empirical fisher approximation for natural gradient descent,”NeurIPS, vol. 32, 2019

    2019

  23. [23]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamedet al., “wav2vec 2.0: A framework for self-supervised learning of speech representations,”Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020

    2020

  24. [24]

    Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks,

    J.-w. Jung, H.-S. Heo, H. Taket al., “Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks,” inICASSP 2022- 2022 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2022, pp. 6367–6371

    2022

  25. [25]

    New insights and perspectives on the natural gradient method,

    J. Martens, “New insights and perspectives on the natural gradient method,” 2014. [Online]. Available: https://arxiv.org/abs/1412.1193

    Pith/arXiv arXiv 2014

  26. [26]

    Optimizing neural networks with kronecker-factored approximate curvature,

    J. Martens and R. Grosse, “Optimizing neural networks with kronecker-factored approximate curvature,” inProceedings of the 32nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, F. Bach and D. Blei, Eds., vol. 37. PMLR, 2015, pp. 2408–2417. [Online]. Available: https://proceedings.mlr.press/v37/martens15.html

    2015

  27. [27]

    An improved empirical fisher approximation for natural gradient descent,

    X. Wu, W. Yu, C. Zhanget al., “An improved empirical fisher approximation for natural gradient descent,” 2024. [Online]. Available: https://arxiv.org/abs/2406.06420

    Pith/arXiv arXiv 2024

  28. [28]

    An investigation into neural net optimization via hessian eigenvalue density,

    B. Ghorbani, S. Krishnan, and Y . Xiao, “An investigation into neural net optimization via hessian eigenvalue density,” inProceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 2019, pp. 2232–2241. [Online]. Available: https://proceedings....

    2019

  29. [29]

    Neural tangent kernel: Convergence and generalization in neural networks,

    A. Jacot, F. Gabriel, and C. Hongler, “Neural tangent kernel: Convergence and generalization in neural networks,” inAdvances in Neural Information Processing Systems 31. Curran Associates, Inc.,

  30. [30]

    RankMe: Assessing the downstream performance of pretrained self- supervised representations by their rank,

    Q. Garrido, R. Balestriero, L. Najmanet al., “RankMe: Assessing the downstream performance of pretrained self- supervised representations by their rank,” 2022. [Online]. Available: https://arxiv.org/abs/2210.02885

    Pith/arXiv arXiv 2022

  31. [31]

    High-dimensional SGD aligns with emerging outlier eigenspaces,

    G. B. Arous, R. Gheissari, J. Huanget al., “High-dimensional SGD aligns with emerging outlier eigenspaces,” 2023. [Online]. Available: https://arxiv.org/abs/2310.03010

    Pith/arXiv arXiv 2023

  32. [32]

    ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection,

    M. Todisco, X. Wanget al., “ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection,” inInterspeech, 2019

    2019

  33. [33]

    ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection,

    J. Yamagishi, X. Wang, M. Todiscoet al., “ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection,” 2021. [Online]. Available: https://arxiv.org/abs/2109.00537

    Pith/arXiv arXiv 2021

  34. [34]

    ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale,

    X. Wang, H. Delgadoet al., “ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale,”ArXiv, vol. abs/2408.08739, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    FakeA VCeleb: A novel audio-video multimodal deepfake dataset,

    H. Khalid, S. Tariq, M. Kimet al., “FakeA VCeleb: A novel audio-video multimodal deepfake dataset,” 2021. [Online]. Available: https://arxiv.org/abs/2108.05080

    Pith/arXiv arXiv 2021

  36. [36]

    Audio deepfake detection: A survey,

    J. Yi, C. Wang, J. Taoet al., “Audio deepfake detection: A survey,”

  37. [37]

    Robust wav2vec 2.0: Analyzing domain shift in self-supervised pre-training,

    W. Hsu, A. Sriram, A. Baevskiet al., “Robust wav2vec 2.0: Analyzing domain shift in self-supervised pre-training,” 2021. [Online]. Available: https://arxiv.org/abs/2104.01027

    Pith/arXiv arXiv 2021

  38. [2018]

    Available: https://arxiv.org/abs/1806.07572

    [Online]. Available: https://arxiv.org/abs/1806.07572

    Pith/arXiv arXiv

  39. [2021]

    Available: https://arxiv.org/abs/2103.02503

    [Online]. Available: https://arxiv.org/abs/2103.02503

    Pith/arXiv arXiv

  40. [2023]

    Available: https://arxiv.org/abs/2308.14970

    [Online]. Available: https://arxiv.org/abs/2308.14970

    Pith/arXiv arXiv