REVIEW 5 major objections 6 minor 35 references
DMDIntel: Interpreting Large Language Models via Dynamic Mode Decomposition
T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Dynamic mode decomposition of a fine-tuned LLM's token-by-token hidden-state trajectory can rank the input tokens that drive a classification decision, outperforming PCA, integrated gradients, and SHAP on the paper's benchmarks.
desk verdict DMDIntel is a genuinely novel single-pass attribution method, but its 'by far outperforms' claim is unsupported by its own tables and the evaluation needs substantial cleanup before the results can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the eigendecomposition of a best-fit linear operator on token-indexed hidden states. Treating the sequence of down-projection MLP outputs as $h_{t+1} \approx A h_t$, the method builds a snapshot matrix, removes the instruction bias, and (for HODMD) stacks delayed snapshots into a Hankel matrix before an SVD-based fit; the eigenmodes $\phi_i$ of $A$ are the low-dimensional spatio-temporal patterns, and their eigenvalues encode growth, decay, and oscillation. Modes are ranked by initial amplitude or by time-averaged amplitude, and each token's state change $\Delta h_t = h_t - h_{t-1}$ is projected onto the top-$k$ modes, with the sum of absolute projections giving the token's attribution score. A layer-selection heuristic picks the decoder layer at the steepest ascent of the average cosine similarity between token vectors and the instruction-bias vector below a threshold of 0.25, which is meant to balance input-token fidelity against instruction-following bias.
What would settle it
Score the same attribution methods against two alternative ground truths: human annotator rankings on the paper's own samples, and causal ground truth from token-deletion or token-perturbation experiments. If DMDIntel's advantage over PCA, integrated gradients, and SHAP shrinks, reverses, or disappears under either alternative, the central claim is refuted; the paper's reported human-to-annotator-model RBO of 0.54-0.70 makes this check decisive. A cheaper version: mask the top-k tokens from DMDIntel, from a random baseline, and from the annotator model, and compare accuracy drops—if DMDIntel's drops are indistinguishable from random, the attribution is not faithful.
Extended reading notes
Core claim
The central claim is that the dominant DMD/HODMD modes of the debiased hidden-state trajectory serve as reliable indicators of which input tokens drive a fine-tuned LLM's classification output, and that ranking tokens by projection onto these modes outperforms PCA, integrated gradients, and SHAP on the paper's matched-count, rank-biased overlap, and recall-at-k metrics. The choice of decomposition is task-dependent: HODMD with time-averaged amplitude ranking wins on sentiment, where meaning is spread across multi-word phrases, while standard DMD with initial-amplitude ranking wins on HateXplain and FakeEdit, where single trigger words dominate. The paper also claims that the linear surrogate is accurate—reconstruction error is of order $10^{-2}$—and that the attribution is stable under one-layer shifts of the selected decoder layer. A token-masking fidelity experiment shows that removing DMDIntel's top tokens lowers accuracy and confidence at rates comparable to the stronger baselines.
Load-bearing premise
The load-bearing premise is that the external annotator model's token rankings are a valid proxy for what truly drives the fine-tuned classifier's decision; the paper's human agreement with that proxy is only moderate (RBO 0.54-0.70), and no experiment causally verifies that the ranked tokens are the ones whose removal changes the prediction.
Editorial extensions
If this is right
- If DMDIntel's attributions are faithful, token-level explanations for fine-tuned decoder-style classifiers can be produced non-intrusively, with a single forward pass, avoiding the many forward and backward passes that gradient and perturbation methods require.
- The task-dependent winner between standard DMD and HODMD suggests that the appropriate temporal memory length is a property of the task: localized single tokens call for near-Markovian dynamics, while phrase-level sentiment calls for delay embedding.
- The layer-selection heuristic and instruction-bias subtraction generalize across the three model families tested, indicating the pipeline may transfer to other fine-tuned decoder-only models without per-model tuning.
- Layer-sensitivity results, with maximum recall deviation around 0.02 across $\ell^*\pm 1$, imply the method does not rest on a fragile choice of decoder layer.
Reading between the lines
- Because the paper scores against one annotator model's rankings, the strongest next test is to score against direct human rankings or against causal tests such as deletion of the attributed tokens; the reported human-to-annotator-model agreement of RBO 0.54-0.70 suggests the two standards could disagree on which method wins.
- The low linear-reconstruction error of the surrogate operator implies DMD modes could support counterfactual experiments, such as predicting the hidden-state trajectory after deleting a token, which would turn the attribution ranking into a directly testable causal claim.
- The pipeline is framed for single-token-output classification; extending it to multi-token generation, as the authors note, requires a different snapshot formulation because generated tokens re-enter the input auto-regressively, and that extension would determine whether dynamical attribution works beyond classification.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces DMDIntel, an input-attribution method for fine-tuned decoder-only LLMs. It extracts per-token hidden states from the MLP down_proj of a selected layer, subtracts an instruction-bias vector obtained from an empty prompt, applies DMD or HODMD to the debiased hidden-state trajectory, and ranks tokens by their projected scores onto the dominant modes. The method is evaluated on sentiment, HateXplain, and FakeEdit classification across Llama-3.2-3B, Qwen3-4B, and Mistral-7B, with PCA, IG, and SHAP as baselines. The paper claims that DMDIntel consistently identifies the highest proportion of ground-truth tokens and preserves their ranked importance, outperforming the baselines by a large margin. It also includes a token-masking fidelity analysis, layer-selection sensitivity experiments, ablations, and a reconstruction-error analysis.
Significance. The dynamical-systems perspective on hidden-state trajectories is a genuinely novel angle for decoder-only LLM attribution, and the single-forward-pass property makes the method practically attractive if its claims hold. The paper is also useful in its breadth of ablations: layer sensitivity (Table 7), denoising (Appendix H.1), threshold choice (Appendix H.2), top-k choice (Appendix H.3), and mode count (Appendix H.4) are all examined, and the reconstruction-error analysis in Appendix F is a constructive check. However, the evaluation as designed primarily measures agreement with GPT-4.1-generated rankings rather than faithfulness to the fine-tuned model, and the paper's own causal fidelity results do not support the strong headline. The core method is coherent and worth further study, but the manuscript's central comparative claims need substantial revision.
major comments (5)
- [Abstract; Section 1; Tables 3-4] The claim that DMDIntel 'by far outperforms' PCA, IG, and SHAP is not supported by the reported numbers. In Table 4 (Mistral-7B-v0.3-inst, HateXplain), PCA beats DMDIntel on all three metrics (MC 2.95 vs. 2.82, RBO 0.30 vs. 0.26, Recall 0.67 vs. 0.65). In Table 3 (Qwen3-4B-inst, Negative Sentiment), PCA has a higher matched count (5.46 vs. 5.43) and equal recall, and the RBO advantage for DMDIntel is only 0.01. These are marginal or reversed differences, not the 'by far outperforms' stated in the abstract and introduction. The claims should be rewritten to match the actual effect sizes, or the evaluation should be extended to demonstrate a robust advantage.
- [Section 5; Section 6.4; Table 6] The ranking metrics are computed against GPT-4.1-generated token importance rankings whose human agreement is only RBO 0.54-0.70 (Section 5). This measures alignment with another model's opinion, not faithfulness to the fine-tuned LLM's decision. The masking experiment in Section 6.4, Table 6, is the only causal check, and it contradicts the 'main drivers' language: for Qwen3-4B-inst Negative Sentiment, DMDIntel's accuracy drop is 10.78 versus IG 15.77 and PCA 12.68; for Llama-3.2-3B-inst on HateXplain hateful, DMDIntel drops 45.50 versus IG 50.66 and PCA 56.88. In these and other cells DMDIntel ranks third, contrary to the text's statement that it 'consistently ranks first or second.' The causal evidence should be made the primary evaluation, and the abstract and introduction should be reconciled with these results.
- [Section 6; Appendix D] The main results report the best of four DMD configurations (DMD or HODMD crossed with amplitude-based or time-averaged-amplitude mode ranking), and the best variant changes by dataset and model. This post hoc selection, without a validation split or correction for multiple comparisons, inflates the reported performance. The full comparison in Appendix D shows many configurations within 0.01-0.02 of each other. The chosen variant should be justified a priori, or the full variant table should be in the main text and treated as a sensitivity analysis rather than as the method's default performance.
- [Section 6; Tables 3-4] No statistical significance tests, confidence intervals, or repeated-run variability are reported, yet differences as small as 0.01 in RBO are treated as evidence of superiority. Since each evaluation uses 1,000 samples per dataset and the metrics are averages, the paper should provide bootstrap confidence intervals or multiple fine-tuning seeds to establish that the observed advantages are not noise. This is especially important because several headline comparisons in Tables 3 and 4 are within 0.01-0.03 on RBO and recall.
- [Appendix I] The paper itself concedes that 'identifying label-intensive tokens does not imply causal dependency' and that non-labeled contextual tokens may be relevant to the decision trajectory. This directly limits the Introduction's claim that the identified tokens are 'the main drivers' of the model's input processing. The abstract and introduction should be revised to reflect this limitation, and the fidelity-by-masking analysis should be presented as the appropriate test of the causal claim rather than as a secondary check.
minor comments (6)
- [Table 15] In Table 15, the Llama-3.2-3B-inst FakeEdit row reports RBO@20 = 3.43 for DMDIntel, which is outside the valid [0,1] range and appears to be a typo for 0.34.
- [Abstract; Section 1] The method name is formatted inconsistently as 'DMDINTEL' and 'DMDIntel', and there are spacing errors such as 'DMDINTELwhichuses' in the abstract; these should be cleaned up.
- [Section 5, Eq. (8)] The RBO formula as written should be checked: the term X_k/k * p^k is not the standard RBO extrapolated agreement, and the expression as shown is not obviously bounded between 0 and 1.
- [References] The GPT-4.1 ground truth is attributed to Achiam et al. (2023), which is the GPT-4 technical report; a GPT-4.1-specific citation or model-card reference should be provided.
- [Section 4.3; Appendix H.2] Algorithm 1 hard-codes the cosine threshold tau = 0.25, and the sensitivity analysis only tests tau = 0.5; a small sweep or a data-driven justification for the fixed value would make the layer-selection heuristic more robust.
- [Appendix F] The reconstruction errors in Table 9 range up to 0.132, which is described as 'quite accurate'; a comparison against the reconstruction error of PCA or a random subspace would help calibrate this claim.
Circularity Check
The central comparison is partly circular: DMDIntel's reported results are the best of four variants selected on the same test metrics; the DMD mathematics itself is a standard fit and the GPT-4.1 ground truth is not definitionally circular.
-
fitted input called prediction
[Section 6 (Dataset based results), preceding Section 6.1; Tables 3-4 and Appendix D Table 8]
"In each setting (model + dataset), we report the metrics obtained for the best performing combination above alongside the baselines (see Appendix D for full results)."
The four DMD/HODMD and ranking variants are all evaluated on the same test sentences, and the variant with the highest reported metric is then labeled as DMDIntel for that setting. Thus the reported DMDIntel numbers are, by construction, the maximum over four configurations. The paper's headline claim that DMDIntel 'consistently identifies the highest proportion of ground-truth tokens' is therefore not a prediction of a fixed method; it is an in-sample selection of the best variant. The variant choice is fitted to the evaluation metric and then presented as the method's performance, which is the fitted-input-called-prediction pattern.
-
fitted input called prediction
[Appendix H.4 (Ablation studies), referring to Table 17]
"We examine the effect of mode subspace dimensionality by projecting token representations onto the top-7 modes compared to top-5 in our original setup. As reported in Table 17, selecting additional modes leads to a noticeable drop in performance metrics for DMDINTEL."
k=5 is not derived from the DMD theory; it is chosen after comparing top-5 vs top-7 on the same test metrics, and the chosen value is then used in all experiments. Similarly, Appendix H.2 tunes the cosine threshold tau (0.25 vs 0.5) against the reported metrics. These hyperparameters are selected using the same ground-truth comparisons that are later used as evidence of superiority, so the evaluation does not independently test the method's configuration.
full rationale
The paper's claimed derivation chain is: hidden states -> debiased data matrix -> DMD/HODMD modes -> token projection scores -> ranking -> comparison to GPT-4.1 GT and to fidelity masking. The DMD/HODMD step is a standard linear fit (Eqs. 9-14 and 15-21) and is not equivalent to the evaluation metric; no load-bearing self-citations exist, and GPT-4.1 is used only to produce ground-truth rankings, not inside DMDIntel, so that anchoring is a validity concern rather than a definitional circularity. The concrete circularity is in the evaluation protocol: the paper evaluates four configurations (DMD/HODMD x amplitude/time-averaged amplitude) and, per model and dataset, reports only the best-performing one. The choice is made after observing the same metrics that are then used to claim consistent superiority, so the 'which configuration works best' statement is a selection rather than a prediction. Appendix H shows that k=5 and tau=0.25 were likewise validated by comparing metrics on the same test sets. This fitted-input-called-prediction pattern makes the reported comparison partially circular. Separately, the paper's own Table 6 shows DMDIntel ranked third in several accuracy-drop cells, contradicting the text's 'consistently first or second'; Appendix I concedes that identifying label-intensive tokens does not imply causal dependency, and human agreement with the GPT-4.1 ground truth is only RBO 0.54-0.70. Those are correctness and external-validity problems, not additional circularity, but they reinforce that the headline claim is weaker than the evidence presented.
Assumptions & free parameters
free parameters (4)
- Cosine similarity threshold tau =
0.25
- Number of modes k =
5
- DMD configuration per dataset =
HODMD-avgamp for Sentiment; DMD-amp for HateXplain and FakeEdit
- HODMD delay parameter d =
adaptive, value not reported
assumptions (4)
- domain assumption The hidden-state transition h_{t+1} = A h_t is a good linear approximation of the fine-tuned LLM at the selected layer.
- ad hoc to paper The layer-wise difference Delta h_t = h_t - h_{t-1} represents the contribution of token w_t.
- domain assumption Subtracting the empty-instruction bias vector b removes prompt-template noise from hidden states.
- domain assumption GPT-4.1-generated token rankings are a valid ground truth for input attribution.
Cite this review
Pith. "Pith review of DMDIntel: Interpreting Large Language Models via Dynamic Mode Decomposition." pith.science (2026). https://pith.science/paper/FVERWLMS
@misc{pith2026260813048,
author = {Pith},
title = {Pith review of: DMDIntel: Interpreting Large Language Models via Dynamic Mode Decomposition},
year = {2026},
howpublished = {\url{https://pith.science/paper/FVERWLMS}},
note = {Machine review of arXiv:2608.13048}
}
read the original abstract
In this work, we introduce DMDIntel which uses dynamic mode decomposition (DMD) to make the predictions made by LLMs in a classification task interpretable. It develops an input attribution pipeline, that first decomposes the hidden states of an LLM into prominent patterns, also known as modes, and then associates ranks to the input tokens based on the projection values on those modes. Rigorous experiments across three datasets and three model families consistently show that the ranked attribution of input tokens obtained using DMDIntel by far outperforms state-of-the-art techniques such as principal component analysis, integrated gradients and SHAP.
Figures
Reference graph
Works this paper leans on
-
[1]
Advances in Neural Information Processing Systems , volume=
Attention is all you need , author=. Advances in Neural Information Processing Systems , volume=
- [2]
-
[3]
Bert: Pre-training of deep bidirectional transformers for language understanding , author=. Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1 (long and short papers) , pages=
work page 2019
-
[4]
2018 , publisher=
Improving language understanding by generative pre-training , author=. 2018 , publisher=
2018
-
[5]
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Voita, Elena and Talbot, David and Moiseev, Fedor and Sennrich, Rico and Titov, Ivan. Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1580
-
[6]
BERT Rediscovers the Classical NLP Pipeline
Tenney, Ian and Das, Dipanjan and Pavlick, Ellie. BERT Rediscovers the Classical NLP Pipeline. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1452
-
[7]
arXiv preprint arXiv:2209.11895 , year=
In-context learning and induction heads , author=. arXiv preprint arXiv:2209.11895 , year=
-
[8]
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=
Transformer feed-forward layers are key-value memories , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=
2021
Show all 35 references
-
[9]
Advances in Neural Information Processing Systems , volume=
Locating and editing factual associations in gpt , author=. Advances in Neural Information Processing Systems , volume=
-
[10]
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=
Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=
2022
-
[11]
Journal of Fluid Mechanics , volume=
Dynamic mode decomposition of numerical and experimental data , author=. Journal of Fluid Mechanics , volume=. 2010 , publisher=
2010
-
[12]
Journal of Open Source Software , year =
Nicola Demo and Marco Tezzele and Gianluigi Rozza , title =. Journal of Open Source Software , year =. doi:10.21105/joss.00530 , url =
-
[13]
and Mezic, Igor and Bagheri, Shervin and Schlatter, Phillpp and Henningson, Dan S
Rowley, Clarence W. and Mezic, Igor and Bagheri, Shervin and Schlatter, Phillpp and Henningson, Dan S. , year=. Spectral analysis of nonlinear flows , volume=. doi:10.1017/S0022112009992059 , journal=
-
[14]
Noack , title =
Gilles Tissot and Laurent Cordier and Nicolas Benard and Bernd R. Noack , title =. Comptes Rendus. M\'ecanique , pages =. 2014 , publisher =. doi:10.1016/j.crme.2013.12.011 , language =
2014 doi
-
[15]
arXiv preprint arXiv:2407.21783 , year=
The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=
-
[16]
arXiv preprint arXiv:2505.09388 , year=
Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=
-
[17]
Advances in Neural Information Processing Systems , volume=
Character-level convolutional networks for text classification , author=. Advances in Neural Information Processing Systems , volume=
-
[18]
Proceedings of the twelfth language resources and evaluation conference , pages=
Fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection , author=. Proceedings of the twelfth language resources and evaluation conference , pages=
-
[19]
Proceedings of the AAAI conference on artificial intelligence , volume=
Hatexplain: A benchmark dataset for explainable hate speech detection , author=. Proceedings of the AAAI conference on artificial intelligence , volume=
-
[20]
International Conference on Machine Learning , pages=
Axiomatic attribution for deep networks , author=. International Conference on Machine Learning , pages=. 2017 , organization=
2017
-
[21]
Advances in Neural Information Processing Systems , volume=
A unified approach to interpreting model predictions , author=. Advances in Neural Information Processing Systems , volume=
-
[22]
Higher Order Dynamic Mode Decomposition , journal =
Le Clainche, Soledad and Vega, Jos\'. Higher Order Dynamic Mode Decomposition , journal =. 2017 , doi =. https://doi.org/10.1137/15M1054924 , abstract =
2017 doi
-
[23]
arXiv preprint arXiv:2308.09124 , year=
Linearity of relation decoding in transformer language models , author=. arXiv preprint arXiv:2308.09124 , year=
-
[24]
arXiv preprint arXiv:2303.08774 , year=
Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=
-
[25]
ACM Trans
Webber, William and Moffat, Alistair and Zobel, Justin , title =. ACM Trans. Inf. Syst. , month = nov, articleno =. 2010 , issue_date =. doi:10.1145/1852102.1852106 , abstract =
2010
-
[26]
Theoretical and Computational Fluid Dynamics , volume=
De-biasing the dynamic mode decomposition for applied Koopman spectral analysis of noisy datasets , author=. Theoretical and Computational Fluid Dynamics , volume=. 2017 , publisher=
2017
-
[27]
2013 , publisher=
Dynamic mode decomposition: Theory and applications , author=. 2013 , publisher=
2013
-
[28]
arXiv preprint arXiv:2009.07896 , year=
Captum: A unified and generic model interpretability library for pytorch , author=. arXiv preprint arXiv:2009.07896 , year=
2009 arXiv
-
[29]
arXiv preprint arXiv:2303.08112 , year=
Eliciting latent predictions from transformers with the tuned lens , author=. arXiv preprint arXiv:2303.08112 , year=
-
[30]
Journal of Real-Time Image Processing , volume=
Compressed dynamic mode decomposition for background modeling , author=. Journal of Real-Time Image Processing , volume=. 2019 , publisher=
2019
-
[31]
2016 , publisher=
Dynamic mode decomposition: data-driven modeling of complex systems , author=. 2016 , publisher=
2016
-
[32]
Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy , pages=
Dynamic mode decomposition for background modeling , author=. Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy , pages=
-
[33]
Intelligent Systems, Technologies and Applications: Proceedings of ISTA 2018 , pages=
Dynamic mode-based feature with random mapping for sentiment analysis , author=. Intelligent Systems, Technologies and Applications: Proceedings of ISTA 2018 , pages=. 2019 , publisher=
2018
-
[34]
arXiv preprint arXiv:2008.06665 , year=
EigenEmo: Spectral utterance representation using dynamic mode decomposition for speech emotion classification , author=. arXiv preprint arXiv:2008.06665 , year=
2008 arXiv
-
[35]
arXiv preprint arXiv:2001.03131 , volume=
Offensive language detection: A comparative analysis , author=. arXiv preprint arXiv:2001.03131 , volume=
2001 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.