Pith. sign in

REVIEW 5 major objections 6 minor 40 references

Removing Spurious Correlation from Neural Network Interpretations

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read The paper claims that conversation topic confounds neuron-level attribution of toxicity in large language models, making toxic behavior look more localized than it is, and that entropy-balancing adjustment spreads the attribution across…

desk verdict Promising causal re-framing of neuron attribution, but the central de-localization claim needs error bars, positivity diagnostics, and validation before it can be trusted. read the letter →

arxiv 2412.02893 v1 pith:RKGRAZWW submitted 2024-12-03 cs.CL cs.AIcs.LGstat.APstat.ME

classification cs.CLcs.AIcs.LGstat.APstat.ME
keywords causalmediationneuronattributiontoxicitylocalizationspuriouscorrelationentropybalancinginterpretabilitylargelanguagemodelsconfounderadjustment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that standard methods for attributing toxic output to specific neurons in large language models are confounded by conversation topic, and it proposes a causal mediation estimator that reweights queries so that topic no longer distorts the attribution. Applying the estimator to two instruction-tuned LLMs, the paper reports that toxicity appears less localized after adjustment: more multilayer perceptron units share responsibility. If correct, this means that mechanistic interpretability findings on toxicity localization may overstate the concentration of harmful behavior, and safety editing based on those attributions may target the wrong units.

What carries the argument

The central object is the Average Indirect Effect (AIE) estimator, a causal mediation quantity that measures how much of a query's effect on toxicity flows through a specific internal node while holding the topic fixed. It is computed with entropy balancing weights that reweight queries so that topic and node activations are balanced, and with a pairwise estimator that avoids generating out-of-distribution inputs. The cancellation of softmax normalization terms in the entropy balancing ratios lets the method compute propensity ratios without retraining, requiring only a single forward pass.

What would settle it

Use a synthetic dataset where a known topic confounder mediates a known fraction of toxicity through a known subset of units; if the estimator does not recover that ground-truth localization, the causal claim fails. Alternatively, if a finer-grained topic model (e.g., 20 clusters or a continuous topic model) reduces or reverses the reported de-localization in Bloomz or Phi-3-mini, the confounding adjustment is not capturing the true confounder.

Watch

Extended reading notes

Core claim

The central claim is that confounding by conversation topic creates spurious correlations in neuron-level attribution of toxicity, and that controlling for topic via entropy balancing changes the estimated contribution of each MLP to toxic generations. Using the DAG where topic $x$ affects both query $q$ and harmfulness $y$, and the query affects mediator $n$ (node activation), the paper estimates the Average Indirect Effect through each MLP with stabilized weights computed by entropy balancing, avoiding out-of-distribution counterfactuals by pairwise comparisons. In experiments with Bloomz 1B7 and Phi-3-mini 3B on RealToxicityPrompts, the normalized contributions to toxicity become more distributed across MLPs, i.e., toxicity is less localized, when topic adjustment is applied.

Load-bearing premise

The attribution is only unconfounded if the three k-means topic clusters capture everything about conversation topic that influences both toxicity and node activations, and if every question has positive probability under every node-activation and topic combination, conditions the paper itself notes may fail.

Editorial extensions

If this is right

  • Attribution maps for toxicity in LLMs should be re-examined with confounder adjustment to avoid overstating localization.
  • Neuron-editing or unlearning methods that rely on unadjusted attribution may suppress topic-general units rather than toxicity-specific ones.
  • The same DAG and estimator can be applied to other confounded behaviors such as gender bias, hallucination, and factuality, as the paper notes.
  • Because the estimator requires only one forward pass, confounder-adjusted attribution is feasible at scale for large models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the de-localization holds under finer topic models, the localization hypothesis itself may need to be reformulated as conditional on unconfoundedness rather than a static property of the model.
  • The pairwise AIE estimator could be adapted to other continuous mediators such as attention heads or feature circuits, not just MLP units.
  • A natural stress test is to introduce a synthetic confounder with known ground-truth mediation, then check whether the estimator recovers the correct ranking of units.
  • Because the paper acknowledges the positivity assumption may be violated by deterministic node activations, overlap diagnostics for the propensity weights would make the causal claim stronger.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a causal mediation approach to remove the effect of conversation topic when attributing toxic behavior to individual MLP units in large language models. The authors introduce a DAG in which topic x confounds the query q and the toxicity outcome y, with node activations n as a mediator, and use entropy balancing to estimate the Average Indirect Effect (AIE) for each MLP. Experiments with Bloomz 1B7 and Phi-3-mini 3B on RealToxicityPrompts show that after adjusting for topic, toxicity contributions become more distributed across MLPs. The paper does not provide code or data.

Significance. If the central claim is correct, the paper offers a principled way to correct neuron attribution for a common confounder, with direct implications for mechanistic interpretability and safety editing. The adaptation of entropy balancing to textual treatments and the focus on the localization hypothesis are timely. However, the empirical support is currently weak: the key estimator is not derived, the positivity assumption is acknowledged as possibly violated but never verified, the main results are presented without uncertainty quantification, and the topic confounder is estimated from the same queries used in the analysis. These issues prevent the paper from being accepted in its present form, though the core idea may be salvageable with additional diagnostics and analyses.

major comments (5)
  1. [Section 3, Eq. (2)] The AIE estimator in Eq. (2) is introduced without a formal derivation. The paper first defines the population quantity μ(q,q') in Eq. (1) and then states an empirical average over pairs, but Eq. (2) replaces the conditional expectation in Eq. (1) with the single observation y_i. As written, it is unclear why this plug-in estimator is unbiased or consistent for the stated causal contrast. The authors should provide a derivation, or at least a clear argument that the weighted sum over observed units estimates the population AIE.
  2. [Section 3, 'A Note on Positivity Assumption'] The paper acknowledges that f(q|n,x)>0 may fail because node activations are deterministic functions of the query, and it proposes PCA dimensionality reduction as a mitigation. However, no overlap diagnostics, weight distributions, or effective sample sizes are reported. Without evidence that the entropy balancing weights are stable and that the denominator terms in Eq. (2) are bounded away from zero, the observed flattening of the De-biased curve in Figure 2 could be an artifact of extreme reweighting, even after Winsorization.
  3. [Section 4, Figure 2] The main empirical claim that toxicity becomes less localized rests solely on visual comparison of two curves without error bars, confidence intervals, or significance tests. The authors should provide bootstrap confidence intervals that reflect both the stochastic approximation (K=200) and the entropy balancing estimation, and report a quantitative localization measure (e.g., a concentration index or slope) with uncertainty.
  4. [Section 4, topic construction] The confounder x is defined by k-means clustering on RoBERTa embeddings of the same queries, with k=3 fixed. Because x is derived from the treatment q, the DAG in Figure 1 (x -> q) is not directly satisfied, and the strong ignorability assumption depends on the cluster solution being a valid proxy for the true topic confounder. The paper should include sensitivity analyses over the number of clusters and the PCA dimension, and demonstrate that the main conclusion is robust to these choices.
  5. [Section 4, baseline definition] The 'Normal' baseline is described as obtained by Eq. (1) without confounding factors, but the exact formula is not given. To make the comparison reproducible, the authors should specify precisely how this baseline is computed, including the form of the propensity weights used when x is omitted.
minor comments (6)
  1. [Section 2.1] The heading 'Backround' should be 'Background'.
  2. [Section 2] The notation for random variables and vector values is introduced but not used consistently; for example, bold x appears both as a random vector and as a value, which can confuse the exposition.
  3. [Section 3, Eq. (1)-(2)] Eq. (1) uses δ = q'−q, while Eq. (2) uses the pair notation (q_i, q_j); the relation between q' and q_j should be stated explicitly.
  4. [Section 3, Algorithm 1] The stochastic approximation with K=200 is not justified; the authors should report sensitivity to this choice.
  5. [Appendix, Figure 3] Figure 3 is referenced as being in the appendix but appears after the references without a caption or explanation; the t-SNE plot and cluster labels should be described.
  6. [General] No code or data are provided, which limits the reproducibility of the empirical claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the de-localization finding is an empirical consequence of reweighting observed attributions, not an assumption encoded in the estimator.

full rationale

The paper's central claim is that adjusting for conversation topic makes toxicity attributions less localized. The AIE estimator in Eq. (2) is a weighted average over observed query pairs, with entropy-balancing weights fitted to balance treatment/confounder moments. The target quantity is not defined in terms of the localization conclusion, and the balance constraints (Gπ = 0) do not enforce any particular ordering or slope of the MLP-by-MLP AIE values. The 'Normal' baseline is the no-confounder special case of Eq. (1); the 'De-biased' estimate adds EB weights for [n,x]. The difference between the two curves is therefore not forced by construction. The topic variable is constructed from query embeddings via k-means, which is a measurement choice rather than a case of defining the confounder in terms of the outcome; weaknesses in that operationalization (k=3, PCA dimension, no overlap diagnostics) are validity concerns, not circularity. The paper explicitly acknowledges the positivity assumption may fail and defers node-masking validation to future work; these are honest limitation statements, not evidence that the result reduces to its inputs. The only self-citation, [Bahadori et al., 2022], supplies a general-purpose entropy-balancing method from prior published work and does not assume the present conclusion, so it is not load-bearing in a circular sense. The finding may be fragile or hard to validate, but no specific derivation step is equivalent to its own inputs.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim depends on standard causal inference assumptions (ignorability, positivity), on the correctness of the proposed DAG, on the operationalization of topic via k-means clusters, and on the validity of entropy balancing as a propensity score estimator. Several free parameters (k, PCA dimension, regularization, sampling size, Winsorization) are chosen without sensitivity analysis. No new entities are postulated.

free parameters (5)
  • Number of topic clusters = 3
    k-means is applied to RoBERTa embeddings of queries; k=3 is chosen without sensitivity analysis, and the resulting clusters are treated as the confounder topic.
  • PCA reduced dimension = 25
    Hidden states and query embeddings are reduced to 25 dimensions before entropy balancing to control computational cost and improve positivity. This choice affects the propensity score estimates.
  • Entropy balancing regularization gamma = not reported
    The regularizer for approximate balancing is mentioned but its value is not given, and the result may depend on it.
  • Stochastic approximation sample size K = 200
    For each toxic query, 200 comparison queries are sampled. This introduces Monte Carlo noise that is not quantified.
  • Winsorization percentile = 5%
    Top and bottom 5% of terms inside the AIE sum are clamped to improve robustness; the effect of this choice is not analyzed.
assumptions (6)
  • domain assumption Strong ignorability: no hidden confounders for the effect of question q on toxicity y given topic x.
    Invoked in Section 2.1 as a classical condition for estimating causal effects; if there are other confounders such as demographic style or prompt format, the adjusted AIE remains biased.
  • domain assumption Positivity: f(q|n,x) > 0 for all q, n, x in the domain.
    Stated in Section 3 'A Note on Positivity Assumption'; the deterministic relationship between activations and queries may violate it, and the paper only reduces dimensionality to alleviate it without verifying the condition.
  • ad hoc to paper The DAG in Figure 1 is the correct causal structure: topic x affects query q and harmfulness y, and query q affects node n and y.
    Assumed in Section 3 and Figure 1; if topic also affects node activations directly (not through q), or if there are other paths, the mediation analysis is misspecified.
  • standard math The entropy balancing weights provide valid estimates of the generalized propensity score ratios.
    Based on Bahadori et al. 2022 and Wang and Zubizarreta 2020; treated as a black-box estimator, but consistency relies on regularity conditions not checked here.
  • ad hoc to paper k-means clusters on RoBERTa embeddings capture the conversation topic confounder.
    Section 4; the choice of k=3 and the embedding model are not justified, and the clusters are derived from the same data used for the analysis.
  • domain assumption GPT-4 toxicity labels are accurate for the outcome.
    Section 4; no validation or agreement statistics are reported, and label noise can attenuate or distort mediation estimates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Removing Spurious Correlation from Neural Network Interpretations." pith.science (2026). https://pith.science/paper/RKGRAZWW

@misc{pith2026241202893,
  author       = {Pith},
  title        = {Pith review of: Removing Spurious Correlation from Neural Network Interpretations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RKGRAZWW}},
  note         = {Machine review of arXiv:2412.02893}
}
read the original abstract

The existing algorithms for identification of neurons responsible for undesired and harmful behaviors do not consider the effects of confounders such as topic of the conversation. In this work, we show that confounders can create spurious correlations and propose a new causal mediation approach that controls the impact of the topic. In experiments with two large language models, we study the localization hypothesis and show that adjusting for the effect of conversation topic, toxicity becomes less localized.

Figures

Figures reproduced from arXiv: 2412.02893 by the authors.

Figure 1
Figure 1. We quantify the Natural Indirect Effect (NIE) in conversations using the above DAG. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Contributions of different MLPs in generation of toxic outputs. We measure the average [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The t-SNE embedding of 1000 randomly selected queries and the clusters identified by [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 22 canonical work pages

  1. [1]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv:2404.14219, 2024

  2. [2]

    End-to-end balancing for causal continuous treatment-effect estimation

    Taha Bahadori, Eric Tchetgen Tchetgen, and David Heckerman. End-to-end balancing for causal continuous treatment-effect estimation. In ICML, pages 1313--1326. PMLR, 2022

  3. [3]

    Doubly robust estimation in missing data and causal inference models

    Heejung Bang and James M Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61 0 (4): 0 962--973, 2005

  4. [4]

    Mechanistic Interpretability for AI Safety--A Review

    Leonard Bereska and Efstratios Gavves. Mechanistic Interpretability for AI Safety--A Review . arXiv:2404.14082, 2024

  5. [5]

    Double/debiased machine learning for treatment and structural parameters

    Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21 0 (1): 0 C1--C68, 2018

  6. [6]

    Applied Causal Inference Powered by ML and AI

    Victor Chernozhukov, Christian Hansen, Nathan Kallus, Martin Spindler, and Vasilis Syrgkanis. Applied Causal Inference Powered by ML and AI. https://causalml-book.org/, 2024

  7. [7]

    Constructing inverse probability weights for marginal structural models

    Stephen R Cole and Miguel A Hern \'a n. Constructing inverse probability weights for marginal structural models. American journal of epidemiology, 168 0 (6): 0 656--664, 2008

  8. [8]

    Dealing with limited overlap in estimation of average treatment effects

    Richard K Crump, V Joseph Hotz, Guido W Imbens, and Oscar A Mitnik. Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96 0 (1): 0 187--199, 2009

Show all 40 references
  1. [9]

    Causal inference without counterfactuals

    A Philip Dawid. Causal inference without counterfactuals. JASA, 95 0 (450): 0 407--424, 2000

  2. [10]

    Fitting science into legal contexts: assessing effects of causes or causes of effects? Sociological Methods & Research, 43 0 (3): 0 359--390, 2014

    A Philip Dawid, David L Faigman, and Stephen E Fienberg. Fitting science into legal contexts: assessing effects of causes or causes of effects? Sociological Methods & Research, 43 0 (3): 0 359--390, 2014

  3. [11]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. R eal T oxicity P rompts: Evaluating neural toxic degeneration in language models. In EMNLP, 2020

  4. [12]

    Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies

    Jens Hainmueller. Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political analysis, pages 25--46, 2012

  5. [13]

    Nonparametric estimation of mediation effects with a general treatment

    Lukang Huang, Wei Huang, Oliver Linton, and Zheng Zhang. Nonparametric estimation of mediation effects with a general treatment. Econometric Reviews, 43 0 (2-4): 0 215--237, 2024

  6. [14]

    Direct and indirect effects of continuous treatments based on generalized propensity score weighting

    Martin Huber, Yu-Chin Hsu, Ying-Ying Lee, and Layal Lettry. Direct and indirect effects of continuous treatments based on generalized propensity score weighting. Journal of Applied Econometrics, 35 0 (7): 0 814--840, 2020

  7. [15]

    A general approach to causal mediation analysis

    Kosuke Imai, Luke Keele, and Dustin Tingley. A general approach to causal mediation analysis. Psychological methods, 15 0 (4): 0 309, 2010

  8. [16]

    Causal inference in statistics, social, and biomedical sciences

    Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge university press, 2015

  9. [17]

    Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data

    Joseph DY Kang, Joseph L Schafer, et al. Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical science, 22 0 (4): 0 523--539, 2007

  10. [18]

    Probabilities of causation for continuous and vector variables

    Yuta Kawakami, Manabu Kuroki, and Jin Tian. Probabilities of causation for continuous and vector variables. arXiv:2405.20487, 2024

  11. [19]

    Bloom: A 176b-parameter open-access multilingual language model

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili \'c , Daniel Hesslow, Roman Castagn \'e , Alexandra Sasha Luccioni, Fran c ois Yvon, Matthias Gall \'e , et al. Bloom: A 176b-parameter open-access multilingual language model. 2023

  12. [20]

    RoBERTa: A Robustly Optimized BERT Pretraining Approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach . arXiv:1907.11692, 2019

  13. [21]

    Evaluating causes of effects by posterior effects of causes

    Zitong Lu, Zhi Geng, Wei Li, Shengyu Zhu, and Jinzhu Jia. Evaluating causes of effects by posterior effects of causes. Biometrika, 110 0 (2): 0 449--465, 2023

  14. [22]

    Sparse feature circuits: Discovering and editing interpretable causal graphs in language models

    Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv:2403.19647, 2024

  15. [23]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. NeurIPS, 35: 0 17359--17372, 2022

  16. [24]

    Progress measures for grokking via mechanistic interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. In ICLR, 2023

  17. [25]

    Zoom in: An introduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5 0 (3), March 2020

  18. [26]

    In-context learning and induction heads

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. arXiv preprint arXiv:2209.11895, 2022

  19. [27]

    Causality

    Judea Pearl. Causality. Cambridge university press, 2009

  20. [28]

    Mediation analysis methods used in observational research: a scoping review and recommendations

    Judith JM Rijnhart, Sophia J Lamp, Matthew J Valente, David P MacKinnon, Jos WR Twisk, and Martijn W Heymans. Mediation analysis methods used in observational research: a scoping review and recommendations. BMC medical research methodology, 21: 0 1--17, 2021

  21. [29]

    Marginal structural models and causal inference in epidemiology

    JM Robins, MA Hern \'a n, and B Brumback. Marginal structural models and causal inference in epidemiology. Epidemiology, 11 0 (5): 0 550--560, 2000

  22. [30]

    Does matching overcome lalonde's critique of nonexperimental estimators? Journal of econometrics, 125 0 (1-2): 0 305--353, 2005

    Jeffrey A Smith and Petra E Todd. Does matching overcome lalonde's critique of nonexperimental estimators? Journal of econometrics, 125 0 (1-2): 0 305--353, 2005

  23. [31]

    A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis

    Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In EMNLP, 2023

  24. [32]

    Investigating gender bias in language models using causal mediation analysis

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. NeurIPS, 33: 0 12388--12401, 2020

  25. [33]

    Minimal dispersion approximately balancing weights: asymptotic properties and practical considerations

    Yixin Wang and Jose R Zubizarreta. Minimal dispersion approximately balancing weights: asymptotic properties and practical considerations. Biometrika, 107 0 (1): 0 93--105, 2020

  26. [34]

    Unlearning bias in language models by partitioning gradients

    Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. Unlearning bias in language models by partitioning gradients. In ACL, pages 6032--6048, 2023

  27. [35]

    Towards best practices of activation patching in language models: Metrics and methods

    Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods. In ICLR, 2024

  28. [36]

    Conditional counterfactual causal effect for individual attribution

    Ruiqi Zhao, Lei Zhang, Shengyu Zhu, Zitong Lu, Zhenhua Dong, Chaoliang Zhang, Jun Xu, Zhi Geng, and Yangbo He. Conditional counterfactual causal effect for individual attribution. In UAI, pages 2519--2528. PMLR, 2023

  29. [37]

    A boosting algorithm for estimating generalized propensity scores with continuous treatments

    Yeying Zhu, Donna L Coffman, and Debashis Ghosh. A boosting algorithm for estimating generalized propensity scores with continuous treatments. Journal of causal inference, 3 0 (1): 0 25--40, 2015

  30. [38]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  31. [39]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  32. [40]

    @7N 37^3b#+]xI[bb ]CxAFgGxx=_ l ЛBPK 2Fi tY # W w(( qx 'ҋ9y |,BP

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.