{"id":"468b08ad-d33d-45e5-820f-e3c26044f37f","arxiv_id":"2505.00555","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Probing, ablation, and causal tracing can expose how small neural-network nuisance estimators in TMLE encode confounders, based on proof-of-concept experiments on synthetic bio-statistical data.","lead":"This Master's thesis applies mechanistic interpretability tools, such as linear probing and causal tracing, to small neural networks used for causal effect estimation in TMLE frameworks. It reports on synthetic data that these tools can reveal which confounders a network represents and how different inputs propagate through its layers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Causal tracing never patches the treatment input A, so the claimed 'treatment vs. confounder pathways' are not actually demonstrated.","rationale":"The reader's weakest assumption identifies the unstated significance threshold in causal tracing. I agree that this is a real problem, but I see an even more load-bearing concern: the treatment pathway claim is not supported by the experimental design at all, because the causal tracing operates on covariate input neurons only. The architecture explicitly separates treatment A from the traced input layer, so the central claim about discovering how treatments are handled cannot be validated by these experiments, regardless of threshold choices. This concern does not change the overall verdict: the paper remains a conditional proof of concept, because the probe/ablation experiments provide some support for representation validation (claim 1), while the pathway discovery (claim 2) needs either a new treatment-input tracing experiment or a substantial softening of the claim. The threshold issue and the treatment-tracing gap both point to the same conclusion that the paper overclaims, so the reader's CONDITIONAL verdict is appropriate, though the specific reason differs.","tokens_in":45529,"tokens_out":4143,"duration_ms":43481,"concrete_test":"Re-run causal tracing starting from the treatment input: for a fixed base input, patch A at the Q-head input (e.g., replace A=0 with A=1 while holding h_shared fixed), then apply the same successive-patching algorithm to identify downstream neurons whose activations change, and compare this resulting pathway with pathways initiated from confounder input neurons. On the DS2 'no-effect' data, the method should show no distinct treatment pathway if it can genuinely detect treatment processing; if the tracing procedure cannot be applied to A because A is not part of the traced input layer, or if no distinct pathway emerges, then the claimed treatment-vs-confounder pathway discovery is unsupported and the claim should be limited to covariate pathways.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (2) states that MI tools can 'discover and visualize the distinct computational pathways employed by the network to process different types of inputs, potentially revealing how confounders and treatments are handled.' The experiments, however, only trace pathways from input neurons corresponding to covariates W. In the fQ,g architecture (Section 5.1.1 and Appendix), the shared layers process W, while treatment A enters only at the Q-head as an additional input to fQ(h_shared, A). Causal tracing in Sections 5.2 and 5.3 initiates activation patches 'at the input layer' by 'changing the activation of only input neuron i', which can only be a covariate neuron. Thus the traced pathways describe how different confounders/covariates are processed, not how treatment is processed. The conclusion in Chapter 6 that causal tracing revealed 'how information from different input covariates, such as confounders versus treatment variables, is processed' overreaches: no treatment-input patch was ever performed. The reader's concern about the unspecified significance threshold is valid and would undermine the reliability of any traced pathway, but even with a fully specified threshold, the treatment-pathway conclusion would not follow from the data collected. The gap is between the experiment's design and the claim's scope, not just a missing implementation detail.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript, a master's thesis, proposes applying mechanistic interpretability (MI) tools to neural networks used for causal inference in biostatistics, specifically multi-task networks that estimate nuisance functions within TMLE. In Experiment 1, linear probes are trained on shared-layer activations to predict a known strong confounder W1, and ablation of the probe-selected neurons is shown to shift the resulting TMLE ATE estimate relative to random and bottom-neuron ablations. Experiments 2 and 3 apply causal tracing from input neurons to generate pathway graphs and Jaccard-overlap matrices, which the paper interprets as revealing distinct computational pathways for confounders versus treatments. The paper concludes that MI can validate nuisance function estimators and reveal how different input covariates are processed, and it claims a third contribution of providing methods to compare mechanisms across statistical, ML, and NN models.","tokens_in":45727,"tokens_out":5972,"duration_ms":56514,"significance":"The work is a useful proof-of-concept that linear probing and ablation can provide external validation of nuisance networks: the experiments use synthetic data with a known true ATE, compare against random and bottom-neuron ablation baselines, and the code is publicly referenced. If the causal-tracing claims were fully supported, the paper would make a modest but real contribution to the emerging practice of auditing black-box causal estimators. The primary experimental evidence for claim (1) is credible; the evidence for claim (2) is not, because treatment information is never patched, and claim (3) is not experimentally addressed.","major_comments":[{"comment":"The causal-tracing experiments patch only input neurons corresponding to covariates W, never the treatment input A; because in fQ,g the treatment A enters only the Q-head (fQ(h_shared,A)) while the shared layers process W, the traced pathways describe how covariates propagate and cannot support the conclusion that the network has distinct 'treatment vs. confounder' pathways. The claim in Chapter 6 that tracing revealed 'how information from different input covariates, such as confounders versus treatment variables, is processed' overreaches the collected data; a treatment-input patch or an explicit head-level intervention would be needed.","section":"§5.2.1, §5.3.1, Chapter 6"},{"comment":"The method marks nodes as 'failed' when activation changes do not exceed a threshold, but the threshold is never specified and no sensitivity analysis is reported; because the success score, the edge set in Figures 5.7-5.9, and all overlap computations depend on this threshold, the pathway graphs are not reproducible or interpretable without reporting it.","section":"§5.2.1"},{"comment":"The abstract's third claimed contribution—providing methodologies for comparing learned mechanisms across statistical, ML, and NN models—is not realized anywhere in the experiments; Sections 5.1-5.3 study only the NN fQ,g, with no cross-model comparison, and Chapter 6 does not present such a method.","section":"Abstract and §5.1-5.3"},{"comment":"The architecture description is internally inconsistent: §5.1.1 and Table 5.1 refer to shared layers h1 through h8 plus h_shared, while the architecture section states 5 shared layers with 100 units, and the training section reports that 'HIDDEN LAYER = 10 and HIDDEN SIZE = 30' gave the best results; the actual network used in Experiments 1 and 3 is therefore ambiguous, preventing reproducibility.","section":"§5.1.1 and Appendix 'Experiment 1'"}],"minor_comments":[{"comment":"Chapter 5 opens with 'this chapter presents experiments1 applying MI techniques' and refers to 'the previous chapter ??'; fix the broken cross-reference.","section":"Chapter 5 opening"},{"comment":"Figure 5.6 is described in §5.2.2 as the no-effect model and in §5.3.2 as the strong-confounder model; the caption and text should state which dataset the figure corresponds to.","section":"§5.2.2 and §5.3.2"},{"comment":"The appendix heading 'All graphs from Experience 3 (5.2)' is mislabeled; the figures it contains appear to belong to the Section 5.2 causal-tracing experiments.","section":"Appendix 'All graphs from Experience 3 (5.2)'"},{"comment":"Section 5.2.1 contains the typo 'follwi metrics,' and the Success Score definition depends on the unspecified threshold noted in the major comments.","section":"§5.2.1"},{"comment":"The combined loss in Section 5.1.1 uses weights wq and wg, while the appendix reports a single alpha = 0.5; specify the relationship between these quantities.","section":"§5.1.1 and Appendix 'Training'"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is clearly a thesis draft in need of editing; the survey chapters are lengthy, and the experimental contribution is a small proof-of-concept. The main risk for the journal is that the abstract and conclusion claim more than the experiments establish. The treatment-pathway claim is fixable by adding a treatment-input tracing experiment, and the threshold needs reporting, so major revision is the appropriate outcome rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first experiment is a solid, reproducible proof of concept: a small MLP estimates TMLE nuisance functions on synthetic data, linear probes clearly decode the known confounder W1 from shared-layer activations, and ablating the top probe-importance neurons moves the TMLE ATE more than ablating random or bottom neurons. That part is clean and worth keeping. Code and synthetic data are on GitHub, and the known ATE gives an honest external benchmark.\n\nThe abstract's second claim does not hold up. Causal tracing is initiated by patching input neurons of the shared layers, which only see covariates W. Treatment A enters later, concatenated at the Q-head, and is never patched. So the traced pathways describe covariate processing, not 'treatment versus confounder' pathways. The Chapter 6 conclusion that the network segregates confounders and treatment variables is not supported by the experiments as designed. The activation-change threshold for marking failed nodes is also never specified, so the pathway graphs are not reproducible.\n\nThe abstract promises a methodology for comparing mechanisms across statistical, ML, and NN models, but no such comparison is actually performed. There are minor internal inconsistencies in the architecture description: the appendix says 5 layers of 100 units, the training text says hidden layer 10 and hidden size 30. These are fixable, but they add noise.\n\nThe probe/ablation experiment stands on its own. The causal-tracing sections need either a redesign that patches A at the Q-head, or a more modest interpretation as covariate-pathway discovery. The paper is a Master's thesis applying standard MI tools to a toy setting, so the contribution is a proof of concept rather than a new method. It deserves peer review—conditional accept with major revision would be a fair outcome. If the venue is interested in MI applied to causal estimation, send it to a serious referee; the first experiment is worth preserving, and the pathway section can be fixed or cut.","headline":"Reproducible probe/ablation proof of concept undermined by an unsupported treatment-pathway claim.","tokens_in":46274,"tokens_out":2126,"would_cite":false,"duration_ms":22825,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62D20","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Mechanistic interpretability tools—probing, ablation, and causal tracing—can open up the neural networks used inside TMLE causal estimators, showing whether a network encodes known confounders and which input pathways drive its estimates.","keywords":["mechanistic interpretability","causal inference","TMLE","nuisance function estimation","linear probing","causal tracing","neural network interpretability","confounding"],"falsifier":"Re-run the causal-tracing procedure of Section 5.2 on the same trained networks while sweeping the activation-change cutoff across a plausible range (e.g., from a small fixed value to a high percentile of the null distribution of activation differences induced by patching). If the reported treatment-versus-confounder pathway separation, the success scores, and the Jaccard-overlap structure change qualitatively across cutoffs, the discovered pathways are threshold artifacts. A second check: train the same architecture on a version of DS1 in which the confounder $W_1$ is permuted to break its link to treatment and outcome; if probes still report high $R^2$ for $W_1$ or ablating probe-important neurons still moves the TMLE ATE, the representation-and-ablation claims fail.","tokens_in":45286,"feed_emoji":"🧠","tokens_out":9790,"duration_ms":90244,"temperature":0.7,"pith_summary":"In bio-statistics, causal effect estimates increasingly rely on neural networks to estimate nuisance functions inside Targeted Minimum Loss-based Estimation (TMLE), but these networks are hard to trust because their internals are opaque. This paper argues that mechanistic interpretability tools developed for large language models transfer to these smaller networks: linear probes can detect whether a known confounder is encoded in the shared layers, ablating the neurons that encode it moves the final average treatment effect estimate, and causal tracing can draw the computational pathways through which inputs such as confounders and treatments propagate. The experiments are carried out on synthetic data with a known true treatment effect, so the internal findings can be checked against ground truth. If the demonstration holds, MI gives biostatisticians a way to validate and debug network-based nuisance estimators before relying on their causal conclusions.","feed_headline":"Probes trace how neural nets use confounders in causal estimates","feed_subtitle":"Linear probing inside TMLE nuisance networks shows which confounders a model actually encodes and uses.","key_machinery":"Three interacting tools carry the argument. Linear probing: train a simple regression on activations at each shared layer to predict the known confounder $W_1$, then rank neurons by the magnitude of the probe weights; high probe $R^2$ is evidence the confounder is represented. Neuron ablation: zero the output of the probe-identified neurons during a forward pass and measure how much the $Q$- and $g$-head losses and the TMLE ATE estimate move; larger movement for top-importance neurons is evidence the representation is causally used. Causal tracing: patch the activation of one input neuron, then successively patch downstream neurons whose activation change exceeds a cutoff, building a directed graph of 'impacted' nodes; sparsity, success, and Jaccard-overlap scores turn these graphs into comparable pathway summaries.","core_discovery":"The central claim is that MI techniques—linear probing, neuron ablation, and causal tracing—can be applied to neural networks that estimate nuisance functions in TMLE, and that they reveal something true about how those networks compute. Concretely, the paper shows that a linear probe trained on shared-layer activations recovers the critical confounder $W_1$ with high $R^2$ at every layer; that ablating the neurons most important to that probe shifts the resulting TMLE estimate of the average treatment effect more than ablating random or least-important neurons; and that causal traces started from individual input neurons produce pathway graphs with measurable overlap structure, which the author interprets as evidence that the network processes confounder and treatment information through partly distinct internal routes. The ground truth for these experiments comes from synthetic datasets with known true ATEs (2.0 in the strong-confounder design, 0 in the no-effect design).","pith_inferences":["A testable next step is to turn the pathway-overlap matrix into a formal diagnostic: under a null model with randomized weights or shuffled inputs, the null distribution of Jaccard overlap between treatment and confounder traces could be estimated, so a reported separation can be tested rather than read off a graph.","If linear decodability and ablation-sensitivity generalize to real electronic health record data, the same protocol could be used as a pre-deployment audit for any neural nuisance model, flagging networks that encode confounders they should not or fail to encode confounders they must adjust for.","The probe-importance ranking also suggests a pruning or regularization strategy: neurons that are important for a confounder could be penalized or removed when the goal is to estimate effects invariant to that confounder, though this is a design choice the paper does not explore.","The author's framing implies a broader claim: MI is not only for frontier models but can serve as a general science-of-models toolkit for statistics, where ground truth about causal structure is often known from the study design."],"forward_implications":["Linear probing can serve as a validation check: a TMLE nuisance network that linearly encodes a known confounder passes a minimal test that it has learned the right input structure.","Ablation of probe-important neurons changes the final ATE estimate, so the internal representation is not a correlation artifact—it is load-bearing for the causal estimate.","Causal tracing provides a visual and quantitative map of how individual input covariates propagate; treatment and confounder inputs can be compared by pathway overlap rather than by inspecting weights.","Because the same tools work on any network with shared layers, the approach extends to other nuisance estimators and to longitudinal variants of TMLE.","If MI validation becomes standard for network-based nuisance estimation, black-box concerns in high-stakes biostatistics shift from 'can we trust it' to 'which confounders are demonstrably used and how'."],"supporting_citations":[{"why":"Founded Targeted Maximum Likelihood Estimation, the causal framework whose nuisance functions the networks estimate and probe.","marker":"[van der Laan and Rubin, 2006]"},{"why":"Formalizes the Targeted Learning roadmap, including the two-stage nuisance-estimation and targeting structure used here.","marker":"[van der Laan and Rose, 2018]"},{"why":"Introduced the linear probing approach used to read the confounder from hidden activations.","marker":"[Niven and Kao, 2019]"},{"why":"Defines interchange interventions and activation patching, the operation causal tracing builds on.","marker":"[Geiger et al., 2021]"},{"why":"Introduced causal tracing, adapted here to trace pathways from input neurons.","marker":"[Meng et al., 2022]"},{"why":"Supplies the definition of neural-network ablation used to test the causal relevance of neurons.","marker":"[Meyes et al., 2019]"},{"why":"Documents the correlation-not-causation limitation of probing, motivating the ablation and tracing experiments.","marker":"[Belinkov, 2022]"}],"fun_headline_variants":["Neural nets spill their causal secrets to linear probes","Probing neural nets: which confounders matter for causal estimates","Mechanistic interpretability unlocks black-box causal nets","Inside TMLE nets: probes identify confounders, ablations shift ATE","Causal tracing reveals how nets handle confounders"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the causal-tracing pathway graphs are not artifacts of an unstated activation-change cutoff: Section 5.2.1 marks downstream neurons as 'failed' whenever their activation change does not exceed a threshold, but the threshold is never specified, so the existence of distinct confounder and treatment pathways depends on an untested tuning choice.","fun_headline_variants_meta":{"raw":{"variants":["Neural nets spill their causal secrets to linear probes","Probing neural nets: which confounders matter for causal estimates","Mechanistic interpretability unlocks black-box causal nets","Inside TMLE nets: probes identify confounders, ablations shift ATE","Causal tracing reveals how nets handle confounders"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000601,"raw_usage":{"total_tokens":2797,"prompt_tokens":925,"completion_tokens":1872,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":1787}},"tokens_in":541,"tokens_out":1872,"duration_ms":14327,"temperature":1.0,"reasoning_tokens":1787,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:39:06.802679+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the causal-tracing procedure of Section 5.2 on the same trained networks while sweeping the activation-change cutoff across a plausible range (e.g., from a small fixed value to a high percentile of the null distribution of activation differences induced by patching). If the reported treatment-versus-confounder pathway separation, the success scores, and the Jaccard-overlap structure change qualitatively across cutoffs, the discovered pathways are threshold artifacts. A second check: train the same architecture on a version of DS1 in which the confounder $W_1$ is permuted to break its link to treatment and outcome; if probes still report high $R^2$ for $W_1$ or ablating probe-important neurons still moves the TMLE ATE, the representation-and-ablation claims fail.","supporting_citations":[{"cited_title":"Causal abstractions of neural networks","cited_arxiv_id":null,"evidence_quote":"Defines interchange interventions and activation patching, the operation causal tracing builds on."}],"review_version":1}