RECORD
Interventional Probing in High Dimensions: An NLI Case Study
Computation and Language · Submitted 20 Apr 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Furthermore, we delve into the limitations of these methods and outline some pitfalls have been obscuring the effectivity of interventional probing studies.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Probing strategies have been shown to detect the presence of various linguistic features in large language models; in particular, semantic features intermediate to the "natural logic" fragment of the Natural Language Inference task (NLI). In the case of natural logic, the relation between the intermediate features and the entailment label is explicitly known: as such, this provides a ripe setting for interventional studies on the NLI models' representations, allowing for stronger causal conjectures and a deeper critical analysis of interventional probing methods. In this work, we carry out new and existing representation-level interventions to investigate the effect of these semantic features on NLI classification: we perform amnesic probing (which removes features as directed by learned linear probes) and introduce the mnestic probing variation (which forgets all dimensions except the probe-selected ones). Furthermore, we delve into the limitations of these methods and outline some pitfalls have been obscuring the effectivity of interventional probing studies.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Probing strategies have been shown to detect the presence of various linguistic features in large language models; in particular, semantic features intermediate to the "natural logic" fragment of the Natural Language Inference task (NLI). | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| s1 | In the case of natural logic, the relation between the intermediate features and the entailment label is explicitly known: as such, this provides a ripe setting for interventional studies on the NLI models' representations, allowing for stronger causal conjectures and a deeper critical analysis of interventional probing methods. | background 0.95 | 0.07 | 0.06 | asserted 0.96 |
| s2 | In this work, we carry out new and existing representation-level interventions to investigate the effect of these semantic features on NLI classification: we perform amnesic probing (which removes features as directed by learned linear probes) and introduce the mnestic probing variation (which forgets all dimensions except the probe-selected ones). | method 0.69 | 0.95 | 0.08 | asserted 0.99 |
| s3 | Furthermore, we delve into the limitations of these methods and outline some pitfalls have been obscuring the effectivity of interventional probing studies. | contribution 0.57 | 0.58 | 0.13 | asserted 0.95 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Probing strategies have been shown to detect the presence of various linguistic features in large language models; in particular, semantic features intermediate to the "natural logic" fragment of the Natural Language Inference task (NLI). In the case of natural logic, the relation between the intermediate features and the entailment label is explicitly known: as such, this provides a ripe setting for interventional studies on the NLI models' representations, allowing for stronger causal conjectures and a deeper critical analysis of interventional probing methods. In this work, we carry out new and existing representation-level interventions to investigate the effect of these semantic features on NLI classification: we perform amnesic probing (which removes features as directed by learned linear probes) and introduce the mnestic probing variation (which forgets all dimensions except the probe-selected ones). Furthermore, we delve into the limitations of these methods and outline some pitfalls have been obscuring the effectivity of interventional probing studies.
Human review
How the literature uses it
2 arXiv papers cite this in 3 sentences
What it cites · 15 sentences without a confident label
Uses the method or tool of 1 paper
-
We consider two projection strategies in particular: the amnesic intervention introduced in [ref] (described further in section [ref]) and our mnestic variation which uses the same INLP technique (section [ref]).
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals · Interventional Probing · 0.98
Uses data or values from 2 papers
-
We use the NLI_XY dataset from [ref] .
Decomposing Natural Logic Inferences in Neural NLI · Dataset · 1.00
-
We use the NLI_XY dataset from [ref] .
Supporting Context Monotonicity Abstractions in Neural NLI Models · Dataset · 1.00
Cites as support 2 papers
-
In [ref] and [ref] , it is shown that when roberta-large-mnli (a model which performs well on benchmarks but poorly on the targeted NLI_XY challenge set) receives additional training on the adversarial HELP dataset [ref] it improves in NLI_XY performance and begins to show high probing performance for the relevant intermediate features, context monotonicity and lexical relations: this is the necessary precondition for doing interventional probing.
Supporting Context Monotonicity Abstractions in Neural NLI Models · NLI Models and Encoding · 0.96
-
In [ref] and [ref] , it is shown that when roberta-large-mnli (a model which performs well on benchmarks but poorly on the targeted NLI_XY challenge set) receives additional training on the adversarial HELP dataset [ref] it improves in NLI_XY performance and begins to show high probing performance for the relevant intermediate features, context monotonicity and lexical relations: this is the necessary precondition for doing interventional probing.
Decomposing Natural Logic Inferences in Neural NLI · NLI Models and Encoding · 0.96
Cites as background 2 papers
-
The application of probing strategies to natural logic components has been explored in [ref] and [ref] .
Decomposing Natural Logic Inferences in Neural NLI · Related Work · 0.90
-
The use of probing as an interpretability strategy dates back as far as works such as [ref] and [ref] , but a core set of work on the detailed development of the methodology includes [ref] .
Understanding intermediate layers using linear classifier probes · Related Work · 1.00
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.