Concept Probing: Where to Find Human-Defined Concepts (Extended Version)

Afonso Leote; Jo\~ao Leite; Manuel de Sousa Ribeiro

REVIEW 3 major objections 4 minor 12 references

Concept Probing: Where to Find Human-Defined Concepts (Extended Version)

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read To probe a neural network for a human-defined concept, pick the layer whose representations are both informative and regular; a weighted score of the two selects that layer in minutes and reaches about 98.6% of oracle accuracy.

desk verdict A solid empirical validation of an existing layer-selection heuristic for concept probing; the core method is not new but the evaluation is broad and useful, with tune-λ and MI-estimator caveats. read the letter →

arxiv 2507.18681 v1 pith:IANB5SYX submitted 2025-07-24 cs.LG cs.AIcs.CVcs.NE

Manuel de Sousa Ribeiro , Afonso Leote , João Leite This is my paper · ORCID

classification cs.LGcs.AIcs.CVcs.NE

keywords conceptprobinglayerselectionmutualinformationuncertaintycoefficientlogisticregressionaccuracyneuralnetworkinterpretabilityrepresentationanalysisprobeefficiency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Concept probes—small classifiers trained on a network's internal activations to detect human-defined concepts—work only if you probe the right layer. This paper claims that the right layer is the one whose representations are both informative about the concept (high normalized mutual information) and regular with respect to it (high logistic-regression accuracy), and it proposes selecting the layer that maximizes a weighted combination of the two. Across six vision models and five concepts each, probes trained from the selected layer reach about 90 percent average accuracy, roughly 98.6 percent of an oracle that knows the best layer in hindsight, while costing about nine minutes of computation instead of hours. The same characterization also reveals where in a model a concept is formed, when it is discarded, and whether a concept is encoded at all.

What carries the argument

The machinery is a two-part layer signature plus a weighted selection rule. Informativeness is measured by the uncertainty coefficient $U(c|f_l(x)) = I(f_l(x);c)/H(c)$, estimated from samples with a mutual-information estimator designed for high-dimensional variables; this says whether the concept's value is present in the activations at all. Regularity is measured by $R(c|f_l(x))$, the five-fold cross-validated accuracy of a logistic regression on the layer's activations; this says how directly and simply the concept can be read out, which in turn predicts how little data a probe needs and how simple the probe can be. Equation (1) combines the two into a single score, with the $k$-normalization of $R$ making the regularity term comparable across concepts with different numbers of values. The claim is that maximizing this score is a proxy for choosing the layer that would give the best probe.

What would settle it

Recompute the layer ranking for one of the large ResNet50 models (CUB or ImageNet) using a different mutual-information estimator, or the same estimator with several smoothness values, and train probes on the layers that win under each ranking. If the ranking flips to a layer whose probe accuracy is substantially below the reported about-90-percent level, or if the reported layer no longer beats the layers-average baseline, the selection criterion is not robust.

Watch

Extended reading notes

Core claim

The paper's central claim is that for any given human-defined concept, a network's layers can be ranked by a single criterion that predicts where a concept probe will succeed, and that no expensive probe training is needed to compute that ranking. A layer is characterized by $U(c|f_l(x))$, the uncertainty coefficient (mutual information normalized by the concept's entropy), and $R(c|f_l(x))$, the cross-validated accuracy of a logistic regression trained on that layer's activations to predict the concept. The selected layer is the one maximizing $\lambda U(c|f_l(x)) + (1-\lambda) k (R(c|f_l(x))-1)/(k-1)$, with $k$ the number of concept values and $\lambda$ controlling the information-regularity trade-off; the paper sets $\lambda=0.26$ after an ablation. Empirically, probes trained on this layer outperform the average over all layers, match or slightly beat an exhaustive validation-based search while being far cheaper, and come within about 1.3 points of the oracle. The paper also shows that the two measures trace interpretable trajectories across layers: task-unrelated concepts stay low in regularity and lose information early, low-level concepts peak before the high-level concepts built on them, and uniformly simple concepts suggest the model itself could be simpler.

Load-bearing premise

The load-bearing premise is that the estimated mutual-information scores rank layers correctly even for very high-dimensional representations; if those estimates are noisy or biased, the selected layer can be wrong even when the regularity term is measured well.

Editorial extensions

If this is right

Layer selection becomes a minutes-long precomputation: about 9 minutes per model-concept pair instead of the 334 minutes needed to train probes on every layer.
Probes from the selected layer reach about 98.6 percent of oracle accuracy and slightly beat an exhaustive validation-based search, so cheaper selection does not cost accuracy.
Because the selected layers are regular, simpler probe models (logistic regression and ridge classifiers) suffice more often, reducing training time and the amount of labeled data needed.
The informativeness–regularity trajectories are themselves interpretable: they distinguish concepts the model does not encode, show when low-level concepts are reused to build high-level ones, and reveal when a task is simple enough that the whole network could be smaller.
The method also beats a unit-selection baseline (input reduce) that searches from the last layer and stops early, missing useful internal representations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An unstated extension is whether the same two-term criterion transfers to transformer and text models; the paper tests only convolutional vision models, so attention-layer behavior remains open.
Because the mutual-information estimator is the least controlled component, a cheaper proxy such as a small probe's validation accuracy on a subsample might give nearly the same layer ranking at lower cost; the paper does not test this.
The layer trajectories could serve as a shortcut-learning diagnostic: a task-relevant concept that is discarded early, or whose regularity drops in the final layers, would flag representations the model may not actually rely on.
The paper fixes $\lambda=0.26$, but its own ablation shows the optimal weight is concept-dependent (Wooden and Rectangular prefer different ranges), so an adaptive per-concept $\lambda$ could improve further.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: the core method is the authors' own NeSy'25 criterion, so this paper is not a new proposal. What it contributes is the broadest empirical validation of that heuristic I've seen: six models, four datasets, thirty concept-model pairs, and sensible baselines. The headline — about 98.6% of oracle probe accuracy — is credible and practically useful.

The evaluation is the strong part. They compare against the layer average, an oracle upper bound, exhaustive validation, and a unit-selection method. Their method is substantially faster than exhaustive search and beats it slightly in accuracy. The per-concept tables in the appendix are consistent with the aggregate, which suggests the effect is real.

Soft spots, in proportion:

- λ is chosen as the value that maximizes average performance on the same benchmark used for the headline result. That is a mild form of fitting. It is tempered by Figure 4, which shows the accuracy surface is flat for λ<0.6.

- The mutual-information term is estimated on representations up to 6.3M units from at most 1,000 samples. Appendix C says the γ smoothness parameter was manually adjusted for the two largest models, but the values are not reported and there is no sensitivity analysis. This is the weakest part of the method's foundation. The impact is limited by λ=0.26, which gives U only 26% weight.

- On the ablation point: the stress-test worried that no ablation removes U entirely. That's not quite right — Figure 4 sweeps λ down to 0, which is exactly the R-only case, and the average accuracy appears similar for λ below 0.6. The paper doesn't quote the λ=0 number or compare it directly to λ=0.26, so the U term's marginal contribution is unquantified. I'd call this a soft spot, not a fatal flaw.

- No repeated seeds or confidence intervals anywhere. With training sets capped at 1,000 samples, that matters.

The central claim holds up: a weighted combination of informativeness and regularity selects layers close to the oracle. The method itself isn't novel, but the evidence is solid enough to make this a useful empirical reference.

Who it's for: anyone doing concept probing who wants a practical layer-selection heuristic with multi-model evidence.

Recommendation: send it to peer review. The referee should ask for the gamma values, the λ=0 comparison in tabular form, and some repeated runs. But it deserves the time.

Referee Report

3 major / 4 minor

Summary. The paper proposes an automatic layer-selection method for concept probing. For each layer of a neural network and a given human-defined concept, the method computes an informativeness score U (normalized mutual information between the layer's activations and concept labels) and a regularity score R (logistic-regression accuracy on the concept labels), then selects the layer maximizing the weighted combination in Eq. (1). The method is evaluated on four datasets (XTRAINS, GTSRB, CUB, ImageNet) and six original models, training five types of probes and comparing against a layer-average baseline, an exhaustive validation-based search, a prior input-reduction method, and an oracle that selects the layer with the highest test accuracy. The headline results report that the proposed method reaches on average 90.1--90.2% probe accuracy, roughly 98.6% of the oracle's accuracy, while taking about 9 minutes per concept instead of several hours for exhaustive validation. The authors also present qualitative observations about how concept informativeness and regularity evolve across layers, arguing that the two characteristics jointly identify suitable probing layers.

Significance. If the central claim holds, the paper provides a practically useful tool: it reduces layer selection for concept probing from an expensive exhaustive search to a fast screening procedure, and the reported accuracy is close to that of a test-oracle upper bound. The empirical study is unusually broad for the probing literature in terms of coverage (four datasets, six original models, thirty concept-model pairs, five probe architectures) and includes a runtime comparison, which is a strength. The conceptual separation between informativeness and regularity is a useful framing and leads to interesting qualitative findings about how different concepts are encoded at different depths. However, the central claim is currently supported by an evaluation in which the global hyperparameter λ in Eq. (1) is selected on the same benchmark used to report the final accuracies, and the mutual-information estimates underlying the informativeness term are not validated in the high-dimensional regime where the largest models operate. These issues need to be addressed before the quantitative claims can be fully trusted.

major comments (3)

[Section 5, Eq. (1), Figure 4, Table 1] The global weight λ is set to 0.26 because it 'results in the best average performance' on the benchmark (Section 5, after Figure 4), and the headline results in Table 1 are then reported using this value. Unless the average performance used for tuning is computed on a separate validation split, this is test-set fitting of a hyperparameter on the same data used to report the final accuracies, which makes the 98.6%-of-oracle figure optimistic. The comparison with the oracle is also asymmetric: the oracle is evaluated per concept using the test labels, while λ is a single value tuned on the aggregate test results. Please clarify the tuning protocol explicitly, and either (a) select λ on a nested validation split and report test results with that fixed λ, (b) report results averaged over a range of stable λ values (e.g., 0.1--0.5), or (c) show that the conclusions in Table 1 are unchanged when λ is chosen independently of the test data. The sentence 'the cost of fine-tuning λ is low -- for a given concept, it leads to considering only, on average, 3.7 different layers' should be explained, since in the paper λ is not actually fine-tuned per concept.
[Appendix C and Section 3] The mutual-information estimates for fCUB and fImageNet use an adjusted γ smoothness parameter whose values are not reported, and the ensemble size is increased to 15 without any sensitivity analysis. Given the dimensionality of the representations (up to 6,322,176 units) and the sample size (at most 1,000), the U term in Eq. (1) may be dominated by estimation noise, especially because the authors themselves cite the formal limitations of mutual-information measurement (McAllester and Stratos, 2020). The paper never reports an ablation with λ = 0 (i.e., the informativeness term removed) or a comparison with an alternative MI estimator. Such an experiment is necessary to establish that the U term contributes to layer selection rather than merely adding noise; if the results without U are similar, the claim that informative and regular layers are jointly selected should be revised.
[Section 5, Tables 1--3 and Appendix C] The entire empirical evaluation uses a single fixed data split (random state 0 in train_test_split, Appendix C), and the reported differences between the proposed method and the best-validation baseline are often small (e.g., fB: 85.6 vs 85.4; several rows in Table 2 are identical or differ by less than one point). Without repeated splits, bootstrap, or another measure of variability, it is difficult to know whether the observed superiority over 'Best Validation' is a reliable effect or split noise. Please add variance estimates over multiple random splits, or at least report per-concept confidence intervals, for the comparisons in Tables 1 and 2.

minor comments (4)

[Related Work, Section 6] The sentence 'Others have studied how the representations of the output concepts of a model evolve across its layers (Noshad et al., 2019; Alain and Bengio, 2017)' appears to attribute a study of output-concept evolution to the mutual-information estimator paper of Noshad et al. (2019); please verify and replace with the correct reference.
[Equation (1)] The rendered form of Eq. (1) is ambiguous: the term involving (1−λ) reads as 'k R(c|fl(x)) − 1/k − 1' in the text, which is not dimensionally consistent. If the intended expression is (1−λ) * (R(c|f_l(x)) − 1/k)/(k − 1), please write it explicitly, since this normalization is central to the method.
[Figure 4 and Section 5] The y-axis of Figure 4 is labeled 'Accuracy' but the figure is used to justify the choice of λ; please state explicitly whether the plotted accuracy is test accuracy or validation accuracy, and how the 'average' over concepts and models is computed.
[Authors' affiliation] The affiliation text contains a spacing artifact ('NOV A LINCS' and 'NOV A School of Science and Technology'), which should be corrected to 'NOVA LINCS' in the final version.

Circularity Check

1 steps flagged · score 4.0 of 10

The λ weight in Eq. 1 is tuned to the same benchmark whose headline accuracy is then reported, so part of the numerical claim is fitted; the layer-selection criterion itself is not circular.

fitted input called prediction [Section 5, 'Results and Discussion', λ-ablation paragraph (before Table 1)]
"In the remainder, we consider λ = 0.26, the value that results in the best average performance."

λ is chosen by maximizing the average probe accuracy over the same set of concepts and models used in Table 1. The 'Our Method' column (90.2% average, 98.6% of oracle) is therefore reported with a weight fitted to that exact evaluation benchmark, so the headline number is partly the result of fitting rather than an independent prediction of the method with a preselected criterion. This does not make layer selection itself circular—U and R are computed from training representations and probe accuracy is on held-out test data—but it inflates the reported result and is a fitted input presented as method performance.

full rationale

The core selection rule (Eq. 1) is not circular: U(c|f_l(x)) is a normalized mutual-information estimate between a layer's activations and the concept labels, and R(c|f_l(x)) is a logistic-regression accuracy computed with 5-fold cross-validation on the training representation set; the selected layer is then evaluated by training fresh probes and measuring accuracy on separate test data. The oracle comparison and the 'Best Validation' baseline are also computed from actual probe training, so the central empirical claim has independent content. The main caveat is that λ, the global weight balancing informativeness and regularity, is tuned to the very benchmark whose averaged accuracy is then reported, making the headline average a fitted rather than a predicted quantity. This is a genuine but limited circular component, since the ablation shows the result is reasonably stable for λ < 0.6 and since U and R are not themselves derived from the reported probe accuracies. The self-citations to the authors' own datasets, corrected CUB labels, and input-reduce baseline are not load-bearing in the derivation: they supply external data or comparison methods, and the claimed selection result does not reduce to those citations. The mutual-information estimation sensitivity noted in Appendix C is a robustness concern rather than a circularity and does not affect this score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The selection rule in Eq. 1 rests on three quantities the reader must take on faith: the mutual-information estimate (U), the logistic-regression accuracy (R), and the weight lambda. Lambda is a free parameter fitted to the benchmark (0.26). The MI estimate has two hand-set hyperparameters (ensemble size 15 and an unreported gamma adjustment). The axioms are the usual supervised-learning assumptions plus the paper-specific heuristic that logistic-regression accuracy stands in for representation regularity. No new entities (particles, forces, dimensions, conserved quantities) are introduced; the 'Reddish' concept is a label computed from pixel thresholds, not an invented entity.

free parameters (3)

lambda (weight between informativeness and regularity) = 0.26
Global weight in Eq. 1 balancing U and R; chosen as the value that results in the best average performance on the benchmark, so it is fitted to the reported accuracy metric.
MI estimator gamma smoothness = not reported
Adjusted for f_CUB and f_ImageNet to allow for a more accurate MI estimate; exact values not given, so the informativeness scores depend on an unreported manual tweak.
MI estimator ensemble size = 15
Set to 15 instead of the Noshad et al. default for more accurate estimation; affects U values across all layers.

assumptions (4)

domain assumption The semantics of a concept is given extensionally by the dataset D; label noise is mostly ignored or revised via the authors' own corrected labels.
Section 2; the paper assumes the provided labels define the concept, and uses revised CUB labels from (de Sousa Ribeiro et al., 2025a).
domain assumption Mutual information and logistic-regression accuracy, estimated on a balanced sample of at most 1000 instances, are faithful proxies for how informative and regular a layer is.
Section 3; the whole selection rule rests on these estimates, and the MI estimator has formal limitations cited via (McAllester and Stratos, 2020).
ad hoc to paper Logistic-regression accuracy R approximates the minimum-description-length / regularity of a layer in a way that indicates how easy a concept is to probe.
Section 3; the paper substitutes cross-entropy with accuracy of a logistic regression as the regularity measure; this is a heuristic choice, not derived.
ad hoc to paper The weighted sum in Eq. 1 with a fixed global lambda (set to 0.26) generalizes across concepts and models.
Section 5; lambda is set globally and assumed transferable; the ablation shows sensitivity for some concepts, e.g., Wooden vs Rectangular in ImageNet.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Concept Probing: Where to Find Human-Defined Concepts (Extended Version)." pith.science (2026). https://pith.science/paper/IANB5SYX

@misc{pith2026250718681,
  author       = {Pith},
  title        = {Pith review of: Concept Probing: Where to Find Human-Defined Concepts (Extended Version)},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IANB5SYX}},
  note         = {Machine review of arXiv:2507.18681}
}

read the original abstract

Concept probing has recently gained popularity as a way for humans to peek into what is encoded within artificial neural networks. In concept probing, additional classifiers are trained to map the internal representations of a model into human-defined concepts of interest. However, the performance of these probes is highly dependent on the internal representations they probe from, making identifying the appropriate layer to probe an essential task. In this paper, we propose a method to automatically identify which layer's representations in a neural network model should be considered when probing for a given human-defined concept of interest, based on how informative and regular the representations are with respect to the concept. We validate our findings through an exhaustive empirical analysis over different neural network models and datasets.

Figures

Figures reproduced from arXiv: 2507.18681 by the authors.

**Figure 2.** Characterization throughout a model’s layers (XTRAINS). [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Characterization throughout a model’s layers (GTSRB, CUB, ImageNet). [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Ablation of λ value over all concepts and models, and for two specific concepts. Model Our Method Layers’ Avg. Oracle % Oracle Best Validation Input Reduce Acc. Time Acc. Acc. Acc. Time Acc. Time fA 89.3 9.6 70.6 89.7 99.6 89.4 276.3 86.5 114.7 fB 85.6 9.2 71.1 87.8 97…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

[1]

Additionally, we present the layer selected by the proposed method for each concept. Table 2 shows the resulting probe accuracy and runtime for each original model and probed concept, for our method, for the layer selected based on the validation set perfor- mance, and for the Input Reduce procedure described in de Sousa Ribeiro and Leite (2021), respecti...

work page 2021
[6]

Opening the Black Box of Deep Neural Networks via Information

Ravid Shwartz-Ziv and Naftali Tishby. Opening the Black Box of Deep Neural Networks via Information. CoRR, abs/1703.00810,

arXiv
[7]

The Caltech-UCSD Birds-200-2011 Dataset

Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001,

work page 2011
[8]

Probed Models In this appendix, we provide the details regarding each probed model – also referred as original models in the paper

15 de Sousa Ribeiro Leote Leite Supplementary Material Appendix A. Probed Models In this appendix, we provide the details regarding each probed model – also referred as original models in the paper. fA, fB, and fC Three models trained on the XTRAINS dataset from Ferreira et al. (2022). These models’ architectures are based on VGGNet, but have fewer convol...

work page 2022
[10]

Dataset Licenses The XTRAINS dataset is publicly available and free to use

and PyTorch 2.7.0 (Paszke et al., 2019). Dataset Licenses The XTRAINS dataset is publicly available and free to use. The GTSRB dataset is publicly available and free to use. The CUB dataset is made available for non-commercial research and educational purposes. The ImageNet dataset is provided underarestrictedaccesslicense, madeavailablefornon-commercialr...

work page 2019
[12]

ProbedConcepts Theconsideredconceptsarereadilyavailableintherespectivedatasets

to further reduce 20 Concept Probing: Where to Find Human-Defined Concepts the representation of the layer they probe from, with a patience value of3 and the ranking of each feature is given by its maximum absolute weight. ProbedConcepts Theconsideredconceptsarereadilyavailableintherespectivedatasets. Please note that, since some of the attributes in CUB ...

work page 2019
[2011]

Pareto Probing: Trad- ing Off Accuracy for Complexity

13 de Sousa Ribeiro Leote Leite Tiago Pimentel, Naomi Saphra, Adina Williams, and Ryan Cotterell. Pareto Probing: Trad- ing Off Accuracy for Complexity. InProcs. of EMNLP’20, 2020a. Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. Information-Theoretic Probing for Linguistic Structure. In Procs. of ACL’2...

arXiv
[2017]

The validation set was used together with early stopping to select the number of boosting rounds

with default parameters. The validation set was used together with early stopping to select the number of boosting rounds. The minimal number of instances at a terminal node is reduced from the default value when the amount of available data is too small for the model to train. We consider the minimum between20 – the default value – and one tenth of the t...

work page 2015

Show all 12 references

[2020]

Multifaceted Feature Visualization: Un- covering the Different Types of Features Learned By Each Neuron in Deep Neural Net- works

Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Multifaceted Feature Visualization: Un- covering the Different Types of Features Learned By Each Neuron in Deep Neural Net- works. CoRR, abs/1602.03616,

arXiv
[2021]

Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction

Eric Chu, Deb Roy, and Jacob Andreas. Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction. abs/2007.12248,

2007 arXiv
[2023]

MobileNetV2 - TorchVision Documentation, 2025a

PyTorch Foundation. MobileNetV2 - TorchVision Documentation, 2025a. URL https://docs.pytorch.org/vision/main/models/generated/torchvision.models. mobilenet_v2.html. Accessed: 2025-01-01. PyTorch Foundation. ResNet50 - TorchVision Documentation, 2025b. URLhttps://docs. pytorch....

2025
[2024]

Explainable Abstract Trains Dataset

Manuel de Sousa Ribeiro, Ludwig Krippahl, and João Leite. Explainable Abstract Trains Dataset. CoRR, abs/2012.12115,

2012 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Additionally, we present the layer selected by the proposed method for each concept. Table 2 shows the resulting probe accuracy and runtime for each original model and probed concept, for our method, for the layer selected based on the validation set perfor- mance, and for the Input Reduce procedure described in de Sousa Ribeiro and Leite (2021), respecti...

work page 2021

[2] [6]

Opening the Black Box of Deep Neural Networks via Information

Ravid Shwartz-Ziv and Naftali Tishby. Opening the Black Box of Deep Neural Networks via Information. CoRR, abs/1703.00810,

arXiv

[3] [7]

The Caltech-UCSD Birds-200-2011 Dataset

Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001,

work page 2011

[4] [8]

Probed Models In this appendix, we provide the details regarding each probed model – also referred as original models in the paper

15 de Sousa Ribeiro Leote Leite Supplementary Material Appendix A. Probed Models In this appendix, we provide the details regarding each probed model – also referred as original models in the paper. fA, fB, and fC Three models trained on the XTRAINS dataset from Ferreira et al. (2022). These models’ architectures are based on VGGNet, but have fewer convol...

work page 2022

[5] [10]

Dataset Licenses The XTRAINS dataset is publicly available and free to use

and PyTorch 2.7.0 (Paszke et al., 2019). Dataset Licenses The XTRAINS dataset is publicly available and free to use. The GTSRB dataset is publicly available and free to use. The CUB dataset is made available for non-commercial research and educational purposes. The ImageNet dataset is provided underarestrictedaccesslicense, madeavailablefornon-commercialr...

work page 2019

[6] [12]

ProbedConcepts Theconsideredconceptsarereadilyavailableintherespectivedatasets

to further reduce 20 Concept Probing: Where to Find Human-Defined Concepts the representation of the layer they probe from, with a patience value of3 and the ranking of each feature is given by its maximum absolute weight. ProbedConcepts Theconsideredconceptsarereadilyavailableintherespectivedatasets. Please note that, since some of the attributes in CUB ...

work page 2019

[7] [2011]

Pareto Probing: Trad- ing Off Accuracy for Complexity

13 de Sousa Ribeiro Leote Leite Tiago Pimentel, Naomi Saphra, Adina Williams, and Ryan Cotterell. Pareto Probing: Trad- ing Off Accuracy for Complexity. InProcs. of EMNLP’20, 2020a. Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. Information-Theoretic Probing for Linguistic Structure. In Procs. of ACL’2...

arXiv

[8] [2017]

The validation set was used together with early stopping to select the number of boosting rounds

with default parameters. The validation set was used together with early stopping to select the number of boosting rounds. The minimal number of instances at a terminal node is reduced from the default value when the amount of available data is too small for the model to train. We consider the minimum between20 – the default value – and one tenth of the t...

work page 2015