REVIEW 4 major objections 5 minor 4 references
Interpreting Social Respect: A Normative Lens for ML Models
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that TCAV, applied to two versions of a toxicity classifier with concept examples built from LGBTIQ+ identity terms and positive self-statements, reveals that the models' internal representations encode different…
desk verdict A genuinely useful dataset and a first-of-its-kind TCAV application, but the central empirical claim is confounded and the paper overreads its own evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is TCAV (Testing with Concept Activation Vectors), a method that represents a human-interpretable concept as a direction in the activation space of a neural network. The direction is learned from a set of example inputs; a TCAV score then measures how strongly moving an input's internal representation along that direction changes the model's prediction. Here the concept directions are learned from LGBTIQ+ identity terms, toxic comments, neutral comments, and Project Respect's positive statements, and the score measures each concept's association with the toxicity prediction. This machinery is what lets the paper translate model internals into a normative claim.
What would settle it
Hold architecture, training data, and training procedure fixed, and rerun the TCAV analysis on a model with and without the specific bias-mitigation step. If the two versions produce indistinguishable TCAV scores for the same concept sets, the paper's attribution of the encoding difference to bias mitigation would be falsified.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that TCAV scores expose a normative difference between model versions. In toxicity@1, both toxic and neutral comments containing LGBTIQ+ identity terms receive high TCAV scores for toxicity, suggesting the model strongly associates identity terms with toxicity regardless of comment sentiment. In toxicity@6, toxic and neutral comments receive very different TCAV scores, indicating the bias-mitigated model no longer conflates identity with toxicity in the same way. The Project Respect positive statements also score differently across versions, and the paper reads this as evidence that the internal representations encode different kinds of information about how social identities relate to normative values.
Load-bearing premise
The central comparison assumes that the only meaningful difference between toxicity@1 and toxicity@6 is the bias-mitigation technique, so that differing TCAV scores can be attributed to that technique rather than to differences in architecture, training data, or other changes between the versions.
Editorial extensions
If this is right
- Bias mitigation can change not only the model's outputs but the internal geometry linking identity concepts to toxicity.
- TCAV with normative concept sets can serve as a pre-deployment audit check for whether a model encodes identity in conflation with harm.
- The Project Respect positive-statement data supplies a concept set for positive value encoding, not just toxicity, extending the probe beyond negative categories.
- The technique is model- and domain-agnostic, so the same concept sets can probe other pre-trained language models and other social categories.
Reading between the lines
- If the result generalizes, TCAV scores could be tracked over model versions as a regression test for value alignment, not just bias.
- The observed differences are correlational; a stronger test would intervene on training data distribution and observe shifts in concept directions.
- The same normative lens could be applied to generative models, asking whether identity terms are internally associated with harm during generation.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a normative lens for interpreting machine learning models by combining TCAV (Testing with Concept Activation Vectors) with a new crowdsourced dataset, Project Respect, containing positive statements from marginalized communities. It applies TCAV to two versions of the Perspective API toxicity model and reports mean TCAV scores with 90% confidence intervals for four concept sets: toxic web comments about LGBTIQ+ identities, neutral web comments about those identities, and two Project Respect positive-statement sets. The authors claim the differing TCAV scores between toxicity@1 and toxicity@6 show that the two models encode different information about the interaction between social identities and normative values, and they argue this demonstrates a usable normative lens for auditing model internals.
Significance. If the empirical result were validated, the paper would make a useful contribution by extending TCAV from visual and generic concepts to normative values in language models, and by introducing Project Respect as a resource for value-sensitive auditing. The paper's strengths are that it uses a well-established interpretability method (TCAV), a concrete artifact (Project Respect), and a clearly stated normative motivation connecting model internals to disparate impact. These are meaningful first steps in an important direction. However, the central empirical claim currently rests on a single qualitative figure, and the missing controls and statistical validation are substantial. The proposed resource and framing are likely to be of interest to the fairness, accountability, and transparency community, even though the evidence in this version is preliminary.
major comments (4)
- [Normative model insights, Figure 1] The comparison between toxicity@1 and toxicity@6 is confounded. The paper states only that version 6 had bias mitigation 'similar to' the techniques in [1], without reporting whether the architecture, training data, training procedure, or other hyperparameters also changed. As a result, any TCAV difference between the two models cannot be attributed specifically to bias mitigation or to a change in normative encoding. The authors should provide the exact model versions and their training details, or use controlled model pairs that differ only in the bias mitigation technique.
- [Normative model insights, concept sets] The four concept sets differ simultaneously along sentiment, presence of identity terms, register, source, and likely sentence length. The TCAV direction for the Project Respect examples may therefore capture general positivity, formality, non-toxicity, or topic rather than 'respect' as a normative value. In particular, since the Project Respect statements are positive by design, their TCAV direction is likely correlated with non-toxicity, so lower TCAV scores in toxicity@6 could follow mechanically from bias mitigation reducing the identity-toxicity correlation, without implying any change in normative encoding. The authors should add matched control concepts (e.g., positive statements about non-marginalized topics, sentiment-matched non-toxic comments from the same source), report TCAV accuracy per concept and concept set sizes, and include randomization or overlap checks to demonstrate that each learned direction corresponds to the intended concept.
- [Figure 1] No statistical test is reported for the difference between toxicity@1 and toxicity@6, so the claim of 'very different results' is not quantitatively established. The figure lists point estimates and 90% confidence intervals, but without a significance test, an effect size, or a report of whether the intervals overlap, the reader cannot assess whether the differences are meaningful. Additionally, the paper omits key TCAV parameters such as the target layer, number of trials, sizes of random baseline sets, and whether the identical example sets were used across both models; these details are necessary for reproducibility and for judging whether the results are robust.
- [Discussion and Conclusion] The conclusion that the two neural networks are 'encoding different kinds of information about the interaction between social identities and normative values' is stronger than the evidence supports. TCAV measures the sensitivity of a model's prediction to movement along a learned concept direction; it does not directly characterize the model's internal representation. The claim should be softened to state that the models' predictions show different sensitivity to the concept directions, or additional probing experiments should be provided to substantiate the representational claim.
minor comments (5)
- [Throughout] Use 'TCAV' consistently; the text alternates between 'tcav' and 'TCAV'.
- [Figure 1] The figure would be easier to interpret with a table of numeric values and explicit labels for the two Project Respect conditions; the current grayscale rendering makes it difficult to distinguish the concepts.
- [Project Respect] Provide the number of submissions, the distribution of identity terms, and the total size of the four concept sets, since the TCAV estimates depend on these quantities.
- [References] Reference [1] is a GitHub README; cite a stable or peer-reviewed description of the Perspective API and its model versions, and clarify the exact relationship between [1] and the bias mitigation applied to toxicity@6.
- [Normative model insights] The phrase 'normative lens' is used informally; a short formal definition of how TCAV scores operationalize normative values would clarify the proposal.
Circularity Check
No significant circularity: the TCAV measurements are external-method empirical observations, and the Project Respect concept set is an interpretive input rather than a fitted target.
full rationale
The paper makes no derived prediction that is defined in terms of its own output. TCAV scores are computed by applying the published TCAV procedure (Kim et al., ICML 2018) to frozen pretrained toxicity models, using concept sets constructed from web comments and Project Respect statements. The Project Respect concept set is an input to the analysis; the conclusion that the two model versions respond differently to this concept set is a direct empirical reading of the TCAV scores, not a quantity fitted to produce that conclusion. The paper's self-citations (Project Respect, Perspective API) describe data-collection infrastructure and are not used as evidence for the interpretability claim. A possible validity concern is that the concept sets differ in sentiment, register, and source, which could affect what the TCAV direction captures, but that is a confound in the experimental design rather than a circularity: no equation or definition reduces the claimed result to its own inputs. Accordingly, no circular step is identified and the score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption The TCAV method, as described in Kim et al. [4], reliably estimates concept importance in neural network internal representations.
- domain assumption The crowdsourced Project Respect statements and the sampled toxic/neutral comment sets accurately represent the intended social identity concepts.
- domain assumption The Perspective API models' internal activations are accessible and comparable across version 1 and version 6.
Cite this review
Pith. "Pith review of Interpreting Social Respect: A Normative Lens for ML Models." pith.science (2026). https://pith.science/paper/HTCFNPI7
@misc{pith2026190807336,
author = {Pith},
title = {Pith review of: Interpreting Social Respect: A Normative Lens for ML Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/HTCFNPI7}},
note = {Machine review of arXiv:1908.07336}
}
read the original abstract
Machine learning is often viewed as an inherently value-neutral process: statistical tendencies in the training inputs are "simply" used to generalize to new examples. However when models impact social systems such as interactions between humans, these patterns learned by models have normative implications. It is important that we ask not only "what patterns exist in the data?", but also "how do we want our system to impact people?" In particular, because minority and marginalized members of society are often statistically underrepresented in data sets, models may have undesirable disparate impact on such groups. As such, objectives of social equity and distributive justice require that we develop tools for both identifying and interpreting harms introduced by models.
Figures
Reference graph
Works this paper leans on
-
[1]
https://github.com/ conversationai/perspectiveapi/blob/ master/api_reference.md#models
Perspective API. https://github.com/ conversationai/perspectiveapi/blob/ master/api_reference.md#models
- [2]
- [3]
-
[4]
B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, et al. Interpretability beyond feature attribution: Quantitative test- ing with concept activation vectors ( tcav). In International Conference on Machine Learning , pages 2673–2682, 2018. 2
work page 2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.