REVIEW 4 major objections 5 minor 44 references
Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims a latent causal value graph organizes LLM responses, that alignment training leaves it structurally different from human value theory, and that the graph predicts and limits steering side effects.
desk verdict A useful recipe for predicting steering side-effects wrapped in an unverified causal claim; the graph is a dependency summary, not evidence of latent value causality. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the value causal graph $G = (V, E)$ over 17 ValueBench value dimensions, constructed with the Peter–Clark (PC) algorithm, a standard passive causal-discovery procedure, run at significance level 0.05 on 2,525 orientation vectors (101 roles times 25 SAE features). Each vector records the model's average ternary score on the value questionnaire under that particular persona or feature-amplification setting. The graph carries the whole argument: its successor and non-successor relations define the prediction metrics, the comparison against Schwartz-style reference graphs, and the claimed precision advantage of SAE steering, which is quantified as the number of source nodes each intervention family touches.
What would settle it
Take a value pair the graph marks with an edge and one it marks as non-adjacent; steer only the cause node, using a single SAE feature at a single token position whose training profile shows it moves that one value, and record on new roles whether the predicted successor flips while the non-successor stays fixed. If the prediction advantage over the reference graphs disappears when interventions are restricted to single nodes, the discovered edges are artifacts of the multi-node interventions.
Extended reading notes
Core claim
The central claim is that LLMs possess a latent causal value graph, a directed structure over value dimensions in which a change to one value propagates along edges to others, and that this graph remains significantly different from the structure of human values described by theories such as Schwartz's quasi-circumplex model. The evidence is predictive: when a role-prompt or SAE intervention changes the model's orientation on a target value, dimensions the learned graph marks as successors change more often than dimensions marked as successors by the reference graphs (0.57–0.74 versus 0.43–0.51 across the two models), while non-successor dimensions change less often. The authors further claim that SAE steering is the more surgical instrument, because it activates only about four source nodes in the graph on average, compared with 7.7–14.6 for persona prompts, which is why it produces fewer unexpected side effects. The reason to accept a graph at all, on this account, is that it predicts what a tester can actually observe after steering.
Load-bearing premise
The load-bearing premise is that the Peter–Clark algorithm applied to the 2,525 role-and-SAE orientation vectors recovers genuine causal edges among value dimensions rather than correlations created by the intervention families acting as common causes on several values at once; if that fails, the graph describes the steering machinery, not the model's values.
Editorial extensions
If this is right
- An operator can consult the graph before steering: it enumerates which value dimensions will shift as side effects, so a target value can be pursued while minimizing collateral change.
- SAE feature steering becomes a precision tool for value control, since it perturbs far fewer causal source nodes than persona-based prompting and therefore triggers fewer unexpected changes.
- Alignment that targets coarse values such as helpfulness, harmlessness, and honesty leaves an internal value structure that differs from human value theory, so effective fine-grained alignment should be guided by the model's own discovered graph rather than by human taxonomies alone.
- The prediction metrics give a quantitative way to compare an LLM's value organization against alternatives — human theory, the model's own self-report, or ValueBench's hierarchy — and in every tested comparison the discovered graph predicts observed steering effects better.
Reading between the lines
- Because the PC step treats the role and SAE interventions as passive observations, its edges could in principle reflect the steering methods acting as common causes on several values at once; a decisive check the paper does not report is whether graphs refit from role-prompt data alone and from SAE data alone agree on edge directions.
- If the graph is stable across models of different sizes, it could serve as a transferable wiring diagram for predicting fine-tuning side effects or for ordering alignment curricula along the model's own causal directions instead of a human ordering.
- The ternary response coding discards the size of each orientation change, so graded scores might reveal weaker causal edges that the binarized metric misses and could narrow the measured gap between LLM and human value structures.
- Since SAE features are layer-specific, steering the same value through different layers and checking whether successor activation patterns change would test whether the causal graph is a property of the whole model or of the readout layer.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that a latent causal value graph underlies the value dimensions of LLMs, that this structure remains largely unchanged by alignment training, and that it differs from human value systems as described by Schwartz's theory. The graph is learned by applying the PC algorithm to 2,525 orientation vectors obtained from 101 role prompts and 25 SAE steering features across 17 value dimensions (Section 3.2). The learned graph is then used to predict which value dimensions should change when another value is steered, comparing against reference graphs generated by GPT-4o, by the LLMs themselves, and by ValueBench's hierarchical structure (Sections 2.3 and 3.2.1). The paper also claims that SAE steering is more targeted than role-based prompting, based on the number of source nodes affected and on steering consistency metrics (Section 3.3).
Significance. If the causal interpretation were justified, the paper would offer a useful tool for explainable and controllable value steering in LLMs, and its comparison with human value theories would be an important empirical finding. The authors are to be commended for using held-out roles and questions in the evaluation, for testing multiple reference graphs, and for providing detailed tables of steering effects. However, the manuscript's central claim of a latent causal value graph is not supported by the methods: the data generation process violates the assumptions of the causal discovery algorithm, the evaluation metric is not an external validity check, and the 'human value system' comparison is based on a language model's output rather than human data. The practical contribution of a steering co-variation graph remains plausible, but the paper as written does not establish its causal or human-comparison conclusions.
major comments (4)
- [Section 3.2, PC application on 2,525 orientation vectors] The PC algorithm is applied to data whose rows are generated by deliberately varying role prompts and SAE features, which are not included among the 17 value nodes. Under PC's assumption of causal sufficiency, the absence of these two manipulated variables means that they act as unobserved common causes of many value dimensions simultaneously. The resulting graph is therefore not identified as a causal graph among values; it can at best be interpreted as a dependency summary of steering-induced co-variation. The held-out role split in Section 3.2.1 does not repair this, because the same confounding mechanism operates on the test rows.
- [Section 2.3, Eq. (1) and Section 3.2.1] The evaluation metric is not an external validity check. The 'expected' successors are defined as successors in the learned graph, and c(v', v) measures the frequency with which v' changes when v changes under the same kind of role and SAE steering interventions used to build the graph. Hence the metric is essentially a train/test evaluation of dependency co-variation, not a test of whether the graph captures causal effects. The higher prediction numbers compared with the GPT-4o reference graph are expected, because the learned graph is fit to the same type of data that defines the evaluation.
- [Section 3.2 and Appendix A.3] The claim that the LLM value structure differs from human value systems is not supported by the evidence. The 'reference graph' representing human values is generated by GPT-4o from a prompt based on Schwartz's theory, rather than from human questionnaire data. The abstract and Remark 1 assert a difference from human value systems; the data only support a difference from one language model's paraphrase of Schwartz's theory. No human subjects data are collected or analyzed anywhere in the manuscript.
- [Section 2.1 and Section 3.2] The description of the procedure as 'passive causal discovery' is misleading and the required assumptions are unmet. PC requires i.i.d. samples from a fixed distribution satisfying Markov, faithfulness, and causal sufficiency. Here the rows are produced by active interventions (role changes and SAE feature amplifications at a chosen layer and strength), so the samples are not i.i.d. from a fixed distribution. The manuscript neither verifies nor discusses any of these assumptions, nor does it report sensitivity of the recovered graph to the PC significance level alpha=0.05, the choice of SAE layer, the amplification magnitude, or the number of selected features. Without such evidence, the causal claim is not identifiable from these data.
minor comments (5)
- [Section 3.2.1, Figure 4 caption] The caption refers to 'Gemma-3B-IT' while the text and Table 1 refer to 'Gemma-2B-IT'; this inconsistency should be corrected.
- [Section 2.3] The notation sr0[v] and the baseline role r0 are introduced informally; the definition should state explicitly that r0 is a fixed baseline role and that sr0[v] is its orientation on value v.
- [Figures 4, 6, and 7] The text and figure captions repeatedly use 'casual graph' where 'causal graph' is intended.
- [Section 3.1] The explanation of the data sampling is confusing: it says 40% of training data are randomly sampled for each role-SAE dyad, but the relationship between this sampling and the 70%/30% question split is not made clear.
- [Limitations] The Limitations section mentions the breadth of ValueBench and the scale of models, but it does not acknowledge the fundamental limitations of the causal discovery step, namely the unobserved confounders introduced by role and SAE steering and the absence of human data for the value comparison.
Circularity Check
No significant circularity: the causal graph is discovered from training interventions and validated on held-out roles and question splits, so the evaluation is not equivalent to the fitting input.
full rationale
The paper's derivation chain is not circular in the sense defined by the review criteria. The value-orientation vectors are computed from questionnaire responses; the PC algorithm is applied to 2,525 training rows (101 roles x 25 SAE features) to produce the graph; and the evaluation in Section 3.2.1 measures, on held-out roles and a disjoint 30% split of value questions, how often successor versus non-successor nodes actually change when a value is steered. The prediction metric is defined in terms of the graph's own successor sets, but it is applied to data not used to fit the graph, so it is an internal-consistency holdout test rather than a fitted input renamed as a prediction. The 'latent causal value graph' label is open to a serious identification critique, because the manipulated role and SAE-feature variables are unobserved common causes that PC's causal-sufficiency assumption does not permit, but that is a correctness or validity problem, not circularity. Self-citations appear only in related-work survey context and do not carry the load-bearing argument. No quoted equation or fitted parameter is equivalent by construction to the claimed output.
Assumptions & free parameters
free parameters (6)
- PC algorithm significance level =
0.05
- Number of SAE steering features =
25
- SAE steering amplitude =
100x
- SAE layer selection =
Gemma layer 12, Llama layer 25
- Value selection threshold =
>20 questions
- Training/validation split =
100 train / 25 test roles; 70%/30% QA splits; 40% sampling
assumptions (5)
- domain assumption PC algorithm assumptions hold: causal sufficiency, faithfulness, acyclicity.
- domain assumption Value orientation vectors measured by averaging ternary responses faithfully represent latent value states.
- domain assumption GPT-4o-based answer judgment is accurate for ambiguous responses.
- domain assumption The 17 selected value dimensions are a sufficient representation of the LLM's value system.
- ad hoc to paper Reference graph generated by GPT-4o guided by Schwartz's theory approximates the human value system.
Cite this review
Pith. "Pith review of Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective." pith.science (2026). https://pith.science/paper/VI7DRTES
@misc{pith2026250100581,
author = {Pith},
title = {Pith review of: Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective},
year = {2026},
howpublished = {\url{https://pith.science/paper/VI7DRTES}},
note = {Machine review of arXiv:2501.00581}
}
read the original abstract
As large language models (LLMs) become increasingly integrated into critical applications, aligning their behavior with human values presents significant challenges. Current methods, such as Reinforcement Learning from Human Feedback (RLHF), typically focus on a limited set of coarse-grained values and are resource-intensive. Moreover, the correlations between these values remain implicit, leading to unclear explanations for value-steering outcomes. Our work argues that a latent causal value graph underlies the value dimensions of LLMs and that, despite alignment training, this structure remains significantly different from human value systems. We leverage these causal value graphs to guide two lightweight value-steering methods: role-based prompting and sparse autoencoder (SAE) steering, effectively mitigating unexpected side effects. Furthermore, SAE provides a more fine-grained approach to value steering. Experiments on Gemma-2B-IT and Llama3-8B-IT demonstrate the effectiveness and controllability of our methods.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, and Andy et al. Jones. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073
arXiv 2022
-
[2]
Pablo Biedma, Xiaoyuan Yi, Linus Huang, Maosong Sun, and Xing Xie. 2024. https://api.semanticscholar.org/CorpusID:269282597 Beyond human norms: Unveiling unique values of large language models through interdisciplinary approaches . ArXiv, abs/2404.12744
arXiv 2024
-
[3]
Joseph Bloom and David Chanin. 2024. Saelens. https://github.com/jbloomAus/SAELens
work page 2024
-
[4]
Denny Borsboom, Marie K Deserno, Mijke Rhemtulla, Sacha Epskamp, Eiko I Fried, Richard J McNally, Donald J Robinaugh, Marco Perugini, Jonas Dalege, Giulio Costantini, et al. 2021. Network analysis of multivariate data in psychological science. Nature Reviews Methods Primers, 1(1):58
work page 2021
-
[5]
Lynn Boschloo, Claudia D van Borkulo, Mijke Rhemtulla, Katherine M Keyes, Denny Borsboom, and Robert A Schoevers. 2015. The network structure of symptoms of the diagnostic and statistical manual of mental disorders. PloS one, 10(9):e0137621
work page 2015
-
[6]
Andrei Boutyline and Stephen Vaisey. 2017. Belief network analysis: A relational approach to understanding the structure of attitudes. American journal of sociology, 122(5):1371--1447
work page 2017
-
[7]
Mark J Brandt, Chris G Sibley, and Danny Osborne. 2019. What is central to political belief system networks? Personality and Social Psychology Bulletin, 45(9):1352--1364
work page 2019
-
[8]
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Ch...
2023
Show all 44 references
-
[9]
Samuel Cahyawijaya, Delong Chen, Yejin Bang, Leila Khalatbari, Bryan Wilie, Ziwei Ji, Etsuko Ishii, and Pascale Fung. 2024. High-dimension human value representation in large language models. arXiv preprint arXiv:2404.07900
2024 arXiv
-
[10]
Giulio Costantini, Daniele Saraulli, and Marco Perugini. 2020. Uncovering the motivational core of traits: The case of conscientiousness. European Journal of Personality, 34(6):1073--1094
2020
-
[11]
Ang \'e lique OJ Cramer, Sophie Van der Sluis, Arjen Noordhof, Marieke Wichers, Nicole Geschwind, Steven H Aggen, Kenneth S Kendler, and Denny Borsboom. 2012. Dimensions of normal personality as networks in search of equilibrium: You can't like parties if you don't like people...
2012
-
[12]
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. http://arxiv.org/abs/2309.08600 Sparse autoencoders find highly interpretable features in language models . (arXiv:2309.08600). ArXiv:2309.08600 [cs]
2023 arXiv
-
[13]
Jonas Dalege, Denny Borsboom, Frenk Van Harreveld, Helma Van den Berg, Mark Conner, and Han LJ Van der Maas. 2016. Toward a formalized account of attitudes: The causal attitude network (can) model. Psychological review, 123(1):2
2016
-
[14]
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783
2024 arXiv
-
[15]
Jacob Elder, Bernice Cheung, Tyler Davis, and Brent Hughes. 2023. Mapping the self: A network approach for understanding psychological and neural representations of self-concept structure. Journal of personality and social psychology, 124(2):237
2023
-
[16]
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of sup...
2022
-
[17]
Xue Feng, Long Wang, and Simon A Levin. 2019. Dynamic analysis and decision-making in disease-behavior systems with perceptions. In 2019 Chinese Control And Decision Conference (CCDC), pages 665--670. IEEE
2019
-
[18]
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093v1
2024 arXiv
-
[19]
Yizhe Huang, Anji Liu, Fanqi Kong, Yaodong Yang, Song-Chun Zhu, and Xue Feng. 2024 a . Efficient adaptation in mixed-motive environments via hierarchical opponent modeling and planning. arXiv preprint arXiv:2406.08002
2024 arXiv
-
[20]
Yizhe Huang, Xingbo Wang, Hao Liu, Fanqi Kong, Aoyang Qin, Min Tang, Song-Chun Zhu, Mingjie Bi, Siyuan Qi, et al. 2024 b . Adasociety: An adaptive environment with social structures for multi-agent decision-making. arXiv preprint arXiv:2411.03865
2024 arXiv
-
[21]
Kosuke Imai. 2022. Causal diagram and social science research. In Probabilistic and Causal Inference: The Works of Judea Pearl, pages 647--654
2022
-
[22]
Yipeng Kang, Tonghan Wang, and Gerard de Melo. 2020. Incorporating pragmatic reasoning communication into emergent language. Advances in neural information processing systems, 33:10348--10359
2020
-
[23]
Fanqi Kong, Yizhe Huang, Song-Chun Zhu, Siyuan Qi, and Xue Feng. 2024. Learning to balance altruism and self-interest based on empathy in mixed-motive games. arXiv preprint arXiv:2410.07863
2024 arXiv
-
[24]
Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2023. https://openreview.net/forum?id=aLLuYpn83y Inference-time intervention: Eliciting truthful answers from a language model . In Thirty-seventh Conference on Neural Information Processing Systems
2023
-
[25]
Yihuan Mao, Yipeng Kang, Peilun Li, Ning Zhang, Wei Xu, and Chongjie Zhang. 2024. Ibgp: Imperfect byzantine generals problem for zero-shot robustness in communicative multi-agent systems. arXiv preprint arXiv:2410.16237
2024 arXiv
-
[26]
David K Marcus, Jonathan Preszler, and Virgil Zeigler-Hill. 2018. A network of dark personality traits: What lies at the heart of darkness? Journal of Research in Personality, 73:56--62
2018
-
[27]
Eric Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller
Samuel Marks, Can Rager, J. Eric Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2024. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv preprint arXiv:2403.19647v2
2024 arXiv
-
[28]
Ouyang, J
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, and et al. A. Ray. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems
2022
-
[29]
Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang, and Guojie Song. 2024. https://arxiv.org/abs/2406.04214 Valuebench: Towards comprehensively evaluating value orientations and understanding of large language models . Preprint, arXiv:2406.04214
2024 arXiv
-
[30]
Julia M Rohrer. 2018. Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in methods and practices in psychological science, 1(1):27--42
2018
-
[31]
Claudia Russo, Francesca Danioni, Ioanaand Zagrean, and Daniela Barni. 2022. Changing personal values through value-manipulation tasks: A systematic literature review based on schwartz’s theory of basic human values. European Journal of Investigation in Health, Psychology and ...
2022
-
[32]
Ois \' n Ryan, Laura F Bringmann, and No \'e mi K Schuurman. 2022. The challenge of generating causal hypotheses using network models. Structural Equation Modeling: A Multidisciplinary Journal, 29(6):953--970
2022
-
[33]
Shalom H Schwartz. 1992. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in experimental social psychology/Academic Press
1992
-
[34]
Shalom H Schwartz. 2012. An overview of the schwartz theory of basic values. Online readings in Psychology and Culture, 2(1):11
2012
-
[35]
Shalom H Schwartz and Klaus Boehnke. 2004. Evaluating the structure of human values with confirmatory factor analysis. Journal of research in personality, 38(3):230--255
2004
-
[36]
Shalom H Schwartz, Jan Cieciuch, Michele Vecchione, Eldad Davidov, Ronald Fischer, Constanze Beierlein, Alice Ramos, Markku Verkasalo, Jan-Erik L \"o nnqvist, Kursad Demirutku, et al. 2012. Refining the theory of basic individual values. Journal of personality and social psych...
2012
-
[37]
Hua Shen, Tiffany Knearem, Reshmi Ghosh, Yu-Ju Yang, Tanushree Mitra, and Yun Huang. 2024. Valuecompass: A framework of fundamental values for human-ai alignment. arXiv preprint arXiv:2049.09586v1
2024
-
[38]
Peter Spirtes, Clark Glymour, and Richard Scheines. 2001. Causation, prediction, and search. MIT press
2001
-
[39]
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295
2024 arXiv
-
[40]
Vazquez, Ulisse Mini, and Monte MacDiarmid
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. 2024. https://arxiv.org/abs/2308.10248 Activation addition: Steering language models without optimization . Preprint, arXiv:2308.10248
2024 arXiv
-
[41]
Jing Yao, Xiaoyuan Yi, Xiting Wang, Yifan Gong, and Xing Xie. 2023. Value fulcra: Mapping large language models to the multidimensional spectrum of basic human values. arXiv preprint arXiv:2311.10766v1
2023 arXiv
-
[42]
Luyao Yuan, Xiaofeng Gao, Zilong Zheng, Mark Edmonds, Ying Nian Wu, Federico Rossano, Hongjing Lu, Yixin Zhu, and Song-Chun Zhu. 2022. In situ bidirectional human-robot value alignment. Science robotics, 7(68):eabm4183
2022
-
[43]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[44]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.