Pith. sign in

REVIEW 3 major objections 5 minor 107 references

Strongly aligning personalized AI assistants to users' historical values locks populations into outdated norms and measurably slows societal adaptation to shifting conditions.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:11 UTC pith:SN6R65NR

load-bearing objection A useful population-level model with a real formal result on value lock-in, but the abstract overreaches: normative mode collapse is driven by social coupling, not by AI alignment strength. the 3 major comments →

arxiv 2607.18506 v1 pith:SN6R65NR submitted 2026-07-20 cs.CY

AI Value Alignment for Evolving Social Norms

Nenad Toma\v{s}ev , Matija Franklin , Simon Osindero This is my paper
classification cs.CY
keywords AI alignmentvalue lock-innormative mode collapseopinion dynamicssocial physicspersonalized assistantspreference changeagent-based simulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that static alignment—an AI assistant that anchors a user to their own past values—is not merely neutral but actively harmful in a changing society. It builds a mathematical model of user-AI pairs in a drifting normative environment and shows analytically that the steady-state tracking error grows with alignment strength, so no fixed positive level of alignment is optimal. Simulations and a spectral analysis further show that strong alignment slows recovery from normative shocks and, combined with strong social coupling, erases sub-cultural diversity. The authors argue that alignment should be temporally adaptive, allowing users to explore and update values. If correct, the model implies that personalized assistants designed to maximize fidelity to historical preferences will systematically degrade population-level adaptation.

Core claim

Under constant environmental drift, total steady-state error strictly increases with alignment strength α for all α>0 (Theorem 4.1). Because the AI model M is an exponential moving average of the user's past values, a stronger pull toward M compounds the lag of both user and AI relative to the drifting optimum, and the trust-dependent composite utility cannot escape this cost. At population level, strong social coupling collapses distinct sub-cultural optima onto the global mean (Theorem 4.2), and with an endogenous environment, alignment drag slows institutional progress ('double stagnation'). The authors read these as formal consequences of violating the principle that rational choice weig

What carries the argument

The coupling of two linear update rules: user values V evolve under a restoring force α(M−V) toward the AI's internal model M, which is itself an exponential moving average of the user's values with learning rate λ; environmental learning pulls V toward the drifting optimum E(t). Alignment strength α acts as a moving historical anchor, the composite utility with usage probability P=exp(−γ||V−M||²) creates a trust trap, and the graph Laplacian's spectral decomposition (Fiedler value) controls the rate of normative mode collapse.

Load-bearing premise

The monotonicity result assumes AI influence is a linear pull proportional to the gap between a historical user model and current values, and that the environment drifts at constant velocity; if real influence depends on capability, persuasion content, or social context, or if environmental change is punctuated and unpredictable, the derived growth of tracking error with alignment strength may not transfer.

What would settle it

Simulate the same user-AI pair with a nonlinear influence term, e.g. α·tanh(M−V) or an influence that saturates with misalignment; if total steady-state error no longer increases monotonically with the strength parameter, the theorem fails. Empirically, measure adaptation speed to a known norm shift in users of strongly vs. weakly personalized assistants; if adaptation is not slowed by greater alignment, the model's core prediction is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • In any environment with constant drift, raising static alignment strength strictly increases steady-state maladaptation; utility is maximized only in the limit α→0⁺.
  • Following a sudden normative shock, strongly aligned populations recover substantially more slowly and can remain in a low-utility equilibrium while still trusting the stale AI.
  • Strong social coupling erases sub-cultural diversity: the steady-state population converges to the global mean environment, with minimal utility loss equal to the variance of local optima.
  • When the environment is co-constructed by the population, strong alignment decelerates institutional and societal change, a 'double stagnation' effect.
  • Adaptive alignment—reducing α when tracking error is high, and optionally increasing the AI learning rate λ—can release users from historical anchors and restore recovery after shocks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable prediction follows: users of strongly personalized assistants should show measurably slower value change on documented normative shifts (e.g. attitudes to remote work) than users of weakly aligned tools, after controlling for information exposure.
  • The monotonicity theorem relies on the linear restoring-force structure; if real influence is saturating or context-dependent (e.g. persuasion quality), the strict result may weaken, though the qualitative lock-in direction likely persists for any influence that points toward historical values.
  • The paper's normative-direction assumption means the model cannot distinguish adaptive from harmful drift; static alignment might be protective in some catastrophic scenarios, a boundary the authors explicitly flag.
  • The framework suggests a cheap pre-screening for agentic evaluations: inexpensive ODE simulations can locate parameter regions where value lock-in and mode collapse concentrate before running large-scale multi-agent LLM simulations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a continuous-time and agent-based 'social physics' model in which each user's value vector V_i is pulled by intrinsic noise, social neighbors, the environment E_i, and an AI alignment force α(M_i−V_i), where M_i is the AI's exponentially moving average model of the user's values. The authors derive the single-agent steady-state tracking error under constant drift E(t)=vt (Eqs. 16-18), prove monotonicity of composite utility error in α (Thm 4.1), analyse collective consensus with Laplacian dynamics (Thm 4.2), and present simulations of drift, shocks, and adaptive alignment. They conclude that static alignment strength causes value lock-in and normative mode collapse, and that adaptive alignment mitigates these risks.

Significance. The analytical core of the paper is mostly sound: Eq. (16) is correct under the stated linear dynamics, Thm 4.1's derivative (Eq. 21) is positive, and Thm 4.2 is a clean Laplacian consensus limit. The paper is unusually transparent about its limitations, and the model isolates a mechanism — historical anchoring via personalized assistants — that deserves study. However, the headline claim that static alignment drives normative mode collapse is not supported by the model's equations, because the mode-collapse steady state is α-independent. The simulation evidence also lacks statistical detail. The contribution is valuable as a tractable 'epistemic bridge' model, but the over-claims must be corrected before publication.

major comments (3)
  1. [Abstract and §6; §4.2, Eqs. (23)-(27)] The mode-collapse result is independent of alignment strength α. In the static-environment steady state, Ṁ_i = λ(V_i − M_i) = 0 forces M_i = V_i, so the term α(M_i − V_i) in Eq. (23) vanishes identically; the equilibrium (25) and the collapse to the global mean E-bar (27) contain no α. The collapse is controlled only by w_soc/w_learn and the graph Laplacian. Accordingly, the abstract's phrase 'normative mode collapse, prominently featured in non-adaptive alignment formulations' and the conclusion's attribution of mode collapse to static alignment are not supported by the model's own equations. This is a load-bearing over-claim: either remove mode collapse from the abstract/conclusion or add an analysis showing that α accelerates convergence to the collapse (the rate in Appendix A.2 depends on α, but the attractor does not).
  2. [§3, Figures 2-10] Simulation results are presented without seed counts, error bars, or any measure of variance. Figures 4 and 5 assert monotone effects ('notable decrease', 'significantly longer recovery periods') from single curves and heatmaps with hand-chosen parameters. Add results over at least 10-20 seeds with confidence bands and a sensitivity analysis for λ, w_soc, γ, or soften the quantitative wording to qualitative model behavior.
  3. [§4.4, Eq. (32), Fig. 17] The adaptive-alignment proposal is the paper's main positive recommendation, but it is supported only by one illustrative continuous-time simulation. No analytic guarantees are given for Eq. (32) (e.g., stability, boundedness of α(t), convergence), and no sensitivity sweep over η, κ, α_target is reported. Since the controller relies on the tracking error ||V−E||, which is not directly observable in practice (acknowledged in §5.5), the prescriptive conclusion that adaptive alignment 'may be preferable' should be presented as a tentative model-based hypothesis, not as an established result.
minor comments (5)
  1. [Eq. (13)] P(t) is used before P(use|align) is defined; define the dependency explicitly.
  2. [§4.1, before Eq. (16)] 'decaying function of the squared alignment gap' should be 'squared distance to the environment optimum' — the alignment gap is ||V−M||, not ||V−E||.
  3. [Figure 7] The y-axis label 'Distance from Optimal Environment' appears to be normalised in the main text but the normalisation is not specified.
  4. [Figures 17-18] The adaptive-controller parameters η, κ, α_target are not listed in Table 1; add them to the caption or text.
  5. [References] Several cited works are dated 2026 and appear to be preprints (Kanwal & Tran; Tsirtsis et al.; Marchal et al.); label them as preprints or forthcoming in the bibliography.

Circularity Check

0 steps flagged

No circular reduction in the analytic derivations; the normative-mode-collapse attribution to static alignment is an over-claim, not a circular step.

full rationale

The core analytic results follow algebraically from the model's own stated update rules (Eqs. 3, 8, 9, 14, 15) under explicitly stated simplifications (E(t)=v t, static environment, mean-field abstraction). For example, Eq. 16 is obtained by transforming to error coordinates and solving V-dot = M-dot = 0; Theorem 4.1 differentiates the resulting expression (Eq. 20) in alpha. These are internal model consequences, not fitted parameters renamed as predictions, and no external dataset is used for calibration. There is therefore no fitted-input-called-prediction or self-definitional circularity. The paper's self-citations (Ashton & Franklin 2022, Franklin et al. 2022, Zhi-Xuan et al. 2025, Leibo et al.) are background and motivation, not load-bearing derivations; no uniqueness theorem or ansatz is imported from the authors' prior work. The skeptical observation about normative mode collapse is best read as a scope/correctness issue rather than circularity: Section 4.2 explicitly states that in the static-environment steady state the alignment term alpha(M-V) vanishes, and Eq. 25 contains no alpha. Thus Theorem 4.2's collapse to the global mean is driven by w_soc/w_learn and graph structure, not by alignment strength. The abstract/conclusion attribution of normative mode collapse to static alignment is therefore unsupported by the model's own math, but it is not a case of the derivation assuming its conclusion; it is an interpretive over-claim. The paper's limitation section (5.5) candidly acknowledges the modelling abstractions and the lack of real-world validation, which further confirms that the gap is empirical validation, not circular derivation. Score 2 reflects minor non-load-bearing self-citation and the noted over-attribution, not a circular reduction.

Axiom & Free-Parameter Ledger

9 free parameters · 6 axioms · 4 invented entities

The model is a deliberately simplified toy system. Every conclusion is a consequence of the hand-chosen linear update rules and the exponential utility function. The free parameters are not fitted to human or AI behavioral data, so the quantitative regimes are illustrative. The central qualitative results—more anchoring leads to more lag; Laplacian consensus erases diversity—are mathematically robust within the model but not empirically grounded.

free parameters (9)
  • alpha (alignment strength) = 0.1 default; swept 0-1
    Central anchor term controlling AI pull on user values; all lock-in results scale with it.
  • lambda (AI learning rate) = 0.1 default; swept
    EMA update rate for the AI value model; determines steady-state alignment gap.
  • w_soc (social influence) = 0.05 default; swept
    Controls consensus strength and normative mode collapse.
  • w_learn (environmental adaptation) = 0.05 default; swept
    The only force pulling users toward the environment; inverse of this weight sets tracking error.
  • w_explore (intrinsic exploration) = 0.001 default; 0.015 in shock runs
    Noise term in value updates; affects denoising and exploration results.
  • beta, gamma (utility and alignment sensitivities) = beta=2.0, gamma=5.0
    Shape the exponential utility and AI-usage probability; not empirically calibrated.
  • delta / drift speed v = 0.002 per step; v in analytical ramp
    Rate of environmental change; tracking error grows with drift speed.
  • shock parameters p_shock, D_shock = p_shock varied; D_shock=0.8, extreme 1.5
    Punctuated-equilibria scenarios that set recovery-time results.
  • adaptive-controller parameters eta, kappa_E, alpha_target, eta_lambda = not fully specified in text
    Ad hoc gains in Section 4.4; no empirical grounding.
axioms (6)
  • domain assumption Values are points in R^D and update linearly.
    Section 2.1; excludes polarisation, backfire effects, hysteresis, and nonlinear belief change, as noted in Limitations.
  • domain assumption The environment optimum E(t) is known to the modeller and is the normatively appropriate target.
    Section 2.3; the authors themselves note that harmful drift cases and resistance to societal processes are not distinguished.
  • ad hoc to paper AI influence on user values is a linear attraction toward the EMA model M: alpha*(M - V).
    Section 2.2, Eq. 8; no empirical derivation; capability and task delegation are abstracted away in the base model.
  • domain assumption AI model updates by exponential moving average: M <- (1-lambda)M + lambda V.
    Eq. 3; different learning mechanisms would change the steady-state lag and recovery dynamics.
  • domain assumption Social influence is mediated by an undirected graph Laplacian.
    Eq. 23; Theorem 4.2 assumes an undirected connected graph; directed, weighted, or scale-free networks require the extensions in Appendix B.
  • ad hoc to paper The analytical tracking-error results assume constant-velocity drift E(t) = v t.
    Section 4.1.1; the paper calls this a 'standard analytical simplification,' but it omits stochastic volatility that would add variance to tracking error.
invented entities (4)
  • alignment strength alpha as a scalar force no independent evidence
    purpose: Models the AI assistant's pull on user values toward its historical model.
    No measurement or calibration is provided; alpha is a hand-set simulation parameter but is central to all conclusions.
  • trust trap no independent evidence
    purpose: Explains sustained high AI usage even when the user-AI pair is misaligned with the environment.
    Emergent from the composite-utility model; qualitative analogy to filter bubbles, not directly observed.
  • double stagnation no independent evidence
    purpose: Describes the feedback where anchored users slow down institutional and environmental change.
    Derived in the endogenous-drift extension; no empirical evidence is offered.
  • sycophancy index s no independent evidence
    purpose: Modulates the user's perceived alignment error, making trust traps harder to escape.
    Motivated by cited sycophancy literature, but the parameter itself is not measured.

pith-pipeline@v1.3.0-alltime-deepseek · 34448 in / 12208 out tokens · 140593 ms · 2026-08-01T15:11:01.178835+00:00 · methodology

0 comments
read the original abstract

AI alignment is essential for the safe deployment of advanced AI systems. Given that values and preferences change over time, culture, social roles, and context, we need to develop a better understanding of the possible long-term consequences of AI alignment, in particular considering the likely ubiquitous future use of personalized AI assistants. We introduce a flexible and extensible mathematical modelling framework, rooted in social physics, aimed at answering macro-level questions regarding the evolving social norms in human populations under the assumption of frequent AI use. Our analysis is part-analytical, and part-simulation, enabling us to characterize the long-term dynamical consequences under a diverse set of starting assumptions. We highlight the risk of value lock-in, and normative mode collapse, prominently featured in non-adaptive alignment formulations. Beyond alignment, we advocate for the wider adoption of these kinds of social physics models as an epistemic bridge: enabling rapid, rigorous, and quantitatively-grounded hypothesis testing for sociotechnical foresight in general AI futures, and acting as a tractable precursor to more computationally expensive large-scale agentic evaluations.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

107 extracted references · 16 linked inside Pith

  1. [1]

    2019 , publisher=

    Human compatible: AI and the problem of control , author=. 2019 , publisher=

    2019

  2. [2]

    2020 , publisher=

    The alignment problem: Machine learning and human values , author=. 2020 , publisher=

    2020

  3. [3]

    2025 , eprint=

    Steerable Pluralism: Pluralistic Alignment via Few-Shot Comparative Regression , author=. 2025 , eprint=

    2025

  4. [4]

    2025 , eprint=

    Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt , author=. 2025 , eprint=

    2025

  5. [5]

    2024 , eprint=

    A theory of appropriateness with applications to generative artificial intelligence , author=. 2024 , eprint=

    2024

  6. [6]

    2022 , publisher=

    What we owe the future: The sunday times bestseller , author=. 2022 , publisher=

    2022

  7. [7]

    2605.16245 , archivePrefix=

    Stratis Tsirtsis and Kai Rawal and Chris Russell and Brent Mittelstadt and Sandra Wachter , year=. 2605.16245 , archivePrefix=

    Pith/arXiv arXiv

  8. [8]

    Concrete problems in

    Amodei, Dario and Olah, Chris and Steinhardt, Jacob and Christiano, Paul and Schulman, John and Man. Concrete problems in. arXiv preprint arXiv:1606.06565 , year=

    Pith/arXiv arXiv

  9. [9]

    International Journal of Human-Computer Studies , volume=

    Does automation bias decision-making? , author=. International Journal of Human-Computer Studies , volume=. 1999 , publisher=

    1999

  10. [10]

    arXiv preprint arXiv:2402.05070 , year=

    A roadmap to pluralistic alignment , author=. arXiv preprint arXiv:2402.05070 , year=

    Pith/arXiv arXiv

  11. [11]

    New media & society , volume=

    I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience , author=. New media & society , volume=. 2011 , publisher=

    2011

  12. [12]

    arXiv preprint arXiv:2108.07258 , year=

    On the opportunities and risks of foundation models , author=. arXiv preprint arXiv:2108.07258 , year=

    Pith/arXiv arXiv

  13. [13]

    2008 , publisher=

    The difference: How the power of diversity creates better groups, firms, schools, and societies-new edition , author=. 2008 , publisher=

    2008

  14. [14]

    Bostrom, Nick , year =

  15. [15]

    arXiv preprint arXiv:1901.00064 , year=

    Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function) , author=. arXiv preprint arXiv:1901.00064 , year=

    Pith/arXiv arXiv 1901

  16. [16]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    Pith/arXiv arXiv

  17. [17]

    arXiv preprint arXiv:2504.12501 , year=

    Reinforcement learning from human feedback , author=. arXiv preprint arXiv:2504.12501 , year=

    Pith/arXiv arXiv

  18. [18]

    2013 , publisher=

    Social acceleration: A new theory of modernity , author=. 2013 , publisher=

    2013

  19. [19]

    2020 , publisher=

    Cultural evolution: People’s motivations are changing, and reshaping the world , author=. 2020 , publisher=

    2020

  20. [20]

    2015 , publisher=

    Foragers, farmers, and fossil fuels: How human values evolve , author=. 2015 , publisher=

    2015

  21. [21]

    The Oxford handbook of digital ethics , pages=

    The challenge of value alignment , author=. The Oxford handbook of digital ethics , pages=. 2022 , publisher=

    2022

  22. [22]

    Ji, Jiaming and Qiu, Tianyi and Chen, Boyuan and Zhang, Borong and Lou, Hantao and Wang, Kaile and Duan, Yawen and He, Zhonghao and Zhou, Jiayi and Zhang, Zhaowei and others , journal=

  23. [23]

    Minds and machines , volume=

    Artificial intelligence, values, and alignment , author=. Minds and machines , volume=. 2020 , publisher=

    2020

  24. [24]

    Advances in Neural Information Processing Systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in Neural Information Processing Systems , volume=

  25. [25]

    Proceedings of the 12th ACM conference on recommender systems , pages=

    How algorithmic confounding in recommendation systems increases homogeneity and decreases utility , author=. Proceedings of the 12th ACM conference on recommender systems , pages=

  26. [26]

    2026 , eprint=

    Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction , author=. 2026 , eprint=

    2026

  27. [27]

    2011 , publisher=

    The filter bubble: What the Internet is hiding from you , author=. 2011 , publisher=

    2011

  28. [28]

    Journal of economic literature , volume=

    Endogenous preferences: The cultural consequences of markets and other economic institutions , author=. Journal of economic literature , volume=. 1998 , publisher=

    1998

  29. [29]

    Beyond Preferences in

    Zhi-Xuan, Tan and Carroll, Micah and Franklin, Matija and Ashton, Hal , journal=. Beyond Preferences in. 2025 , publisher=

    2025

  30. [30]

    arXiv preprint arXiv:2203.10525 , year=

    Recognising the importance of preference change: A call for a coordinated multidisciplinary research effort in the age of AI , author=. arXiv preprint arXiv:2203.10525 , year=

    Pith/arXiv arXiv

  31. [31]

    2022 , organization=

    The problem of behaviour and preference manipulation in AI systems , author=. 2022 , organization=

    2022

  32. [32]

    arXiv preprint arXiv:2603.02960 , year=

    Architecting Trust in Artificial Epistemic Agents , author=. arXiv preprint arXiv:2603.02960 , year=

    arXiv

  33. [33]

    Journal of personality and social psychology , volume=

    Mapping the moral domain , author=. Journal of personality and social psychology , volume=. 2011 , publisher=

    2011

  34. [34]

    2012 , publisher=

    The righteous mind: Why good people are divided by politics and religion , author=. 2012 , publisher=

    2012

  35. [35]

    An overview of the

    Schwartz, Shalom H , journal=. An overview of the

  36. [36]

    Proceedings of the ACM on Human-Computer Interaction , volume=

    The Internet's hidden rules: An empirical study of Reddit norm violations at micro, meso, and macro scales , author=. Proceedings of the ACM on Human-Computer Interaction , volume=. 2018 , publisher=

    2018

  37. [37]

    Ethayarajh, Kawin and Xu, Winnie and Muennighoff, Niklas and Jurafsky, Dan and Kiela, Douwe , journal=

  38. [38]

    1997 , publisher=

    Spectral graph theory , author=. 1997 , publisher=

    1997

  39. [39]

    Czechoslovak mathematical journal , volume=

    Algebraic connectivity of graphs , author=. Czechoslovak mathematical journal , volume=. 1973 , publisher=

    1973

  40. [40]

    Journal of the American Statistical association , volume=

    Reaching a consensus , author=. Journal of the American Statistical association , volume=. 1974 , publisher=

    1974

  41. [41]

    1985 , publisher=

    Culture and the evolutionary process , author=. 1985 , publisher=

    1985

  42. [42]

    Advances in Experimental Social Psychology , volume=

    Moral foundations theory: The pragmatic validity of moral pluralism , author=. Advances in Experimental Social Psychology , volume=. 2013 , publisher=

    2013

  43. [43]

    2012 , publisher=

    Introduction to stochastic control theory , author=. 2012 , publisher=

    2012

  44. [44]

    2010 , publisher=

    Modern control engineering , author=. 2010 , publisher=

    2010

  45. [45]

    Constitutional

    Bai, Yuntao and Kadavath, Saurav and Kundu, Sandipan and Askell, Amanda and Kernion, Jackson and Jones, Andy and Chen, Anna and Goldie, Anna and Mirhoseini, Azalia and McKinnon, Cameron and others , journal=. Constitutional

  46. [46]

    Advances in Experimental Social Psychology , volume=

    Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries , author=. Advances in Experimental Social Psychology , volume=. 1992 , publisher=

    1992

  47. [47]

    2005 , publisher=

    The grammar of society: The nature and dynamics of social norms , author=. 2005 , publisher=

    2005

  48. [48]

    Washington Law Review , volume=

    Privacy as contextual integrity , author=. Washington Law Review , volume=. 2004 , publisher=

    2004

  49. [49]

    2012 , publisher=

    Networked: The new social operating system , author=. 2012 , publisher=

    2012

  50. [50]

    1995 , publisher=

    Value in ethics and economics , author=. 1995 , publisher=

    1995

  51. [51]

    2013 , publisher=

    The epistemology of resistance: Gender and racial oppression, epistemic injustice, and the social imagination , author=. 2013 , publisher=

    2013

  52. [52]

    The Lancet planetary health , volume=

    The relationship between cultural tightness--looseness and COVID-19 cases and deaths: a global analysis , author=. The Lancet planetary health , volume=. 2021 , publisher=

    2021

  53. [53]

    2008 , publisher=

    Convention: A philosophical study , author=. 2008 , publisher=

    2008

  54. [54]

    science , volume=

    Differences between tight and loose cultures: A 33-nation study , author=. science , volume=. 2011 , publisher=

    2011

  55. [55]

    Class: The Anthology , pages=

    Time, work-discipline, and industrial capitalism , author=. Class: The Anthology , pages=. 2017 , publisher=

    2017

  56. [56]

    The secret of our success , year=

    The secret of our success: How culture is driving human evolution, domesticating our species, and making us smarter , author=. The secret of our success , year=

  57. [57]

    arXiv preprint arXiv:2310.13548 , year=

    Towards understanding sycophancy in language models , author=. arXiv preprint arXiv:2310.13548 , year=

    Pith/arXiv arXiv

  58. [58]

    IEEE Transactions on automatic control , volume=

    Consensus problems in networks of agents with switching topology and time-delays , author=. IEEE Transactions on automatic control , volume=. 2004 , publisher=

    2004

  59. [59]

    Model Spec , author=

  60. [60]

    Claude's Character , author=

  61. [61]

    Advances in Neural Information Processing Systems , volume=

    Deep reinforcement learning from human preferences , author=. Advances in Neural Information Processing Systems , volume=

  62. [62]

    Advances in Neural Information Processing Systems , volume=

    Learning to summarize with human feedback , author=. Advances in Neural Information Processing Systems , volume=

  63. [63]

    Advances in Neural Information Processing Systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in Neural Information Processing Systems , volume=

  64. [64]

    Collective constitutional

    Huang, Saffron and Siddarth, Divya and Lovitt, Liane and Liao, Thomas I and Durmus, Esin and Tamkin, Alex and Ganguli, Deep , booktitle=. Collective constitutional

  65. [65]

    arXiv preprint arXiv:2306.16388 , year=

    Towards measuring the representation of subjective global opinions in language models , author=. arXiv preprint arXiv:2306.16388 , year=

    Pith/arXiv arXiv

  66. [66]

    International Conference on Machine Learning , pages=

    Whose opinions do language models reflect? , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  67. [67]

    Carroll, Micah and Foote, Davis and Siththaranjan, Anand and Russell, Stuart and Dragan, Anca , journal=

  68. [68]

    International Conference on Machine Learning , pages=

    Estimating and penalizing induced preference shifts in recommender systems , author=. International Conference on Machine Learning , pages=. 2022 , organization=

    2022

  69. [69]

    arXiv preprint arXiv:2507.09650 , year=

    Cultivating pluralism in algorithmic monoculture: The community alignment dataset , author=. arXiv preprint arXiv:2507.09650 , year=

    arXiv

  70. [70]

    Askell, Amanda and Carlsmith, Joe and Olah, Chris and Kaplan, Jared and Karnofsky, Holden and Anthropic , title =

  71. [71]

    Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages=

    Preference amplification in recommender systems , author=. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages=

  72. [72]

    arXiv preprint arXiv:2009.09153 , year=

    Hidden incentives for auto-induced distributional shift , author=. arXiv preprint arXiv:2009.09153 , year=

    Pith/arXiv arXiv 2009

  73. [73]

    2014 , publisher=

    Transformative experience , author=. 2014 , publisher=

    2014

  74. [74]

    2019 , publisher=

    Choosing for changing selves , author=. 2019 , publisher=

    2019

  75. [75]

    Philosophy & public affairs , pages=

    Rational fools: A critique of the behavioral foundations of economic theory , author=. Philosophy & public affairs , pages=. 1977 , publisher=

    1977

  76. [76]

    Human factors , volume=

    Humans and automation: Use, misuse, disuse, abuse , author=. Human factors , volume=. 1997 , publisher=

    1997

  77. [77]

    Journal of the American Medical Informatics Association , volume=

    Automation bias: a systematic review of frequency, effect mediators, and mitigators , author=. Journal of the American Medical Informatics Association , volume=. 2012 , publisher=

    2012

  78. [78]

    To trust or to think: cognitive forcing functions can reduce overreliance on

    Bu. To trust or to think: cognitive forcing functions can reduce overreliance on. Proceedings of the ACM on Human-computer Interaction , volume=. 2021 , publisher=

    2021

  79. [79]

    Findings of the association for computational linguistics: ACL 2023 , pages=

    Discovering language model behaviors with model-written evaluations , author=. Findings of the association for computational linguistics: ACL 2023 , pages=

    2023

  80. [80]

    arXiv preprint arXiv:2507.21919 , year=

    Training language models to be warm and empathetic makes them less reliable and more sycophantic , author=. arXiv preprint arXiv:2507.21919 , year=

    Pith/arXiv arXiv

Showing first 80 references.