Pith. sign in

REVIEW 3 major objections 4 minor 61 references

Language Agents as Digital Representatives in Collective Decision-Making

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned language models can act as digital representatives of individual people in a consensus-finding task, producing group outcomes that are roughly equivalent in expected payoff and judged similarity to those from the humans…

desk verdict Clean formalization of digital representation, an honest feasibility study, and a proxy-evaluation caveat that has to be taken seriously before the empirical claim is accepted. read the letter →

arxiv 2502.09369 v1 pith:IRX5WUIE submitted 2025-02-13 cs.LG cs.AIcs.CLcs.CY

Daniel Jarrett , Miruna Pîslar , Michiel A. Bakker , Michael Henry Tessler , Raphael Köster , Jan Balaguer , Romuald Elie , Christopher Summerfield
show 1 more author
Andrea Tacchetti
This is my paper · ORCID
classification cs.LGcs.AIcs.CLcs.CY
keywords digitalrepresentativescollectivedecision-makingconsensus-findingvalueequivalencelanguageagentsfine-tuninglargemodelsrepresentativity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a good digital representative of a person in a collective decision is not a clone that says exactly what the person would say, but an agent whose participation in the decision mechanism leads to equivalent final outcomes. It formalizes collective decision-making as an episodic interaction between a group and a decision mechanism, then defines representativity through trajectory-based value equivalence: two profiles are equivalent if, unrolled through the mechanism, they yield the same expected payoffs. In a consensus-finding experiment, the authors fine-tune 1B- and 30B-parameter language models on individuals' past opinions and critiques, then substitute the models' written critiques for the humans' critiques in the mediation pipeline. They report that the resulting consensus statements are roughly equivalent to the human-produced ones under both a learned payoff model and an automated judge, whereas unfine-tuned models clearly degrade the outcomes. A formal result, Proposition 1, ranks three equivalence classes and shows that the trajectory-based class is the appropriate one when the mechanism is invariant to some irrelevant dimensions of language.

What carries the argument

The load-bearing object is a vector-valued Bellman operator $B_{\pi,\tau}$ acting on payoff-value functions $Q: \mathcal{X}\times\mathcal{U}\to\mathbb{R}^n$. The paper defines three equivalence classes of policy profiles through these operators: identical conditionals (clones), equal one-step Bellman effects on a function class $\mathcal{Q}$, and equal $T$-step compositions (trajectory-based value equivalence). Proposition 1 shows $\Pi(\pi^*)\subseteq \Pi(\pi^*,\mathcal{T},\mathcal{Q}) \subseteq \Pi_T(\pi^*,\mathcal{T},\mathcal{Q})$, with the second inclusion proper when mechanisms ignore an irrelevant action subspace $\mathcal{U}_\perp$; this motivates the representativity measure of Equation (16): the worst-case discrepancy, over mechanisms and payoff functions, between the expected outcomes of the true and model profiles.

What would settle it

Recruit a fresh panel of human raters to score the revised consensus statements produced by digital representatives versus those produced by ground-truth critiques, and compare their agreement scores and preference win-rates against the model-based measures; if the human ratings diverge substantially from the payoff model and autorater, the representativity claim would be falsified.

Watch

Extended reading notes

Core claim

Fine-tuned large language models can serve as digital representatives of individual humans in consensus-finding: when their critiques replace the human critiques in the mediator mechanism, the revised consensus statements are roughly equivalent in expected payoff (measured by a learned agreement model) and in automated-judged similarity to the consensuses produced with the humans' own critiques. This holds even though the models were trained only on a standard likelihood objective, not on the equivalence criterion itself. The formal backbone is Proposition 1, which orders the equivalence classes of digital clones (identical conditional behavior), transition-based equivalence (equal one-step Bellman operators), and trajectory-based value equivalence (equal expected payoffs after unrolling the interaction). Under a mechanism class that ignores certain utterance dimensions, the trajectory-based class is strictly larger than the transition-based class, so a representative may freely vary in style as long as the final outcomes coincide.

Load-bearing premise

The evaluation rests on the assumption that the payoff model's agreement scores and the automated judge's win-rates faithfully capture how real humans would rate the consensus statements; if those proxies are biased toward the language models' own output, the measured equivalence could come from model self-similarity rather than fidelity to the person.

Editorial extensions

If this is right

  • Digital representatives could replace human participants in large-scale simulations of consensus-finding, making mechanism design and scenario studies substantially cheaper and faster.
  • The trajectory-based value equivalence criterion provides a principled, task-level definition of representativity that is applicable beyond consensus-finding to other collective decision settings.
  • Fine-tuning on an individual's own past opinions and critiques is the key ingredient: demographic prompting alone performs markedly worse in capturing individual-level style and preferences.
  • Scale helps: the 30B fine-tuned representative reaches win-rates close to the human ground-truth ceiling, while the 1B fine-tuned model still improves over vanilla baselines.
  • The mediation mechanism tolerates a single outlier critique, so full-group substitution is the more sensitive test of representativity; single substitutions tend to wash out differences between models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If human validation confirms the model-based equivalence, the same framework could support 'digital citizens' in deliberative democracy simulations, letting policymakers explore the effect of different deliberation rules before running costly human assemblies.
  • The formal framework is mechanism-agnostic: it should transfer to other collective decision mechanisms such as voting rules or auction formats, where the state-action space is not language but ballot choices or bids.
  • The success of likelihood-based training suggests that directly optimizing the trajectory-equivalence objective (rather than one-step likelihood) could yield better sample efficiency, or may become necessary in settings where the mechanism is not smooth in actions.
  • A natural stress test is distribution shift: evaluate representatives on questions far from the training corpus or on participants with more extreme views to see whether outcome equivalence degrades when the model's prior must do more work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper formalizes collective decision-making as an episodic interaction between participants and a decision mechanism, defines digital representation through equivalence classes of policy profiles (clones, transition-based equivalence, and trajectory-based value equivalence), and proves an ordering result (Proposition 1). It then presents an empirical case study in consensus-finding, fine-tuning 1B and 30B Chinchilla models to generate critiques in place of human participants and evaluating them with log-likelihood, a PaLM2 autorater, and a learned payoff model. The central claim is that fine-tuned language models can act as digital representatives of individual humans, in the sense that substituting their critiques for human critiques yields consensus outcomes with roughly equivalent expected payoff and judged similarity.

Significance. If the central claim is supported, the paper makes a useful conceptual contribution by giving a formal, mechanism-aware definition of what it means for a language agent to represent a human in collective decision-making, and by showing that trajectory-based value equivalence is the right notion. The proof of Proposition 1 is clean in intent, and the empirical study uses a held-out split by both participants and questions, which is a strength. However, the empirical demonstration of feasibility currently rests on model-based proxies from the same research ecosystem (a Chinchilla 1B payoff model from prior work and a PaLM2 autorater), with no human validation of the consensus outcomes. The paper's own formal representativity objective is also not the objective used for training. These gaps are load-bearing for the title claim, even though the authors acknowledge them in Section 5.

major comments (3)
  1. [4.3, Eqs. (21)-(22)] The feasibility claim depends entirely on two model-based proxies: the 1B-parameter Chinchilla payoff model from the authors' prior work and the PaLM2 autorater. No human endorsement data are reported for the consensus statements; Section 5 explicitly defers human validation to future work. If either proxy systematically prefers fluent, mediator-like statements, the reported equivalence between human and digital-representative consensus outcomes could be an artifact of self-similarity among language models rather than fidelity to the represented individuals. This is load-bearing for the central claim, and I request human evaluations on a held-out subset (e.g., human agreement ratings and pairwise preference judgments) or, at minimum, a validation of the payoff model and autorater against human judgments.
  2. [4.1, Eq. (18) vs. Eq. (16)] The digital representatives are trained with a standard log-likelihood objective, i.e., to clone the human critique conditionals, whereas the paper's proposed notion of representativity in Eq. (16) is trajectory-based value equivalence. This mismatch is acknowledged in Section 5, but it means the experiments do not directly instantiate the theoretical framework. The authors should either train with an approximation to the value-equivalence objective or provide evidence that likelihood-trained models also satisfy the trajectory-equivalence criterion in a way that is not solely mediated by the unvalidated proxies.
  3. [Appendix A, Eqs. (29)-(30)] The proof of Proposition 1 factorizes the joint policy as π*((u'||,u'⊥)|x') = π*(u'|||x')π*(u'⊥|x'), which is an independence assumption not stated in the proposition. The intended result can be obtained directly from the invariance hypothesis on Q^t without this factorization, so the statement is likely correct, but the proof as written is invalid at this step. Please revise the proof to avoid the unjustified factorization.
minor comments (4)
  1. [Eq. (16)] In Eq. (16), Q^T is written as Q^T(ω), but Q^T was defined as a function of state and action, Q^T: X × U → R^n; please clarify the notation for evaluating the terminal value function at outcomes.
  2. [Figures 2 and 3] The text reports differences such as "13% difference between ceiling and Vanilla 1B DRs" but does not report confidence intervals or statistical tests; please add error bars and significance tests, or explicitly state their absence and interpret the results accordingly.
  3. [Section 4.3, Eq. (20)] The term "autoreter" appears to be a typo for "autorater"; please correct it for consistency with the rest of the text.
  4. [Section 3.1] The phrase "Expression 10 is simply the singleton class" is slightly misleading, since Π(π*) is an equivalence class of policies, not necessarily a singleton unless Π contains only one policy with the same conditionals; please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: formal result is a proof from definitions, and empirical claims are anchored by held-out human critiques; proxy-based consensus evaluation is an acknowledged limitation, not a circular reduction.

full rationale

The paper's claimed derivation chain is self-contained. Definition 3 and Proposition 1 are a mathematical statement about three equivalence classes, proved in Appendix A directly from the definitions (Eqs. 10-12) without importing the conclusion; the proof constructs a policy and verifies the inclusions, so there is no definitional circularity. The training objective (Eq. 18) maximizes log-likelihood of held-out human critiques, and the first evaluation (Eq. 19, Fig. 2 left) measures log-likelihood on held-out human critiques, an external benchmark that does not depend on the authors' models. The consensus-level evaluation (Eqs. 21-22) uses a payoff model from the authors' prior work [5,6] and a PaLM2 autorater, and Section 5 explicitly acknowledges that human validation is future work; while this is a validity limitation, it is not a circular reduction because the payoff model is a fixed external evaluator, the DRs are not trained to optimize Eq. 21 or Eq. 22, and no equation in the paper defines the reported equivalence in terms of the trained DRs' own outputs. The cited prior work [5,6] contains human data and external validation, so the self-citations are independent support rather than a load-bearing chain.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard MDP machinery plus several domain assumptions about the consensus-finding pipeline. The main ad hoc assumption is the U||/U⊥ action decomposition used only in Proposition 1. The empirical evaluation relies on self-published proxy models for human agreement and on a fine-tuned mediator, which are acknowledged limitations.

assumptions (5)
  • standard math The group decision process is an episodic Markov decision process with a fixed horizon and a terminal-state outcome.
    Used throughout Section 2 to define mechanisms and outcome functions; standard for sequential decision problems.
  • ad hoc to paper The action space can be decomposed into relevant and irrelevant dimensions U = U|| x U⊥, with mechanisms and value functions invariant to U⊥.
    Introduced in Section 3.1 and used in Proposition 1 to show the trajectory-based equivalence class is strictly larger. It is a theoretical convenience, not justified from the language domain.
  • domain assumption The mediator mechanism τ is a fixed black-box function (a fine-tuned 70B Chinchilla) that maps critiques to consensus statements.
    Used in Section 2 Case Study and Section 4; the paper deliberately treats the mediator as a black box and does not model its internal behavior.
  • domain assumption A 1B-parameter payoff model trained by the authors' prior work provides a valid proxy for human agreement scores.
    Used in Section 4.3, Equation 21, to compute payoff discrepancies without running human studies. The authors acknowledge this is a proxy in Section 5.
  • domain assumption Participants do not observe each other's opinions or critiques, so individual behaviors can be treated as independent policies.
    Stated in the Case Study in Section 2; this shapes the MDP formulation and the training and evaluation pipeline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language Agents as Digital Representatives in Collective Decision-Making." pith.science (2026). https://pith.science/paper/IRX5WUIE

@misc{pith2026250209369,
  author       = {Pith},
  title        = {Pith review of: Language Agents as Digital Representatives in Collective Decision-Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IRX5WUIE}},
  note         = {Machine review of arXiv:2502.09369}
}
read the original abstract

Consider the process of collective decision-making, in which a group of individuals interactively select a preferred outcome from among a universe of alternatives. In this context, "representation" is the activity of making an individual's preferences present in the process via participation by a proxy agent -- i.e. their "representative". To this end, learned models of human behavior have the potential to fill this role, with practical implications for multi-agent scenario studies and mechanism design. In this work, we investigate the possibility of training \textit{language agents} to behave in the capacity of representatives of human agents, appropriately expressing the preferences of those individuals whom they stand for. First, we formalize the setting of \textit{collective decision-making} -- as the episodic process of interaction between a group of agents and a decision mechanism. On this basis, we then formalize the problem of \textit{digital representation} -- as the simulation of an agent's behavior to yield equivalent outcomes from the mechanism. Finally, we conduct an empirical case study in the setting of \textit{consensus-finding} among diverse humans, and demonstrate the feasibility of fine-tuning large language models to act as digital representatives.

Figures

Figures reproduced from arXiv: 2502.09369 by the authors.

Figure 1
Figure 1. Consensus-Finding. Observe that this is an instance of a collective decision-making setting: • N is the set of participants, typically 3–5 for each episode; • Ω = X is the space of revised consensus statements; • x 1 ∈ X is the question of interest; • u 1 i ∈ U is the opinion of participant i ∈ N on the question; • x 2 ∈ X is the draft consensus statement taking those into account; • u 2 i ∈ U is the critique of par… view at source ↗
Figure 2
Figure 2. Critique Evaluation. Left: Mean log-likelihood of ground-truth critiques from human participants (from the validation set), evaluated under models πˆi with fine-tuned (“FT’d”) or vanilla digital representatives (“DRs”) with 1B or 30B parameters. The fine-tuned models consistently exhibit higher log-likelihoods compared to their vanilla counterparts, indicating a superior representation of participants’ ground-truth … view at source ↗
Figure 3
Figure 3. Consensus Evaluation. Either one participant’s critique (top), or all participants’ critiques (bottom) are substituted with critiques sampled from their respective digital representatives πˆi . Left: Mean discrepancy in payoffs between the revised consensus generated by the mediator mechanism using participants’ ground-truth critiques, versus using critiques sampled from DRs. Replacing either one or all ground truth… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Variants of Digital Representatives. Mean log-likelihood of ground-truth critiques from human participants (from the validation set), evaluated under various digital representatives. All these DRs have 1B parameters and were fine-tuned on datasets conditioned on divers…
Figure 5
Figure 5. Figure 5: Autorater Critique Evaluations: Ablations. The autorater’s win-rate of sampled critiques against a custom baseline, as specified by the grey bar within each plot (included as a sanity check, expected around 50%). The ground truth (One’s Own Critique) is also included t…
Figure 6
Figure 6. Figure 6: Autorater Consensus Evaluations: Ablations. The autorater’s win-rate of generated consensuses (based on single participant substitutions with DRs) against the baseline (based solely on ground truth critiques). For comparison, various baseline consensus statements (as s…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 42 canonical work pages

  1. [1]

    Collective decision-making

    Thomas Bose, Andreagiovanni Reina, and James AR Marshall. Collective decision-making. Current opinion in behavioral sciences, 16:30–34, 2017

    work page 2017

  2. [2]

    Collective decision making by rational individuals

    Richard P Mann. Collective decision making by rational individuals. Proceedings of the National Academy of Sciences, 115(44):E10387–E10396, 2018

    work page 2018

  3. [3]

    Game theory and mechanism design, volume 4

    Yadati Narahari. Game theory and mechanism design, volume 4. World Scientific, 2014

    work page 2014

  4. [4]

    When freedom is not an endless meeting: A new look at efficiency in consensus- based decision making

    Darcy K Leach. When freedom is not an endless meeting: A new look at efficiency in consensus- based decision making. The Sociological Quarterly, 57(1):36–70, 2016

    work page 2016

  5. [5]

    Fine-tuning language models to find agreement among humans with diverse preferences

    Michiel Bakker, Martin Chadwick, Hannah Sheahan, Michael Tessler, Lucy Campbell- Gillingham, Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matt Botvinick, et al. Fine-tuning language models to find agreement among humans with diverse preferences. Advances in Neural Information Processing Systems, 35:38176–38189, 2022

    work page 2022

  6. [6]

    Bakker, Daniel Jarrett, Hannah Sheahan, Martin J

    Michael Henry Tessler, Michiel A. Bakker, Daniel Jarrett, Hannah Sheahan, Martin J. Chadwick, Raphael Koster, Georgina Evans, Lucy Campbell-Gillingham, Tantum Collins, David C. Parkes, Matthew Botvinick, and Christopher Summerfield. AI can help humans find common ground in democratic deliberation. Science, 386(6719):eadq2852, 2024

    work page 2024

  7. [7]

    Political representation

    Suzanne Dovi. Political representation. The Stanford Encyclopedia of Philosophy, 2006

    work page 2006

  8. [8]

    Smooth imitation learning for online sequence prediction

    Hoang M Le, Andrew Kang, Yisong Yue, and Peter Carr. Smooth imitation learning for online sequence prediction. International Conference on Machine Learning (ICML), 2016

    work page 2016

Show all 61 references
  1. [9]

    Imitation learning (presentation)

    Yisong Yue and Hoang M Le. Imitation learning (presentation). International Conference on Machine Learning (ICML), 2018

    2018

  2. [10]

    An algorithmic perspective on imitation learning

    Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J Andrew Bagnell, Pieter Abbeel, and Jan Peters. An algorithmic perspective on imitation learning. Foundations and Trends in Robotics, 2018

    2018

  3. [11]

    Explaining by imitating: Understanding decisions by interpretable policy learning

    Alihan Hüyük, Daniel Jarrett, Cem Tekin, and Mihaela van der Schaar. Explaining by imitating: Understanding decisions by interpretable policy learning. International Conference on Learning Representations (ICLR), 2021

    2021

  4. [12]

    Generating faithful synthetic data with large language models: A case study in computational social science

    Veniamin Veselovsky, Manoel Horta Ribeiro, Akhil Arora, Martin Josifoski, Ashton Anderson, and Robert West. Generating faithful synthetic data with large language models: A case study in computational social science. arXiv preprint arXiv:2305.15041, 2023

    2023 arXiv

  5. [13]

    Does synthetic data generation of llms help clinical text mining? arXiv preprint arXiv:2303.04360, 2023

    Ruixiang Tang, Xiaotian Han, Xiaoqian Jiang, and Xia Hu. Does synthetic data generation of llms help clinical text mining? arXiv preprint arXiv:2303.04360, 2023

    2023 arXiv

  6. [14]

    The medkit-learn (ing) environment: Medical decision modelling through simulation

    Alex J Chan, Ioana Bica, Alihan Huyuk, Daniel Jarrett, and Mihaela van der Schaar. The medkit-learn (ing) environment: Medical decision modelling through simulation. arXiv preprint arXiv:2106.04240, 2021

    2021 arXiv

  7. [15]

    Carla: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pages 1–16. PMLR, 2017

    2017

  8. [16]

    Social simulacra: Creating populated prototypes for social computing systems

    Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Social simulacra: Creating populated prototypes for social computing systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, ...

    2022

  9. [17]

    Using large language models to simulate multiple humans and replicate human subject studies

    Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337–371. PMLR, 2023

    2023

  10. [18]

    Ai language models cannot replace human research participants

    Jacqueline Harding, William D’Alessandro, NG Laskowski, and Robert Long. Ai language models cannot replace human research participants. AI & SOCIETY, pages 1–3, 2023

    2023

  11. [19]

    Out of one, many: Using language models to simulate human samples

    Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351, 2023

    2023

  12. [20]

    Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023

    John J Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023. 10

    2023

  13. [21]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023

    2023 arXiv

  14. [22]

    Training compute-optimal large language models

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  15. [23]

    The effectiveness of data augmentation in image classification using deep learning

    Jason Wang, Luis Perez, et al. The effectiveness of data augmentation in image classification using deep learning. Convolutional Neural Networks Vis. Recognit, 11(2017):1–8, 2017

    2017

  16. [24]

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664, 2023

    2023 arXiv

  17. [25]

    Value-aware loss function for model-based reinforcement learning

    Amir-massoud Farahmand, Andre Barreto, and Daniel Nikovski. Value-aware loss function for model-based reinforcement learning. In Artificial Intelligence and Statistics, pages 1486–1494. PMLR, 2017

    2017

  18. [26]

    Iterative value-aware model learning

    Amir-massoud Farahmand. Iterative value-aware model learning. Advances in Neural Informa- tion Processing Systems, 31, 2018

    2018

  19. [27]

    The value equivalence principle for model-based reinforcement learning

    Christopher Grimm, André Barreto, Satinder Singh, and David Silver. The value equivalence principle for model-based reinforcement learning. Advances in Neural Information Processing Systems, 33:5541–5552, 2020

    2020

  20. [28]

    Proper value equivalence

    Christopher Grimm, André Barreto, Greg Farquhar, David Silver, and Satinder Singh. Proper value equivalence. Advances in Neural Information Processing Systems, 34:7773–7786, 2021

    2021

  21. [29]

    Approximate value equivalence

    Christopher Grimm, Andre Barreto, and Satinder Singh. Approximate value equivalence. Advances in Neural Information Processing Systems, 35:33029–33040, 2022

    2022

  22. [30]

    Deciding what to model: Value-equivalent sampling for reinforcement learning

    Dilip Arumugam and Benjamin Van Roy. Deciding what to model: Value-equivalent sampling for reinforcement learning. Advances in Neural Information Processing Systems, 35:9024–9044, 2022

    2022

  23. [31]

    Whose opinions do language models reflect? arXiv preprint arXiv:2303.17548, 2023

    Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? arXiv preprint arXiv:2303.17548, 2023

    2023 arXiv

  24. [32]

    Moral mimicry: Large language models produce moral rationalizations tailored to political identity

    Gabriel Simmons. Moral mimicry: Large language models produce moral rationalizations tailored to political identity. arXiv preprint arXiv:2209.12106, 2022

    2022 arXiv

  25. [33]

    How (not) to use sociode- mographic information for subjective nlp tasks

    Tilman Beck, Hendrik Schuff, Anne Lauscher, and Iryna Gurevych. How (not) to use sociode- mographic information for subjective nlp tasks. arXiv preprint arXiv:2309.07034, 2023

    2023 arXiv

  26. [34]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2019

    2019 arXiv

  27. [35]

    Palm 2 technical report

    Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023

    2023 arXiv

  28. [36]

    A reduction from apprenticeship learning to classification

    Umar Syed and Robert E Schapire. A reduction from apprenticeship learning to classification. Advances in neural information processing systems (NeurIPS), 2010

    2010

  29. [37]

    Apprenticeship learning via inverse reinforcement learning

    Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement learning. International conference on Machine learning (ICML), 2004

    2004

  30. [38]

    Apprenticeship learning using irl and gradient methods

    Gergely Neu and Csaba Szepesvári. Apprenticeship learning using irl and gradient methods. Conference on Uncertainty in Artificial Intelligence (UAI), 2007

    2007

  31. [39]

    Apprenticeship learning about multiple intentions

    Monica Babes, Vukosi Marivate, and Michael L Littman. Apprenticeship learning about multiple intentions. International conference on Machine learning (ICML), 2011

    2011

  32. [40]

    Maximum entropy inverse reinforcement learning

    Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. AAAI Conference on Artificial Intelligence (AAAI), 2008

    2008

  33. [41]

    Guided cost learning: Deep inverse optimal control via policy optimization

    Chelsea Finn, Sergey Levine, and Pieter Abbeel. Guided cost learning: Deep inverse optimal control via policy optimization. International conference on machine learning (ICML), 2016. 11

    2016

  34. [42]

    Generative adversarial imitation learning

    Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems (NeurIPS), 2016

    2016

  35. [43]

    Inverse decision modeling: Learning interpretable representations of behavior

    Daniel Jarrett, Alihan Hüyük, and Mihaela Van Der Schaar. Inverse decision modeling: Learning interpretable representations of behavior. In International Conference on Machine Learning, pages 4755–4771. PMLR, 2021

    2021

  36. [44]

    Generative multi-agent behavioral cloning

    Eric Zhan, Stephan Zheng, Yisong Yue, Long Sha, and Patrick Lucey. Generative multi-agent behavioral cloning. arXiv, 2018

    2018

  37. [45]

    Exploring the limitations of behavior cloning for autonomous driving

    Felipe Codevilla, Eder Santana, Antonio M López, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9329–9338, 2019

    2019

  38. [46]

    Multi-agent generative adver- sarial imitation learning

    Jiaming Song, Hongyu Ren, Dorsa Sadigh, and Stefano Ermon. Multi-agent generative adver- sarial imitation learning. Advances in neural information processing systems, 31, 2018

    2018

  39. [47]

    Conditional imitation learning for multi-agent games

    Andy Shih, Stefano Ermon, and Dorsa Sadigh. Conditional imitation learning for multi-agent games. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 166–175. IEEE, 2022

    2022

  40. [48]

    Multi-agent imitation learning for driving simulation

    Raunak P Bhattacharyya, Derek J Phillips, Blake Wulfe, Jeremy Morton, Alex Kuefler, and Mykel J Kochenderfer. Multi-agent imitation learning for driving simulation. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1534–1539. IEEE, 2018

    2018

  41. [49]

    Generating high-fidelity, synthetic time series datasets with doppelganger.ACM Internet Measurement Conference (IMC), 2019

    Zinan Lin, Alankar Jain, Chen Wang, Giulia Fanti, and Vyas Sekar. Generating high-fidelity, synthetic time series datasets with doppelganger.ACM Internet Measurement Conference (IMC), 2019

    2019

  42. [50]

    Cot-gan: Generating sequential data via causal optimal transport

    Tianlin Xu, Li K Wenliang, Michael Munn, and Beatrice Acciaio. Cot-gan: Generating sequential data via causal optimal transport. Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  43. [51]

    Time-series generation by contrastive imitation

    Daniel Jarrett, Ioana Bica, and Mihaela van der Schaar. Time-series generation by contrastive imitation. Advances in Neural Information Processing Systems, 34:28968–28982, 2021

    2021

  44. [52]

    Real-valued (medical) time series generation with recurrent conditional gans

    Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633, 2017

    2017 arXiv

  45. [53]

    Harnessing large-language models to generate private synthetic text

    Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis. Harnessing large-language models to generate private synthetic text. arXiv preprint arXiv:2306.01684, 2023

    2023 arXiv

  46. [54]

    Towards measuring the representation of subjective global opinions in language models

    Esin Durmus, Karina Nyugen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306...

    2023 arXiv

  47. [55]

    The political ideology of conver- sational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation

    Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. The political ideology of conver- sational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation. arXiv preprint arXiv:2301.01768, 2023

    2023 arXiv

  48. [56]

    Communitylm: Probing partisan worldviews from language models

    Hang Jiang, Doug Beeferman, Brandon Roy, and Deb Roy. Communitylm: Probing partisan worldviews from language models. arXiv preprint arXiv:2209.07065, 2022

    2022 arXiv

  49. [57]

    Opiniongpt: Modelling explicit biases in instruction-tuned llms

    Patrick Haller, Ansar Aynetdinov, and Alan Akbik. Opiniongpt: Modelling explicit biases in instruction-tuned llms. arXiv preprint arXiv:2309.03876, 2023

    2023 arXiv

  50. [58]

    Toward a deliberative global citizens’ assembly

    John S Dryzek, André Bächtiger, and Karolina Milewicz. Toward a deliberative global citizens’ assembly. Global Policy, 2(1):33–42, 2011

    2011

  51. [59]

    The assignment of political office by lot

    Fredrik Engelstad. The assignment of political office by lot. Social Science Information , 28(1):23–50, 1989

    1989

  52. [60]

    The mild voice of reason: Deliberative democracy and American national government

    Joseph M Bessette. The mild voice of reason: Deliberative democracy and American national government. University of Chicago Press, 1994

    1994

  53. [61]

    A proposal for importing society’s values

    Jan Leike. A proposal for importing society’s values. Musings on the Alignment Problem, 2023. 12 A Proof of Proposition 1 Proposition 1 (Representational Equivalence) Fix Π and T , and let Q be closed under Bellman updates. Consider the equivalence classes of policy profiles i...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.