Pith. sign in

REVIEW 3 major objections 5 minor 40 references

ALIGN: Prompt-based Attribute Alignment for Reliable, Responsible, and Personalized LLM-based Decision-Making

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ALIGN is an open-source framework that uses prompt-based alignment to make LLM decision-makers follow fine-grained user attributes, and shows this steering improves measured alignment accuracy across backbones and domains.

desk verdict A genuinely useful open-source framework for comparing aligned decision-makers, but the empirical headline is not yet statistically supported. read the letter →

arxiv 2507.09037 v1 pith:J3KYKJGF submitted 2025-07-11 cs.CL cs.AI

Bharadwaj Ravichandran , David Joy , Paul Elliott , Brian Hu , Jadie Adams , Christopher Funk , Emily Veenhuis , Anthony Hoogs
show 1 more author
Arslan Basharat
This is my paper
classification cs.CLcs.AI
keywords LLMalignmentprompt-basedpersonalizationattributepluralisticvaluesmedicaltriagepublicopinionsurveysstructuredoutputgenerationmodularevaluationframework
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that LLM-based decision aids should be judged not only on factual accuracy but on whether their choices can be personalized to a user's values, and it offers ALIGN as a domain-agnostic testbed for that kind of alignment. Its central claim is that a zero-shot, prompt-aligned decision-maker—an LLM given a system prompt that names an attribute target such as 'high moral desert' or 'college graduate'—achieves higher mean alignment accuracy than an unaligned baseline across four LLM backbones. That claim is demonstrated in two settings that lack a single correct answer: demographic alignment on public opinion surveys and value alignment in medical triage. The wider point is that alignment can be treated as a modular, inspectable property of a decision-making system rather than a fixed model characteristic.

What carries the argument

The load-bearing mechanism is the aligned system prompt: a zero-shot instruction that embeds a target attribute into the decision-making context and asks the model to reason before choosing. Around that prompt, ALIGN layers three pieces of machinery: a dataset interface that turns each scenario into a structured question with choices; structured output generation, via the Outlines library, that constrains the model to emit a JSON answer plus an explicit reasoning trace; and a configuration manager, built on Hydra, that makes backbones, attributes, prompt templates, and decoding settings swappable. The Kaleido ADM is the second alignment mechanism, probing a pluralistic value model for attribute relevance and valence per choice. Together these let the same scenario be run under different alignment conditions and compared side by side in a Trame-based interface.

What would settle it

Run a new evaluation where the same ALIGN pipeline is applied to a held-out set of survey questions and triage scenarios with independently collected attribute labels, while paraphrasing each system prompt; if the prompt-aligned decision-maker fails to beat the unaligned baseline on mean alignment accuracy under those conditions, the central claim that prompt-based attribute alignment reliably steers LLM decision-makers would be falsified.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that fine-grained attribute alignment can be implemented and measured entirely at the prompt level, without fine-tuning, and that this prompt-level steering changes decisions in the direction of the target attribute. ALIGN packages this discovery as a comparison framework: the same dataset interface and configuration system hold the baseline, prompt-aligned, and Kaleido decision-makers fixed while only the system prompt and alignment target change. The quantitative backbone of the claim is the alignment-accuracy metric, which scores whether the selected choice is the one closest to the target attribute; the prompt-aligned ADM beats the baseline on mean accuracy for both OpinionQA demographics and MTA value attributes across all four tested backbones, and the Kaleido models achieve the highest MTA scores.

Load-bearing premise

The load-bearing premise is that the alignment-accuracy score derived from the dataset labels measures real alignment to user preferences; if the attribute definitions, answer labels, or the particular OpinionQA subset do not track actual user values, the reported accuracy differences would show prompt compliance rather than true personalization.

Editorial extensions

If this is right

  • If prompt-based alignment works as reported, users can personalize an LLM decision-maker simply by editing the system prompt, with no retraining or fine-tuning, which makes personalization cheap and fast.
  • ALIGN's modular design means other researchers can drop in new alignment algorithms, datasets, or LLM backbones and directly compare alignment accuracy on identical scenarios, effectively standardizing how alignment methods are evaluated.
  • Because outputs are constrained to a structured schema with an explicit reasoning trace, the framework makes it possible to see not only which choice an aligned model made but why, allowing failures to be traced to specific scenarios.
  • The reported per-attribute results imply that alignment gains are not uniform: some attributes and backbones improve more than others, and a few attributes actually drop relative to baseline, suggesting single-attribute steering is only a partial solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is that combining multiple attributes in one prompt, such as fairness plus risk aversion, may yield better personalization than single-attribute steering, since the paper's own results show single attributes can underperform the baseline on some items.
  • The reported accuracy metric tests whether the model picks the label-endorsed choice, not whether a real user feels represented; a direct follow-up would be to run ALIGN-generated prompts with human raters and compare satisfaction against the metric.
  • Because the OpinionQA portion uses an unspecified subset of the dataset, the quantitative results are difficult to reproduce externally; releasing the exact question subset and templates would make the framework's claims independently checkable.
  • The framework's ability to swap backbones exposes a testable question: does prompt-based alignment transfer across model generations, and do smaller models need stronger attribute phrasing to reach the same alignment accuracy?
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents ALIGN, an open-source framework for comparing LLM-based decision-makers that are aligned to fine-grained attributes via prompting. The system combines a Trame-based interactive UI, Hydra-based configuration management, structured output generation with reasoning via Outlines, and pluggable ADM implementations: an unaligned baseline, a prompt-aligned ADM, and two Kaleido variants. The authors demonstrate the framework on two domains, demographic alignment using a subset of OpinionQA and value alignment using the MTA medical triage dataset, across four LLM backbones, and report per-attribute and mean alignment accuracy. The central quantitative claim is that a zero-shot prompt-aligned ADM achieves higher average alignment accuracy than the unaligned baseline, with Kaleido showing the best overall MTA performance.

Significance. If the empirical claims held, ALIGN would be a useful community resource: it lowers the barrier to comparing alignment methods, exposes prompts and reasoning traces for qualitative inspection, and supports swapping backbones and attributes in a reproducible way. The paper's concrete strengths are its open-source implementation, modular backend, configuration tracking, deterministic decoding with fixed seeds, and an interactive UI for side-by-side comparison. The quantitative evaluation, however, is not yet statistically grounded, and the demographic results rely on an unspecified subset of OpinionQA. The contribution is therefore currently a promising system demonstration rather than a validated claim about alignment accuracy; the framework's value is more clearly established than the empirical headline.

major comments (3)
  1. [Section 6, Tables 1 and 2] The load-bearing claim that the prompt-aligned ADM achieves 'higher mean alignment accuracy' compared to the baseline is not statistically supported. No sample sizes, confidence intervals, or significance tests are reported for any cell, so the reader cannot tell whether the differences reflect real alignment or small-sample noise. For example, the OpinionQA Llama-3.3-70B means differ by only 0.2 points (58.6 vs 58.8), and the Inc100k attribute drops from 70.0 to 56.4; on MTA, the unaligned values are exactly 50.0 in 11 of 12 cells, which is consistent with chance on a small two-choice task. Please report the number of items per attribute, confidence intervals, and appropriate tests, or explicitly reframe the quantitative results as an illustrative demonstration rather than a validated comparison.
  2. [Section 4.1 and Appendix A.1] The OpinionQA subset is unspecified. The paper states that it uses 'a subset of the data converted into a steerable benchmark' from Feng et al. (2024), but it does not report how many questions were used, which questions were selected, or how the subset was constructed. Because all OpinionQA results in Table 1 depend on this subset, the results cannot be reproduced or compared with other work, and they may not generalize to the full dataset. Please specify the subset construction, provide the question counts, or use the full steerable benchmark.
  3. [Section 6 and Table 2] The Kaleido comparison is incomplete as a benchmark. The table does not report unaligned metrics for the Kaleido models, and the caption states that these metrics are not computed because Kaleido requires alignment attributes. As a result, the claim that Kaleido models 'demonstrate the best overall performance' is not supported by a comparison against a common baseline, and the reader cannot judge whether the Kaleido advantage over the prompt-aligned ADM is meaningful. Please either compute a comparable baseline for Kaleido or soften the claim to describe absolute accuracy only.
minor comments (5)
  1. [Abstract and Section 1] The sentence 'The entire ALIGN framework is open source' appears nearly verbatim in both the abstract and the introduction; consider stating it once and referring to the GitHub repositories in the other place.
  2. [Section 3.1] There is a missing space in the full text after 'system.' in the sentence ending '...resulting in a more interpretable system.' followed immediately by 'Outlines also provides...'; please fix the typographical error.
  3. [Section 4 and Tables 1-2] The tables would be easier to interpret if the captions stated the number of scenarios per attribute and the exact scoring rule (e.g., binary match versus partial credit), since the reported percentages otherwise float without a denominator.
  4. [Appendix A.2] In the definition of Fairness, the text reads 'Fairness (F)is treating...' with a missing space before 'is'; please correct this typo.
  5. [Full manuscript] The evaluation partly uses the authors' own prompt-aligned ADM from Hu et al. (2024) and the MTA dataset from the same group, with overlapping authors. This is not a circularity problem because the alignment metric comes from prior benchmarks, but the self-evaluation overlap should be stated explicitly for transparency in the experimental setup.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ALIGN's claims rest on new empirical runs against externally defined benchmarks; self-citations are transparent but not load-bearing.

full rationale

ALIGN does not derive its headline 'higher average alignment accuracy' claim from its own definitions. Section 6 says 'we use an accuracy metric proposed in recent benchmarks (Hu et al., 2024; Feng et al., 2024)', so the metric is adopted, not constructed in this paper, and the per-attribute accuracies in Tables 1-2 are produced by the authors' runs of baseline, prompt-aligned, and Kaleido decision-makers. No parameter is fitted to the reported outcomes, and no equation in the paper makes a prediction identical to an input by construction. The main self-citation cluster is the Prompt-Aligned ADM ('Similar to our previous ADM implementation (Hu et al., 2024)') and the MTA dataset (Hu et al., 2024), which shares co-authors; this reduces the independence of the MTA demonstration, but the dataset and metric are published, externally falsifiable resources, and the OpinionQA portion uses the external Feng et al. (2024) formatting. The Limitations section explicitly disclaims that results were 'not concretely linked to any particular task or outcome', a scope caveat rather than a circular step. Accordingly, no circular step can be exhibited; the paper's contribution is framework engineering plus empirical comparison, not a derivation that collapses into its inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

ALIGN is a software framework, not a theory paper. It introduces no fitted constants, no new mathematical axioms, and no new physical or conceptual entities. The assumptions listed above are domain and benchmarking assumptions imported from prior work, and they are the main hidden load-bearing inputs for the reported results.

assumptions (4)
  • domain assumption The alignment-accuracy metric from Hu et al. and Feng et al. correctly measures alignment to the target attribute.
    Invoked in Section 6 as 'we use an accuracy metric proposed in recent benchmarks.' If the metric is flawed, all reported accuracies lose meaning.
  • domain assumption The selected attributes and dataset subsets faithfully represent user values in the two domains.
    Section 4 selects six demographic attributes from OpinionQA and six triage attributes from MTA; the Limitations section acknowledges it is unclear whether these attributes generalize across domains or users.
  • domain assumption Structured output via Outlines constrains the LLM without altering decision quality.
    Section 3.1 relies on Outlines to constrain and parse LLM output, but no validation is provided that constrained decoding does not distort alignment decisions.
  • domain assumption The unaligned baseline prompt is a meaningful reference point for measuring alignment gains.
    Section 6 compares aligned ADMs against a simple unaligned prompt; if this baseline is artificially weak, the reported improvements are inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ALIGN: Prompt-based Attribute Alignment for Reliable, Responsible, and Personalized LLM-based Decision-Making." pith.science (2026). https://pith.science/paper/J3KYKJGF

@misc{pith2026250709037,
  author       = {Pith},
  title        = {Pith review of: ALIGN: Prompt-based Attribute Alignment for Reliable, Responsible, and Personalized LLM-based Decision-Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J3KYKJGF}},
  note         = {Machine review of arXiv:2507.09037}
}
read the original abstract

Large language models (LLMs) are increasingly being used as decision aids. However, users have diverse values and preferences that can affect their decision-making, which requires novel methods for LLM alignment and personalization. Existing LLM comparison tools largely focus on benchmarking tasks, such as knowledge-based question answering. In contrast, our proposed ALIGN system focuses on dynamic personalization of LLM-based decision-makers through prompt-based alignment to a set of fine-grained attributes. Key features of our system include robust configuration management, structured output generation with reasoning, and several algorithm implementations with swappable LLM backbones, enabling different types of analyses. Our user interface enables a qualitative, side-by-side comparison of LLMs and their alignment to various attributes, with a modular backend for easy algorithm integration. Additionally, we perform a quantitative analysis comparing alignment approaches in two different domains: demographic alignment for public opinion surveys and value alignment for medical triage decision-making. The entire ALIGN framework is open source and will enable new research on reliable, responsible, and personalized LLM-based decision-makers.

Figures

Figures reproduced from arXiv: 2507.09037 by the authors.

Figure 1
Figure 1. ALIGN system overview. Across different application domains, ALIGN enables reliable and responsible personalization of LLM-based decision-makers via alignment to a set of fine￾grained attributes. lenging task. This alignment is essential for human trust in AI algorithms, as it enables humans to guide algorithms toward their desired outcomes (Scherrer et al., 2023). One potential solution is to dynamically align algo… view at source ↗
Figure 2
Figure 2. ALIGN system architecture. ALIGN provides an in￾teractive user interface for comparing aligned model outputs, sup￾ported by a modular backend containing various alignment ap￾proaches. User-customizable configurations manage LLM model parameters, prompt templates, and structured generation outputs. text description of the situation, patient descriptions along with vital signs and injuries, as well as available treatm… view at source ↗
Figure 3
Figure 3. Screenshot of the Trame-based (Jourdain et al., 2025) ALIGN system demo app showing an example comparison workflow for a (A) Baseline ADM vs (B) Aligned ADM with alignment to the high moral desert attribute from the Medical Triage Alignment (MTA) dataset. Different components of the user interface are annotated for emphasis (best viewed electronically with zoom). 4.1. OpinionQA Demographic Alignment The OpinionQA da… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Per-attribute alignment accuracy for the baseline vs prompt-aligned ADMs across the six high-low medical triage decision-making attributes (Hu et al., 2024). The radar plots for the full benchmarking suite are provided under Appendix C. 5 [PITH_FULL_IMAGE:figures/full…
Figure 5
Figure 5. Figure 5: Radar plots showing the per-attribute high-low alignment accuracy for both the Medical Triage Alignment (Hu et al., 2024) and OpinionQA (Santurkar et al., 2023; Feng et al., 2024) datasets. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Alicke, M. D. Culpable control and the psychology of blame. Psychological Bulletin, 126 0 (4): 0 556–574, 2000. ISSN 0033-2909. doi:10.1037/0033-2909.126.4.556. URL http://dx.doi.org/10.1037/0033-2909.126.4.556

    doi:10.1037/0033-2909.126.4.556 2000

  3. [3]

    Arawjo, I., Swoopes, C., Vaithilingam, P., Wattenberg, M., and Glassman, E. L. Chainforge: A visual toolkit for prompt engineering and llm hypothesis testing. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp.\ 1--18, 2024

    work page 2024

  4. [4]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  5. [5]

    Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

    2018

  6. [6]

    Training verifiers to solve math word problems, 2021

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems, 2021

    2021

  7. [7]

    A survey for in-context learning

    Dong, Q., Li, L., Dai, D., Zheng, C., Wu, Z., Chang, B., Sun, X., Xu, J., and Sui, Z. A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022

    arXiv 2022

  8. [8]

    I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., et al

    Durmus, E., Nyugen, K., Liao, T. I., Schiefer, N., Askell, A., Bakhtin, A., Chen, C., Hatfield-Dodds, Z., Hernandez, D., Joseph, N., et al. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388, 2023

    arXiv 2023

Show all 40 references
  1. [9]

    E., Baron, J., and Seligman, M

    Eisenberg, A. E., Baron, J., and Seligman, M. E. Individual differences in risk aversion and anxiety. Psychological Bulletin, 87 0 (1): 0 245--251, 1998

    1998

  2. [10]

    and Schmidt, K

    Fehr, E. and Schmidt, K. M. A theory of fairness, competition, and cooperation. The quarterly journal of economics, 114 0 (3): 0 817--868, 1999

    1999

  3. [11]

    Y., Choi, Y., and Tsvetkov, Y

    Feng, S., Sorensen, T., Liu, Y., Fisher, J., Park, C. Y., Choi, Y., and Tsvetkov, Y. Modular pluralism: Pluralistic alignment via multi-llm collaboration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 4151--4171, 2024

    2024

  4. [12]

    From Principles to Practice

    Fetic, L., Fleischer, T., Gr\" u nke, P., Hagendorf, T., Hallensleben, S., Hauer, M., Herrmann, M., Hillerbrand, R., Hustedt, C., Hubig, C., Kaminski, A., Krafft, T., Loh, W., Otto, P., and Puntschuh, M. From Principles to Practice. An Interdisciplinary Framework to Operationa...

    2020

  5. [13]

    A., Haidt, J., Iyer, R., Koleva, S., and Ditto, P

    Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., and Ditto, P. H. Mapping the moral domain. Journal of Personality and Social Psychology, 101 0 (2): 0 366–385, 2011. ISSN 0022-3514. doi:10.1037/a0021847. URL http://dx.doi.org/10.1037/a0021847

    2011 doi

  6. [14]

    Greene, J. D. Beyond point-and-shoot morality: Why cognitive (neuro) science matters for ethics. Ethics, 124 0 (4): 0 695--726, 2014

    2014

  7. [15]

    Measuring massive multitask language understanding, 2021

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding, 2021

    2021

  8. [16]

    and Ones, D

    Hogan, J. and Ones, D. S. Chapter 32 - conscientiousness and integrity at work. In Hogan, R., Johnson, J., and Briggs, S. (eds.), Handbook of Personality Psychology, pp.\ 849--870. Academic Press, San Diego, 1997. ISBN 978-0-12-134645-4. doi:https://doi.org/10.1016/B978-012134...

    1997 doi

  9. [17]

    Language models are alignable decision-makers: Dataset and application to the medical triage domain

    Hu, B., Ray, B., Leung, A., Summerville, A., Joy, D., Funk, C., and Basharat, A. Language models are alignable decision-makers: Dataset and application to the medical triage domain. arXiv preprint arXiv:2406.06435, 2024

    2024 arXiv

  10. [18]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  11. [19]

    Can language models reason about individualistic human values and preferences? In Pluralistic Alignment Workshop at NeurIPS, 2024

    Jiang, L., Levine, S., and Choi, Y. Can language models reason about individualistic human values and preferences? In Pluralistic Alignment Workshop at NeurIPS, 2024

    2024

  12. [20]

    K., Hanna, M

    Johnson, M. K., Hanna, M. M., Clemens-Sewall, M. V., and Staheli, D. P. Responsible AI toolkit ( RAI toolkit 1.0). (January 2024), 2023. URL https://rai.tradewindai.com. [online]

    2024

  13. [21]

    Trame: Platform ubiquitous, scalable integration framework for visual analytics

    Jourdain, S., O'Leary, P., and Schroeder, W. Trame: Platform ubiquitous, scalable integration framework for visual analytics. IEEE Computer Graphics and Applications, March 2025. doi:10.1109/MCG.2025.3540264

    2025

  14. [22]

    A., Earp, B

    Kahane, G., Everett, J. A., Earp, B. D., Caviola, L., Faber, N. S., Crockett, M. J., and Savulescu, J. Beyond sacrificial harm: A two-dimensional model of utilitarian psychology. Psychological review, 125 0 (2): 0 131, 2018

    2018

  15. [23]

    X., Wexler, J., Reif, E., Kallarackal, K., Chang, M., Terry, M., and Dixon, L

    Kahng, M., Tenney, I., Pushkarna, M., Liu, M. X., Wexler, J., Reif, E., Kallarackal, K., Chang, M., Terry, M., and Dixon, L. Llm comparator: Visual analytics for side-by-side evaluation of large language models. In Extended Abstracts of the CHI Conference on Human Factors in C...

    2024

  16. [24]

    S., Lee, Y., Shin, J., Kim, Y.-H., and Kim, J

    Kim, T. S., Lee, Y., Shin, J., Kim, Y.-H., and Kim, J. Evallm: Interactive evaluation of large language model prompts on user-defined criteria. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp.\ 1--21, 2024

    2024

  17. [25]

    Truthfulqa: Measuring how models mimic human falsehoods, 2022

    Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods, 2022

    2022

  18. [26]

    and Lalumi \`e re, M

    Mishra, S. and Lalumi \`e re, M. L. Individual differences in risk-propensity: Associations between personality and behavioral measures of risk. Personality and Individual Differences, 50 0 (6): 0 869--873, 2011

    2011

  19. [27]

    Virtual personas for language models via an anthology of backstories

    Moon, S., Abdulhai, M., Kang, M., Suh, J., Soedarmadji, W., Behar, E., and Chan, D. Virtual personas for language models via an anthology of backstories. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 19864--19897, 2024

    2024

  20. [28]

    T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., Liu, W., et al

    Nori, H., Lee, Y. T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., Liu, W., et al. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. arXiv preprint arXiv:2311.16452, 2023

    2023 arXiv

  21. [29]

    L., Bhagavatula, C., and Choi, Y

    Sakaguchi, K., Bras, R. L., Bhagavatula, C., and Choi, Y. WINOGRANDE: an adversarial winograd schema challenge at scale, 2019

    2019

  22. [30]

    Whose opinions do language models reflect? International Conference on Machine Learning (ICML), 2023

    Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T. Whose opinions do language models reflect? International Conference on Machine Learning (ICML), 2023

    2023

  23. [31]

    Evaluating the moral beliefs encoded in llms

    Scherrer, N., Shi, C., Feder, A., and Blei, D. Evaluating the moral beliefs encoded in llms. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  24. [32]

    S., Wei, J., Chung, H

    Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. Large language models encode clinical knowledge. Nature, 620 0 (7972): 0 172--180, 2023

    2023

  25. [33]

    D., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., et al

    Sorensen, T., Jiang, L., Hwang, J. D., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., et al. Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties. In Proceedings of the AAAI Conference on Artificial Intelligence, vo...

    2024

  26. [34]

    M., Ye, A., Jiang, L., Lu, X., Dziri, N., et al

    Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., et al. A roadmap to pluralistic alignment. arXiv preprint arXiv:2402.05070, 2024 b

    2024 arXiv

  27. [35]

    Webster, D. M. and Kruglanski, A. W. Individual differences in need for cognitive closure. Journal of personality and social psychology, 67 0 (6): 0 1049, 1994

    1994

  28. [36]

    Webster, D. M. and Kruglanski, A. W. Cognitive and social consequences of the need for cognitive closure. European review of social psychology, 8 0 (1): 0 133--173, 1997

    1997

  29. [37]

    V., Zhou, D., et al

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35: 0 24824--24837, 2022

    2022

  30. [38]

    Willard, B. T. and Louf, R. Efficient guided generation for llms. arXiv preprint arXiv:2307.09702, 2023

    2023 arXiv

  31. [39]

    Hydra - a framework for elegantly configuring complex applications

    Yadan, O. Hydra - a framework for elegantly configuring complex applications. Github, 2019. URL https://github.com/facebookresearch/hydra

    2019

  32. [40]

    Hellaswag: Can a machine really finish your sentence?, 2019

    Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. Hellaswag: Can a machine really finish your sentence?, 2019

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.