Pith.

Human-Computer Interaction · arXiv:2611.15732 v1 · submitted 11 Nov 2026 · 74 references

Right Advice, Wrong Reasons: How Decision and Reasoning Alignment Shape Reliance on AI Advisors

Eunkyu Park, Vasudha Varadarajan, Wesley Hanwen Deng, Anisha Reddy, Gunhee Kim, Motahhare Eslami, Jana Schaich Borg, Maarten Sap

The paper says

“When people seek advice from AI on social situations, the system may recommend a decision they fully agree with but justify it through reasoning they disagree with.”

more of the abstract

Conversely, it may offer reasoning they agree with, and yet reach a decision they reject. We distinguish decision alignment, whether an advisor recommends the option favored by a user's measured values, from reason alignment, whether its stated reasons align with the user's values. In this work, we ask how these two forms of alignment shape people's preferences for AI advisors and their willingness to use them in different roles. We conducted a within-subjects study (N=121) in which participants evaluated three AI advisors varying in decision and reason alignment across eight everyday moral dilemmas. Participants then chose which advisors they would consult, allow to draft messages for review, or allow to act on their behalf in new tasks. When seeking advice, participants most often preferred an advisor that reasoned through their prioritized values while recommending the opposing decision (40%). However, when participants had to pick advisors for new tasks, their preferences shifted toward advisors that also recommended decisions aligned with their values, rather than those that only shared their reasons. Also, as participants considered tasks that gave AI advisors greater autonomy, they became more likely to opt out of using any advisor, from 12% when consulting to 38% when allowing an advisor to act on their behalf. Our findings suggest that decision and reason alignment shape advisor preferences differently across contexts of AI use, and that alignment with a user's values does not necessarily imply willingness to delegate.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: holds up only with conditions. Referee: major revision.

Solid user study on decision vs. reason alignment in AI advisors, but the central result depends on an unvalidated mapping from value scores to value-favored decisions. This line is wrong

3 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 74 references say.

The paper says, at Section 3.1.3 and Appendix A.5

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.86):

“Our measure therefore captures alignment with participants’ measured value profiles rather than necessarily matching the decisions they would make.”

Pith says: Objection 1 of 3 · major

The central alignment conditions are constructed from the assumption that the higher-scoring PVQ-21 value within each dilemma's pair identifies the participant's value-favored decision, but this mapping is not validated against participants' actual choices on the dilemmas. The authors acknowledge this in Section 7 ('our measure therefore captures alignment with participants' measured value profiles rather than necessarily matching the decisions they …

more of objection 1

… would make'), yet the Part 2 preference for ReaAligned over FullyAligned is exactly the comparison that would be inflated if the mapping mislabels a substantial share of participants' favored decisions. I request a robustness analysis, for example excluding or re-analyzing participants whose value strength is low, or using the Appendix A.5 validation data to estimate how often the value-favored option coincides with annotators' actual choices. Without such an analysis, the central Part 2 effect is defensible but rests on an untested mapping, and the concern is load-bearing because the ReaAligned, DecAligned, and FullyAligned labels are defined through that mapping.

The paper says, at Section 4.2 and Table 1

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The claim that preferences 'shift toward FullyAligned' in Part 3 relies on comparing conditional selection proportions while the choice set changes: Part 2 forces a choice among the three advisors, whereas Part 3 includes a 'None of them' option. The authors partially address this by conditioning on advisor selection and reporting opt-out separately, but the within-participant change analysis is sensitive …

more of objection 2

… to which participants remain in the conditional set, and the Consult, Draft, and Act rows have n=114, 105, and 93 respectively. I request either a sensitivity analysis that includes participants who opt out in Part 3 with an explicit adjustment, or a clear statement that the 12.9-point shift is estimated only among participants who selected at least one advisor in both parts. This matters because the RQ2 shift is the paper's second central result and the n-reduction across rows raises the possibility that selective opt-out drives part of the apparent preference change.

The paper says, at Section 3.3 and Figure 4B

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The inference treats the eight dilemmas and six tasks as exchangeable within participant, but these are fixed, preselected stimuli and not a random sample from a population of dilemmas. The bootstrap CIs and permutation p-values therefore condition on the chosen stimulus set, with no variance component for the stimulus sample.

more of objection 3

This is a common limitation of such studies, and the paper does not claim generalizability beyond the selected dilemmas; I recommend stating this explicitly in Section 3.3 or Section 7, since the headline percentages (40%, 12.9%) are stimulus-conditional estimates.

7 minor objections

The paper says, at Section 6.2

Pith says: minor objection 1 of 7

'may be explained with different contexts call for different aspects of alignment' is ungrammatical; it should read 'may be explained by the fact that different contexts call for different aspects of alignment'.

The paper says, at Figure 4B

Pith says: minor objection 2 of 7

The caption for Figure 4B is confusing, especially the sentence 'The numbers above zero means alignment in decision or reason increased preference'; please rewrite the caption to state clearly which pairwise differences are shown and what the purple and gold bars represent.

show 5 more

The paper says, at Table 1

Pith says: minor objection 3 of 7

The 'Change, Ask→Part 3' row aggregates the Consult, Draft, and Act roles into a single comparison, but the reader is not told how the eight Part-2 choices and the different numbers of Part-3 tasks are weighted in this aggregate; please state the aggregation rule.

The paper says, at Section 5.2 and Figure 5

Pith says: minor objection 4 of 7

The exploratory Spearman correlations are reported with p-values but without confidence intervals; please add 95% CIs for rho or explicitly label these as exploratory point estimates.

The paper says, at Section 8

Pith says: minor objection 5 of 7

The final paragraph contains a typo: 'when and hwy people prioritize' should be 'when and why people prioritize'.

The paper says, at Appendix A.5

Pith says: minor objection 6 of 7

The human validation criterion (at least 85% recovery and at most 5% difference between endorsers and non-endorsers) is described in prose; a table with per-dilemma recovery rates would improve reproducibility.

The paper says, at Table 13

Pith says: minor objection 7 of 7

The columns labeled 'All Both Dec Rea' appear to be a layout artifact; please clean up the table so that counts for All, FullyAligned, DecAligned, and ReaAligned are clearly and separately labeled.

What would settle it

Re-analyze the data using participants' actual choices on the dilemmas (if collected) or collect a new sample where participants first indicate which option they would choose for each dilemma, then compare the PVQ-based value-favored decision to the participant's stated choice. Compute the misclassification rate per participant and per dilemma. Then re-run the main analysis excluding participants or dilemmas where the misclassification rate exceeds a threshold (e.g., 30%). If the preference for ReaAligned persists in the filtered sample, the concern is mitigated; if it weakens or reverses, the central claim depends on the unvalidated mapping.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic agrees: says it holds up only with conditions 9 s · 35,871 tokens read, 692 written The mapping from PVQ-21 scores to value-favored decisions for each dilemma is not validated against participants' actual choices, so the alignment conditions may not reflect what participants truly prefer.
  3. Editor wrote the verdict above 24 s · 36,719 tokens read, 519 written
  4. Referee major revision: 3 major objections, 7 minor 37 s · 36,248 tokens read, 5,124 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper cleanly separates two things that usually get blurred in human-AI alignment work: whether an AI recommends the decision you'd favor (decision alignment) and whether it reasons from values you hold (reason alignment). The experimental design is careful—within-subjects, counterbalanced colors, validated stimuli, permutation tests. The main finding, that people prefer a reason-aligned advisor when seeking advice but shift toward decision alignment when delegating, is genuinely interesting and not obvious. The opt-out rates climbing from 12% to 38% as autonomy increases is a nice secondary result.

The soft spot is load-bearing and the authors know it. The mapping from PVQ-21 scores to "value-favored decision" for each dilemma is never validated against participants' actual choices. If a participant scores higher on self-transcendence but actually favors the self-enhancement option in a given dilemma, then the ReaAligned advisor is mislabeled—it might be recommending what they actually want. The authors acknowledge this in Section 7 ('people's choices may reflect multiple values and contextual considerations not captured by the PVQ-21'), but they don't test whether the mapping holds. The human validation in Appendix A.5 shows that annotators' personal values predicted their choices, which is suggestive but not a calibration. The exploratory analysis of value strength (Section 5.2) hints that participants with stronger values opt out more, consistent with the mapping being more reliable for them, but this is post-hoc.

That said, this is a limitation of the operationalization, not a fatal flaw. The paper is transparent about it, and the core distinction between decision and reason alignment is valuable regardless. The study is well-executed within its scope.

Who this is for: researchers in human-AI interaction, AI alignment, and FAccT who care about how people evaluate AI advisors. It deserves a serious referee. I'd recommend acceptance with a request that the authors add a robustness check—perhaps a subset analysis where the mapping is most reliable (high value-strength participants) to see if the main results hold.

The strongest objection

The central claim that participants prefer ReaAligned (reasoning aligned, decision misaligned) over FullyAligned and DecAligned depends on the assumption that the value-favored decision is correctly identified for each participant and dilemma. The paper uses PVQ-21 scores to determine which option each participant's values favor, but this mapping is never validated against participants' actual decisions on the dilemmas. The authors acknowledge this limitation (Section 7: 'people's choices may reflect multiple values and contextual considerations not captured by the PVQ-21'), but the concern is load-bearing because if the mapping is wrong for a substantial fraction of participants, then the alignment conditions are mislabeled. For example, a participant who actually favors the self-enhancement option but scores higher on self-transcendence would be assigned a ReaAligned advisor that recommends the option they actually favor, making the apparent preference for ReaAligned an artifact of misclassification. The paper's own human validation (Appendix A.5) shows that annotators' personal values predicted their choices, but this relationship is not used to calibrate the mapping. The exploratory analysis of value strength (Section 5.2) hints that participants with stronger values opt out more, which is consistent with the mapping being more reliable for them, but this is not directly tested.

The reader's weakest_assumption correctly identifies that the mapping from PVQ-21 scores to value-favored decisions is a load-bearing assumption. The paper acknowledges this limitation but does not validate the mapping against participants' actual choices. This is not a fatal flaw, but it means the central result is conditional on the mapping being accurate. A concrete test that checks the misclassification rate and re-runs the analysis on a filtered sample would settle whether this concern actually lands. The paper's own human validation data could potentially be used for this test if participants' choices were recorded. The concern is not that the authors are wrong, but that the evidence for the central claim is weaker than it appears because the alignment conditions may not correspond to what participants actually value in each dilemma. The paper's thoroughness in other areas (stimulus validation, statistical analysis, limitation discussion) makes this the most load-bearing concern because it is the one assumption that, if incorrect, could change the interpretation of the main results.

The referee's report

If the results hold, the paper makes a useful empirical contribution to human-AI alignment research by separating decision alignment from reasoning alignment and showing that their relative importance depends on the AI's role. The study is carefully designed: the stimulus construction is documented in detail (Appendix A) with independent LLM and human validation, the within-subjects design and permutation/bootstrap analyses are appropriate, and the authors are explicit about the limits of their operationalization. The machine-verifiable components (validation pipelines, codebooks with high intercoder reliability, and a falsifiable opt-out gradient) are strengths, and the main 'ReaAligned preference at ask stage' is a counterintuitive finding that usefully complicates alignment-as-agreement narratives. The reported increase in opt-outs across consult, draft, and act roles is a clean result with practical implications for agentic AI design.

Pith's reading of the paper (a machine's summary, not the authors')

When people ask an AI for advice on a moral dilemma, the system can recommend the decision they would favor while justifying it with values they reject, or argue against their lean while reasoning from values they endorse. This paper tests which form of alignment people actually prefer, and whether that preference survives when the AI moves from giving advice to drafting messages or acting on the user's behalf. Using a within-subjects study of 121 participants, it finds that when seeking advice people most often chose the advisor that reasoned from their prioritized values while recommending the opposing decision (40% of selections), more than one aligned with both decision and reasoning (31%) or with the decision alone (29%). That preference shifted when the same advisors were considered for new tasks with greater autonomy, and opt-out rates rose from 12% when consulting to 38% when letting an advisor act for them. The paper's central claim is that decision alignment and reasoning alignment shape advisor preferences differently across roles, so that agreeing with a user's values does not by itself guarantee permission to delegate.

What carries the argument. The central object is the contrast between decision alignment and reasoning alignment, operationalized through three advisor conditions crossed over eight everyday moral dilemmas chosen so their two options pull on opposite poles of the same Schwartz value axis. Each participant's PVQ-21 scores determine which option their values favor and which value they prioritize, so the three advisors can be constructed per participant and per dilemma. The design's work is to isolate whether people respond to what an advisor recommends or to the value-based reasons it gives, and to test whether that response survives when the advisor takes on roles granting more autonomy — consulting, drafting a message for review, or acting on the user's behalf.

Core claim. The paper distinguishes an AI advisor's decision (the option it recommends) from its reasoning (the values it invokes to justify that decision), and builds three advisors for each participant from their PVQ-21 value profile: one aligned with both the value-favored decision and the value-based reasoning (FullyAligned), one aligned only with the decision (DecAligned), and one aligned only with the reasoning (ReaAligned). When choosing whom to ask about a dilemma, participants most often selected the ReaAligned advisor (40% of selections) — the one that reasoned from their values while recommending the opposing decision — over FullyAligned (31%) and DecAligned (29%), with the ReaAligned advantage over DecAligned at 10.1 points (p = .002) and over FullyAligned at 8.3 points (p = .006). This preference reversed when participants chose the same advisors for new tasks with more autonomy: the within-participant contrast between FullyAligned and ReaAligned shifted by 12.9 percentage points toward FullyAligned (p = .001), and no advisor was consistently preferred within those tasks. Separately, the proportion of choices selecting no advisor rose from 12% for consultation to 38% for letting an advisor act on the participant's behalf. The authors conclude that decision and reasoning alignment shape advisor preferences differently across contexts, and that alignment with a user's values does not necessarily imply willingness to delegate.

The premise it rests on. The study assumes that the PVQ-21 value questionnaire and the mapping of the eight dilemmas to Schwarz value poles correctly identify, for each participant and dilemma, which decision their values favor — so that the three advisor conditions are genuinely aligned or misaligned as labelled; the paper acknowledges that people's choices can reflect multiple values and contextual considerations the questionnaire does not capture.

If the paper is right (Pith's inferences, not the authors')

Evaluating personalized AI alignment must separate the decision an advisor recommends from the values in its reasoning, since these dimensions affect preferences independently and differently across roles.

An AI advisor can be preferred for advice even when it recommends against the option a user's values favor, provided its reasoning is grounded in those values — a devil's-advocate role participants seemed to welcome when reading advice.

Willingness to delegate is not guaranteed by value alignment: participants opted out more as advisor autonomy grew, from 12% when consulting to 38% when letting an advisor act on their behalf.

Agreement with an advisor's judgment is an inconclusive signal of alignment, since the same recommendation can be supported by values the user does not prioritize.

Alignment evaluations should be role-sensitive, distinguishing whether a system is meant to support deliberation, produce drafts for review, or act on a user's behalf.

The preference for reason-aligned, decision-misaligned advice may generalize beyond moral dilemmas to any domain where users retain final control and want to stress-test their own leanings, suggesting a default expectation of 'deliberative complementarity' rather than decision congruence for advisory AI.

The rising opt-out with increasing autonomy hints that users treat delegation as a distinct decision governed by accountability and outcome stakes, not merely by value similarity; a testable extension would hold alignment constant while varying who bears responsibility for outcomes.

Because the value-favored decision is inferred from questionnaire scores rather than stated intentions, a natural robustness check is whether the 40% ReaAligned preference and the 12.9-point shift toward FullyAligned survive when alignment is defined against participants' self-reported preferences on each dilemma.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

When people seek advice from AI on social situations, the system may recommend a decision they fully agree with but justify it through reasoning they disagree with. Conversely, it may offer reasoning they agree with, and yet reach a decision they reject. We distinguish decision alignment, whether an advisor recommends the option favored by a user's measured values, from reason alignment, whether its stated reasons align with the user's values. In this work, we ask how these two forms of alignment shape people's preferences for AI advisors and their willingness to use them in different roles. We conducted a within-subjects study (N=121) in which participants evaluated three AI advisors varying in decision and reason alignment across eight everyday moral dilemmas. Participants then chose which advisors they would consult, allow to draft messages for review, or allow to act on their behalf in new tasks. When seeking advice, participants most often preferred an advisor that reasoned through their prioritized values while recommending the opposing decision (40%). However, when participants had to pick advisors for new tasks, their preferences shifted toward advisors that also recommended decisions aligned with their values, rather than those that only shared their reasons. Also, as participants considered tasks that gave AI advisors greater autonomy, they became more likely to opt out of using any advisor, from 12% when consulting to 38% when allowing an advisor to act on their behalf. Our findings suggest that decision and reason alignment shape advisor preferences differently across contexts of AI use, and that alignment with a user's values does not necessarily imply willingness to delegate.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 1.00

“Our findings suggest that decision and reason alignment shape advisor preferences differently across contexts of AI use, and that alignment with a user's values does not necessarily imply willingness to delegate.”

How the main claim is stated: hedged possibility 0.99

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.30 · human participants 0.98 · controlled comparison 0.11 · randomized assignment 0.05 · states quantitative result 0.99 · compares with prior results 0.09 · mentions code or data availability 0.02

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
When people seek advice from AI on social situations, the system may recommend a decision they fully agree with but justify it through reasoning they disagree with. background 0.99 0.07 0.03 hedged possibility 1.00
Conversely, it may offer reasoning they agree with, and yet reach a decision they reject. background 0.99 0.09 0.02 hedged possibility 0.98
We distinguish decision alignment, whether an advisor recommends the option favored by a user's measured values, from reason alignment, whether its stated reasons align with the user's values. contribution 0.67 0.39 0.03 asserted 0.95
In this work, we ask how these two forms of alignment shape people's preferences for AI advisors and their willingness to use them in different roles. contribution 0.66 0.22 0.03 unclear 0.84
We conducted a within-subjects study (N=121) in which participants evaluated three AI advisors varying in decision and reason alignment across eight everyday moral dilemmas. method 0.96 0.97 0.02 asserted 0.99
Participants then chose which advisors they would consult, allow to draft messages for review, or allow to act on their behalf in new tasks. method 0.99 0.94 0.02 asserted 0.99
When seeking advice, participants most often preferred an advisor that reasoned through their prioritized values while recommending the opposing decision (40%). result 1.00 0.95 0.09 asserted 1.00
However, when participants had to pick advisors for new tasks, their preferences shifted toward advisors that also recommended decisions aligned with their values, rather than those that only shared their reasons. result 1.00 0.95 0.16 asserted 1.00
Also, as participants considered tasks that gave AI advisors greater autonomy, they became more likely to opt out of using any advisor, from 12% when consulting to 38% when allowing an advisor to act on their behalf. result 1.00 0.96 0.07 asserted 0.98
Our findings suggest that decision and reason alignment shape advisor preferences differently across contexts of AI use, and that alignment with a user's values does not necessarily imply willingness to delegate. interpretation 1.00 0.91 0.93 hedged possibility 0.99

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 7,633 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (15)
Figure 1 of the paper
Figure 1. Advisor preferences shift from evaluating advice to considering AI use. (A) A participant’s value profile and an everyday dilemma. (B) Their value scores determine the three AI advisor’s reply on every dilemma, and advisor colors are counterbalanced across participants. (C) Participants most often chose the reasoning-aligned advisor when selecting whom to ask for advice, but this preference weakened when they considered using the same advisors for new tasks. When people seek advice from AI on social situations, the system may recommend a decision they fully agree with but justify it…
Figure 2 of the paper
Figure 2. Dilemmas and advisor replies. Top: Eight dilemmas, each involving a pair of competing Schwartz values. Bottom: An example crossing two recommendations with two value-based justifications. Advisor alignment conditions are assigned relative to each participant’s value profile; labels shown here assume benevolence is ranked above achievement. Only three of the four replies are presented to each participant. Full set of dilemmas and replies are shown in Appendix B separate model, GPT-4.1-mini, checked whether each reply clearly expressed its intended decision and the reason, and whether…
Figure 3 of the paper
Figure 3. Study procedure. (1) Participants report their AI use and AI literacy and complete the PVQ-21. Their value scores determine the three advisors’ alignment condition. Advisor colors are counterbalanced. (2) Across eight dilemmas, participants compare the three advisors’ replies and choose whom they would want to ask (RQ1). (3) For six tasks, participants choose which advisor, if any, they would Consult and let Act for them. For five friends’ messages, they choose whether to write the reply themselves or let an advisor Draft it (RQ2). A recap of each advisor’s decisions is available…
Figure 4 of the paper
Figure 4. Advisor selection across study settings. (A) The proportion of each participant’s advisor choices in Part 2 with 95% CIs. (B) Within-person gaps between Part 2 and Part 3 between two advisors that differ in the decision or reason. The numbers above zero means alignment in decision or reason increased preference (purple: FullyAligned−ReaAligned) or decision (gold: FullyAligned−DecAligned). selection proportions conditional on choosing an advisor, excluding “None of them.” This accounts for the difference in choice sets: participants were required to select an advisor in Part 2 but…
Figure 5 of the paper
Figure 5. Opt-out rates by value strength. Bars show the proportion opt-outs in Con￾sult, Draft, and Act, grouped by value strength. Because advisor alignment was defined relative to partici￾pants’ competing values, we additionally explored whether value strength was associated with opting out of AI assis￾tance across Consult, Draft, and Act roles. We define value strength as how much more strongly a participant prioritizes one competing value over another, measured by the mean absolute difference between their PVQ-21 scores across eight dilemmas. For visualization and analysis, we grouped…
Figure 6 of the paper
Figure 6. Prompt for mapping dilemma tensions onto Schwartz values.
Figure 7 of the paper
Figure 7. Prompt for generating a vignette, two decision options, and four advisor replies (GPT-5.5).
Figure 8 of the paper
Figure 8. Prompt for recovering the expressed value and recommended decision (one evaluation per reply).
Figure 9 of the paper
Figure 9. Prompt for rating reply quality (one evaluation per reply).
Figure 10 of the paper
Figure 10. Prompt for distinguishing value-based reasons behind the same recommendation (one evaluation per
Figure 11 of the paper
Figure 11. Prompt for checking whether options are opposed and which option serves value P (one evaluation
Figure 12 of the paper
Figure 12. Where the eight dilemmas sit on Schwartz’s value circle. Each colored line joins the two values one
Figure 13 of the paper
Figure 13. Background survey and Part 1. Representative screens for prior AI use, AI literacy, and the PVQ-21 value questionnaire
Figure 14 of the paper
Figure 14. Part 2: Choosing whom to ask. Participants compared three responses to each dilemma and selected an advisor; they subsequently indicated their overall preferred advisor and explained why. Part 3. After the full interaction, choose which one to use for different tasks
Figure 15 of the paper
Figure 15. Part 3: Prospective AI use. Representative screens for selecting advisors for tasks, reviewing earlier responses, and choosing whether to have an advisor draft a message. Participants could also choose no advisor

Figures are the authors', reproduced from the paper.

References (74)
  1. [1] Hasan Amin, Ming Yin, and Rajiv Khanna. 2026. Align when they want, complement when they need! human- centered ensembles for adaptive human-AI collaboration. InProceedings of the Fortieth AAAI Conference on Artificial Intelligence and Thirty-Eighth Conference on Innovative App... on Pithdoi:10.1609/aaai.v40i21.38786
  2. [2] Anthropic. 2026. Introducing Claude Opus 4.8. https://www.anthropic.com/news/claude-opus-4-8. Model identifier: claude-opus-4-8. Accessed October 19, 2026 on Pith
  3. [3] Anthropic. 2026. Introducing Claude Opus 5.5. https://www.anthropic.com/claude-opus-5-5 on Pith
  4. [4] Anthropic. 2026. Introducing Claude Sonnet 5.5. https://www.anthropic.com/claude-sonnet-5-5 on Pith
  5. [5] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073(2022) on PithPith's reviewarXiv
  6. [6] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. InProceedings of the 2021 CHI Conference on Human Facto... on Pithdoi:10.1145/3411764.3445717
  7. [7] Anat Bardi and Shalom H Schwartz. 2003. Values and behavior: Strength and structure of relations.Personality and social psychology bulletin29, 10 (2003), 1207–1220 on Pith
  8. [8] Yochanan E Bigman and Kurt Gray. 2018. People are averse to machines making moral decisions.Cognition181 (2018), 21–34 on Pith
  9. [9] Robert Böhm, Moritz Jörling, Leonhard Reiter, and Christoph Fuchs. 2023. People devalue generative AI’s competence but not its advice in addressing societal and personal challenges.Communications Psychology1, 1 (2023), 32 on Pith
  10. [10] Silvia Bonaccio and Reeshad S Dalal. 2006. Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences.Organizational behavior and human decision processes101, 2 (2006), 127–151 on Pith
  11. [11] Noah Castelo, Maarten W Bos, and Donald R Lehmann. 2019. Task-dependent algorithm aversion.Journal of marketing research56, 5 (2019), 809–825 on Pith
  12. [12] Alan Chan, Carson Ezell, Max Kaufmann, Kevin Wei, Lewis Hammond, Herbie Bradley, Emma Bluemke, Nitarshan Rajkumar, David Krueger, Noam Kolt, Lennart Heim, and Markus Anderljung. 2024. Visibility into AI Agents. In Proceedings of the 2024 ACM Conference on Fairness, Accountabil... on Pithdoi:10.1145/3630106.3658948
  13. [13] Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstanti... on Pith
  14. [14] Aaron Chatterji, Thomas Cunningham, David J Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman. 2025.How people use chatgpt. Technical Report. National Bureau of Economic Research on Pith
  15. [15] Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky. 2026. Sycophantic AI decreases prosocial intentions and promotes dependence.Science391, 6792 (2026), eaec8352 on Pith
  16. [16] Chun-Wei Chiang, Zhuoran Lu, Zhuoyan Li, and Ming Yin. 2024. Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil’s Advocate. InProceedings of the 29th International Conference on Intelligent User Interfaces (Greenville, SC, USA)(IUI ’24). Association for Comp... on Pith
  17. [17] Yu Ying Chiu, Liwei Jiang, and Yejin Choi. 2025. DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life. InThe Thirteenth International Conference on Learning Representations. https://openreview.net/forum? id=PGhiPGBf47 on Pith
  18. [18] Yu Ying Chiu, Michael S. Lee, Rachel Calcott, Brandon Handoko, Paul de Font-Reaulx, Raphaël Millière, Paula Rodriguez, Chen Bo Calvin Zhang, Ziwen Han, Udari Madhushani Sehwag, Yash Maurya, Christina Q Knight, Harry R. Lloyd, Florence Bacus, Conor Downey, Mantas Mazeika, Bing ... on Pith
  19. [19] Rudrajit Choudhuri, Christian Bird, Carmen Badea, Marco Gerosa, and Anita Sarma. 2026. You Shall Not Pass! Where and Why Developers Draw The Line on AI Autonomy.arXiv preprint arXiv:2607.00533(2026) on PithPith's reviewarXiv
  20. [20] Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep Reinforcement Learning from Human Preferences. InAdvances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and... on Pith
show 54 more
  1. [21] Kathleen M Eisenhardt. 1989. Agency theory: An assessment and review.Academy of management review14, 1 (1989), 57–74 on Pith
  2. [22] Cedric Faas, Richard Uth, Sarah Sterz, Markus Langer, and Anna Maria Feit. 2026. AI Support in Safety-Critical Tasks: How Perceived Autonomy and Meaningfulness Shape Human-AI Joint Performance. InProceedings of the 2026 ACM Conference on Fairness, Accountability, and Transpare... on Pithdoi:10.1145/3805689.3812265
  3. [23] Kevin J Feng, David W McDonald, and Amy X Zhang. 2025. Levels of autonomy for ai agents.arXiv preprint arXiv:2506.12469(2025) on PithPith's reviewarXiv
  4. [24] Andreas Fügener, Jörn Grahl, Alok Gupta, and Wolfgang Ketter. 2022. Cognitive challenges in human–artificial intelligence collaboration: Investigating the path toward productive delegation.Information systems research33, 2 (2022), 678–696 on Pith
  5. [25] Iason Gabriel. 2020. Artificial intelligence, values, and alignment: I. gabriel.Minds and machines30, 3 (2020), 411–437 on Pith
  6. [26] Alice Gao, Andrew N. Meltzoff, Maarten Sap, and Katharina Reinecke. 2026. Framing an AI with Values Reduces AI Reliance in AI-supported Writing Tasks. InProceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency(Montreal, QC, Canada)(FAccT ’26). Assoc... on Pithdoi:10.1145/3805689.3812369
  7. [27] Francesca Gino, Jen Shang, and Rachel Croson. 2009. The impact of information from similar or different advisors on judgment.Organizational behavior and human decision processes108, 2 (2009), 287–302 on Pith
  8. [28] Jan Gogoll and Matthias Uhl. 2018. Rage against the machine: Automation in the moral domain.Journal of Behavioral and Experimental Economics74 (2018), 97–103 on Pith
  9. [29] Kunal Handa, Alex Tamkin, Miles McCain, Saffron Huang, Esin Durmus, Sarah Heck, Jared Mueller, Jerry Hong, Stuart Ritchie, Tim Belonax, et al. 2025. Which economic tasks are performed with ai? evidence from millions of claude conversations.arXiv preprint arXiv:2503.04761(2025) on PithPith's reviewarXiv
  10. [30] William Hart, Dolores Albarracín, Alice H Eagly, Inge Brechan, Matthew J Lindberg, and Lisa Merrill. 2009. Feeling validated versus being correct: a meta-analysis of selective exposure to information.Psychological bulletin135, 4 (2009), 555 on Pith
  11. [31] Jess Hohenstein, Rene F Kizilcec, Dominic DiFranzo, Zhila Aghajari, Hannah Mieczkowski, Karen Levy, Mor Naaman, Jeffrey Hancock, and Malte F Jung. 2023. Artificial intelligence in communication impacts language and social relationships.Scientific reports13, 1 (2023), 5487 on Pith
  12. [32] Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023. Co-Writing with Opinionated Language Models Affects Users’ Views. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems(Hamburg, Germany)(CHI ’23). Association for Comp... on Pithdoi:10.1145/3544548.3581196
  13. [33] Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny T. Liang, Sydney Levine, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jack Hessel, Jon Borchardt, Taylor Sorensen, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, and Yejin Choi. 2... on Pithdoi:10.1038/s42256-024-00969-6
  14. [34] Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan. 2024. "I’m Not Sure, But... ": Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust. In Proceedings of the 2024 ACM Conference on Fa... on Pithdoi:10.1145/3630106.3658941
  15. [35] Sara Kingsley, Jiayin Zhi, Wesley Hanwen Deng, Jaimie Lee, Sizhe Zhang, Motahhare Eslami, Kenneth Holstein, Jason I Hong, Tianshi Li, and Hong Shen. 2024. Investigating what factors influence users’ rating of harmful algorithmic bias and discrimination. InProceedings of the AA... on Pith
  16. [36] Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. 2024. The benefits, risks and bounds of personalizing the alignment of large language models to individuals.Nature Machine Intelligence6, 4 (2024), 383–392 on Pith
  17. [37] Nils Köbis, Zoe Rahwan, Raluca Rilla, Bramantyo Ibrahim Supriyatno, Clara Bersch, Tamer Ajaj, Jean-François Bonnefon, and Iyad Rahwan. 2025. Delegation to artificial intelligence can increase dishonest behaviour.Nature646, 8083 (2025), 126–134. 18 Eunkyu Park et al on Pith
  18. [38] Sebastian Krügel, Andreas Ostermaier, and Matthias Uhl. 2023. ChatGPT’s inconsistent moral advice influences users’ judgment.Scientific Reports13, 1 (2023), 4569 on Pith
  19. [39] Aakriti Kumar, Ren-Hui Michelle Tham, and Mark Steyvers. 2025. Seeing Things Differently: The Role of Differing Perspectives in Advice-Taking. InProceedings of the Annual Meeting of the Cognitive Science Society, Vol. 47 on Pith
  20. [40] Deepak Kumar, Yousef Anees AbuHashem, and Zakir Durumeric. 2024. Watch your language: Investigating content moderation with large language models. InProceedings of the International AAAI Conference on Web and Social Media, Vol. 18. 865–878 on Pith
  21. [41] Ethan Landes, Kathryn B Francis, and Jim AC Everett. 2026. People defer to AI moral advice, but not blindly.Cognition 272 (2026), 106504 on Pith
  22. [42] Julie Anne Lee, Anat Bardi, Paul Gerrans, Joanne Sneddon, Hester Van Herk, Uwana Evers, and Shalom Schwartz on Pith
  23. [43] John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance.Human factors46, 1 (2004), 50–80 on Pith
  24. [44] Margarita Leib, Nils Köbis, Rainer Michael Rilke, Marloes Hagens, and Bernd Irlenbusch. 2024. Corrupted by algorithms? How AI-generated and human-written advice shape (dis) honesty.The Economic Journal134, 658 (2024), 766–784 on Pith
  25. [45] Zhuoyan Li, Zhuoran Lu, and Ming Yin. 2023. Modeling human trust and reliance in ai-assisted decision making: A markovian approach. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 6056–6064 on Pith
  26. [46] Jiarui Liu, Yueqi Song, Yunze Xiao, Mingqian Zheng, Lindia Tjuatja, Jana Schaich Borg, Mona T. Diab, and Maarten Sap. 2025. Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics. In Proceedings of the 2025 Conference on Empirical Metho... on Pithdoi:10.18653/v1/2025.emnlp-main.831
  27. [47] Tao Long and Lydia B. Chilton. 2026. SPEAR: Five Principles for Interactive Human-Agent Alignment. InProceedings of the 2026 ACM Conference on Human-AI Complementarity and Alignment(Alexandria, VA, USA)(HCOMP ’26). Association for Computing Machinery, New York, NY, USA. https:... on PithPith's reviewarXiv
  28. [48] Brian Lubars and Chenhao Tan. 2019. Ask not what AI can do, but what AI should do: Towards a framework of task delegability.Advances in neural information processing systems32 (2019) on Pith
  29. [49] Roger C Mayer, James H Davis, and F David Schoorman. 1995. An integrative model of organizational trust.Academy of management review20, 3 (1995), 709–734 on Pith
  30. [50] Siddharth Mehrotra, Catholijn M Jonker, and Myrthe L Tielman. 2021. More similar values, more trust?-the effect of value similarity on trust in human-agent interaction. InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 777–783 on Pith
  31. [51] Tim Miller. 2023. Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven Decision Support using Evaluative AI. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency(Chicago, IL, USA)(FAccT ’23). Association for Computing Machinery,... on Pithdoi:10.1145/3593013.3594001
  32. [52] Jimin Mun, Wei Bin Au Yeong, Wesley Hanwen Deng, Jana Schaich Borg, and Maarten Sap. 2025. Why (not) use AI? Analyzing people’s reasoning and conditions for AI acceptability. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 8. 1771–1784 on Pith
  33. [53] OpenAI. 2026. Introducing GPT-5.5. April 23, 2026. https://openai.com/index/introducing-gpt-5-5/ on Pith
  34. [54] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems35 ... on Pith
  35. [55] Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan, Mingxi Yan, Gunhee Kim, Maarten Sap, and Motahhare Eslami. 2026. Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations. In Proceedings of the 2026 Conference on Empirical Methods in Nat... on Pithdoi:10.18653/2026.emnlp-main.2050
  36. [56] Shalom H Schwartz, Jan Cieciuch, Michele Vecchione, Eldad Davidov, Ronald Fischer, Constanze Beierlein, Alice Ramos, Markku Verkasalo, Jan-Erik Lönnqvist, Kursad Demirutku, et al. 2012. Refining the theory of basic individual values.Journal of Personality and Social Psychology... on Pith
  37. [57] Julia Sepúlveda Coelho and Scott A Hale. 2026. What Do People Actually Want From AI? Mapping Preference Plurality. InProceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency(Montreal, QC, Canada)(FAccT ’26). Association for Computing Machinery, New ... on Pithdoi:10.1145/3805689.3812398
  38. [58] Mrinank Sharma, Meg Tong, Tomek Korbak, David Duvenaud, Amanda Askell, Sam Bowman, Esin Durmus, Zac Hatfield-Dodds, Scott Johnston, Shauna Kravec, et al. 2024. Towards understanding sycophancy in language models. InInternational Conference on Learning Representations, Vol. 202... on Pith
  39. [59] Hua Shen, Tiffany Knearem, Reshmi Ghosh, Kenan Alkiek, Kundan Krishna, Yachuan Liu, Savvas Petridis, Yi-Hao Peng, Li Qiwei, Chenglei Si, Yutong Xie, Jeffrey P. Bigham, Frank Bentley, Joyce Chai, Zachary Chase Lipton, Qiaozhu Right Advice, Wrong Reasons: How Decision and Reason... on Pith
  40. [60] Michael Siegrist, George Cvetkovich, and Claudia Roth. 2000. Salient Value Similarity, Social Trust, and Risk/Benefit Perception.Risk Analysis20, 3 (2000), 353–362. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/0272-4332.203034 doi:10.1111/0272-4332.203034 on Pithdoi:10.1111/0272-4332.203034
  41. [61] Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. 2025. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267(2025) on PithPith's reviewarXiv
  42. [62] Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L Gordon, Niloofar Mireshghallah, Christopher Michael Ryt- ting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. 2024. Position: A Roadmap to Pluralistic Alignment. InForty-first International C... on Pith
  43. [63] Mary Steffel, Elanor F. Williams, and Jaclyn Perrmann-Graham. 2016. Passing the buck: Delegating choices to others to avoid responsibility and blame.Organizational Behavior and Human Decision Processes135 (2016), 32–44. doi:10.1016/j.obhdp.2016.04.006 on Pithdoi:10.1016/j.obhdp.2016.04.006
  44. [64] Alex Tamkin, Miles McCain, Kunal Handa, Esin Durmus, Liane Lovitt, Ankur Rathi, Saffron Huang, Alfred Mountfield, Jerry Hong, Stuart Ritchie, Michael Stern, Brian Clarke, Landon Goldberg, Theodore R. Sumers, Jared Mueller, William McEachen, Wes Mitchell, Shan Carter, Jack Clar... on PithPith's reviewarXiv
  45. [65] Michael Terry, Chinmay Kulkarni, Martin Wattenberg, Lucas Dixon, and Meredith Ringel Morris. 2024. Interactive AI Alignment: Specification, Process, and Evaluation Alignment. arXiv:2311.00710 [cs.HC] https://arxiv.org/abs/2311. 00710 on PithPith's reviewarXiv
  46. [66] Suzanne Tolmeijer, Markus Christen, Serhiy Kandul, Markus Kneer, and Abraham Bernstein. 2022. Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans,... on Pithdoi:10.1145/3491102.3517732
  47. [67] Matt Twyman, Nigel Harvey, and Clare Harries. 2008. Trust in motives, trust in competence: Separate factors determining the effectiveness of risk communication.Judgment and Decision Making3, 1 (2008), 111–120. doi:10.1017/ S1930297500000218 on Pith
  48. [68] Oleksandra Vereschak, Gilles Bailly, and Baptiste Caramiaux. 2021. How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies.Proc. ACM Hum.-Comput. Interact.5, CSCW2, Article 327 (Oct. 2021), 39 pages. doi:10.1145/3476068 on Pithdoi:10.1145/3476068
  49. [69] Kyra Wilson and Aylin Caliskan. 2025. Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval. InProceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society(San Jose, California, USA)(AIES ’24). AAAI Press, 1578–1590 on Pith
  50. [70] Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency(Barcelona, Spain)(FAT* ’20). A... on Pithdoi:10.1145/3351095.3372852
  51. [71] Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky, and Maarten Sap. 2025. REL-A.I.: An Interaction- Centered Approach To Measuring Human-LM Reliance. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computationa... on Pithdoi:10.18653/v1/2025.naacl-
  52. [72] Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, and Maarten Sap. 2024. Relying on the Unreliable: The Impact of Language Models’ Reluctance to Express Uncertainty. InProceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). Asso... on Pithdoi:10.18653/v1/2024.acl-long.198
  53. [74] <IDENTIFYING RUBRIC ITEM 1> on Pith
  54. [2022] Are value–behavior relations stronger than previously thought? It depends on value importance.European journal of personality36, 2 (2022), 133–148 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment