{"id":"cc331dc4-8664-4b8b-8443-9745501e3b77","arxiv_id":"2611.15732","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"When seeking AI advice, people prefer advisors that reason from their values even if recommending the opposite decision, but this preference shifts toward decision-aligned advisors when delegating tasks.","lead":"This paper reports a user study (N=121) examining how people choose among AI advisors that either agree with their preferred decision, their reasoning, or both, across everyday moral dilemmas. The study finds that when seeking advice, participants most often prefer an advisor that reasons from their values but recommends the opposite decision, yet this preference shifts toward decision-aligned advisors when considering tasks with greater AI autonomy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The mapping from PVQ-21 scores to value-favored decisions for each dilemma is not validated against participants' actual choices, so the alignment conditions may not reflect what participants truly prefer.","rationale":"The reader's weakest_assumption correctly identifies that the mapping from PVQ-21 scores to value-favored decisions is a load-bearing assumption. The paper acknowledges this limitation but does not validate the mapping against participants' actual choices. This is not a fatal flaw, but it means the central result is conditional on the mapping being accurate. A concrete test that checks the misclassification rate and re-runs the analysis on a filtered sample would settle whether this concern actually lands. The paper's own human validation data could potentially be used for this test if participants' choices were recorded. The concern is not that the authors are wrong, but that the evidence for the central claim is weaker than it appears because the alignment conditions may not correspond to what participants actually value in each dilemma. The paper's thoroughness in other areas (stimulus validation, statistical analysis, limitation discussion) makes this the most load-bearing concern because it is the one assumption that, if incorrect, could change the interpretation of the main results.","tokens_in":35871,"tokens_out":692,"duration_ms":8886,"concrete_test":"Re-analyze the data using participants' actual choices on the dilemmas (if collected) or collect a new sample where participants first indicate which option they would choose for each dilemma, then compare the PVQ-based value-favored decision to the participant's stated choice. Compute the misclassification rate per participant and per dilemma. Then re-run the main analysis excluding participants or dilemmas where the misclassification rate exceeds a threshold (e.g., 30%). If the preference for ReaAligned persists in the filtered sample, the concern is mitigated; if it weakens or reverses, the central claim depends on the unvalidated mapping.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that participants prefer ReaAligned (reasoning aligned, decision misaligned) over FullyAligned and DecAligned depends on the assumption that the value-favored decision is correctly identified for each participant and dilemma. The paper uses PVQ-21 scores to determine which option each participant's values favor, but this mapping is never validated against participants' actual decisions on the dilemmas. The authors acknowledge this limitation (Section 7: 'people's choices may reflect multiple values and contextual considerations not captured by the PVQ-21'), but the concern is load-bearing because if the mapping is wrong for a substantial fraction of participants, then the alignment conditions are mislabeled. For example, a participant who actually favors the self-enhancement option but scores higher on self-transcendence would be assigned a ReaAligned advisor that recommends the option they actually favor, making the apparent preference for ReaAligned an artifact of misclassification. The paper's own human validation (Appendix A.5) shows that annotators' personal values predicted their choices, but this relationship is not used to calibrate the mapping. The exploratory analysis of value strength (Section 5.2) hints that participants with stronger values opt out more, which is consistent with the mapping being more reliable for them, but this is not directly tested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper reports a within-subjects user study (N=121) in which three AI advisors are constructed from each participant's PVQ-21 value profile, varying whether the advisor's recommended decision and/or its stated value-based reasons align with the participant's measured values. The advisors are evaluated across eight everyday moral dilemmas, and the study examines both which advisor participants would ask for advice (Part 2) and which advisors they would consult, allow to draft, or allow to act on their behalf in new tasks (Part 3). The main empirical claims are that participants most often prefer the reason-aligned advisor (ReaAligned, 40%) when asking for advice, that this preference shifts toward the fully aligned advisor when selecting advisors for new tasks (within-participant contrast shifts 12.9 percentage points, p=.001), and that opt-out rates increase from 12% (consult) to 38% (act). The paper also reports exploratory analyses on value strength and open-text explanations of advisor preferences.","tokens_in":36248,"tokens_out":5124,"duration_ms":36916,"significance":"If the results hold, the paper makes a useful empirical contribution to human-AI alignment research by separating decision alignment from reasoning alignment and showing that their relative importance depends on the AI's role. The study is carefully designed: the stimulus construction is documented in detail (Appendix A) with independent LLM and human validation, the within-subjects design and permutation/bootstrap analyses are appropriate, and the authors are explicit about the limits of their operationalization. The machine-verifiable components (validation pipelines, codebooks with high intercoder reliability, and a falsifiable opt-out gradient) are strengths, and the main 'ReaAligned preference at ask stage' is a counterintuitive finding that usefully complicates alignment-as-agreement narratives. The reported increase in opt-outs across consult, draft, and act roles is a clean result with practical implications for agentic AI design.","major_comments":[{"comment":"The central alignment conditions are constructed from the assumption that the higher-scoring PVQ-21 value within each dilemma's pair identifies the participant's value-favored decision, but this mapping is not validated against participants' actual choices on the dilemmas. The authors acknowledge this in Section 7 ('our measure therefore captures alignment with participants' measured value profiles rather than necessarily matching the decisions they would make'), yet the Part 2 preference for ReaAligned over FullyAligned is exactly the comparison that would be inflated if the mapping mislabels a substantial share of participants' favored decisions. I request a robustness analysis, for example excluding or re-analyzing participants whose value strength is low, or using the Appendix A.5 validation data to estimate how often the value-favored option coincides with annotators' actual choices. Without such an analysis, the central Part 2 effect is defensible but rests on an untested mapping, and the concern is load-bearing because the ReaAligned, DecAligned, and FullyAligned labels are defined through that mapping.","section":"Section 3.1.3 and Appendix A.5","authors_acknowledge":"Our measure therefore captures alignment with participants’ measured value profiles rather than necessarily matching the decisions they would make.","authors_acknowledge_p":0.86},{"comment":"The claim that preferences 'shift toward FullyAligned' in Part 3 relies on comparing conditional selection proportions while the choice set changes: Part 2 forces a choice among the three advisors, whereas Part 3 includes a 'None of them' option. The authors partially address this by conditioning on advisor selection and reporting opt-out separately, but the within-participant change analysis is sensitive to which participants remain in the conditional set, and the Consult, Draft, and Act rows have n=114, 105, and 93 respectively. I request either a sensitivity analysis that includes participants who opt out in Part 3 with an explicit adjustment, or a clear statement that the 12.9-point shift is estimated only among participants who selected at least one advisor in both parts. This matters because the RQ2 shift is the paper's second central result and the n-reduction across rows raises the possibility that selective opt-out drives part of the apparent preference change.","section":"Section 4.2 and Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The inference treats the eight dilemmas and six tasks as exchangeable within participant, but these are fixed, preselected stimuli and not a random sample from a population of dilemmas. The bootstrap CIs and permutation p-values therefore condition on the chosen stimulus set, with no variance component for the stimulus sample. This is a common limitation of such studies, and the paper does not claim generalizability beyond the selected dilemmas; I recommend stating this explicitly in Section 3.3 or Section 7, since the headline percentages (40%, 12.9%) are stimulus-conditional estimates.","section":"Section 3.3 and Figure 4B","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"'may be explained with different contexts call for different aspects of alignment' is ungrammatical; it should read 'may be explained by the fact that different contexts call for different aspects of alignment'.","section":"Section 6.2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The caption for Figure 4B is confusing, especially the sentence 'The numbers above zero means alignment in decision or reason increased preference'; please rewrite the caption to state clearly which pairwise differences are shown and what the purple and gold bars represent.","section":"Figure 4B","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The 'Change, Ask→Part 3' row aggregates the Consult, Draft, and Act roles into a single comparison, but the reader is not told how the eight Part-2 choices and the different numbers of Part-3 tasks are weighted in this aggregate; please state the aggregation rule.","section":"Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The exploratory Spearman correlations are reported with p-values but without confidence intervals; please add 95% CIs for rho or explicitly label these as exploratory point estimates.","section":"Section 5.2 and Figure 5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The final paragraph contains a typo: 'when and hwy people prioritize' should be 'when and why people prioritize'.","section":"Section 8","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The human validation criterion (at least 85% recovery and at most 5% difference between endorsers and non-endorsers) is described in prose; a table with per-dilemma recovery rates would improve reproducibility.","section":"Appendix A.5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The columns labeled 'All Both Dec Rea' appear to be a layout artifact; please clean up the table so that counts for All, FullyAligned, DecAligned, and ReaAligned are clearly and separately labeled.","section":"Table 13","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"The paper cleanly separates two things that usually get blurred in human-AI alignment work: whether an AI recommends the decision you'd favor (decision alignment) and whether it reasons from values you hold (reason alignment). The experimental design is careful—within-subjects, counterbalanced colors, validated stimuli, permutation tests. The main finding, that people prefer a reason-aligned advisor when seeking advice but shift toward decision alignment when delegating, is genuinely interesting and not obvious. The opt-out rates climbing from 12% to 38% as autonomy increases is a nice secondary result.\n\nThe soft spot is load-bearing and the authors know it. The mapping from PVQ-21 scores to \"value-favored decision\" for each dilemma is never validated against participants' actual choices. If a participant scores higher on self-transcendence but actually favors the self-enhancement option in a given dilemma, then the ReaAligned advisor is mislabeled—it might be recommending what they actually want. The authors acknowledge this in Section 7 ('people's choices may reflect multiple values and contextual considerations not captured by the PVQ-21'), but they don't test whether the mapping holds. The human validation in Appendix A.5 shows that annotators' personal values predicted their choices, which is suggestive but not a calibration. The exploratory analysis of value strength (Section 5.2) hints that participants with stronger values opt out more, consistent with the mapping being more reliable for them, but this is post-hoc.\n\nThat said, this is a limitation of the operationalization, not a fatal flaw. The paper is transparent about it, and the core distinction between decision and reason alignment is valuable regardless. The study is well-executed within its scope.\n\nWho this is for: researchers in human-AI interaction, AI alignment, and FAccT who care about how people evaluate AI advisors. It deserves a serious referee. I'd recommend acceptance with a request that the authors add a robustness check—perhaps a subset analysis where the mapping is most reliable (high value-strength participants) to see if the main results hold.","headline":"Solid user study on decision vs. reason alignment in AI advisors, but the central result depends on an unvalidated mapping from value scores to value-favored decisions.","tokens_in":36719,"tokens_out":519,"duration_ms":24276},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"People prefer AI advice that matches their values, not their call — until the AI has to act for them.","keywords":["AI alignment","human-AI interaction","decision alignment","reasoning alignment","AI advice","value alignment","reliance and delegation","Schwartz values"],"falsifier":"A replication in which participants state their own value-favored decision for each dilemma, and advisors are built from those stated preferences rather than from PVQ-21 inference, checking whether the 40% preference for the reason-aligned, decision-misaligned advisor and the 12.9-point shift toward the fully aligned advisor when delegating still appear.","tokens_in":37340,"feed_emoji":"🤖","tokens_out":4976,"duration_ms":27684,"temperature":0.7,"pith_summary":"When people ask an AI for advice on a moral dilemma, the system can recommend the decision they would favor while justifying it with values they reject, or argue against their lean while reasoning from values they endorse. This paper tests which form of alignment people actually prefer, and whether that preference survives when the AI moves from giving advice to drafting messages or acting on the user's behalf. Using a within-subjects study of 121 participants, it finds that when seeking advice people most often chose the advisor that reasoned from their prioritized values while recommending the opposing decision (40% of selections), more than one aligned with both decision and reasoning (31%) or with the decision alone (29%). That preference shifted when the same advisors were considered for new tasks with greater autonomy, and opt-out rates rose from 12% when consulting to 38% when letting an advisor act for them. The paper's central claim is that decision alignment and reasoning alignment shape advisor preferences differently across roles, so that agreeing with a user's values does not by itself guarantee permission to delegate.","feed_headline":"People prefer AI advice matching their values, not their call","feed_subtitle":"Seeking advice, users favored an advisor reasoning from their values even against their lean — until it had to act for them.","key_machinery":"The central object is the contrast between decision alignment and reasoning alignment, operationalized through three advisor conditions crossed over eight everyday moral dilemmas chosen so their two options pull on opposite poles of the same Schwartz value axis. Each participant's PVQ-21 scores determine which option their values favor and which value they prioritize, so the three advisors can be constructed per participant and per dilemma. The design's work is to isolate whether people respond to what an advisor recommends or to the value-based reasons it gives, and to test whether that response survives when the advisor takes on roles granting more autonomy — consulting, drafting a message for review, or acting on the user's behalf.","core_discovery":"The paper distinguishes an AI advisor's decision (the option it recommends) from its reasoning (the values it invokes to justify that decision), and builds three advisors for each participant from their PVQ-21 value profile: one aligned with both the value-favored decision and the value-based reasoning (FullyAligned), one aligned only with the decision (DecAligned), and one aligned only with the reasoning (ReaAligned). When choosing whom to ask about a dilemma, participants most often selected the ReaAligned advisor (40% of selections) — the one that reasoned from their values while recommending the opposing decision — over FullyAligned (31%) and DecAligned (29%), with the ReaAligned advantage over DecAligned at 10.1 points (p = .002) and over FullyAligned at 8.3 points (p = .006). This preference reversed when participants chose the same advisors for new tasks with more autonomy: the within-participant contrast between FullyAligned and ReaAligned shifted by 12.9 percentage points toward FullyAligned (p = .001), and no advisor was consistently preferred within those tasks. Separately, the proportion of choices selecting no advisor rose from 12% for consultation to 38% for letting an advisor act on the participant's behalf. The authors conclude that decision and reasoning alignment shape advisor preferences differently across contexts, and that alignment with a user's values does not necessarily imply willingness to delegate.","pith_inferences":["The preference for reason-aligned, decision-misaligned advice may generalize beyond moral dilemmas to any domain where users retain final control and want to stress-test their own leanings, suggesting a default expectation of 'deliberative complementarity' rather than decision congruence for advisory AI.","The rising opt-out with increasing autonomy hints that users treat delegation as a distinct decision governed by accountability and outcome stakes, not merely by value similarity; a testable extension would hold alignment constant while varying who bears responsibility for outcomes.","Because the value-favored decision is inferred from questionnaire scores rather than stated intentions, a natural robustness check is whether the 40% ReaAligned preference and the 12.9-point shift toward FullyAligned survive when alignment is defined against participants' self-reported preferences on each dilemma."],"forward_implications":["Evaluating personalized AI alignment must separate the decision an advisor recommends from the values in its reasoning, since these dimensions affect preferences independently and differently across roles.","An AI advisor can be preferred for advice even when it recommends against the option a user's values favor, provided its reasoning is grounded in those values — a devil's-advocate role participants seemed to welcome when reading advice.","Willingness to delegate is not guaranteed by value alignment: participants opted out more as advisor autonomy grew, from 12% when consulting to 38% when letting an advisor act on their behalf.","Agreement with an advisor's judgment is an inconclusive signal of alignment, since the same recommendation can be supported by values the user does not prioritize.","Alignment evaluations should be role-sensitive, distinguishing whether a system is meant to support deliberation, produce drafts for review, or act on a user's behalf."],"supporting_citations":[{"why":"Schwartz's refined theory of basic values and the PVQ-21 questionnaire; the value framework that defines what alignment means for each participant.","marker":"[56]"},{"why":"MoReBench; supplies the pool of 200 everyday moral dilemmas and expert-written rubrics from which the eight stimuli were selected.","marker":"[18]"},{"why":"DailyDilemmas; the original source of the everyday dilemma scenarios used within MoReBench.","marker":"[17]"},{"why":"The RLHF preference-alignment paradigm the paper contrasts with value-based reasoning alignment in motivating the study.","marker":"[54]"},{"why":"Prior evidence that shared values foster trust in human-agent interaction, which the study extends by separating decision from reasoning alignment.","marker":"[50]"},{"why":"The advice-taking literature on seeking alternative perspectives, used to interpret the preference for the reason-aligned advisor.","marker":"[10]"},{"why":"The tension between aligning with user preferences and providing complementary assistance, which frames the interpretation of the ReaAligned preference.","marker":"[1]"}],"fun_headline_variants":[],"cache_read_input_tokens":0,"weakest_assumption_plain":"The study assumes that the PVQ-21 value questionnaire and the mapping of the eight dilemmas to Schwarz value poles correctly identify, for each participant and dilemma, which decision their values favor — so that the three advisor conditions are genuinely aligned or misaligned as labelled; the paper acknowledges that people's choices can reflect multiple values and contextual considerations the questionnaire does not capture.","fun_headline_variants_meta":{"raw":{"variants":[]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.00011,"raw_usage":{"cost":0.000110474,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000110474,"upstream_inference_prompt_cost":0.00011011,"upstream_inference_completions_cost":3.64e-07},"total_tokens":1212,"prompt_tokens":1210,"completion_tokens":2,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":0}},"tokens_in":1210,"tokens_out":2,"cost_source":"openrouter_usage","duration_ms":2044,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":0,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T05:37:36.377568+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"A replication in which participants state their own value-favored decision for each dilemma, and advisors are built from those stated preferences rather than from PVQ-21 inference, checking whether the 40% preference for the reason-aligned, decision-misaligned advisor and the 12.9-point shift toward the fully aligned advisor when delegating still appear.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Schwartz's refined theory of basic values and the PVQ-21 questionnaire; the value framework that defines what alignment means for each participant."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MoReBench; supplies the pool of 200 everyday moral dilemmas and expert-written rubrics from which the eight stimuli were selected."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DailyDilemmas; the original source of the everyday dilemma scenarios used within MoReBench."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The RLHF preference-alignment paradigm the paper contrasts with value-based reasoning alignment in motivating the study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior evidence that shared values foster trust in human-agent interaction, which the study extends by separating decision from reasoning alignment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The advice-taking literature on seeking alternative perspectives, used to interpret the preference for the reason-aligned advisor."},{"cited_title":"Proceedings of the AAAI Conference on Artificial Intelligence , author=","cited_arxiv_id":null,"evidence_quote":"The tension between aligning with user preferences and providing complementary assistance, which frames the interpretation of the ReaAligned preference."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}