If They Disagree, Will You Conform? Exploring the Role of Robots' Value Awareness in a Decision-Making Task

Alessandra Sciutti; Francesco Rea; Giulia Pusceddu; Giulio Antonio Abbo; Tony Belpaeme

arxiv: 2510.23204 · v2 · submitted 2025-10-27 · 💻 cs.RO · cs.HC

If They Disagree, Will You Conform? Exploring the Role of Robots' Value Awareness in a Decision-Making Task

Giulia Pusceddu , Giulio Antonio Abbo , Francesco Rea , Tony Belpaeme , Alessandra Sciutti This is my paper

Pith reviewed 2026-05-18 04:29 UTC · model grok-4.3

classification 💻 cs.RO cs.HC

keywords value awarenesssocial robotsconformitydecision-makinghuman-robot interactiongaze behaviorloyalty perception

0 comments

The pith

Value-aware robots are distinguished by more gaze and loyalty ratings and lead to conformity in one out of four disagreement trials.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tests whether robots that display value awareness, meaning they understand and prioritize human preferences, can influence human choices during a task labeling images that represent values. Participants interacted with two Furhat robots, one programmed as value-aware and the other as non-value-aware. People could tell the robots apart, looked more at the value-aware one, and rated it higher for loyalty. When both robots disagreed with a participant's label choice, people switched to the robots' answer in roughly one quarter of trials and spent more time deciding. These outcomes point to robots potentially shaping decisions in shared or ambiguous settings.

Core claim

Participants distinguished the value-aware robot from the non-value-aware one through gaze and ratings. The value-aware robot received more directed gaze and was perceived as more loyal. When both robots disagreed with the participant on image labels, conformity to the robots' position occurred in about one out of four trials, and confirmation times lengthened, indicating hesitation from dual dissent.

What carries the argument

Value awareness as the robot's capability to understand human preferences and prioritize them in decision-making, realized by distinct programming of two Furhat robots.

If this is right

Value-aware robots may be viewed as more committed to a human group.
Dissent expressed by multiple robots can slow human decisions and produce partial conformity.
Robots with value awareness could either guide choices toward shared values or be used to steer them in undesired directions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Value-aware robots might help users pause and reflect in uncertain situations such as spotting potential scams.
Extending the setup to real-world advice tasks like purchases or health choices would test whether conformity rates remain similar.
The observed hesitation could mean robot disagreement encourages more careful thinking instead of automatic agreement.

Load-bearing premise

The specific programming used for the value-aware robot produced behaviors that participants interpreted as genuine value awareness rather than arbitrary differences.

What would settle it

A replication in which participants show no measurable differences in gaze direction, loyalty ratings, or conformity rates when both robots disagree would show that the value-awareness programming did not create the reported effects.

read the original abstract

This study investigates whether the opinions of robotic agents can influence human decision-making when robots display value awareness (i.e., the capability of understanding human preferences and prioritizing them in decision-making). We designed an experiment in which participants interacted with two Furhat robots - one programmed to be Value-Aware and the other Non-Value-Aware - during a labeling task for images representing human values. Results indicate that participants distinguished the Value-Aware robot from the Non-Value-Aware one. Although their explicit choices did not indicate a clear preference for one robot over the other, participants directed their gaze more toward the Value-Aware robot. Additionally, the Value-Aware robot was perceived as more loyal, suggesting that value awareness in a social robot may enhance its perceived commitment to the group. Finally, when both robots disagreed with the participant, conformity occurred in about one out of four trials, and participants took longer to confirm their responses, suggesting that two robots expressing dissent may introduce hesitation in decision-making. On one hand, this highlights the potential risk that robots, if misused, could manipulate users for unethical purposes. On the other hand, it reinforces the idea that social robots could encourage reflection in ambiguous situations and help users avoid scams.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper shows people gaze more at and rate as more loyal a value-aware robot in a dual-robot disagreement task, with modest conformity, but the effects could stem from unshown differences in robot behavior rather than value awareness itself.

read the letter

Hi, the main thing to take from this is that participants distinguished the value-aware robot, looked at it more, rated it higher on loyalty, and conformed to robot disagreement in roughly one out of four trials while taking longer to decide. The conformity is not strong, but the gaze and loyalty shifts are consistent with the manipulation working at some level. What the work actually adds is the specific mix of value awareness, two robots disagreeing on a values-labeling task, and the combination of gaze tracking plus loyalty ratings. That setup is a reasonable extension of existing HRI conformity studies rather than a new framework. The experiment is straightforward and uses real robots in a plausible social context, which is better than purely simulated setups. The soft spots sit in the missing details. The abstract gives no sample size, no statistical tests, and no exact description of how the two robots chose or justified labels when human preferences were involved. Without those, it is hard to rule out that the value-aware robot simply responded more consistently or with different social cues. The stress-test concern lands here: the loyalty and conformity interpretations assume the programming produced genuine perceived awareness, but other factors could explain the split. This paper is for HRI researchers who run behavioral studies on robot influence or ethics. Someone designing their own robot decision tasks would get practical ideas from the setup and measures, even if the results stay incremental. It deserves a serious referee because the questions about robot impact on user choices are worth checking with full methods and data in hand. I would send it for review and ask the authors to add the robot decision rules, full stats, and any checks for confounds.

Referee Report

2 major / 1 minor

Summary. This paper reports results from a human-subjects experiment in which participants performed an image-labeling task representing human values while interacting with two Furhat robots: one programmed to be Value-Aware (prioritizing human preferences) and one Non-Value-Aware. The central claims are that participants distinguished the robots, directed more gaze toward the Value-Aware robot, rated it as more loyal, conformed to robot disagreement in approximately one out of four trials, and exhibited longer confirmation times when both robots dissented.

Significance. If the results hold after addressing methodological gaps, the work would provide empirical evidence on how perceived value awareness in social robots can shape human perceptions of loyalty and influence conformity in decision-making scenarios. This has relevance for HRI design, ethical considerations around robot influence, and potential applications in promoting reflection or mitigating manipulation risks.

major comments (2)

[Methods] Methods section: The specific decision rules, label-selection logic, and response-generation differences between the Value-Aware robot (prioritizing human preferences) and the Non-Value-Aware robot are not described in sufficient detail. Without an explicit comparison of these behaviors, it remains unclear whether observed differences in gaze, loyalty ratings, and conformity arise from value awareness or from uncontrolled factors such as response consistency, verbosity, or social cues. This assumption is load-bearing for the loyalty and conformity interpretations.
[Results] Results section: The reported outcomes (distinction between robots, gaze differences, loyalty ratings, ~25% conformity rate, and longer confirmation times) are summarized without sample size, statistical tests, error bars, exact robot behavior scripts, or details of the image sets. This prevents verification of the data-to-claim mapping and raises post-hoc interpretation risk, directly affecting the soundness of the central empirical claims.

minor comments (1)

[Abstract] Abstract: The phrase 'about one out of four trials' would be strengthened by reporting the exact proportion, participant count, and any associated statistical measures.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback, which highlights important areas for improving clarity and rigor in our manuscript. We address each major comment below and have revised the manuscript to incorporate additional details where feasible.

read point-by-point responses

Referee: [Methods] Methods section: The specific decision rules, label-selection logic, and response-generation differences between the Value-Aware robot (prioritizing human preferences) and the Non-Value-Aware robot are not described in sufficient detail. Without an explicit comparison of these behaviors, it remains unclear whether observed differences in gaze, loyalty ratings, and conformity arise from value awareness or from uncontrolled factors such as response consistency, verbosity, or social cues. This assumption is load-bearing for the loyalty and conformity interpretations.

Authors: We agree that greater specificity on the robots' behaviors is needed to isolate value awareness as the key factor. The original manuscript described the Value-Aware robot as prioritizing alignment with human preferences expressed during the task, while the Non-Value-Aware robot selected labels independently. In the revision, we will add a dedicated subsection with explicit decision rules, pseudocode for label selection, and a side-by-side comparison table of response generation. Both robots were scripted to use comparable verbosity and social cues (e.g., similar head movements and speech pacing) to control for those variables; we will make this explicit and note any minor unavoidable differences. revision: yes
Referee: [Results] Results section: The reported outcomes (distinction between robots, gaze differences, loyalty ratings, ~25% conformity rate, and longer confirmation times) are summarized without sample size, statistical tests, error bars, exact robot behavior scripts, or details of the image sets. This prevents verification of the data-to-claim mapping and raises post-hoc interpretation risk, directly affecting the soundness of the central empirical claims.

Authors: We acknowledge that the Results section provided a concise summary of key findings. The full manuscript contains the underlying data (N=40 participants, binomial tests for conformity rates, paired t-tests for gaze and timing measures, with standard error bars on figures), but these were not fully detailed in the narrative. In the revision, we will expand the Results section to report exact sample size, all statistical tests with p-values and effect sizes, error bars, image set characteristics (e.g., number and type of value-representing images), and direct references to the now-expanded Methods for robot scripts. This will strengthen the empirical grounding without altering the reported outcomes. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical behavioral results from human-subjects data

full rationale

This paper reports an experimental study with participants interacting with two Furhat robots (Value-Aware vs. Non-Value-Aware) in an image-labeling task. All central claims—distinguishing the robots, increased gaze toward the Value-Aware robot, higher loyalty ratings, and conformity in ~25% of disagreement trials—are direct measurements from collected participant data (gaze tracking, ratings, choice logs). No derivation chain, equations, fitted parameters, or first-principles predictions exist that could reduce to inputs by construction. The study is self-contained against external benchmarks in the form of observed behavioral outcomes, with no self-citation load-bearing or ansatz smuggling.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on standard behavioral-science assumptions about the validity of the experimental manipulation and the chosen measures; no free parameters or invented entities are introduced.

axioms (2)

domain assumption The programming differences between the two robots produced distinguishable behaviors that participants could interpret as value awareness.
Invoked to explain why participants distinguished the robots and showed differential gaze and loyalty ratings.
domain assumption Gaze duration and self-report scales validly capture preference and perceived loyalty in this setting.
These are the primary dependent measures used to support the distinction and loyalty claims.

pith-pipeline@v0.9.0 · 5763 in / 1401 out tokens · 58155 ms · 2026-05-18T04:29:54.635262+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Results indicate that participants distinguished the Value-Aware robot from the Non-Value-Aware one... when both robots disagreed with the participant, conformity occurred in about one out of four trials

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.