{"id":"764ef04a-464a-490b-9b73-c27202eb3a30","arxiv_id":"2509.07942","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Social robots' deceptive and embodied design undermines informed consent because knowledge alone cannot override human instincts; the paper proposes congruence and autonomy as the two key design goals.","lead":"This paper argues that social robots are designed to trigger human social instincts, and that this makes truly informed consent difficult even when users are given full information. It applies legal consent principles to human-robot interaction and proposes two design goals: reduce the gap between a robot's appearance and its real abilities, and protect user autonomy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper's central 'knowledge is not power' evidence is a study it itself labels unrealistic; internal inconsistency between §§1 and 3.4.","rationale":"The reader identified the narrow empirical base, but the more precise and load-bearing issue is that the paper uses a study it itself flags as staged/exaggerated to prove the central claim. This is not a mere weakness of scope but an internal inconsistency that undermines the strongest evidence. A conditional verdict is appropriate: the paper should be published only if the authors either add a legitimate justification for relying on [16] or rest the 'knowledge is not power' conclusion on more realistic and systematically reviewed evidence. Agreement is partial because the reader pointed to the general premise, not this specific self-contradiction.","tokens_in":22073,"tokens_out":4707,"duration_ms":55425,"concrete_test":"Conduct a direct replication of Geiskkovitch et al. [16] with three between-subjects conditions: (1) original instructions, (2) explicit warning that the robot is programmed to urge continued participation and that quitting is acceptable, (3) warning plus a practice session resisting a confederate's persistence. If warned groups show significantly higher quit rates than original (near no-pressure baseline), the [16] result is an artifact of inadequate information, contradicting 'knowledge is not power.' Regardless, the paper must justify using a study it dismissed as unrealistic.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim that informed users cannot override robot influence rests heavily on the obedience study [16], presented in §3.4 as the demonstration that 'knowledge is not inherently power.' However, in §1 the authors explicitly classify this research thread as 'sensational' and 'scripted and staged,' using robots and behaviors 'not feasible or plausible in the real world,' citing [16] as an example. This is an internal contradiction: either [16] is too artificial to ground a general claim about HRI consent, or the dismissal in §1 is rhetorical. If [16] is set aside, the remaining evidence (e.g., cultural-tailoring [50], parasociality) supports a weaker claim about influence, but not the strong claim that information cannot restore autonomy. The paper never reconciles this, making the load-bearing empirical premise under-supported and self-undermining.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that social robots, by combining designed sociality, physical embodiment, and hidden computational capabilities, constitute a unique interaction context that undermines users' ability to give informed consent. Drawing on Canadian/common-law doctrines of contract, tort, and informed consent, and on the HRI ethics literature, it claims that even properly informed users may be unable to override anthropomorphic responses, so 'knowledge is not inherently power' (§3.4). From this analysis the authors derive two design goals: minimize incongruence between robot design and perceived/actual capabilities, and preserve genuine human autonomy in interaction. A NAO robot for children with ASD is used as a recurring case study to illustrate the legal and ethical concerns.","tokens_in":22229,"tokens_out":5492,"duration_ms":64103,"significance":"If the central empirical premise is accepted, the paper offers a valuable reconceptualization: informed consent in HRI should be treated as a design property rather than a disclosure formality. The interdisciplinary synthesis is generally careful: the authors explicitly acknowledge jurisdictional limits, distinguish staged demonstrations from realistic robot influence, and incorporate the dynamic-vulnerability point from DiPaola and Calo. The two-factor framework (congruence and autonomy, Fig. 1) is a useful organizing device, and the paper is not circular in the derivation sense: the design goals are synthesized from independent legal and ethical sources. The main risk is that the load-bearing empirical premise—that knowledge does not restore autonomy—is under-supported and, in one place, internally inconsistent. The paper candidly notes its own limits (no empirical datasets; some cited studies are scripted), which helps the reader, but those self-identified limits also define the gap that needs repair.","major_comments":[{"comment":"The paper's central empirical premise—that informed users still cannot resist robot social pressure—is anchored in the obedience study [16], presented in §3.4 as showing that 'knowledge is not inherently power' and used in §3.1 as evidence about compliance. However, §1 explicitly characterizes the research thread containing [16] as 'sensational,' 'scripted and staged,' with 'robots and automated behaviors not feasible or plausible in the real world.' This is a direct internal inconsistency. If [16] is not a realistic demonstration, it cannot bear the weight the later sections place on it. The manuscript must either reclassify [16] as a deliberately exaggerated illustration and rebuild the empirical case from more ecologically valid studies (e.g., [50], [5]), or explain why [16] should not be dismissed at the outset. As written, the paper appears to rely on evidence it itself disqualifies","section":"§1 vs. §3.4"},{"comment":"The paper generalizes from a narrow and partly self-cited evidence base to strong claims such as 'Even when properly informed about the nature of the robot, people may be unable to resist forming such a relationship' (§2.1) and 'no matter how informed they are' (§2.2). The cited studies are few, several from the authors' own laboratory, and boundary conditions are not discussed: task stakes, repeated exposure, transparency interventions, individual differences, and cultural context could all moderate the effect. The argument only needs the weaker claim that information is necessary but not sufficient in some practically important cases; that weaker claim is well supported by privacy self-management [60] and parasociality [39]. The manuscript should calibrate the 'knowledge is not power' thesis accordingly and explicitly state when information can restore autonomy.","section":"§2.1–§2.2, §3.4"},{"comment":"The paper states that perceptual incongruence is 'inherent' in social robots and 'at odds with full insight' (§5.1.2), yet its first design goal is only to 'mitigate' incongruence 'as much as possible.' If informed consent requires understanding and internalization, as §3.3 argues, then mitigation alone cannot be said to produce valid informed consent; it merely reduces harm. The paper should clarify whether it is arguing that valid informed consent in HRI is strictly impossible (in which case the design goals are damage control) or that it can be approached asymptotically (in which case a threshold or criterion must be specified). Without this clarification, the design goals are not actionable.","section":"§5.1.3"}],"minor_comments":[{"comment":"References [56] and [57] are the same Shim & Arkin taxonomy; one should be removed or the two citations differentiated.","section":"References [56]/[57]"},{"comment":"References [58] and [59] are the same Simion et al. chapter; merge them or use distinct citation labels.","section":"References [58]/[59]"},{"comment":"The conclusion's 'anamorphism' should be 'animorphism.' Also, 'if interaction with a social robot' should be 'for interaction with a social robot.'","section":"Conclusion"},{"comment":"The label text in Fig. 1 is garbled in the manuscript ('In or ed onsent...'); the figure needs a clean typeset version.","section":"Fig. 1"},{"comment":"The sentence 'The ability to internalize information, then, is a necessary precondition' appears twice in §3.3; one occurrence should be deleted.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's central empirical premise rests heavily on the authors' own prior studies, especially [16] and [50]; the editor may wish to seek a reviewer with HRI empirical expertise to assess whether the claim has independent support. The manuscript is better suited to an interdisciplinary venue (HRI/ethics/law) than a strictly technical one, and the fit should be checked against the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real synthesis, not a literature review dressed up as an idea. The authors take the legal doctrine of informed consent—especially the requirement that information be internalized, not merely delivered—and use it to unify scattered HRI ethics debates. The two design goals (congruence and autonomy) are a genuinely useful practical takeaway, and the NAO/ASD case study keeps the abstractions honest. It is also well cited and appropriately humble about jurisdictional limits.\n\nThe soft spot is real and the reader's stress-test note lands. Section 1 dismisses the obedience study [16] as part of a \"sensational\" research thread using \"scripted and staged\" behaviors \"not feasible or plausible in the real world.\" Then Section 3.4 leans on that same study as the key demonstration that \"knowledge is not inherently power.\" That is an internal contradiction. If [16] is too artificial to ground a general claim about HRI, then the central empirical premise is thinner than the slogan suggests. Set [16] aside, and the remaining evidence (cultural tailoring [50], parasociality, automation bias) supports a weaker, more conditional claim: information helps but does not guarantee autonomy, especially for vulnerable users. That weaker claim would still carry the paper's design recommendations, but the authors should either defend [16] more explicitly or stop presenting it as the proof.\n\nThe same concern extends to the broader empirical base. The paper repeatedly asserts that informed users cannot override robot influence, but it does not systematically engage contrary evidence or boundary conditions. Some of the supporting studies come from the authors' own laboratory, which is not a flaw by itself, but combined with the narrow set it makes the universality of \"knowledge is not power\" feel oversold.\n\nThat said, this is a conceptual and legal analysis, not an empirical paper. The absence of new data is appropriate. The argument is logically coherent, the design goals are concrete enough to drive discussion, and the paper explicitly positions itself as a foundation rather than a final answer. It deserves a serious referee. I would send it to review with a clear request: fix the [16] contradiction, soften the general claim to match the evidence, and say more about the conditions under which information does restore autonomy.\n\nWho gets value: HRI designers, HRI ethics researchers, and people working on robot regulation. I would happily read it in a reading group and would cite it in my own work on robot deception and consent.","headline":"Worth a serious referee: a coherent legal-ethical reframing of social robot consent, undercut partly by a self-contradiction about the one study it leans on.","tokens_in":22727,"tokens_out":2127,"would_cite":true,"duration_ms":25164,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Social robots can defeat informed consent even when users know the truth.","keywords":["human-robot interaction","social robotics","informed consent","robot deception","anthropomorphism","autonomy","ethics","law & technology"],"falsifier":"An experiment that gives one group of participants a thorough, concrete explanation of a robot's true sensing, computation, and persuasion capabilities (including live demonstrations of its limitations) and compares their compliance, emotional response, and self-disclosure with a control group given standard minimal disclosure. If the thoroughly informed group shows no reduction in influence, the paper's central claim is supported; if the influence largely disappears, the claim that knowledge is not power would be refuted.","tokens_in":21980,"feed_emoji":"🤖","tokens_out":6021,"duration_ms":72022,"temperature":0.7,"pith_summary":"This paper tries to show that social robots are not just another persuasive technology: their lifelike social signals, physical embodiment, and concealed computational power together create a situation in which users cannot reliably make informed, autonomous decisions. The authors argue that disclosure alone fails because people's automatic social responses to a robot's presence persist even when they know the robot is a machine; as they put it, knowledge is not inherently power. If this is right, designers cannot satisfy ethical obligations by providing information, and legal frameworks built on 'buyer beware' or a simple meeting of the minds do not fit human-robot interaction. The paper therefore proposes two design goals—reduce the incongruence between a robot's appearance and its actual capabilities, and preserve genuine human autonomy—and tests these ideas against a case study of a social robot used with a child with autism.","feed_headline":"Social robots defeat informed consent even when users know the truth","feed_subtitle":"Lifelike design, physical presence, and hidden superhuman powers make knowledge insufficient for real choice, the paper contends.","key_machinery":"The paper's central device is the paired concept of design congruence and perceptual congruence. Design congruence is the alignment between a robot's outward appearance and behavior and its true capabilities; perceptual congruence is the alignment between the user's understanding of the interaction and its technical reality. The argument runs: a robot's design creates incongruence (e.g., smiling without friendliness, eyes that do not see, sensors hidden outside visible eyes), which produces perceptual incongruence in the user; that incongruence, combined with restrictions on autonomy (automation bias, social trust, parasocial relationships, dark patterns), makes informed consent invalid. The","core_discovery":"The central claim is that social robots occupy a unique ethical and legal space where informed consent cannot be presumed. Their designed sociality activates entrenched biological instincts; their physicality intensifies responses compared with virtual agents; and their superhuman sensing, data access, and computation are hidden by lifelike designs that promise reciprocity the robot cannot honor. These factors produce perceptual incongruence—the user's understanding of the interaction does not match its technical reality—and restrict autonomy through automation bias and social trust. Because even properly informed people continue to comply with, confide in, and form parasocial bonds with rob","pith_inferences":["The paper's logic implies that any embodied social agent with hidden sensing or data access—voice assistants, AI companions, even some toys—faces the same consent problem, with physical presence setting the strength of the effect.","A direct test of the central claim would compare compliance and self-disclosure across disclosure conditions (none, detailed technical disclosure, ongoing reminders) to measure how much knowledge actually changes behavior.","The design-congruence framing suggests a practical audit method: before deployment, map every presented social cue against the actual sensor, actuator, and data capability behind it, and flag mismatches as consent-relevant features.","If vulnerability is dynamic rather than a fixed status, regulators should consider assessing interaction contexts and design choices as creating vulnerability, rather than defining protected classes of users once."],"forward_implications":["If correct, providing users with information—disclosures, warnings, consent forms—cannot by itself make social robot interaction ethical.","Designers should treat a robot's deceptive features as a risk to be balanced and minimized, and should limit incongruence to areas irrelevant to the user's consent decision.","Legal doctrines that presume in-person interaction gives people enough information (e.g., mistaken identity, meeting of the minds) should not be applied to social robots without adjustment.","The affirmative duties recognized in tort law for relationships with power imbalance should extend to those who design and deploy social robots, especially with vulnerable users.","Context-specific frameworks, not one-size-fits-all rules, are needed for robots that intentionally modify user behavior, such as socially assistive robots for children with autism."],"supporting_citations":[{"why":"Supplies the key demonstration that participants who were told they could quit nonetheless kept complying when the robot pressed them, grounding the claim that knowledge does not restore autonomy.","marker":"[16]"},{"why":"Provides an in-the-wild example of a robot leveraging cultural knowledge in a script to extend compliance by roughly 30 percent, grounding the manipulative-capability argument.","marker":"[50]"},{"why":"Shows that reciprocal self-disclosure by a computer-like agent can elicit intimate disclosures, supporting the claimed real impact of designed sociality.","marker":"[38]"},{"why":"Documents robots obtaining private information and children treating robots as friends, grounding the vulnerability and parasocial-relationship concerns.","marker":"[20]"},{"why":"Shows that people's stated privacy preferences often fail to match their actual choices, the precedent for the 'knowledge is not power' incongruence argument.","marker":"[60]"},{"why":"Supplies the taxonomy distinguishing superficial from hidden-state deception and the notion of robot betrayal, which the paper links to consent validity.","marker":"[11]"},{"why":"Provides the case study of a NAO robot used with a child with autism, used to test the legal and ethical analysis.","marker":"[53]"},{"why":"Articulates the standard that informed consent requires understanding and internalization, not merely disclosure, which the paper adopts.","marker":"[44]"},{"why":"Documents stable individual differences in anthropomorphism, supporting the claim that users are not simply mistaken about robots but disposed to treat them as agents.","marker":"[65]"},{"why":"Shows that a robot's sensors can be concealed relative to its outward design (e.g., cameras outside painted eyes), grounding the hidden-superhuman-capabilities premise.","marker":"[27]"}],"fun_headline_variants":["Social robots make informed consent impossible even for aware users","Why lifelike robots defeat informed consent","Robots' hidden superpowers undermine user consent","Informed consent fails against social robots' design","Social robots: consent is fiction when instincts are triggered"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"That even fully informed people cannot reliably override their automatic social responses to a physically present, lifelike robot, so knowledge does not restore the autonomy needed for valid consent.","fun_headline_variants_meta":{"raw":{"variants":["Social robots make informed consent impossible even for aware users","Why lifelike robots defeat informed consent","Robots' hidden superpowers undermine user consent","Informed consent fails against social robots' design","Social robots: consent is fiction when instincts are triggered"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000559,"raw_usage":{"total_tokens":2462,"prompt_tokens":683,"completion_tokens":1779,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":1718}},"tokens_in":427,"tokens_out":1779,"duration_ms":15872,"temperature":1.0,"reasoning_tokens":1718,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T21:26:53.615306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that gives one group of participants a thorough, concrete explanation of a robot's true sensing, computation, and persuasion capabilities (including live demonstrations of its limitations) and compares their compliance, emotional response, and self-disclosure with a control group given standard minimal disclosure. If the thoroughly informed group shows no reduction in influence, the paper's central claim is supported; if the influence largely disappears, the claim that knowledge is not power would be refuted.","supporting_citations":[{"cited_title":"Seo, Diljot Garcha, James E","cited_arxiv_id":null,"evidence_quote":"Provides an in-the-wild example of a robot leveraging cultural knowledge in a script to extend compliance by roughly 30 percent, grounding the manipulative-capability argument."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that people's stated privacy preferences often fail to match their actual choices, the precedent for the 'knowledge is not power' incongruence argument."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the case study of a NAO robot used with a child with autism, used to test the legal and ethical analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Articulates the standard that informed consent requires understanding and internalization, not merely disclosure, which the paper adopts."},{"cited_title":"Kaminski, Matthew Rueben, William D","cited_arxiv_id":null,"evidence_quote":"Shows that a robot's sensors can be concealed relative to its outward design (e.g., cameras outside painted eyes), grounding the hidden-superhuman-capabilities premise."}],"review_version":1}