{"id":"06975de2-b5b4-4d5b-b8e1-650ae0b51086","arxiv_id":"1908.07572","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A position paper proposing that HRI adopt CHI's research through design methods so designers can explore what robots should be, not just how to build them.","lead":"This workshop paper argues that human-robot interaction research should adopt research through design, an exploratory design method established in human-computer interaction. It claims this would let HRI researchers ask what robots are worth designing before perfecting how to build them.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on unverified transfer of CHI acceptance norms to HRI; Section 4's pathway (publish in CHI, then RtD enters HRI naturally) is asserted, not demonstrated.","rationale":"The reader's verdict is CONDITIONAL and identifies the transfer of CHI acceptance norms to HRI as the weakest assumption. I agree. The paper's argument is clear and its examples are genuine, and Section 3's candid acknowledgment of RtD's challenges is a strength. However, the central benefit claim is normative and predictive: it asserts that HRI will benefit and that the benefit is achievable through a specific community pathway. The paper offers no data on HRI review norms, no pilot evidence that CHI publications change HRI acceptance, and no mechanism beyond 'joining forces.' Since this is a workshop position paper, the lack of empirical support is understandable, but it does make the claim conditional. The correct verdict remains CONDITIONAL: the proposal is reasonable as a hypothesis, but the causal community-level claim needs evidence or explicit reframing. No technical error was found, and the critique is directed at the argument, not the authors.","tokens_in":3891,"tokens_out":3678,"duration_ms":548090,"concrete_test":"Perform a longitudinal citation-context analysis: collect RtD/speculative design papers on HRI topics published in CHI/DIS (e.g., Luria et al. 2019, Auger 2014) and code every subsequent citation made in HRI-venue papers (HRI, JHR, IJSR) over a 5-year window as (a) methodological precedent for accepting exploratory design contributions, or (b) background/related work. Then check whether any new HRI-venue paper is accepted whose primary contribution is framed as RtD or exploratory design. If no such paper appears and citations are only background, Section 4's 'enter HRI more naturally' pathway is falsified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that HRI would benefit from adopting RtD and that this would expand the community's accepted design boundaries—rests on Section 4's premise that publishing RtD on HRI topics in CHI will create a group of researchers and 'allow RtD to enter HRI more naturally.' This is a claim about HRI's reviewing and publication culture. The paper itself, in Section 3, concedes that RtD is 'not structured or controlled' and that reviewing it from a 'user study' perspective would reveal 'plenty of methodology gaps,' yet it offers no evidence that HRI reviewers will change their criteria. The CHI examples (Whisper, Drift Table) show that RtD can be accepted in CHI after years of effort; they do not show that the acceptance pathway transfers to HRI, whose design research has historically emphasized task performance and controlled evaluation. If HRI venues continue to require internal validity, then even RtD work published in CHI will remain outside HRI's accepted contribution types, and the claimed benefit to the HRI community is unrealized. The authors themselves note they 'would like to examine ... how to encourage HRI researchers to accept RtD,' which is an admission that the mechanism is untested. The claim is not internally inconsistent, but it is a load-bearing empirical prediction with no supporting evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This workshop paper argues that the HRI community should adopt Research through Design (RtD) methods from CHI in order to expand the accepted forms of design contribution beyond task-oriented evaluation. It characterizes RtD as exploratory design that asks \"what is the right thing to design,\" gives several CHI examples (Whisper, the Drift Table, Odom et al.'s virtual-possessions work), mentions Auger's speculative design robots as a related HRI-adjacent example, and reports the authors' own use of Speed Dating with User Enactments to explore robot re-embodiment. The paper then identifies the main barrier—RtD is unstructured and does not satisfy standard \"user study\" review criteria—and concludes with a proposal that HRI designers publish RtD work on HRI topics in CHI venues, creating a cross-community group that will allow RtD to enter HRI more naturally.","tokens_in":4113,"tokens_out":2974,"duration_ms":30476,"significance":"The paper is a position statement rather than an empirical study, and as such its contribution is to articulate a plausible gap between CHI and HRI design-research norms and to propose a bridging strategy. The argument is internally coherent, and the illustrative examples are real and clearly relevant to the CHI side of the discussion. The authors also deserve credit for explicitly acknowledging the central difficulty—that RtD is unstructured by definition and will look methodologically weak when reviewed as a user study. If the central normative claim is correct, the paper points to a concrete shift in how HRI might evaluate design research, moving beyond task performance toward exploratory knowledge about robots' roles and contexts. The main weakness is that the empirical premises about HRI's current design publications and about how CHI publishing norms might transfer to HRI are asserted rather than supported; the authors themselves concede the mechanism is still to be examined.","major_comments":[{"comment":"The central recommendation—that publishing RtD on HRI topics in CHI will \"create a growing group of researchers... and allow RtD to enter HRI more naturally\"—is load-bearing but asserted without evidence. Section 3 explicitly concedes that the authors \"would like to examine... how to encourage HRI researchers to accept RtD,\" which admits that the transfer mechanism is untested. Because the practical value of the paper's proposal depends entirely on this cross-community influence, the manuscript should either reframe this as an explicit hypothesis with a description of how it could be evaluated, or supply evidence that CHI publication patterns have previously shifted acceptance criteria in HRI.","section":"Section 4, final paragraph"},{"comment":"The premise that \"the majority of design papers published in HRI create and test a particular robot task or function\" is stated without any citation or systematic review of the HRI literature. This empirical claim is the basis for identifying the gap that the paper asks to close. As written, the claim is unsupported, and the argumentative weight is too large for an anecdotal generalization. The authors should at least cite an existing review of HRI design research or narrowly qualify the claim as their personal observation.","section":"Section 1, third paragraph"},{"comment":"The examples (Whisper, the Drift Table, Odom et al., Auger) establish that RtD and speculative design have produced valued knowledge in CHI and adjacent communities, but they do not demonstrate that HRI's current design practices are inferior or that HRI would benefit from adopting RtD. To support \"the HRI community could greatly benefit,\" the paper would need a structured analysis of HRI design contributions or an explicit labeling of the benefit claim as a hypothesis to be debated in the workshop. Without one of these, the gap between the evidence and the conclusion remains open.","section":"Section 2"}],"minor_comments":[{"comment":"The sentence \"With some notable exceptions, the majority of design papers published in HRI create and test a particular robot task or function\" has a subject-verb agreement issue: \"papers... create and test\" is correct, but the phrase \"in HRI creates\" is grammatically awkward; revise for clarity.","section":"Section 1"},{"comment":"The DOI in the footer is a placeholder (\"https://doi.org/10.1145/nnnnnnn.nnnnnnn\") and should be replaced or removed before any archival submission.","section":"Page 1, header/DOI"},{"comment":"Reference [14] (Overgoor and Funk, \"IdleBot: exploring the design of serendipitous artifacts\") lists the venue as the ACM SIGACCESS Conference on Computers and Accessibility, which appears inconsistent with the topic of serendipitous artifacts in domestic settings; please verify the correct publication venue and update the entry.","section":"References, [14]"},{"comment":"The authors' own Speed Dating study is used as an illustrative case, but its placement in the argument should make explicit that it is an example of method use rather than evidence that HRI reviewers would accept such work; a single sentence clarifying this would avoid the appearance of circular support.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"This is clearly a short workshop position paper, so the major comments should not be read as demanding a full empirical evaluation in this venue. However, if the authors intend to develop the manuscript into a full archival paper, the unsupported claims about HRI publication patterns and the CHI-to-HRI influence pathway will need substantially more grounding. The paper is well within the scope of the HRI community's ongoing discussion of design methods, and the authors' reflection on their own RtD process is a useful first-person data point."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a short workshop paper arguing that HRI should adopt Research through Design (RtD) as a recognized contribution type. It's not a new empirical result, and its central causal claim—that publishing RtD on HRI topics in CHI will \"allow RtD to enter HRI more naturally\"—is asserted, not shown. But as an advocacy piece, it does its job well.\n\nThe authors clearly articulate the gap between CHI's exploratory design tradition and HRI's task-performance orientation. The examples are well chosen: Whisper illustrates design fixation, Drift Table shows RtD generating knowledge about ludic interaction, and their own re-embodiment work demonstrates a concrete HRI application of speed dating with user enactments. They're also upfront about RtD's limitations, admitting in Section 3 that the method is unstructured and would look weak if reviewed as a user study. That honesty is a real strength.\n\nThe soft spot is exactly where the stress-test note lands: Section 4. The claim that publishing in CHI will create a bridge that lets RtD enter HRI \"more naturally\" is a community-sociology prediction with no supporting evidence. The paper itself concedes the mechanism is untested when it says it \"would like to examine how to encourage HRI researchers to accept RtD.\" That's an admission the load-bearing step is still a hypothesis. The anecdotal evidence even cuts both ways: Auger's speculative design work was published in the Journal of HRI, suggesting some openness already exists, but the authors don't analyze that case or extract lessons from it.\n\nThat said, this is a workshop position paper, not a journal article. The tight format excuses some of the hand-waving, and the central proposal—that RtD can help HRI ask what the right thing to design is—is genuinely worth discussing. The paper would be stronger if it framed the adoption claim as a hypothesis and sketched how to test it, rather than presenting it as an expected outcome.\n\nAll this makes it a fine workshop submission, a reasonable reading-group item, and a legitimate position piece for the field. I wouldn't cite it as evidence that the adoption pathway works, but I'd cite it as a clear statement of the proposal. It deserves a serious referee, not a desk reject.","headline":"A clear, honest position paper advocating Research through Design for HRI; the proposal is reasonable, but the key claim about how RtD will get adopted is asserted, not demonstrated.","tokens_in":4653,"tokens_out":2137,"would_cite":false,"duration_ms":222480,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Human-robot interaction should adopt Research through Design so the field can ask what is worth designing, not just how to design it.","keywords":["research through design","human-robot interaction","exploratory design","design methods","user enactments","CHI","HRI","speculative design"],"falsifier":"A review of HRI conference submissions over the next several years showing that exploratory RtD papers are consistently rejected for lacking controlled user studies would contradict the paper's expectation that RtD can enter HRI through CHI.","tokens_in":3653,"feed_emoji":"🤖","tokens_out":5355,"duration_ms":43418,"temperature":0.7,"pith_summary":"This workshop paper argues that human-robot interaction (HRI) research should adopt Research through Design (RtD), an exploratory design method already established in the CHI community. RtD asks 'what is the right thing to design' rather than 'how to make a chosen thing work better.' The authors contend that most HRI design work tests a particular robot function, and that adding RtD would let the community question underlying assumptions, avoid fixation, and discover when robots are worth designing at all. The paper draws on CHI examples and the authors' own RtD study of robot re-embodiment to show how exploratory methods can explore design spaces that lab testing cannot reach.","feed_headline":"HRI should ask what to design, not just how to design it","feed_subtitle":"Exploratory design from CHI could expand robot research beyond task performance.","key_machinery":"The central mechanism is Research through Design (RtD), defined as inquiry in which designers generate new knowledge through the process of designing and making. A key operational tool is Speed Dating with User Enactments, where participants experience brief, 'brought-to-life' scenarios using actors, prototypes, and props, then reflect on their needs and values. This method allows exploration of interactions that do not yet exist without requiring controlled experiments or internal validity. The paper also invokes speculative design as a related exploratory approach for questioning assumptions about robots in domestic life.","core_discovery":"The central claim is that the HRI community can and should accept exploratory design research, specifically Research through Design, as a valid contribution alongside controlled user studies. On the paper's account, RtD generates knowledge through the process of designing and making artifacts, allowing researchers to investigate 'wicked' problems where the design space is open. In HRI, where design papers typically create and test a robot task, adopting RtD would expand the boundaries of the field to include questions about when designing robots is the right thing to do. The paper supports this by presenting examples such as the Drift Table and speculative robots, and by describing the authors' use of Speed Dating with User Enactments to study re-embodiment of social presence across robot bodies.","pith_inferences":["If RtD becomes accepted, HRI might also embrace other critical design practices like speculative design and value-sensitive design, broadening what counts as a design contribution.","RtD could be used to ask normative questions about robot autonomy and social roles, such as whether certain tasks should be automated at all, which current task-performance evaluation cannot address.","A testable extension: measure whether HRI papers that include RtD-style explorations lead to different research questions or robot concepts than papers that start from a predefined task.","The transfer from CHI to HRI may require explicit review criteria for RtD, since journal and conference reviewers in HRI often expect internal validity."],"forward_implications":["HRI design contributions could include exploratory knowledge about what to design, not only evaluations of existing robot functions.","RtD could help the field avoid design fixation, as seen in the Whisper example, by re-framing assumptions about user behavior.","The authors' re-embodiment study could be shared with the HRI audience once RtD is accepted, addressing questions of when social presence should move between bodies.","Publishing RtD work at CHI could create a cross-community group that gradually introduces RtD into HRI.","HRI designers could identify which RtD methods need adaptation for robot-specific questions."],"supporting_citations":[{"why":"Establishes that design practice itself can produce research knowledge, the foundation of RtD.","marker":"[10]"},{"why":"Defines Research through Design as a method for interaction design research and introduces the 'right thing' framing.","marker":"[15]"},{"why":"Supplies the distinction between 'getting the design right' and 'getting the right design,' central to the paper's argument.","marker":"[3]"},{"why":"Provides the Speed Dating with User Enactments method used in the authors' RtD study.","marker":"[5]"},{"why":"Presents the Drift Table as a canonical RtD example that explores ludic engagement with technology in the home.","marker":"[9]"},{"why":"Demonstrates speculative design in HRI, questioning why robots are not adopted as domestic objects.","marker":"[1]"},{"why":"Introduces IdleBot as an exploratory robot that questions whether and when robots should be engaging.","marker":"[14]"}],"fun_headline_variants":["Bring design exploration from CHI into HRI","Research through Design: a path for HRI to explore","Let HRI ask what to design, not just how","Beyond tasks: design research for robots","Embrace exploratory design in HRI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the premise that the HRI community will eventually accept exploratory design work as a legitimate contribution, even though the paper itself notes that the method is unstructured and lacks internal validity.","fun_headline_variants_meta":{"raw":{"variants":["Bring design exploration from CHI into HRI","Research through Design: a path for HRI to explore","Let HRI ask what to design, not just how","Beyond tasks: design research for robots","Embrace exploratory design in HRI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1577,"prompt_tokens":793,"completion_tokens":784,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":409,"completion_tokens_details":{"reasoning_tokens":713}},"tokens_in":409,"tokens_out":784,"duration_ms":7691,"temperature":1.0,"reasoning_tokens":713,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:03:47.540409+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A review of HRI conference submissions over the next several years showing that exploratory RtD papers are consistently rejected for lacking controlled user studies would contradict the paper's expectation that RtD can enter HRI through CHI.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that design practice itself can produce research knowledge, the foundation of RtD."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Research through Design as a method for interaction design research and introduces the 'right thing' framing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the distinction between 'getting the design right' and 'getting the right design,' central to the paper's argument."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Speed Dating with User Enactments method used in the authors' RtD study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Presents the Drift Table as a canonical RtD example that explores ludic engagement with technology in the home."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates speculative design in HRI, questioning why robots are not adopted as domestic objects."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces IdleBot as an exploratory robot that questions whether and when robots should be engaging."}],"review_version":1}