{"id":"4180d375-ddf6-4378-af3c-a8eea00bd637","arxiv_id":"2502.02725","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A two-phase co-design study with 13 older adults produced an initial design space and three principles for on-body, movable robots.","lead":"Researchers ran co-design workshops with 13 older adults to imagine what small robots that move on the body should do and how they should behave. The result is a structured design space with three principles covering social norms, physical comfort, and clear utility.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's core contribution—the two-level design space with anchors and units—is not shown to be derived from the data; the published codebook and methods do not operationalize these constructs.","rationale":"The reader's weakest assumption is external validity: the framework is assumed to be informative for older adults generally despite a small, homogeneous sample and a single design probe. That is a real concern, and Section VIII acknowledges it, so it functions as a known boundary condition rather than a hidden defect. The more fundamental threat is internal: the paper's advertised contribution is a specific relational structure of scopes, factors, anchors, and units, but the reported analysis does not demonstrate that this structure was derived from the data in a checkable way. The codebook contains many content-level categories, yet the distinctive elements of the framework—design anchors, design units, and the Level 1/Level 2 separation—are defined only at a high level and are not traced to specific participant statements or coded excerpts. Without that audit trail, the central claim cannot be independently verified or falsified, even with a perfect sample. This is not a criticism of qualitative methods generally; the two-phase co-design process, the use of a tangible probe, and the concrete application vignettes are genuine strengths. But the leap from workshop material to the abstracted design space is the load-bearing step, and it is currently opaque. Because the paper is transparent about many limitations and the framework is presented as 'initial,' the existing CONDITIONAL verdict remains appropriate. I would make the condition explicit: release the coded data and a derivation audit, or provide an independent coding replication, before the design space is treated as a validated characterization rather than a well-motivated hypothesis.","tokens_in":16547,"tokens_out":5602,"duration_ms":58925,"concrete_test":"Release a de-identified coding matrix mapping every transcript and worksheet excerpt to the codebook categories and to the final design-space elements (scopes, factors, DP1–DP3, and any named design units), then ask two independent qualitative coders who have not seen Fig. 4 to re-code a stratified 30% sample of the excerpts using the published codebook and to redraw the design space. Compute inter-coder agreement on which excerpts support each anchor and unit; require κ ≥ 0.6 and stable DP–factor associations. If the structure does not reproduce, or if no explicit units can be named, the central framework should be reported as a preliminary interpretive proposal rather than an empirical characterization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the design space in Fig. 4 being an empirically grounded characterization, not an interpretive gloss. Section III-D states that after thematic coding, the authors 'mapped interactions between these concepts (units and anchors)' through 'iterative team discussions,' and Section V defines design units only as factors that 'should be considered jointly.' Yet no design unit is ever enumerated, and the Supplementary codebook (Table III) contains no codes for 'anchor,' 'unit,' 'scope,' or 'design principle.' The three DPs are asserted as 'key design principles derived from our workshops' (Section V) without a single coded excerpt or co-occurrence table linking them to participant statements. This is not a sample-size issue: with the same 13 participants, a different research team could reasonably organize the material into a different framework, and nothing in the paper would permit adjudication. The framework is therefore currently unfalsifiable as presented. The limitation is acknowledged in general terms in Section VIII, but the missing piece is not just broader recruitment; it is an audit trail showing which data points generated each scope, factor, anchor, and unit.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a two-phase co-design study with 13 independently living older adults, using the authors' Calico on-body robot as a design probe. In phase 1, exploratory workshops elicited a broad range of imagined applications; in phase 2, application-focused workshops converged on three domains (massage, physical therapy, walking) to develop concrete interaction designs. The authors present two co-designed applications (a fall-risk-mitigating walking sentinel and a gamified physical therapy coach) and then characterize a two-level design space with Level 1 scopes (context, human, application) and Level 2 factors (robot, communication), linked by design anchors and design units. Three design principles are proposed: DP1 (adapt to social norms), DP2 (practical interaction design), and DP3 (clear utility and cohesive identity). The paper also offers reflections on co-designing with older adults and acknowledges demographic and methodological limitations.","tokens_in":16771,"tokens_out":3014,"duration_ms":30434,"significance":"If the proposed framework is accepted, it would provide one of the first structured maps of the interaction design space for on-body, movable robots, a topic with little prior HRI literature. The study's strengths include a clearly described two-phase method, the use of a tangible design probe, the inclusion of participant quotes, and a published codebook. The paper also makes a practical contribution by identifying concrete design factors (e.g., embodiment, topology, autonomy, communication modality and content) and by demonstrating how bodystorming can surface embodied design constraints. The claims are modestly scoped as an 'initial' framework, and the authors are transparent about the small, homogeneous participant sample. However, the central analytical constructs—design anchors and design units—are not shown to follow from the data, and this is the main weakness that limits the framework's current evidentiary status.","major_comments":[{"comment":"The central constructs of the design space are not operationalized in the reported analysis. Section III-D states that, after thematic coding, the authors 'mapped interactions between these concepts (units and anchors) through iterative team discussions,' but the Supplementary codebook (Table III) contains no codes for 'anchor,' 'unit,' or 'design principle,' and Section V introduces these terms only with informal definitions. Because the two-level design space in Fig. 4 is the paper's main contribution, the manuscript must provide an audit trail—for example, representative coded excerpts, co-occurrence tables, or an analytic memo showing which participant data generated each scope, factor, anchor, and unit. Without this, readers cannot distinguish a data-grounded characterization from an interpretive gloss, and the framework cannot be independently adjudicated.","section":"III-D and V"},{"comment":"Design units are asserted but never enumerated. Section V defines design units only as Level 2 factors that 'should be considered jointly,' and the text gives no list of what the design units actually are, nor does Fig. 4 clearly mark them. This matters because Section VI-A motivates open research questions by appealing to 'the interplay within design units' (e.g., 'embodiments suitable for multiple on-body robots'). The manuscript should enumerate the proposed design units explicitly and, for each, provide evidence from the workshops (e.g., the bodystorming example where visual feedback was abandoned for haptic feedback due to posture).","section":"V"},{"comment":"The scope of the claimed design space is broader than the evidence supports. The abstract and Section V present 'a design space' and 'key design principles' for on-body robots for older adults, yet the study draws on 13 participants who are predominantly Caucasian, independently living, and recruited with a single inclusion criterion (age 65+), and it uses a single design probe (Calico) across short workshops. The limitations section acknowledges these points in general terms, but the framing should be tightened: either qualify the contribution as an initial design space for this particular participant group and probe (which is defensible), or add transferability evidence such as comparison with other older-adult subgroups or other on-body robot probes. This is load-bearing because the paper's title and abstract generalize beyond the empirical basis.","section":"VIII and Table I"}],"minor_comments":[{"comment":"The sentence 'We also accounted for participants' enthusiasm, for certain ideas' contains an intrusive comma after 'enthusiasm'; the comma should be removed or the phrase rephrased.","section":"III-B"},{"comment":"The phrase 'designing on-robots to convey more complex information' appears to contain a typo; it should likely read 'designing on-body robots' or 'designing the robot.'","section":"V-E"},{"comment":"The visual guide is dense, and the main text does not explain the graphical conventions (solid vs. dashed lines, box vs. circle shapes). A legend or caption note defining each graphical element would make the figure more self-contained.","section":"Fig. 4 and Supplementary Fig. 1"},{"comment":"The paper uses 'Modality' as a design factor in the text, while Fig. 4 and the Supplementary materials use 'Input Modality' and 'Output Modality' separately. This naming inconsistency should be reconciled.","section":"V-E"},{"comment":"The caption heading 'ORGANIZATION OF PHASE 1 APPLICATIONS ALONG ROBOT TIME ON USER AND CO-PRESENSE CONTEXT DIMENSIONS' contains a typo: 'CO-PRESENSE' should be 'CO-PRESENCE.'","section":"Supplementary Table II"}],"recommendation":"major_revision","confidential_remarks":"The core issue is not the sample size per se but the missing analytic connection between the raw data and the central framework. The authors should be encouraged to add supplementary material that traces each design scope, factor, anchor, and unit to specific participant statements or coded excerpts. This is fixable within the manuscript's scope. I also note that Calico is the authors' own system and that several co-design references are from the same research group; this is not inherently problematic, but it makes the independence of the 'initial design space' framing worth scrutinizing in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honestly, this is a useful paper for the on-body robot and HRI community. The authors ran a careful two-phase co-design with 13 older adults, used Calico as a physical probe, and produced something the field lacked: an initial design space plus three design principles for movable on-body robots. The quotes are vivid and the principles—adapt to social norms, practical interaction design, clear utility and cohesive identity—match the data they show. The methods are described in enough detail to be replicated, and the limitations section is unusually honest.\n\nThe soft spots are real but not fatal. The sample is 13 mostly Caucasian, independently living older adults, all from a single cultural context; one author-built probe; and workshops that are necessarily short. The paper acknowledges this. My bigger concern is the central framework. Section III-D says the authors mapped 'units and anchors' through iterative team discussions, and Section V defines design units only as factors that 'should be considered jointly.' But the supplementary codebook has no codes for anchors, units, scopes, or design principles, and no design unit is ever enumerated. So the two-level design space is presented as a derived result, but a reader cannot verify that derivation from the data provided. That is a transparency gap, not a misconduct. It also means the framework is hard to falsify: with the same transcripts, another team could reasonably organize the material differently. I'd want to see a table linking codes to each scope, factor, anchor, and unit, and at least one explicit example of a design unit.\n\nThat said, this is the kind of work that deserves referee time. The area is young, the question is meaningful, and the study is well-designed apart from the reporting gap. I'd send it out and ask for the audit trail as a major revision condition, not desk reject. If the authors can make the anchor/unit derivation explicit, this becomes a solid reference for HRI and wearable-robot researchers working with older adults.","headline":"Useful co-design study with a real transparency gap: the two-level design space needs an explicit audit trail before I'd rely on it.","tokens_in":17260,"tokens_out":2995,"would_cite":true,"duration_ms":28052,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-phase co-design with 13 older adults yields a two-level design space and three principles for on-body robots that move across the body.","keywords":["on-body robots","wearable robots","co-design","older adults","human-robot interaction","design space","design principles"],"falsifier":"If a co-design study with a more diverse group of older adults—varying in physical and cognitive abilities—systematically produced design themes that cannot be mapped onto the proposed Level 1 scopes or Level 2 factors, the framework's comprehensiveness would be disproven.","tokens_in":16404,"feed_emoji":"🤖","tokens_out":6120,"duration_ms":48569,"temperature":0.7,"pith_summary":"This paper aims to establish a structured design space for on-body robots—small wearable robots that move across the body—specifically for older adults. The authors ran two phases of co-design workshops with 13 independently living older adults, first diverging on possible applications and then converging on three concrete applications (massage, physical therapy, and walking). From these workshops they derive a two-level design space: context, human, and application on the first level, and robot and communication factors on the second. They also articulate three design principles: adapt to social norms, practical interaction design, and clear utility with cohesive identity. If this framework is useful, it gives researchers a shared vocabulary and an initial map for building and evaluating on-body robots for aging users.","feed_headline":"13 older adults co-design a map for on-body robots","feed_subtitle":"A two-level design space and three principles show how movable body-worn robots should look, speak, and behave for aging users.","key_machinery":"The central object is the two-level design space itself, composed of Level 1 scopes (context, human, application) and Level 2 factors (robot: embodiment, topology, perception, autonomy, adoption; communication: modality, content). Design anchors are the boundaries that yield the three design principles, and design units indicate which Level 2 factors should be considered together. The paper uses Calico, a small track-based robot probe, to ground participants' design thinking across brainstorming, experience-flow worksheets, and bodystorming sessions, with thematic analysis converting workshop data into the framework.","core_discovery":"The central claim is that the interaction design space for on-body robots for older adults can be characterized by a two-level structure in which scoping concepts—context, human, and application—bound the exploration of design factors for the robot and for communication. Design anchors define boundaries that translate into three design principles (DP1, DP2, DP3), and design units show which factors must be considered jointly, such as embodiment and topology or modality and content. The authors argue that this framework is grounded in the lived experiences and design activities of the older adult participants themselves, and that it can serve as an initial structured map for research and prototyping rather than a fixed specification.","pith_inferences":["If the framework generalizes beyond the participant group, it could serve as a generative checklist for designing on-body robots for other populations with sensory or motor constraints, such as people who are blind or have limited mobility.","The emphasis on co-presence suggests a single robot may need to dynamically adjust its expressiveness depending on whether the user is alone, with family, or in public—an idea the paper mentions but does not fully develop.","A longitudinal deployment study with a functional on-body robot in daily life would likely surface additional factors, such as long-term maintenance and changing trust, that short workshops cannot capture."],"forward_implications":["Researchers can use the design space to identify and structure open research questions, such as how to design olfactory output or how to manage multiple on-body robots distributed across the body.","Prototype builders can start from the Level 1 scopes and then systematically explore Level 2 factors to generate feasible on-body robot concepts for older adults.","The three design principles give concrete guidance: keep the robot discreet and context-aware for social norms, make physical interactions comfortable and non-intrusive, and ensure the robot has clear utility and a cohesive identity.","The framework highlights that communication with on-body robots should be multimodal and content-aware, balancing supportive and informative cues to avoid sensory overload."],"supporting_citations":[{"why":"Supplies the design probe, Calico, used to ground all workshops with a tangible on-body robot.","marker":"[15]"},{"why":"Defines a canonical example of on-body mobile robots, establishing the class of systems the design space targets.","marker":"[17]"},{"why":"Provides the participatory-design grounding for working with older adults as design partners.","marker":"[20]"},{"why":"Supports the value of sharing older adults' personal narratives in technology design, shaping the workshop approach.","marker":"[35]"},{"why":"Supplies long-term co-design guidelines that structure the two-phase workshop methodology.","marker":"[45]"},{"why":"Provides the thematic analysis method used to derive the design factors and principles from workshop data.","marker":"[52]"}],"fun_headline_variants":["Older adults map on-body robot design","Co-design with 13 seniors shapes robot principles","On-body robots need a design space, older adults provide it","How older adults want on-body robots to look and act","A roadmap for on-body robots from older adult co-design"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework is assumed to be informative for older adults generally, even though it is built from 13 independently living, mostly Caucasian participants, one track-based robot probe, and short workshop sessions.","fun_headline_variants_meta":{"raw":{"variants":["Older adults map on-body robot design","Co-design with 13 seniors shapes robot principles","On-body robots need a design space, older adults provide it","How older adults want on-body robots to look and act","A roadmap for on-body robots from older adult co-design"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000122,"raw_usage":{"total_tokens":1028,"prompt_tokens":811,"completion_tokens":217,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":427,"completion_tokens_details":{"reasoning_tokens":140}},"tokens_in":427,"tokens_out":217,"duration_ms":2844,"temperature":1.0,"reasoning_tokens":140,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T11:20:39.720462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If a co-design study with a more diverse group of older adults—varying in physical and cognitive abilities—systematically produced design themes that cannot be mapped onto the proposed Level 1 scopes or Level 2 factors, the framework's comprehensiveness would be disproven.","supporting_citations":[{"cited_title":"Calico: Relocatable on-cloth wearables with fast, reliable, and precise locomotion,","cited_arxiv_id":null,"evidence_quote":"Supplies the design probe, Calico, used to ground all workshops with a tangible on-body robot."},{"cited_title":"Rovables: Miniature on-body robots as mobile wearables,","cited_arxiv_id":null,"evidence_quote":"Defines a canonical example of on-body mobile robots, establishing the class of systems the design space targets."},{"cited_title":"Maximizing the benefits of participatory design for human–robot interaction research with older adults,","cited_arxiv_id":null,"evidence_quote":"Provides the participatory-design grounding for working with older adults as design partners."},{"cited_title":"Personal narratives in technology design: the value of sharing older adults’ stories in the design of social robots,","cited_arxiv_id":null,"evidence_quote":"Supports the value of sharing older adults' personal narratives in technology design, shaping the workshop approach."},{"cited_title":"Long-term co-design guidelines: empowering older adults as co-designers of social robots,","cited_arxiv_id":null,"evidence_quote":"Supplies long-term co-design guidelines that structure the two-phase workshop methodology."},{"cited_title":"Thematic analysis of qualitative data: Amee guide no. 131,","cited_arxiv_id":null,"evidence_quote":"Provides the thematic analysis method used to derive the design factors and principles from workshop data."}],"review_version":1}