{"id":"132bd053-c409-468b-9ea8-34b799e8703f","arxiv_id":"2607.21802","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A collaborative-filtering layer that learns per-user appropriateness scores consistently improves robot action selection over aggregate baselines on two human-robot interaction datasets.","lead":"This paper adds a recommender-system layer to robot action scoring, learning each person's preferences from their past ratings of robot behavior in different scenes. It reports that this personalized layer consistently beats a one-size-fits-all approach across two datasets and several model architectures.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported CF gains may be driven by per-user intercepts rather than collaborative filtering; missing bias-only ablation","rationale":"The reader identified cold-start/user-wise splitting as the weakest assumption. I agree that is a real limitation, but I find a more fundamental attribution concern: the architecture confounds collaborative-filtering interaction terms with simple per-user and per-scene biases, and no ablation separates them. Without a bias-only control, the paper's claim that the 'collaborative-filtering module' yields the gains is not fully supported—even though the overall StARS system may still beat the non-personalized baseline. This concern is concrete and testable with the released code. The reader's CONDITIONAL verdict remains appropriate; my read does not move it to a different category, but it strengthens the need for additional experiments.","tokens_in":13245,"tokens_out":6015,"duration_ms":67495,"concrete_test":"Run the same training protocol and splits with three models: (i) full StARS; (ii) a bias-only variant with the same content head but no user–scene interaction term (set p_u⊙q̄_i to zero, keeping b_u, b_i, action offsets, and content residual); (iii) the scene-only baseline. Compare RMSE/MSE and Pearson r on both datasets. If the bias-only variant achieves ≥80% of the full StARS improvement over the scene-only baseline, or if its metrics are within 0.02 RMSE of full StARS, the collaborative-filtering interaction is not supported as the source of gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim attributes the improvements to a collaborative-filtering module, but the model in Eq. (5) bundles per-user offsets b_u, per-scene offsets b_i, content residuals, and the interaction term (p_u ⊙ q̄_i)ᵀ r_a. The scene-only baseline contains no user identifier, so any user-specific signal—even a simple per-user average offset—will reduce error. The paper never ablates the user/scene bias or interaction components, so Tables II and III cannot distinguish between gains from genuinely collaborative signal-sharing across users and gains from memorizing each user's mean rating. On MannersDB+, users average only 20.8 ratings, so a per-user intercept alone could plausibly capture much of the variance. If a bias-only variant (e.g., ŷ = b_u + b_i + h_iᵀr_a + action offset, omitting p_u ⊙ q̄_i) recovers most of the RMSE/Pearson improvement, then the claim that collaborative filtering is the mechanism is overstated; the practical conclusion would reduce to 'adding user IDs helps'. This is load-bearing because it determines whether the recommender-system formulation is actually doing the work, not merely an evaluation edge case.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reformulates socially appropriate robot action prediction as a personalised rating prediction problem. StARS augments a differentiable scene encoder with a multi-task matrix-factorization collaborative-filtering (CF) module: Eq. (5) combines shared user/scene embeddings, action embeddings, user/scene/global offsets, and residual content fusion. Training is two-stage (scene-encoder pre-training, then end-to-end joint optimisation). The method is evaluated on MannersDB+ with five scene-only backbones and on SocNav1 with six GNN backbones, using RMSE/MSE, Pearson r and CCC, with Bouckaert-Frank-corrected paired t-tests and Holm adjustment. Tables II and III report consistent StARS improvements over the scene-only baselines in all 11 backbone-dataset pairs with small adjusted p-values, and Figure 3 reports learning curves for two StARS variants under varying training fractions.","tokens_in":13467,"tokens_out":3938,"duration_ms":43723,"significance":"If the empirical claims hold, StARS is a useful model-agnostic personalisation layer for socially appropriate robot action selection, and the paper's public code, two datasets, eleven backbones, and corrected significance testing are genuine strengths. However, the manuscript does not yet establish that the CF interaction term, rather than simple per-user offsets, is responsible for the reported gains, and the evaluation protocol assumes that target users are already known at training time. These gaps are load-bearing for the paper's central claim that collaborative filtering is the mechanism and that the method supports practical personalised deployment.","major_comments":[{"comment":"The central claim attributes the improvement to a collaborative-filtering module, but Eq. (5) bundles several user-specific components: the offset b_u, the scene offset b_i, the content residual h_i^T r_a, and the interaction term (p_u ⊙ (q_i + h_i))^T r_a. The scene-only baseline contains no user identifier, so any user-specific signal, including a simple per-user mean offset, would reduce error. With MannersDB+ having only 20.8 annotations per annotator on average, a per-user intercept could plausibly capture much of the variance. The paper never ablates the interaction term. Please add a bias-only variant (e.g., yhat = h_i^T r_a + b_u + b_i + b, without p_u ⊙ qbar_i) and report the same metrics and tests. If such a variant recovers most of the RMSE/Pearson/CCC improvement, the conclusion should be weakened from 'collaborative filtering helps' to 'adding user IDs helps'.","section":"§III-B, Eq. (5); Tables II and III"},{"comment":"The evaluation splits the full set of annotated user–scene rows randomly into train and test, so the same annotators appear in both splits. Because Eq. (5) requires a user embedding p_u for every prediction, this protocol does not measure performance for new users with no prior ratings (cold start). The Discussion claims real-world deployment where the model is 'initialised from offline annotations' and adapts user embeddings, but no user-wise split or cold-start experiment is reported. Please add an evaluation where some users are held out entirely, or otherwise state clearly that the reported gains apply only to known users with existing ratings. This is necessary to support the deployment-facing claims in Section VI.","section":"§IV-D, Eq. (5); Discussion"},{"comment":"The RQ3 claim that the RS formulation is 'particularly beneficial' or 'particularly effective' in the low-data regime is not supported by the presented learning curves. Figure 3 shows only StARS performance as a function of training fraction, with no comparison to the corresponding scene-only baseline or to a bias-only variant at the same fractions. The curves could simply reflect the backbone's own data-scaling behaviour. Please include the scene-only baseline (and ideally the bias-only ablation) in Figure 3, and report significance or confidence intervals for the gap; otherwise the sparsity-robustness conclusion is unsubstantiated.","section":"§V-C, Fig. 3; RQ3"}],"minor_comments":[{"comment":"Typo: 'across-validation–corrected paired t-test' should read 'cross-validation–corrected paired t-test'.","section":"§IV-E"},{"comment":"The notation is a bit dense: h_i appears both as part of the hybrid scene embedding qbar_i = q_i + h_i and as a separate residual term. A one-sentence explanation of why the second h_i is needed, and how it differs from the first, would improve readability.","section":"§III-B, Eq. (5)"},{"comment":"The algorithm says z-score statistics are computed 'from training split D', while the text says normalization is applied per training split. Please clarify whether mu_a and sigma_a are recomputed in each outer fold and whether the inner validation split is included in these statistics.","section":"Algorithm 1, §IV-D"},{"comment":"The learning curves show only five-fold means without error bars or pointwise variability. Adding confidence intervals or per-fold points would make the claimed diminishing-returns trend easier to assess.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is solid and the writing is clear, but the missing bias-only ablation and the train/test user leakage are substantive enough that the current wording overstates the contribution. Both are fixable within the manuscript's scope, so I do not recommend rejection. The paper would be much stronger with a user-wise split and a bias-only baseline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First the good news: this is the first computational evaluation of collaborative filtering for personalized socially appropriate robot actions, and the evaluation is done carefully. Eleven backbones across two datasets, five-fold CV, Bouckaert–Frank paired t-tests with Holm correction, and code are all provided. The gains from adding the StARS layer are large and consistent, so the empirical claim that conditioning on user identity improves prediction is solid. The multi-task MF with residual scene fusion is a modest but sensible extension of standard matrix factorization, and the scene-encoder agnosticism is a nice practical property.\n\nThe main soft spot is exactly what the stress test flagged: the model bundles per-user bias b_u, per-scene bias b_i, and the interaction term (p_u ⊙ q_i)^T r_a into one package, and the scene-only baseline has no user identifier at all. There is no ablation that strips out the interaction and keeps only the bias terms. So the paper's central claim—that collaborative filtering is the mechanism—is not actually supported. A bias-only variant (predict b_u + b_i + content) might recover most of the RMSE/Pearson gains, in which case the conclusion reduces to \"adding user IDs helps.\" That is still useful, but it changes the story. This is a load-bearing gap, not a nitpick.\n\nSecond soft spot: the split is row-wise, so the same annotators appear in both train and test. The model needs a user embedding at test time, which is fine for a returning user but not for deployment to unseen users. The RQ3 \"sparsity\" experiment subsamples rows but still includes every user in training; it does not measure per-user cold start or extreme sparsity. The paper acknowledges cold start as future work, but the abstract and discussion make broader deployment claims, so this should be called out.\n\nThe citation pattern is fine; self-citation to the authors' conceptual HRI paper is appropriate as motivation, and the other references look standard. No invented entities or circular derivation.\n\nVerdict: a solid empirical contribution worth refereeing, but it needs an ablation separating bias from interaction, plus a user-wise split or cold-start experiment, before the collaborative-filtering-specific claims can be trusted. The core result—that personalization helps—will likely survive that test; the mechanism claim may not.\n\nI'd bring this to reading group, and I'd cite it if I were working on HRI personalization.","headline":"Useful empirical demonstration that recommender-style personalization improves social appropriateness prediction, but the mechanism claim is undercut by a missing bias-only ablation and cold-start evaluation.","tokens_in":13988,"tokens_out":2235,"would_cite":true,"duration_ms":25927,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A collaborative-filtering layer added to any scene encoder lets a robot predict how appropriate each action is for a specific user, and the gains over averaged human ratings are consistent across two datasets and eleven backbones.","keywords":["socially appropriate robot actions","personalization","collaborative filtering","recommender systems","matrix factorization","human-robot interaction","social navigation","inter-subject variability"],"falsifier":"Take either dataset, split by annotator identity rather than by row so no annotator appears in both training and test, and retrain StARS; if the gains over aggregated baselines disappear or reverse, the personalisation effect does not generalize to unseen users. A second check: give the model a cold-start user with only a few ratings and see whether it still beats an aggregated baseline that ignores identity.","tokens_in":13084,"feed_emoji":"🤖","tokens_out":3360,"duration_ms":34450,"temperature":0.7,"pith_summary":"This paper argues that social appropriateness in human-robot interaction is not universal: the same robot action in the same scene can be judged appropriate by one person and inappropriate by another. It reformulates the problem as a recommender-system task, treating annotators as users, scenes as items, and appropriateness scores for candidate actions as ratings. The proposed StARS framework augments any differentiable scene encoder with a collaborative-filtering layer based on matrix factorization, learning user, scene, and action embeddings jointly so each user receives personalized scores. Across two datasets and eleven backbone models, StARS consistently reduces prediction error and improves agreement with human raters, with statistically reliable gains even under sparse preference feedback.","feed_headline":"Recommender trick makes robot actions fit individual tastes","feed_subtitle":"The same scene can call for different robot behavior per person; a matrix-factorization layer captures who likes what.","key_machinery":"Multi-task matrix factorization with residual fusion. User and scene embeddings are shared across actions while each action has its own embedding; the hybrid scene embedding is the scene encoder's content embedding plus a collaborative scene embedding, and the personalized score is the dot product of an action embedding with the Hadamard product of the user embedding and hybrid scene embedding, plus a residual content term and user, scene, and global offsets. This combines collaborative signal (who rates what) with content signal (scene features), and the residual content shortcut stabilizes prediction when collaborative factors are weak.","core_discovery":"StARS treats social appropriateness as a user-item rating problem and shows that collaborative filtering can be layered on top of any differentiable scene encoder without redesigning it. The prediction for a user, scene, and action is a dot product between an action embedding and a user-personalized scene representation, where the scene representation is the sum of a collaborative scene embedding and a content embedding from the scene encoder, plus a residual content term and offset terms. This structure lets the model share statistical strength across users and scenes while still outputting user-specific scores. Evaluated on MannersDB+ and SocNav1, StARS outperforms the scene-only version o","pith_inferences":["If the method is evaluated with a user-wise split (no overlap of annotators between training and test), the gains may shrink or vanish, because the model must then infer a user embedding from scratch; the paper only reports random-split results.","The same architecture could be applied to other personalization-centric HRI tasks, such as preferred assistance levels in collaborative assembly or preferred communication style, treating the user as the 'user' and interaction contexts as 'items'.","A testable extension: use cold-start users (no prior ratings) and initialize their embeddings from scene-only predictions; if aggregated baselines then become competitive, the value of collaborative filtering lies in warm-start personalization.","The residual content shortcut suggests the model is robust when collaborative factors are weak; one could intentionally ablate the residual term to measure its contribution."],"forward_implications":["Socially appropriate robot action selection can be personalized without retraining or replacing the underlying perception or decision model; the collaborative-filtering layer is an add-on.","The improvements are systematic, not architecture-specific: all eleven backbone-dataset pairs show the same direction of gain.","Personalization works in sparse regimes: meaningful gains appear with a fraction of the data, which matters because collecting per-user ratings is costly.","The structure of inter-subject disagreement is learnable: rating variation correlates with user-scene interactions, not just noise.","Recommender-system methods are transferable to HRI preference modelling, opening a path to rank alternative robot actions by user-specific appropriateness."],"fun_headline_variants":["Robot actions get personal with recommender-style ratings","Collaborative filtering tailors robot behavior to each person","StARS: recommend the right robot move for you","Preference modeling makes robots socially appropriate per user","A recommender system teaches robots what you deem polite"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The reported gains assume the target user is already known when training starts: ratings are split randomly, so the same individuals appear in training and test, and every personalised score requires a user embedding.","fun_headline_variants_meta":{"raw":{"variants":["Robot actions get personal with recommender-style ratings","Collaborative filtering tailors robot behavior to each person","StARS: recommend the right robot move for you","Preference modeling makes robots socially appropriate per user","A recommender system teaches robots what you deem polite"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000151,"raw_usage":{"total_tokens":1018,"prompt_tokens":704,"completion_tokens":314,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":240}},"tokens_in":448,"tokens_out":314,"duration_ms":4089,"temperature":1.0,"reasoning_tokens":240,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T06:36:56.986069+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take either dataset, split by annotator identity rather than by row so no annotator appears in both training and test, and retrain StARS; if the gains over aggregated baselines disappear or reverse, the personalisation effect does not generalize to unseen users. A second check: give the model a cold-start user with only a few ratings and see whether it still beats an aggregated baseline that ignores identity.","supporting_citations":[],"review_version":1}