{"id":"d66f91aa-2ca9-404a-be8e-38282c7b431a","arxiv_id":"2501.01367","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Features learned from users' exploratory clicks on robot behaviors outperform self-supervised features for preference elicitation across visual, auditory, and kinetic signals.","lead":"This paper introduces CLEA, a way to learn what features of robot behavior matter to users by watching which behaviors they choose to explore while customizing a robot. In two user studies, features learned this way beat self-supervised features at predicting and explaining user preferences.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The learning signal assumes 'ignored' behaviors were considered and rejected, but the interface did not log which items were actually viewed; unviewed items may dominate the negative set and CLEA may learn interface exposure patterns rather than preference.","rationale":"The reader's weakest-assumption analysis correctly identifies the most load-bearing point: CLEA derives all of its human supervision from the explored/ignored partition, and the interface design makes 'ignored' a very noisy proxy for 'disliked.' I do not see a stronger concern. The paper otherwise has real independent support: a held-out evaluation population, multiple baseline algorithms, four evaluation criteria, and extensive appendices, including a direct-reward-modeling comparison. The kinetic modality's null completeness result is evidence that the effect is not uniformly strong, but it does not by itself invalidate the central claim. The ignored-label concern is testable and could be resolved by re-analyzing interaction logs or by a small instrumented follow-up study. Since the reader already issued a CONDITIONAL verdict on the basis of this same concern, my stress-test does not change the verdict: the paper should be accepted only if the authors either verify the semantics of ignored behaviors or clearly qualify the claim as contingent on that assumption.","tokens_in":22828,"tokens_out":3424,"duration_ms":40498,"concrete_test":"Audit the original interaction logs (or rerun the customization session with logging) to reconstruct, for each presented set Di, the actually-viewed subset V_i from scroll position, filter state, and viewport timestamps. Retrain all CLEA variants with the negative set defined as Dig_i = V_i \\ Dex_i (viewed but not selected) instead of Dig_i = Di \\ Dex_i, then rerun the full held-out ranking evaluation (Sec. V) and compare test preference accuracy and AUC Alignment. If results are statistically unchanged, the assumed ignored-label semantics are not load-bearing; if the CLEA advantage shrinks or reverses, the central claim currently rests on an unverified assumption about unseen items.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (1) partitions each presented set Di into explored (psi=1) and ignored (psi=0) behaviors, and the CLEA triplet loss (Eqs. 2-4) treats every ignored item as a negative. The load-bearing semantic claim is that psi=0 means 'evaluated and found irrelevant.' The interface (Sec. IV-A, Fig. 4) presented up to 100 behaviors through scrolling and keyword filtering, and participants selected appealing items to evaluate on the physical robot. The procedure did not log which items were actually visible, scrolled past, or filtered away, so most of the ignored set is plausibly composed of behaviors the user never considered. If so, the contrastive objective separates selected behaviors from arbitrary unselected database entries, and the learned features may encode interface exposure patterns rather than preference. This concern is central because exploratory action labels are the only human signal CLEA uses; if those labels are mostly noise, the reported gains in completeness, simplicity, minimality, and explainability could be interface artifacts. The kinetic modality result, where Random tied for highest test preference accuracy, further suggests the learned preference signal is weak in at least one modality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes contrastive learning from exploratory actions (CLEA), a method for training trajectory-feature encoders from implicit labels generated during open-ended robot customization. In a signal-design study, 25 users browsed up to 100 visual, auditory, and kinetic behavior options, selecting some for physical-robot evaluation; the paper treats selected behaviors as positives and unselected behaviors as negatives in a symmetric triplet loss (Eqs. 2-4). The learned features are evaluated on a disjoint set of 42 users who rank behaviors, comparing CLEA, CLEA+AE, and CLEA+VAE with Random, Pretrained, AE, and VAE baselines on four criteria: completeness (test preference accuracy), simplicity and minimality (AUC of alignment of a linear reward model), and explainability (cosine similarity to nearest exemplar). The authors report that CLEA-based features outperform self-supervised features on most metrics in visual and auditory modalities, with the kinetic completeness comparison showing no significant differences.","tokens_in":23074,"tokens_out":7511,"duration_ms":69158,"significance":"The central idea is attractive: exploratory search is already required for customization, so using its byproduct as preference labels avoids proxy tasks entirely. The evaluation has real strengths: separate training and testing populations, no overlap between training objective and evaluation metrics, predefined hypotheses from Bobu et al.'s framework, Bonferroni-corrected post-hoc tests, and appendices with pseudocode and training details. If the label assumption is verified, CLEA would be a practical and scalable way to obtain user-aligned behavior representations. The reported gains are substantial in several metrics (e.g., visual completeness of CLEA+AE). The main risk is not circularity but label validity: the negative set may contain behaviors the user never considered, and at least one modality shows a null completeness result.","major_comments":[{"comment":"Equation (1) partitions each presented set Di into explored (psi=1) and ignored (psi=0), and the CLEA triplet loss (Eqs. 2-4) treats every ignored behavior as a negative. The semantic claim is that psi=0 means the user evaluated the behavior and found it irrelevant. In the RoSiD exploratory interface, however, participants could scroll through and keyword-filter up to 100 options, and the study did not log which options were actually visible or attended. A large fraction of Dig is therefore likely to be behaviors the user never considered, so the negative label may reflect interface exposure rather than stated preference. Because this partition is the only human supervision used to train CLEA, the load-bearing assumption is currently unverified. I ask the authors to (a) add viewport/scroll/filter logging and report results when negatives are restricted to options verifiably displayed but not selected, or (b) provide an explicit label-noise analysis showing that the learned features are not driven by exposure patterns.","section":"III-B, Eq. (1); IV-A, Fig. 4"},{"comment":"The abstract claims CLEA features outperform self-supervised features on completeness, but Appendix E reports that in the kinetic modality there were no significant differences between algorithms, with Random, CLEA, and CLEA+AE empirically highest (p = .008, eta^2 = .247). This directly undermines the completeness claim for one of the three modalities and the unqualified statement in Section VI that CLEA 'significantly increased performance in all modalities.' Please either qualify the completeness claims to the visual and auditory modalities, or provide additional evidence for the kinetic modality.","section":"V-D, 'Evaluating Completeness'; Appendix E"},{"comment":"The text states that 'a CLEA-based feature space has the highest AUC Alignment in 13 of the 15 experiments,' but Table I shows that in several dimension-modality cells the best-performing algorithm is not flagged as significant, and in some cells a non-CLEA baseline has the highest value (e.g., VAE at Visual-16, AE at Kinetic-8). Counting rank-order wins without requiring significance overstates the support for H2. Please report the number of cells in which a CLEA-based method significantly outperforms all baselines after correction, and keep the '13 of 15' claim only if it refers to significant wins or is rewritten accordingly.","section":"V-D, 'Evaluating Simplicity and Minimality'; Table I"},{"comment":"The evaluation of minimality compares AUC Alignment across feature spaces of dimensions 8, 16, 32, 64, and 128, but the paper does not specify how the lower-dimensional feature spaces are constructed from the 128-dimensional encoders. If these are truncations or linear projections of a 128-d embedding, the minimality comparison does not evaluate the ability of each algorithm to learn a low-dimensional representation; it evaluates the information retained by the first d coordinates. Please state the construction and, if appropriate, retrain or report results for encoders with output dimension d.","section":"V-D, 'Evaluating Simplicity and Minimality'; Appendix C-F"}],"minor_comments":[{"comment":"The same symbol RH is used for the user's true reward function and for the learned reward model; this makes Eq. (6) and the later discussion of omega_true and omega_est confusing. Please use distinct notation, e.g., R* and R-hat.","section":"III-A, Eq. (5)"},{"comment":"'CLAE+V AE' appears to be a typo for 'CLEA+V AE'; please correct it.","section":"V-D, Minimality paragraph"},{"comment":"Hyperparameter selection reports only the chosen margin and KL weight; reporting performance across the sweep or describing the validation procedure would help readers gauge sensitivity.","section":"Appendix C"},{"comment":"Error bars are described as 'mean standard error across participants'; since TPA is a per-participant proportion, consider also reporting per-participant distributions or confidence intervals.","section":"Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid candidate but the central training-label assumption needs empirical support. The kinetic completeness null and the underspecified low-dimensional feature construction are also addressable. I do not see circularity or novelty concerns; this is a genuine extension of the authors' prior workshop paper with a new held-out evaluation. Scope is appropriate for a robotics/HRI venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one. It's a solid HRI/learning paper with a genuinely new data source: users customizing a robot naturally select behaviors to test and ignore the rest, and the authors turn that into triplet training for feature spaces. The evaluation is careful—held-out users, multiple baselines (Random, pretrained foundation models, AE/VAE, and CLEA variants), Bonferroni-corrected tests, and four criteria from Bobu et al. The visual and auditory results are convincing: CLEA-based features dominate on test preference accuracy, linear reward AUC, and explainability. The qualitative examples in Appendix I are also striking—CLEA embeddings find semantically related images (maps, containers, lightbulbs) where autoencoders find structural lookalikes.\n\nThe soft spot is the central assumption behind Eq. (1). The interface showed up to 100 behaviors with scrolling and filtering but didn't log which items were actually seen. So 'ignored' likely includes plenty of behaviors the user never considered. The triplet loss treats those as negatives—items the user rejected—when they may just be items that never entered the screen. That's a real gap, and it's exactly the kind of thing that could make the learned features encode interface exposure patterns rather than preference. I don't think it sinks the paper: selection is at least preference-correlated, and the held-out ranking results would be hard to explain if the signal were pure interface artifact. But the authors should either log viewport data in future work or analyze how many 'ignored' items were likely unseen. The kinetic modality is a second soft spot: Random ties for highest TPA, and no significant completeness difference among algorithms. The paper's abstract overstates slightly when it says CLEA outperformed self-supervised features across all four metrics—that's true on average for visual/auditory, but kinetic is mixed and Random is competitive.\n\nOne more smaller issue: the simplicity count. The text says CLEA-based features won 13 of 15 experiments and CLEA+VAE was best in 10. Counting asterisks in Table I gives 12 CLEA-based wins and 9 for CLEA+VAE. It's a minor internal inconsistency, but it should be fixed.\n\nOverall: this is a serious paper that deserves referee time. The idea is portable, the experiments are reproducible (code/data on project page, clear training details), and the limitations section is honest about familiarity and summaries. I'd recommend major revision, mostly to address the ignored-set semantics and tighten the claims.","headline":"CLEA is a genuinely useful new data source for preference-aligned features; the main unresolved question is what 'ignored' behaviors actually mean when users never saw them.","tokens_in":23511,"tokens_out":3665,"would_cite":true,"duration_ms":33999,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Users' natural exploratory actions while customizing a robot — browsing, selecting, ignoring — are implicit preference labels, and contrastive features learned from them outperform self-supervised features on all four tested criteria.","keywords":["preference elicitation","contrastive learning","exploratory search","robot behavior representation","reward learning","human-robot interaction","multimodal signals","self-supervised learning"],"falsifier":"Train CLEA twice on the same logged exploratory actions, once with every unselected behavior as a negative and once with only behaviors the user demonstrably viewed (via gaze tracking or interaction logs) as negatives, then compare held-out ranking accuracy on the 42-participant task; if accuracy does not change, the ignored-as-negative assumption is not load-bearing, and if it falls, the assumption carries the result.","tokens_in":22650,"feed_emoji":"🤖","tokens_out":6711,"duration_ms":60291,"temperature":0.7,"pith_summary":"The paper asks where robot-behavior features that capture what users actually care about can come from without forcing users to label data. It proposes that exploratory search — the selecting and ignoring users naturally do while customizing a robot — is itself a preference signal, and trains trajectory features with a contrastive triplet loss on those actions. Using data from 25 users designing signals for a Kuri robot, the authors learn features and then evaluate them with 42 new users ranking behaviors. They report that CLEA-based features outperform self-supervised features across visual, auditory, and kinetic modalities on completeness, simplicity, minimality, and explainability.","feed_headline":"Browsing beats labeling for teaching robots your preferences","feed_subtitle":"Natural browsing actions beat self-supervised features on all four tested preference criteria in a Kuri robot study.","key_machinery":"The central object is the CLEA loss, a symmetric triplet loss. For each set of behaviors a user was shown, equation (1) splits it into explored ($\\psi(\\xi)=1$, selected or tested) and ignored ($\\psi(\\xi)=0$) behaviors; equations (2)–(4) sample two behaviors from one side and one from the other, then push the same-side features closer than the cross-side feature by a margin $\\alpha$, summed over all users and datasets. This trains a feature map $\\Phi: \\Xi \\to \\mathbb{R}^d$ so that Euclidean distance in feature space reflects population-level preference agreement rather than physical reconstruction. Downstream, a Bradley-Terry model over rankings, equation (5), turns the features into a reward function $R_H$.","core_discovery":"The paper's claim is that exploratory actions are a legitimate and usable preference signal: users who scroll, filter, select, and ignore robot behaviors while customizing a robot are implicitly partitioning behaviors into categories that matter to them, and a symmetric triplet loss over that partition learns features that align with user preferences across modalities. The authors train on 25 users' exploratory actions in a signal-design task with a Kuri robot and evaluate on 42 new users who ranked behaviors. CLEA-based feature spaces outperformed self-supervised autoencoder and VAE baselines, as well as random and pretrained networks, on completeness, simplicity, minimality, and explainability across all three modalities.","pith_inferences":["A testable extension the authors leave implicit: applying the same explored/ignored partition to other logged browsing data, such as recommender scrolls or video thumbnail selections, would show whether CLEA transfers to any interface where ignoring is observable.","The ignored-as-negative assumption could be probed directly by training on gaze-filtered negatives; if accuracy holds, the signal comes from true rejection rather than from interface exposure.","CLEA learns a population-level embedding; adapting the embedding to a single user after a few of their own exploratory actions is a natural next step the paper does not demonstrate."],"forward_implications":["Robot customization interfaces can generate training data as a by-product of the user's main task, so personalization no longer requires a separate labeling phase.","CLEA can be combined with existing feature learners such as autoencoders and variational autoencoders, and the paper argues the same loss can extend to trajectory similarity queries and labeled behaviors.","Because CLEA features transfer to a new population of users, a single round of exploratory-action data collection can serve many downstream users.","CLEA feature spaces support simple linear reward models in low dimensions, meaning preference elicitation can need fewer user queries and simpler models.","Explainability-by-example improves: top-ranked behaviors land near exemplars from the customization session in CLEA feature spaces."],"supporting_citations":[{"why":"Supplies the four evaluation criteria — completeness, simplicity, minimality, explainability — that organize the paper's hypotheses and analysis.","marker":"[3]"},{"why":"Supplies the triplet-loss formulation for perceptual judgments and the test preference accuracy metric used to measure completeness.","marker":"[4]"},{"why":"Supplies the RoSiD signal-design interface and the customization session from which the exploratory-action training data are collected.","marker":"[7]"},{"why":"Defines exploratory search, the human-computer interaction concept the paper reframes as a source of preference labels.","marker":"[50]"},{"why":"Supplies Bayesian inverse reward learning and the AUC alignment metric used to evaluate simplicity and minimality.","marker":"[8]"},{"why":"Supplies the Bradley-Terry preference model used to convert rankings into pairwise preference probabilities.","marker":"[70]"},{"why":"Supports modeling intuitive perceptual decisions with triplet losses, a foundation for the CLEA objective.","marker":"[58]"}],"fun_headline_variants":["Browsing, not labeling, reveals robot preferences","Exploratory actions outperform self-supervised features in robot learning","Natural exploration is a better signal for robot preference elicitation","CLEA: learn robot traits from what users browse, not ask","Your browsing history teaches robots your preferences better than labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a behavior a user scrolls past without selecting is a true negative — actively deemed irrelevant — rather than simply unseen.","fun_headline_variants_meta":{"raw":{"variants":["Browsing, not labeling, reveals robot preferences","Exploratory actions outperform self-supervised features in robot learning","Natural exploration is a better signal for robot preference elicitation","CLEA: learn robot traits from what users browse, not ask","Your browsing history teaches robots your preferences better than labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000398,"raw_usage":{"total_tokens":2064,"prompt_tokens":910,"completion_tokens":1154,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":1085}},"tokens_in":526,"tokens_out":1154,"duration_ms":11317,"temperature":1.0,"reasoning_tokens":1085,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:28:09.290449+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CLEA twice on the same logged exploratory actions, once with every unselected behavior as a negative and once with only behaviors the user demonstrably viewed (via gaze tracking or interaction logs) as negatives, then compare held-out ranking accuracy on the 42-participant task; if accuracy does not change, the ignored-as-negative assumption is not load-bearing, and if it falls, the assumption carries the result.","supporting_citations":[{"cited_title":"Exploratory search: from finding to understanding,","cited_arxiv_id":null,"evidence_quote":"Defines exploratory search, the human-computer interaction concept the paper reframes as a source of preference labels."},{"cited_title":"Faceted metadata for image search and browsing,","cited_arxiv_id":null,"evidence_quote":"Supplies the Bradley-Terry preference model used to convert rankings into pairwise preference probabilities."},{"cited_title":"Learning perceptual ker- nels for visualization design,","cited_arxiv_id":null,"evidence_quote":"Supports modeling intuitive perceptual decisions with triplet losses, a foundation for the CLEA objective."}],"review_version":1}