{"id":"fca445ea-0333-4a25-b54e-3171e9c02e7f","arxiv_id":"2504.20782","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Personalized reinforcement-learning agents trained with individual user feedback produced higher user satisfaction and engagement than non-adaptive interfaces in two app domains.","lead":"The authors built a system that trains a separate reinforcement-learning agent for each user, using the user's likes and dislikes about interface changes as part of the reward signal. In a 33-person study, users rated these personalized adaptive interfaces as more satisfying and engaging than fixed non-adaptive interfaces.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's causal claim that human feedback improves UX is untested: the experiment lacks an adaptive-without-feedback arm, so the observed effect could stem from RL adaptation or novelty rather than feedback integration.","rationale":"The reader's weakest assumption correctly identifies the missing adaptive-without-feedback condition as the load-bearing gap. I reviewed the full text, including Section 4.4 (design), Section 5.2 (LMM results), and Section 5.3 (discussion), and the strongest statistically supported claim is that the combined personalized-feedback adaptive system outperforms a non-adaptive interface. The abstract and conclusions make the stronger claim that human feedback is the cause of the improvement. That causal attribution requires a contrast between adaptive systems with and without the personalized feedback reward modifier, which the experiment does not provide. The prior study [15] is mentioned as having found no significant improvement with a general feedback model, which actually underscores the need for a controlled comparison in the current setting. The paper's internal evidence therefore does not bridge the gap. The crossover/grouo confound is real but secondary; the primary issue is sufficient on its own to reject the paper's central claim as stated. The concrete test I propose would settle whether the feedback modifier is necessary for the observed UX gains; if it is not, the paper's title, abstract, and conclusions overstate the contribution.","tokens_in":15172,"tokens_out":3295,"duration_ms":33853,"concrete_test":"Run a three-arm experiment using the same instrumentation, tasks, and crossover design: (A) adaptive with personalized human feedback, (B) adaptive using only the predictive HCI reward (no personalized feedback), and (C) non-adaptive. Fit the same LMM with technique as a three-level factor. The central claim that human feedback enhances UX requires A to significantly outperform B on both QUIS and UES; if A and B do not differ significantly while both exceed C, the effect is attributable to RL-based adaptation, not to feedback integration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and conclusions, is that 'incorporating human feedback into RL-driven adaptations significantly enhances UX.' The experimental design (Section 4.4) compares only two conditions: Adaptive with personalized human feedback and Non-Adaptive. There is no Adaptive-without-feedback control. Therefore, the significant LMM results in Section 5.2 (e.g., satisfaction estimate -0.97, p<0.001) can only establish that the combined feedback-augmented adaptive system outperforms a static UI. They cannot isolate the contribution of human feedback. The paper's own hypotheses H_n1/H_n2 are framed as 'AUIs that use predictive HCI models with personalized human feedback' versus non-adaptive UIs, so the statistical test matches the narrower claim, not the abstract's causal one. The prior study [15] is not a substitute: it compared different system versions in a separate experiment, and its feedback contrast was not statistically significant. Section 5.3 explicitly attributes the current significance to individualized feedback models, but that attribution is unsupported by the current design. A secondary concern is the crossover analysis: 'group' is confounded with carryover and treatment-by-period interaction, and the non-significance of group does not fully rule out carryover bias, though this is less load-bearing than the missing control.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper extends an existing RL-based framework for adaptive user interfaces (AUIs) by adding a personalized human-feedback module: users provide offline pairwise preference judgments on UI adaptation clips, a per-user reward-modifier MLP is trained on these judgments, and a GA3C RL agent is trained for each user with a dual reward combining a predictive HCI model and the personalized modifier. The authors then conduct a within-subject crossover experiment with 33 participants across two domains (trip planning and e-learning), comparing an adaptive UI with human feedback against a non-adaptive UI. Linear mixed models on QUIS satisfaction and UES engagement scores show significantly higher satisfaction and engagement for the adaptive condition, with no significant domain or group effects. The abstract and conclusions claim that incorporating human feedback into RL-driven adaptations significantly enhances user experience.","tokens_in":15363,"tokens_out":5777,"duration_ms":61623,"significance":"If the stated causal claim were supported, this would be a meaningful contribution: it would be one of the first empirical demonstrations of RLHF applied to adaptive user interfaces, with per-user agents trained from individual preference data and evaluated in a real user study. The manuscript has several strengths: it provides replication links for the framework and experimental materials, uses standardized questionnaires with reported Cronbach's alpha (QUIS 0.97, UES 0.90), checks LMM assumptions (Shapiro-Wilk and Levene's tests), and uses a balanced crossover design. It is also not circular: the outcome questionnaires were not used as training signals for the reward models. However, the central causal claim about the specific contribution of human feedback is not supported by the experimental design, because the design lacks an adaptive-without-feedback control arm. This is a load-bearing issue for the paper's main contribution.","major_comments":[{"comment":"The central claim, stated in the abstract and repeated in Sections 5.3 and 7, is that 'incorporating human feedback into RL-driven adaptations significantly enhances UX.' The experiment, however, compares only two conditions: Adaptive-with-human-feedback and Non-Adaptive. There is no Adaptive-without-feedback arm, nor an Adaptive-with-generic-feedback arm. Therefore the significant LMM estimates (satisfaction: -0.97, p < 0.001; engagement: -0.23, p = 0.016) can only establish that the combined feedback-augmented adaptive system outperforms a static interface. They cannot isolate the contribution of human feedback, which could plausibly arise from RL adaptation itself, from the offline preference session, or from novelty effects. The paper's own hypotheses H_n1/H_n2 are framed as comparing 'AUIs that use predictive HCI models with personalized human feedback' against non-adaptive UIs, so the statistical test matches the narrower claim, not the abstract's causal one. The prior study [15], which compared adaptive systems with and without a general feedback model, is a separate experiment with different procedures, tasks, and participant populations, and its feedback contrast was not statistically significant; it is not a valid control. To support the abstract claim, the design would need a third condition (e.g., adaptive RL without human feedback, or adaptive with a non-personalized feedback model) under identical conditions in the same experiment. Absent that, the manuscript must be revised to state the supported claim: the proposed feedback-augmented adaptive framework improves UX relative to non-adaptive UIs, with the specific contribution of the feedback mechanism remaining an open question.","section":"Abstract; Sections 4.4 and 5.2"},{"comment":"The paper's conclusion that 'there is no carryover effect' is too strong. The LMM includes group as a fixed effect, but group is completely confounded with period, treatment order, and the treatment-period interaction. The non-significance of the group coefficients does not constitute a test for carryover; it only indicates no detectable difference between the four sequence groups. A proper carryover analysis would either model the treatment-by-period interaction explicitly or analyze first-period data only. With only 33 participants, the test for group effects is likely underpowered, so the absence of significance provides little evidence for the absence of carryover. This does not necessarily invalidate the main technique effect, but the claim that carryover was ruled out is not established by the reported analysis.","section":"Section 5.2 and Section 4.4.3"},{"comment":"The discussion attributes the significant improvements in this study to the individualized preference models, contrasting it with the non-significant results in prior work [15]. This cross-experiment comparison is not supported by the data: the two studies differ in multiple respects (participant recruitment, experimental procedure, task domains, and the exact system version), and the current study lacks the control condition needed to separate the effect of individualization from the effect of having feedback at all, or from RL adaptation itself. The statement in Section 5.3 that the improvements 'emphasize the effectiveness of personalized adaptation strategies' therefore goes beyond what the experiment can establish. The discussion should be reworded to present the individualized-feedback mechanism as a design choice whose incremental benefit is not directly tested.","section":"Section 5.3"}],"minor_comments":[{"comment":"There are multiple typos and grammatical issues, including 'leaning process' (Abstract), 'adaption' (Abstract), 'continuosly' (Section 1), and 'a most 120 minutes' (Section 4.6.3). A thorough proofreading pass is needed.","section":"Throughout"},{"comment":"Section 4.4 describes three sessions (User Feedback, RL Agent Training, User Interaction), but Section 4.6.3 says 'each participant attended two sessions in total.' This is inconsistent: the RL agent training is evidently an offline process that does not require the participant, but the text should clarify that the participant attends two sessions while the overall methodology has three phases.","section":"Section 4.4 vs Section 4.6"},{"comment":"Table 4 labels the residual variance as 'α²', which is an unusual notation; the standard symbol for residual variance in LMM notation is σ². Please correct this for clarity.","section":"Table 4"},{"comment":"The analysis plan states that 'group or sequence (which was confounded with carryover and treatment*period interaction)' was included as a fixed factor, but it does not explain how the authors would distinguish these effects if the group term had been significant. This is related to the major comment on carryover; the text should be clearer about the limitations of the design.","section":"Section 4.7"}],"recommendation":"reject","confidential_remarks":"The missing adaptive-without-feedback condition is the central problem. The paper's abstract and conclusions claim a causal role for human feedback that the experimental design cannot support, and this cannot be repaired by editorial changes alone; it would require either a new experimental arm or a substantial reframing that would change the paper's contribution. The authors' prior work [15] is not a suitable substitute for the missing control. I would not recommend inviting a major revision unless the journal is willing to consider a substantially weakened claim about the feedback mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version. This paper trains a separate RL agent for each user, with a reward modifier learned from that user's pairwise feedback on UI adaptation clips, layered onto a predictive HCI reward. It then runs a crossover study comparing the resulting adaptive UI against a static interface in two domains. The comparison it actually ran is statistically competent, and beating static UIs on satisfaction and engagement is a real, new result. But the abstract's headline claim—that incorporating human feedback significantly enhances UX—is not supported, because there's no adaptive-without-feedback arm. The design only isolates the combined system, not the contribution of feedback itself.\n\nWhat's genuinely good: the per-user RLHF approach is a reasonable and under-explored idea when applied to UI adaptation. The empirical execution is careful. The stats seem mostly sound: linear mixed models, checked normality and variance, reported Cronbach's alpha, random effects, ICCs. They also ship replication links for the framework and the feedback-collection app, which deserves credit.\n\nSoft spots, in order of significance. One: missing control. The paper's own hypotheses are worded narrowly—adaptive with personalized feedback vs non-adaptive—so the tests match those hypotheses, but the abstract, Section 5.3, and the conclusions draw the causal inference the design can't support. In fact, Section 5.3 explicitly credits individualized feedback for the significance, which is the load-bearing overreach. Two: the 'no carryover' conclusion rests on a non-significant group effect in a design where group is confounded with carryover and treatment-by-period. That's a common limitation, but the wording overstates what the test can show. Three: 33 CS Master's students, convenience sampled, two task domains; external validity is thin. The paper lists some threats, so at least it's aware.\n\nMy take: the narrow empirical result is publishable, but the claim needs reining in or the experiment needs another arm. The second is a lot of work; the first is just honest writing. I'd send it to referees rather than desk reject, because the framework extension and the controlled comparison are useful to the adaptive-UI community, and the overclaim is correctable. A serious referee could push them to fix the framing and tighten the carryover discussion. I wouldn't cite the main claim as established, but I'd keep the paper in mind as an example of per-user RLHF in UI adaptation.","headline":"Genuinely useful empirical work on per-user RLHF for UI adaptation, undermined by an abstract that claims more than the design can show.","tokens_in":15963,"tokens_out":2934,"would_cite":false,"duration_ms":30959,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tying UI adaptation to a per-user reinforcement learning agent improves satisfaction and engagement.","keywords":["Adaptive user interfaces","Reinforcement learning","Human feedback","User experience","Personalized RL agents","User satisfaction","User engagement","Crossover experiment"],"falsifier":"Run the same experiment with a third condition: an adaptive interface using the same RL pipeline but with only the predictive HCI reward and no personalized preference modifier; if participants rate this no-feedback adaptive arm as high as the personalized-feedback arm, the central claim about human feedback is falsified.","tokens_in":14907,"feed_emoji":"🖥️","tokens_out":5508,"duration_ms":54549,"temperature":0.7,"pith_summary":"This paper tries to establish that incorporating personalized human feedback into a reinforcement-learning-driven UI adaptation framework measurably improves user experience. It does so by training a separate reinforcement learning agent for each participant, with a reward signal built from a general predictive HCI model plus a per-user preference model derived from pairwise video comparisons. In a crossover experiment with 33 participants across e-learning and trip-planning domains, the adaptive interfaces scored significantly higher than non-adaptive interfaces on both user satisfaction and user engagement, with no significant domain effect. If right, this supports the value of individualized human-in-the-loop reward design for adaptive interfaces.","feed_headline":"Personalized RL agents lift UX over static interfaces","feed_subtitle":"In a 33-user crossover study, per-user feedback-trained adaptive UIs beat static ones on satisfaction and engagement.","key_machinery":"The load-bearing mechanism is a dual-source reward model: the reinforcement learning agent's reward is the sum of a baseline predictive HCI model (general engagement prediction) and a personalized reward modifier, a fully connected multilayer perceptron trained on the participant's pairwise preferences over 64 adaptation video clips. This per-user modifier shifts the policy so each agent is unique; agents are trained with GA3C, an asynchronous actor-critic method. The evaluation machinery is a balanced two-treatment, two-domain crossover design analyzed with a linear mixed model that includes period, sequence, and carryover-with-sequence, treating subject as a random effect.","core_discovery":"The paper's claim is that a UI adaptation system that learns a unique reinforcement-learning policy for each individual, using that person's stated preferences as a reward modifier, produces a significantly better user experience than a fixed, non-adaptive interface. The empirical evidence is two linear mixed models in which the non-adaptive condition is estimated 0.97 points lower on satisfaction (95% CI [-1.20, -0.74], p < 0.001) and 0.23 points lower on engagement (95% CI [-0.41, -0.04], p = 0.016); domain is not significant and the design shows no carryover. The paper interprets this as a step beyond its earlier general human-feedback model, which failed to reach significance, and as evidence that personalization at the level of individual reward models matters.","pith_inferences":["Editorial inference: because the design has no adaptive-without-feedback control arm, the strongest statement the data license is that RL-based personalized adaptation beats static; isolating the contribution of human feedback itself would require a third arm with the same RL adaptation but no personalized reward modifier.","Editorial inference: the per-user training cost is substantial (roughly one million steps per user over three days on a high-end GPU), so practical deployment would likely need shared or cluster-based preference models; the paper itself lists cluster-based models as future work.","Editorial inference: the pairwise video-comparison setup is an offline preference elicitation; a natural next test is whether feedback collected during live interaction, rather than from pre-generated clips, strengthens or changes the observed effects."],"forward_implications":["Adaptive interfaces trained with a per-user reward modifier can be expected to outperform static interfaces on satisfaction and engagement, at least in short controlled tasks.","The absence of a significant domain effect suggests the benefit transfers between e-learning and trip-planning, at least for the adaptation actions studied.","Individual differences dominate the model (ICC 0.76 for satisfaction and 0.55 for engagement), implying that user-specific random variation carries much of the observed variance.","The technique effect on satisfaction (-0.97 on a 10-point scale) is larger relative to its scale than the effect on engagement (-0.23 on a 5-point scale), so personalized adaptation may shape perceived satisfaction more strongly than engagement."],"supporting_citations":[{"why":"Supplies the original RL-based UI adaptation framework and replication package that this paper extends.","marker":"[16]"},{"why":"The authors' earlier empirical study with a general, non-personalized human-feedback model; its non-significant result motivates individualization.","marker":"[15]"},{"why":"Provides the GA3C GPU-parallel actor-critic algorithm used to train each participant's unique RL agent.","marker":"[4]"},{"why":"Underpins the dual-source reward and preference-modelling mechanism for deep RL from human preferences.","marker":"[12]"},{"why":"Supplies the taxonomy of human feedback encodings that motivates the comparative-feedback collection format.","marker":"[38]"},{"why":"Provides the crossover-design and linear-mixed-model guidelines that structure the analysis.","marker":"[33]"},{"why":"The QUIS questionnaire used to measure user satisfaction.","marker":"[26]"},{"why":"The UES questionnaire used to measure user engagement.","marker":"[28]"}],"fun_headline_variants":["Per-user RL beats static UI in UX study","Feedback-trained RL UIs top static for satisfaction","Personalized RL policies lift UX: engagement too","Individual RL agents improve adaptive UI over fixed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that human feedback itself drives the UX gains assumes that the adaptive and non-adaptive conditions differ only in the personalized feedback, yet no adaptive-without-feedback arm was run and group is confounded with carryover.","fun_headline_variants_meta":{"raw":{"variants":["Per-user RL beats static UI in UX study","Feedback-trained RL UIs top static for satisfaction","Personalized RL policies lift UX: engagement too","Individual RL agents improve adaptive UI over fixed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1308,"prompt_tokens":950,"completion_tokens":358,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":312}},"tokens_in":566,"tokens_out":358,"duration_ms":4516,"temperature":1.0,"reasoning_tokens":312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:19:59.638864+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same experiment with a third condition: an adaptive interface using the same RL pipeline but with only the predictive HCI reward and no personalized preference modifier; if participants rate this no-feedback adaptive arm as high as the personalized-feedback arm, the central claim about human feedback is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The QUIS questionnaire used to measure user satisfaction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The authors' earlier empirical study with a general, non-personalized human-feedback model; its non-significant result motivates individualization."},{"cited_title":"Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback","cited_arxiv_id":"2402.02423","evidence_quote":"Supplies the taxonomy of human feedback encodings that motivates the comparative-feedback collection format."}],"review_version":1}