{"id":"ff72ddeb-9c07-4ced-b159-a09ae42517f3","arxiv_id":"2412.20297","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A smart-fabric LLM agent that learns from textual feedback via a token-level DualCUT alignment method, with an 11.3% Overcooked-AI score gain claimed.","lead":"The paper introduces FaGeL, an embodied agent that combines smart fabric sensing with large language models to autonomously generate tasks and adapt them through feedback. It also proposes DualCUT, a token-level alignment algorithm, and reports an 11.3% score improvement over a baseline in a virtual cooking game.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 11.3% improvement is not auditable, and the Overcooked experiments cannot separate the DualCUT alignment mechanism from generic self-training on the agent's own annotations.","rationale":"The reader's verdict is REJECT with HIGH confidence. My stress-test agrees with the rejection but sharpens the reason. The strongest claim is the 11.3% number, and that number is both unreported and uncomputable from the paper. The surrounding description also fails to provide the minimal scientific controls needed to attribute the gain to the proposed mechanism. The annotator loop means the 'human feedback' is self-generated, so any alignment claimed is with the model's own judgments, not with human values. This is not an external-consensus issue; it is an internal-evidence issue: the paper's own text permits LLM-drafted feedback, and the experiment uses exactly that. A proper control would separate the effect of extra training from the effect of the DualCUT loss. Since no such control is present, the central claim is unsupported. The reader's weakest_assumption about self-generated reflections being a proxy for human feedback is on point, and I expand it by emphasizing the missing quantitative baseline and the missing ablation. Hence the verdict remains REJECT (UNCHANGED).","tokens_in":12802,"tokens_out":6598,"duration_ms":61133,"concrete_test":"Ask the authors to release the per-game scores for the 10 games for ProAgent and FaGeL-evolution (with seeds), and to run two controls: (i) ProAgent fine-tuned with the same number of MLE updates on its own successful trajectories (no DualCUT, no annotations), and (ii) FaGeL-evolution with the annotator's feedback labels randomly permuted. If control (i) or (ii) reproduces the 11.3% gain within noise, the improvement is not attributable to DualCUT's token-level alignment. Report mean±std over the 10 games and the exact formula used for the 11.3%.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim (Contribution 5: '11.3% scoring performance improvement in 10 games') is not backed by any reproducible measurement in §V-A. No table lists per-game scores, means, standard deviations, seeds, or the formula behind the 11.3%; Fig. 5 is a single-episode score trace and Fig. 4 reports average timesteps per completion, not the claimed score increase. The second, more conceptual weakness is the lack of an ablation that isolates the DualCUT mechanism. In §IV-D, the reflector/annotator generates analysis annotations from the agent's own evaluation of the current state, and §V-A says the model is updated with DualCUT every 1000 timesteps. The baseline ProAgent is frozen, so the comparison conflates 'DualCUT on self-generated reflections' with 'any additional parameter updates on the agent's own outputs'. The improvement could come from the extra gradient steps on in-distribution self-generated text, from the annotator reinforcing the model's prior mistakes, or from the token-level contrastive loss. Only the last of these is the paper's claimed contribution. Furthermore, because the paper explicitly allows feedback to be 'drafted by the LLM' (§III-A), the 'implicit human feedback' claim is not demonstrated: no human annotations or preferences enter the loop, so the alignment result is circular with respect to the model's own judgments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes FaGeL, an embodied agent combining smart-fabric sensing with an LLM-based controller. The agent is intended to autonomously mine user needs from multimodal sensor data, generate tasks, and align its behavior through textual feedback processed by a new DualCUT algorithm that extends Contrastive Unlikelihood Training with separate positive and negative token-level signals. The paper reports a case study of a sleep-disorder scenario, a t-SNE visualization of generated tasks, and simulated Overcooked-AI experiments in which FaGeL-evolution is claimed to improve ProAgent's score by 11.3% over 10 games. A token-level saliency map is proposed as an interpretability tool.","tokens_in":13095,"tokens_out":5446,"duration_ms":47055,"significance":"If the findings were reliably established, the paper would advance the underexplored problem of aligning embodied agents through implicit, non-intrusive feedback, and the DualCUT algorithm could be a useful token-level alternative to response-level alignment methods. The paper has several genuine strengths: it clearly formulates the problem of textual feedback for alignment, provides a concrete extension of CUT with positive and negative token identification, and describes a full system vision from fabric sensing to task execution. However, the current validation does not establish the core claims: the headline performance number is not auditable, the feedback in the simulated experiments is generated by the agent's own annotator rather than by humans, and no ablation isolates the contribution of DualCUT. These limitations are not presentation details; they concern the validity and interpretability of the central results.","major_comments":[{"comment":"The headline claim that FaGeL achieves an 11.3% scoring performance improvement in 10 games is not backed by any reproducible measurement. No table lists per-game scores, means, standard deviations, seeds, or the formula defining the 11.3% figure. Figure 5 is a single-episode score trace, and Figure 4 reports average timesteps per completion, which is not the claimed scoring improvement. Without this information, the central quantitative result cannot be verified or compared with future work.","section":"§V-A (Contribution 5, p. 3)"},{"comment":"The feedback used in the Overcooked-AI experiments is generated by the agent's own annotator rather than by a human user. Section IV-D states that the agent 'reflects on whether there were reasoning errors in the previous time slice' and that 'this reflection generates analysis annotations'; Section III-A explicitly permits feedback to be 'drafted by the LLM based on the analysis of the current state.' Consequently, the experiments do not demonstrate alignment with human preferences; they demonstrate self-training on the model's own judgments. The claim of 'implicit human feedback' is therefore circular with respect to the model's internal estimates.","section":"§IV-D and §III-A"},{"comment":"The evaluation does not isolate the DualCUT mechanism from generic self-training. The baseline ProAgent is frozen, while the FaGeL variant receives additional parameter updates every 1000 timesteps on self-generated annotations. Any improvement could be due to the extra in-distribution gradient steps, the MLE term in Eq. (9) that reinforces non-U tokens, or the contrastive/unlikelihood losses. Without an ablation that applies, for example, standard fine-tuning or the original CUT to the same self-annotated data, the specific contribution of DualCUT to the reported improvement is not established.","section":"§V-A vs. §III-C"},{"comment":"The algorithm depends on eight hyperparameters (λ1, λ2, λ3, λ4, α, β, σ1, σ2), but the manuscript does not report their values or a sensitivity analysis. Since the token-selection criteria and the dynamic scaling terms in Eqs. (7) and (8) are highly sensitive to these thresholds, the reader cannot judge whether the reported behavior is robust or an artifact of particular settings.","section":"§III-C, Eqs. (5) to (9)"},{"comment":"The token-level saliency map is presented as the first such visualization during LLM training, but it appears to display the model's own next-token probabilities under different feedback conditions. As such, it is a visualization of the model's internal probability assignments, not an independent explanation of why certain tokens are selected. The paper does not compare this saliency map with established interpretability tools (e.g., gradient-based or attention-based attributions), nor does it validate that the visualized saliency corresponds to human-interpretable causes.","section":"§V-B, Fig. 6"},{"comment":"The validation of task mining is limited to a t-SNE plot of 1000 generated tasks without any quantitative metric (e.g., diversity, relevance, or user agreement), and the DPO-based user preference initialization is described without reporting the rating dataset size, training details, or evaluation. The case study in Table I is anecdotal and does not demonstrate that the system's outputs improve over time in a measurable way.","section":"§IV-C and §V"}],"minor_comments":[{"comment":"The section heading appears as 'R ALATED WORK' in the provided manuscript; this should be corrected to 'RELATED WORK.'","section":"Section II heading"},{"comment":"The indicator function uses the notation x → y without a definition; the meaning of the arrow in the indicator is unclear and should be specified.","section":"Eq. (3)"},{"comment":"The footnote states that ProAgent surpasses its previous SOTA by 0.51% in the Cramped Room scenario, but no source or experimental basis is provided for this number.","section":"Footnote to Contribution 5"},{"comment":"Several references are incomplete (e.g., [1], [9], [35] lack full bibliographic details such as venue, volume, or page numbers), which hinders reproducibility.","section":"References"},{"comment":"The caption of Figure 2 lists multiple subfigures (a) through (d) but does not explain the content of each subfigure; a legend or detailed caption would improve clarity.","section":"Figure 2"},{"comment":"The term 'intelligence evolution' is used repeatedly without a formal definition; the paper would benefit from a precise problem statement defining the evolution objective and its success criteria.","section":"§III (general)"}],"recommendation":"reject","confidential_remarks":"The paper's central claims require human-in-the-loop validation and reproducible experimental details; the current manuscript does not meet the standard for publication. The authors might be encouraged to refocus the contribution on the DualCUT algorithm with rigorous offline experiments, an ablation against simple fine-tuning, and a human-subject study of the implicit-feedback claim. As it stands, the self-generated feedback loop makes the 'human-machine collaboration' contribution largely unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line first: this paper has a real kernel of an idea—DualCUT extends CUT with a positive-token criterion and an extra threshold—but the evidence for it is much weaker than the claims. The fabric-sensing and task-mining material is largely framing; the only quantitative payoff is the 11.3% Overcooked improvement, and that number is not supported by anything reproducible in the text.\n\nWhat is genuinely new: the DualCUT equations are a nontrivial extension of CUT, and the authors describe the token-level positive/negative weighting mechanism clearly. The sleep-disorder case study is a useful end-to-end illustration, and the architecture figures make the system easier to understand than many comparable papers. I give credit for that.\n\nThe soft spots are the ones the stress test flags, and they are not minor. First, the Overcooked evaluation has no per-game table, no means, no standard deviations, no seeds, and no hyperparameter values. Figure 4 shows average timesteps per completion; Figure 5 shows a single episode. The 11.3% in Contribution 5 is a bare assertion.\n\nSecond, and more seriously, the 'implicit human feedback' in the experiment is generated by the agent's own reflector/annotator. Section III-A explicitly allows feedback to be drafted by the LLM, and in the Overcooked setup that is exactly what happens. So the experiment tests self-training on the model's own reflections, not human-agent alignment. The baseline ProAgent is frozen, so the comparison conflates 'DualCUT on self-generated annotations' with 'any extra gradient updates on the model's own outputs.' Without an ablation that separates the DualCUT loss from generic self-training, the central mechanism is unproven.\n\nThe claim of being first with token-level saliency maps is also overbroad; no comparison with existing interpretability methods is given.\n\nWho should read this: people working on CUT-style alignment or non-intrusive feedback might find DualCUT worth discussing, and the circularity is a useful cautionary example for evaluation design. But nobody should cite the 11.3% number.\n\nMy recommendation: send it to peer review rather than desk-reject—there is enough of a concrete algorithm to merit referee scrutiny—but mark it for major revision and insist on real human annotations, a proper ablation, and full reporting of variance and hyperparameters.","headline":"A plausible token-level alignment extension (DualCUT) wrapped in an embodied-agent story, but the headline 11.3% result is not auditable and the 'human feedback' loop is actually self-feedback.","tokens_in":13596,"tokens_out":3071,"would_cite":false,"duration_ms":30183,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An embodied LLM agent improves cooperative play by 11.3% using only its own textual reflections as feedback.","keywords":["smart fabric","embodied agent","large language models","AI alignment","implicit feedback","DualCUT","token-level saliency map","Overcooked-AI"],"falsifier":"Rerun the Overcooked evolution loop with feedback annotations whose polarity is inverted, praising errors and criticizing successes; if the agent still improves by about 11%, the improvement cannot be from aligning with feedback content, and the central claim is falsified.","tokens_in":12601,"feed_emoji":"🤖","tokens_out":5107,"duration_ms":42265,"temperature":0.7,"pith_summary":"The paper tries to establish that an embodied agent can improve its collaborative behavior using only implicit textual feedback, without explicit ratings or preferences. It introduces FaGeL, which combines smart fabric sensors with an LLM to sense the user's state, generate tasks, and refine its policy. The central experiment shows an 11.3% score improvement over the ProAgent baseline in an Overcooked-AI cooperation game, using the agent's own reflections as feedback. If true, this suggests a path toward agents that adapt to human preferences in daily environments without demanding constant evaluation.","feed_headline":"Self-reflecting fabric agent beats ProAgent by 11.3%","feed_subtitle":"Smart-fabric agent improves Overcooked-AI scores 11.3% using only its own textual reflections.","key_machinery":"The central mechanism is DualCUT, a token-level alignment loss built on Contrastive Unlikelihood Training. It uses two contrastive criteria, one for erroneous tokens and one for satisfactory tokens, based on differences in token probabilities under negative feedback, positive feedback, and no feedback. Dynamic sigmoid weights scale penalties and rewards, and a token-level saliency map shows which tokens are reinforced or suppressed during fine-tuning.","core_discovery":"The central claim is that the DualCUT algorithm lets an LLM-driven embodied agent align its behavior with human preferences by exploiting both positive and negative textual feedback at the token level. DualCUT extends Contrastive Unlikelihood Training with explicit criteria and dynamic weights for identifying good and bad tokens, and a token-level saliency map visualizes the fine-tuning. In the Cramped Room Overcooked-AI setup, the FaGeL evolution algorithm, starting from ProAgent, achieved an 11.3% improvement over 10 games purely from observation and self-generated annotations, with no human guidance.","pith_inferences":["The reported gain likely conflates self-reflection with human alignment, since the feedback is generated by the same model being trained; a human-in-the-loop version may show a smaller or different effect.","Token-level saliency visualization could be used to compare other alignment methods, such as DPO or RLHF, and expose which tokens they reinforce.","A testable extension is to replace the annotator with real human-written feedback and measure whether the 11.3% gain persists.","The single Cramped Room layout leaves open whether the improvement generalizes to other Overcooked layouts or longer horizons."],"forward_implications":["Embodied agents can improve collaborative performance from observation alone, without explicit user ratings.","Token-level saliency maps give developers a concrete view of which words in an instruction or feedback drive alignment.","Smart fabric sensing provides a non-intrusive way to collect the contextual data that feeds implicit feedback.","The DualCUT approach could transfer to other cooperative or instruction-following tasks with textual feedback.","The result suggests that a zero-shot planner like ProAgent can be refined online through self-reflection."],"supporting_citations":[{"why":"Serves as the state-of-the-art baseline ProAgent that the FaGeL evolution starts from and beats.","marker":"[9]"},{"why":"Contrastive Unlikelihood Training, the alignment method DualCUT directly extends.","marker":"[19]"},{"why":"Overcooked-AI, the cooperative benchmark environment used for all performance experiments.","marker":"[34]"},{"why":"Llama3-8b-instruct, the base language model used in the Overcooked experiments.","marker":"[36]"},{"why":"The Wearable 2.0 hardware setup that FaGeL's fabric sensing module follows.","marker":"[35]"}],"fun_headline_variants":["Fabric agent learns by self-reflection, beats ProAgent by 11.3%","Self-reflecting fabric agent boosts Overcooked-AI score 11.3%","DualCUT enables fabric LLM agent to self-improve 11.3%","Fabric-powered agent evolves via implicit feedback, wins 11.3%","Token-level alignment lifts fabric agent 11.3% over ProAgent"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the annotator's self-generated textual reflections are a valid proxy for genuine human feedback; in the Overcooked experiments the 'human feedback' is produced by the agent's own annotator component, not by a person.","fun_headline_variants_meta":{"raw":{"variants":["Fabric agent learns by self-reflection, beats ProAgent by 11.3%","Self-reflecting fabric agent boosts Overcooked-AI score 11.3%","DualCUT enables fabric LLM agent to self-improve 11.3%","Fabric-powered agent evolves via implicit feedback, wins 11.3%","Token-level alignment lifts fabric agent 11.3% over ProAgent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000709,"raw_usage":{"total_tokens":3168,"prompt_tokens":896,"completion_tokens":2272,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":2165}},"tokens_in":512,"tokens_out":2272,"duration_ms":15428,"temperature":1.0,"reasoning_tokens":2165,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:23:01.196922+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the Overcooked evolution loop with feedback annotations whose polarity is inverted, praising errors and criticizing successes; if the agent still improves by about 11%, the improvement cannot be from aligning with feedback content, and the central claim is falsified.","supporting_citations":[{"cited_title":"Wearable 2.0: Enabling human-cloud integration in next generation healthcare systems,","cited_arxiv_id":null,"evidence_quote":"The Wearable 2.0 hardware setup that FaGeL's fabric sensing module follows."},{"cited_title":"Proagent: Building proactive cooperative ai with large language models,","cited_arxiv_id":null,"evidence_quote":"Serves as the state-of-the-art baseline ProAgent that the FaGeL evolution starts from and beats."},{"cited_title":"Reasons to Reject? Aligning Language Models with Judgments","cited_arxiv_id":"2312.14591","evidence_quote":"Contrastive Unlikelihood Training, the alignment method DualCUT directly extends."},{"cited_title":"On the utility of learning about humans for human- ai coordination,","cited_arxiv_id":null,"evidence_quote":"Overcooked-AI, the cooperative benchmark environment used for all performance experiments."},{"cited_title":"Llama 3 model card,","cited_arxiv_id":null,"evidence_quote":"Llama3-8b-instruct, the base language model used in the Overcooked experiments."}],"review_version":1}