{"id":"a4917745-9f2b-4886-9fa0-f397cbbe9fed","arxiv_id":"2411.12255","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A hierarchical robot imitation-learning model with a proportional error-feedback term improves tracking of pre-recorded upper-layer outputs, especially when the lower layer is a stateless MLP rather than an LSTM.","lead":"Researchers added a feedback loop to a two-level neural network for robot imitation learning, feeding the difference between the current output and a target state back into the network's input. In writing tests with a robot arm, the loop improved tracking of pre-recorded trajectories for one unlearned character, but results were mixed and the target states were assumed to be known in advance.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Angular Error metric in Eq. (3) is a signed cumulative sum, so positive and negative deviations cancel; the reported tracking improvements may be an artifact and the central claim is not yet supported.","rationale":"The reader's identified weakest assumption (upper-layer outputs are known in advance, Section III.C) is a real scope limitation and is explicitly acknowledged by the authors, who defer the full learned upper layer to future work. I agree with the conditional verdict but think the more immediately damaging issue is that the headline quantitative evidence for 'tracking improvement' is based on a non-magnitude metric. This is internal to the evaluation and can be checked from the already-collected data. If the Angular Error decreases are reproduced by RMSE/MAE, the paper's scoped claim remains plausible; if not, the central claim is unsupported. I therefore keep the verdict unchanged rather than moving to reject, because the proposed mechanism is simple and the IoU improvement for '4' provides some partial support.","tokens_in":7742,"tokens_out":9007,"duration_ms":95071,"concrete_test":"Recompute tracking error from the recorded joint-angle time series behind Fig. 9 as per-joint RMSE and MAE over the task horizon (and, if possible, normalized by task duration), comparing w/ feedback vs. w/o feedback for MLP and LSTM on 'A', '4', and 'B'. If the ordering or sign of the feedback effect changes relative to the reported Angular Error, that confirms cancelling errors; if RMSE/MAE also improve for the same conditions, the tracking claim survives this objection. Reporting the histogram of signed one-step errors would make the cancellation explicit.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing problem is not the acknowledged use of recorded upper-layer states, but the metric used to measure the claimed improvement. Eq. (3) defines Angular Error as Σ(θ^upper − θ^res) with no absolute value, no square, and no division by the number of samples, despite the subsection being titled 'MSE of Angles.' A signed cumulative sum can be near zero even when the robot tracks poorly, because positive and negative deviations cancel over time or across joints. Consequently, the widespread decreases in Angular Error in Fig. 9—e.g., for 'B', MLP w/ feedback gives 3.31 vs. 6.27 without feedback while its IoU actually falls from 0.035 to 0.025—do not establish that feedback improves tracking. Since the IoU evidence is positive for only one untrained character ('4'), the central claim largely rests on a metric that may simply reflect error-sign balance. This is a correctness risk in the paper's own experimental setting, not in the unvalidated full-system extension.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes an error-feedback mechanism for hierarchical bilateral-control-based imitation learning. The upper layer supplies future state plans, and the lower layer predicts the next state from the current state and the upper-layer plan. The difference between the lower-layer prediction and the one-step-ahead upper-layer output is added to the upper-layer input (Eq. (1)). The lower layer is implemented either as an LSTM or as a stateless MLP, and experiments are performed on a character-writing task with characters 'A', '4', and 'B', using pre-recorded upper-layer outputs. The authors report IoU and a quantity called Angular Error, and claim that the feedback model improves tracking of the upper-layer outputs and improves accuracy for untrained characters, particularly with the MLP lower layer.","tokens_in":8014,"tokens_out":6161,"duration_ms":64149,"significance":"If the claims were fully supported, the idea of deliberately using a stateless MLP as the low-level controller in a hierarchical network and closing a simple error-feedback loop is a useful step toward integrating neural networks with control-theoretic ideas. The authors are transparent about using pre-recorded upper-layer states and about the limitation that a learned upper layer is not tested. However, the current experimental evidence is not sufficient to establish the central claim: the main angular metric is mathematically flawed, the independent IoU evidence is positive for only one untrained character, and no statistical tests accompany the five-run comparisons. The contribution is therefore conditional on a corrected and more rigorous evaluation.","major_comments":[{"comment":"The Angular Error metric as written is a signed cumulative sum, not a mean squared error despite the subsection title. Positive and negative deviations cancel over time and across joints, so a small or decreased value does not establish accurate tracking. This is load-bearing: in Fig. 9, character 'B' with the MLP shows IoU decreasing from 0.035 to 0.025 while Angular Error decreases from 6.27 to 3.31, the opposite of what improved tracking should produce. The authors should replace Eq. (3) with a per-sample absolute or squared error, normalize by sample count, and report per-joint or otherwise non-cancelling summary statistics before interpreting Angular Error decreases as evidence of improved tracking. In addition, because the feedback signal in Eq. (1) is the same one-step difference being measured in Eq. (3), a reduction in this metric is partly a check that the loop is doing what it was designed to do rather than an independent validation.","section":"Section IV.D.2, Eq. (3)."},{"comment":"The claim that the model improves accuracy for previously untrained characters is supported by only one clear IoU increase: the MLP on '4' (0.127 to 0.204). The LSTM on '4' improves only slightly (0.034 to 0.045), and for 'B' there is no IoU improvement; the MLP's IoU actually decreases from 0.035 to 0.025. With only five runs and overlapping standard deviations, and without significance tests, the evidence is too thin for the abstract's statement that the model demonstrated improved accuracy in writing previously untrained characters. The authors should add appropriate statistical testing or restrict the claim to the specific condition where the IoU evidence is positive.","section":"Section V.B, Fig. 9."},{"comment":"The evaluation assumes that upper-layer outputs are known in advance from pre-recorded bilateral control, as the authors explicitly acknowledge. This tests only whether the lower layer can track a perfectly known reference; it does not test closed-loop behavior when the upper layer is a learned model whose predictions contain errors. The abstract's statement that 'Through autonomous control with error feedback, we confirmed that the lower layer could effectively track the output of the upper layer' should be qualified to this oracle-upper-layer setting, and the paper should not imply that the full autonomous hierarchical system has been validated.","section":"Section III.C and Section V.B."}],"minor_comments":[{"comment":"The sentence 'In the character-writing task, this model demonstrated improved accuracy in writing previously untrained characters.' appears twice in the abstract and should be deduplicated.","section":"Abstract."},{"comment":"The notation f^{upper} is overloaded: the left side is a new input vector while the right side contains both f^{upper}_{k+10} and f^{upper}_{k+1}; the authors should define the dimension and timing of each quantity more explicitly.","section":"Section III.B, Eq. (1)."},{"comment":"The heading 'MSE of Angles' is inconsistent with Eq. (3), which is neither a mean nor a squared error; if a mean squared error was intended, the equation should be corrected accordingly.","section":"Section IV.D.2."},{"comment":"There are grammatical errors in the future-work section, such as 'an lower-layer'; the manuscript should be proofread for articles and plural forms.","section":"Section VII."},{"comment":"The results in Figs. 8 and 9 are presented in a compressed table-like format with small numeric values; a regular table with clear column headers would improve readability and reduce transcription errors.","section":"Figures 8 and 9."}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the authors are honest about the oracle upper-layer assumption, but the main angular-error metric is flawed and the independent IoU evidence is limited to one condition. I would invite a revision that corrects the metric, adds statistical analysis, and tempers the claims accordingly; without those changes the paper is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my read on arXiv:2411.12255. The core idea is simple and worth a look: add an error-feedback term to the lower layer of the hierarchical bilateral-control imitation learning model from Hayashi et al. [8]. The lower layer gets the upper layer's ten-step-ahead state plus the difference between the one-step-ahead prediction and its own output. The twist is using a stateless MLP instead of an LSTM, on the rationale that a Markovian lower layer is easier to control. That is a reasonable extension, and the observation that the MLP benefits more from feedback than the LSTM is genuinely interesting.\n\nThe paper does a few things well. It clearly separates the Markovian and non-Markovian parts of the task, it uses IoU of the drawn character as an independent metric, and it explicitly acknowledges that the upper-layer states are pre-recorded rather than produced by a learned upper model. The comparison framework between LSTM and MLP is clean.\n\nThe soft spot is load-bearing and it is in the evaluation. Equation (3) defines Angular Error as a signed cumulative sum of (θ_upper − θ_res) with no absolute value, no square, and no normalization. The section is titled 'MSE of Angles,' but the formula is not an MSE. Positive and negative deviations cancel, so a low Angular Error can mean the trajectory oscillates around the reference rather than tracking it. That is exactly what the results suggest. For character 'B', the MLP with feedback shows a big drop in Angular Error (3.31 vs 6.27) while IoU actually falls from 0.035 to 0.025. If the metric were a true MSE or MAE, the improvement would likely shrink or disappear. With only one untrained character showing an IoU gain (MLP for '4': 0.204 vs 0.127), the evidence for the paper's central claim is thin. There are also no significance tests over the five runs, and no code or data released.\n\nWho should read this? Researchers working on bilateral-control imitation learning or hierarchical models with feedback might find the idea worth exploring. The paper deserves a serious referee, but as it stands it needs major revision: fix the metric, report per-time-step errors or an uncancelled statistic, and show that the IoU gains are consistent across more than one character and robust to reruns. If the authors can do that, the contribution becomes credible.\n\nMy recommendation: send to peer review with a request for heavy revision, not desk reject. The idea is not flawed; the evidence is.","headline":"Plausible error-feedback idea for hierarchical imitation learning, but the main tracking metric is a signed sum that can cancel out, so the central claim is not yet supported.","tokens_in":8483,"tokens_out":3137,"would_cite":false,"duration_ms":30996,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Feeding predicted-state errors back into the lower-layer network improves a robot's ability to write characters it never trained on.","keywords":["error feedback","imitation learning","bilateral control","hierarchical neural network","stateless MLP","LSTM","character writing"],"falsifier":"Run the identical writing task with a learned upper-layer neural network (trained to predict states) instead of recorded data, and measure Angular Error and IoU for characters '4' and 'B'; if the MLP-with-feedback configuration no longer beats the no-feedback baseline, the claimed benefit is an artifact of using ground-truth reference commands.","tokens_in":7545,"feed_emoji":"✍️","tokens_out":7606,"duration_ms":67079,"temperature":0.7,"pith_summary":"In bilateral control-based imitation learning, a neural network predicts the robot's next command from its current state, but a feedforward network has no way to correct its own output errors during autonomous operation. The paper proposes a hierarchical structure in which an upper layer supplies long-horizon state predictions and a lower layer predicts the next state, with the difference between the lower output and the upper one-step prediction fed back into the lower layer's input. The authors show that this error-feedback model reduces angular tracking error and, for one unlearned character, improves the drawn shape's overlap (IoU). They also find that a stateless MLP benefits more from feedback than an LSTM, because it has no internal memory to carry old errors forward. The work is a step toward treating the lower layer as a controllable plant rather than a fixed feedforward mapping.","feed_headline":"Error feedback lets a robot write characters it never trained on","feed_subtitle":"A two-layer model with a stateless MLP tracks planned motion closely enough to improve drawing of unseen characters.","key_machinery":"The central mechanism is the error-feedback model: the upper layer outputs a one-step predicted state $f^{upper}_{k+1}$ and a ten-step target $f^{upper}_{k+10}$; the lower layer outputs $\\hat{f}_{k+1}$; the correction $e = f^{upper}_{k+1} - \\hat{f}_{k+1}$ is added to the ten-step target, giving $f^{upper} = f^{upper}_{k+10} + e$, and this corrected value is fed to the lower layer as the goal to track. The lower layer is a four-layer MLP with no internal state, which keeps the loop Markovian and easy to control, and it is compared against a three-layer LSTM lower layer. The mechanism is applied only during autonomous operation, not during training.","core_discovery":"The central claim is that output errors in a hierarchical imitation-learning network can be corrected by adding a feedback term $f^{upper}_{k+1} - \\hat{f}_{k+1}$ to the upper-layer command that the lower layer receives. With this feedback, the lower layer tracks the upper layer's predicted states more closely than in the conventional hierarchical model. On the learned character 'A', feedback did not raise IoU but did lower Angular Error; on the unlearned character '4', feedback with an MLP lower layer increased IoU and cut Angular Error roughly in half; on 'B', feedback again reduced Angular Error, with MLP best, while IoU stayed low because the pen lost contact with the board. The paper interprets the MLP advantage as a Markovian property: without an internal state, the lower layer responds to the current feedback rather than being influenced by past memories.","pith_inferences":["The paper's oracle assumption means the real benefit of feedback with a learned upper layer remains untested; a concrete next experiment is to close the loop with a trained upper-layer network and see whether the feedback still stabilizes tracking.","The same feedback idea could be applied to other hierarchical policy architectures with a slow planner and a fast executor, where a stateless executor with error correction behaves like a classical servo tracking a reference trajectory.","Because the correction is just a proportional error term, the framework invites connections to PID control: adding integral or derivative terms on the error $e$ may improve tracking on trajectories like 'B' where the lower layer must extrapolate far from its training distribution.","The MLP's advantage suggests that for feedback-based control, intentionally removing recurrency and trading it for a shorter sampling period can be a design choice rather than a limitation."],"forward_implications":["The error-feedback model can be applied to any hierarchical imitation-learning setup where an upper layer provides multi-step predictions, without retraining the lower layer.","Using a stateless MLP lower layer yields better tracking than an LSTM when feedback is present, because the correction is not diluted by internal memory.","The model suppresses errors from extrapolating to unlearned motions: on character '4', feedback with MLP raised IoU and halved angular error.","The feedback correction works even when the lower layer is fed with upper-layer outputs for characters never seen in training.","Tracking the upper layer more closely reduces joint-angle error while not always improving the drawn shape, indicating that shape quality also depends on contact with the writing surface."],"supporting_citations":[{"why":"Supplies the independently learnable hierarchical model that this paper extends with error feedback.","marker":"[8]"},{"why":"Supplies the bilateral control-based imitation learning scheme used to collect the training demonstrations.","marker":"[4]"},{"why":"Supplies the cross-structure hand hardware used for the character-writing task.","marker":"[6]"},{"why":"Supplies the reaction-force estimation observer used to obtain torque estimates for the upper-layer outputs.","marker":"[13]"},{"why":"Supplies the time-series data augmentation and sampling strategy used to build the training set.","marker":"[14]"}],"fun_headline_variants":["Feedback fixes robot writing errors on new characters","Error feedback sharpens robot drawing of unseen letters","Robot learns to write new characters via error feedback","Feedback improves robot writing on unfamiliar characters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole demonstration assumes the upper-layer outputs are already known and taken from pre-recorded bilateral control data, so the feedback loop has never been tested against errors produced by a learned upper-layer model.","fun_headline_variants_meta":{"raw":{"variants":["Feedback fixes robot writing errors on new characters","Error feedback sharpens robot drawing of unseen letters","Robot learns to write new characters via error feedback","Feedback improves robot writing on unfamiliar characters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000284,"raw_usage":{"total_tokens":1639,"prompt_tokens":872,"completion_tokens":767,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":711}},"tokens_in":488,"tokens_out":767,"duration_ms":8472,"temperature":1.0,"reasoning_tokens":711,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:45:02.511641+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical writing task with a learned upper-layer neural network (trained to predict states) instead of recorded data, and measure Angular Error and IoU for characters '4' and 'B'; if the MLP-with-feedback configuration no longer beats the no-feedback baseline, the claimed benefit is an artifact of using ground-truth reference commands.","supporting_citations":[{"cited_title":"An independently learnable hierar- chical model for bilateral control-based imitation learning applications,","cited_arxiv_id":null,"evidence_quote":"Supplies the independently learnable hierarchical model that this paper extends with error feedback."},{"cited_title":"Imitation learning for nonprehen- sile manipulation through self-supervised learning considering motion speed,","cited_arxiv_id":null,"evidence_quote":"Supplies the bilateral control-based imitation learning scheme used to collect the training demonstrations."},{"cited_title":"Soft and rigid object grasping with cross-structure hand using bilateral control-based imitation learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the cross-structure hand hardware used for the character-writing task."},{"cited_title":"Torque sensorless control in multidegree-of-freedom manipulator,","cited_arxiv_id":null,"evidence_quote":"Supplies the reaction-force estimation observer used to obtain torque estimates for the upper-layer outputs."},{"cited_title":"From virtual demonstration to real-world manipulation using lstm and mdn,","cited_arxiv_id":null,"evidence_quote":"Supplies the time-series data augmentation and sampling strategy used to build the training set."}],"review_version":1}