{"id":"9c0b32c1-5032-46bc-8af4-3151535817c8","arxiv_id":"2411.17458","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A robot policy trained on corrupted RGB plus uncorrupted depth maps kept 82-88% success from near-dark to very bright camera exposures, where baseline imitation learning collapsed.","lead":"This paper combines a new image-corruption augmentation, AugBlender, with monocular depth estimates to make imitation-learning robots tolerate changing camera exposure. If the results hold, a low-cost robot arm could keep manipulating objects under poor lighting without adding depth sensors.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported robustness gain is confounded with dataset scale/composition: no DP+CombinedDataset baseline exists, so the central attribution to AugBlender-plus-depth is unsupported.","rationale":"The reader identified the same weakest assumption: the absence of a DP+CombinedDataset baseline. I agree, and this concern is decisive for the paper's central claim. The paper explicitly separates 'data compensation' (Section 5.1) from 'method compensation' (Section 5.2), and states that the combined dataset alone improved robustness. Yet Table 1 only reports baselines trained on the original or varied datasets. Therefore the observed advantage of Ours cannot be causally attributed to AugBlender or Depth Anything V2 without an equal-data control. The paper has no code, no data release, and no error bars, but those are secondary; the confound alone is sufficient to reject the central claim as stated. I am not claiming the mechanism is false: a carefully matched experiment could well show that fused corrupted-RGB and clean-depth genuinely helps. But the burden is on the authors to provide the missing control. Since the reader's verdict is already REJECT and this concern independently supports that verdict, no adjustment is needed.","tokens_in":11432,"tokens_out":2465,"duration_ms":25027,"concrete_test":"Train vanilla Diffusion Policy on exactly the same combined dataset used for Ours (same number of demonstrations, same 62.5/37.5 fixed/varied split, same training budget and hyperparameters), then evaluate on the same 10 exposure settings with at least 20 trials per cell. If DP+CombinedDataset achieves success rates close to Ours, the reported advantage is attributable to data scale/composition rather than to the proposed perception pipeline; if it collapses at extreme exposures, the method attribution is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that fusing AugBlender-corrupted RGB with clean monocular depth produces robustness across camera exposures. Table 1 cannot support that claim because the 'Ours' condition is the only one trained on the combined dataset defined in Section 4.4: 62.5% fixed-exposure demos plus 37.5% varied-exposure demos. The baselines are trained on either the original fixed-exposure dataset or the smaller varied-exposure dataset, so dataset scale and composition differ between Ours and every baseline. The paper's own Section 5.1 shows that the combined dataset alone 'improved robustness,' and Section 5.2 argues that the proposed methods add further robustness, but no DP+CombinedDataset control is reported. Without that control, the 82-88% average success rates of Ours may be explained entirely by having more data and exposure diversity, rather than by AugBlender or depth fusion. This is the pivot between the paper's own 'data compensation' and 'method compensation' discussion, and it is the load-bearing omission for the headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a multimodal perception pipeline for real-robot imitation learning: AugBlender, a stochastic RGB augmentation scheme, plus monocular depth maps from Depth Anything V2, fused through separate FPN encoders before a transformer-based Diffusion Policy. The central empirical claim is that this pipeline maintains 82–88% average success across camera exposures from 10 to 170 ms on three manipulation tasks, whereas baselines collapse at extreme exposures. The paper also argues that the robustness comes from method compensation rather than from the larger combined training dataset used for the proposed model.","tokens_in":11672,"tokens_out":7689,"duration_ms":69757,"significance":"If the attribution were supported, the result would be practically valuable: it demonstrates a low-cost (single arm, two RGB cameras, one RTX 3090) way to improve lighting robustness in visuomotor policies, using an off-the-shelf monocular depth model and a composable augmentation module. The paper also makes a useful conceptual distinction between data compensation and method compensation, and it attempts a real-robot evaluation with 10 exposure levels. The pipeline is plausible and the low-cost setup is a strength, as is the use of an externally pretrained depth model and an explicit algorithmic description of AugBlender. However, as reported, the main attribution is not established because the proposed model is the only condition trained on the enlarged combined dataset; no control with the same data but without AugBlender or depth is reported, and no uncertainty estimates are provided.","major_comments":[{"comment":"The proposed system is the only condition in Table 1 that can be trained on the combined dataset defined in §4.4 (62.5% fixed-exposure demos plus 37.5% varied-exposure demos). The paper's own §5.1 states that the model trained on the combined dataset 'demonstrated improved robustness' relative to the original and varied datasets. Consequently, the large success-rate advantage of Ours over DP+Depth and DP+AugBlender in Table 1 is confounded with training-data scale and exposure composition. A DP+CombinedDataset baseline, trained with the same data budget and no AugBlender or depth inputs, is the minimal control needed to support the attribution in the abstract and §5. Without it, the headline statement that 'our approach significantly boosts the success rate' is not established.","section":"Table 1; §4.4; §5.1"},{"comment":"The central empirical comparison rests on success-rate point estimates with no error bars, confidence intervals, per-cell trial counts, or multiple training seeds. The text says each model was tested 20–50 times per exposure level and that 2–3 human evaluators had to agree unanimously, but Table 1 does not report n, and no inter-rater agreement statistic is given. Given that the claimed effect is large and precisely the kind of result that can be driven by one favorable seed or evaluation batch, the authors should report seed-level variance, per-cell trial counts, and a statistical comparison (or at least confidence intervals) before the robustness claim can be assessed.","section":"§4.3; Table 1"},{"comment":"The section concludes that 'increasing training data diversity alone does not enhance model robustness to varying exposure levels,' but the evidence shown is inconsistent with this conclusion. DP+Varied Data is trained on the smaller varied-exposure dataset, not on the combined dataset that §5.1 credits with improved robustness. Thus the paper's own data-compensation result—that a larger combined dataset improves robustness—undercuts the method-compensation interpretation. The missing DP+CombinedDataset row would resolve this contradiction; in its absence the paragraph's causal conclusion should be withdrawn or substantially weakened.","section":"§5.2"},{"comment":"Depth maps for training episodes were preprocessed with the ViT-B variant of Depth Anything V2, while inference uses the ViT-S variant. This train/inference mismatch changes the input distribution of the depth channel that the policy is supposed to rely on under corrupted RGB. The paper asserts that the lighter model does not 'significantly compromis[e] depth estimation quality,' but no experiment or quantitative comparison is provided. Please either use the same depth model in both phases or report a comparison showing that the mismatch has no material effect on the reported success rates.","section":"§3.1"}],"minor_comments":[{"comment":"The pseudocode has presentation issues: 'if ξ > βthen' is missing a space, and the else branch applies 'ai(xt)' without explicitly defining which augmentation ai is selected. The role of λ in line 4, where λ is set to 1 when ξ < β, should also be explained in the surrounding text.","section":"§3.2; Algorithm 1"},{"comment":"The sentence 'it is larger than prvevious dataset as we intutively mix them up' contains typos, and the three dataset sizes are not reported numerically; please give exact episode or demonstration counts for the original, varied, and combined datasets.","section":"§4.4"},{"comment":"The key AugBlender hyperparameters β, α, k, and λ are fixed (e.g., β=0.16) without any sensitivity study; since the ID/OOD balance is a central design choice, reporting at least a small sweep over β and λ would strengthen the method description.","section":"§3.2"},{"comment":"The explanation that 'present noise in the generated policy trajectories' caused CupStack failures at exposure 10 is not backed by trajectory-level evidence; either provide such evidence or phrase this passage as a hypothesis.","section":"§5.3"},{"comment":"Future Work contains spacing artifacts ('PointV oxelNet', 'V oxFormer'), and the reference list has issues: [14] and [15] are duplicates and [45] is an incomplete citation of Vaswani et al.","section":"§7; References"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the manuscript reports a real-robot study and the proposed pipeline is plausible, but the experimental design has a load-bearing confound. I recommend major revision rather than rejection because the missing DP+CombinedDataset control and the requested uncertainty estimates are obtainable with the existing setup. If the authors cannot produce the control, the paper should be reframed as a data-combination robustness study rather than a method paper. There is also an incremental-novelty concern relative to AugMix, but the empirical contribution is the main value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper asks a real question—can you make an RGB-only imitation-learning policy robust to camera exposure changes by fusing corrupted RGB with monocular depth?—and the exposure sweep from 10 to 170 ms on three real-robot tasks is the right kind of stress test. But the central attribution is not supported as presented. The Ours condition is the only one trained on the larger combined dataset (62.5% fixed + 37.5% varied exposure), and there is no DP+CombinedDataset control. The paper's own Section 5.1 says the combined dataset 'demonstrated improved robustness,' so the headline 82–88% average success could be a data-scale and data-diversity effect, not AugBlender plus depth. That is a load-bearing omission.\n\nWhat is genuinely new: the specific combination of AugBlender-style out-of-distribution color augmentation with off-the-shelf Depth Anything V2 in a Diffusion Policy is not in the cited imitation-learning literature. The intuition is sensible—corrupt the RGB during training so the policy learns to lean on depth when lighting degrades—and the setup is hardware-light (two RGB cameras, one RTX 3090). The paper also reports per-exposure success rates rather than a single aggregate, which is more informative than usual.\n\nThe soft spots are real. Beyond the missing control, there are no error bars or seed variance; 20–50 trials per exposure cell with 2–3 human evaluators and no inter-rater reliability statistics; AugBlender's beta=0.16 and other hyperparameters are fixed with no sensitivity analysis; and no code or data are provided. The DP+Varied Data baseline performing worse than vanilla DP is interesting but only strengthens the need for the combined-data control, since the model trained on the combined data is exactly the one that succeeds. Also, AugBlender is an incremental variant of AugMix, and the depth model is used off-the-shelf, so the algorithmic delta is modest.\n\nWho this is for: practitioners working on cheap robustness fixes for visuomotor policies. The paper is not ready in current form, but the question is legitimate and the missing experiment is well-defined. I would send it to peer review with a clear request for the DP+CombinedDataset baseline, multiple seeds, and code/data release; if the results survive that control, the claim becomes credible.","headline":"Real-robot exposure robustness, but the missing combined-data control makes the main attribution unsupported.","tokens_in":12186,"tokens_out":2893,"would_cite":false,"duration_ms":26938,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A policy that fuses corrupted RGB frames with monocular depth maps keeps 82-88 percent success across camera exposures from 10 to 170 ms, where standard imitation-learning baselines collapse.","keywords":["imitation learning","robot manipulation","monocular depth estimation","image augmentation","lighting robustness","camera exposure","Diffusion Policy","multimodal perception"],"falsifier":"Train a vanilla Diffusion Policy on the same combined dataset (62.5 percent fixed at 120 ms, 37.5 percent varied between 50 and 160 ms) without AugBlender or depth, and evaluate it at the ten test exposures. If its average success approaches the proposed model's 82-88 percent, the paper's attribution to the fused perception pipeline collapses; if it stays near the other baselines, the attribution is supported.","tokens_in":11265,"feed_emoji":"🤖","tokens_out":9207,"duration_ms":80844,"temperature":0.7,"pith_summary":"This paper sets out to show that an imitation-learning robot policy can stay reliable when its cameras' exposure changes drastically, a setting where ordinary policies fail. The proposed pipeline corrupts RGB frames with AugBlender during training and fuses them with depth maps produced by a monocular depth model trained on internet-scale data. Across three manipulation tasks and ten exposure levels from 10 to 170 ms, the full model averages 82-88 percent success while the baselines collapse, especially at low exposures. The authors argue this makes low-cost, video-only robot systems more deployable in varied lighting without adding depth sensors.","feed_headline":"Depth maps rescue robot policies from lighting collapse","feed_subtitle":"RGB plus depth keeps 82-88 percent success across extreme camera exposures, where standard policies fail.","key_machinery":"The carrying mechanism is a two-branch perception module feeding a transformer-based denoising diffusion policy. AugBlender, an extension of AugMix, applies random color-only corruptions such as hue, saturation, solarization, and gamma changes, with mixing weights drawn from a Dirichlet distribution and a logic gate $\\beta=0.16$ deciding whether the output is an in-distribution or out-of-distribution image; restricting corruptions to color keeps the geometry aligned with depth. The depth branch runs the monocular estimator Depth Anything V2 on the same frames, producing spatial maps that stay consistent under exposure change. The fused features pass through separate per-view ResNet34 Feature Pyramid Network encoders and then into the action prediction module, so the policy can condition on depth when the RGB channel is corrupted.","core_discovery":"The paper's central claim is that fusing corrupted RGB observations with uncorrupted monocular depth maps yields an imitation-learning perception module that stays stable under exposure variation. Concretely, the model combining AugBlender and Depth Anything V2 reaches average success rates of 88 percent on CupStack, 82 percent on PickSmall, and 83 percent on PickBig across exposures of 10 to 170 ms, whereas the vanilla Diffusion Policy baseline averages 23, 43, and 47 percent on those tasks. At the three lowest exposures, the proposed model keeps an 81 percent average on CupStack while all four baselines score zero at the lowest two levels. The paper attributes this to the depth channel remaining informative when RGB is nearly unusable, and to AugBlender teaching the policy to lean on depth when RGB departs from training conditions.","pith_inferences":["The paper leaves open whether dataset scale matters: the combined dataset is larger and only the full model trains on it, so part of the gain could come from data volume rather than the perception modules.","If the mechanism is really depth reliability under lighting change, swapping Depth Anything V2 for a different monocular depth estimator should preserve most of the benefit; if not, the gain may come from DINOv2 features or from augmentation alone.","The paper's own CupStack failure at exposure 10 suggests depth preserves geometry but not color identity, so tasks requiring color discrimination or strict sequencing will need a third cue, such as segmentation or object embeddings, in near-darkness.","A standardized exposure-robustness benchmark for manipulation policies would let this single-robot evaluation be compared across labs and methods."],"forward_implications":["If the central claim holds, an RGB-only imitation learning stack can gain exposure stability by adding monocular depth and color-only augmentation, with no hardware change.","The full model's average success is 82-88 percent across 10-170 ms exposures on all three tasks, more than double the second-best baseline on CupStack.","At extreme low exposures (10-40 ms), the proposed model keeps an 81 percent average success on CupStack while every baseline scores zero at 10-20 ms.","The ablation pattern suggests depth mainly rescues high-exposure performance while AugBlender widens the usable exposure range, so the two components act together rather than either alone.","Because the depth estimator runs in real time on a single RTX 3090, the stability gain is available to low-cost two-camera robot setups."],"supporting_citations":[{"why":"It supplies the monocular depth maps fused with corrupted RGB, and its exposure-stable estimates carry the central claim.","marker":"[49]"},{"why":"It is the backbone of Depth Anything V2, so the depth branch inherits its corruption-resistant features.","marker":"[36]"},{"why":"It provides the mixing scheme that AugBlender extends, including the Dirichlet weight sampling and chained augmentations.","marker":"[13]"},{"why":"It is the imitation-learning policy that every compared model shares, fixing the action-prediction module.","marker":"[6]"},{"why":"It supplies the denoising-diffusion machinery underlying the transformer-based action prediction.","marker":"[14]"}],"fun_headline_variants":["Depth maps armor robot policies against lighting collapse","AugBlender plus depth keep robot policies robust to exposure","Spatial perception framework beats RGB alone under exposure changes","Depth channel preserves robot success when RGB goes dark"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method-compensation claim depends on the combined training dataset being neutral; because only the full model trains on the larger mixed-exposure dataset, the extra data could explain part of the gain.","fun_headline_variants_meta":{"raw":{"variants":["Depth maps armor robot policies against lighting collapse","AugBlender plus depth keep robot policies robust to exposure","Spatial perception framework beats RGB alone under exposure changes","Depth channel preserves robot success when RGB goes dark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1342,"prompt_tokens":845,"completion_tokens":497,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":435}},"tokens_in":461,"tokens_out":497,"duration_ms":5349,"temperature":1.0,"reasoning_tokens":435,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:07:14.984021+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a vanilla Diffusion Policy on the same combined dataset (62.5 percent fixed at 120 ms, 37.5 percent varied between 50 and 160 ms) without AugBlender or depth, and evaluate it at the ten test exposures. If its average success approaches the proposed model's 82-88 percent, the paper's attribution to the fused perception pipeline collapses; if it stays near the other baselines, the attribution is supported.","supporting_citations":[{"cited_title":"Depth any- thing v2, 2024","cited_arxiv_id":null,"evidence_quote":"It supplies the monocular depth maps fused with corrupted RGB, and its exposure-stable estimates carry the central claim."},{"cited_title":"Dinov2: Learning robust visual features with- out supervision, 2024","cited_arxiv_id":null,"evidence_quote":"It is the backbone of Depth Anything V2, so the depth branch inherits its corruption-resistant features."},{"cited_title":"Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan","cited_arxiv_id":null,"evidence_quote":"It provides the mixing scheme that AugBlender extends, including the Dirichlet weight sampling and chained augmentations."},{"cited_title":"Diffusion policy: Visuomotor policy learning via action dif- fusion, 2024","cited_arxiv_id":null,"evidence_quote":"It is the imitation-learning policy that every compared model shares, fixing the action-prediction module."}],"review_version":1}