{"id":"a5359bae-98c6-4205-bb04-cf3b3141c953","arxiv_id":"2607.10309","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"An affordable vision-and-hardware-keyboard AIoT platform quantifies a 1160% sim-to-real HNS drop for DQN on Breakout while enabling real-world training to ~38% of human performance.","lead":"Researchers built a sub-$400 camera-plus-emulated-keyboard platform so an edge device can play Atari games in the physical world. It measures a 1160% sim-to-real performance collapse for DQN and shows direct real-world training reaches only ~38% of human level after 10 M steps.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The 1160% gap numbers are solid for this Breakout setup, but the claim that the platform is a general AIoT Sim-to-Real benchmark rests on an unvalidated architectural analogy between game loops and broader sensor-actuator systems.","rationale":"The reader correctly isolates the representativeness assumption as the weakest link supporting the “AIoT” and “universal” framing. The quantitative results themselves (Tables II–VI, Figs. 7–16) are free of calculation errors, use recommended statistics (IQM + CIs), and are backed by open code; the 1160 % wording is merely a re-expression of the HNS difference and does not hide any inconsistency. No stronger internal flaw (circular derivation, missing artifact, or mis-specified baseline) is present. Consequently the CONDITIONAL verdict already assigned by the reader stands without adjustment; multi-game or multi-algorithm validation is precisely the broader check the paper’s own future-work section anticipates.","tokens_in":28933,"tokens_out":612,"duration_ms":32593,"concrete_test":"Execute the exact Section V protocol (10 M step DQN, 100-episode IQM HNS with 95 % CIs) on two additional ALE games (Pong and SpaceInvaders) under identical camera crop, 66 ms key-press, and serial-reward hardware; if either game yields a sim-to-real IQM HNS ratio below 50, the 1160 % figure cannot be presented as a platform-level characteristic transferable to general AIoT.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, Sec. VII) that the platform “provides a substantial foundation for qualitative and quantitative evaluations of RL in real-world AIoT systems” and is effectively universal requires that the single-camera, fixed-crop, 66 ms key-press Breakout loop (Sec. IV-A, Algs. 2–3, Figs. 3–6) captures the essential sensor–actuator–edge dynamics of the AIoT applications invoked in Secs. I–II (smart buildings, traffic, power grids, etc.). This is asserted only via the minimal architecture of Fig. 1; no non-game task is ever run, no quantitative comparison to existing AIoT simulators or physical systems is supplied, and the dominant observed failure modes (visual domain shift in Fig. 12, total collapse under a 5-pixel crop shift in Figs. 13–16) are classic vision-based RL brittleness rather than generic AIoT phenomena. If the analogy does not hold, the carefully measured 11.6 → 0.01 HNS collapse and the 0.38 HNS real-world training result remain valid only for this video-game platform, not for AIoT broadly.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The manuscript introduces a low-cost (<USD 400 hardware) real-world platform intended as a Sim-to-Real benchmark for RL in AIoT. An edge-device agent observes a host computer’s screen via USB camera and issues actions through a hardware-emulated keyboard (Teensy), while rewards and termination signals travel over a serial link. Three configurations are compared: pure simulation (ALE/Gymnasium), camera-input only, and the full real-world loop. Using Breakout and standard DQN for 10 M steps, the authors report that a simulation-trained agent reaches an IQM human-normalized score (HNS) of ~11.6 in simulation yet collapses to ~0.01 when deployed on the physical platform (a 1160 % degradation relative to human level), while direct real-world training reaches ~0.38 HNS. Additional ablations show brittleness to a 5-pixel crop shift. The platform, code, and evaluation protocol are released open-source.","tokens_in":29294,"tokens_out":1449,"duration_ms":23396,"significance":"Affordable, safety-constrained physical platforms for RL remain scarce outside robotics; the open-source release, sub-USD-400 bill of materials, and careful reporting (100-episode tests, 95 % CIs on mean/median/IQM/optimality gap, learning curves) are genuine contributions that lower the barrier to real-world experimentation. The measured 11.6\to0.01 HNS collapse and the 0.38 HNS real-world training ceiling are concrete, reproducible numbers for this setup. If the community adopts the platform even as a vision-based edge-control testbed, it can support systematic study of domain shift, latency, and sample efficiency. The broader claim that the platform constitutes a “universal” or “substantial foundation” for general AIoT (smart buildings, traffic, power grids) is more aspirational and currently rests on an architectural analogy rather than multi-domain evidence.","major_comments":[{"comment":"Abstract, §§I–III and VII assert that the platform supplies a “universal” / “substantial foundation” for Sim-to-Real evaluation of RL in AIoT systems (smart buildings, traffic, power grids, etc.). The only empirical support is a single-camera, fixed-crop, 66 ms key-press Breakout loop (Algs. 2–3, Figs. 3–6). No non-game task, no continuous-control or multi-agent AIoT scenario, and no quantitative comparison to existing AIoT simulators or physical plants is provided. The dominant failure modes (visual domain shift in Fig. 12, total collapse under a 5-pixel crop shift in Figs. 13–16) are classic vision-based RL brittleness, not generic AIoT phenomena. Either the generality claim must be substantially scoped to “vision-based edge control with discrete actuation” or additional evidence (at least one non-game task or a mapping study) is required; otherwise the carefully measured gap remains v","section":"Abstract, §§I–III, VII"},{"comment":"§V reports results exclusively for Breakout under a single algorithm (DQN) and a 10 M-step budget (far below the 200 M-frame ALE standard). While the authors correctly note computational constraints, a platform intended as a general benchmark needs at least a second game or a second algorithm (e.g., PPO/A3C, already mentioned in §VIII) to demonstrate that the measured gap and real-world trainability are not artifacts of one environment–algorithm pair. Without that, the claim that the platform “validates” Sim-to-Real evaluation for AIoT (§V-G) is overstated.","section":"§V, §V-G"},{"comment":"The observation pipeline relies on a manually chosen, fixed crop of the camera FOV (Fig. 4, Alg. 1) and a hand-tuned 66 ms key-press duration. These free parameters are acknowledged but never ablated systematically beyond the 5-pixel shift. Because the entire Sim-to-Real gap is attributed to “real-world factors,” the contribution of these design choices versus irreducible physical noise/latency must be quantified; otherwise it is unclear how much of the 1160 % degradation is platform-specific rather than fundamental.","section":"§IV-A, Fig. 4, Algs. 1–3"}],"minor_comments":[{"comment":"The 1160 % figure is arithmetic (HNS 11.6 – 0.01 ≈ 11.6 relative to human baseline 1.0) but is repeatedly phrased as “performance degradation relative to the human-level performance,” which can be misread as a 11.6-fold drop from human level rather than from the agent’s own sim performance. Clarify the reference point in the abstract and §V-E.","section":"Abstract, §V-E"},{"comment":"Figs. 7–11 and 14–16 use non-standard Unicode axis labels that render poorly; replace with ordinary text or vector fonts for production.","section":"Figs. 7–11, 14–16"},{"comment":"Table I’s “˜✓” notation for IoT status of Atari/robots is unexplained in the caption; define the symbols explicitly.","section":"Table I"},{"comment":"§II-C notes that the Physical Atari platform [26] appeared after the present work was completed; a short side-by-side comparison of design goals (robotic joystick vs. AIoT keyboard/serial) would help readers situate the contribution.","section":"§II-C"},{"comment":"Minor typos: “HV AC” → “HVAC”, “A VWC” → “AVWC”, “tokamak” capitalization consistency, and “Sim-to-real” vs. “Sim-to-Real” throughout.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The empirical core (gap measurement + real-world training feasibility on a cheap platform) is solid and publishable after scoping. The main risk is over-selling the work as a general AIoT benchmark when the evidence is confined to vision-based Atari. If the authors refuse to narrow the claim, the paper fits better in a systems/benchmarks venue than a general AI journal. Open-source code is a clear plus for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core is concrete: a sub-$400 camera + Teensy-emulated-keyboard setup, three matched systems (sim, camera-only, full real), and careful numbers. After 10 M DQN steps the sim agent sits at ~11.6 IQM HNS in simulation and collapses to ~0.01 on the physical loop—the 1160% figure is just that arithmetic against the human baseline. Direct real-world training reaches ~0.38 HNS. They report 100-episode tests with CIs, learning curves, wall-clock (0.3 vs 8–10 days), and a 5-pixel crop-shift ablation that kills performance. Code and build notes are public. That is real engineering evidence, not a rehash of MuJoCo-to-robot papers or the later Physical Atari box.\n\nWhat is new is the deliberately motor-free edge loop aimed at AIoT-style sensing/actuation/timing, plus the first quantitative HNS gap on that hardware. For groups that cannot buy robots or buildings, this is a practical, safe way to study vision noise and control latency.\n\nThe soft spot is proportion, not fraud. Only Breakout, only DQN, 10 M frames, manual crop. More importantly, the abstract and conclusion sell a “substantial foundation” / near-universal AIoT benchmark. The support is the minimal sensor–edge–actuator diagram; no non-game task, no comparison to building/traffic/grid systems. The failures they measure (domain shift, crop brittleness) are classic vision RL, not generic AIoT dynamics. That overclaim should be dialed back; it does not erase the platform or the gap numbers for this setup.\n\nMath and baselines are standard ALE HNS; related work is fair. No circular fitting. This is for people who need a cheap physical RL testbed or who write sim-to-real methods and want a reproducible stress case. It deserves a serious referee as a platform paper—expect them to demand multi-game/algorithm checks and a narrower claim. I would engage with the hardware and the measured collapse; I would not treat the AIoT-universality sentence as established.","headline":"Clean empirical gap numbers on a cheap physical Breakout loop; the “universal AIoT benchmark” framing is the soft part, not the measurements.","tokens_in":29886,"tokens_out":550,"would_cite":true,"duration_ms":14830,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A sub-$400 camera-and-keyboard platform measures a 1160% Sim-to-Real collapse for RL game agents while proving real-world training still works.","keywords":["reinforcement learning","Sim-to-Real gap","AIoT","real-world benchmark","deep Q-network","edge computing","Atari Breakout","human-normalized score"],"falsifier":"Train and evaluate the same DQN agent on a second, independently built copy of the platform whose camera crop, lighting or key-hold timing differ by only a few pixels or milliseconds; if the reported 1160% gap and 0.38 HNS numbers disappear or reverse, the measured gap is an artifact of the particular hardware instance rather than a transferable AIoT property.","tokens_in":29832,"feed_emoji":"📷","tokens_out":761,"duration_ms":6351,"temperature":0.7,"pith_summary":"Most reinforcement-learning research for Autonomous Internet of Things systems stays inside simulators because real trials are expensive and sometimes dangerous. Without a shared physical testbed, it is hard to know how large the gap between simulation and reality actually is, or whether an algorithm that works in simulation still works when sensors and actuators are real. This paper builds an inexpensive physical AIoT loop in which an edge computer watches a game screen with a camera and issues keystrokes through a hardware-emulated keyboard. On that loop a simulation-trained deep Q-network agent that scores far above human level inside the simulator collapses to near-random performance once deployed, a 1160% drop relative to human skill. The same platform, however, lets the identical algorithm learn directly from real camera and keyboard experience and reach roughly 38% of human performance after ten million steps. The result is a concrete, low-cost, safety-safe benchmark that makes both the gap and the residual feasibility of real-world RL measurable for AIoT researchers.","feed_headline":"RL agent drops 1160% from sim to real on a $400 AIoT rig","feed_subtitle":"Camera-and-keyboard Breakout loop still lets DQN learn to 38% of human level from scratch.","key_machinery":"The real-world AIoT benchmark loop: a fixed-crop camera observation, a Teensy-based hardware keyboard that holds each key for 66 ms, and a serial reward channel, all running at real-time frame rates with no step-wait. This closed physical loop supplies the states, actions and rewards that let the Sim-to-Real gap and algorithmic robustness be measured as differences in human-normalized score.","core_discovery":"A camera-plus-emulated-keyboard Breakout platform costing under 400 dollars reveals that a simulation-trained DQN agent loses essentially all of its super-human performance when the observation and action streams become physical, yet the same algorithm can still learn from scratch on the physical loop and reach about 38% of human-normalized score after ten million steps. That pair of numbers quantifies both a severe Sim-to-Real gap and residual algorithmic robustness for AIoT-style RL.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["1160% Sim-to-Real drop for RL on $400 AIoT Breakout rig","Sub-$400 camera-keyboard rig exposes RL Sim-to-Real cliff","Sim DQN collapses 1160% on real AIoT; real DQN hits 38% human","Cheap vision-keyboard Breakout platform quantifies RL gap","Real AIoT DQN reaches 38% human after 10M steps; sim fails hard"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The claim that a single-camera, fixed-crop, 66-millisecond key-press video-game loop is a good enough stand-in for the sensor-actuator-edge interactions found in general AIoT systems such as buildings, traffic or power grids.","fun_headline_variants_meta":{"raw":{"variants":["1160% Sim-to-Real drop for RL on $400 AIoT Breakout rig","Sub-$400 camera-keyboard rig exposes RL Sim-to-Real cliff","Sim DQN collapses 1160% on real AIoT; real DQN hits 38% human","Cheap vision-keyboard Breakout platform quantifies RL gap","Real AIoT DQN reaches 38% human after 10M steps; sim fails hard"]},"model":"grok-4.5","effort":"low","cost_usd":0.005262,"raw_usage":{"total_tokens":1533,"prompt_tokens":936,"num_sources_used":0,"completion_tokens":116,"cost_in_usd_ticks":52620000,"prompt_tokens_details":{"text_tokens":936,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":481,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":936,"tokens_out":116,"duration_ms":4240,"temperature":1.0,"reasoning_tokens":481,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T12:43:47.573704+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train and evaluate the same DQN agent on a second, independently built copy of the platform whose camera crop, lighting or key-hold timing differ by only a few pixels or milliseconds; if the reported 1160% gap and 0.38 HNS numbers disappear or reverse, the measured gap is an artifact of the particular hardware instance rather than a transferable AIoT property.","supporting_citations":[],"review_version":1}