{"id":"02ffb341-8208-46a7-aba7-3fa0055321f4","arxiv_id":"2507.09144","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"I2-World forecasts 3D occupancy over 3 seconds using an intra/inter tokenizer and reports state-of-the-art results, but the gains come mainly from oracle conditioning on the future ego pose at test time.","lead":"A new 4D scene forecasting model for self-driving cars splits occupancy compression into spatial and temporal tokenizers and steers future-scene generation with the car's planned motion. It reports a large accuracy and speed improvement over prior occupancy forecasting models, but its evaluation hands the model the car's ground-truth future trajectory that rivals do not get.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA forecast claim depends on test-time access to the ground-truth future ego transform (Sec. 4.1); without it, Table 4 shows I2-World falls to ~17 mIoU, on par with OccWorld.","rationale":"The reader identifies the test-time use of the ground-truth transformation matrix as the critical flaw, and I agree. The paper explicitly says so in Sec. 4.1 and the ablation in Table 4 confirms the dependence: without the transform, I2-World's forecasting mIoU is 17.12, essentially identical to OccWorld's 17.14. Therefore, the 25.1% mIoU and 36.9% IoU improvements are not forecasting gains; they arise from oracle conditioning on the future ego pose. The architectural contributions (intra/inter scene tokenizer, encoder-decoder) are plausible and the paper gives honest ablations, but the central SOTA claim does not hold under a fair protocol. The fix is straightforward — either withhold the GT transform from all methods or provide equivalent future ego-motion conditioning to the baselines — so the work could be resubmitted with a corrected protocol. A minor additional inconsistency: the Intro reports a 42.9% IoU gain while the Abstract and Table 1 imply 36.9%; this is cosmetic next to the protocol issue. I therefore keep the reader's rejection, i.e., UNCHANGED.","tokens_in":13741,"tokens_out":6221,"duration_ms":72324,"concrete_test":"Rerun the I2-Former evaluation (I2-World-O on Occ3D-nus) exactly as in Table 1 but with the transformation matrix predicted by the Intra-Encoder instead of the ground-truth matrix at every future step, i.e., remove the test-time oracle of Sec. 4.1. If average mIoU drops to the ~17% region (near OccWorld) rather than 39.7%, the SOTA claim is attributable to the leaked future ego pose and the comparison should be re-framed or rejected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 states: 'For evaluation, we employ the ground-truth transformation matrix to guide generation.' The matrix T^{t+k+1}_{t+k} is the future relative ego pose between timesteps t+k and t+k+1 (Sec. 3.3), so the model is given the future ego motion at test time. The comparison baselines (OccWorld, DOME, UniScene, DFIT-OccWorld) are not given any such oracle; the paper does not claim they use future pose, so the comparison is not apples-to-apples. The paper's own ablation (Table 4) shows the baseline with no conditioning reaches 17.12 mIoU / 27.2 IoU, essentially equal to OccWorld's 17.14 mIoU / 26.63 IoU. Adding translation conditioning (+11.6 mIoU) and rotation (+5.5 mIoU) produces the reported SOTA. Thus the central claim that I2-World outperforms prior methods by 25.1% mIoU and 36.9% IoU for 4D occupancy forecasting is not supported as an unconditional forecasting result; it is a conditional generation result with leaked future pose. The tokenizer contribution, the paper's other headline contribution, does not by itself lift forecasting performance above existing methods without this oracle.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes I2-World, a 4D occupancy forecasting framework that decouples scene tokenization into an intra-scene multi-scale residual quantizer and an inter-scene temporal quantizer, and combines these with an encoder-decoder autoregressive transformer conditioned on a transformation matrix. The authors report state-of-the-art results on Occ3D-nuScenes for 4D occupancy forecasting, with mIoU of 39.73 and IoU of 49.80, along with high efficiency (2.9 GB training memory and 37.0 FPS). They also report zero-shot generalization on Occ3D-Waymo and demonstrate controllable generation via commands and transformation matrices.","tokens_in":14013,"tokens_out":5952,"duration_ms":64593,"significance":"The tokenizer design is interesting and the authors provide a released codebase and extensive ablations. However, the central forecasting claim is not supported because the evaluation protocol supplies the ground-truth future ego transformation matrix at test time, while the comparison baselines do not receive this future information. The paper's own ablation shows that removing this conditioning drops performance to 17.12 mIoU, essentially identical to OccWorld's 17.14 mIoU. Thus the reported 25.1% mIoU improvement is not an unconditional forecasting result; it is a conditional generation result with an oracle. The tokenizer contribution, while potentially valuable for reconstruction, does not by itself lift forecasting performance above existing methods without this leaked future pose.","major_comments":[{"comment":"The evaluation protocol invalidates the headline claim. Section 4.1 states: 'For evaluation, we employ the ground-truth transformation matrix to guide generation.' Section 3.3 defines T^{t+k+1}_{t+k} as the transformation matrix between consecutive future timesteps, which encodes the future ego motion. At test time, the model is therefore given the answer's ego trajectory. Table 4 shows the effect of this conditioning: without translation or rotation conditioning, mIoU is 17.12; adding translation raises it to 28.74, adding rotation to 20.34, and adding both to 34.25; the full model reaches 39.73. Thus the majority of the reported forecasting gain comes from the ground-truth future pose, not from the proposed tokenizer or architecture. The central claim of state-of-the-art 4D occupancy forecasting is therefore not supported as an unconditional forecasting result.","section":"Section 4.1 and Section 3.3, Table 4"},{"comment":"The comparison with baselines is not apples-to-apples. The baselines in Table 1 (OccWorld, DOME, UniScene, DFIT-OccWorld, etc.) do not receive the ground-truth future transformation matrix, whereas I2-World does. The paper's own ablation in Table 4 shows that without this oracle, I2-World achieves 17.12 mIoU and 27.75 IoU, essentially equal to OccWorld's 17.14 mIoU and 26.63 IoU. Therefore, the claimed gains of 25.1% mIoU and 36.9% IoU conflate the model's forecasting ability with the information leaked by the oracle. To support the headline claim, the authors would need to either evaluate without the ground-truth future pose and report those numbers as primary, or give the same oracle information to all baselines and show that I2-World still outperforms them under a fair protocol.","section":"Section 4.2, Table 1 vs Table 4"}],"minor_comments":[{"comment":"The phrasing 'nearly requires 2.9 GB of training memory' is awkward; consider 'requires only 2.9 GB' or 'requires approximately 2.9 GB'.","section":"Section 1 and abstract"},{"comment":"The text states that the I2-World-STC variant 'outperforms prior methods by 50.9% (18.97 vs. 12.57) in mIoU', but Table 1 lists no baseline with 12.57 mIoU; the closest STC comparison is DOME-STC at 14.53 mIoU, which would be a 30.6% improvement. Please clarify or correct this number.","section":"Section 4.2"},{"comment":"The column header 'With' is ambiguous; the caption explains it denotes the Intra-Encoder, but this should be stated explicitly in the caption rather than only in the text.","section":"Table 4 caption"},{"comment":"The symbol B_t is reused for both the original feature map and the residual being updated, which makes the update equations difficult to follow; using distinct symbols (e.g., R_t for the residual) would improve readability.","section":"Section 3.2, Eqs. (2)-(4)"},{"comment":"The conclusion says the paper presents a framework for '3D scene generation', but the paper's topic is 4D occupancy forecasting; this wording is inconsistent.","section":"Section 5 (Conclusion)"},{"comment":"In 'the I2Former undergoes 48 epochs of training', there is a missing space between 'I2' and 'Former'.","section":"Section 4.1"},{"comment":"The abstract and Section 4.2 highlight 37.0 FPS, but Table 1 shows 37.04 FPS for I2-World-O and only 4.21 FPS for I2-World-STC; please clarify which configuration the efficiency claim refers to.","section":"Table 1 and Section 4.2"}],"recommendation":"reject","confidential_remarks":"The central issue is the test-time use of the ground-truth future transformation matrix, which the paper itself discloses in Section 4.1 and whose impact is quantified in Table 4. This is not a matter of framing or presentation; it changes the meaning of the main experimental claim. I see no way to support the stated state-of-the-art forecasting result without substantially reworking the evaluation and likely the contribution framing. If the authors were to reposition the work as a controllable world model with oracle-based conditioning and compare against equally-conditioned baselines, a resubmission might be viable, but the current manuscript's central claim is unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper has a genuinely reasonable idea—decoupling scene tokenization into intra-scene and inter-scene residual quantization with a shared codebook—but its headline forecasting claim is not supported by the evidence. At evaluation time the model is given the ground-truth future ego transformation matrix, and baselines are not. The paper's own ablation shows that conditioning is where almost all of the reported gain comes from.\n\nWhat's actually new: the I2-Scene Tokenizer combines multi-scale residual quantization (RQ-VAE style) with a temporal residual quantization that aligns historical features via ego-pose matrices. That's a clean and plausible efficiency story, and the numbers support it: 2.9 GB training memory, 37 FPS, and the tokenizer ablation in Table 3 shows real reconstruction gains from inter-scene alignment. Code is promised, which helps reproducibility.\n\nThe soft spot is not subtle. Section 4.1 states: 'For evaluation, we employ the ground-truth transformation matrix to guide generation.' That matrix encodes exactly how the ego vehicle will move in the future. Table 4 then shows that without this conditioning, mIoU is 17.12—almost identical to OccWorld's 17.14. Translation conditioning adds 11.6 mIoU, rotation another 5.5, and the combined system reaches 39.73. So the 25.1% improvement over prior methods is a conditional generation result with leaked future ego pose, not a forecasting result. Calling it 'state-of-the-art 4D occupancy forecasting' is misleading. The tokenizer alone does not lift forecasting performance above existing methods, as the ablation demonstrates.\n\nOne could argue the transformation matrix is a legitimate control input, and the controllable generation demos in Fig 4 are interesting. But then the comparison should be against other controllable world models under the same protocol, not against unconditional forecasting baselines. That's a fixable flaw, and the design itself is worth discussing.\n\nThe failure case in Fig 4 (unseen reversing transformation produces unrealistic behavior) is honestly reported. The citation pattern looks fine; the self-citation to STCOcc is relevant since they build their STC variant on it.\n\nBottom line: this paper deserves to go out for review because the tokenizer idea is substantive and the evaluation issue, while central, is correctable with a changed protocol (e.g., predict the transform from history, or give all baselines the same oracle). As submitted, the central claim fails. If you're looking for an efficient conditional occupancy generator, this is a useful reference; if you want an unconditional forecasting result, the numbers don't deliver.","headline":"The tokenizer design is a legitimate efficiency contribution, but the SOTA forecasting claim rests on test-time access to the ground-truth future ego pose, and the paper's own ablation shows the gains vanish without it.","tokens_in":14575,"tokens_out":3241,"would_cite":false,"duration_ms":34826,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"I²-World claims that decoupling 4D occupancy tokenization into intra-scene and inter-scene residual quantizers, and conditioning an encoder-decoder forecaster on a predicted transformation matrix, outperforms prior 4D occupancy world…","keywords":["4D occupancy forecasting","world models","residual quantization","intra-inter tokenization","autonomous driving","transformation matrix conditioning","encoder-decoder transformer","zero-shot generalization"],"falsifier":"Run the Occ3D-nuScenes forecasting benchmark once with the ground-truth transformation matrix withheld from I²-World, so it must predict its own ego motion as the baselines implicitly do, and once with the same ground-truth matrix supplied to the baselines; if the mIoU gap over DOME and UniScene disappears or falls well below 25.1%, the headline advantage depends on the oracle pose input.","tokens_in":13524,"feed_emoji":"🚗","tokens_out":10865,"duration_ms":104879,"temperature":0.7,"pith_summary":"The paper is trying to establish that 4D occupancy forecasting for autonomous driving can be made more accurate and much cheaper by separating spatial compression from temporal compression. Rather than tokenizing each 3D frame alone or tokenizing whole spatiotemporal stacks, I²-World first compresses the current scene with multi-scale residual quantization and then adds residual temporal tokens obtained from historical frames aligned by the ego pose. This keeps the token count close to that of a 3D tokenizer while carrying motion information, which the authors say is why their model outperforms prior occupancy world models by 25.1% mIoU and 36.9% IoU on Occ3D-nuScenes. The forecasting transformer is encoder-decoder instead of decoder-only: it predicts a $4\\times4$ transformation matrix from the current scene and planned motion, then conditions the decoder on that matrix to generate the next scene tokens autoregressively. If the claims hold, occupancy world models become practical for latency-critical driving, needing only 2.9 GB of training memory and inferring at 37.0 FPS.","feed_headline":"Two-pass tokenizer lifts 4D scene forecasting by 25.1%","feed_subtitle":"Separating spatial and temporal compression keeps detail and motion, enabling 3-second forecasts at 37 FPS with 2.9 GB.","key_machinery":"The load-bearing mechanism is the I²-Scene Tokenizer's two-stream residual quantization. An Intra-Scene Tokenizer applies $S$ scales of multi-scale residual quantization to one shared codebook, and an Inter-Scene Tokenizer keeps $G$ historical feature maps in a memory queue, aligns each to the current frame with an ego-pose transformation matrix $T_{t-g}^{t}$, and quantizes the residual that remains. A single shared codebook keeps spatial and temporal tokens in the same discrete space, and the predicted transformation matrix $T_{t+k+1}^{t+k}$ is injected into the Inter-Decoder, making generation spatially consistent with ego motion. This is what lets the model keep the compactness of 3D tokenizers while carrying temporal dynamics, and it also provides the handle for fine-grained control of generated scenes.","core_discovery":"The central claim is that dynamic 4D scene forecasting does not require a monolithic 4D tokenizer or a decoder-only autoregressive world model. The I²-Scene Tokenizer decouples tokenization: an intra-scene stream quantizes the current occupancy map at multiple resolutions by residual subtraction against a shared codebook, and an inter-scene stream quantizes the residual left after subtracting temporally aligned historical feature maps, so motion enters the token stream as residual modifications of spatial codes rather than as extra spatial dimensions. The I²-Former then predicts a transformation matrix from the current tokens and an action-plan embedding, and it uses that matrix plus a queue of historical tokens to condition the next-timestep token map. On Occ3D-nuScenes, the occupancy-input variant reports 39.73 mIoU and 49.80 IoU averaged over 1s, 2s, and 3s forecasts, surpassing the previous best results by 25.1% mIoU and 36.9% IoU, and the camera-based variant reports 18.97 mIoU and 28.77 IoU, surpassing prior camera-based baselines by 50.9% mIoU and 40.9% IoU. The same frozen model also transfers zero-shot to Occ3D-Waymo, roughly doubling forecasting mIoU over a copy-paste baseline at both 10 Hz and 2 Hz sampling rates.","pith_inferences":["Not claimed by the paper, but implied by its setup: the reported margin may partly reflect an information asymmetry, because evaluation gives I²-World the ground-truth future ego transformation matrix while comparison baselines do not receive future information; a symmetric comparison with predicted poses would test how much of the gain is architectural.","The residual intra/inter tokenization recipe is not tied to occupancy grids; the same align-and-quantize-the-residual idea could be applied to point cloud, radar, or video sequence tokenizers, where motion is also a sparse residual on top of static structure.","Because the transformation matrix is a compact, interpretable control channel, a natural deployment extension is to learn the pose forecast and use it at inference, turning the oracle-conditioned evaluation into a fully closed-loop system that predicts its own future motion."],"forward_implications":["Averaged over 1s, 2s, and 3s forecasts on Occ3D-nuScenes, the occupancy-input I²-World-O reports 39.73 mIoU and 49.80 IoU, improving on the prior best by 25.1% mIoU and 36.9% IoU.","With camera-based occupancy predictions as input, I²-World-STC reports 18.97 mIoU and 28.77 IoU, beating prior camera-based baselines by 50.9% mIoU and 40.9% IoU.","The model requires about 2.9 GB of training memory and runs at 37.0 FPS on an RTX 4090, so real-time 4D occupancy forecasting is feasible with much smaller compute than LLM- or diffusion-based world models.","Frozen tokenizer and forecaster transfer zero-shot to Occ3D-Waymo, raising forecasting mIoU from 28.34 to 43.73 at 10 Hz and from 17.17 to 36.38 at 2 Hz over a copy-paste baseline.","Transformation-matrix conditioning provides two levels of controllability: high-level action commands such as turn left or right, and fine-grained per-step spatial transformation for scenario-specific generation."],"supporting_citations":[{"why":"Supplies the base VQ-VAE-style scene tokenizer, the BEV encoder-decoder design, the autoregressive forecasting setup, and the FPS measurement convention I²-World builds on and compares against.","marker":"[59]"},{"why":"The prior 4D tokenizer plus diffusion world model whose 36.36 IoU is the strongest baseline for the headline IoU gain.","marker":"[11]"},{"why":"Provides the residual-quantization mechanism that the multi-scale intra-scene tokenizer extends.","marker":"[17]"},{"why":"Supplies the next-scale residual prediction framing used for hierarchical intra-scene quantization.","marker":"[39]"},{"why":"Establishes the codebook representation and vector-quantization loss used to train the tokenizer.","marker":"[41]"},{"why":"Defines the Occ3D-nuScenes benchmark and evaluation protocol for 4D occupancy forecasting, and the Occ3D-Waymo benchmark used for zero-shot tests.","marker":"[40]"},{"why":"Provides the nuScenes dataset and ego-pose information that underlies Occ3D-nuScenes and the alignment step.","marker":"[6]"},{"why":"Reports the closest mIoU baseline (31.76 mIoU) that the paper's 25.1% mIoU improvement is computed against.","marker":"[18]"},{"why":"Supplies the predicted occupancy inputs used by the camera-based I²-World-STC variant.","marker":"[24]"}],"fun_headline_variants":["Decoupled tokenizers speed 4D forecasting 25%+","Separate spatial, temporal codes lift 4D forecast accuracy","Two-stream tokenization boosts 4D scene forecasting by 25%","Efficient 4D forecasting via dual tokenizer, 37 FPS","Split tokenization achieves SOTA 4D occupancy forecasting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise of the reported gains is that it is fair to give I²-World the ground-truth future ego transformation matrix during evaluation while the comparison methods do not receive that future information; if that oracle pose were removed or given to the baselines too, the 25.1% mIoU margin could shrink.","fun_headline_variants_meta":{"raw":{"variants":["Decoupled tokenizers speed 4D forecasting 25%+","Separate spatial, temporal codes lift 4D forecast accuracy","Two-stream tokenization boosts 4D scene forecasting by 25%","Efficient 4D forecasting via dual tokenizer, 37 FPS","Split tokenization achieves SOTA 4D occupancy forecasting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000641,"raw_usage":{"total_tokens":3055,"prompt_tokens":1154,"completion_tokens":1901,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":770,"completion_tokens_details":{"reasoning_tokens":1810}},"tokens_in":770,"tokens_out":1901,"duration_ms":15052,"temperature":1.0,"reasoning_tokens":1810,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:02:32.092201+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Occ3D-nuScenes forecasting benchmark once with the ground-truth transformation matrix withheld from I²-World, so it must predict its own ego motion as the baselines implicitly do, and once with the same ground-truth matrix supplied to the baselines; if the mIoU gap over DOME and UniScene disappears or falls well below 25.1%, the headline advantage depends on the oracle pose input.","supporting_citations":[{"cited_title":"Occworld: Learning a 3d occupancy world model for autonomous driving","cited_arxiv_id":null,"evidence_quote":"Supplies the base VQ-VAE-style scene tokenizer, the BEV encoder-decoder design, the autoregressive forecasting setup, and the FPS measurement convention I²-World builds on and compares against."},{"cited_title":"Autoregressive image generation using resid- ual quantization","cited_arxiv_id":null,"evidence_quote":"Provides the residual-quantization mechanism that the multi-scale intra-scene tokenizer extends."},{"cited_title":"Visual autoregressive modeling: Scalable image generation via next-scale prediction","cited_arxiv_id":null,"evidence_quote":"Supplies the next-scale residual prediction framing used for hierarchical intra-scene quantization."},{"cited_title":"Neural discrete representation learning","cited_arxiv_id":null,"evidence_quote":"Establishes the codebook representation and vector-quantization loss used to train the tokenizer."},{"cited_title":"Occ3d: A large-scale 3d occupancy prediction benchmark for au- tonomous driving","cited_arxiv_id":null,"evidence_quote":"Defines the Occ3D-nuScenes benchmark and evaluation protocol for 4D occupancy forecasting, and the Occ3D-Waymo benchmark used for zero-shot tests."},{"cited_title":"Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom","cited_arxiv_id":null,"evidence_quote":"Provides the nuScenes dataset and ego-pose information that underlies Occ3D-nuScenes and the alignment step."},{"cited_title":"Uniscene: Unified occupancy-centric driving scene generation","cited_arxiv_id":null,"evidence_quote":"Reports the closest mIoU baseline (31.76 mIoU) that the paper's 25.1% mIoU improvement is computed against."},{"cited_title":"Stcocc: Sparse spatial-temporal cascade renova- tion for 3d occupancy and scene flow prediction","cited_arxiv_id":null,"evidence_quote":"Supplies the predicted occupancy inputs used by the camera-based I²-World-STC variant."}],"review_version":1}