{"id":"dbf1ad81-37a7-494a-9593-71c4e76893e3","arxiv_id":"2506.11302","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"TARDIS is a transformer world model trained on STRIDE, a graph-structured street-view dataset, with claimed abilities in controllable image generation, georeferencing, self-driving actions, and temporal simulation.","lead":"A team from Tera AI and universities built STRIDE, a dataset that rewires 131,000 Google Street View panoramas from San Mateo into 6.3 million road-travel sequences, and trained TARDIS, a 1B-parameter world model on it. The paper claims one model can generate street views from movement commands, guess GPS locations, drive along roads, and simulate seasonal change, but the headline evaluations leak information and lack fair baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Georeferencing is likely driven by move-command leakage and dense-city memorization rather than image understanding; the SOTA claim needs an action-ablated baseline before it can stand.","rationale":"The reader's weakest_assumption identifies exactly this confound: the georeferencing evaluation conditions on ground-truth move commands and uses a holdout embedded in the same dense road network. The paper's central claim is that TARDIS integrates perception, state, and action to achieve state-of-the-art georeferencing, and that claim collapses if the move command alone determines the next coordinate. The proposed ablation would settle whether image input contributes anything beyond odometry and memorization. The paper does have independent value in the dataset release, the 27x augmentation construction, and qualitative generation results, but the headline quantitative claims are not currently supported by the reported protocol. I therefore concur with the reader's REJECT verdict; a conditional accept would require the action-ablated evaluation, the odometry baseline, and the holdout-distance analysis to be added and the numbers re-reported.","tokens_in":15363,"tokens_out":3682,"duration_ms":42600,"concrete_test":"Re-run the georeferencing evaluation with the action tokens ablated (mask displacement/heading, or set them to neutral 'move 0, heading 0') while keeping the same observations and state tokens. If the 60% within 10m collapses, the result depends on move-command leakage. Also implement an odometry-only baseline: use the ground-truth starting coordinate plus the true displacement/heading to compute the next coordinate with no image input; if this baseline matches or exceeds TARDIS, imagery contributes little. Additionally, report the distance from each spatiotemporal test node to its nearest training node; if the median is below, say, 20m, the 'unseen area' is dense interpolation rather than held-out geography.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the georeferencing evaluation (Section 4.2) does not isolate image-based localization. In STRIDE, each sample contains observation O_n, state S_n (lat/lon/month/year), and action A_n (displacement, heading, month/year offsets) (Eq. 2). The action is derived from the ground-truth path, so when TARDIS is prompted with the true move command before predicting the next coordinate, it can compute the next node by dead reckoning from a known current location; the visual observation is not needed. The paper's 'self-masking of tokens which fall outside the expected radius defined by the move command' actively uses the action to restrict the coordinate search, encoding the answer into the prompt. Under this protocol, 60% within 10m and 90% within 20m are expected from odometry plus in-city memorization. The baseline comparison is also not matched: SVG is restricted to a 50m search radius, does not receive move commands, and is evaluated without the benefit of training on the same road network. Finally, the spatiotemporal holdout (bottom 10% of a 9.2x7.5 km area, with 130k training nodes on the same streets) is not a true out-of-distribution test; nearby training nodes provide dense interpolation and memorization cues. If georeferencing fails without action leakage, three of the paper's headline contributions (georeferencing, self-control, and spatiotemporal world modeling) rest on an artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces STRIDE, a large-scale dataset that reorganizes Google Street View panoramas and OpenStreetMap data into 6.3M 'visual sentences' of observation-state-action tokens spanning 82B tokens, and TARDIS, a 1B-parameter transformer-based autoregressive world model trained on this dataset. The authors claim state-of-the-art georeferencing (60% of predictions within 10m error), controllable photorealistic image generation (with a 41% FID improvement over Chameleon7B), autonomous self-control (77.4% road adherence at 4m lane width), and temporal reasoning (linear SSIM decay with R²=0.94). The dataset and code are released.","tokens_in":15640,"tokens_out":4970,"duration_ms":53274,"significance":"If the claims were substantiated, STRIDE would be a valuable new resource for spatiotemporal world modeling, and TARDIS would demonstrate a promising unified architecture for navigation, generation, and temporal reasoning. The paper is commendable for releasing training code, model checkpoints, and a sizable dataset, and the graph-based composition of street-view imagery into navigable state-action trajectories is a creative and potentially reusable idea. However, as we argue below, the experimental protocol used to support the headline claims is flawed in several load-bearing ways, and some quantitative claims appear to be unsupported by the reported measurements. The significance of the contribution therefore rests on corrections that have not yet been demonstrated.","major_comments":[{"comment":"The georeferencing evaluation leaks the answer. In Eq. (2), the action tokens include displacement and heading derived from the ground-truth path. Section 4.2 states that TARDIS is prompted with these spatial commands and that 'self-masking of tokens which fall outside the expected radius defined by the move command' is used. Given the current coordinates in the prompt and the true move command, the next coordinate is determined up to tokenization error; the visual observation is not needed. The 60% within 10m result is therefore an expected consequence of dead reckoning plus in-city memorization, not image-based georeferencing. The comparison to SVG is unfair because SVG is restricted to a 50m search radius, does not receive move commands, and was not trained on the same road network. Please provide an evaluation that masks or abandons the true move command, and that fairly matches the baseline.","section":"Section 4.2"},{"comment":"The claims of '41% FID improvement over Chameleon7B' and 'linear SSIM decay R2=0.94 over 5-year intervals' are not supported by any reported measurement. The cited figures (Fig. 3, Fig. 9, Figs. 20-22) show qualitative images and perplexity plots, not FID or SSIM values. Either report the actual quantitative measurements with evaluation details, or remove these claims from the abstract and contributions list.","section":"Section 1 and Section 4.1"},{"comment":"The spatiotemporal holdout is not a true out-of-distribution test. The 'bottom 10% of the training area' is embedded within a 9.2x7.5 km city where the remaining ~130k training nodes lie on the same road network, often immediately adjacent to the test nodes. Dense training coverage on the same streets enables memorization and interpolation, so the claimed generalization to 'unseen areas' is overstated. The temporal holdout (2023-2024) is likewise within the same city. To support the generalization claim, evaluate on a geographically disjoint area or a different city with a separate training set.","section":"Section 2.3"},{"comment":"The self-control evaluation lacks any baseline or random policy comparison. The metric (road adherence) is computed for actions generated by TARDIS, but without comparing to, e.g., a random heading/distance policy, a nearest-neighbor policy, or an action-only model that ignores the image, it is not possible to determine whether the image observation contributes to the result. Since TARDIS is trained on road-legal transitions, the 77.4% road-adherence rate may reflect memorized road layout rather than visual understanding. Add at least one non-trivial, non-oracle baseline.","section":"Section 4.3"}],"minor_comments":[{"comment":"The number of panoramic images is reported as 135k in Section 2.2 but Table 1 lists 130k (total) or 105k (training); please reconcile these inconsistencies.","section":"Section 2.2 and Table 1"},{"comment":"The use of perplexity as a quantitative proxy for image quality should be justified; the paper itself notes that visual metrics are often not representative, and perplexity of a language model over image tokens is a weak proxy for photorealism or control.","section":"Section 4.1"},{"comment":"The phrase 'we dynamically allow self-masking of tokens which fall outside the expected radius defined by the move command' needs a precise definition of 'expected radius' and of how the masking is applied; as written, it strengthens the impression of answer leakage.","section":"Section 4.2"},{"comment":"The tokenization table lists 'Year' with possible range 2000-2030, but the temporal distribution in Figure 19 only goes to 2024; clarify whether the model is required to predict beyond 2030 and how the token range was chosen.","section":"Appendix D"},{"comment":"The phrase '734 days of wall time, or about 16 hours on our 128 VM cluster' is confusing; clarify the relationship between total CPU-time, wall-time, and cluster size.","section":"Appendix E"},{"comment":"The dataset is released under CC-BY-NC-4.0, which may restrict commercial and downstream use; this should be stated explicitly in the main text, as it could affect the dataset's utility for the autonomous-driving community.","section":"Appendix J.2"}],"recommendation":"reject","confidential_remarks":"The paper's central georeferencing claim appears to be the product of an evaluation protocol that supplies the answer (the true move command) to the model. In addition, at least two headline quantitative claims (41% FID improvement, R²=0.94 SSIM temporal decay) are not backed by any reported measurements. These are not presentation issues; they are load-bearing flaws in the evidence for the paper's main contributions. The dataset itself is a substantial artifact, but the manuscript's central model claims are not currently supported. I would recommend rejection, though the authors could potentially resubmit a revised version with an action-ablated georeferencing evaluation and all quantitative claims properly reported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nShort version: this paper has a genuinely useful dataset idea, but the evaluation overstates what the model does. The spatiotemporal graph formulation—month/year offsets as actions, 27x panorama permutations, and the learned transition graph from GSV+OSM—is new and worth building on. Releasing the dataset and training code is a real plus. The headline results, though, do not hold up as reported.\n\nThe georeferencing comparison (Sec. 4.2) is confounded. TARDIS is prompted with the true move command (distance and heading from the ground-truth path) and is allowed to self-mask coordinate tokens outside the expected radius. That makes the next node computable by dead reckoning from a known current location; the image barely matters. SVG gets no such action input and is restricted to a 50m search radius, so the 60%-vs-<10% within 10m comparison is not a fair test of image-based localization. The spatiotemporal holdout is also not a clean out-of-distribution test: it is the bottom 10% of a 9.2x7.5 km area with dense training nodes on the same road network, so interpolation and memorization are plausible confounds.\n\nThe generation and temporal numbers are asserted but not measured. The intro claims a 41% FID improvement over Chameleon and R2=0.94 for linear SSIM decay, but the experiments report perplexity, not FID or SSIM. The cited figures show qualitative samples and perplexity plots; the actual metric values are absent. Self-control also has no baseline—77.4% road adherence with no comparison policy is hard to interpret.\n\nThat said, the dataset and the code release are real, and the paper is candid about its limitations: single region, sparse post-2020 temporal sampling, projection artifacts. The central model claims are simply not supported by the reported measurements.\n\nMy recommendation: send it to peer review—the dataset deserves scrutiny and the evaluation flaws need to be surfaced—but expect major revision. The authors should ablate the move-command leakage, report actual FID/SSIM numbers, and add a baseline for self-control. A solid dataset paper is underneath, but the current preprint overreaches.","headline":"The dataset is a real contribution; the model claims are not supported by the reported evaluation—georeferencing leaks the answer through the move command, and the FID/SSIM numbers are missing from the experiments.","tokens_in":16249,"tokens_out":2325,"would_cite":true,"duration_ms":25099,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single autoregressive world model, trained on a graph of street panoramas with coordinates and move commands, can localize images to within 10 meters, generate the next view under instruction, and choose its own legal driving actions.","keywords":["STRIDE","TARDIS","world model","georeferencing","spatiotemporal dataset","autoregressive transformer","controllable image generation","self-control"],"falsifier":"Run TARDIS on the georeferencing benchmark with image tokens replaced by a constant placeholder image while keeping the true state and action tokens; if coordinate error stays near its current 60%-within-10m level, the images are not doing the localizing work and the result is odometry-plus-memory. A complementary check is a no-image baseline that receives only the previous coordinates and the move command and predicts the next coordinate.","tokens_in":15140,"feed_emoji":"🗺️","tokens_out":5140,"duration_ms":51124,"temperature":0.7,"pith_summary":"This paper claims that real-world driving environments can be modeled as a unified autoregressive sequence of observations, states, and actions across both space and time, and that a transformer trained on such sequences gains a range of embodied abilities. To show this, it builds STRIDE, a dataset of 82B tokens derived from 131k Google StreetView panoramas in San Mateo, permuted into 6.3M visual sentences by stitching panoramic views into graph-structured drivable routes with coordinates, months, years, and movement commands. It trains TARDIS, a 1B-parameter generative world model, on this dataset. TARDIS reportedly georeferences held-out images to within 10m error 60% of the time, generates photorealistic next views under spatial and temporal instruction, and autonomously generates movement commands that stay on a road 77.4% of the time at a 4m lane width. If true, this is evidence that a single sequence-prediction framework can fuse perception, localization, and control in dynamic real-world environments.","feed_headline":"World model georeferences, generates, and steers street views","feed_subtitle":"A single model maps, simulates, and drives through street scenes it has never seen.","key_machinery":"The load-bearing object is the spatiotemporal token sequence V = {s_start, N_1, ..., N_n, s_end}, where each node N_i packages an egocentric observation O (1024 VQGAN tokens of a 360° panorama projected toward travel direction), spatial state S (latitude, longitude), temporal state S (month, year), and action A (displacement in meters, heading in degrees, Δmonth, Δyear). This representation turns the physical road network into a graph the model can traverse, and the paper formalizes the traversal as five coupled functions (fss, fts, fsa, fta, fO) that map observation to state, state to action, and state-plus-action to the next observation. The Markovian structure, validated by OpenStreetMap-grounded transitions, is what lets a single transformer trained as a next-token predictor perform georeferencing, generation, and self-control without task-specific heads.","core_discovery":"The central claim is that spatial and temporal navigation of the real world reduce to one auto-regressive prediction problem. On STRIDE, each step of a journey is a token block N_i = {image, latitude, longitude, month, year, displacement, heading, Δmonth, Δyear}, and the model is trained to predict the next block from the previous ones. The paper argues that because node transitions are grounded in the OpenStreetMap road network, the process is Markovian: the next observation depends only on the current observation, state, and action. From this single formulation TARDIS derives four capabilities: coordinate prediction (georeferencing), conditional photorealistic image generation, self-generated navigation actions, and explicit temporal reasoning, each demonstrated on data held out in space or time. The result, as the authors state it, is a promising direction toward generalist agents that understand and manipulate the spatial and temporal aspects of their material environments.","pith_inferences":["If the georeferencing is genuinely image-driven, the imagery itself encodes absolute position at meter level in an area the model never saw, which implies that a world model trained on dense street-view permutations could serve as a learned visual compass usable when GPS is unavailable.","The same observation-state-action tokenization could transfer to other embodied settings—indoor navigation, drone flight, or robot manipulation—wherever a graph of traversable nodes with sensor observations and motion commands can be assembled.","A natural stress test the paper does not run is to train on one city and georeference in a second, unseen city; the fixed 16K context and single-region training leave open whether the learned notion of space generalizes or is memorized road-network topology.","The temporal control dimension is effectively a learned simulator that can hallucinate the same place at different dates; such a simulator could be used as data augmentation for downstream perception models that must be robust to seasonal and infrastructural change."],"forward_implications":["A single model trained on observation-state-action sequences can output meter-level geographic coordinates from a street image alone, outperforming a dedicated aerial-matching geolocalization method given only a 50m search space.","The same model can be instructed to move and watch the world change accordingly, including temporal commands that alter seasons or infrastructure, which is a step toward controllable simulation of real environments.","The model's self-generated actions keep it on legal road segments 77.4% of the time at 4m lane width on held-out areas, suggesting that action generation emerges from the world-model objective rather than a hand-designed policy.","Because time is an explicit token dimension, the model can estimate position and generate views across years, with image dissimilarity scaling linearly (R²=0.94) over 5-year intervals.","The dataset construction method itself—permuting a small set of panoramas into 6.3M visual sentences with 27x augmentation—provides a recipe for building large spatiotemporal world-model datasets from modest image collections."],"supporting_citations":[{"why":"Provides the SVG baseline that TARDIS is compared against in georeferencing, and the contrastive aerial-matching method that TARDIS claims to outperform.","marker":"[1]"},{"why":"Supplies the VQGAN image tokenizer used by TARDIS and serves as the comparison model (Chameleon-7B) for controllable image generation.","marker":"[15]"},{"why":"OpenStreetMap data grounds the road network that defines permissible transitions between nodes in STRIDE.","marker":"[14]"},{"why":"The LLaMA architecture is the base transformer design that TARDIS follows for its 1B-parameter model.","marker":"[56]"},{"why":"Provides the hyperparameter and sequential-modeling conventions that TARDIS adopts for training from scratch.","marker":"[30]"}],"fun_headline_variants":["One model maps, simulates, and drives street scenes","From street view to GPS: a single world model","One transformer predicts location, image, and action","Street-level world model: see, place, and steer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The georeferencing result stands or falls on whether the true move command (distance and heading) given to the model does not effectively reveal the next location, since the evaluation measures coordinate error by masking the coordinate tokens but leaving the action tokens intact.","fun_headline_variants_meta":{"raw":{"variants":["One model maps, simulates, and drives street scenes","From street view to GPS: a single world model","One transformer predicts location, image, and action","Street-level world model: see, place, and steer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000502,"raw_usage":{"total_tokens":2452,"prompt_tokens":941,"completion_tokens":1511,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":1447}},"tokens_in":557,"tokens_out":1511,"duration_ms":11265,"temperature":1.0,"reasoning_tokens":1447,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:11:44.092541+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TARDIS on the georeferencing benchmark with image tokens replaced by a constant placeholder image while keeping the true state and action tokens; if coordinate error stays near its current 60%-within-10m level, the images are not doing the localizing work and the result is odometry-plus-memory. A complementary check is a no-image baseline that receives only the previous coordinates and the move command and predicts the next coordinate.","supporting_citations":[{"cited_title":"Statewide visual geolocalization in the wild","cited_arxiv_id":null,"evidence_quote":"Provides the SVG baseline that TARDIS is compared against in georeferencing, and the contrastive aerial-matching method that TARDIS claims to outperform."},{"cited_title":"Planet dump retrieved from https://planet.osm.org","cited_arxiv_id":null,"evidence_quote":"OpenStreetMap data grounds the road network that defines permissible transitions between nodes in STRIDE."},{"cited_title":"Sequential modeling enables scalable learning for large vision models","cited_arxiv_id":null,"evidence_quote":"Provides the hyperparameter and sequential-modeling conventions that TARDIS adopts for training from scratch."}],"review_version":1}