{"id":"0dc6508f-a120-4d98-a8bd-d23589f3154f","arxiv_id":"1908.08908","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Scene-LSTM that learns common human movement paths in grid cells and subgrids of a scene, combined with a Pedestrian-LSTM and data filters, reduces trajectory prediction error on several public benchmarks.","lead":"This paper presents a trajectory prediction system that learns common walking paths in a scene and combines them with each pedestrian's own movement pattern to forecast future positions. Because it reports substantially lower prediction errors than prior methods on several public datasets, it may be relevant for autonomous driving and robot navigation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported baseline numbers are internally inconsistent with the ablation: PMrel (a plain relative-coordinate LSTM) is 0.66 ADE in Table 2 but the same LSTM baseline is 1.74 ADE in Table 1, so the claimed improvement cannot be cleanly attributed to the Scene-LSTM.","rationale":"The reader focused on whether common paths learned in the first half of a video become stale in the second half. That is a reasonable limitation of the scene prior, but it is secondary: even if the hard filter works perfectly, the quantitative claim is not trustworthy because the reported baselines cannot be reconciled with the paper's own ablation. The PMrel row is the crux: it is defined as the Pedestrian Movement LSTM with relative locations and no scene data, which is the same architecture as the 'LSTM' baseline. A 1.08 m ADE gap between two executions of the same model cannot be explained by the model; it indicates the two numbers were produced under different training/evaluation conditions. The paper's assertion that all methods were given the same two-stage protocol is not enough: the numbers in Table 1 must be reproduced with that protocol. Also, no code is released and no error bars are given, so one cannot independently verify the 0.56 ADE. Since the central claim is empirical and depends on this comparison, I would not accept the current version until the baseline protocol is reconciled; if the comparison is indeed uncontrolled, the claim should be rejected. The hard-filter staleness concern could be revisited once the protocol is fixed, but it is not the first-order problem.","tokens_in":11584,"tokens_out":12797,"duration_ms":127203,"concrete_test":"Rerun the Table 1 'LSTM' baseline using the exact two-stage protocol described in Section 5 (Stage 1: leave-one-out pretraining on four videos; Stage 2: 10 epochs fine-tuning on the first 50% frames of the held-out video; test on the remaining 50%), with the same 8-frame observation / 12-frame prediction setup and the same negative log-likelihood loss, and compare the resulting ADE/FDE to Table 2 row 2 (PMrel). If the rerun matches PMrel (~0.66 ADE), Table 1's LSTM baseline was not evaluated under the same protocol and the comparison is invalid; if it matches ~1.74 ADE, PMrel must have an undisclosed advantage and the ablation is misleading. Either result settles whether the claimed improvement is attributable to Scene-LSTM.","verdict_should_be":"REJECT","load_bearing_attack":"Table 2 row 2 (PMrel) and Table 1 column 'LSTM' describe the same model: a per-pedestrian LSTM with relative displacement input and Gaussian output. Under the paper's stated protocol ('we apply the same training and testing procedures for all methods'), both should receive the same Stage 1 pretraining plus Stage 2 fine-tuning on 50% of the target video. Their reported averages differ by 1.08 m ADE (0.66 vs 1.74) and 1.81 m FDE (1.36 vs 3.17). Since PMrel alone also beats every Table 1 baseline and comes within 0.10 ADE / 0.21 FDE of the full Scene-LSTM, the headline reduction from 0.79 to 0.56 ADE appears to come from the two-stage, target-video fine-tuning protocol and/or an uncontrolled difference in how baselines were evaluated, not from the scene-memory mechanism. The paper does not disclose whether the Table 1 baseline numbers were rerun under Stage 2 or taken from prior papers; if the latter, the comparison is not controlled and the central claim is unsupported as stated. This is the load-bearing weakness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Scene-LSTM, a trajectory-prediction model that couples a per-pedestrian LSTM (Pedestrian-LSTM) with a scene-level LSTM (Scene-LSTM) operating on a two-level grid. A hard filter decides whether to use scene information based on whether the pedestrian is in a linear grid cell or a common subgrid path, and a soft filter selects the relevant scene memory via an attention-like gating mechanism. The model is evaluated on ETH, UCY, Town Center, PETS09, and Grand Central, reporting lower average ADE and FDE than baselines including Linear, LSTM, Social-LSTM, and SGAN variants. An ablation study isolates the contributions of relative coordinates, scene data, hard filter at grid/subgrid levels, and the soft filter.","tokens_in":11817,"tokens_out":3944,"duration_ms":40216,"significance":"If the quantitative claims are substantiated, the two-level grid scene memory with hard/soft filtering is a simple and interpretable alternative to visual-feature-based scene models, and the additional evaluation on Town Center, PETS09, and Grand Central is a useful generalization check. The paper's strengths include a clear problem formulation, a systematic ablation of PMabs vs PMrel, and explicit consideration of when scene data should or should not be used. However, the reported numerical comparisons are undermined by an internal inconsistency between the LSTM baseline in Table 1 and the identical PMrel model in Table 2, and by the absence of controlled hyperparameter selection, error bars, and significance tests. The central claim of a large improvement therefore is not yet supported as stated.","major_comments":[{"comment":"PMrel in Table 2 is a per-pedestrian LSTM with relative displacement input and Gaussian output, which is the same model as the LSTM baseline in Table 1. Under the paper's stated protocol ('we apply the same training and testing procedures for all methods'), these numbers should agree or nearly agree, but they differ by 1.08 m ADE (0.66 vs 1.74) and 1.81 m FDE (1.36 vs 3.17). This discrepancy indicates that the Table 1 baselines were either not run under the same two-stage fine-tuning protocol or were taken from prior publications. Since PMrel alone comes within 0.10 ADE and 0.21 FDE of the full Scene-LSTM, the headline improvement from 0.79 to 0.56 ADE cannot be cleanly attributed to the scene-memory mechanism. The authors must rerun all baselines under the identical Stage 1 plus Stage 2 protocol and report consistent numbers.","section":"§5.2, Table 2 vs §5.1, Table 1"},{"comment":"The grid and subgrid sizes (selected as 8x8 in Figure 5b) and the stage-2 training fraction of 50% are tuned on the same five ETH/UCY videos used for the main results, with no held-out validation split. This selection on the test data can inflate reported gains. Additionally, no error bars, multiple seeds, or significance tests are reported, so the phrase 'significantly outperforms' is not statistically supported. Please fix hyperparameters using a separate validation set or nested cross-validation, and report variance and significance measures.","section":"§5.1, §5.4, Figure 5"},{"comment":"The hard filter's core decisions depend on unspecified algorithmic criteria: 'a grid cell is characterized as linear if all human trajectories in this grid cell are linear' and 'a common subgrid path is a path between two subgrids commonly travelled by a number of pedestrians greater than p = 3'. No precise definition of linearity (e.g., curvature threshold, number of points, fitting residual) or of how a trajectory is parsed into subgrid paths is provided. These details are load-bearing because they determine which scene data are used at prediction time, and without them the method cannot be reproduced or compared fairly.","section":"§4, Implementation Details, Hard filter"},{"comment":"The average NDE advantage over SGAN-20V-1 (1.00 vs 1.36) is not consistent across sequences: on ETH-Univ Scene-LSTM is worse (1.88 vs 1.55) and on UCY-Zara02 it is also worse (0.93 vs 0.88). The paper acknowledges this qualitatively, but reporting only sequence-level numbers without error bars or significance tests leaves it unclear whether the average difference is meaningful. Per-sequence confidence intervals or a paired test over multiple seeds would clarify this.","section":"§5.1, Table 1, NDE results"}],"minor_comments":[{"comment":"The abstract contains 'a pedestrians future locations' missing an apostrophe, and the author affiliation contains 'Univesity' and later 'filer' in the contributions list.","section":"Abstract and author affiliation"},{"comment":"The reported prediction horizons 'Tpred = 4.8 and 6.4 seconds' are not tied to the observed-frame length or the frame rates of Town Center, PETS09S2, and Grand Central; please specify the number of frames and frame rates for each dataset.","section":"§5.4"},{"comment":"The red and blue lines mentioned in the text are not labeled directly in the figure; please add a legend and axis labels so the reader can interpret the grid/subgrid-size experiments.","section":"Figure 5"},{"comment":"The metrics are described as 'mean square error (Euclidean distance)', which is inaccurate because the reported values are average Euclidean distances, not squared errors; please rephrase to 'average Euclidean distance' or 'mean squared Euclidean distance' consistently.","section":"§5, Metrics"}],"recommendation":"major_revision","confidential_remarks":"The central idea is promising, but the quantitative comparison must be made internally consistent. If the authors cannot reproduce a comparable LSTM baseline under the same protocol, the claimed improvement over prior methods is unsupported and the paper should be rejected. The current inconsistency is fixable by rerunning baselines, so I recommend major revision rather than rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper for the architecture, not for the numbers. The Scene-LSTM idea — a two-level grid that learns common routes from observed trajectories, plus a hard filter for linear/non-linear regions and a soft filter to gate the scene memory per pedestrian — is genuinely interesting and not something I've seen in earlier scene-aware trajectory work. The ablation study suggests the components do something: PMrel alone gets 0.66 ADE, and the full model gets 0.56 on ETH/UCY. That 0.1 m gain is plausible and worth investigating.\n\nThe problem is that the quantitative comparison to prior methods cannot be trusted. The LSTM baseline in Table 1 is reported at 1.74 ADE, but the PMrel ablation in Table 2 is 0.66 ADE. These are the same model: a per-pedestrian LSTM with relative displacements and Gaussian output. Under the stated protocol (\"we apply the same training and testing procedures for all methods\"), they should be identical. They are not. This means the Table 1 baseline numbers were probably taken from earlier papers rather than rerun under the Stage 2 fine-tuning protocol. If the baselines weren't rerun, the comparison is uncontrolled, and the headline reduction from 0.79 to 0.56 ADE over SGAN-20V-1 does not support the claim that the scene memory is responsible. The more modest comparison in the ablation — 0.66 vs 0.56 — is the only controlled one, and it still supports the method, but with a much smaller effect.\n\nTwo other soft spots. The grid and subgrid sizes are tuned on ETH/UCY and then tested on ETH/UCY, which can inflate the result. And the Stage 2 fine-tuning on 50% of the target video is a strong protocol, but unless every baseline gets the same fine-tuning the comparison is not fair. No error bars or code either.\n\nWho should read this: anyone designing scene-conditioned prediction models will find the filtering mechanism worth thinking about. But the paper should be cited cautiously, if at all, until the baselines are rerun.\n\nRecommendation: send it to review, but insist on a controlled re-evaluation of all baselines under the same protocol, error bars, and code. If the authors do that, the core result might hold up, but at a smaller scale than claimed.","headline":"The scene-memory idea is real, but the quantitative story collapses when you notice the same LSTM model is reported as 0.66 and 1.74 ADE across two tables.","tokens_in":12365,"tokens_out":4702,"would_cite":false,"duration_ms":45448,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scene-LSTM couples a two-level scene grid of common paths with per-pedestrian LSTM motion to beat social-interaction models on five benchmark datasets.","keywords":["human trajectory prediction","LSTM","scene information","common human movement","two-level grid","hard filter","soft filter","crowd analysis"],"falsifier":"Swap the training and testing halves of each video in Stage 2 (train on the second 50% of frames, test on the first 50%). If the ADE and FDE gains over the Pedestrian-LSTM-only model shrink or disappear, then the hard filter is exploiting the particular half of the video rather than a stable property of the scene.","tokens_in":11349,"feed_emoji":"🚶","tokens_out":7613,"duration_ms":71624,"temperature":0.7,"pith_summary":"This paper sets out to show that the next positions of a pedestrian in a static crowd can be predicted more accurately by learning the common paths people already take through each local part of the scene than by modeling social interactions alone. The authors build a two-level grid over the video frame, train a Scene-LSTM for every grid cell to absorb all observed movements in that cell, and train a separate Pedestrian-LSTM for each person's own relative motion. A hard filter restricts the scene memory to non-linear cells and to subgrids crossed by at least three people, and a soft filter multiplies the scene memory by a gating vector derived from the pedestrian's location and recent motion. On ETH, UCY, Town Center, PETS09, and Grand Central they report average displacement error falling from 0.79 to 0.56 meters and final displacement error from 1.65 to 1.15 meters compared with the best method they compare against. If the claim holds, recurrent scene-specific structure is a cheaper and stronger cue than the interaction graphs most current predictors rely on.","feed_headline":"Scene grid of common paths shrinks trajectory prediction error","feed_subtitle":"Coupling a scene-memory LSTM with pedestrian motion cuts average error from 0.79 to 0.56 meters on benchmark crowds.","key_machinery":"The load-bearing mechanism is the two-level grid paired with the Scene Data Filter. Each grid cell carries a Scene-LSTM whose memory represents how people move through that cell; a cell is labeled linear if all training trajectories inside it are straight, and in linear cells the scene memory is ignored because the Pedestrian-LSTM alone suffices. For non-linear cells, an $m\\times m$ subgrid parses each trajectory into subgrid paths, and paths traveled by more than $p=3$ pedestrians define 'common subgrids' that the hard filter treats as trustworthy. A soft filter then gates the selected cell memory with a sigmoid vector derived from the pedestrian's current subgrid and LSTM state, so the scene influence is tuned to the individual's recent behavior. The filtered scene vector is added to the pedestrian hidden state before a bivariate Gaussian is sampled for the next displacement; this coupling is what lets common local motion steer predictions without contaminating them with rare trajectories.","core_discovery":"The central claim is that a static scene's commonly traveled paths, encoded in a grid of LSTM cells, are sufficient to beat models built on social interaction. Each grid cell's Scene-LSTM is updated with the concatenation of a one-hot vector marking the pedestrian's subgrid position and that pedestrian's LSTM hidden state, so the cell memory accumulates typical local motion. The hard filter classifies each cell as linear or non-linear at preprocessing time; only non-linear cells are kept, their trajectories are parsed into subgrid paths, and only subgrids shared by more than $p=3$ pedestrians count as common. At prediction time, a pedestrian whose current subgrid is common receives the cell's hidden state, and the soft filter element-wise multiplies that state with a sigmoid gate computed from the one-hot location and the pedestrian hidden state, then adds the result into the pedestrian hidden state before sampling the next displacement from a bivariate Gaussian. The authors report that this combined model reduces average ADE from 0.79 to 0.56 meters and FDE from 1.65 to 1.15 meters on the five ETH/UCY sequences, and outperforms the same comparison methods on Town Center, PETS09, and Grand Central.","pith_inferences":["A natural stress test is to make the hard filter time-aware: recompute common subgrids over a sliding window, which would let the model follow shifts in crowd flow after events or train arrivals rather than freezing the first half of the video.","The same grid-plus-filter recipe could be applied to other agents, such as vehicles, cyclists, or warehouse robots, wherever recurring local paths dominate motion; the $p=3$ threshold would become a per-scene hyperparameter.","The dependency on stage-2 training suggests an implicit test: if the scene memory is trained on a different time of day or on a disjoint set of pedestrians and the gain shrinks, the 'common' paths are partly person-specific rather than purely scene-specific.","Combining the soft-filtered scene memory with a social pooling layer looks like the natural next architecture; the ablation's NDE numbers suggest the grid-level scene memory and the subgrid-level filter capture complementary parts of the error."],"forward_implications":["If the reported gains are real, trajectory prediction in static crowded scenes can rely primarily on scene-specific common paths, reducing the need for expensive social-interaction modeling.","The two-level grid and hard/soft filtering can be recomputed from raw trajectories alone, so the method should transfer to a new static scene after a short re-training pass on half of the new video's frames.","The ablation results imply that subgrid granularity is the controlling factor: too coarse a grid merges distinct paths, too fine a grid fragments them, and an 8x8 cell/subgrid split best captures common motion.","For robotics and tracking applications that require one best-guess trajectory, the model's single sampled prediction with lower FDE means better long-horizon endpoint estimates than the compared single-trajectory baselines.","On the three additional datasets, the improvements hold at longer horizons (6.4 seconds) as well as shorter ones, so the common-path signal is not only a short-range effect."],"supporting_citations":[{"why":"Supplies the Pedestrian-LSTM motion model and the social pooling baseline, and the leave-one-out training protocol that Stage 1 adopts.","marker":"[1]"},{"why":"Provides the SGAN-20V-1 and SGAN-20VP-1 comparison models whose ADE/FDE numbers Scene-LSTM is measured against.","marker":"[7]"},{"why":"Supplies the ETH dataset (Hotel and Univ sequences) used in the main evaluation.","marker":"[10]"},{"why":"Supplies the UCY dataset, the ADE/NDE/FDE metrics, and the energy-based interaction baseline.","marker":"[14]"},{"why":"Supplies the Town Center surveillance sequence used to test generalization to a new scene.","marker":"[4]"},{"why":"Supplies the PETS09S2 videos used to test generalization to short crowded sequences.","marker":"[5]"},{"why":"Supplies the Grand Central long-duration video used to test generalization to dense, complex crowds.","marker":"[23]"}],"fun_headline_variants":["Scene-memory LSTM cuts pedestrian path error by 29%","Common-path grid sharpens LSTM trajectory forecasts","Gridded scene knowledge shrinks ADE from 0.79 to 0.56 m","Scene grid, not social cues, key to pedestrian prediction","Typical routes in scene grid boost LSTM accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the common paths learned from the first half of a video's frames remain valid for the second half, so a pedestrian standing in a common subgrid can safely be steered by that learned path.","fun_headline_variants_meta":{"raw":{"variants":["Scene-memory LSTM cuts pedestrian path error by 29%","Common-path grid sharpens LSTM trajectory forecasts","Gridded scene knowledge shrinks ADE from 0.79 to 0.56 m","Scene grid, not social cues, key to pedestrian prediction","Typical routes in scene grid boost LSTM accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000274,"raw_usage":{"total_tokens":1639,"prompt_tokens":945,"completion_tokens":694,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":606}},"tokens_in":561,"tokens_out":694,"duration_ms":7822,"temperature":1.0,"reasoning_tokens":606,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:25:30.028382+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Swap the training and testing halves of each video in Stage 2 (train on the second 50% of frames, test on the first 50%). If the ADE and FDE gains over the Pedestrian-LSTM-only model shrink or disappear, then the hard filter is exploiting the particular half of the video rather than a stable property of the scene.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the Pedestrian-LSTM motion model and the social pooling baseline, and the leave-one-out training protocol that Stage 1 adopts."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Provides the SGAN-20V-1 and SGAN-20VP-1 comparison models whose ADE/FDE numbers Scene-LSTM is measured against."},{"cited_title":"In: Computer graphics forum","cited_arxiv_id":null,"evidence_quote":"Supplies the ETH dataset (Hotel and Univ sequences) used in the main evaluation."},{"cited_title":"In: Computer Vision, 2009 IEEE 12th International Conference on","cited_arxiv_id":null,"evidence_quote":"Supplies the UCY dataset, the ADE/NDE/FDE metrics, and the energy-based interaction baseline."},{"cited_title":"In: CVPR","cited_arxiv_id":null,"evidence_quote":"Supplies the Town Center surveillance sequence used to test generalization to a new scene."},{"cited_title":"In: Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance","cited_arxiv_id":null,"evidence_quote":"Supplies the PETS09S2 videos used to test generalization to short crowded sequences."},{"cited_title":"In: 2012 IEEE Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the Grand Central long-duration video used to test generalization to dense, complex crowds."}],"review_version":1}