{"id":"544c2785-c229-4cf7-b1a6-62ab81507af3","arxiv_id":"2501.13848","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"ScenePTP fuses SGCN interaction features with OneFormer semantic scene features via cross-attention and reports 0.252/0.372 ADE/FDE, but only on a four-scene subset.","lead":"A trajectory-prediction model that combines social graphs with semantic scene features reports large accuracy gains on the ETH/UCY benchmarks. The headline improvement is hard to trust because the reported average excludes the UNIV scene that earlier baselines include.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The four-scene AVG in Table 1 is internally consistent because baseline AVGs are also over four scenes; the load-bearing gap is instead that the standard five-scene benchmark is not run, and the §3.5 cross-attention is never spatially grounded to pedestrian positions.","rationale":"","tokens_in":7772,"tokens_out":9203,"duration_ms":88780,"concrete_test":"Re-run the complete ETH/UCY benchmark under the standard leave-one-out protocol on all five scenes, including UNIV, and report per-scene and AVG ADE/FDE with at least three random seeds; if the UNIV frames truly are unavailable, obtain the students003 video or clearly relabel all claims as four-scene results and compare against baselines re-evaluated on the same four scenes. This single check settles whether the headline SOTA claim generalizes beyond the chosen subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption is not exactly right. In Table 1, the AVG column is the mean of the four listed scene columns for every model, including SGCN: (0.57+0.31+0.29+0.22)/4=0.35. So ScenePTP's 0.252/0.372 is compared on the same four-scene subset, not against five-scene averages. The real protocol concern is that the paper's headline 'significantly outperforms existing state-of-the-art approaches' is made without completing the standard ETH/UCY five-scene leave-one-out evaluation: UNIV is dropped in Section 4.1, with no result or code to check it. Since UNIV is a standard and typically harder scene, the SOTA claim is not established. A second, more methodological load-bearing concern is Section 3.5: Eq. (1) uses graph features as Q and scene features as K/V, but the paper never specifies how H_scene is associated with each pedestrian's position (localized crops, homography-aligned feature maps, coordinate embeddings, or a global descriptor). Without spatial grounding, cross-attention cannot 'prioritize relevant environmental factors that influence each pedestrian's movement'; the mechanism claimed for the large HOTEL gain (0.31 to 0.145 ADE) is underspecified and untested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ScenePTP, a pedestrian trajectory prediction model that combines Sparse Graph Convolutional Network (SGCN) interaction features with scene features extracted via Real-ESRGAN image enhancement, OneFormer semantic segmentation, and ResNet-18 visual features. The two feature streams are fused with a cross-attention module and a temporal convolutional network predicts future positions. The authors report ADE/FDE of 0.252/0.372 on a four-scene subset of ETH/UCY (ETH, HOTEL, ZARA1, ZARA2), claiming significant improvement over state-of-the-art baselines, with the largest gain on HOTEL. A two-configuration ablation suggests that adding semantic maps improves over frame features alone.","tokens_in":8005,"tokens_out":7138,"duration_ms":64374,"significance":"If the reported results are reproducible and correctly attributed, the proposed integration of semantic scene features with sparse graph interactions would be a practically relevant contribution, since the claimed margins over SGCN on HOTEL and ETH are large. The paper addresses an important problem and uses standard datasets and metrics, and the high-level architecture is clearly presented. However, the significance is currently not established: the central SOTA claim depends on an incomplete evaluation (UNIV omitted), the cross-attention mechanism that is supposed to ground scene features to individual pedestrians is underspecified, no statistical validation is provided, and the ablation does not isolate the proposed scene-fusion branch from the SGCN backbone. These issues prevent the reader from assessing whether the improvement is due to the method or to evaluation and implementation choices.","major_comments":[{"comment":"The standard ETH/UCY evaluation is a five-scene leave-one-out protocol, but results are reported on only four scenes because UNIV frames are said to be unavailable. I checked the AVG row in Table 1: for every listed model, including SGCN, the AVG is the mean of the same four scene columns, so the aggregate comparison is internally consistent and not an apples-to-oranges arithmetic comparison. The load-bearing problem is different: the abstract and Section 4.3 claim that the method 'significantly outperforms existing state-of-the-art approaches' on the ETH/UCY benchmark, while UNIV, a standard and typically challenging scene, is simply absent, with no result, no baseline numbers, and no code to reproduce the four-scene numbers. The SOTA claim over the full benchmark is therefore not established. The authors must either provide a UNIV result or explicitly and consistently scope every claim in the abstract, Section 4.3, and the conclusion to the four-scene subset.","section":"§4.1, Table 1"},{"comment":"The cross-attention fusion is underspecified to the point that the paper's central mechanism is not a well-defined operation. The text says graph features serve as Q while scene features serve as K/V, but it never states whether H_scene is a single global vector, a flattened feature map, or a set of tokens, nor how scene feature locations are associated with each pedestrian's spatial coordinates (localized crops, homography alignment, coordinate embeddings, or a global descriptor). If H_scene is a global descriptor, then the softmax in Eq. (1) has exactly one key-value pair and the attention weight is trivially 1 for every pedestrian, so the model cannot 'prioritize relevant environmental factors that influence each pedestrian's movement' as claimed. Since the large reported HOTEL improvement (ADE 0.31 to 0.145) is attributed to exactly this mechanism, the architecture must be specified at the tensor level and ablated before the result can be interpreted.","section":"§3.5, Eq. (1)"},{"comment":"All reported ADE/FDE numbers are single runs with no error bars, no multiple seeds, and no statistical significance test, despite the abstract and Section 4.3 using the words 'significantly' and 'significant margins.' The margins over SGCN are large on some scenes, but per-scene variance is unknown and the model's advantage is not uniform. At minimum, the authors should report mean and standard deviation over several seeds, or an equivalent confidence interval, to support the superiority claim. Without this, the reader cannot distinguish a genuine improvement from training noise or favorable initialization.","section":"§4.3, Table 1"},{"comment":"The ablation study compares 'w/o Maps' (frame features only) with 'w/ Maps' (frame features plus semantic maps), but both configurations include the full cross-attention module and the scene-feature branch. There is no ablation that removes the scene branch entirely and uses only SGCN features, so the paper never isolates the contribution of the proposed scene-feature fusion over the SGCN backbone. Given that the SGCN baseline numbers in Table 1 are taken from the original SGCN paper rather than from the authors' own reimplementation, the reported improvement cannot be attributed to the proposed module without a controlled comparison using the same training pipeline, data pre-processing, and evaluation protocol.","section":"§4.4, Table 2"}],"minor_comments":[{"comment":"The ADE formula sums over τ = 1 to T, but the problem definition in Section 3.1 defines future ground truth as Y_i^{t:T} = {(x_i^τ, y_i^τ) | τ = t+1, ..., T}. The summation should run over the prediction horizon (τ = t+1 to T) or use a consistently defined index set.","section":"§3.6, Eq. (6)"},{"comment":"The paper gives no implementation details that would allow reproduction: no batch size, learning rate schedule, number of epochs, optimizer settings (beyond 'stochastic gradient descent'), attention head count, TCN kernel size, hidden dimensions, or train/validation splitting procedure, and no code link is provided.","section":"§4"},{"comment":"Reference [3] (Eigentrajectory) appears in the bibliography but is never cited in the text, and Reference [16] (Holistic LSTM) is cited but the surrounding sentence does not clearly connect to the citation.","section":"References"},{"comment":"The qualitative analysis of semantic segmentation and image restoration relies only on visual examples; adding a quantitative metric such as mIoU over the relevant scene regions would make the claimed advantage of OneFormer plus Real-ESRGAN concrete and comparable.","section":"§4.5–§4.6"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the omission of UNIV appears to be data-driven, but the abstract's unqualified 'significantly outperforms' claim is overbroad. The authors should be asked to either obtain UNIV evaluation data or to rewrite the paper so that all contributions and comparisons are explicitly scoped to the four reported scenes. The underspecified cross-attention module is another serious concern that requires architectural clarification before any acceptance recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline result of this paper should not be taken at face value. The authors report ADE 0.252/FDE 0.372 on ETH/UCY and claim to 'significantly outperform' state-of-the-art, but the average is computed over four scenes, not the usual five. They disclose the UNIV exclusion (no frame data), which is honest, but it means the standard benchmark is not completed. If UNIV turns out to be a harder scene for their method, the whole improvement could shrink or vanish. The stress-test note got this right: the AVG column in Table 1 is internally consistent – SGCN's 0.35 is the mean of its four listed scenes – so the comparison within the four-scene subset is fair. The gap is the missing scene, not a miscalculation.\n\nWhat is actually new here is modest but real: the specific combination of Real-ESRGAN image restoration, OneFormer semantic segmentation, and cross-attention fusion of scene features into an SGCN. The ablation (Table 2) shows that adding semantic maps improves ADE/FDE on all four scenes, which is a legitimate, reproducible observation. The paper is transparent about the UNIV exclusion and about the fixed pre-trained modules.\n\nThe soft spots are in the mechanism and the evidence. Section 3.5's cross-attention is never spatially grounded: the scene features H_scene serve as Key/Value, but the paper doesn't say whether this is a global image descriptor, per-pedestrian crops, or a coordinate-aligned feature map. Without that, the claim that the model 'prioritizes relevant environmental factors' per pedestrian is hand-waving. That matters because the HOTEL gain (0.31 to 0.145 ADE) is huge; a plausible alternative is that the improvement comes from something else in the pipeline (e.g., better input resolution) rather than from grounded scene reasoning. There are also no error bars or multiple seeds, and the baselines stop at SGCN (2021); newer models like Trajectron++, AgentFormer, or EqMotion are absent. The phrase 'significantly outperforms' is asserted, not tested.\n\nWho should read this? Someone working on scene-conditioned trajectory prediction might be interested in the specific feature-extractor choices and the ablation result. But the headline claim is unsupported as reported. I would not reject it out of hand – it's a salvageable paper that needs a five-scene evaluation, a specified grounding mechanism, and variance reporting. As it stands, I wouldn't cite it for the SOTA numbers, though I'd be happy to see a revised version.","headline":"Plausible architecture, but the headline SOTA claim rests on a four-scene subset and an unspecified attention mechanism.","tokens_in":8617,"tokens_out":3729,"would_cite":false,"duration_ms":31846,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scene features push pedestrian path error down to 0.25 m on standard benchmarks.","keywords":["pedestrian trajectory prediction","scene features","semantic segmentation","cross-attention","sparse graph convolution","temporal convolutional network","ETH/UCY datasets","autonomous vehicles"],"falsifier":"Re-run the comparison on the exact same four-scene subset for all baselines, or obtain the UNIV frames and evaluate ScenePTP on UNIV as well; if the per-scene margins over SGCN shrink or vanish when the scene sets are matched, the claimed state-of-the-art improvement is not supported.","tokens_in":7559,"feed_emoji":"🚶","tokens_out":3635,"duration_ms":31823,"temperature":0.7,"pith_summary":"This paper tries to establish that pedestrian trajectory prediction improves substantially when scene context is explicitly fused with pedestrian interaction features. The proposed model, ScenePTP, first extracts visual and semantic scene features using image restoration, semantic segmentation, and a convolutional backbone, then combines them with sparse graph features of pedestrian interactions via a cross-attention mechanism. On the ETH/UCY scenes with available frames, it reports an average displacement error of 0.252 meters and a final displacement error of 0.372 meters, beating prior baselines. If these numbers hold, the work supports the claim that environmental context matters as much as social interactions for accurate trajectory forecasting.","feed_headline":"Scene features push pedestrian path error down to 0.25 m","feed_subtitle":"Fusing semantic maps with sparse pedestrian graphs via cross-attention beats prior baselines on ETH/UCY.","key_machinery":"The cross-attention fusion module is the central mechanism: graph features from the sparse graph interaction module form the Query, while scene features from the semantic/visual extraction pipeline form the Key and Value, with a residual connection that adds the original graph features back to the attention output. This lets the model select relevant environmental cues for each pedestrian's motion while retaining its learned social interaction representation, and the refined features are passed through a temporal convolutional network to predict the future trajectory.","core_discovery":"The paper claims that pedestrian trajectory prediction on the ETH/UCY benchmarks improves significantly when detailed scene information is added to a sparse graph interaction model. The scene representation is built by enhancing raw frames with Real-ESRGAN, running OneFormer semantic segmentation, and extracting features with ResNet-18 plus convolutional layers; these are concatenated and refined by an MLP. The graph features from SGCN serve as queries in a cross-attention layer that keys on the scene features, with a residual connection preserving the original interaction signal, and a temporal convolutional network produces the final trajectories. On the four scenes with available public frames (ETH, HOTEL, ZARA1, ZARA2), the model reports ADE/FDE of 0.252/0.372 meters, including a HOTEL ADE of 0.145 meters versus 0.31 for the SGCN baseline.","pith_inferences":["The paper's headline average is not directly comparable to baseline averages because ScenePTP's AVG is computed over only four scenes (ETH, HOTEL, ZARA1, ZARA2) while baselines like SGCN are averaged over all five including UNIV; if UNIV is a relatively high-error scene, part of the reported margin could come from excluding it.","The semantic segmentation backbone is trained on Cityscapes, an urban driving dataset, and its transfer to the ETH/UCY scenes may itself be a source of the gains; a segmentation model fine-tuned on those scenes might perform differently.","The same cross-attention fusion pattern could be applied to other context modalities, such as occupancy grids, map data, or audio cues, suggesting a general recipe for conditioning graph-based motion forecasting on arbitrary environment information."],"forward_implications":["If the reported results are correct, adding semantic scene context to a sparse graph interaction model yields a large accuracy gain on the ETH/UCY subset, with HOTEL showing the most dramatic improvement.","The ablation study indicates that semantic maps contribute consistently across all tested scenes, so future trajectory predictors should treat scene segmentation as a standard input rather than an optional extra.","The architecture shows that a pre-trained segmentation backbone can be frozen and combined with a graph-based interaction model through cross-attention, avoiding end-to-end retraining of the segmentation stage.","The use of image restoration before segmentation suggests that input image quality is a practical bottleneck for scene-aware trajectory prediction."],"supporting_citations":[{"why":"SGCN is the sparse graph interaction module that provides the pedestrian interaction features, and its published numbers are the main baseline the paper compares against.","marker":"[20]"},{"why":"The ETH and UCY datasets supply the trajectories and video frames used in all experiments, and the standard leave-one-out evaluation protocol comes from this line of work.","marker":"[15]"},{"why":"Real-ESRGAN performs the image enhancement that the paper shows improves semantic segmentation quality and downstream prediction.","marker":"[23]"},{"why":"OneFormer produces the semantic segmentation maps that are a core component of the scene feature representation.","marker":"[9]"},{"why":"ResNet-18 extracts visual features from the enhanced frames, providing the appearance-based part of the scene representation.","marker":"[7]"},{"why":"The cross-attention mechanism described in this citation is used to fuse scene features with graph features.","marker":"[10]"},{"why":"Social-STGCNN provides the temporal convolutional network used for final prediction and also serves as a baseline model in the comparison table.","marker":"[11]"},{"why":"Cityscapes is the dataset used to train the OneFormer segmentation model that produces the semantic maps.","marker":"[4]"}],"fun_headline_variants":["Scene-aware model predicts pedestrian paths with 0.25m error","Cross-attention fuses scene and social cues to cut path error","Pedestrian path prediction improved by semantic scene features","Sparse graph plus scene context yields 0.252 ADE","How scene features sharpen pedestrian trajectory forecasts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline average improvement rests on comparing ScenePTP's four-scene average (ETH, HOTEL, ZARA1, ZARA2) with baselines' five-scene averages that include UNIV, so if UNIV is a harder-than-average scene, the gain may come from dropping it rather than from the method.","fun_headline_variants_meta":{"raw":{"variants":["Scene-aware model predicts pedestrian paths with 0.25m error","Cross-attention fuses scene and social cues to cut path error","Pedestrian path prediction improved by semantic scene features","Sparse graph plus scene context yields 0.252 ADE","How scene features sharpen pedestrian trajectory forecasts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000323,"raw_usage":{"total_tokens":1795,"prompt_tokens":907,"completion_tokens":888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":807}},"tokens_in":523,"tokens_out":888,"duration_ms":7815,"temperature":1.0,"reasoning_tokens":807,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:32:06.947484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison on the exact same four-scene subset for all baselines, or obtain the UNIV frames and evaluate ScenePTP on UNIV as well; if the per-scene margins over SGCN shrink or vanish when the scene sets are matched, the claimed state-of-the-art improvement is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SGCN is the sparse graph interaction module that provides the pedestrian interaction features, and its published numbers are the main baseline the paper compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The ETH and UCY datasets supply the trajectories and video frames used in all experiments, and the standard leave-one-out evaluation protocol comes from this line of work."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Real-ESRGAN performs the image enhancement that the paper shows improves semantic segmentation quality and downstream prediction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OneFormer produces the semantic segmentation maps that are a core component of the scene feature representation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The cross-attention mechanism described in this citation is used to fuse scene features with graph features."}],"review_version":1}