{"id":"6fea449a-c6ac-4609-a46b-f9a393f9ec85","arxiv_id":"2508.15336","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"MediaPipe pose landmarks fed to LSTM, GRU, and 1D CNN predict pedestrian crossing intent on a private 60-video dataset, with GRU reaching 89.24% test AUC and 1D CNN running at 1 ms inference.","lead":"This study uses pose landmarks from MediaPipe and three sequence models to predict whether a pedestrian will cross a road. GRU achieved the highest test AUC (89.24%) while 1D CNN was fastest, but the small private dataset and missing code limit the result.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported AUCs likely reflect train/test leakage: a random 90:10 split of overlapping 15-frame windows from only 60 clips almost certainly places the same video in both train and test. No video-wise split is described, so generalization claims are unverified.","rationale":"The reader's explicit weakest assumption was manual label noise, but the reader's rationale also mentioned an ambiguous train/test split that may leak video sequences. My stress-test focuses on the split leakage because it is the single most load-bearing threat to the central claim: even if labels were perfect, a random window-level split from only 60 clips would make the reported test AUCs an unreliable measure of generalization. This does not contradict the reader's conditional verdict; it strengthens the condition under which the paper should be accepted. The paper provides no code or data, so the split cannot be verified post hoc, but a concrete re-analysis with video-wise splitting would settle the concern. I therefore keep the verdict as CONDITIONAL/UNCHANGED rather than escalating to REJECT, because the issue is addressable and the paper's contribution is framed as a feasibility study rather than a state-of-the-art benchmark claim.","tokens_in":8404,"tokens_out":2072,"duration_ms":22780,"concrete_test":"Obtain the source video/participant IDs for all 15,585 training and 1,823 test windows and check whether any test window originates from a video clip also present in training. Then re-run the full pipeline with a leave-videos-out split (e.g., train on 48 clips, validate on 6, test on the remaining 6, never tuning on test) or leave-subject-out split over the 20 volunteers, and report test AUC/accuracy for each model on this split. If the GRU test AUC drops substantially below 89.24% or the model ordering changes, the reported Table 1 results are leakage artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that MediaPipe pose landmarks plus GRU/LSTM achieve test AUC above 85% for crossing-intent prediction. The load-bearing evidence is Table 1, but the test protocol in Section IV appears to be a random 90:10 split of 15-frame sequences drawn from only 60 short video clips (about 300 frames each; trainset 15,585 and testset 1,823 data points). With overlapping sliding windows, a random split at the sequence/frame level puts many windows from the same video clip and same actor into both training and test sets. Recurrent and convolutional models can then memorize clip-specific pose dynamics, and the test set contains near-duplicate temporal contexts already seen in training. This inflates accuracy and AUC and invalidates the comparison between GRU, LSTM, and 1D CNN as evidence of generalization to new pedestrians or new road-crossing episodes. The paper never states that the split was performed video-wise or subject-wise; the phrase 'a few more videos were collected and kept aside to test the model' is also not reconciled with the 90:10 split. Without a clean split, the abstract's claim that GRU is better at intent prediction is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end framework for predicting pedestrian road-crossing intent from video. Pose landmarks are extracted with MediaPipe (33 keypoints, X/Y coordinates), and three sequence models—LSTM, GRU, and 1D CNN—are trained on 15-frame windows to classify the next frame as crossing/not crossing. Experiments on a self-collected dataset of 60 videos (about 17,408 windows) report that GRU achieves the best test AUC (89.24%), followed by LSTM (86.38%) and 1D CNN (74.27%), with 1D CNN being fastest (1 ms inference). The authors claim the framework can support autonomous driving applications.","tokens_in":8735,"tokens_out":2939,"duration_ms":34437,"significance":"If the reported results were valid, the paper would offer a lightweight, landmark-based alternative to full-video models for pedestrian intent prediction, with potential real-time application. The use of MediaPipe for pose extraction and the comparison of three sequence models are reasonable design choices, and the authors are transparent about computational constraints. However, the evaluation methodology has fundamental weaknesses—most notably the lack of a video-wise train/test split and the reliance on subjective, retrospectively modified labels—that cast doubt on the quantitative claims. The paper does not release code or data, so the results are not independently checkable. The core idea is plausible, but the evidence presented does not currently establish the claimed superiority of GRU over LSTM or the generalizability of the models.","major_comments":[{"comment":"The reported test performance is likely inflated by train/test leakage. The dataset consists of 60 video clips, each roughly 300 frames, and the 15-frame windows are almost certainly overlapping. The paper states only that the dataset was split into trainset and testset with a 90:10 ratio (Section IV), giving 15,585 and 1,823 data points. Without a video-wise or subject-wise split, many windows from the same clip—and even the same actor—appear in both training and test sets. Recurrent and convolutional models can then memorize clip-specific pose dynamics, so the test AUC values in Table 1 do not measure generalization to new pedestrians or new crossing episodes. The sentence in Section III that 'a few more videos were collected and kept aside to test the model' is also inconsistent with the 90:10 split described in Section IV. The authors should rerun the experiments with grouping by vid","section":"Section IV, Table 1 and data split"},{"comment":"The ground-truth labels are subjective and may include future information. Section III states that labels were assigned manually by observing frames individually, and Section IV adds that frames depicting backtracking were reclassified from 'crossing' to 'not crossing.' This reclassification is done after the fact: a frame at time t is labeled 0 because the pedestrian later decides to backtrack, even though at time t the crossing intention may have been present. Thus the labels are retrospective, not predictive, and the model is trained to recognize a label that depends on future events beyond the 15-frame input window. No inter-annotator agreement is reported, and no protocol for prospective labeling is defined. The authors should define a labeling rule that uses only information up to the current frame, and report annotation reliability.","section":"Sections III and IV, labeling protocol"},{"comment":"Table 1 reports single point estimates for accuracy, AUC, and inference time without any measure of variance. With only 60 clips and class imbalance, the difference between GRU AUC (89.24%) and LSTM AUC (86.38%) may not be statistically significant. The paper also reports only ROC AUC; given class imbalance, precision-recall AUC or a confusion matrix would be more informative. The authors should provide confidence intervals, multiple runs, or k-fold cross-validation with video-wise grouping to support the claim that GRU is 'better in predicting intent' than LSTM.","section":"Section IV, evaluation statistics"}],"minor_comments":[{"comment":"There are numerous typographical errors: 'fame' for frame, 'Long Short Tern Memory' for Long Short-Term Memory, 'Robicqet' for Robicquet, 'Fe-Fei' for Fei-Fei, 'Savarse' for Savarese, and 'SLTM' for LSTM in reference [10]. A careful proofread is needed.","section":"Throughout"},{"comment":"The cross-references to figures are inconsistent. The text says 'The pose landmarks are shown in Figure 1' but Figure 1 is a training data flowchart and Figure 2 shows key points. The figure labels should be corrected.","section":"Section III, figures"},{"comment":"The convolution equation uses `𝑤𝑖 𝑥𝑡−𝑖` with the summation index going forward; typically the kernel index is reversed. This is notationally confusing and should be fixed.","section":"Section III, equation for 1D CNN"},{"comment":"The reported average inference time of 43 ms per frame includes MediaPipe and intermediate data handling, but the paper does not specify whether this is on CPU or GPU; the earlier inference times in Table 1 are presumably GPU. Clarify the hardware and the measurement conditions.","section":"Section IV, inference time"},{"comment":"The phrase 'the existing pretrained models stayed updated' is unclear. It would be helpful to state explicitly that MediaPipe was chosen because it is actively maintained by Google.","section":"Section II, literature review"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim depends entirely on a clean evaluation, and the current test protocol is not acceptable. However, the flaw is fixable within the scope of the paper: the authors can redo the experiments with a video-wise split, report variance, and clarify the labeling protocol. If the results survive that re-evaluation, the paper may be publishable. I would not reject outright, but the current version is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a small, honest, but methodologically under-powered study. The new bit is using MediaPipe 2D pose landmarks (66 coordinates per frame) as the sole input to LSTM/GRU/1D-CNN sequence models for pedestrian crossing intent. That specific input representation is a reasonable thing to try, and the paper is transparent about its setup: about 60 single-pedestrian clips, 20 volunteers, manual frame-level labeling, and a comparison of three standard sequence models. The authors also clearly state the single-pedestrian limitation and the speed/accuracy tradeoff. So there is a kernel of a useful empirical result here, if the evaluation holds up.\n\nThe problem is that the reported numbers (GRU AUC 89.24%, LSTM 86.38%, 1D-CNN 74.27%) are probably inflated by train/test leakage. The split is described as a random 90:10 split of 15-frame sliding windows. With only 60 videos and roughly 300 frames each, a random split at the window level almost certainly puts windows from the same video, same actor, and overlapping time steps into both train and test. Recurrent models can then memorize clip-specific pose dynamics, and the test set contains near-duplicates of training contexts. The paper never says the split was done video-wise or subject-wise, and the sentence about 'a few more videos were collected and kept aside to test the model' is never reconciled with the 90:10 split. That is a load-bearing weakness, not a cosmetic one.\n\nThe other soft spots are real but less severe: manual labels with no inter-annotator agreement, post-hoc reclassification of backtracking frames, no error bars or repeated splits, and no comparison on PIE or JAAD even though PIE is cited and discarded due to size. No code or data are released, so the numbers cannot be independently checked. These are all fixable, but as reported the central comparative claim about GRU versus LSTM is not established.\n\nStill, this is not a paper to dismiss outright. The idea is simple, cheap, and potentially useful for low-resource autonomous driving scenarios. A serious referee could ask for a video-wise split, error bars, and a public benchmark, and the work could become a modest but valid empirical note. I'd send it to review if the authors are willing to address the leakage issue; otherwise I'd encourage them to revise first.","headline":"The core claim is plausible but the current evaluation almost certainly leaks training and test data across the same video clips, so the reported AUCs don't yet establish generalization.","tokens_in":9201,"tokens_out":1264,"would_cite":false,"duration_ms":15941,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pipeline that turns 15 frames of body-keypoint coordinates into a crossing-intent prediction finds GRU the most accurate sequence model, with 1D CNN fastest.","keywords":["pedestrian intent prediction","road crossing behaviour","pose detection","MediaPipe","LSTM","GRU","1D CNN","sequence modelling"],"falsifier":"Run the same MediaPipe + 15-frame-window pipeline with the three models on a public pedestrian-intention dataset (e.g., PIE) or have two independent annotators relabel the same 60 clips; if GRU's AUC no longer exceeds LSTM's on a larger sample, or the labelers disagree substantially, the ranking and the absolute AUC values in Table 1 would not transfer.","tokens_in":8334,"feed_emoji":"🚶","tokens_out":6919,"duration_ms":60924,"temperature":0.7,"pith_summary":"The paper asks whether a computer can predict, from a short video of a person near a road, whether that person is about to cross. It builds an end-to-end pipeline that extracts 33 body keypoints per frame with MediaPipe, feeds 15-frame sequences of x-y coordinates into three sequence models, and labels the next frame as crossing or not. On a private dataset of 60 videos from 20 volunteers, the GRU achieved the best test AUC (89.24%), slightly above the LSTM (86.38%) and higher than the 1D CNN (74.27%), while the 1D CNN had the fastest inference (1 ms per window vs. 2–3 ms). The authors claim that pose landmarks alone carry enough signal to anticipate crossing intent, and that the speed/accuracy trade-off favors GRU for accuracy and CNN for speed. The same framework is proposed for autonomous vehicle decision-making and for other movement-intent sequence tasks.","feed_headline":"GRU beats LSTM at reading pedestrian crossing intent","feed_subtitle":"MediaPipe pose landmarks plus 15-frame windows give GRU 89% AUC on a private video set; CNN runs in 1 ms.","key_machinery":"The central object is the sliding-window sequence of MediaPipe pose landmarks: 33 body keypoints per frame, reduced to their X and Y coordinates (66 values per frame), with a window of 15 frames used to classify the 16th frame as crossing or not. This representation converts raw video into a compact temporal signal. The comparison is carried by three sequence models—LSTM and GRU (two-layer, 50 hidden units, dropout 0.5) and a 1D CNN (kernel size 3, global average pooling)—all trained with binary cross-entropy and evaluated by accuracy and AUC on a common held-out test set.","core_discovery":"On its own terms, the paper's central claim is that the coordinates of a single pedestrian's body keypoints, extracted per frame by MediaPipe and read as 15-frame sequences, are sufficient for sequence models to infer whether the pedestrian is about to cross a road. The paper reports that GRU is the most accurate of the three models at this binary intent task on the held-out test set (AUC 89.24%), LSTM is close behind (86.38%), and 1D CNN is markedly lower (74.27%) but fastest (1 ms per window). The authors also claim that end-to-end processing including pose extraction runs at about 43 ms per frame on a consumer laptop, too slow for 30 fps video but usable at lower frame rates. The practica","pith_inferences":["The recurrent models' edge over the 1D CNN may come from their ability to retain information across the 15-frame window; testing wider CNN kernels or dilated convolutions could close the gap.","The reported 43 ms per frame is dominated by MediaPipe and data handling rather than the sequence model, so optimizing pose extraction could make even the GRU reach 30 fps on similar hardware.","The manual labelling and lack of inter-annotator agreement mean a public-dataset replication would be the direct way to test whether GRU's ranking transfers to other annotators and scenes.","Scaling to multiple pedestrians in a frame, which the paper lists as future work, will require assigning landmarks to individuals and may degrade the clean single-person performance."],"forward_implications":["For this task, pose landmarks alone suffice: no background, object detector, or trajectory history is needed to get AUC above 86% for the recurrent models.","GRU gives the best accuracy/AUC trade-off, making it the candidate where precision matters more than milliseconds; 1D CNN is the candidate where speed matters most.","A 15-frame window (half a second at 30 fps) is enough to signal intent, which means the system can warn a vehicle about a likely crossing almost immediately.","The end-to-end 43 ms/frame latency means the current, unoptimised pipeline cannot keep up with 30 fps video, but it can run in real time on lower-frame-rate or downsampled feeds.","The same sequence-modelling setup can be transferred to other movement-intent tasks, such as sports analytics or detecting suspicious movement near restricted areas."],"supporting_citations":[{"why":"Sets up the task of predicting pedestrian assertiveness from observable behaviour, giving the paper its problem baseline.","marker":"[1]"},{"why":"Prior work on whether pedestrians want to cross; provides a comparison point in intention estimation.","marker":"[2]"},{"why":"The LSTM-based crossing-intention model this study adapts by swapping trajectory features for pose landmarks.","marker":"[3]"},{"why":"Supplies the LSTM/RNN equations the paper uses to describe the recurrent models.","marker":"[4]"},{"why":"Demonstrates the MediaPipe-plus-LSTM pipeline for sign language, the template for this study's architecture.","marker":"[14]"},{"why":"Shows pose estimation plus CNN for suspicious-movement classification, an analogue for the CNN variant.","marker":"[16]"},{"why":"The PIE dataset, which the study evaluated and set aside due to size; defines the public benchmark this private dataset is compared against.","marker":"[17]"},{"why":"Original LSTM paper underpinning the recurrent architecture and its four gates.","marker":"[20]"}],"fun_headline_variants":["GRU best for pedestrian crossing intent; CNN fastest","GRU edges LSTM in crossing intent accuracy, CNN fastest","Pedestrian crossing predictor: GRU accurate, CNN fast","Crossing intent: GRU beats LSTM, CNN leads in speed"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that a human watching each frame can reliably label whether the pedestrian intends to cross, with no inter-annotator agreement reported, so the model scores inherit whatever noise or bias is in those manual labels.","fun_headline_variants_meta":{"raw":{"variants":["GRU best for pedestrian crossing intent; CNN fastest","GRU edges LSTM in crossing intent accuracy, CNN fastest","Pedestrian crossing predictor: GRU accurate, CNN fast","Crossing intent: GRU beats LSTM, CNN leads in speed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1155,"prompt_tokens":701,"completion_tokens":454,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":445,"completion_tokens_details":{"reasoning_tokens":396}},"tokens_in":445,"tokens_out":454,"duration_ms":5401,"temperature":1.0,"reasoning_tokens":396,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T17:57:49.240748+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same MediaPipe + 15-frame-window pipeline with the three models on a public pedestrian-intention dataset (e.g., PIE) or have two independent annotators relabel the same 60 clips; if GRU's AUC no longer exceeds LSTM's on a larger sample, or the labelers disagree substantially, the ranking and the absolute AUC values in Table 1 would not transfer.","supporting_citations":[{"cited_title":"Predicting pedestrian road-crossing assertiveness for autonomous vehicle control,","cited_arxiv_id":null,"evidence_quote":"Sets up the task of predicting pedestrian assertiveness from observable behaviour, giving the paper its problem baseline."},{"cited_title":"Do they want to cross? understanding pedestrian intention for behavior prediction,","cited_arxiv_id":null,"evidence_quote":"Prior work on whether pedestrians want to cross; provides a comparison point in intention estimation."},{"cited_title":"Prediction of pedestrian crossing intentions at intersections based on long short-term memory recurrent neural network,","cited_arxiv_id":null,"evidence_quote":"The LSTM-based crossing-intention model this study adapts by swapping trajectory features for pose landmarks."},{"cited_title":"Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network,","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM/RNN equations the paper uses to describe the recurrent models."},{"cited_title":"American sign language recognition for alphabets using MediaPipe and LSTM,","cited_arxiv_id":null,"evidence_quote":"Demonstrates the MediaPipe-plus-LSTM pipeline for sign language, the template for this study's architecture."},{"cited_title":"Suspicious human activity recognition using 2d pose estimation and convolutional neural network,","cited_arxiv_id":null,"evidence_quote":"Shows pose estimation plus CNN for suspicious-movement classification, an analogue for the CNN variant."},{"cited_title":"Pie: A large-scale dataset and models for pedestrian intention estimation and trajectory prediction,","cited_arxiv_id":null,"evidence_quote":"The PIE dataset, which the study evaluated and set aside due to size; defines the public benchmark this private dataset is compared against."}],"review_version":1}