{"id":"03d01045-66d7-40ab-bc66-99ccc492a41d","arxiv_id":"2508.20762","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Adding skip connections between Swin Transformer stages raises the CARLA Driving Score of the authors' end-to-end driving model from 29.7 (x13 CNN baseline) to 37.1 on Town05, in a single reported evaluation run.","lead":"This paper adds skip-stage connections to a Swin Transformer backbone inside an end-to-end CARLA driving model, claiming a higher Driving Score than a CNN-based baseline. The result is a modest, plausibly useful architecture tweak, but the evidence is a single route with no error bars and only one self-cited comparison model.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed DS gain (37.10 vs 29.71) is not established: one run, no variance, post-hoc selection on the test route, and a 14-point implementation-library gap.","rationale":"The reader's verdict is REJECT because the headline result is a single-run/post-hoc comparison. My read agrees and sharpens the point: the paper's own cross-library numbers--23.27 (mmsegmentation) vs 37.10 (Official PyTorch) for the same architecture--show that the claimed 7.39-point gain over x13 is within the range the authors attribute to framework effects. The skip-stage idea is plausible, and the within-OP comparison stage4 (32.73) vs stage1 to 4 (37.10) is suggestive, but it too is a single measurement. The concrete test would distinguish a real architectural effect from evaluation noise/implementation choice. Secondary issues (mis-cited complexity, duplicate Table III rows, future-tense ablation) weaken trust but are not load-bearing for the central claim. Thus the reader's REJECT should stand until controlled evaluation is provided.","tokens_in":17126,"tokens_out":7311,"duration_ms":69412,"concrete_test":"Fix one implementation (Official PyTorch) and one data pipeline. Train x13, SKGE-Swin-tiny (stage 4, no skip), and SKGE-Swin-tiny (stage 1 to 4) for at least 10 seeds. Evaluate each seed on Town05 long and on a held-out route (e.g., Town01 long), reporting mean +/- SD of DS/RC/IP. Pre-register stage selection on a validation split, then apply a paired bootstrap test for stage 1 to 4 vs x13 and vs stage 4. If the advantage remains after this procedure and exceeds inter-seed SD on both routes, the claim is supported; if not, the current 37.10 result is an evaluation/selection artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison is uncontrolled in three compounding ways. First, Section IV.C says evaluation was on the Town05 long route; no seeds, repetitions, or variance are reported, so there is no way to know whether 37.10 vs 29.71 is larger than run-to-run DS noise. Second, the winning configuration (SKGE-Swin-tiny, stage 1 to 4, Official PyTorch) was selected after comparing candidate variants on that same test evaluation (Tables IV-V, Fig. 3 caption), so the reported DS is a max-over-configurations, not a pre-registered estimate. Third, the architecture is confounded with implementation: the same SKGE-Swin (stage 1 to 4) in mmsegmentation scores 23.27 (Table V), below the x13 baseline, while the Official PyTorch version scores 37.10; Section V.C.2 itself attributes such discrepancies to framework-level differences (10-20%) and says performance is 'highly sensitive to the implementation framework.' Since x13's implementation/library is unspecified, the 7.39-point 'improvement' could be a framework artifact rather than an effect of skip-stage Swin. The central claim therefore rests on a single, post-hoc selected, framework-dependent measurement; as reported, it is not stable evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SKGE-Swin, a modification of the Swin Transformer that adds skip-stage connections to preserve high-resolution spatial features across stages, and applies it to BEV feature extraction and semantic segmentation in an end-to-end autonomous driving model built on the earlier x13 architecture. The model is trained with multi-task losses on CARLA data and evaluated on the Town05 long route in ClearNoon. The central quantitative claim is a Driving Score of 37.10 for SKGE-Swin-tiny (stage 1→4, float32, Official PyTorch) versus 29.71 for the x13 baseline, with additional ablation results over skip-stage configurations and framework implementations.","tokens_in":17292,"tokens_out":6989,"duration_ms":65991,"significance":"If the reported Driving Score improvement were statistically robust and attributable to the skip-stage design, the paper would offer a simple, potentially useful architectural modification for transformer-based end-to-end driving. The paper also provides a fairly complete CARLA evaluation pipeline and an ablation over skip-stage configurations. However, the evaluation protocol is not sufficiently controlled to support the central claim: the result rests on a single run, the winning configuration is selected on the same test evaluation, and the architecture is confounded with implementation framework. As presented, the evidence does not establish that SKGE-Swin outperforms the baseline.","major_comments":[{"comment":"The evaluation is a single run on the Town05 long route in ClearNoon. No seeds, repetitions, or variance estimates are reported. CARLA's Driving Score is affected by stochastic NPC behavior and route-specific events, so a 7.39-point gap (37.10 vs 29.71) cannot be distinguished from run-to-run noise. The stated superiority claim requires multiple runs with mean/std (or a statistical test) across seeds and preferably additional routes/weather.","section":"Section IV.C / Tables II & V"},{"comment":"The winning skip-stage configuration (stage 1→4) was selected by comparing many variants on the same Town05 test evaluation used to report the headline result. Thus 37.10 is a maximum over configurations, not a pre-registered or held-out estimate. This selection-on-test bias inflates the apparent superiority. The authors should fix the configuration on a validation set and report the test result for that fixed configuration only.","section":"Section V.C / Fig. 3 / Tables IV-V"},{"comment":"The same SKGE-Swin-tiny (stage 1→4) obtains DS 23.27 with the mmsegmentation implementation and DS 37.10 with Official PyTorch. The paper itself attributes such differences to implementation frameworks and states performance is 'highly sensitive to the implementation framework.' Since the x13 baseline's framework is not specified, the claimed gain over x13 may be a framework artifact rather than an effect of skip-stage Swin. A controlled comparison requires matching frameworks/codebases across all models.","section":"Section V.C.2 / Table V"},{"comment":"The paper inconsistently identifies the best model: Table II and V identify SKGE-Swin-tiny (stage 1→4) float32/Official PyTorch as the 37.10 result, but Section V.A text highlights 'SKGE-Swin-base (stage 1→4) + LiDAR' with DS 31.50, and the conclusion refers to 'SKGE-Swin (stage 1→4)' without size/precision/framework. This ambiguity makes the central result difficult to reproduce or interpret.","section":"Section V.A / Tables II, IV, V"},{"comment":"The text claims 'SKGE-Swin (stage 1→4) shows the best performance' for BEV feature extraction test loss, but Table VII lists SKGE-Swin (stage 1→4) at 0.4690, worse than SKGE-Swin (stage 3) at 0.4668 and SKGE-Swin (stage 2→4) at 0.4683, and much worse than OP SKGE-Swin-tiny (stage 4) at 0.4287. The ablation rationale for selecting stage 1→4 is therefore not supported by the reported numbers.","section":"Section V.C.2 / Table VII"}],"minor_comments":[{"comment":"The abstract says 'an ablation study will be conducted,' but the ablation study is already presented in Section V.C. Use present/past tense.","section":"Abstract"},{"comment":"'This stage aimed to evaluate the results of each model test conducted previously' appears to describe training, not evaluation. Reword for clarity.","section":"Section IV.B"},{"comment":"There are two rows labeled 'SKGE-Swin-base (stage 3)*' with different VRAM/FPS values. One is likely a typo; please correct the label or data.","section":"Table III"},{"comment":"Reference [34] is about PLS regression models, not deep learning framework optimization; it does not support the claim that framework-level differences cause 10-20% performance deviations.","section":"References"},{"comment":"The text mentions 'SKGE-Swin-base (stage 1→4) + LiDAR' with DS 31.50, but this row is not in Table II; it appears only in Table V. Add an explicit cross-reference.","section":"Section V.A"},{"comment":"The introduction refers to 'Chapter 2' and 'Chapter 3'; in a journal/arXiv paper these should be 'Section.'","section":"Section I"}],"recommendation":"reject","confidential_remarks":"The paper compares only against x13, the authors' own prior work, and does not include established CARLA baselines such as TransFuser or LAV. The architectural novelty is incremental (skip-stage connections in a Swin backbone), and the evaluation issues described in the main report—single-run results, test-set configuration selection, and framework confounding—leave the contribution unvalidated. Even though the flaws could in principle be addressed with a substantially revised experimental protocol, the current manuscript does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core claim is a roughly 7-point Driving Score gain (37.10 vs 29.71) from adding skip-stage connections to a Swin Transformer inside the authors' own SDC/BEV end-to-end driving pipeline. The idea is plausible and the paper is coherent; applying stage-to-stage skip connections to a Swin backbone in this specific setup is new, and the ablation across skip configurations is a reasonable exploratory step. The qualitative behavior analysis is also a nice addition.\n\nThe problem is the evaluation. The headline number comes from a single run on a single route (Town05 long, ClearNoon), with no seeds, repetitions, or error bars. That alone makes the 7-point difference uninterpretable. Worse, the winning configuration (stage 1→4, Official PyTorch) was selected after comparing variants on that same test evaluation, so the reported score is a max over a small set, not a principled estimate. There is also a serious implementation confound: the same architecture in mmsegmentation scores 23.27, below the x13 baseline, while the Official PyTorch version scores 37.10. The paper itself says framework differences can cause 10-20% swings, and the baseline's framework isn't specified. So the claimed improvement could easily be a framework artifact.\n\nThe internal inconsistencies (abstract in future tense about an ablation that is present, duplicate rows in Table III, contradictory statements about which model performs best, the mis-citation of [12] as the Swin source) lower confidence in the manuscript's care, but they are secondary to the evaluation problem.\n\nThis is a work-in-progress report for people in end-to-end driving who are curious about skip connections in Swin-style backbones. It might motivate a better-controlled study, but as written it doesn't demonstrate its central claim.\n\nI would send it to a referee with the expectation of major revision: the evaluation must include multiple routes, multiple seeds, variance reporting, a specified baseline framework, and a pre-registered configuration selection. If those can't be provided, reject.","headline":"Plausible incremental idea, but the headline Driving Score gain is not established by the reported single-route, single-run, post-hoc-selected evaluation.","tokens_in":17976,"tokens_out":2130,"would_cite":false,"duration_ms":21463,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Skip-stage Swin Transformer hits 37.10 driving score in CARLA","keywords":["autonomous driving","end-to-end learning","Swin Transformer","skip connections","waypoint prediction","bird's-eye view","CARLA","multitask learning"],"falsifier":"Re-run the best configuration and the x13 baseline on several CARLA routes, including Town05 long and at least one unseen town, with at least five independent seeds per model, and compare mean Driving Score and its spread; if the 37.10-versus-29.71 gap does not persist, or another skip-stage configuration wins on the held-out routes, the paper's central claim is refuted.","tokens_in":16855,"feed_emoji":"🚗","tokens_out":9164,"duration_ms":91899,"temperature":0.7,"pith_summary":"The paper's aim is to show that an end-to-end driving model navigates better when its feature extractor is a Swin Transformer with skip-stage connections rather than a CNN. The proposed SKGE-Swin keeps high-resolution spatial details from early layers and combines them, via bilinear interpolation and addition, with the globally attended features of a deep stage. On the CARLA Town05 long route the model reaches a Driving Score of 37.10, beating the 29.71 of the CNN-based x13 baseline and approaching the expert score of 42.63. The contribution matters because end-to-end driving depends on understanding both fine local structure and distant context, which the paper argues CNNs handle poorly at intersections and with oncoming traffic.","feed_headline":"Skip-stage Swin Transformer hits 37.10 driving score in CARLA","feed_subtitle":"Adding early-to-deep skip connections beats the CNN baseline's 29.71 and approaches the expert's 42.63.","key_machinery":"The load-bearing mechanism is the SKGE block, a Swin Transformer hierarchy in which feature maps from an early stage are resized with bilinear interpolation and added element-wise to a deeper stage. This lets the deep stage combine its own global, semantic features from shifted-window self-attention with the high-resolution spatial details that patch merging would otherwise discard. The paper systematically varies which stages are connected and finds that feeding stage 1 into stage 4, inside the BEV feature extractor, works best.","core_discovery":"On the paper's own terms, the discovery is that a Swin Transformer backbone equipped with a skip-stage connection extracts better bird's-eye-view features for end-to-end driving than the CNN backbone it replaces. The winning configuration—SKGE-Swin-tiny (the small Swin variant) with features from stage 1 bilinearly upsampled and added to stage 4—reaches a Driving Score of 37.10 on the CARLA Town05 long route, against 29.71 for the x13 CNN baseline and 42.63 for the expert. The same configuration also behaves more safely at intersections in qualitative tests, for example checking left before turning right. This is presented as evidence that global attention combined with preserved high-resolu","pith_inferences":["Because the paper shows the same named architecture scoring 37.10 in one implementation and 23.27 in another, the reported advantage is tied to the full training and inference pipeline, not just to skip-stage wiring; a clean test would vary the skip connection while holding the pipeline fixed.","The single-route evaluation offers a natural next experiment: repeated and multi-route runs would establish whether the stage-1-to-4 advantage is stable or an artifact of choosing the best configuration from the test route.","The same skip-stage idea could transfer to other hierarchical vision encoders or to BEV tasks like motion forecasting and object detection; the paper does not test these, but the mechanism—preserving high-resolution detail into deep global stages—is task-general.","The intersection behavior suggests a measurable design target: an intersection look-ahead or anticipation metric could make the qualitative advantage quantitative for future models."],"forward_implications":["If the central claim holds, a Swin Transformer with a stage-1-to-stage-4 skip is a viable replacement for a CNN encoder in end-to-end waypoint prediction, at the cost of roughly 2.5 times more GPU memory (1048–1382 MiB vs 556 MiB).","The skip direction matters: forwarding early features into deeper stages helps, while the reverse stage-4-to-1 skip hurts performance, supporting the paper's claim that semantic mismatch is the reason.","The transformer's benefit is concentrated in BEV feature extraction; as a semantic segmentation backbone it did not beat the CNN baseline, so the architecture should be placed on the BEV encoder rather than the segmentation path.","Mixed-precision float16 speeds up the best model (22.82 to 27.49 FPS) with identical memory use, but also lowered Driving Score in the reported run (31.60 vs 37.10), so precision and driving quality can diverge.","Qualitatively, the model shows anticipatory behavior in situations where the CNN baseline collides, such as looking left before a right turn and braking for vehicles appearing at the image edge."],"supporting_citations":[{"why":"Supplies the base end-to-end SDC/BEV architecture whose encoders the paper re-engineers with the SKGE-Swin block.","marker":"[6]"},{"why":"Provides the Swin Transformer with shifted-window self-attention and hierarchical stages that the skip connections modify.","marker":"[10]"},{"why":"Contributes the skip-connection concept used to preserve early-stage spatial information in deeper layers.","marker":"[11]"},{"why":"Supplies the dataset and the imitation baseline context used for training and comparison.","marker":"[19]"},{"why":"Defines the route configuration (long/short/mini) used to collect data and evaluate driving.","marker":"[24]"},{"why":"Grounds the claim that reverse skip stages cause feature mismatch, justifying the early-to-deep skip direction.","marker":"[32]"},{"why":"Supports the suitability of transformers for bird's-eye-view representation, motivating the Swin BEV encoder.","marker":"[33]"}],"fun_headline_variants":["Skip-stage Swin Transformer outscores CNN in CARLA driving: 37.10","Swin Transformer with skip-stage boosts self-driving CARLA score","Skip-stage Swin beats CNN baseline on autonomous vehicle waypoints","Global attention plus skip connections lift CARLA driving score","Swin Transformer outperforms CNN for end-to-end driving in CARLA"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The headline gap rests on a single evaluation run over one route (Town05 long, ClearNoon), with the winning skip-stage configuration chosen from that same evaluation; repeated runs, other routes, or held-out configuration selection could change the comparison.","fun_headline_variants_meta":{"raw":{"variants":["Skip-stage Swin Transformer outscores CNN in CARLA driving: 37.10","Swin Transformer with skip-stage boosts self-driving CARLA score","Skip-stage Swin beats CNN baseline on autonomous vehicle waypoints","Global attention plus skip connections lift CARLA driving score","Swin Transformer outperforms CNN for end-to-end driving in CARLA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000283,"raw_usage":{"total_tokens":1493,"prompt_tokens":717,"completion_tokens":776,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":695}},"tokens_in":461,"tokens_out":776,"duration_ms":8183,"temperature":1.0,"reasoning_tokens":695,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:51:12.349510+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the best configuration and the x13 baseline on several CARLA routes, including Town05 long and at least one unseen town, with at least five independent seeds per model, and compare mean Driving Score and its spread; if the 37.10-versus-29.71 gap does not persist, or another skip-stage configuration wins on the held-out routes, the paper's central claim is refuted.","supporting_citations":[{"cited_title":"End-to-end autonomous driving with semantic depth cloud mapping and multi-agent,","cited_arxiv_id":null,"evidence_quote":"Supplies the base end-to-end SDC/BEV architecture whose encoders the paper re-engineers with the SKGE-Swin block."},{"cited_title":"Swin transformer: Hierarchical vision transformer using shifted windows,","cited_arxiv_id":null,"evidence_quote":"Provides the Swin Transformer with shifted-window self-attention and hierarchical stages that the skip connections modify."},{"cited_title":"Deep residual learning for image recognition,","cited_arxiv_id":null,"evidence_quote":"Contributes the skip-connection concept used to preserve early-stage spatial information in deeper layers."},{"cited_title":"Trans- Fuser: Imitation with Transformer-Based sensor fusion for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset and the imitation baseline context used for training and comparison."},{"cited_title":"Multi-modal fusion transformer for end-to-end autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Defines the route configuration (long/short/mini) used to collect data and evaluate driving."}],"review_version":1}