{"id":"b2a74c63-dff4-4098-acf2-1945d89761ac","arxiv_id":"2504.19722","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new public traffic light dataset and a map-based detection, association, and smoothing framework improve traffic light perception in autonomous driving, with 99.33% accuracy within 120 meters in a small real-world evaluation.","lead":"ATLAS, a new public traffic light dataset with 33,044 images, three camera fields of view, 25 state-pictogram classes, and rain footage, is used to train YOLO detectors. The authors also describe a map-based association and smoothing framework that made reliable traffic-light decisions in 230 minutes of autonomous driving in Karlsruhe.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reliability claim rests on 230 minutes of no-override driving and a single 10-minute annotated drive, but the paper never reports how many intersections, state transitions, or adverse conditions were encountered, so the headline numbers lack a statistical denominator.","rationale":"The paper is a competent systems contribution: the ATLAS dataset fills real gaps in traffic-light perception data, the detector evaluation is sensible, and the association and decision-module design are plausible and well motivated. The reader's conditional verdict is appropriate. I do not see an internal inconsistency or a fatal flaw in the architecture; the map dependence flagged by the reader is a genuine boundary condition but it is stated in the framework description and is not, by itself, a violation of the central claim, since the framework is explicitly a camera-plus-HD-map system. My more pressing concern is that the reliability headline is statistically underdetermined. The two strongest quantitative pillars, 230 minutes without override and 99.33% accuracy with no erroneous state changes within 120 meters, are reported without the denominators needed to evaluate them: number of intersections, number of state transitions, weather and illumination conditions, and number of relevant signal groups. Moreover, the decision-module parameters appear to have been tuned on the same evaluation data used to report the accuracy, so the in-sample nature of the result is unclear. This does not change the verdict from the reader's CONDITIONAL, but it sharpens the condition: the authors should report the event counts and provide a held-out deployment evaluation before the reliability claim is taken as established. No ad hominem is intended; the issue is the strength of the evidence relative to the strength of the claim.","tokens_in":11055,"tokens_out":7271,"duration_ms":82829,"concrete_test":"Compute from the recorded logs the number of traffic-light-controlled intersections, the number of signal-group state transitions, and the counts of yellow and red-yellow events in both the 230-minute run and the 10-minute annotated drive; report per-transition error counts with a 95% confidence interval. Then re-run the full framework on a held-out second 10-minute drive without re-tuning the circular-buffer parameters, and report the same statistics. If the transition counts are low (for example, fewer than 20 state changes) or the held-out error rate differs materially from the reported 99.33%, the reliability claim is not supported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section V is that the framework is 'reliable, stable, and capable of detecting traffic lights from greater distances.' The quantitative support is (i) no manual override in 230 minutes of autonomous driving, and (ii) no erroneous state changes within 120 meters plus 99.33% accuracy in one 10-minute annotated drive. For these observations to license the claim, we need to know how many opportunities to err the system actually faced. The paper does not report the number of traffic-light-controlled intersections, the number of signal-group state transitions, the number of yellow or red-yellow events, the number of unique mapped traffic lights exercised, or the weather and time-of-day conditions during the deployment runs. A 230-minute no-override result is much less informative if the route contained only a handful of relevant signal changes, and the 99.33% frame-level accuracy figure is likewise sensitive to state-change density: the 0.67% discrepancy is attributed entirely to the 103ms decision-module latency, but without transition counts it is impossible to judge whether this is a stable property or an artifact of a short recording. Additionally, the decision-module parameters (buffer size 9, three-second weight decay, pictogram mismatch downweighting of 0.5) are described as optimal 'in our experiments' without a held-out split or sensitivity analysis, and the same 10-minute evaluation drive appears to be the tuning and reporting set. The reliance on HD-map priors is a real boundary condition, but even when the map is available, the reliability claim is not yet statistically anchored.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ATLAS, a publicly released traffic-light dataset with 25 pictogram-state classes, multi-camera fields of view, and rain annotations, totaling 33,044 images and 72,998 bounding boxes. The authors train several YOLO variants on ATLAS combined with DTLD and report improved mAP on the ATLAS test split. They then present a modular perception framework consisting of a camera-based detector, a ray-projection and Hungarian-algorithm association module that links detections to HD-map traffic-light positions, and a circular-buffer decision module that aggregates weighted detector outputs over time to reach a stable signal-group state. The framework is deployed in an autonomous vehicle and evaluated during 230 minutes of autonomous driving with no safety-driver override, plus a 10-minute hand-annotated autonomous drive reporting 99.33% accuracy within 120 meters, no erroneous state changes within that range, an average end-to-end reaction time of 184 ms, and an average first-detection distance of 169.5 meters.","tokens_in":11327,"tokens_out":5848,"duration_ms":62907,"significance":"If the system-level results hold, the paper makes a useful contribution: ATLAS fills concrete gaps in public traffic-light data (rare pictogram-state combinations, rain, multiple FOVs), and the deployment study provides a rare full-stack evaluation with a breakdown of latency sources. The authors also compare fairly against prior map-based association work and openly report the 230-minute no-override record and the 10-minute annotated evaluation. However, the headline reliability claims rest on a very small statistical base: one 10-minute segment from one vehicle in one city, no reported counts of intersections or state transitions, point estimates without variance, and decision-module hyperparameters that may have been tuned on the same evaluation segment. These issues are load-bearing for the central claim that the framework is reliable, stable, and capable of long-range detection.","major_comments":[{"comment":"The fine-grained reliability evidence is a single 10-minute annotated segment from one vehicle in Karlsruhe, and the 230-minute no-override record is reported without the number of traffic-light-controlled intersections, signal-group state transitions, unique mapped traffic lights, or weather and time-of-day conditions encountered. As a result, the headline numbers \"no erroneous state changes were detected within a range of 120 meters\" and \"99.33% accuracy\" have no statistical denominator and cannot, on their own, support the general reliability claim stated in Section V. Please report the counts of transitions, intersections, and frames used to compute these figures, and provide per-intersection results or a confidence interval for the accuracy and latency estimates.","section":"§V, Fig. 6b"},{"comment":"The decision-module hyperparameters (circular buffer size 9, linear weight decay to zero over three seconds, and halving the weight for mismatched pictograms) are described as optimal \"in our experiments\" without a held-out split or sensitivity analysis. If the same 10-minute annotated drive used in Section V was also used to select these parameters, the reported 103 ms state-change latency and 99.33% accuracy are optimistically biased. Please specify the tuning protocol, state explicitly whether the evaluation segment was excluded from tuning, and report how the system-level metrics vary with buffer size, decay time, and pictogram mismatch factor.","section":"§IV.B.4"},{"comment":"The metric definitions need to be reconciled: the paper reports \"no erroneous state changes were detected within a range of 120 meters\" yet also reports a 0.67% accuracy loss within that range, which is attributed entirely to latency. Clarify whether delayed state transitions are counted as erroneous state changes, define the exact frame-level counting rule used for the 99.33% figure, and report the number of state-change events from which the average 103 ms latency is computed. Without this, it is not possible to judge whether the 0.67% discrepancy is a stable property of the system or an artifact of a short recording with few transitions.","section":"§V, criteria A and C"},{"comment":"The association and decision modules require an HD map with accurate 3D traffic-light positions, pictogram labels, and signal groups for every operated route, and the evaluation is limited to such mapped routes. The concluding claims in Section V, and the abstract's statement of a \"reliable perception framework,\" should be explicitly scoped to map-based operation. If map creation effort, map errors, or localization failures were measured, that information should be reported, since it bears directly on how broadly the reliability result can be expected to generalize.","section":"§IV.B.1 and §V"}],"minor_comments":[{"comment":"Equation (1) as typeset contains a malformed normalization term, with a stray \"r\" before the square-root expression; the formula should be corrected so that the ray equation can be verified.","section":"§IV.B.2, Eq. (1)"},{"comment":"Table I would benefit from a legend or footnote explaining the checkmark encoding; the single long row of checkmarks makes it difficult to see which classes are missing for each dataset, especially for the less common pictograms.","section":"Table I"},{"comment":"Table III reports single-run mAP values without variance or the number of random seeds; a short statement on training reproducibility would help assess whether the reported differences among YOLO variants are meaningful.","section":"Table III"},{"comment":"Figure 6b is difficult to read at the printed size; a zoomed excerpt around the two reported exceptions near 30 and 220 seconds would make it easier for a reader to verify the \"no erroneous state changes within 120 m\" claim.","section":"Fig. 6b"},{"comment":"The annotation protocol labels pictograms of far-away traffic lights using later recorded images or Google Street View; this should be stated explicitly in the dataset documentation, since it means the long-distance pictogram labels are not always derived from the pixels in the annotated frame.","section":"§III.B"},{"comment":"The statement that the 381 ms worst-case latency is \"still faster than human drivers\" compares a perception-only latency with a full human perception-brake reaction time; the comparison should be qualified to avoid overstating the system-level margin.","section":"§V, latency comparison"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the central weakness is the thin statistical basis of the reliability claim, which is fixable by additional analysis (transition and intersection counts, held-out tuning or sensitivity analysis, and clearer metric definitions) rather than by new technical machinery. The dataset release and full-stack deployment are genuine strengths, and I see no citation or novelty concerns. The paper is a reasonable fit for IV, but the authors should be asked to either strengthen the evaluation or substantially scope down the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe useful half of this paper is the ATLAS dataset. It is a genuine new resource: 33k images, 25 state-pictogram classes including the straight-right pictogram missing from every public dataset, synchronized tele/medium/wide camera streams, and a rain subset. That fills concrete gaps. The detector table is honest: adding ATLAS to DTLD training lifts ATLAS test mAP50 from roughly 0.53 to 0.72 with no loss on DTLD. The framework itself is not new — Hungarian matching and a circular buffer are standard — but the integration is sensible, and the association cost cap plus pictogram mismatch downweighting are reasonable engineering choices. I would trust the dataset and reuse it.\n\nThe soft spots are all on the evaluation side. The headline numbers — 230 minutes with no override, 99.33% accuracy, 184 ms latency — come from one vehicle in one city, with no variance, no baseline comparison for association or smoothing, and no count of how many intersections or signal-group transitions the system actually faced. In a 10-minute annotated drive, a 99.33% figure can look great or mediocre depending on how many red/yellow events occurred. The paper does not say. That is not a fatal flaw, but it is a load-bearing omission: the concluding sentence says the framework is 'reliable,' and the evidence does not license that word yet. The decision-module parameters (buffer 9, 3 s decay, 0.5 mismatch weight) are reported as optimal without a held-out split or sensitivity analysis, which makes the accuracy number look tuned in the same run it is reporting. The reliance on a hand-annotated HD map with signal groups is a real boundary condition; the authors acknowledge it but do not quantify how much of the result depends on map quality. The citation pattern is fine.\n\nWho should read this: anyone working on traffic light perception or robot perception evaluation. The dataset is the thing; the paper is worth a serious referee because the dataset deserves public scrutiny and the framework has enough substance to be improved. But a referee should insist on reporting the denominator — number of intersections, transitions, weather conditions — and on a held-out parameter split or sensitivity bounds.\n\nI would not desk-reject it. Send it out.\n\nBest","headline":"The ATLAS dataset is a real contribution; the reliability claim outruns the evidence.","tokens_in":11918,"tokens_out":2168,"would_cite":true,"duration_ms":22172,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims its modular camera-and-map traffic light perception framework recognizes and decides signal states with no erroneous state changes within 120 meters and an average 184 ms reaction time.","keywords":["traffic light perception","autonomous driving","ATLAS dataset","object detection","HD map association","Hungarian algorithm","circular buffer","real-time decision module"],"falsifier":"A concrete check would be to repeat the 10-minute annotated evaluation on a route where the HD-map traffic light positions are intentionally shifted by 2 to 5 meters relative to the camera localization. If the association module starts pairing detections with the wrong mapped lights or produces erroneous state changes within 120 meters, the claimed reliability depends on map accuracy rather than on the matching and buffering design. A second check is to hand-annotate all 230 minutes of the autonomous drive; any erroneous state change inside 120 meters would contradict the stated accuracy bound directly.","tokens_in":10840,"feed_emoji":"🚦","tokens_out":5693,"duration_ms":56853,"temperature":0.7,"pith_summary":"This paper aims to establish that camera-based traffic light perception can be made dependable enough to drive through urban intersections autonomously. The authors propose a complete framework: a detector finds traffic lights, an association stage matches detections to 3D positions stored in an HD map, and a decision module buffers recent detections to produce stable signal states. They also release ATLAS, a new annotated dataset covering pictograms, weather conditions, and multi-camera views, and show that training on it improves detection accuracy. The payoff is that an autonomous vehicle gets a concrete, measurable basis for stopping or proceeding: no erroneous state changes within 120 meters and 184 ms average latency.","feed_headline":"Perception framework keeps traffic-light states correct inside 120 meters","feed_subtitle":"Camera-plus-HD-map system reacts in 184 ms and smooths detections through circular buffers.","key_machinery":"The load-bearing mechanism is the association-plus-decision chain. Each detected bounding box is turned into a 3D ray using camera intrinsics and extrinsics, and the distance from each ray to each mapped traffic light reference point is computed over a 180-meter region; the assignment of detections to mapped lights is then solved as a weighted complete bipartite graph with the Hungarian algorithm, with edge costs capped at 10 meters and success requiring a matched edge under 2 meters. After association, every traffic light keeps a circular buffer of recent detections, weighted by detector confidence, linearly decayed over three seconds, and halved when the predicted pictogram (the direction arrows on the light face, such as straight, left, or right) does not match the map's pictogram; the final state is the highest cumulative weight in the buffer, and for a signal group—a set of lights that jointly control one movement and must show the same state—the highest-confidence member decides. This buffered voting is what prevents flicker and suppresses single-frame misdetections.","core_discovery":"The paper's central claim is that a perception stack built from three stages—a YOLO-family detector, a ray-to-map association solved as a minimum-cost bipartite matching, and a per-traffic-light circular-buffer decision module—is reliable enough for an autonomous vehicle to act on traffic light states in real urban traffic. The authors report that during 230 minutes of fully autonomous driving in Karlsruhe the safety driver never had to override the decision module, and that in a hand-annotated 10-minute segment no erroneous state change occurred within 120 meters of an intersection, with a calculated accuracy of 99.33% inside that range. They further report an average end-to-end signal-change reaction time of 184 ms and a first-detection association distance averaging 169.5 meters, and they attribute the stability to the buffered decision logic rather than to any single detector.","pith_inferences":["A natural test of the framework's generality is to run it in a city with a differently annotated HD map, or with map positions deliberately perturbed; the association threshold of 2 meters and the 10-meter cost cap would have to absorb those errors, and the reported reliability would only transfer if they do.","The same association method could be applied to other 2D-to-3D landmark tasks, such as attaching detected signs or lane markings to a map under localization drift.","The circular-buffer weighting scheme is detector-agnostic and could be paired with any future traffic-light detector, so the reported stability may outlive the specific YOLO models used here."],"forward_implications":["If the framework is as reliable as reported, autonomous vehicles can plan smooth braking rather than emergency stops, since red lights are recognized early enough at 120 meters and beyond (average first association at 169.5 meters).","The end-to-end 184 ms reaction time to a signal change is comfortably inside human perception-brake times, so the decision module itself is not the bottleneck for intersection safety.","Combining ATLAS with an existing dataset raises detector mAP50 on ATLAS from roughly 0.53 to 0.70–0.72 across several YOLO variants, which suggests dataset coverage, not architecture choice, is the limiting factor.","Because the decision module reasons over signal groups, a single momentarily misclassified or missed light can be outvoted by another light in the same group, making the system robust to occlusions that hide one signal."],"supporting_citations":[{"why":"Supplies the modified YOLOv8 detector that the framework reuses and extends, along with the prior analysis of open-source dataset gaps that motivates ATLAS.","marker":"[9]"},{"why":"Provides the DTLD training and test data combined with ATLAS, and the comparison baseline for detection performance.","marker":"[16]"},{"why":"Establishes the map-based projection approach for traffic light mapping and detection that the projection stage builds on.","marker":"[19]"},{"why":"The prior map-plus-detector relevance approach that the paper compares against and improves on with global min-cost association and signal-group reasoning.","marker":"[6]"},{"why":"Supplies the Hungarian algorithm used to solve the bipartite association exactly.","marker":"[28]"},{"why":"Provides the localization and SLAM-based world coordinates used to place mapped traffic lights in the HD map.","marker":"[27]"},{"why":"The autonomous driving stack that consumes signal group states for planning and braking decisions.","marker":"[29]"}],"fun_headline_variants":["99.33% traffic light state accuracy within 120 meters","184 ms reaction to traffic light changes in autonomous driving","No overrides in 230 min: traffic light framework proves reliable","ATLAS dataset and buffered logic improve traffic light perception"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole reliability argument assumes an HD map with accurate 3D traffic light positions, pictogram labels, and signal groups is available for every route the vehicle drives; without those priors, the association and decision modules cannot identify which detected light is relevant.","fun_headline_variants_meta":{"raw":{"variants":["99.33% traffic light state accuracy within 120 meters","184 ms reaction to traffic light changes in autonomous driving","No overrides in 230 min: traffic light framework proves reliable","ATLAS dataset and buffered logic improve traffic light perception"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000333,"raw_usage":{"total_tokens":1817,"prompt_tokens":879,"completion_tokens":938,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":869}},"tokens_in":495,"tokens_out":938,"duration_ms":8818,"temperature":1.0,"reasoning_tokens":869,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:45:04.585596+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check would be to repeat the 10-minute annotated evaluation on a route where the HD-map traffic light positions are intentionally shifted by 2 to 5 meters relative to the camera localization. If the association module starts pairing detections with the wrong mapped lights or produces erroneous state changes within 120 meters, the claimed reliability depends on map accuracy rather than on the matching and buffering design. A second check is to hand-annotate all 230 minutes of the autonomous drive; any erroneous state change inside 120 meters would contradict the stated accuracy bound directly.","supporting_citations":[{"cited_title":"TLD-READY: Traffic Light Detection - Relevance Estimation and Deployment Analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the modified YOLOv8 detector that the framework reuses and extends, along with the prior analysis of open-source dataset gaps that motivates ATLAS."},{"cited_title":"The DriveU traf- fic light dataset: Introduction and comparison with existing datasets,","cited_arxiv_id":null,"evidence_quote":"Provides the DTLD training and test data combined with ATLAS, and the comparison baseline for detection performance."},{"cited_title":"Traffic light mapping and detection,","cited_arxiv_id":null,"evidence_quote":"Establishes the map-based projection approach for traffic light mapping and detection that the projection stage builds on."},{"cited_title":"Traffic Light Recog- nition Using Deep Learning and Prior Maps for Autonomous Cars,","cited_arxiv_id":null,"evidence_quote":"The prior map-plus-detector relevance approach that the paper compares against and improves on with global min-cost association and signal-group reasoning."},{"cited_title":"The Hungarian method for the assignment problem,","cited_arxiv_id":null,"evidence_quote":"Supplies the Hungarian algorithm used to solve the bipartite association exactly."}],"review_version":1}