{"id":"737eff77-ad78-46d2-8867-2f30b0f49895","arxiv_id":"2508.06529","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A single transformer-based network jointly performs object detection, drivable area segmentation, and lane line segmentation on BDD100K with reported state-of-the-art accuracy at real-time speed, along with a dilated label correction for fairer lane line evaluation.","lead":"This paper presents a real-time neural network, RMT-PPAD, that simultaneously detects vehicles, segments drivable areas, and marks lane lines on the BDD100K driving dataset. It reports state-of-the-art scores on all three tasks at 32.6 frames per second, plus a fix for an inconsistency in how lane line labels are evaluated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The lane-line SOTA claim is conditional on the new 8-pixel evaluation protocol; if that protocol is not exactly matched by every baseline's training labels, the headline IoU/ACC gains are an artifact of the metric change.","rationale":"The reviewer's conditional verdict is already keyed to this issue, and my read agrees. The paper's other contributions — the GCA module, the adaptive segmentation decoder, and the real-time performance — are supported by internal ablations and by re-running open-source baselines; I found no internal inconsistency there that would overturn the conditional acceptance. The lane-label protocol is the one place where the headline number depends on a nonstandard evaluation change whose exact calibration to prior training labels is unverified. If the premise is true, the comparison is fair and the contribution stands; if false, the lane-line SOTA collapses. This does not change the verdict: it remains conditional on release of code/checkpoints and on verifying the protocol. I therefore keep UNCHANGED rather than moving to accept or reject.","tokens_in":17036,"tokens_out":7360,"duration_ms":91470,"concrete_test":"Check each released checkpoint (RMT-PPAD, YOLOP, YOLOPX, HybridNet, A-YOLOM(n/s)) under the original 2-pixel BDD100K labels and under the authors' dilated labels; also inspect each repo's lane-mask generation code to determine the training label width. If RMT-PPAD is not the best under the original protocol, or if any baseline's training masks are not 8-pixel, the SOTA claim must be relabeled as 'best on the new protocol' rather than unconditional.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RMT-PPAD's headline lane-line numbers (IoU 56.8%, ACC 84.7%, Table III) are obtained after dilating BDD100K test labels from the original 2-pixel masks to 8-pixel masks, as proposed in Section III-D. The paper states that prior methods train with 8-pixel masks and evaluate with 2-pixel masks, so the dilation is a fair correction. This premise is load-bearing. If any baseline's training masks were generated differently (different width or different skeleton-to-mask code), the dilated test labels will not match what that model was trained to predict, and the comparison is biased. The manuscript admits the dilation is not exact: original labels are not strictly 2 pixels, and the 7x7 elliptical structuring element yields 8- or 9-pixel widths with 'systematic error' on diagonal lanes (Section III-D, Table I). It does not quantify how well the dilated masks match the actual training masks of YOLOP, YOLOPX, HybridNet, or A-YOLOM, nor does it report per-method IoU under both the old and new protocols. Because the protocol change substantially inflates IoU (Fig. 6: YOLOPX IoU goes from 0.2339 to 0.5851 on one sample), the 4.7-point gap over YOLOPX in Table III could be partly a metric artifact. The 'state-of-the-art' claim is therefore only valid under the corrected protocol, and only if the training-mask premise holds for every baseline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript introduces RMT-PPAD, a multi-task transformer for panoptic driving perception that jointly handles vehicle detection, drivable-area segmentation, and lane-line segmentation. It proposes a Gate Control with Adapter (GCA) module to fuse shared and task-specific features, an adaptive segmentation decoder that learns per-task multi-scale weighting, and a lane-line evaluation correction in which test masks are dilated from roughly 2 px to 8 px to match the training masks used by prior work. On BDD100K, the authors report mAP50 84.9%, Recall 95.4%, drivable-area mIoU 92.6%, lane-line IoU 56.8% and ACC 84.7% at 32.6 FPS on a single RTX 4090, claiming SOTA over open-source panoptic-driving MTL baselines. Ablations on a 10K-image toy split and gradient-similarity histograms support the negative-transfer claim, and qualitative real-world results are provided.","tokens_in":17288,"tokens_out":6940,"duration_ms":72283,"significance":"The paper has real strengths: the authors release code and weights, re-run all comparison methods under the same dilated-label evaluation protocol (the right way to handle the metric change), and provide a quantitative gradient-similarity analysis rather than only qualitative claims. If the lane-label premise is correct and the toy-set ablations transfer, the GCA module and the adaptive decoder are useful architectural contributions with immediate practical value for real-time autonomous-driving perception. However, the headline lane-line SOTA is tightly coupled to an unverified assumption about every baseline's training-label width, and the main negative-transfer evidence is obtained on a reduced dataset; as presented, the significance is moderate and the strongest claims need additional support.","major_comments":[{"comment":"The headline lane-line results are conditional on the premise that every compared baseline trained with 8-pixel-wide lane masks and was evaluated against 2-pixel masks, so that dilating the test masks to 8 pixels restores a fair comparison. The manuscript does not verify this premise for YOLOP, HybridNet, YOLOPX, A-YOLOM(n), or A-YOLOM(s); it only cites [14] and states the practice in [3]-[5], [11]-[13]. It also does not measure how well the 7x7 elliptical dilation, which the authors concede yields 8- or 9-pixel widths and a systematic error on diagonals in Section III-D, matches the actual training masks of each baseline. Because Fig. 6 shows the protocol change taking YOLOPX IoU from 0.2339 to 0.5851 on one sample, the 4.7-point IoU margin over YOLOPX in Table III could be a metric artifact rather than a model advantage. I ask the authors to (i) state how RMT-PPAD's own training masks are generated, (ii) confirm from released code or documentation the training-label width of each baseline, and (iii) report each method's lane IoU/ACC under both the original 2-pixel protocol and the dilated 8-pixel protocol so the effect of the protocol change is visible per method.","section":"Section III-D, Table III"},{"comment":"The central evidence that GCA alleviates negative transfer is produced on a toy dataset of 10K training and 2K validation images, not on the full BDD100K split. The numbers in Table IV (e.g., RMT-PPAD mAP50 78.3% and mIoU 91.3%) are far from the full-model results in Table III (84.9% and 92.6%), so the conclusion that GCA 'even outperforms the single-task baselines in mAP50 and mIoU' and 'fosters synergy across tasks' is established only in a reduced-data regime. The paper's contribution statements in the abstract and Section I do not carry this caveat. The authors should either repeat the single-task/vanilla-MTL/with-GCA comparison on the full training set (or a substantially larger subset with full epochs), or explicitly restrict the negative-transfer claims to the toy setting and explain why the trend is expected to transfer.","section":"Section IV-A.3, Section IV-C, Table IV"},{"comment":"The segmentation mask thresholds used for the reported BDD100K numbers (0.45 for drivable area, 0.9 for lane lines) are selected on the validation set from Table V, and all results are single-run without error bars or seed variation. This raises two issues: the margins in Table III (for example, 4.7 IoU over HybridNet in lane lines) may be threshold-dependent, and it is not clear whether the baselines were evaluated with their own optimal thresholds or with the same fixed thresholds. The paper should report the sensitivity of each method to the threshold, give multi-seed variance for at least the main comparison, and state the exact evaluation protocol for all baselines.","section":"Section IV-A.3, Table V"}],"minor_comments":[{"comment":"Sentences such as \"We through experiments and ablation studies to demonstrate...\" and \"RMT-PPAD achieve outperforming while maintaining real-time performance\" contain grammatical errors and need revision.","section":"Section I"},{"comment":"The sentence \"YOLOP [5] YOLOP introduces...\" has a duplicated subject and should be corrected.","section":"Section II-C"},{"comment":"The notation \"A-YOLOM(n)\" and \"A-YOLOM(s)\" is used without defining what (n) and (s) mean; the authors should state that these are the nano and small variants, or otherwise explain the suffix.","section":"Section IV-A.3, Table III"},{"comment":"The block labeled \"S Deconv\" is ambiguous; a full name or a clearer caption would help the reader understand the decoder structure.","section":"Figure 5"},{"comment":"The text says the real-road dataset \"includes four scenarios: snow, cloudy, and daytime,\" which lists only three scenarios, while Fig. 10 caption lists \"nighttime, daytime, cloudy, and highway\"; please reconcile the enumeration.","section":"Section IV-D"},{"comment":"\"Table I(c) and I(e) shows\" should be \"show,\" and the phrase \"previous work labels in Table I(b) and I(d)\" would be clearer if the subfigures were referred to by their explicit labels (a)-(e).","section":"Section III-D"}],"recommendation":"major_revision","confidential_remarks":"The comparison set is largely anchored to the author's own A-YOLOM baseline and omits YOLOPv2 (cited as [13]), which is a natural open-source competitor for the panoptic-driving perception task; the authors should justify this omission or add the comparison. The paper would also benefit from an explicit statement of which parts of the evaluation are performed on the validation set (threshold selection, lane-label dilation verification, and toy-set ablations) so that readers can judge the risk of overfitting to the reported protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent engineering paper. The genuinely new thing is not the network—GCA is a reasonable adapter-plus-attention module, and the adaptive segmentation decoder is a nice touch—but the lane-label evaluation analysis. They identify that prior work trains on 8-pixel-wide lane masks and evaluates on 2-pixel-wide masks, which systematically penalizes correct predictions. Their confusion-matrix demonstration is convincing, and dilating test labels to match training is the right instinct. Credit also for re-running all baselines under the corrected protocol rather than just quoting old numbers.\n\nThe main tables hold up as far as I can tell. They report SOTA among the open-source MTL models compared, on detection, drivable area, and lane lines. The ablation on the toy dataset is honest, and the gradient-similarity histograms support the claim that GCA reduces conflicting gradients.\n\nSoft spots, in proportion: The lane-line SOTA is conditional on the claim that every baseline was trained with 8-pixel masks. They assert this and cite the SAD convention, but they don't verify each repo's training label generation. The dilation itself is approximate—7x7 elliptical structuring element, 8- or 9-pixel widths, systematic error on diagonals—so the headline IoU/ACC gains could be partly a metric artifact if any baseline's training mask differs. This is fixable: show per-method IoU under both old and new protocols, or inspect the training code of each baseline. Right now the paper gives one example figure but no per-method protocol comparison.\n\nOther issues are minor: single runs with no error bars, thresholds tuned on the validation set, and real-world evaluation is qualitative with unreleased data. The toy-dataset ablations are explicitly stated, so that's fine.\n\nThe lane-label issue deserves a serious referee. If the premise holds, this is a useful correction that could affect many papers in the area. Even if it doesn't fully hold, the paper's internal comparison is consistent because all baselines are re-evaluated under the same protocol. I'd accept this for peer review and ask the authors to strengthen the verification of training-label widths. I'd cite the evaluation fix in my own work if I were active in lane detection.","headline":"Solid MTL engineering with a genuinely useful evaluation fix; the lane-label SOTA number is conditional on a plausible but unverified premise.","tokens_in":17864,"tokens_out":4536,"would_cite":true,"duration_ms":51873,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RMT-PPAD claims one real-time transformer network beats all open-source multi-task baselines on BDD100K detection, drivable area, and lane lines at 32.6 FPS.","keywords":["multi-task learning","panoptic driving perception","object detection","drivable area segmentation","lane line segmentation","negative transfer","real-time transformer","BDD100K"],"falsifier":"Download the lane-line annotation files and the training and evaluation scripts of YOLOP, HybridNet, YOLOPX, and A-YOLOM, and measure the actual pixel-width distributions of lane masks in training versus test. If the training labels are not predominantly 8 pixels wide or the test labels are not predominantly 2 pixels wide, the paper's unfair-evaluation diagnosis and the resulting lane-IoU rankings in Table III collapse. Alternatively, re-evaluate all models on the original 2-pixel test labels; if RMT-PPAD does not still lead in lane IoU, the label-dilation fix, not the model, is carrying the lane result.","tokens_in":16785,"feed_emoji":"🚗","tokens_out":10823,"duration_ms":109990,"temperature":0.7,"pith_summary":"RMT-PPAD is a real-time, transformer-based multi-task network that performs object detection, drivable-area segmentation, and lane-line segmentation in a single forward pass. The paper claims that on BDD100K it beats every open-source multi-task driving-perception model it compares with on all three tasks: mAP50 of 84.9%, Recall of 95.4%, drivable-area mIoU of 92.6%, and lane-line IoU of 56.8% with accuracy of 84.7%, while running at 32.6 FPS. The central engineering claim is that a lightweight gate-control-with-adapter (GCA) module fuses shared and task-specific features so that the tasks help rather than hurt one another. The paper also claims that lane-line evaluations in earlier work were unfair because models were trained on 8-pixel-wide lane labels but tested against 2-pixel-wide labels, and that dilating the test labels to 8 pixels gives a fairer comparison. If the paper is right, real-time multi-task panoptic perception can be achieved without manually designed task-specific heads, and a simple label-alignment fix can change published rankings.","feed_headline":"RMT-PPAD tops BDD100K on all three driving tasks at 32.6 FPS","feed_subtitle":"RMT-PPAD beats earlier multi-task models on detection, drivable area, and lanes—and fixes an unfair lane-label test.","key_machinery":"The central object is the gate control with adapter (GCA), a lightweight module inserted at three scales between the efficient hybrid encoder and the task decoders. For each scale, a task adapter, built from $1\\times1$ convolutions and a depthwise separable convolution, extracts task-specific features while leaving shared features intact; a gate then applies squeeze-and-excitation-style channel attention, spatial attention, and a fusion gate module, clipping the combined gate to $[0.05, 0.95]$ so no stream is fully suppressed. The fusion rule is $out = shared + gate \\cdot (task - shared)$, a residual interpolation that lets each task emphasize useful shared features and suppress conflicting ones. A second mechanism is the adaptive segmentation decoder, which stacks the three scale feature maps and multiplies them by softmax-normalized learnable weights $\\alpha \\in \\mathbb{R}^{2\\times3}$, one row per segmentation task, avoiding manually designed heads for drivable area versus lane lines. The third mechanism is the label-alignment fix: dilating lane-line test labels to the 8-pixel width used in training, which the paper argues makes lane-line IoU a faithful measure.","core_discovery":"The paper's claim is that a single RT-DETR-based network with six lightweight GCA modules can jointly predict vehicles, drivable area, and lane lines better than all open-source multi-task baselines on BDD100K while still running above the 30 FPS camera sampling rate. The load-bearing novelty is the GCA module: an adapter produces task-specific features from shared multi-scale features, and a dynamic gate combining channel attention, spatial attention, and a fusion gate computes $out = shared + gate \\cdot (task - shared)$ with the gate clipped to $[0.05, 0.95]$, which the authors say alleviates negative transfer; their gradient-cosine-similarity analysis supports this by showing fewer conflicting gradients in the three task pairs. The paper further claims that the previous lane-line protocol was inconsistent: models were trained on 8-pixel-wide lane labels but evaluated against 2-pixel-wide test labels, so more accurate predictions accumulated false positives and were scored worse; dilating the test labels with a $7\\times7$ elliptical structuring element restores agreement with visual quality. A unified segmentation decoder with a learnable softmax-normalized weight tensor $\\alpha$ over the three scales learns task-appropriate preferences, such as drivable area relying more on high-level features and lane lines relying more on low-level detail.","pith_inferences":["Our inference: the label-width inconsistency probably affects any benchmark where training and test annotations use different line thicknesses, so the same dilation audit could be applied to other road-marking or lane datasets.","Our inference: the GCA module is task-agnostic and could be inserted into other multi-task dense-prediction models; its benefit could be tested by measuring gradient cosine similarity before and after insertion on depth, segmentation, and detection combinations.","Our inference: the learned weights $\\alpha$ offer a diagnostic for which feature scales each task relies on; monitoring them under domain shift might reveal when a model starts depending on the wrong resolution.","Our inference: the real-world dashcam evaluation in the paper is qualitative, so a quantitative extension with manually labelled real-road frames would be needed to confirm the claimed stability in practice."],"forward_implications":["If the GCA result holds, multi-task driving perception can be built with a shared backbone plus a lightweight gating module and no task-specific segmentation heads, reducing design and engineering cost.","If the lane-label dilation correction is accepted, the published lane-line IoU rankings of open-source models such as YOLOP, HybridNet, YOLOPX, and A-YOLOM should be recomputed, since some models are penalized more than others by thick predictions compared with 2-pixel ground truth.","The 32.6 FPS result implies the model runs above the 30 FPS camera sampling rate on a single RTX 4090 without inference accelerators, meeting the paper's definition of real-time for autonomous driving.","The gradient cosine-similarity evidence indicates that the GCA reduces conflicts among the three tasks, so joint training need not degrade individual task performance."],"supporting_citations":[{"why":"Supplies the real-time DETR backbone and detection decoder that RMT-PPAD extends into a multi-task model.","marker":"[15]"},{"why":"Provides the BDD100K dataset, splits, and annotations used for every comparison and ablation.","marker":"[16]"},{"why":"Establishes the 8-pixel lane-line training-label convention that creates the train/test width mismatch.","marker":"[14]"},{"why":"Defines the lane-line accuracy formula that ignores false positives and documents the negative-transfer trend this model targets.","marker":"[5]"},{"why":"YOLOPX is the main reproduced baseline and the source of the lane metric opinions the paper revisits.","marker":"[3]"},{"why":"A-YOLOM is a reproduced baseline whose lane predictions illustrate the label-width evaluation problem.","marker":"[4]"},{"why":"HybridNet is a reproduced baseline in the main comparison table and ablation context.","marker":"[11]"},{"why":"Defines the bipartite matching loss and set-prediction formulation used for the detection task.","marker":"[21]"},{"why":"Focal loss is used in both segmentation losses to handle foreground-background imbalance.","marker":"[34]"},{"why":"Tversky loss is used for lane-line segmentation to penalize false negatives.","marker":"[35]"}],"fun_headline_variants":["RMT-PPAD: one transformer, three tasks, 32.6 FPS","Fixing unfair lane-label test boosts multi-task driving AI","Adaptive gating prevents task interference in real-time driving","One network, three tasks: panoptic driving at 32.6 FPS","RMT-PPAD: best on BDD100K for detection, drivable area, lanes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results hinge on the claim that earlier multi-task models trained lane lines with 8-pixel-wide labels and were tested against 2-pixel-wide labels, so dilating the test labels to 8 pixels is a fair correction rather than a change of the task.","fun_headline_variants_meta":{"raw":{"variants":["RMT-PPAD: one transformer, three tasks, 32.6 FPS","Fixing unfair lane-label test boosts multi-task driving AI","Adaptive gating prevents task interference in real-time driving","One network, three tasks: panoptic driving at 32.6 FPS","RMT-PPAD: best on BDD100K for detection, drivable area, lanes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000932,"raw_usage":{"total_tokens":4069,"prompt_tokens":1101,"completion_tokens":2968,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":717,"completion_tokens_details":{"reasoning_tokens":2866}},"tokens_in":717,"tokens_out":2968,"duration_ms":23105,"temperature":1.0,"reasoning_tokens":2866,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:36:13.662810+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Download the lane-line annotation files and the training and evaluation scripts of YOLOP, HybridNet, YOLOPX, and A-YOLOM, and measure the actual pixel-width distributions of lane masks in training versus test. If the training labels are not predominantly 8 pixels wide or the test labels are not predominantly 2 pixels wide, the paper's unfair-evaluation diagnosis and the resulting lane-IoU rankings in Table III collapse. Alternatively, re-evaluate all models on the original 2-pixel test labels; if RMT-PPAD does not still lead in lane IoU, the label-dilation fix, not the model, is carrying the lane result.","supporting_citations":[{"cited_title":"Detrs beat yolos on real-time object detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the real-time DETR backbone and detection decoder that RMT-PPAD extends into a multi-task model."},{"cited_title":"Bdd100k: A diverse driving dataset for heterogeneous multitask learning,","cited_arxiv_id":null,"evidence_quote":"Provides the BDD100K dataset, splits, and annotations used for every comparison and ablation."},{"cited_title":"Learning lightweight lane detection cnns by self attention distillation,","cited_arxiv_id":null,"evidence_quote":"Establishes the 8-pixel lane-line training-label convention that creates the train/test width mismatch."},{"cited_title":"Yolop: You only look once for panoptic driving perception,","cited_arxiv_id":null,"evidence_quote":"Defines the lane-line accuracy formula that ignores false positives and documents the negative-transfer trend this model targets."},{"cited_title":"Yolopx: Anchor- free multi-task learning network for panoptic driving perception,","cited_arxiv_id":null,"evidence_quote":"YOLOPX is the main reproduced baseline and the source of the lane metric opinions the paper revisits."},{"cited_title":"You only look at once for real-time and generic multi-task,","cited_arxiv_id":null,"evidence_quote":"A-YOLOM is a reproduced baseline whose lane predictions illustrate the label-width evaluation problem."},{"cited_title":"Tversky loss function for image segmentation using 3d fully convolutional deep networks,","cited_arxiv_id":null,"evidence_quote":"Tversky loss is used for lane-line segmentation to penalize false negatives."}],"review_version":1}