{"id":"b7d0dca7-233d-459e-9e55-0608ec952813","arxiv_id":"2506.15313","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An online HD map model that swaps ResNet and Swin backbones for DINOv2 and adds auxiliary BEV segmentation heads reports mAP gains of about 1.5 points over MapQR on nuScenes.","lead":"MapFM uses the DINOv2 foundation model as its camera backbone and adds auxiliary bird's-eye-view segmentation losses to an existing DETR-style decoder for online HD map generation. On nuScenes it reports mAP 67.8 to 69.0, about 1.5 points above the MapQR baseline, with gains attributed partly to the auxiliary road-surface segmentation head.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline margins in Tables 1-3 are single-run mAP differences of 0.4-1.5 points with no error bars or significance testing, so the claimed MapFM gain over MapQR may be within run-to-run seed noise.","rationale":"I read the full manuscript in good faith. The method is clearly described: a DINOv2 image encoder, a BEVFormer-style BEV encoder, an auxiliary BEV road-surface segmentation head, and a MapQR vector decoder, with a weighted loss combining point, classification, direction, BEV/PV segmentation, and Dice-based surface losses. I found no internal inconsistency or broken derivation; the pipeline is plausible and the ablation tables point in the expected direction. The serious weakness is empirical rather than architectural: every comparison is a single run, and the reported gains (0.4-1.5 mAP) are comparable to the seed-to-seed variance typically observed in DETR-style map construction on nuScenes. The reader's weakest assumption identified exactly this issue, and I agree with that assessment. I would not change the conditional verdict: the paper deserves acceptance if repeated-seed evaluation confirms the margins, but should be toned down if the variance is large. The concrete test proposed above would settle whether the concern lands.","tokens_in":8413,"tokens_out":4567,"duration_ms":46191,"concrete_test":"Run each of the three key configurations (MapQR ResNet50, MapFM DINOv2-small 24ep, MapFM DINOv2-base 16ep) for at least 3 seeds each with identical hyperparameters, and repeat the Table 2 auxiliary-head on/off pair for 3 seeds. Report mean and standard deviation per category (APdiv, APped, APbound) and a paired bootstrap 95% confidence interval for the mAP difference MapFM-minus-MapQR. If any confidence interval includes zero, or if the seed standard deviation of either configuration is 0.8 mAP or larger, the stated gains should be reclassified as preliminary rather than established. Also verify that the released repository reproduces Table 1 within 0.3 mAP under the same single-seed protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on small, single-run mAP differences. Section 4.1 reports no repeated seeds, and every table cell in Tables 1-5 is one number per configuration. The main comparison is 67.8 vs 66.3 (MapFM DINOv2-small vs MapQR ResNet50, +1.5 mAP) and 69.0 vs 66.3 (DINOv2-base, +2.7 mAP); the auxiliary-head ablation shows gains of only 0.4 mAP on ResNet50 (Table 2) and 1.1 mAP on DINOv2-small at 14 epochs (Table 3). DETR-style map prediction on nuScenes is known to be sensitive to seed and training noise, and the margins here are the same order as typical run-to-run variation in this model family. Without repeated runs or confidence intervals, the evidence cannot distinguish a genuine mechanism from favorable initialization. This is the load-bearing weakness: the architecture is coherent and the losses are well-specified, but the empirical basis for the advertised improvement is not statistically grounded.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MapFM, an extension of the MapQR end-to-end vectorized HD map prediction framework. MapFM replaces the ResNet/Swin image backbone with a fine-tuned DINOv2 foundation model and adds an auxiliary road-surface segmentation head that predicts BEV drivable-area and pedestrian-crossing masks, trained with a Dice loss. The total loss combines MapTRv2/MapQR losses with the new surface loss. Experiments on nuScenes report that MapFM with DINOv2-small reaches 67.8 mAP at 24 epochs and MapFM with DINOv2-base reaches 69.0 mAP at 16 epochs, compared with 66.3 mAP for the MapQR ResNet50 baseline. Ablations address the auxiliary head, DINOv2 feature aggregation strategies, and frozen versus fine-tuned transfer learning.","tokens_in":8663,"tokens_out":3262,"duration_ms":31642,"significance":"The paper addresses a practical and timely problem, online vectorized HD mapping for autonomous driving. Replacing standard backbones with DINOv2 and adding a lightweight BEV segmentation head is a plausible, low-cost improvement that, if confirmed, would be useful to the field. The auxiliary-head ablation in Table 2 and Table 3 supports the direction of the claimed effect. The manuscript also makes its source code publicly available, which is a strength. The main weakness is that all benchmark claims rest on single-run mAP differences of 0.4 to 1.5 points with no repeated seeds, error bars, or significance testing, so the numerical headline is not yet statistically grounded.","major_comments":[{"comment":"The central claim of improvement over MapQR is based on single-run mAP values with no error bars or significance testing. For example, Table 1 reports MapFM DINOv2-small at 67.8 mAP versus MapQR ResNet50 at 66.3 mAP, a margin of 1.5 points, and the auxiliary-head ablation in Table 2 shows a 0.4-point gain for ResNet50. These margins are of the same order as typical run-to-run variation in DETR-style map prediction models, so the evidence cannot currently distinguish the proposed contributions from training noise. The authors should report mean and standard deviation over at least three seeds and, if appropriate, a paired significance test or confidence intervals for the key comparisons.","section":"Section 4.1, Tables 1-5"},{"comment":"The experiments do not provide a controlled comparison that isolates the effect of the DINOv2 backbone from the training budget. Table 3 gives MapFM without the auxiliary head (essentially MapQR with DINOv2-small) only at 14 epochs (63.0 mAP), while the MapQR baseline in Table 1 is at 24 epochs (66.3 mAP). Table 5 reports DINOv2 fine-tune transfer strategies at 14 epochs only. The improved headline numbers in Table 1 therefore confound the backbone change with a longer training schedule. The authors should include MapQR with DINOv2-small and DINOv2-base at identical epoch counts to the baseline, and MapFM without the auxiliary head at the same final epochs as the full MapFM.","section":"Tables 1, 3, and 5"},{"comment":"The main model configuration is not fully specified. The method text says DINOv2 is 'fine-tuned,' but Table 5 evaluates only 'Fine-tune DINOv2's Last Layer' at 14 epochs, which reaches 65.2 mAP, while Table 1 reports DINOv2-small at 24 epochs reaching 67.8 mAP. The authors should state explicitly which transfer-learning strategy, which set of DINOv2 layers, and which feature-aggregation choice are used in the headline configurations, and ideally ablate this choice at the final training budget rather than only at 14 epochs.","section":"Section 3.2 and Table 5"},{"comment":"The conclusion that using only the last DINOv2 block is the best feature-aggregation strategy is supported by a 0.2 mAP difference (64.1 vs. 63.9) from a single run at 12 epochs. This is insufficient evidence to prefer one aggregation method over another, especially since this choice is used in the main model. The feature-aggregation ablation should be repeated with at least three seeds or combined into the controlled comparison at the final training budget.","section":"Table 4"}],"minor_comments":[{"comment":"The equation uses Q_BEV but the text defines Q_Enc as the learnable BEV queries; the notation should be made consistent.","section":"Section 3.3, Eq. (1)"},{"comment":"The word 'significantly' is used to describe performance gains, but no statistical significance test is reported; please replace it with a quantitative statement or add significance testing.","section":"Abstract and Section 4.2"},{"comment":"There are several typographical errors, including 'Backnone' in Section 3.1, 'milti-camera' in Section 3.1, and 'approache' in Section 2.1; a proofreading pass is needed.","section":"Throughout"},{"comment":"The label 'Fine-tune DINOv2's Last Layer' is ambiguous regarding whether this means only the final transformer block or the final linear layer; please clarify.","section":"Table 5"},{"comment":"The figure caption says 'Green– drivable area mask, Blue– pedestrian crossing area mask' but an extra comma and spacing appear; also, the qualitative comparison in Figure 4 would benefit from zoomed-in views that make differences between MapQR and MapFM visible.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's core architectural idea is coherent and the ablations are directionally supportive, but the empirical case is currently built on single-run numbers. I would ask the authors for multi-seed statistics on the key comparisons and for controlled experiments that vary the backbone at fixed training epochs. The claim of being the first to apply DINOv2 as a direct feature extractor for vectorized HD mapping may also deserve a careful prior-art check, though this alone would not change my recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a solid, incremental engineering paper. The two tested ideas — swapping the ResNet/Swin backbone for a fine-tuned DINOv2 and adding an auxiliary BEV road-surface segmentation head — are each straightforward applications of known components, but the combination on MapQR is new and the ablations are sensible. The paper does not reshape the field, but it gives the online HD-map subfield a useful data point: DINOv2 features help, and multi-task supervision helps a little more.\n\nWhat it does well: the method section is readable, the loss is fully specified, and Tables 2, 4, and 5 are real ablations of the two main contributions. The authors also looked at feature aggregation strategies and frozen vs. fine-tuned backbones, which is more than many such papers do. Source code is promised, which matters.\n\nThe soft spots are real and mostly statistical. Every result is a single run on nuScenes validation, with no error bars or significance tests. The headline gains are 1.5 mAP (DINOv2-small, 24 epochs) and 2.7 mAP (DINOv2-base, 16 epochs) over the MapQR ResNet50 baseline. Those numbers are plausible, but DETR-style map training is known to be seed-sensitive, and the auxiliary-head ablation gains are only 0.4 mAP (ResNet50) and 1.1 mAP (DINOv2-small, 14 epochs). Without multiple seeds, the paper cannot distinguish a genuine mechanism from favorable initialization. The stress-test note is correct on that point.\n\nSmaller issues: the abstract and intro use \"significantly\" loosely, and the claim of being the \"first application\" of foundation models for vectorized HD mapping is undercut by BEVCar's DINOv2 use in BEV segmentation (cited by the authors). That framing is not dishonest, but it is stronger than the evidence supports.\n\nThe paper deserves a serious referee. It is clearly written, the experimental setup is mostly standard, and the code release will let others check reproducibility. My recommendation: send it to peer review, but ask the authors to add multiple seeds (at least three) for the main comparison and the auxiliary-head ablation, or provide confidence intervals. If the gains survive that, the paper is a fine workshop-or-conference contribution.\n\nBest.","headline":"Useful incremental result on MapQR with DINOv2 backbone and auxiliary BEV segmentation, but the reported gains are single-run margins that may not clear seed noise.","tokens_in":9176,"tokens_out":1050,"would_cite":false,"duration_ms":12676,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that swapping in a fine-tuned DINOv2 encoder and adding an auxiliary BEV road-surface segmentation head raises nuScenes vectorized HD map mAP from 66.3 to 69.0.","keywords":["online HD map construction","vectorized map","DINOv2","foundation model","bird's-eye view","multi-task learning","auxiliary segmentation","nuScenes"],"falsifier":"Train MapFM and the MapQR baseline multiple times with several random seeds under identical settings on nuScenes and compare the mAP distributions; if the 2.7-point best-case gap falls within one standard deviation of the runs, the central claim is not supported.","tokens_in":8229,"feed_emoji":"🗺️","tokens_out":6146,"duration_ms":52901,"temperature":0.7,"pith_summary":"MapFM is an end-to-end model for online vectorized HD map construction that makes two changes to the MapQR architecture: it replaces the ResNet or Swin image backbone with a fine-tuned DINOv2 foundation model, and it adds an auxiliary head that predicts drivable-area and pedestrian-crossing masks in bird's-eye view. The paper's central claim is that these two changes raise prediction quality on the nuScenes benchmark, with MapFM reaching 67.8 mAP using DINOv2-small at 24 epochs and 69.0 mAP using DINOv2-base at 16 epochs, compared to 66.3 mAP for the MapQR baseline with ResNet50. Ablations attribute part of the gain, roughly 0.4 to 1.0 mAP depending on backbone, to the auxiliary segmentation head, with the rest coming from the stronger features of the foundation backbone. A sympathetic reader would care because these are drop-in component swaps, not new decoder designs, and better online maps matter directly for downstream planning and localization.","feed_headline":"DINOv2 backbone lifts HD map mAP to 69.0 on nuScenes","feed_subtitle":"MapFM swaps ResNet for a fine-tuned DINOv2 and adds a road-surface head, lifting mAP from 66.3 to 69.0.","key_machinery":"The load-bearing components are the DINOv2 image encoder, used as a foundation backbone for surround-view camera features, and the Auxiliary Road Surface Segmentation Head (ARSS), a dense BEV head that predicts drivable-area and pedestrian-crossing masks from the BEV features and is supervised with Dice loss. Between them sits the BEVFormer BEV encoder, which projects multi-view DINOv2 features into a shared BEV representation, and the MapQR vector map decoder, which models map elements as ordered point sequences through a scatter-and-gather query mechanism. The ARSS head's role is to force the BEV features to encode road-surface context, which the paper argues makes the vectorized map prediction easier.","core_discovery":"The paper's discovery is that a large self-supervised vision transformer, fine-tuned on the driving task, transfers to online HD map prediction better than the task-specific backbones previously used, and that dense road-surface supervision makes the BEV features more useful for vectorized map elements. The authors build MapFM on MapQR's scatter-and-gather query decoder and a BEVFormer-style BEV encoder, swap in DINOv2, and add the Auxiliary Road Surface Segmentation Head (ARSS) that predicts drivable area and pedestrian crossing masks with Dice loss. On the nuScenes validation set the best configuration, MapFM with DINOv2-base trained for 16 epochs, reaches 69.0 mAP, surpassing MapQR's 66.3 mAP with ResNet50 and 66.1 mAP with SwinT; the DINOv2-small 24-epoch variant reaches 67.8 mAP. Adding the ARSS head to MapQR improves mAP by 0.4 points with ResNet50 and 1.0 with SwinT, and improves MapFM with DINOv2-small from 63.0 to 64.1 at 14 epochs. The paper also finds fine-tuning DINOv2's last layer markedly outperforms frozen DINOv2 or fine-tuned RADIOv2.5, and that last-layer features beat multi-layer aggregation strategies.","pith_inferences":["If the single-run results hold across seeds, self-supervised vision transformers like DINOv2 could become the default camera backbone for BEV map construction, with dense auxiliary segmentation serving as a cheap task adapter.","The same ARSS recipe could be tested on temporal or streaming map methods and on radar or LiDAR fusion pipelines, since the segmentation head only consumes BEV features and should be agnostic to sensor input.","The paper compares backbones within a fixed decoder; a testable extension is whether the gain from DINOv2 persists under decoders that use different point representations, such as Bezier curves or pivot points."],"forward_implications":["Under the same training recipe, a DINOv2-small backbone with the ARSS head beats the MapQR baseline with ResNet50 or SwinT by 1.5 to 1.7 mAP on nuScenes.","DINOv2-base reaches 69.0 mAP in only 16 epochs, suggesting foundation backbones also reduce training time needed to exceed prior methods.","The auxiliary road-surface segmentation head yields a consistent mAP gain of 0.4 to 1.0 points across ResNet50, SwinT, and DINOv2-small backbones, so the multi-task supervision transfers across architectures.","Frozen foundation features are not enough for this task: fine-tuning DINOv2's last layer gives 65.2 mAP versus 52.4 mAP frozen, so task adaptation is required even with strong representations.","For this pipeline, using only the last DINOv2 block (64.1 mAP) outperforms combining blocks 4, 8, and 12 by concatenation (63.9) or a multi-layer CNN (62.6), so feature aggregation from multiple layers is not beneficial here."],"supporting_citations":[{"why":"Supplies the MapQR baseline and decoder whose point-set query mechanism MapFM keeps; the reported gains are measured against it.","marker":"[19]"},{"why":"DINOv2 is the foundation model whose fine-tuned features carry the proposed improvement.","marker":"[20]"},{"why":"BEVFormer's BEV encoder is used to lift multi-camera DINOv2 features into the shared BEV representation.","marker":"[13]"},{"why":"MapTRv2 provides the auxiliary perspective-view and BEV segmentation heads and losses that MapFM reuses.","marker":"[15]"},{"why":"MapTR defines the map-element matching, Chamfer-distance thresholds, and mAP evaluation protocol used for all comparisons.","marker":"[14]"},{"why":"nuScenes is the dataset on which all models are trained and validated.","marker":"[3]"}],"fun_headline_variants":["Fine-tuned DINOv2 lifts HD map mAP to 69.0","Multi-task supervision sharpens online HD maps","MapFM: foundation model beats task-specific backbones","DINOv2 transfer boosts nuScenes HD mapping by 2.7 mAP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported mAP differences are treated as real signal rather than seed-dependent noise, since all comparisons use single-run validation numbers without error bars or significance tests.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuned DINOv2 lifts HD map mAP to 69.0","Multi-task supervision sharpens online HD maps","MapFM: foundation model beats task-specific backbones","DINOv2 transfer boosts nuScenes HD mapping by 2.7 mAP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1669,"prompt_tokens":978,"completion_tokens":691,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":615}},"tokens_in":594,"tokens_out":691,"duration_ms":5813,"temperature":1.0,"reasoning_tokens":615,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:36:22.188364+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MapFM and the MapQR baseline multiple times with several random seeds under identical settings on nuScenes and compare the mAP distributions; if the 2.7-point best-case gap falls within one standard deviation of the runs, the central claim is not supported.","supporting_citations":[{"cited_title":"In: European Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Supplies the MapQR baseline and decoder whose point-set query mechanism MapFM keeps; the reported gains are measured against it."},{"cited_title":"International Journal of Computer Vision pp","cited_arxiv_id":null,"evidence_quote":"MapTRv2 provides the auxiliary perspective-view and BEV segmentation heads and losses that MapFM reuses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MapTR defines the map-element matching, Chamfer-distance thresholds, and mAP evaluation protocol used for all comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"nuScenes is the dataset on which all models are trained and validated."}],"review_version":2}