Pith. sign in

REVIEW 5 major objections 7 minor 64 references

MATS: A novel multi-modality multi-task learning framework for 3D perception in autonomous driving

T0 review · 5 major / 7 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single shared BEV map is too thin for multi-task 3D perception; diverse fused maps plus task-routed experts fix the conflict.

desk verdict Solid multi-task BEV systems paper with real nuScenes gains and clean ablations; the “diverse carriers” story is underspecified but the empirical pattern is still useful. read the letter →

arxiv 2607.24224 v1 pith:7DRI2YW4 submitted 2026-07-27 cs.CV

Junchen Huo , Wanming Hao , Song Wang , Enqing Chen , Shouyi Yang , Guanghui Wang This is my paper · ORCID
classification cs.CV
keywords multi-modalityfusionmulti-tasklearningBEVrepresentationMixture-of-Experts3DobjectdetectionmapsegmentationautonomousdrivingnuScenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autonomous driving systems fuse camera and LiDAR into a bird’s-eye-view (BEV) map and try to run several perception jobs—object detection and map segmentation—from that one map. This paper argues that one fused map cannot carry what those jobs need at once, so joint training often hurts accuracy relative to separate models. MATS instead builds several lightly fused BEV maps by recalibrating camera–LiDAR features in parallel branches, then routes each task through its own Mixture-of-Experts path that picks the maps it needs while sharing a little common knowledge. On the large nuScenes benchmark with camera and LiDAR, the joint model beats strong multi-task baselines and largely closes the gap to single-task training, with especially large gains on map segmentation. A sympathetic reader cares because multi-task multi-sensor stacks are cheaper to deploy than a fleet of specialist models, but only if joint training does not quietly degrade the very tasks it is meant to share.

What carries the argument

Modality-adaptive multi-branch BEV fusion plus task-specific MoE: parallel identical fusion branches each do a 3×3 fuse then channel recalibration to emit diverse BEV maps; separate detection/segmentation gates pick sparse specialized experts while always-on shared experts supply task-invariant features.

What would settle it

On nuScenes validation with the same encoders and batch size, remove multi-branch diversity (force one fused map or shared-branch heads) or collapse task-specific MoE routing and check whether joint mAP/NDS and mean mIoU fall back to the negative-transfer levels of the UniTR/BEVFusion multi-task baselines.

Watch

Extended reading notes

Core claim

The central claim is that multi-modality multi-task 3D perception fails mainly because a single fused BEV feature map lacks capacity for conflicting task needs, and that a simple multi-branch modality-adaptive fusion producing diverse BEV candidates, combined with a task-specific Mixture-of-Experts that routes each task to specialized and shared experts, restores performance so joint detection and map segmentation on camera–LiDAR nuScenes data exceeds prior multi-task methods and reduces negative transfer.

Load-bearing premise

That a few parallel, identically built fusion branches and sparse task gates are enough diversity to resolve multi-task conflict, rather than needing stronger geometric alignment or much larger model capacity.

Editorial extensions

If this is right

  • Joint camera–LiDAR stacks can keep one deployed network for detection and map segmentation without the large segmentation collapse seen in single-map multi-task baselines.
  • Simple parallel fusion branches can replace intricate single-map fusion designs when the goal is multi-task BEV perception.
  • Task-conditioned expert routing plus a small shared expert set is a practical way to balance gradients so the weaker task (segmentation) is not dominated in backprop.
  • The same fusion-plus-task-MoE pattern extends to occupancy as a third task with measurable recovery from negative transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If representational diversity at BEV resolution is the real bottleneck, similar multi-carrier fusion may help other BEV multi-task pairs (tracking, prediction) without redesigning sensor alignment.
  • The gradient-ratio plot suggests future work could treat task gradient balance as an explicit design target rather than only an observed side effect of MoE routing.
  • Because memory cost of extra low-resolution BEV maps is small, deployment cost may favor several cheap fused maps over one heavily engineered shared map.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes MATS, a multi-modality multi-task framework for joint 3D object detection and BEV map segmentation on nuScenes. Instead of fusing camera and LiDAR features into a single shared BEV map, MATS uses N=3 identically architected, independently weighted fusion branches (3×3 conv + SE-style channel recalibration) to produce multiple fused BEV maps, followed by a task-specific sparse MoE with detection experts, segmentation experts, and shared experts with separate noisy top-k gates per task. On the nuScenes validation set, MATS reports 67.5 mAP / 70.8 NDS / 64.6 mean mIoU, improving over a retrained UniTR baseline (66.1 / 70.1 / 59.8) under matched batch size, and shows gains when grafted onto both BEVFusion and UniTR encoders and onto the DAOcc two-/three-task pipeline. Ablations cover the two components, the branch count N, the number of shared experts, and alternative fusion architectures; a gradient-ratio scatter plot (Fig. 4) is offered as evidence that MATS balances detection and segmentation gradients.

Significance. If the results hold, this is a useful and practically relevant contribution: negative transfer in multi-task BEV perception is a real obstacle to deployment, and a lightweight, drop-in remedy with consistent gains across two encoders and a second baseline (DAOcc) would be of interest to the autonomous-driving perception community. The paper's strongest assets are empirical hygiene rather than theory: matched-protocol retrained baselines, parameter counts in Table II, a full component ablation, sweeps over N and s, an alternative-architecture control (Table V), and a three-task generalization test. No parameter-free derivations or machine-checked claims are involved, and none are needed; the evidentiary standard here is experimental, and the experimental coverage is broader than typical for the subfield, though it stops short of directly validating the claimed mechanism (diverse carriers) and reports no seed variance.

major comments (5)
  1. [§I, §III-B/C, Fig. 4] The central mechanistic claim — that N parallel fusion branches 'generate diverse fused BEV feature maps' and that this diversity is what alleviates negative transfer — is never measured. All N branches share the identical 3×3-conv + SE-style recalibration architecture (Eq. 1, Fig. 2), receive identical inputs, and are trained with no diversity/orthogonality regularizer; they differ only in random initialization, which for small identical conv blocks frequently yields highly correlated solutions. The supporting evidence is indirect: Table III/V ablations show an outcome, and Fig. 4 shows balanced gradients, not diverse carriers. Please provide direct evidence: (a) inter-branch feature similarity (e.g., CKA or cosine similarity between branch outputs on held-out data), and (b) an analysis of the gating distributions eG_det vs eG_seg showing the two tasks actually route to different branch
  2. [§IV-B, Fig. 4, Eq. (9)] The attribution of the segmentation gains to gradient balancing is confounded with the multi-task loss weights. Eq. (9) sets α=1.0, β=2.0 'following the baselines,' but no ablation of α/β is reported anywhere. Fig. 4 (a single training snapshot, one seed) shows that MATS has a more balanced γ than the baseline, but the obvious control — retraining the UniTR baseline with a larger β to balance γ directly — is missing. Without it, the paper cannot distinguish 'architecture balances gradients and thereby helps segmentation' from 'any reweighting of the segmentation loss helps segmentation.' A small sweep of β in the baseline (and ideally in MATS) alongside the γ measurement would make the §IV-B causal story testable.
  3. [§IV-D.4, Table V] The capacity control is not parameter-audited. Table V compares multi-branch fusion against SB-MC and SB-MH alternatives, but unlike Table II (which commendably reports parameter counts), no parameter or FLOP counts are given for the three variants. SB-MC triples the output channel dimension of a single branch; whether its parameter count matches the 3-branch design is not stated. Since the argument of §IV-D.4 is that the gain comes from architecture (decoupled gradient paths) rather than capacity, matched-parameter reporting is load-bearing for that conclusion.
  4. [§IV-E vs Table VI] The text and the table disagree. The text states: 'the detection mAP and the occupancy mIoU in two-task learning increase from 62.3% to 63.9% and from 46.8% to 49.2%.' Table VI shows DAOcc(STL) detection at 62.3, DAOcc(MTL) at 59.5, and DAOcc+MATS(MTL) at 61.6 — 63.9 appears nowhere in the table, and the MTL-to-MTL gain is 59.5→61.6 (+2.1), not 62.3→63.9. Similarly the three-task paragraph's '+1.9% mAP' matches 57.2→59.1, so the two-task sentence appears to contain wrong numbers. Please correct and double-check all quantitative statements against the tables; this bears directly on the generalization claim of §IV-E.
  5. [§IV, Tables I–V] All results appear to be single runs. The headline detection gain over retrained UniTR is +1.4 mAP / +0.7 NDS (Table I), and several ablation differences in Tables III–V are in the 0.3–1.0 range, which is comparable to typical seed-to-seed variance on nuScenes detection with small batch size (the authors themselves retrain at batch size 12 due to GPU limits). At minimum, the key claims (Table I main row, Table III component ablation) should report mean±std over ≥3 seeds, or the paper should acknowledge single-run noise explicitly and avoid 'significant margin' language for sub-1-point differences.
minor comments (7)
  1. [§III-C, Eq. (8)] Eq. (8): the noisy top-k gating formula deviates from Shazeer et al. [46] and is ambiguously typeset — 'log(1+e^x)' applies softplus to x rather than to (x·W_n), and the multiplicative/additive structure of the noise term is unclear. The standard form is softmax(x·W_g + ε·softplus(x·W_n)). Please clarify whether this is a modification or a typesetting error.
  2. [§I, §IV] The abstract/introduction motivate the framework by deployment cost, but only parameter counts (Table II) are reported — no latency, FPS, or memory measurements. Given that N=3 branches plus MoE experts add inference cost, at least a runtime comparison against UniTR on the same hardware would substantiate the deployment narrative.
  3. [§IV-A/B, Table I] Evaluation is restricted to the nuScenes validation set. Test-server numbers for the main configuration would strengthen the SOTA claim, which currently rests on val-set comparisons with a mixture of retrained and literature numbers (Table I mixes the two; the ∗ marking helps, but literature numbers were obtained at larger batch sizes).
  4. [Abstract, §IV-B] The claim 'outperforms SOTA by a significant margin' (abstract, §IV-B) is overstated on segmentation relative to single-task models: X-Align (L+C) reports 65.7 mean mIoU vs MATS 64.6 in Table I itself. Please scope the claim to multi-task learning, as the table actually supports.
  5. [Throughout] Typos/notation: 'UnitR' for UniTR (§II-A); 'shard ones' should be 'shared ones' (Fig. 3 caption); 'the objection task' should be 'the detection task' (§IV-B, after Eq. 11); 'nuScene' vs 'nuScenes' inconsistent (§IV-A, §IV-D.2); 'RetNet-50' presumably means ResNet-50 (§IV-E).
  6. [Fig. 4] Fig. 4: state at which training iteration/epoch the gradient snapshot is taken, how many batches are aggregated, and whether the 256 points are per-kernel ratios (the text implies this). A time series of γ over training would be more convincing than a single scatter.
  7. [Abstract] Code and models are promised only 'upon publication.' An anonymous release at review time would materially improve verifiability, especially given the retraining-based comparisons in Tables I–II.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical multi-task CV system evaluated on external nuScenes metrics; design choices are ablated, not laundered into predictions.

full rationale

MATS is a standard empirical systems paper. Its central claims (67.5% mAP / 70.8% NDS / 64.6% mIoU on nuScenes val; gains over retrained UniTR and other multi-task baselines) are measured against external benchmark metrics that are not functions of the model’s own fitted parameters. The modality-adaptive multi-branch fusion (Eq. 1, N branches) and task-specific MoE (Eqs. 6–8) are architectural proposals whose hyperparameters (N=3, k=2, s=1, loss weights α/β following baselines) are selected by ablation tables, not presented as first-principles derivations or out-of-sample “predictions” forced by a fit. There is no self-definitional loop, no uniqueness theorem imported from overlapping authors, and no renaming of a known closed-form result. Skeptical concerns about whether parallel identical SE-style branches actually produce diverse carriers (unmeasured CKA/gating, capacity controls) are evidence-strength / correctness issues, not circularity: nothing in the derivation chain reduces by construction to its inputs. Score 0; steps empty.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

Load-bearing content is architectural and empirical, not axiomatic physics. The claim rests on standard deep-learning practice (BEV encoders, focal/L1 losses, sparse MoE gating), the modeling bet that parallel lightweight fusion branches create useful diversity, and hyperparameters selected on val ablations. No new physical entities; free parameters are design knobs (N, expert counts, loss weights) and the usual trained weights.

free parameters (5)
  • Number of fusion branches N = 3
    Chosen by ablation on val (Table IV); peak at N=3. Directly affects claimed diversity of BEV carriers.
  • Specialized experts n, top-k, shared experts s = n=4, k=2, s=1
    MoE capacity and sparsity set by hand/ablation (n=4, k=2, s=1 best in Fig. 5).
  • Multi-task loss weights α, β = α=1.0, β=2.0
    Fixed to 1.0 and 2.0 following baselines; segmentation up-weighted, influencing gradient-balance narrative.
  • Detection loss component weights α_cls, α_heatmap, α_bbox = 1.0, 1.0, 0.25
    Set to 1.0, 1.0, 0.25 for baseline parity; not derived.
  • Training schedule (lr, epochs, batch, GPUs) = lr=3e-3, 20 epochs, bs=12
    AdamW 3e-3, 20 epochs, batch 12 on 4×4090D; authors note this is smaller than original SOTA setups and retrain baselines accordingly.
assumptions (5)
  • domain assumption Camera and LiDAR features can be aligned and fused usefully in a shared discrete BEV grid without pixel/point-level correspondence.
    Standard BEV-fusion premise throughout §I–III; inherited from BEVFusion/UniTR line.
  • domain assumption A single shared BEV map is representationally insufficient for detection (instance geometry) vs map segmentation (global layout), causing negative transfer.
    Stated as motivation in abstract and §II-B; motivates multi-map design.
  • ad hoc to paper Sparse noisy top-k gating and even-weighted shared experts adequately assign task-specific vs task-invariant computation.
    Eqs. (6)–(8) and Fig. 3; design choice validated only empirically.
  • ad hoc to paper Global average pooling + 1×1 channel recalibration after a 3×3 fuse models “global cross-modality dependencies” enough to produce diverse useful maps.
    §III-B Eq. (1) and Fig. 2; SE-style block treated as modality-adaptive fusion.
  • domain assumption nuScenes val mAP/NDS/mIoU under the authors’ retrain protocol are fair proxies for SOTA comparison.
    §IV-A/B; batch-size-matched retrains of BEVFusion/UniTR assumed sufficient.
invented entities (2)
  • Modality-adaptive multi-branch BEV fusion module (MATS fusion)
    purpose: Generate N diverse fused BEV maps via parallel independent SE-style fusion branches.
    Named module; mechanism is standard conv+channel attention stacked in parallel—not a new physical entity, but a paper-specific construct the claims depend on.
  • Task-specific MoE with det/seg/shared expert groups and dual gates
    purpose: Decouple multi-task gradients and route BEV candidates per task while sharing invariant experts.
    Specialization of sparse MoE to two perception tasks; evidence is internal ablations only.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MATS: A novel multi-modality multi-task learning framework for 3D perception in autonomous driving." pith.science (2026). https://pith.science/paper/7DRI2YW4

@misc{pith2026260724224,
  author       = {Pith},
  title        = {Pith review of: MATS: A novel multi-modality multi-task learning framework for 3D perception in autonomous driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DRI2YW4}},
  note         = {Machine review of arXiv:2607.24224}
}
read the original abstract

Multi-modality data from different sensors provides rich complementary information for 3D perception, becoming an essential component in reliable autonomous driving systems. Current research typically designs intricate and complex fusion strategies to integrate information from multimodal data on a unified bird's-eye-view (BEV) feature map for the joint learning of multiple perception tasks. However, such a single feature map hardly carries sufficient information to simultaneously meet the requirements of various perception tasks, leading to a very limited perception performance. To mitigate this limitation, this paper proposes MATS, a novel multi-modality multi-task learning approach with modality-adaptive BEV fusion and task-specific Mixture-of-Experts (MoE) for 3D perception. Specifically, a simple modality-adaptive BEV fusion module is designed to adaptively recalibrate the BEV features by modeling the global cross-modality dependencies, generating diverse BEV feature maps for various perception tasks. For joint multi-task learning, this paper proposes a task-specific MoE module to decouple the tasks and enable the network to automatically choose the appropriate BEV feature candidates for each specific task. To validate the effectiveness of the proposed approach, we conduct extensive experiments on the large-scale benchmark nuScenes. With the camera- and LiDAR-modality input data, the proposed approach outperforms the state-of-the-art (SOTA) by a significant margin. Furthermore, the experimental results on the single tasks show that the proposed approach significantly outperforms the baselines. The code and trained models will be available upon publication.

Figures

Figures reproduced from arXiv: 2607.24224 by the authors.

Figure 1
Figure 1. Illustration of the overall architecture of the proposed MATS. The multi-modality inputs from camera and LiDAR sensors are processed through [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the simple BEV fusion network utilized in the modality [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the task-specific MoE module. The experts are split into [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: We visualize the γ values computed on the last shared layer preceding the decoder block. It is observed that the gradient magnitude associated with the segmentation loss is dominated by that of the detection loss in the baseline. In contrast, MATS effectively balances …
Figure 5
Figure 5. Figure 5: The experimental results along with the increase of the number of [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visualization results of the baseline UniTR and the proposed MATS on multi-task 3D perception, including 3D object detection and BEV map [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 2 linked inside Pith

  1. [1]

    Modeling interactions between autonomous agents in a multi-agent self-awareness architecture,

    A. S. Alemaw, G. Slavic, P. Zontone, L. Marcenaro, D. M. Gomez, and C. Regazzoni, “Modeling interactions between autonomous agents in a multi-agent self-awareness architecture,”IEEE Transactions on Multimedia, 2025

    2025

  2. [2]

    Privacy-concealing coopera- tive perception for bev scene segmentation,

    S. Wang, L. Li, M. Santos, and G. Wang, “Privacy-concealing coopera- tive perception for bev scene segmentation,” in2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2026, pp. 22 232–22 236

    2026

  3. [3]

    Nitedr: Nighttime image de-raining with cross-view sensor cooperative learning for dynamic driving scenes,

    C. Shi, L. Fang, H. Wu, X. Xian, Y . Shi, and L. Lin, “Nitedr: Nighttime image de-raining with cross-view sensor cooperative learning for dynamic driving scenes,”IEEE Transactions on Multimedia, vol. 26, pp. 9203–9215, 2024

    2024

  4. [4]

    Ubtransformer: Uncertainty-based transformer model for complex scenarios detection in autonomous driving,

    K. Wang, Q. Ma, X. Li, C. Shen, R. Leng, and J. Lu, “Ubtransformer: Uncertainty-based transformer model for complex scenarios detection in autonomous driving,”IEEE Transactions on Multimedia, 2025

    2025

  5. [5]

    Physical adversarial attacks for camera-based smart systems: Current trends, categorization, applications, research challenges, and future outlook,

    A. Guesmi, M. A. Hanif, B. Ouni, and M. Shafique, “Physical adversarial attacks for camera-based smart systems: Current trends, categorization, applications, research challenges, and future outlook,”IEEE Access, vol. 11, pp. 109 617–109 668, 2023

    2023

  6. [6]

    Lif-seg: Lidar and camera image fusion for 3d lidar semantic segmentation,

    L. Zhao, H. Zhou, X. Zhu, X. Song, H. Li, and W. Tao, “Lif-seg: Lidar and camera image fusion for 3d lidar semantic segmentation,”IEEE Transactions on Multimedia, vol. 26, pp. 1158–1168, 2023

    2023

  7. [7]

    Synet: A synergistic network for 3d object detection through geometric-semantic-based multi- interaction fusion,

    X. Zhang, K. Bi, S. Chan, S. Lu, and X. Zhou, “Synet: A synergistic network for 3d object detection through geometric-semantic-based multi- interaction fusion,”IEEE Transactions on Multimedia, 2025

    2025

  8. [8]

    Multi-sensor fusion and cooperative perception for autonomous driving: A review,

    C. Xiang, C. Feng, X. Xie, B. Shi, H. Lu, Y . Lv, M. Yang, and Z. Niu, “Multi-sensor fusion and cooperative perception for autonomous driving: A review,”IEEE Intelligent Transportation Systems Magazine, vol. 15, no. 5, pp. 36–58, 2023

    2023

Show all 64 references
  1. [9]

    Cg-mae: Bev masked autoencoders based on cross-modal guidance for 3d object detection in autonomous driving,

    J. Huo, S. Wang, E. Chen, Y . Ding, and S. Yang, “Cg-mae: Bev masked autoencoders based on cross-modal guidance for 3d object detection in autonomous driving,”Computers, Materials & Continua, pp. 1–21, 2026

    2026

  2. [10]

    Transfusion: Robust lidar-camera fusion for 3d object detection with transformers,

    X. Bai, Z. Hu, X. Zhu, Q. Huang, Y . Chen, H. Fu, and C.-L. Tai, “Transfusion: Robust lidar-camera fusion for 3d object detection with transformers,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1090–1099

    2022

  3. [11]

    Mapfusion: A novel bev feature fusion network for multi-modal map construction,

    X. Hao, Y . Diao, M. Wei, Y . Yang, P. Hao, R. Yin, H. Zhang, W. Li, S. Zhao, and Y . Liu, “Mapfusion: A novel bev feature fusion network for multi-modal map construction,”Information Fusion, vol. 119, p. 103018, 2025. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 ...

    2025

  4. [12]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,

    Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. L. Rus, and S. Han, “Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,” in2023 IEEE international conference on robotics and automation (ICRA). IEEE, 2023, pp. 2774–2781

    2023

  5. [13]

    Mta: Multimodal task alignment for bev perception and captioning,

    Y . Ma, B. Yaman, X. Ye, J. Luo, F. Tao, A. Mallik, Z. Wang, and L. Ren, “Mta: Multimodal task alignment for bev perception and captioning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2026, pp. 670–679

    2026

  6. [14]

    M3net: Multimodal multi-task learning for 3d detection, segmentation, and occupancy prediction in autonomous driving,

    X. Chen, S. Shi, T. Ma, J. Zhou, S. See, K. C. Cheung, and H. Li, “M3net: Multimodal multi-task learning for 3d detection, segmentation, and occupancy prediction in autonomous driving,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 2, 2025, pp. ...

    2025

  7. [15]

    Fuller: Unified multi-modality multi-task 3d perception via multi-level gradient calibration,

    Z. Huang, S. Lin, G. Liu, M. Luo, C. Ye, H. Xu, X. Chang, and X. Liang, “Fuller: Unified multi-modality multi-task 3d perception via multi-level gradient calibration,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3502–3511

    2023

  8. [16]

    M2bev: multi-camera joint 3d detection and seg- mentation with unified birds-eye view representation,

    E. Xie, Z. Yu, D. Zhou, J. Philion, A. Anandkumar, S. Fidler, P. Luo, and J. M. Alvarez, “M2bev: multi-camera joint 3d detection and seg- mentation with unified birds-eye view representation,”arXiv preprint arXiv:2204.05088, 2022

    2022 arXiv

  9. [17]

    Unitr: A unified and efficient multi-modal transformer for bird’s-eye- view representation,

    H. Wang, H. Tang, S. Shi, A. Li, Z. Li, B. Schiele, and L. Wang, “Unitr: A unified and efficient multi-modal transformer for bird’s-eye- view representation,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 6792–6802

    2023

  10. [18]

    Maskbev: Towards a unified framework for bev detection and map seg- mentation,

    X. Zhao, X. Zhang, D. Yang, M. Sun, M. Li, S. Wang, and L. Zhang, “Maskbev: Towards a unified framework for bev detection and map seg- mentation,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 2652–2661

    2024

  11. [19]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 621–11 631

    2020

  12. [20]

    Deep learning for 3d point clouds: A survey,

    Y . Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3d point clouds: A survey,”IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 12, pp. 4338–4364, 2020

    2020

  13. [21]

    Automotive radars: A review of signal processing techniques,

    S. M. Patole, M. Torlak, D. Wang, and M. Ali, “Automotive radars: A review of signal processing techniques,”IEEE Signal Processing Magazine, vol. 34, no. 2, pp. 22–35, 2017

    2017

  14. [22]

    Static multitarget- based autocalibration of rgb cameras, 3-d radar, and 3-d lidar sensors,

    S. Agrawal, S. Bhanderi, K. Doycheva, and G. Elger, “Static multitarget- based autocalibration of rgb cameras, 3-d radar, and 3-d lidar sensors,” IEEE Sensors Journal, vol. 23, no. 18, pp. 21 493–21 505, 2023

    2023

  15. [23]

    High dimensional frustum pointnet for 3d object detection from camera, lidar, and radar,

    L. Wang, T. Chen, C. Anklam, and B. Goldluecke, “High dimensional frustum pointnet for 3d object detection from camera, lidar, and radar,” in2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 1621–1628

    2020

  16. [24]

    Ezfusion: A close look at the integration of lidar, millimeter-wave radar, and camera for accurate 3d object detection and tracking,

    Y . Li, J. Deng, Y . Zhang, J. Ji, H. Li, and Y . Zhang, “Ezfusion: A close look at the integration of lidar, millimeter-wave radar, and camera for accurate 3d object detection and tracking,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 182–11 189, 2022

    2022

  17. [25]

    Camera, lidar, and radar sensor fusion based on bayesian neural network (clr-bnn),

    R. Ravindran, M. J. Santora, and M. M. Jamali, “Camera, lidar, and radar sensor fusion based on bayesian neural network (clr-bnn),”IEEE Sensors Journal, vol. 22, no. 7, pp. 6964–6974, 2022

    2022

  18. [26]

    Mt-detr: Robust end-to-end multimodal detection with confidence fusion,

    S.-Y . Chu and M.-S. Lee, “Mt-detr: Robust end-to-end multimodal detection with confidence fusion,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2023, pp. 5252–5261

    2023

  19. [27]

    Rcm-fusion: Radar-camera multi-level fusion for 3d object detection,

    J. Kim, M. Seong, G. Bang, D. Kum, and J. W. Choi, “Rcm-fusion: Radar-camera multi-level fusion for 3d object detection,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 18 236–18 242

    2024

  20. [28]

    Bridging the view disparity between radar and camera features for multi-modal fusion 3d object detection,

    T. Zhou, J. Chen, Y . Shi, K. Jiang, M. Yang, and D. Yang, “Bridging the view disparity between radar and camera features for multi-modal fusion 3d object detection,”IEEE Transactions on Intelligent Vehicles, vol. 8, no. 2, pp. 1523–1535, 2023

    2023

  21. [29]

    Simple- bev: What really matters for multi-sensor bev perception?

    A. W. Harley, Z. Fang, J. Li, R. Ambrus, and K. Fragkiadaki, “Simple- bev: What really matters for multi-sensor bev perception?” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 2759–2765

    2023

  22. [30]

    Multifusionnet: Spatio-temporal camera-radar fusion in dynamic urban environments,

    A. Hazarika, M. Fotouhi, M. Rahmati, P. Arabshahi, and W. Cheng, “Multifusionnet: Spatio-temporal camera-radar fusion in dynamic urban environments,”IEEE Sensors Journal, 2025

    2025

  23. [31]

    Rcbevdet: Radar-camera fusion in bird’s eye view for 3d object detection,

    Z. Lin, Z. Liu, Z. Xia, X. Wang, Y . Wang, S. Qi, Y . Dong, N. Dong, L. Zhang, and C. Zhu, “Rcbevdet: Radar-camera fusion in bird’s eye view for 3d object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 928–14 937

    2024

  24. [32]

    Eliminating cross-modal conflicts in bev space for lidar-camera 3d object detection,

    J. Fu, C. Gao, Z. Wang, L. Yang, X. Wang, B. Mu, and S. Liu, “Eliminating cross-modal conflicts in bev space for lidar-camera 3d object detection,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 16 381–16 387

    2024

  25. [33]

    Henet: Hybrid encoding for end-to-end multi-task 3d perception from multi-view cameras,

    Z. Xia, Z. Lin, X. Wang, Y . Wang, Y . Xing, S. Qi, N. Dong, and M.-H. Yang, “Henet: Hybrid encoding for end-to-end multi-task 3d perception from multi-view cameras,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 376–392

    2024

  26. [34]

    Adversarial multi-task learning for liver tumor segmentation, dynamic enhancement regression, and classification,

    X. Xiao, Q. V . Hu, T. H. Kim, and G. Wang, “Adversarial multi-task learning for liver tumor segmentation, dynamic enhancement regression, and classification,”arXiv preprint arXiv:2511.20793, 2025

    2025

  27. [35]

    Quadbev: An efficient quadruple-task perception framework via birds’- eye-view representation,

    Y . Li, Y . Li, X. Yang, M. Yu, Z. Huang, X. Wu, and C. Yeo, “Quadbev: An efficient quadruple-task perception framework via birds’- eye-view representation,” in2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2024, pp. 2405– 2412

    2024

  28. [36]

    Msc-bench: Benchmarking and analyzing multi-sensor corrup- tion for driving perception,

    X. Hao, G. Liu, Y . Zhao, Y . Ji, M. Wei, H. Zhao, L. Kong, R. Yin, and Y . Liu, “Msc-bench: Benchmarking and analyzing multi-sensor corrup- tion for driving perception,” in2025 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2025, pp. 1–6

    2025

  29. [37]

    Sgformer: Semantic-geometry fusion transformer for multi-modal 3d panoptic segmentation,

    H. Yu, S. Chan, X. Zhou, and X. Zhang, “Sgformer: Semantic-geometry fusion transformer for multi-modal 3d panoptic segmentation,” inPro- ceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 9, 2025, pp. 9616–9625

    2025

  30. [38]

    Graphbev: Towards robust bev feature alignment for multi-modal 3d object detection,

    Z. Song, L. Yang, S. Xu, L. Liu, D. Xu, C. Jia, F. Jia, and L. Wang, “Graphbev: Towards robust bev feature alignment for multi-modal 3d object detection,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 347–366

    2024

  31. [39]

    Cmgfa: A bev segmentation model based on cross-modal group-mix attention feature aggregator,

    X. Kuang, R. Niu, C. Hua, C. Jiang, H. Zhu, Z. Chen, and B. Yu, “Cmgfa: A bev segmentation model based on cross-modal group-mix attention feature aggregator,”IEEE Robotics and Automation Letters, 2024

    2024

  32. [40]

    Filter-based deep-compression with global average pooling for convolutional net- works,

    T.-Y . Hsiao, Y .-C. Chang, H.-H. Chou, and C.-T. Chiu, “Filter-based deep-compression with global average pooling for convolutional net- works,”Journal of Systems Architecture, vol. 95, pp. 9–18, 2019

    2019

  33. [41]

    Convolution in convolution for network in network,

    Y . Pang, M. Sun, X. Jiang, and X. Li, “Convolution in convolution for network in network,”IEEE transactions on neural networks and learning systems, vol. 29, no. 5, pp. 1587–1597, 2017

    2017

  34. [42]

    Deep learning using rectified linear units (relu),

    A. F. Agarap, “Deep learning using rectified linear units (relu),”arXiv preprint arXiv:1803.08375, 2018

    2018 arXiv

  35. [43]

    Adaptive mixtures of local experts,

    R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,”Neural computation, vol. 3, no. 1, pp. 79–87, 1991

    1991

  36. [44]

    Long-tailed recog- nition by routing diverse distribution-aware experts,

    X. Wang, L. Lian, Z. Miao, Z. Liu, and S. X. Yu, “Long-tailed recog- nition by routing diverse distribution-aware experts,” inInternational Conference on Learning Representations, 2021

    2021

  37. [45]

    Adamv-moe: Adaptive multi-task vision mixture-of-experts,

    T. Chen, X. Chen, X. Du, A. Rashwan, F. Yang, H. Chen, Z. Wang, and Y . Li, “Adamv-moe: Adaptive multi-task vision mixture-of-experts,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17 346–17 357

    2023

  38. [46]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,

    N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” inInternational Conference on Learning Representations, 2017

    2017

  39. [47]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988

    2017

  40. [48]

    X-align: Cross-modal cross-view alignment for bird’s-eye-view segmentation,

    S. Borse, M. Klingner, V . R. Kumar, H. Cai, A. Almuzairee, S. Yoga- mani, and F. Porikli, “X-align: Cross-modal cross-view alignment for bird’s-eye-view segmentation,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 3287–3297

    2023

  41. [49]

    Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers,

    Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Yu, and J. Dai, “Bevformer: learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  42. [50]

    Pointpillars: Fast encoders for object detection from point clouds,

    A. H. Lang, S. V ora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 12 697–12 705

    2019

  43. [51]

    Center-based 3d object detection and tracking,

    T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detection and tracking,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11 784–11 793

    2021

  44. [52]

    Focalformer3d: focusing on hard instance for 3d object JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 12 detection,

    Y . Chen, Z. Yu, Y . Chen, S. Lan, A. Anandkumar, J. Jia, and J. M. Alvarez, “Focalformer3d: focusing on hard instance for 3d object JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 12 detection,” inProceedings of the IEEE/CVF International Conference on Computer Visi...

    2015

  45. [53]

    Safdnet: A simple and effective network for fully sparse 3d object detection,

    G. Zhang, J. Chen, G. Gao, J. Li, S. Liu, and X. Hu, “Safdnet: A simple and effective network for fully sparse 3d object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 477–14 486

    2024

  46. [54]

    Pointpainting: Se- quential fusion for 3d object detection,

    S. V ora, A. H. Lang, B. Helou, and O. Beijbom, “Pointpainting: Se- quential fusion for 3d object detection,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 4604– 4612

    2020

  47. [55]

    Futr3d: A unified sensor fusion framework for 3d detection,

    X. Chen, T. Zhang, Y . Wang, Y . Wang, and H. Zhao, “Futr3d: A unified sensor fusion framework for 3d detection,” inproceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 172–181

    2023

  48. [56]

    Multimodal virtual point 3d detection,

    T. Yin, X. Zhou, and P. Kr ¨ahenb¨uhl, “Multimodal virtual point 3d detection,”Advances in Neural Information Processing Systems, vol. 34, pp. 16 494–16 507, 2021

    2021

  49. [57]

    Mbfusion: A new multi-modal bev feature fusion method for hd map construction,

    X. Hao, H. Zhang, Y . Yang, Y . Zhou, S. Jung, S.-I. Park, and B. Yoo, “Mbfusion: A new multi-modal bev feature fusion method for hd map construction,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 15 922–15 928

    2024

  50. [58]

    Maptr: Structured modeling and learning for online vectorized hd map construction,

    B. Liao, S. Chen, X. Wang, T. Cheng, Q. Zhang, W. Liu, and C. Huang, “Maptr: Structured modeling and learning for online vectorized hd map construction,” inInternational Conference on Learning Representations, 2023

    2023

  51. [59]

    Smab: Simple multimodal attention for effective bev fusion,

    A. Mustajbasic, S. Chen, E. Stenberget al., “Smab: Simple multimodal attention for effective bev fusion,” in2025 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2025, pp. 1766–1772

    2025

  52. [60]

    Henet++: Hybrid encoding and multi-task learning for 3d perception and end-to-end autonomous driving,

    Z. Xia, Z. Lin, Y . Wang, and M.-H. Yang, “Henet++: Hybrid encoding and multi-task learning for 3d perception and end-to-end autonomous driving,”arXiv preprint arXiv:2511.07106, 2025

    2025

  53. [61]

    Unisparsebev: A multi-task learning framework with unified sparse query for autonomous driving,

    H. Zhou, Y . Zhang, and H. Qi, “Unisparsebev: A multi-task learning framework with unified sparse query for autonomous driving,”IEEE Transactions on Circuits and Systems for Video Technology, 2026

    2026

  54. [62]

    Daocc: 3d object detection assisted multi- sensor fusion for 3d occupancy prediction,

    Z. Yang, Y . Dong, J. Wang, H. Wang, L. Ma, Z. Cui, Q. Liu, H. Pei, K. Zhang, and C. Zhang, “Daocc: 3d object detection assisted multi- sensor fusion for 3d occupancy prediction,”IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  55. [63]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inInternational Conference on Learning Representations, 2019

    2019

  56. [64]

    Super-convergence: Very fast training of neural networks using large learning rates,

    L. N. Smith and N. Topin, “Super-convergence: Very fast training of neural networks using large learning rates,” inArtificial intelligence and machine learning for multi-domain operations applications, vol. 11006. SPIE, 2019, pp. 369–386

    2019

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.