Pith. sign in

REVIEW 4 major objections 5 minor 55 references

TEM^3-Learning: Time-Efficient Multimodal Multi-Task Learning for Advanced Assistive Driving

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single lightweight network jointly recognizes driver emotion, driver behavior, traffic context, and vehicle behavior in real time, achieving state-of-the-art accuracy on the AIDE benchmark.

desk verdict Plausible lightweight MTL architecture, but the SOTA claim is unsupported because the two strongest published AIDE baselines are missing from the comparison. read the letter →

arxiv 2506.18084 v1 pith:DJMHWN4O submitted 2025-06-22 cs.CV

Wenzhuo Liu , Yicheng Qiao , Zhen Wang , Qiannan Guo , Zilong Chen , Meihua Zhou , Xinran Li , Letian Wang
show 3 more authors
Zhiwei Li Huaping Liu Wenshuo Wang
This is my paper
classification cs.CV
keywords multi-tasklearningmultimodalfusionassistivedrivingdriveremotionrecognitionbehaviortrafficcontextvehicleMamba
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that multimodal multi-task learning for assistive driving can be both accurate and fast. It proposes TEM3-Learning, a two-stage architecture that extracts temporal-spatial features from multi-view images with a Mamba-based module and adaptively fuses three input modalities for each of four tasks with task-specific gates. On the AIDE dataset, the network reports state-of-the-art accuracy on all four tasks with fewer than 6 million parameters and 142.32 FPS inference, which would make it suitable for real-time ADAS deployment. The ablations attribute the gains to the MTS-Mamba and MGMI components and to joint training across driver-state and traffic-environment tasks.

What carries the argument

The central machinery is the two-stage feature pipeline. The first stage, MTS-Mamba, takes multi-view sequential images (front/left/right exterior, interior, driver face/body), concatenates the 16 frames per view, and applies a state-space model with forward and backward scanning to capture bidirectional temporal dependencies, plus local (average-pool) and global (adaptive-pool) spatial branches whose outputs are combined under a temporal weight $W_{ssm}$ with a residual connection. The second stage, MGMI, concatenates the MTS-Mamba outputs for exterior and interior images with a 3D CNN's joint features, forms task-shared features via self-attention, and then applies four task-specific gating units (convolution + batch norm + sigmoid) that compute weighted sums of the three modality features per task. The gating mechanism is the load-bearing element that lets each task emphasize its most relevant modality, addressing negative transfer.

What would settle it

Running the released code on the AIDE test split with the same preprocessing and comparing per-task accuracies to the reported DER 75.00, DBR 69.31, TCR 96.29, VBR 86.11 ($\beta_{macc}$ 81.68) would settle the claim; if a faithful re-implementation falls outside a reasonable margin (for example, more than 1-2 points) or does not beat the best baseline under identical conditions, the central claim fails.

Watch

Extended reading notes

Core claim

The paper claims that a multimodal, multi-task network can outperform all previous single-modality or independently-fused methods on the AIDE assistive-driving benchmark while remaining fast enough for real-time use: with fewer than 6 million parameters, it reports a mean accuracy $\beta_{macc}$ of 81.68%, an improvement of 3.48%-9.32% over the compared state-of-the-art models, and an inference speed of 142.32 FPS. This result is attributed to MTS-Mamba's forward-backward temporal scanning plus global-local spatial attention, which extracts temporal-spatial features cheaply from multi-view sequential images, and to MGMI's task-specific gating, which reweights the three input modalities (vehicle-exterior images, vehicle-interior images, and driver joints) for each of the four tasks and thereby mitigates negative transfer.

Load-bearing premise

The reported accuracy and speed gains assume that the comparison baselines were trained and evaluated under the exact same protocol (data split, input size, preprocessing, and hardware), so the differences reflect the architecture rather than implementation choices.

Editorial extensions

If this is right

  • If the reported results hold, a single model can serve four ADAS recognition tasks simultaneously, simplifying onboard perception stacks.
  • The roughly 6-million-parameter count and 142.32 FPS inference suggest the model can run under real-time constraints typical of assistive-driving systems.
  • The MGMI gating design implies that modality weights can be adapted per task during inference, so tasks needing different sensors (TCR using exterior images, DER using interior images and joints) can share one backbone without performance loss.
  • The ablation showing that joint training across driver-state and traffic-environment tasks improves both groups implies that assistive-driving perception benefits from coupling these task families rather than solving them separately.
  • The pattern of shared feature extraction across similar modalities may generalize to other multimodal multi-task learning problems where parameter efficiency matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The forward-backward scanning in MTS-Mamba is a simplified bidirectional state-space model; a natural extension would be to test whether full selective-scan mechanisms from more recent Mamba variants yield further gains at equal parameter count.
  • Because the gating weights are learned per task, the model could be used as a probe to quantify how much each modality contributes to each task, and analyzing gate activations on AIDE may reveal stable modality-task couplings that could guide sensor selection in future ADAS designs.
  • The reported FPS was measured on an L40S GPU; the parameter efficiency suggests the network may also run on embedded automotive hardware with quantization or pruning, but that is a testable hypothesis the paper does not verify.
  • The decision to share one MTS-Mamba across multiple views rather than using separate backbones is the main source of parameter savings, and this design principle could transfer to other multi-view video understanding tasks beyond driving.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. TEM^3-Learning proposes a multimodal multi-task architecture for four assistive-driving recognition tasks on the AIDE dataset: driver emotion recognition, driver behavior recognition, traffic context recognition, and vehicle behavior recognition. The framework combines a Mamba-based temporal-spatial feature extractor (MTS-Mamba) for multi-view image sequences and a gated multimodal feature integrator (MGMI) that uses self-attention and task-specific gating to fuse image and joint features. The paper claims state-of-the-art accuracy on all four tasks, fewer than 6 million parameters, and 142.32 FPS inference speed, with ablations supporting each module.

Significance. If the reported results are reproducible and the comparison is complete, the paper would demonstrate a lightweight and real-time-capable multimodal multi-task architecture, a valuable contribution for ADAS applications. The ablations are well structured and appear to isolate the contributions of MTS-Mamba and MGMI, including the temporal scanning, global-local spatial extraction, self-attention, and multi-gating components, which is a strength. However, the central state-of-the-art claim depends on a comparison table that omits the two most relevant AIDE baselines, and the efficiency metrics lack a described measurement protocol. These issues must be addressed before the significance claims can be accepted.

major comments (4)
  1. [Section IV-E, Table I] The claim of state-of-the-art accuracy on all four AIDE tasks is not supported by the comparison in Table I because the two most relevant published methods, UMD-Net [30] and MMTL-UNIAD [31], both by the authors' group and both listed in the bibliography, are neither cited in the body nor included in the table. Without their β_macc results on the same AIDE split, the reader cannot verify that the proposed model outperforms the strongest baselines. The authors should add these methods to Table I and discuss the comparison, or explicitly justify their exclusion.
  2. [Section IV-E, FPS column] The paper reports 142.32 FPS for the proposed model and includes FPS values for some baselines, but it does not state how these numbers were measured. It is unclear whether all models were timed on the same L40S GPU with the same batch size, input resolution, precision, and whether preprocessing and I/O are included. Several baselines (e.g., the ResNet/TransE rows, I3D, SlowFast, TimeSFormer, Video Swin) have no FPS entry. Please provide a complete measurement protocol and report FPS for all baselines under identical conditions; if some FPS values are taken from the original papers, state this and explain hardware differences.
  3. [Section III-C, Equation (6)] Equation (6) is ambiguous about tensor shapes. The attention output softmax(Q·K^T/√d)·V is described as being reshaped to R^{C×H×W}, but the gating argument BN(Conv2D(...)) seems to be applied to this attention output, while each H_i is elementwise multiplied by the gate. It is not clear how the per-modality gates σ_i^r are derived from the task-shared features, nor what the dimensions of Q, K, V, and the attention weights are after the 3×1 convolutions shown in Figure 4. Please define all tensor shapes and the exact flow from the attention output to the three gates, so that MGMI can be reproduced unambiguously.
  4. [Section IV-A and all result tables] The experiments report single-run accuracy with no error bars, repeated trials, or statistical significance testing. Given the small dataset size (2,898 samples) and the modest claimed improvements (e.g., a β_macc improvement of 3.48%-9.32%), the absence of variance estimates makes it difficult to assess whether the differences are meaningful. Please report results over multiple random seeds, or at least discuss training stability and provide mean±std for the reported metrics.
minor comments (5)
  1. [Section IV-A] The dataset split is described as 'training, testing, and validation sets with proportions of 65%, 15%, and 20%, respectively,' which is unusual order and sums to 100% but does not clarify which set is the held-out test set. Please specify the exact split (e.g., train/val/test) and whether the reported results are on the test or validation set.
  2. [Section III-B, Equation (3)] The state-space equations (2)-(3) use B and C as shared state parameters but the matrix dimensions of A, d_state, and d_dim are not fully specified. In particular, it is unclear what 'state unit vector' and 'channel unit vector' mean in the context of the forward/backward scanning. Please clarify the mathematical definitions.
  3. [Section III-C] The text says H_1, H_2 ∈ R^{C×H×W} are extracted by MTS-Mamba, but there are two MTS-Mamba branches—one for vehicle-exterior and one for vehicle-interior images. Please explicitly state that H_1 and H_2 are the outputs of the two MTS-Mamba branches, and define the output shape of the 3D CNN branch H_3.
  4. [Table I] The 'Pattern' column in Table I is not explained in the text. Please define what the pattern labels (e.g., 2D, 2D+Timing, 3D) mean and how they relate to the architecture descriptions.
  5. [General] Some formatting issues: Table VI uses '✓' symbols while the text uses 'w/' and 'w/o' notation; the reference list contains entries [3], [12], [14], [15], [23], [26], [29], [40], [42], [46], and [54] that are self-citations or related works not directly discussed in the body. Please check that all numbered references are cited in the text.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the accuracy/FPS claims are empirical benchmark measurements on AIDE; one comparison-completeness gap involving the authors' own prior AIDE methods prevents a clean SOTA verdict but is not a definitional circularity.

full rationale

The paper's central quantitative claims are benchmark results, not first-principles derivations. Equations (1)-(7) define the total loss, SSM weight update, temporal-spatial feature output, MGMI fusion, and beta_macc metric; none of these defines a target quantity in terms of the quantity it is supposed to predict, and no fitted parameter is renamed as a prediction. MTS-Mamba and MGMI are new architectures whose contributions are tested by ablations against held-out AIDE labels (Tables II-VI), i.e., by external, falsifiable measurements rather than by construction. The numerous self-citations (refs [3],[10],[12],[14],[23],[26],[28]-[31],[40],[42],[46],[54]) are mostly related-work context; the load-bearing methodological citations (Mamba [16], AIDE [51], MMoE [34]) are independent. The one flagged gap is Section IV-E/Table I: the 'state-of-the-art' comparison follows the setup of [51] and omits UMD-Net [30] and MMTL-UNIAD [31], two same-group AIDE methods present in the bibliography, so the SOTA conclusion is not fully established by the table; this is a missing-comparison/correctness issue, not a circular reduction. Accordingly, circularity is low (2/10) rather than zero because of the density of self-referential reporting in the SOTA claim, but the core empirical result remains independent of its inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim, that the architecture achieves SOTA on AIDE, rests on the correctness of the AIDE annotations, the effectiveness of Mamba and MMoE-style gating, and the assumption that baseline comparisons are fair. The model itself has hand-chosen hyperparameters (state dimension n, scaling gamma) that are not specified.

free parameters (2)
  • Mamba state dimension n = not reported
    In Eq. (3), the state transition matrix A in R^{C x n} and the SSM state dimension n control the temporal modeling capacity; n is a hand-chosen hyperparameter that is never specified.
  • Scaling factor gamma = not reported
    In Eq. (4), gamma controls the residual contribution of the temporal-spatial features; its value is not given and is presumably tuned.
assumptions (4)
  • domain assumption The AIDE dataset provides reliable ground-truth labels for the four tasks
    All experiments and claims of SOTA rest on the correctness and representativeness of the AIDE annotations (Section IV-A).
  • domain assumption Mamba selective state spaces are an effective and efficient temporal feature extractor
    The design of MTS-Mamba assumes the SSM machinery from Gu and Dao (2023) will capture temporal dependencies better than CNNs or full attention, which is an empirical bet.
  • domain assumption Multi-gate mixture-of-experts style gating reduces negative transfer
    MGMI inherits the MMoE assumption (Ma et al., 2018) that task-specific gating can decouple conflicting tasks; this is a modeling assumption, not a proven guarantee.
  • domain assumption Input modalities are temporally synchronized at 16 FPS
    The preprocessing section states synchronization, and the 16-frame sequences assume this alignment is accurate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TEM^3-Learning: Time-Efficient Multimodal Multi-Task Learning for Advanced Assistive Driving." pith.science (2026). https://pith.science/paper/DJMHWN4O

@misc{pith2026250618084,
  author       = {Pith},
  title        = {Pith review of: TEM^3-Learning: Time-Efficient Multimodal Multi-Task Learning for Advanced Assistive Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJMHWN4O}},
  note         = {Machine review of arXiv:2506.18084}
}
read the original abstract

Multi-task learning (MTL) can advance assistive driving by exploring inter-task correlations through shared representations. However, existing methods face two critical limitations: single-modality constraints limiting comprehensive scene understanding and inefficient architectures impeding real-time deployment. This paper proposes TEM^3-Learning (Time-Efficient Multimodal Multi-task Learning), a novel framework that jointly optimizes driver emotion recognition, driver behavior recognition, traffic context recognition, and vehicle behavior recognition through a two-stage architecture. The first component, the mamba-based multi-view temporal-spatial feature extraction subnetwork (MTS-Mamba), introduces a forward-backward temporal scanning mechanism and global-local spatial attention to efficiently extract low-cost temporal-spatial features from multi-view sequential images. The second component, the MTL-based gated multimodal feature integrator (MGMI), employs task-specific multi-gating modules to adaptively highlight the most relevant modality features for each task, effectively alleviating the negative transfer problem in MTL. Evaluation on the AIDE dataset, our proposed model achieves state-of-the-art accuracy across all four tasks, maintaining a lightweight architecture with fewer than 6 million parameters and delivering an impressive 142.32 FPS inference speed. Rigorous ablation studies further validate the effectiveness of the proposed framework and the independent contributions of each module. The code is available on https://github.com/Wenzhuo-Liu/TEM3-Learning.

Figures

Figures reproduced from arXiv: 2506.18084 by the authors.

Figure 1
Figure 1. Comparison of four mainstream algorithm frameworks: (a) single [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall pipeline of TEM3 -Learning. MTS-Mamba and 3D CNN are used to extract multimodal features from vehicle-exterior images, vehicle-interior images, and driver joints, respectively. The multi￾task learning-based gated multimodal feature integrator (MGMI) adaptively fuses these features, enabling multi-task recognition. III. METHODOLOGY This section presents the overall structure and key modules of the propose… view at source ↗
Figure 3
Figure 3. The structure diagram of MTS-Mamba, which includes the forward and backward scanning mechanisms, as well as the global-local spatial feature [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Structure of the multi-task learning-based gated multimodal feature [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 50 canonical work pages

  1. [30]

    Umd-net: A unified multi-task assistive driving network based on multimodal fusion.IEEE Transactions on Intelligent Transportation Systems, 2025

    Wenzhuo Liu, Yicheng Qiao, Zhiwei Li, Wenshuo Wang, Wei Zhang, Jiayin Zhu, Yanhuan Jiang, Li Wang, Hong Wang, Huaping Liu, et al. Umd-net: A unified multi-task assistive driving network based on multimodal fusion.IEEE Transactions on Intelligent Transportation Systems, 2025

    work page 2025

  2. [31]

    Mmtl-uniad: A unified framework for multimodal and multi-task learning in assistive driving perception

    Wenzhuo Liu, Wenshuo Wang, Yicheng Qiao, Qiannan Guo, Jiayin Zhu, Pengfei Li, Zilong Chen, Huiming Yang, Zhiwei Li, Lening Wang, et al. Mmtl-uniad: A unified framework for multimodal and multi-task learning in assistive driving perception. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 6864–6874, 2025

    work page 2025

  3. [1]

    Emerging trends in autonomous vehicle perception: Multimodal fusion for 3d object detection.World Electric Vehicle Journal, 15(1):20, 2024

    Simegnew Yihunie Alaba, Ali C Gurbuz, and John E Ball. Emerging trends in autonomous vehicle perception: Multimodal fusion for 3d object detection.World Electric Vehicle Journal, 15(1):20, 2024

    work page 2024

  4. [2]

    Is space-time attention all you need for video understanding? InInternational Conference on Machine Learnin (ICML), page 4, 2021

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? InInternational Conference on Machine Learnin (ICML), page 4, 2021

    work page 2021

  5. [3]

    Han Bi, Ge Yu, Yu He, Wenzhuo Liu, and Zijie Zheng. Vm-bhinet: Vision mamba bimanual hand interaction network for 3d interacting hand mesh recovery from a single rgb image.Proceedings of the ACM on Computer Graphics and Interactive Techniques, 8(1):1–16, 2025

    work page 2025

  6. [4]

    Relational multi-task learning: Modeling relations between data and tasks.arXiv preprint arXiv:2303.07666, 2023

    Kaidi Cao, Jiaxuan You, and Jure Leskovec. Relational multi-task learning: Modeling relations between data and tasks.arXiv preprint arXiv:2303.07666, 2023

    arXiv 2023

  7. [5]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. Inproceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6299–6308, 2017

    work page 2017

  8. [6]

    Umt-net: A uniform multi-task network with adaptive task weighting.IEEE Transactions on Intelligent Vehicles, 9(1):2304– 2317, 2023

    Sihan Chen, Lianqing Zheng, Libo Huang, Jie Bai, Xichan Zhu, and Zhixiong Ma. Umt-net: A uniform multi-task network with adaptive task weighting.IEEE Transactions on Intelligent Vehicles, 9(1):2304– 2317, 2023

    work page 2023

Show all 55 references
  1. [7]

    Adamv- moe: Adaptive multi-task vision mixture-of-experts

    Tianlong Chen, Xuxi Chen, Xianzhi Du, Abdullah Rashwan, Fan Yang, Huizhong Chen, Zhangyang Wang, and Yeqing Li. Adamv- moe: Adaptive multi-task vision mixture-of-experts. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17346–17357, 2023

    2023

  2. [8]

    Multi-task learning for real-time autonomous driving leveraging task-adaptive attention generator

    Wonhyeok Choi, Mingyu Shin, Hyukzae Lee, Jaehoon Cho, Jaehyeon Park, and Sunghoon Im. Multi-task learning for real-time autonomous driving leveraging task-adaptive attention generator. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14732–14739. I...

    2024

  3. [9]

    Multinet: Multi-modal multi-task learning for autonomous driving

    Sauhaarda Chowdhuri, Tushar Pankaj, and Karl Zipser. Multinet: Multi-modal multi-task learning for autonomous driving. In2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1496–1504. IEEE, 2019

    2019

  4. [10]

    Textnerf: A novel scene-text image synthesis method based on neural radiance fields

    Jialei Cui, Jianwei Du, Wenzhuo Liu, and Zhouhui Lian. Textnerf: A novel scene-text image synthesis method based on neural radiance fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22272–22281, 2024

    2024

  5. [11]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6202–6211, 2019

    2019

  6. [12]

    A segmentation method based on boundary fracture correction for froth scale measurement.Applied Intelligence, pages 1–22, 2024

    Yongqi Gan, Wenzhuo Liu, Jianwang Gan, and Guoying Zhang. A segmentation method based on boundary fracture correction for froth scale measurement.Applied Intelligence, pages 1–22, 2024

    2024

  7. [13]

    Enhanced multi-task learning and knowledge graph-based recommender system.IEEE Transactions on Knowledge and Data Engineering, 35(10):10281–10294, 2023

    Min Gao, Jian-Yu Li, Chun-Hua Chen, Yun Li, Jun Zhang, and Zhi- Hui Zhan. Enhanced multi-task learning and knowledge graph-based recommender system.IEEE Transactions on Knowledge and Data Engineering, 35(10):10281–10294, 2023

    2023

  8. [14]

    Sifdrivenet: Speed and image fusion for driving behavior classification network.IEEE Transactions on Computational Social Systems, 2023

    Yan Gong, Jianli Lu, Wenzhuo Liu, Zhiwei Li, Xinmin Jiang, Xin Gao, and Xingang Wu. Sifdrivenet: Speed and image fusion for driving behavior classification network.IEEE Transactions on Computational Social Systems, 2023

    2023

  9. [15]

    Multi-modal fusion technology based on vehicle information: A survey.arXiv preprint arXiv:2211.06080, 2022

    Yan Gong, Jianli Lu, Jiayi Wu, and Wenzhuo Liu. Multi-modal fusion technology based on vehicle information: A survey.arXiv preprint arXiv:2211.06080, 2022

    2022 arXiv

  10. [16]

    Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  11. [17]

    Temporal information fusion network for driving behavior prediction

    Chenghao Guo, Haizhuang Liu, Jiansheng Chen, and Huimin Ma. Temporal information fusion network for driving behavior prediction. IEEE Transactions on Intelligent Transportation Systems, 24(9):9415– 9424, 2023

    2023

  12. [18]

    Cmt: Convolutional neural networks meet vision transformers

    Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. Cmt: Convolutional neural networks meet vision transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12175– 12185, June 2022

    2022

  13. [19]

    Coarse to fine-based image–point cloud fusion network for 3d object detection.Information Fusion, 112:102551, 2024

    Meilan Hao, Zhongkang Zhang, Lei Li, Kejian Dong, Long Cheng, Prayag Tiwari, and Xin Ning. Coarse to fine-based image–point cloud fusion network for 3d object detection.Information Fusion, 112:102551, 2024

    2024

  14. [20]

    Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotem- poral 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6546–6555, 2018

    2018

  15. [21]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016

    2016

  16. [22]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications.arXiv preprint arXiv:1704.04861, 2017

    Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications.arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  17. [23]

    Mfe-ssnet: Multi-modal fusion-based end-to-end steering angle and vehicle speed prediction network.Automotive Innovation, pages 1–14, 2024

    Yi Huang, Wenzhuo Liu, Yaoyu Li, Lei Yang, Hanqi Jiang, Zhiwei Li, and Jun Li. Mfe-ssnet: Multi-modal fusion-based end-to-end steering angle and vehicle speed prediction network.Automotive Innovation, pages 1–14, 2024

    2024

  18. [24]

    Multi-task learning with attention for end-to-end autonomous driving

    Keishi Ishihara, Anssi Kanervisto, Jun Miura, and Ville Hautamaki. Multi-task learning with attention for end-to-end autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2902–2911, 2021

    2021

  19. [25]

    Knowledge distillation for multi- task learning

    Wei-Hong Li and Hakan Bilen. Knowledge distillation for multi- task learning. InComputer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pages 163–176. Springer, 2020

    2020

  20. [26]

    Mipd: A multi-sensory interactive perception dataset for embodied intelligent driving.arXiv preprint arXiv:2411.05881, 2024

    Zhiwei Li, Tingzhen Zhang, Meihua Zhou, Dandan Tang, Pengwei Zhang, Wenzhuo Liu, Qiaoning Yang, Tianyu Shen, Kunfeng Wang, and Huaping Liu. Mipd: A multi-sensory interactive perception dataset for embodied intelligent driving.arXiv preprint arXiv:2411.05881, 2024

    2024 arXiv

  21. [27]

    Loss-balanced task weighting to reduce negative transfer in multi-task learning

    Shengchao Liu, Yingyu Liang, and Anthony Gitter. Loss-balanced task weighting to reduce negative transfer in multi-task learning. InProceedings of the AAAI conference on artificial intelligence, volume 33, pages 9977–9978, 2019

    2019

  22. [28]

    Glmdrivenet: Global–local multimodal fusion driving behavior classification network.Engineering Applications of Artificial Intelligence, 129:107575, 2024

    Wenzhuo Liu, Yan Gong, Guoying Zhang, Jianli Lu, Yunlai Zhou, and Junbin Liao. Glmdrivenet: Global–local multimodal fusion driving behavior classification network.Engineering Applications of Artificial Intelligence, 129:107575, 2024

    2024

  23. [29]

    Fmdnet: Feature-attention- embedding-based multimodal-fusion driving-behavior-classification network.IEEE Transactions on Computational Social Systems, 2024

    Wenzhuo Liu, Jianli Lu, Junbin Liao, Yicheng Qiao, Guoying Zhang, Jiayin Zhu, Bozhang Xu, and Zhiwei Li. Fmdnet: Feature-attention- embedding-based multimodal-fusion driving-behavior-classification network.IEEE Transactions on Computational Social Systems, 2024

    2024

  24. [32]

    Video swin transformer

    Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3202– 3211, 2022

    2022

  25. [33]

    Epnet++: Cascade bi-directional fusion for multi-modal 3d object detection.IEEE transactions on pattern analysis and machine intelligence, 45(7):8324–8341, 2022

    Zhe Liu, Tengteng Huang, Bingling Li, Xiwu Chen, Xi Wang, and Xiang Bai. Epnet++: Cascade bi-directional fusion for multi-modal 3d object detection.IEEE transactions on pattern analysis and machine intelligence, 45(7):8324–8341, 2022

    2022

  26. [34]

    Modeling task relationships in multi-task learning with multi- gate mixture-of-experts

    Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. Modeling task relationships in multi-task learning with multi- gate mixture-of-experts. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1930–1939, 2018

    1930

  27. [35]

    Shuf- flenet v2: Practical guidelines for efficient cnn architecture design

    Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shuf- flenet v2: Practical guidelines for efficient cnn architecture design. In European Conference on Computer Vision (ECCV), pages 116–131, 2018

    2018

  28. [36]

    Drive&act: A multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles

    Manuel Martin, Alina Roitberg, Monica Haurilet, Matthias Horne, Simon Reiß, Michael V oit, and Rainer Stiefelhagen. Drive&act: A multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles. InProceedings of the IEEE/CVF International Conference on C...

    2019

  29. [37]

    Driver emotion recognition with a hybrid attentional multimodal fusion framework.IEEE Transactions on Affective Computing, 14(4):2970–2981, 2023

    Luntian Mou, Yiyuan Zhao, Chao Zhou, Bahareh Nakisa, Moham- mad Naim Rastgoo, Lei Ma, Tiejun Huang, Baocai Yin, Ramesh Jain, and Wen Gao. Driver emotion recognition with a hybrid attentional multimodal fusion framework.IEEE Transactions on Affective Computing, 14(4):2970–2981, 2023

    2023

  30. [38]

    Dlt-net: Joint detection of drivable areas, lane lines, and traffic objects.IEEE Transactions on Intelligent Transportation Systems, 21(11):4670–4679, 2019

    Yeqiang Qian, John M Dolan, and Ming Yang. Dlt-net: Joint detection of drivable areas, lane lines, and traffic objects.IEEE Transactions on Intelligent Transportation Systems, 21(11):4670–4679, 2019

    2019

  31. [39]

    Mobilenetv2: Inverted residuals and linear bottlenecks

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4510–4520, 2018

    2018

  32. [40]

    Bssnet: A real-time semantic segmentation network for road scenes inspired from autoencoder.IEEE Transactions on Circuits and Systems for Video Technology, 2023

    Xiaoqiang Shi, Zhenyu Yin, Guangjie Han, Wenzhuo Liu, Li Qin, Yuanguo Bi, and Shurui Li. Bssnet: A real-time semantic segmentation network for road scenes inspired from autoencoder.IEEE Transactions on Circuits and Systems for Video Technology, 2023

    2023

  33. [41]

    Very deep convolu- tional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    Karen Simonyan and Andrew Zisserman. Very deep convolu- tional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  34. [42]

    Graph-based target association for multi-drone collaborative perception under imperfect detection conditions.Drones, 9(4):300, 2025

    Qifan Tan, Xuqi Yang, Cheng Qiu, Wenzhuo Liu, Yize Li, Zhengxia Zou, and Jing Huang. Graph-based target association for multi-drone collaborative perception under imperfect detection conditions.Drones, 9(4):300, 2025

    2025

  35. [43]

    Learning spatiotemporal features with 3d convo- lutional networks

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convo- lutional networks. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4489–4497, 2015

    2015

  36. [44]

    Attention is all you need.Advances in Neural Information Processing Systems (NIPS), 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in Neural Information Processing Systems (NIPS), 30, 2017

    2017

  37. [45]

    Openoc- cupancy: A large scale benchmark for surrounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xingang Wang. Openoc- cupancy: A large scale benchmark for surrounding semantic occupancy perception. InProceedings of the IEEE/CVF International Conference on Computer Vision, ...

    2023

  38. [46]

    Path planning for air- ground robot considering modal switching point optimization

    Xiaoyu Wang, Kangyao Huang, Xinyu Zhang, Honglin Sun, Wenzhuo Liu, Huaping Liu, Jun Li, and Pingping Lu. Path planning for air- ground robot considering modal switching point optimization. In2023 International Conference on Unmanned Aircraft Systems (ICUAS), pages 87–94. IEEE, 2023

    2023

  39. [47]

    Yolop: You only look once for panoptic driving perception.Machine Intelligence Research, 19(6):550–562, 2022

    Dong Wu, Man-Wen Liao, Wei-Tian Zhang, Xing-Gang Wang, Xiang Bai, Wen-Qing Cheng, and Wen-Yu Liu. Yolop: You only look once for panoptic driving perception.Machine Intelligence Research, 19(6):550–562, 2022

    2022

  40. [48]

    Yang Xing, Chen Lv, Dongpu Cao, and Efstathios Velenis. Multi-scale driver behavior modeling based on deep spatial-temporal representa- tion for intelligent vehicles.Transportation research part C: emerging technologies, 130:103288, 2021

    2021

  41. [49]

    Fusionpainting: Multimodal fusion with adaptive attention for 3d object detection

    Shaoqing Xu, Dingfu Zhou, Jin Fang, Junbo Yin, Zhou Bin, and Liangjun Zhang. Fusionpainting: Multimodal fusion with adaptive attention for 3d object detection. In2021 IEEE International Intel- ligent Transportation Systems Conference (ITSC), pages 3047–3054. IEEE, 2021

    2021

  42. [50]

    Cross-task knowledge distillation in multi-task recommendation

    Chenxiao Yang, Junwei Pan, Xiaofeng Gao, Tingyu Jiang, Dapeng Liu, and Guihai Chen. Cross-task knowledge distillation in multi-task recommendation. InProceedings of the AAAI conference on artificial intelligence, volume 36, pages 4318–4326, 2022

    2022

  43. [51]

    Aide: A vision-driven multi-view, multi-modal, multi-tasking dataset for assistive driving perception

    Dingkang Yang, Shuai Huang, Zhi Xu, Zhenpeng Li, Shunli Wang, Mingcheng Li, Yuzheng Wang, Yang Liu, Kun Yang, Zhaoyu Chen, et al. Aide: A vision-driven multi-view, multi-modal, multi-tasking dataset for assistive driving perception. InProceedings of the IEEE/CVF International ...

    2023

  44. [52]

    Yolopx: Anchor-free multi-task learning network for panoptic driving perception.Pattern Recognition, 148:110152, 2024

    Jiao Zhan, Yarong Luo, Chi Guo, Yejun Wu, Jiawei Meng, and Jingnan Liu. Yolopx: Anchor-free multi-task learning network for panoptic driving perception.Pattern Recognition, 148:110152, 2024

    2024

  45. [53]

    Shuf- flenet: An extremely efficient convolutional neural network for mobile devices

    Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shuf- flenet: An extremely efficient convolutional neural network for mobile devices. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6848–6856, 2018

    2018

  46. [54]

    Oblique convolution: A novel convolution idea for redefining lane detection.IEEE Transactions on Intelligent Vehicles, 2023

    Xinyu Zhang, Yan Gong, Jianli Lu, Zhiwei Li, Shixiang Li, Shu Wang, Wenzhuo Liu, Li Wang, and Jun Li. Oblique convolution: A novel convolution idea for redefining lane detection.IEEE Transactions on Intelligent Vehicles, 2023

    2023

  47. [55]

    Driving behavior prediction considering cognitive prior and driving context.IEEE Transactions on Intelligent Trans- portation Systems, 22(5):2669–2678, 2020

    Dong Zhou, Hongyi Liu, Huimin Ma, Xiang Wang, Xiaoqin Zhang, and Yuhan Dong. Driving behavior prediction considering cognitive prior and driving context.IEEE Transactions on Intelligent Trans- portation Systems, 22(5):2669–2678, 2020

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.