Pith. sign in

REVIEW 5 major objections 4 minor 53 references

On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance

T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that fusing three top-down guidance factors—driver intention, semantic context, and traffic rules—with bottom-up visual features sets a new state of the art for on-road object importance estimation, and introduces the…

desk verdict The TOI dataset is a real contribution, but the model's central adaptive traffic-rule gate is non-differentiable as written, so the headline 20-23% AP gains shouldn't be trusted until the training story is fixed. read the letter →

arxiv 2411.17152 v1 pith:T3H5NZRX submitted 2024-11-26 cs.RO cs.CV

Zhixiong Nan , Yilong Chen , Tianfei Zhou , Tao Xiang This is my paper · ORCID
classification cs.ROcs.CV
keywords on-roadobjectimportanceestimationtop-downguidancedriverintentionsemanticcontexttrafficrulesobject-laneinteractionTOIdatasetautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles the question of which objects in a driving scene deserve the driver's attention. It does two things: it releases a larger public dataset for the task, named TOI, and it proposes a model that combines three top-down cues—driver intention, semantic context, and traffic rules—with bottom-up visual features. The authors argue that existing methods use at most a single top-down cue, which is insufficient for dynamic and diverse traffic. They report that their model outperforms prior state-of-the-art methods by 23.1% Average Precision on the existing public benchmark and by 20.0% AP on their new dataset. If correct, this establishes a new benchmark for the task and evidence that multi-fold top-down guidance improves importance estimation.

What carries the argument

The load-bearing mechanism is the multi-fold top-down guidance architecture, specifically its two novel modules. DISG (Driver Intention and Semantics Guidance) multiplies semantic segmentation features by fixed intention masks and refines object spatial features through multi-head cross-attention. TRG (Traffic Rule Guidance) converts lane-marking coordinates into lane features, applies cross-attention to object temporal features, and then adaptively penalizes weak object-lane interactions with a learned gating coefficient. That adaptive weighting is credited with the single largest performance gain, a 30.4% AP improvement.

What would settle it

Using the same trained model, replace the lane detector's outputs on TOI with oracle lane annotations; a large AP drop would show the claimed traffic-rule gains depend on upstream lane detection accuracy, while a small gap would show the mechanism is robust.

Watch

Extended reading notes

Core claim

The paper's central claim is that on-road object importance estimation requires fusing driver intention, semantic context, and traffic-rule guidance simultaneously with bottom-up object appearance and motion features, and it introduces the first model to do so. The DISG module uses semantic segmentation maps and hand-designed intention masks (for turning left, going straight, and turning right) to refine object spatial features through cross-attention. The TRG module represents traffic rules via lane-marking coordinates and an adaptive object-lane interaction weighting that suppresses weak lane-object relations. On the TOI dataset—9,858 frames, 28 scenes, and 44,120 object annotations built by re-annotating an established autonomous-driving benchmark—the model reaches 60 AP versus 50 for the strongest baseline, and on the previous public dataset it improves AP by 23.1% over the second-best method.

Load-bearing premise

The traffic-rule pathway treats lane markings detected by a pretrained lane detector as a reliable and complete proxy for traffic rules; when lanes are missed or falsely detected, the importance estimates shift accordingly, as the paper's own failure examples show.

Editorial extensions

If this is right

  • Traffic-rule modeling should become a standard component in importance and risk estimation systems, rather than being limited to visual saliency.
  • The TOI dataset provides a public, roughly three-times-larger benchmark, enabling more complex models to be trained and compared on this task.
  • The reported gains of 23.1% AP and 20.0% AP suggest that combining multiple top-down factors is more effective than any single one.
  • The adaptive object-lane interaction demonstrates that not all objects are equally affected by lanes, so the model learns when to disable that cue.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported margin partly depends on the lane detector's accuracy; the paper's own failure examples show that missing or false lane markings shift importance estimates, so degrading lane detection will likely reduce the TRG benefit.
  • The three hand-coded intention masks are a coarse discretization of driver intent, and a finer or learnable intention representation might yield further gains, though the authors note a learnable mask did not converge.
  • Because object bounding boxes are assumed known, the model's practical end-to-end performance will also depend on upstream detection quality.
  • Because TOI retains extra sensor labels from the underlying benchmark, it could support future multimodal importance-estimation research beyond the RGB-video setting explored here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces TOI, a new dataset for on-road object importance estimation, built by re-annotating KITTI with 9,858 frames, 28 scenes, and 44,120 object-level importance annotations, and proposes a model that combines bottom-up object features with three top-down guidance factors: driver intention, semantic context, and traffic rules. The model consists of an Object Feature Extraction (OFE) module, a Driver Intention and Semantics Guidance (DISG) module, a Traffic Rule Guidance (TRG) module, and an Object Importance Estimation module. Experiments on TOI and the Ohn-Bar dataset report large improvements over seven baselines, including 23.1% and 20.0% Average Precision gains over the Goal method, and ablations attribute the gains to the DISG and TRG modules. The paper claims to be the first on-road object importance estimation model that fuses multi-fold top-down guidance with bottom-up features.

Significance. If the claims are validated, the TOI dataset could be a useful community resource, and the idea of explicitly integrating driver intention, semantic context, and traffic rules into object importance estimation is a plausible step beyond single-fold guidance methods. The annotation procedure with double-checking and triple-discussion is a strength, and the paper provides detailed architecture and implementation descriptions, as well as explicit discussion of limitations. However, the central quantitative claims are not currently supported: the proposed adaptive weighting in TRG is non-differentiable as written and cannot learn the behavior attributed to it, hyperparameters appear to be selected on the test set, all results are single-run point estimates without error bars, and the comparison may be affected by the assumption of ground-truth bounding boxes. These issues are load-bearing for the claimed state-of-the-art results, so the paper requires substantial revision before the findings can be accepted.

major comments (5)
  1. [Section 4.4, Eqs. (12)-(14)] The object-lane interaction weighting is non-differentiable as written. Because pc is a hard step function of p, the derivative of fo-l with respect to the MLP parameters is zero almost everywhere, so no gradient from the loss in Eq. (16) can update the MLP that produces p. The paper does not describe a straight-through estimator, Gumbel-softmax relaxation, REINFORCE estimator, or auxiliary loss that would provide a training signal. Consequently, the claim that the module learns to 'adaptively penalize the cases in which object-lane relation is weak' is not supported, and the 30.4% AP gain attributed to this weighting in Section 4.4 and Table 5 (#2 vs #3) cannot be interpreted as the result of learned adaptive behavior. Please specify a concrete differentiable training mechanism and rerun the ablation, or revise the claims to describe a fixed, non-learned gate.
  2. [Section 5.1 / Table 7 / Appendix D.2] Hyperparameter selection appears to have been performed on the test set. The paper describes only a random training/testing split (8,121:1,737 frames) and reports in Table 7 the AP/F1 values used to choose a, b, and alpha; the values of a and b change AP from 49 to 60. If these choices were made by evaluating on the test split, the final reported margins are partially a product of test-set tuning. A separate validation split (or nested cross-validation) should be used for hyperparameter selection, and final metrics should be reported only on a held-out split not used for model selection.
  3. [Tables 2-6] All experimental results are single-run point estimates with no error bars, confidence intervals, or significance tests. Since the headline claims are 'large margins' (23.1% and 20.0% AP improvements), it is important to report mean and standard deviation over multiple runs, or at least a paired significance test, before concluding that the proposed model reliably outperforms the baselines.
  4. [Appendix D.3 / Table 2] The proposed model assumes ground-truth object bounding boxes are given, but the paper does not state whether the same ground-truth boxes were provided to all seven baselines during evaluation. If the baselines were evaluated with detector outputs while the proposed model used ground-truth boxes, the comparison in Table 2 would not be apples-to-apples and the state-of-the-art claim would not be established. Please specify the object-input protocol used for every baseline and justify the fairness of the comparison.
  5. [Appendix C / Section 4.4] The traffic-rule pathway depends on lane markings detected by the frozen CLRNet model, and Appendix C provides two qualitative failure cases but no quantitative analysis of lane-detection errors on TOI. Because TRG is credited with a large AP gain, an estimate of the frequency and impact of lane-detection failures (e.g., CLRNet precision/recall on TOI and how often such failures change importance rankings) is needed to assess the robustness of the central result.
minor comments (4)
  1. [Table 2] The row labeled 'Yolo' contains numeric entries and no input checkmarks; it is unclear whether this is a baseline method, a detector, or a formatting artifact, and the table should be cleaned and explained.
  2. [Section 2 / Section 3 / Appendix D.2] There are several typos, including 'Gaoet al.' (should be 'Gao et al.'), 'dateset' and 'date formats' (should be 'dataset' and 'data formats'), and 'SDG optimizer' (should be 'SGD optimizer').
  3. [Eq. (15) / Table 8] The softmax in Eq. (15) is not explicitly specified over which dimension; based on the architecture in Table 8, the MLP outputs two logits per object, so the softmax should be described as a per-object two-class softmax over the importance/unimportance classes.
  4. [Abstract / Section 3] The paper states that TOI will be publicly released and provides implementation details, but no dataset or code URL is included; a link to the dataset and code would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the architecture is trained and evaluated on annotated data; top-down guidance signals are external inputs, and self-citations are not load-bearing.

full rationale

The paper's derivation chain is a conventional supervised learning pipeline: TOI provides importance annotations, and the model in Eqs. (1)-(16) maps video, ego-velocity, semantic maps, and CLRNet lane points to importance scores; no equation defines the ground-truth labels in terms of the model's own outputs, so the target is not an input by construction. The DISG and TRG top-down factors come from independent external sources (IMU angular velocity for beta, DeepLabv3 semantic maps, CLRNet lane detections), not from the importance labels, and the self-citations [5,27,28,29] serve only to position related attention-prediction work rather than to justify a load-bearing premise. Ablations compare actual AP values on annotated splits, and the hyperparameter selection in Table 7 is model selection, not a fitted parameter renamed as a prediction. Appendix C's lane-detection failure cases and Appendix D.3's assumed ground-truth boxes are correctness or applicability limitations, not circular reductions; likewise the non-differentiable gate in Eqs. (12)-(14) is a trainability concern outside circularity. The empirical claims therefore stand or fall on the experiments, not on a self-referential derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

Four scalar parameters are tuned outside the training loss: the mask values a,b, the penalty alpha, and the turning threshold beta. The first three are explicitly selected by test-set performance in Tab. 7; beta is chosen from IMU statistics in Appendix D.2. All four affect the top-down guidance that the paper credits for large parts of the improvement.

free parameters (4)
  • a (low mask value) = 1
    Value in intention masks Eq. (6); selected together with b to maximize test AP/F1 in Tab. 7.
  • b (high mask value) = 1.5
    Selected from {1, 1.5, 2, 2.5} on test set in Tab. 7; b=1.5 gives AP 60 vs b=1 gives 49.
  • alpha (penalty coefficient) = 0.001
    In Eq. (13); tested values 0.1, 0.01, 0.001; chosen 0.001 because it yields best test AP/F1 in Tab. 7.
  • beta (turning threshold) = 2.2
    Angular velocity threshold Eq. (7); set from statistical analysis of IMU data (Sec. D.2), reflects ego turning left/right/straight.
assumptions (5)
  • domain assumption Driver intention is reducible to three discrete behaviors (turning left, going straight, turning right) inferable from ego-car angular velocity.
    Eqs. (6)-(7) use angular velocity threshold beta to select a fixed mask; Appendix C acknowledges real intentions are finer-grained.
  • domain assumption Lane markings detected by CLRNet are a reliable proxy for traffic rule in on-road importance.
    Eqs. (10)-(14) build the TRG module on detected lane coordinates; Appendix C documents failure cases when lane detection misses or falsely detects markings.
  • domain assumption Ground-truth object bounding boxes are available during inference.
    Appendix D.3 states the ground truth bounding boxes are assumed known, so reported performance excludes detection errors.
  • domain assumption The manual importance annotations are reliable despite the subjective nature of the task.
    Section 3 describes double-checking and triple-discussion protocols but reports no inter-annotator agreement metric.
  • domain assumption The beta threshold derived from IMU statistics is stable across the 28 TOI scenes.
    Section D.2 sets beta=2.2 from IMU analysis; if driving behavior distribution shifts between scenes, the intention masks are mis-assigned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance." pith.science (2026). https://pith.science/paper/T3H5NZRX

@misc{pith2026241117152,
  author       = {Pith},
  title        = {Pith review of: On-Road Object Importance Estimation: A New Dataset and A Model with Multi-Fold Top-Down Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T3H5NZRX}},
  note         = {Machine review of arXiv:2411.17152}
}
read the original abstract

This paper addresses the problem of on-road object importance estimation, which utilizes video sequences captured from the driver's perspective as the input. Although this problem is significant for safer and smarter driving systems, the exploration of this problem remains limited. On one hand, publicly-available large-scale datasets are scarce in the community. To address this dilemma, this paper contributes a new large-scale dataset named Traffic Object Importance (TOI). On the other hand, existing methods often only consider either bottom-up feature or single-fold guidance, leading to limitations in handling highly dynamic and diverse traffic scenarios. Different from existing methods, this paper proposes a model that integrates multi-fold top-down guidance with the bottom-up feature. Specifically, three kinds of top-down guidance factors (ie, driver intention, semantic context, and traffic rule) are integrated into our model. These factors are important for object importance estimation, but none of the existing methods simultaneously consider them. To our knowledge, this paper proposes the first on-road object importance estimation model that fuses multi-fold top-down guidance factors with bottom-up feature. Extensive experiments demonstrate that our model outperforms state-of-the-art methods by large margins, achieving 23.1% Average Precision (AP) improvement compared with the recently proposed model (ie, Goal).

Figures

Figures reproduced from arXiv: 2411.17152 by the authors.

Figure 1
Figure 1. The crucial factors considered by human drivers when estimating on-road object importance. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overview of multi-fold top-down guidance aware model. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of object-lane interaction weighting. on both sides of the road. The results make sense since the static cars on roadsides are factually not interacting with lanes. In Fig. 3c and Fig. 3d, oncoming cars from the opposite direction and the car on the current lane are not penalized, since these cars are interacting with lanes. We note the yellow mask do not signal the important object. Instead, it indica… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison with baselines (i.e., Goal [8], Ohn-Bar [33], and Zhang [50]). Red boxes represent important objects and green boxes denote unimportant objects. Four scenarios (a)-(d) are illustrated in [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Failure examples. Top row is GT and bottom row is object importance estimation. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 24 canonical work pages

  1. [1]

    IEEE Transactions on Intelligent Transportation Systems 23(6), 5573–5585 (2022)

    Amadori, P.V ., Fischer, T., Demiris, Y .: Hammerdrive: A task-aware driving visual attention model. IEEE Transactions on Intelligent Transportation Systems 23(6), 5573–5585 (2022). https://doi.org/10.1109/TITS.2021.3055120

    work page arXiv 2022

  2. [2]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Caesar, H., Bankiti, V ., Lang, A.H., V ora, S., Liong, V .E., Xu, Q., Krishnan, A., Pan, Y ., Baldan, G., Beijbom, O.: nuscenes: A multimodal dataset for autonomous driving. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    work page 2020

  3. [3]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4), 834–848 (2018)

    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convo- lutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4), 834–848 (2018). https://doi.org/10.1109/TPAMI.2017.2699184

    arXiv 2018

  4. [4]

    IEEE Transactions on Systems, Man, and Cybernetics: Systems 53(9), 5831–5847 (2023)

    Chen, L., Li, Y ., Huang, C., Xing, Y ., Tian, D., Li, L., Hu, Z., Teng, S., Lv, C., Wang, J., Cao, D., Zheng, N., Wang, F.Y .: Milestones in autonomous driving and intelligent vehicles—part i: control, computing system design, communication, hd map, testing, and human behaviors. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53(9), 5831–5847...

    arXiv 2023

  5. [5]

    In: IEEE/CVF International Conference on Computer Vision

    Chen, Y ., Nan, Z., Xiang, T.: Fblnet: Feedback loop network for driver attention prediction. In: IEEE/CVF International Conference on Computer Vision. pp. 13325–13334 (2023)

    work page 2023

  6. [6]

    IEEE Transactions on Intelligent Transportation Systems 17(7), 2051–2062 (2016)

    Deng, T., Yang, K., Li, Y ., Yan, H.: Where does the driver look? top-down-based saliency detection in a traffic driving environment. IEEE Transactions on Intelligent Transportation Systems 17(7), 2051–2062 (2016). https://doi.org/10.1109/TITS.2016.2535402

    arXiv 2016

  7. [7]

    IEEE Transactions on Intelligent Transportation Systems 23(6), 4959–4971 (2022)

    Fang, J., Yan, D., Qiao, J., Xue, J., Yu, H.: Dada: Driver attention prediction in driving accident scenarios. IEEE Transactions on Intelligent Transportation Systems 23(6), 4959–4971 (2022). https://doi.org/10.1109/TITS.2020.3044678

    work page arXiv 2022

  8. [8]

    In: Proceedings of the IEEE International Conference on Robotics and Automation

    Gao, M., Tawari, A., Martin, S.: Goal-oriented object importance estimation in on-road driving videos. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 5509–5515 (2019). https://doi.org/10.1109/ICRA.2019.8793970

    arXiv 2019

Show all 53 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3354–3361 (2012). https://doi.org/10.1109/CVPR.2012.6248074

    2012

  2. [10]

    In: Proceedings of the IEEE International Conference on Computer Vision

    Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1440–1448 (2015). https://doi.org/10.1109/ICCV .2015.169

    2015 doi

  3. [11]

    IEEE Transactions on Intelligent Transportation Systems 21(8), 3135–3151 (2020)

    Guo, J., Kurup, U., Shah, M.: Is it safe to drive? an overview of factors, metrics, and datasets for driveability as- sessment in autonomous driving. IEEE Transactions on Intelligent Transportation Systems 21(8), 3135–3151 (2020). https://doi.org/10.1109/TITS.2019.2926042

    2020

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90 9 A PREPRINT - NOVEMBER 27, 2024

    2016 doi

  5. [13]

    Neural Computation 9(8), 1735–1780 (1997)

    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    1997 doi

  6. [15]

    IEEE Transactions on Industrial Electronics 69(2), 1800–1808 (2022)

    Hu, Z., Lv, C., Hang, P., Huang, C., Xing, Y .: Data-driven estimation of driver attention using calibration-free eye gaze and scene features. IEEE Transactions on Industrial Electronics 69(2), 1800–1808 (2022). https://doi.org/10.1109/TIE.2021.3057033

    2022

  7. [16]

    IEEE Transactions on Intelligent Transportation Systems 23(12), 24343–24354 (2022)

    Hu, Z., Zhang, Y ., Li, Q., Lv, C.: A novel heterogeneous network for modeling driver attention with multi-level visual content. IEEE Transactions on Intelligent Transportation Systems 23(12), 24343–24354 (2022). https://doi.org/10.1109/TITS.2022.3208004

    2022

  8. [17]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)

    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998). https://doi.org/10.1109/34.730558

    1998 doi

  9. [18]

    IEEE Transactions on Image Processing 26(9), 4446–4456 (2017)

    Kruthiventi, S.S.S., Ayush, K., Babu, R.V .: Deepfix: A fully convolutional neural network for predicting human eye fixations. IEEE Transactions on Image Processing 26(9), 4446–4456 (2017). https://doi.org/10.1109/TIP.2017.2710620

    2017

  10. [19]

    arXiv preprint arXiv:2312.01659 (2023)

    Kung, C.H., Yang, C.C., Pao, P.Y ., Lu, S.W., Chen, P.L., Lu, H.C., Chen, Y .T.: Riskbench: A scenario-based benchmark for risk identification. arXiv preprint arXiv:2312.01659 (2023)

    2023 arXiv

  11. [20]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 45(11), 13683–13698 (2023)

    Li, C., Chan, S.H., Chen, Y .T.: Droid: Driver-centric risk object identification. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(11), 13683–13698 (2023). https://doi.org/10.1109/TPAMI.2023.3294305

    2023

  12. [21]

    In: Proceedings of the IEEE International Conference on Robotics and Automation

    Li, J., Gang, H., Ma, H., Tomizuka, M., Choi, C.: Important object identification with semi-supervised learning for au- tonomous driving. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 2913–2919 (2022). https://doi.org/10.1109/ICRA46639.20...

    2022

  13. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, W.H., Hong, F.T., Zheng, W.S.: Learning to learn relation for important people detection in still images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4998–5006 (2019). https://doi.org/10.1109/CVPR.2019.00514

    2019

  14. [25]

    In: Proceedings of the International Conference on Human-Computer Interaction

    Liu, Y ., Zhang, J., Li, Y ., Hansen, P., Wang, J.: Human-computer collaborative interaction design of intelligent vehicle—a case study of hmi of adaptive cruise control. In: Proceedings of the International Conference on Human-Computer Interaction. pp. 296–314 (2021)

    2021

  15. [26]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Malla, S., Choi, C., Dwivedi, I., Hee Choi, J., Li, J.: Drama: Joint risk localization and captioning in driving. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 1043–1052 (2023). https://doi.org/10.1109/W ACV56688.2023.00110

    2023

  16. [27]

    IEEE Transactions on Image Processing 30, 8293–8305 (2021)

    Nan, Z., Jiang, J., Gao, X., Zhou, S., Zuo, W., Wei, P., Zheng, N.: Predicting task-driven attention via integrating bottom-up stimulus and top-down guidance. IEEE Transactions on Image Processing 30, 8293–8305 (2021)

    2021

  17. [28]

    Pattern Recognition 103, 107314 (2020)

    Nan, Z., Shu, T., Gong, R., Wang, S., Wei, P., Zhu, S.C., Zheng, N.: Learning to infer human attention in daily activities. Pattern Recognition 103, 107314 (2020)

    2020

  18. [29]

    IEEE Transactions on Circuits and Systems for Video Technology34(8), 6762–6773 (2024)

    Nan, Z., Xiang, T.: Third-person view attention prediction in natural scenarios with weak information dependency and human-scene interaction mechanism. IEEE Transactions on Circuits and Systems for Video Technology34(8), 6762–6773 (2024)

    2024

  19. [30]

    IEEE Access 11, 62932–62941 (2023)

    Nitta, Y ., Isogawa, M., Yonetani, R., Sugimoto, M.: Importance rank-learning of objects in urban scenes for assisting visually impaired people. IEEE Access 11, 62932–62941 (2023). https://doi.org/10.1109/ACCESS.2023.3287147

    2023

  20. [31]

    In: Proceedings of the IEEE Intelligent Vehicles Symposium

    Niu, Y ., Ding, M., Zhang, Y ., Ohtani, K., Takeda, K.: Auditory and visual warning information generation of the risk object in driving scenes based on weakly supervised learning. In: Proceedings of the IEEE Intelligent Vehicles Symposium. pp. 1572–1577 (2022). https://doi.or...

    2022

  21. [32]

    IEEE Transactions on Intelligent Vehicles 1(1), 90–104 (2016)

    Ohn-Bar, E., Trivedi, M.M.: Looking at humans in the age of self-driving and highly automated vehicles. IEEE Transactions on Intelligent Vehicles 1(1), 90–104 (2016). https://doi.org/10.1109/TIV .2016.2571067

    2016

  22. [33]

    Pattern Recognition 64, 425–436 (2017)

    Ohn-Bar, E., Trivedi, M.M.: Are all objects equal? deep spatio-temporal importance prediction in driving videos. Pattern Recognition 64, 425–436 (2017)

    2017

  23. [34]

    Organization, W.H.: World health statistics 2023 (2023)

    2023

  24. [35]

    looking at the right stuff

    Pal, A., Mondal, S., Christensen, H.I.: “looking at the right stuff” – guided semantic-gaze for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11880–11889 (2020). https://doi.org/10.1109/CVPR42600.2020.01190 10 A ...

    2020

  25. [36]

    In: Proceedings of the IEEE International Conference on Intelligent Computing and Human-Computer Interaction

    Qiu, Y .: Human-centered hmi design for level 3 automated driving takeover process. In: Proceedings of the IEEE International Conference on Intelligent Computing and Human-Computer Interaction. pp. 43–54 (2023). https://doi.org/10.1109/ICHCI58871.2023.10277815

    2023

  26. [37]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

    Ramanishka, V ., Chen, Y .T., Misu, T., Saenko, K.: Toward driving scene understanding: A dataset for learning driver behavior and causal reasoning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

    2018

  27. [38]

    IEEE Transactions on Intelligent Transportation Systems 24(8), 8339–8347 (2023)

    Sharma, N., Garg, R.D.: Real-time iot-based connected vehicle infrastructure for intelligent transportation safety. IEEE Transactions on Intelligent Transportation Systems 24(8), 8339–8347 (2023). https://doi.org/10.1109/TITS.2023.3263271

    2023

  28. [39]

    IEEE Transactions on Vehicular Technology71(3), 2319–2332 (2022)

    Sidorenko, G., Thunberg, J., Sjöberg, K., Fedorov, A., Vinel, A.: Safety of automatic emergency braking in platooning. IEEE Transactions on Vehicular Technology71(3), 2319–2332 (2022). https://doi.org/10.1109/TVT.2021.3138939

    2022

  29. [40]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 45(1), 1036–1054 (2023)

    Singh, G., Akrigg, S., Maio, M.D., Fontana, V ., Alitappeh, R.J., Khan, S., Saha, S., Jeddisaravi, K., Yousefi, F., Culley, J., Nicholson, T., Omokeowa, J., Grazioso, S., Bradley, A., Gironimo, G.D., Cuzzolin, F.: Road: The road event awareness dataset for autonomous driving. ...

    2023

  30. [41]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Suzuki, T., Kataoka, H., Aoki, Y ., Satoh, Y .: Anticipating traffic accidents with adaptive loss and large-scale incident db. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3521–3529 (2018). https://doi.org/10.1109/CVPR.2018.00371

    2018

  31. [42]

    IEEE Transactions on Intelligent Transportation Systems 23(9), 15434–15449 (2022)

    Tang, X., Yu, J., Su, Y .: Modeling driver’s visual fixation behavior using white-box representations. IEEE Transactions on Intelligent Transportation Systems 23(9), 15434–15449 (2022). https://doi.org/10.1109/TITS.2022.3140759

    2022

  32. [43]

    IEEE/CAA Journal of Automatica Sinica 9(7), 1335–1338 (2022)

    Tian, H., Deng, T., Yan, H.: Driving as well as on a sunny day? predicting driver’s fixation in rainy weather conditions via a dual-branch visual model. IEEE/CAA Journal of Automatica Sinica 9(7), 1335–1338 (2022). https://doi.org/10.1109/JAS.2022.105716

    2022

  33. [44]

    Cognitive psychology 12(1), 97–136 (1980)

    Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognitive psychology 12(1), 97–136 (1980)

    1980

  34. [45]

    Proceedings of the IEEE 109(4), 377–398 (2021)

    Wan, J., Li, X., Dai, H.N., Kusiak, A., Martínez-García, M., Li, D.: Artificial-intelligence-driven customized man- ufacturing factory: Key technologies, applications, and challenges. Proceedings of the IEEE 109(4), 377–398 (2021). https://doi.org/10.1109/JPROC.2020.3034808

    2021

  35. [46]

    In: 2022 IEEE Intelligent Vehicles Symposium

    Wu, T., Sachdeva, E., Akash, K., Wu, X., Misu, T., Ortiz, J.: Toward an adaptive situational awareness support system for urban driving. In: 2022 IEEE Intelligent Vehicles Symposium. pp. 1073–1080 (2022)

    2022

  36. [48]

    IEEE Transactions on Intelligent Transportation Systems 23(7), 7941–7951 (2022)

    Yu, S.Y ., Malawade, A.V ., Muthirayan, D., Khargonekar, P.P., Faruque, M.A.A.: Scene-graph augmented data-driven risk assessment of autonomous vehicle decisions. IEEE Transactions on Intelligent Transportation Systems 23(7), 7941–7951 (2022). https://doi.org/10.1109/TITS.2021.3074854

    2022

  37. [49]

    IEEE Access 8, 58443–58469 (2020)

    Yurtsever, E., Lambert, J., Carballo, A., Takeda, K.: A survey of autonomous driving: Common practices and emerging technologies. IEEE Access 8, 58443–58469 (2020). https://doi.org/10.1109/ACCESS.2020.2983149

    2020

  38. [50]

    In: Proceedings of the IEEE International Conference on Robotics and Automation

    Zhang, Z., Tawari, A., Martin, S., Crandall, D.: Interaction graphs for object importance estimation in on-road driv- ing videos. In: Proceedings of the IEEE International Conference on Robotics and Automation. pp. 8920–8927 (2020). https://doi.org/10.1109/ICRA40945.2020.9197104

    2020

  39. [51]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zheng, T., Huang, Y ., Liu, Y ., Tang, W., Yang, Z., Cai, D., He, X.: Clrnet: Cross layer refinement network for lane detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 888–897 (2022). https://doi.org/10.1109/CVPR52688.2022.00097

    2022

  40. [52]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhou, H., Qiao, B., Yang, L., Lai, J., Xie, X.: Texture-guided saliency distilling for unsupervised salient object detec- tion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7257–7267 (2023). https://doi.org/10.1109/CVPR52729.2023.0...

    2023

  41. [53]

    Bottom-up feature : the low-level information extracted directly from the input images or video frames using backbone networks

  42. [54]

    Top-down guidance: the high-level information including semantic understanding, prior knowledge, specific goals, etc

  43. [55]

    Ego-car: the car capturing video sequences that are used as the input of the model

  44. [56]

    Intention driving path: the path from the ego-car current position to the intention destination

  45. [57]

    B Additional Experimental Results B.1 Qualitative Comparison Figure 4: Qualitative comparison with baselines (i.e., Goal [8], Ohn-Bar [33], and Zhang [50])

    Intention behaviors: the actions that the driver intends to perform based on their goals ( e.g., turning left, going straight, and turning right). B Additional Experimental Results B.1 Qualitative Comparison Figure 4: Qualitative comparison with baselines (i.e., Goal [8], Ohn-...

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.