pith. sign in

arxiv: 2605.22550 · v1 · pith:AWIGHYOBnew · submitted 2026-05-21 · 💻 cs.CV

MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding

Varun A. Paturkar , Shankar Gangisetty , C.V. Jawahar This is my paper

Pith reviewed 2026-05-22 06:03 UTC · model grok-4.3

classification 💻 cs.CV
keywords two-wheelerrider behaviormultimodal datasetaction recognitionmaneuver classificationtraffic safetyeye gazetelemetry
0
0 comments X

The pith

The MOTOR dataset shows that fusing RGB video, eye gaze, and telemetry improves two-wheeler rider behavior recognition and legality classification.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the MOTOR dataset to address the lack of resources for studying two-wheeler rider behavior in dense traffic compared to four-wheeler research. It contains 1,629 sequences totaling over 25 hours of data from 16 riders, with synchronized front, rear, and helmet videos, wearable eye-gaze tracking, on-road audio, and telemetry. Annotations cover traffic context, rider state, 12 maneuvers including unconventional ones, and legality labels. Benchmarks using CNN and Transformer models for action recognition find that combining RGB, gaze, and telemetry inputs delivers the strongest results on both behavior recognition and maneuver legality tasks.

Core claim

MOTOR is a multimodal dataset of 1,629 sequences with more than 25 hours of video collected from 16 riders in dense, unstructured traffic. It synchronizes front, rear, and helmet camera views with rider eye-gaze from wearable trackers, on-road audio, and telemetry including GPS, accelerometer, and gyroscope, plus annotations for 12 riding maneuvers and legality categories of Legal, Illegal, or Unspecified. Benchmarks establish that multimodal fusion of RGB, gaze, and telemetry consistently outperforms single-modality approaches in rider behavior recognition and legality classification.

What carries the argument

The MOTOR dataset, which synchronizes multiple video views, eye-gaze, audio, and telemetry with annotations for maneuvers and legality to support multimodal rider behavior analysis.

If this is right

  • Multimodal fusion of RGB, gaze, and telemetry yields better results than any single input for recognizing rider behaviors.
  • The same fusion approach improves accuracy in classifying whether maneuvers are legal or illegal.
  • The dataset supplies a benchmark resource for developing behavior analysis tools in intelligent transportation systems.
  • Annotations for both conventional and unconventional maneuvers enable targeted study of safety-relevant actions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The dataset could support training of real-time safety alerts for riders in high-density traffic environments.
  • Extending collection to more riders and cities would help check how well the multimodal gains hold in new settings.
  • The use of wearable gaze and telemetry alongside video might apply to monitoring other road users such as cyclists.

Load-bearing premise

Sequences recorded from 16 riders under the studied conditions represent typical two-wheeler rider behavior in dense, unstructured traffic across the Global South.

What would settle it

Collecting new multimodal recordings from additional riders in different traffic settings and finding that models no longer show performance gains when gaze and telemetry are added to RGB video.

Figures

Figures reproduced from arXiv: 2605.22550 by C.V. Jawahar, Shankar Gangisetty, Varun A. Paturkar.

Figure 1
Figure 1. Figure 1: Comparison of traffic contexts and accident statistics [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of rider behaviors in the MOTOR dataset. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Data capture setup: Three cameras are oriented towards front-view (ego-vehicle, helmet-mounted), rare-view, and eye-gaze derived from eye-tracking cameras (Aria [19] or Pupil [20] glasses). task-specific: CDBV [34] collected bike-mounted egocen￾tric views, MoRe [35] targeted motorcycle re-identification, while riding dynamics [32] and powered two-wheeler pat￾terns [31] explored specific motion and traffic … view at source ↗
Figure 4
Figure 4. Figure 4: Data samples helmet-view. (a) Ego-rider weaves through dense, slow traffic, overtaking multiple vehicles across lanes. (b) Rider squeezes through a narrow gap between a bus and a car, narrowly avoiding the bus. (c) Rider rides in the wrong lane against dense oncoming traffic, disrupting flow. (d) Rider turns head fully toward a roadside building, diverting gaze from the road amid fast-moving traffic. to cr… view at source ↗
Figure 5
Figure 5. Figure 5: Data samples of legal and illegal maneuvers. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Annotation Instances and data statistics. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Confusion matrices for rider behavior classification using S3D [21], ResNet3D [22], MViTv2 [24], and SwinT [23] on the MOTOR dataset. highlight some key lessons for two-wheeler behavior: gaze provides complementary attention cues, telemetry captures dominant kinematic patterns such as lean and speed, and unconventional behaviors remain challenging to classify due to their variability and overlap with conve… view at source ↗
read the original abstract

Two-wheelers account for a disproportionately high share of road fatalities in the Global South. Research on two-wheeler rider behavior, however, lags far behind four-wheelers, where multimodal datasets have driven major advances in Advanced Driver Assistance Systems (ADAS). To address this gap, we present the MOtorized TwO-wheeler Rider (MOTOR) dataset, the first large-scale, multi-view, multimodal resource dedicated to two-wheelers in dense, unstructured traffic. MOTOR comprises 1,629 sequences (25+ hours of video data) collected from 16 riders and integrates synchronized front, rear, and helmet videos, rider eye-gaze from wearable trackers, on-road audio, and telemetry (GPS, accelerometer, gyroscope). Rich annotations capture traffic context, rider state, 12 riding maneuvers spanning conventional and unconventional behaviors, and legality labels (Legal, Illegal, Unspecified). We benchmark rider behavior recognition and maneuver legality classification using state-of-the-art video action recognition backbones (CNN and Transformer-based), extended with multimodal fusion, and find that combining RGB, gaze, and telemetry consistently yields the best performance. MOTOR thus provides a unique foundation for advancing safety-critical understanding of two-wheeler riding. It offers the research community a benchmark to develop and evaluate models for behavior analysis, legality-aware prediction, and intelligent transportation systems. Dataset and code is available at https: //varuniiith.github.io/MOTOR-Dataset/

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper introduces the MOTOR dataset as the first large-scale multimodal resource for two-wheeler rider behavior in dense, unstructured traffic. It comprises 1,629 sequences (25+ hours) from 16 riders, with synchronized front/rear/helmet videos, eye-gaze from wearables, on-road audio, and telemetry (GPS, accelerometer, gyroscope). Annotations cover traffic context, rider state, 12 maneuvers (conventional and unconventional), and legality labels (Legal/Illegal/Unspecified). Benchmarks using CNN/Transformer video backbones with multimodal fusion show that RGB + gaze + telemetry consistently achieves the best results on behavior recognition and maneuver legality classification. The dataset and code are publicly released.

Significance. If the benchmarks use rider-disjoint splits and annotation protocols are shown to be reliable, MOTOR would be a significant contribution by providing the first dedicated multimodal benchmark for two-wheeler safety analysis in Global South contexts. This could drive advances in ADAS, legality-aware prediction, and intelligent transportation systems, analogous to how multimodal datasets advanced four-wheeler research. The public release of data and code is a clear strength that enhances reproducibility and community utility.

major comments (2)
  1. [Benchmarking section] Benchmarking section (results on behavior recognition and legality classification): The central claim that 'combining RGB, gaze, and telemetry consistently yields the best performance' is load-bearing for the paper's utility as a benchmark. The manuscript must explicitly describe the train/test split strategy and confirm whether it is rider-disjoint. With only 16 riders, any leakage (e.g., random sequence splits or per-sequence but same-rider) allows models to exploit stable rider-specific traits such as posture, helmet fit, or typical speed profiles that correlate with both labels and extra modalities, which would inflate the reported multimodal gains and undermine cross-rider generalization claims.
  2. [Dataset Collection and Annotations section] Dataset Collection and Annotations section: Quantitative details on annotation reliability (e.g., inter-annotator agreement scores for the 12 maneuvers and legality labels) are missing. Without these, it is difficult to assess the quality of the ground truth that underpins all benchmark outcomes; this is especially critical given the subjective nature of 'unconventional' maneuvers and legality judgments in unstructured traffic.
minor comments (3)
  1. [Abstract] The dataset URL in the abstract contains a space after the colon ('https: //varuniiith.github.io/MOTOR-Dataset/'); this should be corrected for accessibility.
  2. [Results tables/figures] Table or figure captions for the benchmark results should include exact train/test split ratios and whether they are rider-independent to aid immediate interpretation.
  3. [Discussion or Conclusion] Consider adding a short limitations paragraph discussing the modest number of riders (16) and geographic scope when claiming broader applicability to Global South traffic.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the MOTOR dataset paper. The comments highlight important aspects of benchmarking rigor and annotation quality that we address below. We have revised the manuscript accordingly to strengthen these sections.

read point-by-point responses

  1. Referee: [Benchmarking section] Benchmarking section (results on behavior recognition and legality classification): The central claim that 'combining RGB, gaze, and telemetry consistently yields the best performance' is load-bearing for the paper's utility as a benchmark. The manuscript must explicitly describe the train/test split strategy and confirm whether it is rider-disjoint. With only 16 riders, any leakage (e.g., random sequence splits or per-sequence but same-rider) allows models to exploit stable rider-specific traits such as posture, helmet fit, or typical speed profiles that correlate with both labels and extra modalities, which would inflate the reported multimodal gains and undermine cross-rider generalization claims.

    Authors: We agree that explicit confirmation of a rider-disjoint split is essential for validating cross-rider generalization claims, especially with a modest number of riders. The original manuscript described the overall split ratios but did not explicitly state the rider-disjoint protocol. In the revised manuscript, we have added a clear description in the Benchmarking section: the 1,629 sequences are partitioned such that all sequences from 12 riders form the training set and sequences from the remaining 4 riders form the test set, with no rider overlap between splits. This protocol has also been documented in the public dataset release to support reproducibility. revision: yes

  2. Referee: [Dataset Collection and Annotations section] Dataset Collection and Annotations section: Quantitative details on annotation reliability (e.g., inter-annotator agreement scores for the 12 maneuvers and legality labels) are missing. Without these, it is difficult to assess the quality of the ground truth that underpins all benchmark outcomes; this is especially critical given the subjective nature of 'unconventional' maneuvers and legality judgments in unstructured traffic.

    Authors: We acknowledge that the manuscript did not include quantitative inter-annotator agreement metrics, which are important for assessing ground-truth reliability given the subjective elements in maneuver and legality labeling. In the revised version, we have added these details to the Dataset Collection and Annotations section. Specifically, we report Cohen's kappa scores computed over a subset of sequences annotated by three independent annotators: 0.83 for the 12 maneuver classes and 0.76 for the legality labels (Legal/Illegal/Unspecified), with disagreements resolved by majority vote. The annotation protocol is also expanded for clarity. revision: yes

Circularity Check

0 steps flagged

No circularity in dataset collection or empirical benchmarks

full rationale

The MOTOR paper is a data resource and benchmark report with no mathematical derivations, equations, fitted parameters, or first-principles claims. It describes collection of 1,629 sequences from 16 riders, annotations for maneuvers and legality, and empirical results from training CNN/Transformer backbones with multimodal fusion, reporting that RGB+gaze+telemetry yields best performance. These are direct experimental outcomes on the provided data splits rather than any reduction by construction, self-definition, or load-bearing self-citation. No steps match the enumerated circularity patterns, and the work remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an empirical dataset paper; no mathematical derivations, fitted parameters, or postulated entities appear in the abstract.

pith-pipeline@v0.9.0 · 5792 in / 1079 out tokens · 44814 ms · 2026-05-22T06:03:25.423955+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    MOTOR comprises 1,629 sequences (25+ hours of video data) collected from 16 riders and integrates synchronized front, rear, and helmet videos, rider eye-gaze from wearable trackers, on-road audio, and telemetry... combining RGB, gaze, and telemetry consistently yields the best performance on rider behavior recognition and maneuver legality classification.

  • IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    We benchmark rider behavior recognition... using state-of-the-art video action recognition backbones (CNN and Transformer-based), extended with multimodal fusion

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages

  1. [1]

    Highway statistics 2022,

    U. D. of Transportation Federal Highway Administration, “Highway statistics 2022,” 2022. [Online]. Available: https://highways.dot.gov/

    work page 2022

  2. [2]

    Transport database,

    Eurostat, “Transport database,” 2024. [Online]. Available: https: //ec.europa.eu/eurostat/web/transport/database

    work page 2024

  3. [3]

    Annual report: 2023–2024,

    M. of Road Transport and I. Highways, “Annual report: 2023–2024,”

    work page 2023

  4. [4]

    Available: https://morth.nic.in/en/annual-report

    [Online]. Available: https://morth.nic.in/en/annual-report

    work page

  5. [5]

    E mobility country profile,

    A. T. Observatory, “E mobility country profile,” 2023. [Online]. Available: https://asiantransportobservatory.org/documents/ 67/Indonesia 20231002b.pdf

    work page 2023

  6. [6]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesaret al., “nuscenes: A multimodal dataset for autonomous driving,” inCVPR, 2020

    work page 2020

  7. [7]

    Scalability in perception for autonomous driving: Waymo open dataset,

    P. Sunet al., “Scalability in perception for autonomous driving: Waymo open dataset,” inCVPR, 2020

    work page 2020

  8. [8]

    Large car-following data based on lyft level-5 open dataset,

    G. Liet al., “Large car-following data based on lyft level-5 open dataset,” inITSC, 2023

    work page 2023

  9. [9]

    Bdd100k: A diverse driving dataset for heterogeneous multitask learning,

    F. Yuet al., “Bdd100k: A diverse driving dataset for heterogeneous multitask learning,” inCVPR, 2020

    work page 2020

  10. [10]

    Idd: A dataset for exploring problems of autonomous navigation in unconstrained environments,

    G. Varmaet al., “Idd: A dataset for exploring problems of autonomous navigation in unconstrained environments,” inWACV, 2019

    work page 2019

  11. [11]

    The apolloscape dataset for autonomous driving,

    X. Huanget al., “The apolloscape dataset for autonomous driving,” in CVPR Workshops, 2018

    work page 2018

  12. [12]

    Toward driving scene understanding: A dataset for learning driver behavior and causal reasoning,

    V . Ramanishkaet al., “Toward driving scene understanding: A dataset for learning driver behavior and causal reasoning,” inCVPR, 2018

    work page 2018

  13. [13]

    Car that knows before you do: Anticipating maneuvers via learning temporal driving models,

    A. Jainet al., “Car that knows before you do: Anticipating maneuvers via learning temporal driving models,” inICCV, December 2015

    work page 2015

  14. [14]

    Meteor: A dense, heterogeneous, and unstructured traffic dataset with rare behaviors,

    R. Chandraet al., “Meteor: A dense, heterogeneous, and unstructured traffic dataset with rare behaviors,” inICRA, 2023

    work page 2023

  15. [15]

    Early anticipation of driving maneuvers,

    A. Wasiet al., “Early anticipation of driving maneuvers,” inECCV, 2024

    work page 2024

  16. [16]

    Unigen: Unified modeling of initial agent states and trajectories for generating autonomous driving scenarios,

    R. Mahjourianet al., “Unigen: Unified modeling of initial agent states and trajectories for generating autonomous driving scenarios,” inICRA, 2024

    work page 2024

  17. [17]

    Road accidents in india 2022,

    MORTH, “Road accidents in india 2022,” 2023. [Online]. Available: https://morth.nic.in/road-accident-in-india

    work page 2022

  18. [18]

    Icpr 2024 competition on rider intention predic- tion,

    S. Gangisettyet al., “Icpr 2024 competition on rider intention predic- tion,” inICPR, 2024

    work page 2024

  19. [19]

    myeye2wheeler: A two-wheeler indian driver real-world eye-tracking dataset,

    B. V . Kumaret al., “myeye2wheeler: A two-wheeler indian driver real-world eye-tracking dataset,” inITSC, 2024

    work page 2024

  20. [20]

    Project aria: A new tool for egocentric multi-modal ai research,

    J. Engelet al., “Project aria: A new tool for egocentric multi-modal ai research,”arXiv, 2023

    work page 2023

  21. [21]

    Pupil: An open source platform for pervasive eye tracking and mobile gaze-based interaction,

    M. Kassneret al., “Pupil: An open source platform for pervasive eye tracking and mobile gaze-based interaction,”arXiv, 2014

    work page 2014

  22. [22]

    Rethinking spatiotemporal feature learning: Speed- accuracy trade-offs in video classification,

    S. Xieet al., “Rethinking spatiotemporal feature learning: Speed- accuracy trade-offs in video classification,” inECCV, 2018

    work page 2018

  23. [23]

    A closer look at spatiotemporal convolutions for action recognition,

    D. Tranet al., “A closer look at spatiotemporal convolutions for action recognition,” inCVPR, 2018

    work page 2018

  24. [24]

    Video swin transformer,

    Z. Liuet al., “Video swin transformer,” inCVPR, 2022

    work page 2022

  25. [25]

    Mvitv2: Improved multiscale vision transformers for classification and detection,

    Y . Li, C.-Y . Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, and C. Feichtenhofer, “Mvitv2: Improved multiscale vision transformers for classification and detection,” inCVPR, 2022

    work page 2022

  26. [26]

    Idd-x: A multi-view dataset for ego-relative important object localization and explanation in dense and unstructured traffic,

    C. Parikhet al., “Idd-x: A multi-view dataset for ego-relative important object localization and explanation in dense and unstructured traffic,” inICRA, 2024

    work page 2024

  27. [27]

    Aide: A vision-driven multi-view, multi-modal, multi- tasking dataset for assistive driving perception,

    D. Yanget al., “Aide: A vision-driven multi-view, multi-modal, multi- tasking dataset for assistive driving perception,” inICCV, 2023

    work page 2023

  28. [28]

    The oxford robotcycle project: A multimodal urban cycling dataset for assessing the safety of vulnerable road users,

    E. Panagiotakiet al., “The oxford robotcycle project: A multimodal urban cycling dataset for assessing the safety of vulnerable road users,” IEEE Transactions on Field Robotics, 2025

    work page 2025

  29. [29]

    Dmd: A large-scale multi-modal driver monitoring dataset for attention and alertness analysis,

    J. D. Ortegaet al., “Dmd: A large-scale multi-modal driver monitoring dataset for attention and alertness analysis,” inECCV, 2020

    work page 2020

  30. [30]

    Predicting the driver’s focus of attention: the dr(eye)ve project,

    A. Palazziet al., “Predicting the driver’s focus of attention: the dr(eye)ve project,”IEEE TPAMI, 2019

    work page 2019

  31. [31]

    Look both ways: Self-supervising driver gaze estimation and road scene saliency,

    I. Kasaharaet al., “Look both ways: Self-supervising driver gaze estimation and road scene saliency,” inECCV, 2022

    work page 2022

  32. [32]

    Powered two-wheeler riding pattern recognition using a machine-learning framework,

    F. Attalet al., “Powered two-wheeler riding pattern recognition using a machine-learning framework,”IEEE Transactions on Intelligent Transportation Systems, 2015

    work page 2015

  33. [33]

    Data-driven methodology for the investigation of riding dynamics: A motorcycle case study,

    M. Bartolozziet al., “Data-driven methodology for the investigation of riding dynamics: A motorcycle case study,”IEEE Transactions on Intelligent Transportation Systems, 2023

    work page 2023

  34. [34]

    Motorcycle safety gear recognition with deep learning,

    J. A. Sanchez-Rodriguezet al., “Motorcycle safety gear recognition with deep learning,” inTEMSCON LATAM, 2024

    work page 2024

  35. [35]

    Cdbv: A driving dataset with chinese characteristics from a bike view,

    Y . Heet al., “Cdbv: A driving dataset with chinese characteristics from a bike view,”IEEE Access, 2019

    work page 2019

  36. [36]

    More: A large-scale motorcycle re-identification dataset,

    A. Figueiredoet al., “More: A large-scale motorcycle re-identification dataset,” inWACV, 2021

    work page 2021

  37. [37]

    Motorized two wheelers in indian cities,

    EMBARQ, “Motorized two wheelers in indian cities,” 2014

    work page 2014

  38. [38]

    Telemetry extraction for gopro,

    “Telemetry extraction for gopro,” 2024. [Online]. Available: https: //goprotelemetryextractor.com/telemetry-overlay-gps-video-sensors

    work page 2024

  39. [39]

    Indian motor vehicle driving regulation 2017,

    MORTH, “Indian motor vehicle driving regulation 2017,”

    work page 2017

  40. [40]

    Available: https://morth.nic.in/sites/default/files/ Motor-Vehicle-Driving-Regulation-2017.pdf

    [Online]. Available: https://morth.nic.in/sites/default/files/ Motor-Vehicle-Driving-Regulation-2017.pdf

    work page 2017

  41. [41]

    Focal loss for dense object detection,

    T.-Y . Linet al., “Focal loss for dense object detection,” inICCV, 2017

    work page 2017

  42. [42]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszkeet al., “Pytorch: An imperative style, high-performance deep learning library,”Advances in neural information processing systems, 2019

    work page 2019