RECORD
MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark
Computer Vision and Pattern Recognition · Submitted 29 Mar 2024
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim This dataset provides a challenging test-bed for studying multi-camera tracking under diverse real-world complexities and includes an additional input modality of spatially aligned and temporally synchronized RGB and thermal cameras, which enhances the accuracy of multi-camera tracking.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. This task has practical applications in various fields, such as visual surveillance, crowd behavior analysis, and anomaly detection. However, due to the difficulty and cost of collecting and labeling data, existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting, which limits their ability to model real-world dynamics and generalize to diverse camera configurations. To address this issue, we present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments - campus and factory - across various time, weather, and season conditions. This dataset provides a challenging test-bed for studying multi-camera tracking under diverse real-world complexities and includes an additional input modality of spatially aligned and temporally synchronized RGB and thermal cameras, which enhances the accuracy of multi-camera tracking. MTMMC is a super-set of existing datasets, benefiting independent fields such as person detection, re-identification, and multiple object tracking. We provide baselines and new learning setups on this dataset and set the reference scores for future studies. The datasets, models, and test server will be made publicly available.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. | background 1.00 | 0.04 | 0.03 | asserted 1.00 |
| s1 | This task has practical applications in various fields, such as visual surveillance, crowd behavior analysis, and anomaly detection. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | However, due to the difficulty and cost of collecting and labeling data, existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting, which limits their ability to model real-world dynamics and generalize to diverse camera configurations. | gap 0.99 | 0.06 | 0.05 | asserted 1.00 |
| s3 | To address this issue, we present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments - campus and factory - across various time, weather, and season conditions. | contribution 0.97 | 0.94 | 0.18 | asserted 1.00 |
| s4 | This dataset provides a challenging test-bed for studying multi-camera tracking under diverse real-world complexities and includes an additional input modality of spatially aligned and temporally synchronized RGB and thermal cameras, which enhances the accuracy of multi-camera tracking. | contribution 0.82 | 0.77 | 0.16 | asserted 1.00 |
| s5 | MTMMC is a super-set of existing datasets, benefiting independent fields such as person detection, re-identification, and multiple object tracking. | contribution 0.67 | 0.51 | 0.11 | asserted 1.00 |
| s6 | We provide baselines and new learning setups on this dataset and set the reference scores for future studies. | contribution 0.95 | 0.95 | 0.14 | asserted 1.00 |
| s7 | The datasets, models, and test server will be made publicly available. | contribution 0.75 | 0.65 | 0.08 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. This task has practical applications in various fields, such as visual surveillance, crowd behavior analysis, and anomaly detection. However, due to the difficulty and cost of collecting and labeling data, existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting, which limits their ability to model real-world dynamics and generalize to diverse camera configurations. To address this issue, we present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments - campus and factory - across various time, weather, and season conditions. This dataset provides a challenging test-bed for studying multi-camera tracking under diverse real-world complexities and includes an additional input modality of spatially aligned and temporally synchronized RGB and thermal cameras, which enhances the accuracy of multi-camera tracking. MTMMC is a super-set of existing datasets, benefiting independent fields such as person detection, re-identification, and multiple object tracking. We provide baselines and new learning setups on this dataset and set the reference scores for future studies. The datasets, models, and test server will be made publicly available.
Human review
How the literature uses it
1 arXiv paper cite this in 1 sentence
Papers citing it
Cites it as background 1 paper
-
Recent multimodal MOT methods mainly focus on RGB-T fusion and cross-modal association, including MTMMC [ref] , HGT-Track [ref] , UniRTL and UnisMOT [ref] , VT-MOT and PFTrack [ref] , and MCTrack [ref] .
SMAC: Spatial-Modal Joint Modeling and Adaptive Representation Collapse for Multimodal Object Tracking · Multimodal Multi-Object Tracking · 0.97
What it cites · 8 sentences without a confident label
Builds on 1 paper
-
Building upon prior research [ref] , we have developed a conditional attention module to utilize multi-modal data more effectively.
RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation · Implementation Details · 0.87
Uses the method or tool of 4 papers
-
This module uses a BAM attention block [ref] , processing the differential between RGB and thermal features to explicitly highlight the complementary nature of these modalities.
BAM: Bottleneck Attention Module · Implementation Details · 0.99
-
In our experiment, we employed four state-of-the-art trackers: JDE [ref] , QDTrack [ref] , CenterTrack [ref] , and ByteTrack [ref] , and our analysis focuses on three main aspects:
ByteTrack: Multi-Object Tracking by Associating Every Detection Box · Sub Tasks: Detection, Re-ID, and MOT · 0.99
-
Training Our framework is constructed using two well-established public codebases: mmtracking [ref] and fast-reid [ref] .
FastReID: A Pytorch Toolbox for General Instance Re-identification · Implementation Details · 1.00
Uses data or values from 1 paper
-
Specifically, we trained models MTMMC-Person and COCO-Person datasets and then tested their generalization performance using the MOT17 [ref] dataset, which presents a variety of real-world tracking scenarios.
MOT16: A Benchmark for Multi-Object Tracking · Sub Tasks: Detection, Re-ID, and MOT · 0.87
Compares with 1 paper
-
In fig:overview, we present a detailed comparison of our MTMMC dataset with two of the largest datasets in MTMC: MTA [ref] and MMPTrack [ref] .
MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark · Data Characteristics · 0.99
Cites as support 2 papers
-
As a result, existing MTMC benchmarks are either short in duration [ref] , have low video resolution [ref] , or provide inconsistent person IDs [ref] , making them unsuitable for training generic deep trackers for real-world use cases.
An equalised global graphical model-based approach for multi-camera object tracking · Related Work · 0.96
-
As a result, existing MTMC benchmarks are either short in duration [ref] , have low video resolution [ref] , or provide inconsistent person IDs [ref] , making them unsuitable for training generic deep trackers for real-world use cases.
The WILDTRACK Multi-Camera Person Dataset · Related Work · 0.96
Cites as background 6 papers
-
Notably, several offline global association techniques have been developed, such as hierarchical clustering [ref] , correlation clustering [ref] , matrix factorization [ref] , and adaptive locality-based association [ref] .
Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Project · Related Work · 0.88
-
To facilitate this process, various constraints have been employed, including time conflicts [ref] , linear motion patterns [ref] , camera network topology [ref] , geometric cues [ref] , and spatial locality [ref] .
Real-time 3D Deep Multi-Camera Tracking · Related Work · 0.83
-
Recent advancements in MOT, thanks to benchmarks such as MOT17 [ref] , BDD100K [ref] , Waymo [ref] , and TAO [ref] have led to the development of more effective and efficient trackers [ref] .
TrackFormer: Multi-Object Tracking with Transformers · Introduction · 0.93
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.