OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction

Alexandre Alahi; Kaouther Messaoud; Lan Feng; Po-Chien Luan; Yang Gao

REVIEW 3 major objections 5 minor 17 references

OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single FPS number unlocks zero-shot trajectory prediction.

desk verdict A clean controlled study of FPS conditioning in trajectory prediction, but the headline 70% zero-shot claim is confounded and the ETH-UCY SOTA statement contradicts the paper's own table. read the letter →

arxiv 2507.23657 v1 pith:ZSSO2BHT submitted 2025-07-31 cs.CV

Yang Gao , Po-Chien Luan , Kaouther Messaoud , Lan Feng , Alexandre Alahi This is my paper

classification cs.CV

keywords humantrajectorypredictionzero-shottransferframerateconditioningpre-trainingtransformertemporaldynamicsmulti-modalUniHuMotion++

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that pre-trained human trajectory predictors fail on unseen datasets mainly because they are locked to the frame rate and horizon they were trained on, not because transfer itself is impossible. It isolates temporal shift from dataset shift and shows that explicitly feeding the model the frame rate as a small piece of metadata is more effective than data-unaware discrete models or continuous-time models. The proposed OmniTraj encodes frames per second (FPS) into its input tokens, pre-trains on 859 hours of heterogeneous motion data from 12 datasets, and reduces zero-shot prediction error by over 70% on unseen datasets. After fine-tuning, it also reaches best published results on NBA, JTA, WorldPose, and competitive results on ETH-UCY.

What carries the argument

The load-bearing component is the Frame Rate Encoding module, defined as $E_r = \text{MLP}_r(r)$, which converts the scalar frame rate $r$ into a latent vector added element-wise to the multi-modal input tokens before the transformer processes them. Around this sit three supporting structures: UniHuMotion++, a data container that stores trajectories with their FPS and horizon metadata and unifies 859 hours of motion from 12 datasets; the Cross-Modality Encoder that fuses trajectory, pose, and bounding-box tokens with shared parameters; and the Decoupled Interaction Modules, a Historical Interaction Encoder plus a Predictive Interaction Decoder with ego-centric cross-attention that separately models past and future social interactions. A spatial-temporal masking strategy during pre-training makes the model robust to missing and variable-length observations. The FPS embedding does the work of normalizing temporal resolution so the transformer does not have to rediscover it.

What would settle it

Evaluate OmniTraj zero-shot on a dataset whose FPS lies outside the pre-training range, for example 30 FPS when the model saw only 1-25 FPS during training; if prediction error jumps sharply, the claim that explicit FPS conditioning handles arbitrary temporal dynamics is falsified.

Watch

Extended reading notes

Core claim

The central claim is that temporal generalization in trajectory prediction can be bought cheaply: rather than learning frame-rate-agnostic internals, the model should be told the temporal resolution of its input. OmniTraj embeds the scalar FPS value with a small MLP and adds that embedding element-wise to the input tokens, so the same transformer can interpret motion recorded at 1, 2.5, 5, 10, or 25 FPS. In a controlled zero-shot study on NBA data, where the model is trained on 5 and 2.5 FPS setups and evaluated on a 1 FPS setup, this FPS-aware design reduces MinADE20 by 36.9% and MinFDE20 by 51.0% relative to the same model without the FPS encoder, and it beats both a discrete pre-trained baseline and a continuous-time stochastic differential equation model. The paper also shows the same mechanism transfers to entirely unseen datasets: OmniTraj lowers zero-shot ADE on Trajnet++ and SDD by more than 70% compared with the prior pre-trained model, and its multi-modal pre-training gives it an extra edge even when only trajectories are used at inference.

Load-bearing premise

The controlled zero-shot results assume that a single scalar frame-rate value captures all relevant temporal differences between datasets; if motion speed, sensor noise, or irregular frame sampling also shift, the FPS embedding alone may not generalize.

Editorial extensions

If this is right

Pre-trained trajectory models could be deployed directly to new sensors, cameras, or recording hardware with different frame rates, without retraining or fine-tuning on the target setup.
Explicit metadata conditioning may replace complex continuous-time internal dynamics as the default strategy for temporal generalization in trajectory forecasting.
Multi-modal pre-training improves zero-shot transfer even when only trajectory data is available at inference, which matters for real deployments where pose or bounding boxes are not reliably available.
In few-shot settings the pre-trained model outperforms a prior model fine-tuned on 200 samples using only 2 training samples, suggesting the learned representation is already near the target task.
A model conditioned on FPS and horizon in this way remains comparatively robust when only two observed frames are available, degrading 11% in ADE versus 57% for a dataset-specific baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves implicit that the same recipe could be applied to other nuisance metadata, such as sensor noise level, observation length, or camera viewpoint, turning each into a cheap conditioning input and potentially enabling zero-shot transfer along those axes too.
The FPS scalar captures a global temporal scale but not irregular sampling patterns or dropped frames; conditioning on per-timestep timestamps or a frame-drop mask would be a natural testable extension that the current design does not cover.
The controlled NBA study varies only FPS while keeping agents and motion style similar, so the broad 'varying temporal dynamics' claim would be stronger if tested across datasets that differ simultaneously in speed, scene type, and FPS, which the Trajnet++ and SDD results begin to do.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, the paper's core experiment is worth reading: they partition NBA motion into two temporal setups (5 FPS, 2.5 FPS) and test on a third (1 FPS), isolating frame-rate shift from dataset shift. In that controlled setting, explicit FPS conditioning beats both a discrete model that ignores FPS and a continuous-time SDE baseline. That is a clean result and a useful methodological contribution. Second, the paper's headline numbers overshoot. The 'over 70% reduction' in the abstract compares OmniTraj pre-trained on 12 datasets to Multi-Transmotion pre-trained on 7, so data scale and architecture are confounded with the FPS mechanism. And the 'state-of-the-art on ETH-UCY' claim is contradicted by their own Table 9, where SingularTrajectory achieves 0.21/0.32 versus OmniTraj's 0.21/0.35.

The controlled study is not synthetic in a pejorative sense—it uses real basketball tracking data resampled to different temporal resolutions. But it is a single dataset and a single domain, so the evidence that the FPS embedding generalizes across datasets with different motion statistics is missing. The actual cross-dataset zero-shot evaluations (Trajnet++ and SDD) are both at 2.5 FPS, a rate already present in the pretraining mixture (JRDB-Pose, JTA), so those results do not test frame-rate generalization at all. They test whether a bigger pretraining corpus and a better architecture transfer. That's not nothing, but it's not the claimed FPS effect.

Other soft spots: the FPS-encoder variants in Table 2 were presumably selected after looking at validation performance, which is standard researcher degrees of freedom but worth flagging. Reproducibility is partial—code is public but without a commit hash, and the data framework release is deferred. The paper is transparent about many split choices and provides a detailed supplement.

Who is this for? People building deployable trajectory predictors, especially for robotics and surveillance, will find the FPS-conditioning idea directly actionable and the UniHuMotion++ framework useful. It is not a theoretical advance. The paper deserves a serious referee: the controlled study is a legitimate experiment, and the overclaims are fixable in revision. I would recommend conditional acceptance with a requirement to re-analyze the 70% claim, to either correct or qualify the ETH-UCY statement, and to add an experiment that varies FPS across real datasets rather than only within NBA.

Referee Report

3 major / 5 minor

Summary. The paper proposes OmniTraj, a Transformer-based human trajectory prediction model pre-trained on a large heterogeneous motion dataset (UniHuMotion++, 12 datasets, 859 hours). The central idea is to condition the model on a scalar frame-rate value by adding an MLP-encoded embedding to the input tokens, motivated by a controlled NBA resampling study (Setups 1/2 at 5 and 2.5 FPS, tested on 1 FPS) in which FPS-aware encoders improve prediction. The authors report zero-shot results on Trajnet++ and SDD where OmniTraj reduces ADE by about 70% relative to Multi-Transmotion, and fine-tuned results on NBA, JTA, WorldPose, and ETH-UCY.

Significance. If fully supported, the paper would make a useful empirical contribution: it offers a simple, low-cost mechanism (frame-rate embedding) for adapting pre-trained trajectory models to new temporal setups, along with a publicly released codebase and a unified data framework. The controlled ablation with several FPS-encoder variants (FiLM, codebook, MLP-concat, MLP-latent-sum) and the comparison against a continuous-time baseline (TrajSDE) are valuable. The strongest direct evidence is the NBA Setup 1/2/3 study, where the MLP latent-sum FPS encoder improves MinADE20 from 1.87 to 1.18. However, the headline claims go beyond this evidence: the 70% reduction is drawn from a confounded comparison, the cross-dataset zero-shot evaluation does not actually vary FPS, and the ETH-UCY state-of-the-art claim is contradicted by the paper's own Table 9.

major comments (3)

[Abstract, Conclusion, Table 1, Table 2] The claim that 'explicitly conditioning on the frame rate enables OmniTraj to achieve state-of-the-art zero-shot transfer performance, reducing prediction error by over 70%' is not supported by the controlled experiment. The only experiment that varies the FPS encoder, Table 2, reports MinADE20 of 1.87 (w/o FPS-encoder) versus 1.18 (MLP latent-space summation), a 36.9% reduction (and 51.0% for MinFDE20). The 70%+ figure in Table 1 compares OmniTraj (multi-modal pre-training) to Multi-Transmotion, which involves different pre-training data scale, architecture, and training objective; the improvement cannot be attributed to FPS conditioning. Please separate the data-scale/architecture comparison from the FPS-mechanism ablation and revise the abstract and conclusion accordingly.
[Datasets and Results; Table 1 and Table 2] The only experiment that varies FPS is the NBA resampling study (Setup 1/2/3). The zero-shot evaluations on Trajnet++ and SDD in Table 1 are both at 2.5 FPS, a frame rate already present in the pre-training corpus (e.g., JRDB-Pose and JTA in Table 8). Thus the paper has not demonstrated that FPS conditioning transfers across datasets with different motion statistics, sensor noise, or frame-drop patterns; in the NBA study, the learned scalar embedding may simply act as a setup tag. A cross-dataset evaluation with a genuinely unseen FPS/horizon combination is needed, or the claims should be explicitly restricted to frame-rate variation within a dataset.
[Supplementary Material, Table 9 and Abstract] The abstract's claim that OmniTraj 'achieves state-of-the-art results on four datasets, including ... ETH-UCY' is contradicted by the paper's own Table 9. OmniTraj's average ADE/FDE is 0.21/0.35, while SingularTrajectory reports 0.21/0.32 and EqMotion reports 0.21/0.35. OmniTraj does not achieve the best average FDE on this benchmark, so the state-of-the-art claim for ETH-UCY is not supported by the reported numbers.

minor comments (5)

[Frame Rate Encoding] The scalar FPS embedding is added element-wise to all input tokens, but the paper does not discuss whether the same scalar is used for all agents in a scene or how irregular sampling (e.g., missing frames, per-agent frame drops) would be handled. A sentence acknowledging this limitation would clarify the scope of the mechanism.
[Table 2] Please state explicitly whether all models in the NBA Setup 1/2/3 study are trained from scratch or initialized from pre-trained checkpoints. The current text says 'All models are trained on a mixture of Setup 1 and 2,' which leaves the role of pre-training in this controlled study ambiguous.
[Tables 3-5] The reported improvements over the previous state of the art are small (e.g., NBA MinADE20 0.73 vs. 0.75 in Table 3, JTA ADE 0.90 vs. 0.91 in Table 4) and no variance or significance information is given. Please report standard deviations or at least note that these differences are within typical run-to-run variability.
[Figures 3 and 4] The captions of Figures 3 and 4 are identical, and the axes are unclear: one panel is labeled 'Data percentage' while the text describes 'Number of samples used for few-shot learning.' Please make the captions distinct and define the axes.
[References] There are several typographical issues in the reference list, e.g., 'V on Marcard' should be 'Von Marcard,' and the Social-GAN row in Table 9 contains '0.61/1/21' which appears to be a typo for '0.61/1.21.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the FPS-conditioning claim is supported by a held-out temporal-setup ablation, not by construction or self-citation.

full rationale

OmniTraj is an empirical pre-training paper rather than a derivation, and its load-bearing claims do not reduce to their inputs by construction. The FPS encoder Er = MLPr(r) is trained only on Setups 1 and 2 (r = 5, 2.5) and tested on Setup 3 (r = 1), so the Table 2 improvement is a genuine extrapolation along the frame-rate axis rather than a fitted parameter being renamed as a prediction. The zero-shot results in Table 1 are on Trajnet++ and SDD, which the paper explicitly excludes from pre-training, and the 70% gain is an empirical comparison against the Multi-Transmotion baseline, not an identity. The authors' prior work (Multi-Transmotion, UniHuMotion, Social-Transmotion) is cited for baselines, data splits, and the UniHuMotion++ framework, but no uniqueness theorem or central premise is justified solely by those self-citations; the key design conclusion is validated by an ablation against TrajSDE, an external continuous-time baseline. The abstract's phrasing that FPS conditioning 'enables' the 70% reduction is not strictly supported by Table 1, since both target datasets are at 2.5 FPS and the FPS variable is only varied in the synthetic NBA study; however, that is an external-validity or framing concern, not circularity, because the paper's equations do not make the target result equivalent to its inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters beyond learned weights; the model's 7.5M parameters are trained, not fitted constants. Key assumptions are domain-level (sufficiency of FPS, pooling validity) rather than mathematical axioms.

assumptions (3)

domain assumption A single scalar frame-rate value r is a sufficient conditioning signal to normalize temporal dynamics across datasets.
The FPS encoder E_r = MLP_r(r) is added to input tokens; if higher-order temporal statistics (e.g., motion blur, frame drop patterns) matter, this conditioning could be insufficient. Introduced in Section 'Frame Rate Encoding'.
domain assumption Trajectories from the 12 datasets can be pooled into a common data container without explicit coordinate-frame normalization or scene-context alignment.
UniHuMotion++ mixes NBA court coordinates, ETH-UCY world coordinates, and driving-agent coordinates. The paper does not describe a global coordinate normalization, so cross-dataset transfer relies on implicit standardization, which may affect zero-shot results.
domain assumption Downsampling or upsampling trajectories to a common representation captures the underlying motion without aliasing artifacts.
The framework resamples data to different FPS and horizons; no anti-aliasing or interpolation details are given, so reliability of resampled trajectories is assumed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction." pith.science (2026). https://pith.science/paper/ZSSO2BHT

@misc{pith2026250723657,
  author       = {Pith},
  title        = {Pith review of: OmniTraj: Pre-Training on Heterogeneous Data for Adaptive and Zero-Shot Human Trajectory Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZSSO2BHT}},
  note         = {Machine review of arXiv:2507.23657}
}

read the original abstract

While large-scale pre-training has advanced human trajectory prediction, a critical challenge remains: zero-shot transfer to unseen dataset with varying temporal dynamics. State-of-the-art pre-trained models often require fine-tuning to adapt to new datasets with different frame rates or observation horizons, limiting their scalability and practical utility. In this work, we systematically investigate this limitation and propose a robust solution. We first demonstrate that existing data-aware discrete models struggle when transferred to new scenarios with shifted temporal setups. We then isolate the temporal generalization from dataset shift, revealing that a simple, explicit conditioning mechanism for temporal metadata is a highly effective solution. Based on this insight, we present OmniTraj, a Transformer-based model pre-trained on a large-scale, heterogeneous dataset. Our experiments show that explicitly conditioning on the frame rate enables OmniTraj to achieve state-of-the-art zero-shot transfer performance, reducing prediction error by over 70\% in challenging cross-setup scenarios. After fine-tuning, OmniTraj achieves state-of-the-art results on four datasets, including NBA, JTA, WorldPose, and ETH-UCY. The code is publicly available: https://github.com/vita-epfl/omnitraj

Figures

Figures reproduced from arXiv: 2507.23657 by the authors.

**Figure 1.** OmniTraj: A pre-trained trajectory predictor that adapts to varying frame rates and horizons while excelling in zero-shot transfer. them inflexible when adapting to new scenarios with different frame setups, as shown in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** OmniTraj: a decoupled interactive transformer with explicit frame-rate awareness. The model applies an FPS embedding to input tokens for temporal conditioning. The decoupled design first uses the Historical Interaction Encoder (HIE) to process observed social dynamics. Then, the Predictive Interaction Decoder (PID) models future interactions using learned queries and an ego-centric cross-attention mechanism to refin… view at source ↗

**Figure 4.** Few-shot learning performance on the SDD (Robicquet et al. 2016) dataset. Models Trajnet++ SDD ADE (gain) ADE (gain) Multi-Transmotion (Gao, Luan, and Alahi 2024) 3.40 3.58 OmniTraj (Traj-only pre-training) 1.57 (53.8%) 1.91 (46.6%) OmniTraj (Multi-modal pre-training) 1.01, (70.2%) 0.93, (74.0%) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Qualitative results on the NBA (Linou, Linou, and de Boer 2016) dataset. Models Input modality ADE/FDE Social-LSTM (Alahi et al. 2016) T 2.66/6.30 Directional-LSTM (Kothari, Kreiss, and Alahi 2021) T 2.62/6.25 Autobots (Girgis et al. 2022) T 2.84/5.80 Social-Transmotio…

**Figure 6.** Figure 6: Relationship between performance and data percentage used in pre-training. Data Scaling Effect To examine the data scalability, we pre-train our model on different data proportions and evaluate its validation performance. This analysis examines how the model benefits …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 12 canonical work pages

[6]

IEEE Robotics and Automation Letters, 6(2): 295–302

Multimodal deep generative models for trajectory prediction: A conditional variational autoencoder approach. IEEE Robotics and Automation Letters, 6(2): 295–302. 2 Ivanovic, B.; Song, G.; Gilitschenski, I.; and Pavone, M

work page
[9]

In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14424–14432

Social-stgcnn: A social spatio-temporal graph con- volutional neural network for human trajectory prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14424–14432. 2, 6, 12 Monti, A.; Porrello, A.; Calderara, S.; Coscia, P.; Ballan, L.; and Cucchiara, R. 2022. How many observations are enough? knowledge distill...

work page 2022
[12]

InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part XVIII 16, 683–700

Trajectron++: Dynamically-feasible trajectory fore- casting with heterogeneous data. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part XVIII 16, 683–700. Springer. 2, 6, 11, 12 Schmerling, E.; Leung, K.; V ollprecht, W.; and Pavone, M

work page 2020
[289]

12 Bae, I.; Park, J.-H.; and Jeon, H.-G

Springer. 12 Bae, I.; Park, J.-H.; and Jeon, H.-G. 2022b. Non-probability sampling network for stochastic human trajectory predic- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 6477–6487. 12 Bae, I.; Park, Y .-J.; and Jeon, H.-G. 2024. SingularTrajec- tory: Universal Trajectory Predictor Using Diffusion Mode...

arXiv 2024
[565]

3, 5, 11, 12 Rudenko, A.; Palmieri, L.; Herman, M.; Kitani, K

Springer. 3, 5, 11, 12 Rudenko, A.; Palmieri, L.; Herman, M.; Kitani, K. M.; Gavrila, D. M.; and Arras, K. O. 2020. Human motion tra- jectory prediction: A survey. The International Journal of Robotics Research, 39(8): 895–935. 2 Saadatnejad, S.; Gao, Y .; Messaoud, K.; and Alahi, A. 2024. Social-Transmotion: Promptable Human Trajectory Predic- tion. In I...

work page 2020
[1805]

2 Caesar, H.; Bankiti, V .; Lang, A

PMLR. 2 Caesar, H.; Bankiti, V .; Lang, A. H.; V ora, S.; Liong, V . E.; Xu, Q.; Krishnan, A.; Pan, Y .; Baldan, G.; and Beijbom, O

work page
[2013]

6m: Large scale datasets and predictive methods for 3d human sensing in natural environments

Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intel- ligence, 36(7): 1325–1339. 3, 11, 12 Ivanovic, B.; Leung, K.; Schmerling, E.; and Pavone, M

work page
[2014]

% % % % JRDB-Pose (Vendrow et al. 2023) R 2.5 9 12 1.1h

with an initial learning rate of 1 × 10−4, which de- cays by a factor of 0.1 after 80% of the total 30 epochs. Pre-training is conducted on six NVIDIA H100 GPUs, each with 80GB of memory. For supervision, we use L2 loss to guide the predicted trajectory outputs. Regarding masking strategies, we adopt a 30% modality masking (Saadatnejad et al. 2024) to lea...

work page 2024

Show all 17 references

[2016]

In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Nether- lands, October 11-14, 2016, Proceedings, Part VIII 14, 549–

Learning social etiquette: Human trajectory under- standing in crowded scenes. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Nether- lands, October 11-14, 2016, Proceedings, Part VIII 14, 549–

2016
[2018]

In 2018 IEEE International Con- ference on Robotics and Automation (ICRA) , 3399–3406

Multimodal probabilistic model-based planning for human-robot interaction. In 2018 IEEE International Con- ference on Robotics and Automation (ICRA) , 3399–3406. IEEE. 2 Sun, J.; Li, Y .; Chai, L.; Fang, H.-S.; Li, Y .-L.; and Lu, C

2018
[2020]

In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 11621–11631

nuscenes: A multimodal dataset for autonomous driv- ing. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 11621–11631. 3, 5, 7, 11, 12 Chen, H.; Wang, J.; Shao, K.; Liu, F.; Hao, J.; Guan, C.; Chen, G.; and Heng, P.-A. 2023. Traj-mae: Mas...

2023 arXiv
[2021]

In 2020 25th international conference on pattern recognition (ICPR), 10335–10342

Transformer networks for trajectory forecasting. In 2020 25th international conference on pattern recognition (ICPR), 10335–10342. IEEE. 2 Gu, T.; Chen, G.; Li, J.; Lin, C.; Rao, Y .; Zhou, J.; and Lu, J. 2022a. Stochastic Trajectory Prediction via Motion Indeter- minacy Diffu...

2020 arXiv
[2022]

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 6467–

Human Trajectory Prediction With Momentary Ob- servation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 6467–
[2023]

Advances in Neural Information Processing Systems, 36: 27582–27593

trajdata: A unified interface to multiple human trajec- tory datasets. Advances in Neural Information Processing Systems, 36: 27582–27593. 2 Jiang, T.; Billingham, J.; M ¨uksch, S.; Zarate, J.; Evans, N.; Oswald, M. R.; Polleyfeys, M.; Hilliges, O.; Kaufmann, M.; and Song, J. ...

2024 arXiv
[2024]

2013), AMASS (Mahmood et al

methodology to split NBA SportVU (Linou, Linou, and de Boer 2016), Human3.6M (Ionescu et al. 2013), AMASS (Mahmood et al. 2019), and 3DPW (V on Marcard et al. 2018). Similarly, we followed ScenarioNet (Li et al. 2023a) for NuScenes (Caesar et al. 2020), WOMD (Sun et al. 2020),...

2016
[3920]

11 Li, R.; Li, C.; Ren, D.; Chen, G.; Yuan, Y .; and Wang, G. 2023b. Bcdiff: Bidirectional consistent diffusion for instan- taneous trajectory prediction. Advances in Neural Informa- tion Processing Systems, 36: 14400–14413. 2 Li, Y .; Li, C.; Lv, R.; Li, R.; Yuan, Y .; and Wa...

2025 arXiv
[6476]

2 Sun, P.; Kretzschmar, H.; Dotiwalla, X.; Chouard, A.; Pat- naik, V .; Tsui, P.; Guo, J.; Zhou, Y .; Chai, Y .; Caine, B.; Va- sudevan, V .; Han, W.; Ngiam, J.; Zhao, H.; Timofeev, A.; Ettinger, S.; Krivokon, M.; Gao, A.; Joshi, A.; Zhao, S.; Cheng, S.; Zhang, Y .; Shlens, J....

2020 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [6]

IEEE Robotics and Automation Letters, 6(2): 295–302

Multimodal deep generative models for trajectory prediction: A conditional variational autoencoder approach. IEEE Robotics and Automation Letters, 6(2): 295–302. 2 Ivanovic, B.; Song, G.; Gilitschenski, I.; and Pavone, M

work page

[2] [9]

In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14424–14432

Social-stgcnn: A social spatio-temporal graph con- volutional neural network for human trajectory prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14424–14432. 2, 6, 12 Monti, A.; Porrello, A.; Calderara, S.; Coscia, P.; Ballan, L.; and Cucchiara, R. 2022. How many observations are enough? knowledge distill...

work page 2022

[3] [12]

InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part XVIII 16, 683–700

Trajectron++: Dynamically-feasible trajectory fore- casting with heterogeneous data. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part XVIII 16, 683–700. Springer. 2, 6, 11, 12 Schmerling, E.; Leung, K.; V ollprecht, W.; and Pavone, M

work page 2020

[4] [289]

12 Bae, I.; Park, J.-H.; and Jeon, H.-G

Springer. 12 Bae, I.; Park, J.-H.; and Jeon, H.-G. 2022b. Non-probability sampling network for stochastic human trajectory predic- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 6477–6487. 12 Bae, I.; Park, Y .-J.; and Jeon, H.-G. 2024. SingularTrajec- tory: Universal Trajectory Predictor Using Diffusion Mode...

arXiv 2024

[5] [565]

3, 5, 11, 12 Rudenko, A.; Palmieri, L.; Herman, M.; Kitani, K

Springer. 3, 5, 11, 12 Rudenko, A.; Palmieri, L.; Herman, M.; Kitani, K. M.; Gavrila, D. M.; and Arras, K. O. 2020. Human motion tra- jectory prediction: A survey. The International Journal of Robotics Research, 39(8): 895–935. 2 Saadatnejad, S.; Gao, Y .; Messaoud, K.; and Alahi, A. 2024. Social-Transmotion: Promptable Human Trajectory Predic- tion. In I...

work page 2020

[6] [1805]

2 Caesar, H.; Bankiti, V .; Lang, A

PMLR. 2 Caesar, H.; Bankiti, V .; Lang, A. H.; V ora, S.; Liong, V . E.; Xu, Q.; Krishnan, A.; Pan, Y .; Baldan, G.; and Beijbom, O

work page

[7] [2013]

6m: Large scale datasets and predictive methods for 3d human sensing in natural environments

Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intel- ligence, 36(7): 1325–1339. 3, 11, 12 Ivanovic, B.; Leung, K.; Schmerling, E.; and Pavone, M

work page

[8] [2014]

% % % % JRDB-Pose (Vendrow et al. 2023) R 2.5 9 12 1.1h

with an initial learning rate of 1 × 10−4, which de- cays by a factor of 0.1 after 80% of the total 30 epochs. Pre-training is conducted on six NVIDIA H100 GPUs, each with 80GB of memory. For supervision, we use L2 loss to guide the predicted trajectory outputs. Regarding masking strategies, we adopt a 30% modality masking (Saadatnejad et al. 2024) to lea...

work page 2024