REVIEW 4 major objections 4 minor 21 references
C$^{2}$INet: Realizing Incremental Trajectory Prediction with Prior-Aware Continual Causal Intervention
T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper claims that a continual causal-intervention module, which estimates scene-specific confounders and stores their priors in a bounded memory queue, reduces catastrophic forgetting and beats prior continual and causal baselines in…
desk verdict Useful continual-learning machinery, but the causal claim does not survive contact with the paper's own equations; the method is closest to a VAE with learned priors plus memory replay. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a backdoor-adjusted variational objective, meaning an objective that estimates the interventional distribution $P(Y \mid \mathrm{do}(X))$ by conditioning on and averaging over the confounder, with a continually updated prior. The named object is the prior queue: a memory of pseudo-features $U_i$ whose encodings $Q(C\mid U_i)$ approximate the scenario-specific distribution of the confounder $P(C)$. Each new task adds a pseudo-feature, the mixture weights $\alpha_k$ are optimized alternately with the prior in a min-max loop, and a pruning rule drops redundant pseudo-features once the queue exceeds capacity $\gamma$. The causal engine is the Gaussian product that merges the trajectory encoder and confounder encoding into $P(Z\mid X,C)$, so predictions are made from the intervened representation.
What would settle it
Train the model on a synthetic dataset where the ground-truth confound is known, such as a rule that shifts all right turns in one scene, then test whether the learned $Q(C\mid X)$ encodes that confound and whether predictions degrade when the confounder channel is removed. If the model performs the same when its 'confounder' is replaced by an arbitrary latent dimension, the causal intervention claim is not doing the work.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that the confounding influence of scene-specific context $C$ on trajectory predictions can be removed continuously, not just in a fixed training set. The authors derive an evidence lower bound for the interventional distribution $P(Y \mid \mathrm{do}(X))$ using a variational encoder $Q(C\mid X)$, and combine the trajectory encoder's Gaussian $P(Z\mid X)$ with the confounder posterior into a mixed latent distribution $P(Z\mid X,C)$ used for prediction. They then keep a representative prior for each task in a memory queue, update it in a min-max loop with mixture weights, and prune redundant pseudo-features when the queue grows too large. In their experiments, C2INet in online and offline modes often delivers the best average ADE/FDE and shows less performance decay on previously seen tasks than the compared baselines.
Load-bearing premise
The load-bearing premise is that a single hidden 'environment' variable, learned only from the observed trajectories, really captures every scene-specific bias, and that the causal-adjustment formulas used to remove it are valid for this model. If that premise is wrong, the intervention step is just an ordinary variational encoder with a stored prior, and the gains may come from the replay memory and regularization instead.
Editorial extensions
If this is right
- If the central claim is right, continual trajectory prediction can adopt causal-debiasing tools that were designed for fixed datasets, without requiring full access to old data.
- The ablation results imply that replay memory alone is not enough: keeping priors diverse through symmetric KL divergence and optimizing mixture weights both contribute to the gain.
- Because the module is plug-and-play, existing RNN- and CNN-based trajectory encoders can be converted to the continual setting by adding the confounder encoder and prior queue.
- Storing pseudo-features rather than raw trajectories keeps memory cost tied to the number of scenes, which would matter for deployment on vehicles or edge devices.
Reading between the lines
- The paper leaves implicit that its causal story is only as strong as the identifying assumption; a safer reading is that the memory queue plus KL regularization is the reliable part, and the confounder encoder may simply be a useful auxiliary variable.
- A concrete next test would be a controlled synthetic domain shift with a known ground-truth confound, checking whether $C$ recovers it and whether removing it is necessary for the reported gains.
- The task-order analysis implies a practical curriculum rule: onboard noisier scenes earlier in the sequence, because later forgetting then dampens their harmful bias.
- The bounded prior queue points toward a deployment pattern where each new city or agent class gets one slot in the queue, making adaptation a matter of learning and pruning a single pseudo-feature.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes C2INet, a continual-learning framework for multi-agent trajectory prediction. The authors model environmental context as a confounder C, use backdoor adjustment to define P(Y|do(X)), and approximate the resulting interventional distribution with a variational objective in which an encoder Q(C|X) estimates C and a memory queue stores scenario-specific priors. They introduce online and offline modes, a min-max training scheme with pruning, and report experiments on ETH-UCY, a synthetic dataset, and SDD, comparing against several continual-learning and causal baselines. The paper's central claim is that C2INet consistently improves trajectory prediction while mitigating confounders and catastrophic forgetting.
Significance. If the causal-intervention formalism were sound, the paper would address a real and currently relevant problem: continual trajectory prediction under environmental shift. The work has practical strengths: it embeds the method in two backbones, evaluates on multiple datasets, includes an ablation study, and reports five-seed averages. However, the theoretical core is not sound as written. The derivation of Eq. (3) loses the latent representation Z and an unknown posterior term, so the optimized objective is not a valid ELBO for the backdoor-adjusted interventional distribution. The continual-prior update in Eq. (9) minimizes a quantity that is not a KL divergence, and the prior in Eq. (8) is constructed from the same posterior it is meant to constrain. Since the manuscript's distinctive contribution is the causal-intervention mechanism, these issues undermine the central claim and reduce the contribution to a regularized continual VAE with a memory queue.
major comments (4)
- [Sec. 2.3, Eq. (3)] The variational derivation is invalid. In the transition between the third and fourth lines of Eq. (3), the expectation over P(Z) is silently replaced by an expectation over Q(C|X) alone, while the factor P(Z|X,C) is dropped; the integration over Z disappears. The paper also drops the term KL(Q(C|X)||P(C|X,Y,Z)) and treats the remaining expression as a lower bound, but dropping a KL term without replacing it by a bound is not a valid ELBO manipulation. Consequently, optimizing Eq. (3) does not implement P(Y|do(X)) under the stated graphical model; it is a conditional VAE objective with an added regularizer. This is load-bearing because the paper's causal-intervention claim rests on Eq. (3).
- [Sec. 3, Eqs. (7) and (9)] The first-order expansion of the KL divergence in Eq. (7) is asserted without conditions on α_{K-1}, and the resulting derivative term, written as −α_{K-1}(P̂_K(C) log(Q_i(C|X)/M_{≤K−1}(C)) − 1), is not a legitimate functional derivative of the KL expression. More seriously, Eq. (9) minimizes KL[P̂_K(C) || Σ_i Q_i(C|X)/M_{≤K−1}(C) − 1], but the second argument is not a probability distribution: it can be negative and does not integrate to one. Therefore the quantity inside the proposed KL is not a valid density, and the optimization problem is not well-defined as a variational inference step. The continual-prior algorithm needs to be redefined with a proper divergence or a correctly normalized surrogate.
- [Sec. 3, Eq. (8)] The prior P(C) is defined as an average of the model's own posterior Q(C|U_i) over trainable pseudo-features U_i. These pseudo-features are then optimized under KL constraints that involve the same posterior family. The prior is therefore not an independent environmental distribution, which is required for the backdoor adjustment in Eq. (1) to be interpreted causally. As a result, the claimed debiasing mechanism collapses into a learned prior in a regularized VAE, and the causal identification assumptions in Sec. 2.2 are untested. The authors need to either provide a causal ground-truth or sensitivity analysis, or explicitly reframe the method as a continual prior-learning method without causal claims.
- [Sec. 5 and Appendix A.3.4/A.3.5] The empirical evaluation does not establish that the reported gains come from the causal mechanism or even from the proposed continual prior, because several critical hyperparameters are tuned on the same tasks used for evaluation. The queue capacity γ is selected via Fig. 8 on ETH-UCY, the task-order sensitivity is measured on the same dataset in Fig. 9, and the weights α are optimized online during training. Since the objective already reduces to a CVAE-like loss, the performance improvement over baselines could be explained by memory replay and KL regularization. The authors should report a validation/tuning protocol separate from test tasks, include variance estimates, and compare against a non-causal variational continual-learning baseline with matched capacity.
minor comments (4)
- [Sec. 2.3, text after Eq. (3)] The sentence 'According to the Markov property, P(Y|X,Z,C) = P(Y|X,Z)P(Z|X,C)' is not a standard use of the Markov property in this graph; the equality mixes a conditional distribution over Y with a density over Z and should be written with the appropriate factorization and integrations, if it is intended at all.
- [Fig. 1 and Sec. 2.3] The notation is inconsistent: Fig. 1 labels a 'Trajectory Prior P(X)' and a 'Distribution Mixture P(Z|X,C)', but the paper's equations use P(C) as the prior and define P(Z|X,C) as a mixture of Gaussians; the caption and the equations should be aligned.
- [Table 1 caption and formatting] The table columns and the role of the leading colon in task names are not clearly explained; the repeated baseline names in the header make the table difficult to read, and the absence of standard deviations makes it hard to assess whether the reported improvements are statistically significant.
- [Algorithm 1, line 12] The condition `if j mod L/⌊2γ⌋ = 0` is ambiguously formatted and could be undefined for some values of γ; please rewrite the modulo condition unambiguously.
Circularity Check
The causal-intervention claim is undercut by construction: the 'environmental prior' (Eq.8) is an average of the model's own posterior Q(C|U), and the ELBO (Eq.3) drops the Z integration and P(Z|X,C), so the optimized objective is a conditional VAE with a self-fitted prior rather than a backdoor-adjusted P(Y|do(X)); the empirical benchmark results remain independent evidence.
-
self definitional
[Section 3, Eq.8 and Eq.9; used in Eq.11]
"P_i(C) = 1/|M_i| * sum_{U_i in M_i} Q(C|U_i), (8) where M_i is the prior set related to the i-th scene stored in the memory queue. ... The prior P*_K(C) = Q(C|U_K) is computed from the trainable pseudo feature set U_K of the current task K, while M_{<=K-1}(C) is the optimal prior maintained up to round K-1."
The prior P(C), which is supposed to represent the environment-specific confounder distribution for backdoor adjustment, is defined in Eq.8 as an average of the model's own variational posterior Q(C|U_i) over trainable pseudo-features. Eq.9 then optimizes U_K (hence the prior) by minimizing KL[P_K(C) || sum_i Q_i(C|X)/M_{<=K-1}(C) - 1]. Therefore the KL constraint in Eq.11 aligns Q(C|X) not with an independently identified confounder prior, but with a transformation of the very same learned posterior Q(C|U). The claimed 'prior-aware' causal intervention reduces to a self-constraint on the encoder; no independent environmental prior enters the derivation, so the debiasing attributed to the causal prior is fitted by construction.
-
other
[Section 2.3, Eq.3]
"= E_{P(Z)Q(C|X)} log P(Y|X,Z,C)P(Z|X,C)P(C)Q(C|X) / [P(C|X,Y,Z)Q(C|X)] = E_{Q(C|X)} log P(Y|X,Z,C)P(C)Q(C|X) / [P(C|X,Y,Z)Q(C|X)] = E[logP(Y|X,Z,C)] + KL(...) - KL(...) >= E[logP(Y|X,Z,C)] - KL(Q(C|X)||P(C)). This is called the ELBO for intervening confounding factors."
The preceding line still contains an expectation over P(Z) and the factor P(Z|X,C); the next equality silently replaces that with an expectation only over Q(C|X) and deletes P(Z|X,C). The paper then names the truncated expression 'the ELBO for intervening confounding factors' and builds the training loss Eq.11 from it. Thus the objective actually optimized is not a valid lower bound on the backdoor-adjusted P(Y|do(X)): it is, by construction, a conditional-VAE objective with the self-referential prior of Eq.8. The central causal claim is therefore equivalent to the chosen objective rather than derived from backdoor adjustment.
full rationale
The empirical evaluation is not itself circular: Table 1 compares the method against baselines on ETH-UCY, Synthetic, and SDD, and the ablation study shows the added KL/weight/pruning modules contribute to the reported ADE/FDE numbers. However, the paper's distinctive claim — that C2INet performs causal intervention on confounding factors — is assembled from self-referential pieces. Eq.8 defines the environmental prior as an average of the model's own posterior over trainable pseudo-features, and Eq.9/10 fit both the prior components and the blending weights to the current task's aggregate posterior, so the 'alignment' of prior and posterior is a constraint of Q toward Q. Independently, Eq.3 drops the Z-integration and P(Z|X,C) while renaming the remainder the ELBO for intervention, so the training loss Eq.11 is not a valid backdoor-adjusted objective. Tuning the queue capacity gamma and the alpha weights on the evaluated tasks is ordinary hyperparameter selection and is not counted as circularity here, but it reinforces that the debiasing mechanism is fitted to the same scenarios. I found no load-bearing self-citation: the cited pseudo-feature and VampPrior ideas (Egorov et al.; Tomczak & Welling) are prior work, and the circularity comes from how those ideas are used to define P(C), not from citation alone. Overall the central causal claim reduces by construction to a conditional VAE with a learned posterior-derived prior, while the trajectory-prediction numbers retain independent empirical content; score 6.
Assumptions & free parameters
free parameters (8)
- prior blending weights alpha_j =
optimized via Eq.10
- pseudo-features U_i =
trainable vectors in memory queue
- trainable noise epsilon for pseudo-features =
not specified, paper writes N(1,0) which is degenerate
- environment weights w_i =
unspecified
- queue capacity gamma =
45, chosen as best in Fig.8
- number of pseudo-features per task |U| =
not specified
- latent dimension d of Q(C|X) =
not specified
- initial prior P_1(C) =
assumed valid, not specified
assumptions (5)
- domain assumption A single latent confounder C is sufficient to model all environment-specific biases and the causal graph C to X, X to Z with C, X to Y with Z is correct.
- domain assumption The backdoor adjustment conditions hold, so P(Y|do(X)) can be expressed as an expectation over P(C) as in Eq.1-2.
- ad hoc to paper The variational lower bound remains valid after dropping the KL(Q(C|X) parallel to P(C|X,Y,Z)) term in Eq.3.
- ad hoc to paper The first-order Taylor expansion of the KL in Eq.7 around small alpha_{K-1} is accurate.
- ad hoc to paper The expression in Eq.9 is a valid KL divergence to a probability distribution.
invented entities (2)
-
Confounder C, the environmental context variable
-
Pseudo-features U_i stored in the prior queue
Cite this review
Pith. "Pith review of C$^{2}$INet: Realizing Incremental Trajectory Prediction with Prior-Aware Continual Causal Intervention." pith.science (2026). https://pith.science/paper/FHFD2JBT
@misc{pith2026241112313,
author = {Pith},
title = {Pith review of: C$^2$INet: Realizing Incremental Trajectory Prediction with Prior-Aware Continual Causal Intervention},
year = {2026},
howpublished = {\url{https://pith.science/paper/FHFD2JBT}},
note = {Machine review of arXiv:2411.12313}
}
abstract
Trajectory prediction for multi-agents in complex scenarios is crucial for applications like autonomous driving. However, existing methods often overlook environmental biases, which leads to poor generalization. Additionally, hardware constraints limit the use of large-scale data across environments, and continual learning settings exacerbate the challenge of catastrophic forgetting. To address these issues, we propose the Continual Causal Intervention (C$^{2}$INet) method for generalizable multi-agent trajectory prediction within a continual learning framework. Using variational inference, we align environment-related prior with posterior estimator of confounding factors in the latent space, thereby intervening in causal correlations that affect trajectory representation. Furthermore, we store optimal variational priors across various scenarios using a memory queue, ensuring continuous debiasing during incremental task training. The proposed C$^{2}$INet enhances adaptability to diverse tasks while preserving previous task information to prevent catastrophic forgetting. It also incorporates pruning strategies to mitigate overfitting. Comparative evaluations on three real and synthetic complex datasets against state-of-the-art methods demonstrate that our proposed method consistently achieves reliable prediction performance, effectively mitigating confounding factors unique to different scenarios. This highlights the practical value of our method for real-world applications.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[3]
Uncertainty-aware short-term motion prediction of traffic actors for autonomous driving
Nemanja Djuric, Vladan Radosavljevic, Henggang Cui, Thi Nguyen, Fang-Chieh Chou, Tsung- Han Lin, Nitin Singh, and Jeff Schneider. Uncertainty-aware short-term motion prediction of traffic actors for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2095–2104,
-
[5]
11 Alexey Kamenev, Lirui Wang, Ollin Boer Bohan, Ishwar Kulkarni, Bilal Kartal, Artem Molchanov, Stan Birchfield, David Nistér, and Nikolai Smolyanskiy. Predictionnet: Real-time joint proba- bilistic traffic prediction for planning, control, and simulation. In 2022 International Conference on Robotics and Automation (ICRA), pp. 8936–8942. IEEE,
work page 2022
-
[7]
Auto-encoding variational bayes
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,
-
[11]
Judea Pearl. The do-calculus revisited. arXiv preprint arXiv:1210.4852,
-
[15]
Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data
Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone. Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, pp. 683–700. Springer,
work page 2020
-
[16]
Sen- tence embedding alignment for lifelong relation extraction
Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. Sen- tence embedding alignment for lifelong relation extraction. arXiv preprint arXiv:1903.02588 ,
arXiv 1903
-
[19]
Spatial-Channel Transformer Network for Trajectory Prediction on the Traffic Scenes
Jingwen Zhao, Xuanpeng Li, Qifan Xue, and Weigong Zhang. Spatial-channel transformer network for trajectory prediction on the traffic scenes. arXiv preprint arXiv:2101.11472,
-
[20]
Long short term memory for driver intent prediction
Alex Zyner, Stewart Worrall, James Ward, and Eduardo Nebot. Long short term memory for driver intent prediction. In 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 1484–1489. IEEE,
work page 2017
Show all 21 references
-
[21]
eth" task, the task complexity increases significantly, resulting in an ADE of over 1.3 for all models. Additionally, the performance on the previously completed
14 A A PPENDIX A.1 T RAINING PROCESS The following Alg.1 provides a detailed description of the training process for the proposed C2INet method. Algorithm 1 Training Process of C2INet. Input: The training trajectories from K task domains {Xi, Yi}K i=1, with a maximum prior que...
2018
-
[2007]
Grip: Graph-based interaction-aware trajectory prediction
Xin Li, Xiaowen Ying, and Mooi Choo Chuah. Grip: Graph-based interaction-aware trajectory prediction. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 3960–3966. IEEE,
2019
-
[2012]
Lc-llm: Explainable lane-change intention and trajectory predictions with large language models
Mingxing Peng, Xusen Guo, Xianda Chen, Meixin Zhu, Kehua Chen, Xuesong Wang, Yinhai Wang, et al. Lc-llm: Explainable lane-change intention and trajectory predictions with large language models. arXiv preprint arXiv:2403.18344,
-
[2015]
It is not the journey but the destination: Endpoint conditioned tra- jectory prediction
Karttikeya Mangalam, Harshayu Girase, Shreyas Agarwal, Kuan-Hui Lee, Ehsan Adeli, Jitendra Malik, and Adrien Gaidon. It is not the journey but the destination: Endpoint conditioned tra- jectory prediction. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, Au...
2020
-
[2016]
Expanding the deployment envelope of behavior prediction via adaptive meta-learning
Boris Ivanovic, James Harrison, and Marco Pavone. Expanding the deployment envelope of behavior prediction via adaptive meta-learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 7786–7793. IEEE,
2023
-
[2017]
Adversarial autoencoders
Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders. arXiv preprint arXiv:1511.05644,
-
[2018]
Learning social eti- quette: Human trajectory understanding in crowded scenes
Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. Learning social eti- quette: Human trajectory understanding in crowded scenes. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part ...
2016
-
[2019]
Human trajectory prediction via counterfactual analysis
Guangyi Chen, Junlong Li, Jiwen Lu, and Jie Zhou. Human trajectory prediction via counterfactual analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 9824–9833, 2021a. Weihuang Chen, Fangfang Wang, and Hongbin Sun. S2tnet: Spatio-temporal ...
2020
-
[2020]
Multimodal trajectory predictions for autonomous driving using deep convolutional networks
Henggang Cui, Vladan Radosavljevic, Fang-Chieh Chou, Tsung-Han Lin, Thi Nguyen, Tzu-Kuo Huang, Jeff Schneider, and Nemanja Djuric. Multimodal trajectory predictions for autonomous driving using deep convolutional networks. In 2019 international conference on robotics and autom...
2019
-
[2021]
Intention- aware long horizon trajectory prediction of surrounding vehicles using dual lstm networks
Long Xin, Pin Wang, Ching-Yao Chan, Jianyu Chen, Shengbo Eben Li, and Bo Cheng. Intention- aware long horizon trajectory prediction of surrounding vehicles using dual lstm networks. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 1441–14...
2018
-
[2022]
Multi-head attention based probabilistic vehicle trajectory prediction
Hayoung Kim, Dongchan Kim, Gihoon Kim, Jeongmin Cho, and Kunsoo Huh. Multi-head attention based probabilistic vehicle trajectory prediction. In 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1720–1725. IEEE,
2020
-
[2023]
Spatio-temporal graph transformer networks for pedestrian trajectory prediction
13 Cunjun Yu, Xiao Ma, Jiawei Ren, Haiyu Zhao, and Shuai Yi. Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16, pp. 507–523. Springer,
2020
-
[2024]
Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence
12 Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint arXiv:1810.11910,
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.