REVIEW 2 major objections 6 minor 45 references
Temporal Knowledge Propagation for Image-to-Video Person Re-identification
T0 review · 2 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Propagating temporal knowledge from a video encoder into an image encoder closes much of the image-to-video person re-identification gap.
desk verdict Solid I2V Re-ID paper with large gains and strong ablations; the 'temporal knowledge' mechanism is plausible but not fully isolated from teacher capacity. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the TKP loss $L_{TKP}=L^F_{TKP}+L^D_{TKP}$. Here $L^F_{TKP}$ is the mean squared error between image features $i_{nt}$ and video frame features $f_{nt}$ for every frame $t$ of every clip $n$, and $L^D_{TKP}$ is the squared Frobenius norm of the difference between the image distance matrix $D^{img}\in\mathbb{R}^{NT\times NT}$ and the video frame distance matrix $D^{vid}\in\mathbb{R}^{NT\times NT}$. These losses are added to classification and integrated triplet losses, and they carry the temporal knowledge from the video representation network (ResNet-50 with non-local blocks) to the image representation network (plain ResNet-50), while the TKP gradient is blocked from updating the video network.
What would settle it
Train TKP on MARS, then test with query images taken from a different camera or from later video frames rather than the first frame; if top-1 accuracy drops back to the no-transfer baseline (or below the 8.5-point improvement), the transferred temporal knowledge does not generalize beyond the training video distribution.
Extended reading notes
Core claim
The central discovery is that temporal knowledge—resilience to occlusion, blur, and pose variation that a video model acquires by seeing frames in relation to one another—can be transferred into a single-image feature extractor. The transfer is achieved by the TKP loss, which combines a feature-level mean squared error between image features and corresponding video frame features with a distance-level Frobenius norm between the cross-sample Euclidean distance matrices of the two modalities. Both image and video networks are ResNet-50 based; the video network adds non-local blocks, and during training the TKP loss is not back-propagated through the video network, preserving its temporal modeling ability. The result is that image features pay attention to more of the person foreground and the image-video feature distributions align, which directly improves image-to-video retrieval.
Load-bearing premise
The method assumes that the temporal knowledge a video network learns from training frames can be transferred to a still-image encoder by fitting its outputs, and that this transfer holds for query images that may come from different cameras, poses, or image quality than the training frames.
Editorial extensions
If this is right
- I2V Re-ID systems can use a single still image as query while the gallery remains video, with accuracy approaching video-to-video levels.
- The transfer improves I2I Re-ID as well (MARS mAP from 49.2 to 55.0), so the robustness gain lives in the image features, not just the matching protocol.
- Any pair of teacher/student architectures that differ in temporal modeling can adopt the same two-term transfer loss.
- The method achieves state-of-the-art I2V results on MARS (top-1 75.6, mAP 65.1) and Duke (top-1 77.9, mAP 75.9) without changing test-time computation.
Reading between the lines
- The success of combining feature and distance distillation suggests a general recipe for cross-modal capability transfer: match both the raw outputs and the relational structure of the embedding space; the same recipe could apply to transferring temporal knowledge to single-image action recognition or person search from a single frame.
- Because adding TKP to the baseline also lifts I2I performance, the method indirectly indicates that a portion of the I2V gap is feature-asymmetry rather than missing information; an adversarial or normalization-based alignment might recover part of the same gain.
- A natural extension is to train the image network on frames from multiple camera domains while keeping the video network fixed, then test whether the transferred robustness survives a domain shift in the query images.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Temporal Knowledge Propagation (TKP) for image-to-video person re-identification (I2V Re-ID). The method trains an image representation network (ResNet-50) and a video representation network (ResNet-50 with non-local blocks) jointly. The TKP loss has two terms: a feature-matching term (Eq. 3, MSE between image frame features and video frame features) and a distance-matching term (Eq. 4, matching cross-sample distance matrices), with gradients of both terms blocked from the video network. These are combined with classification and integrated triplet losses. At test time, query image features come from the image network and gallery video features from the video network. Experiments on MARS, Duke, and iLIDS-VID report large gains over a baseline (MARS: 67.1% to 75.6% top-1, 55.5% to 65.1% mAP) and state-of-the-art comparisons. Extensive ablations cover the non-local blocks, gradient blocking, a pre-trained teacher, triplet-loss variants, and clip length T.
Significance. If the mechanism claim is accepted, the paper is a solid empirical contribution: it reports consistent, substantial improvements over a strong baseline on two multi-camera datasets, includes a well-designed gradient-blocking analysis, and releases code. The ablation suite is unusually thorough for this area, and the I2I/V2V comparisons show that the image features improve without degrading the video features. The main weakness is that the central conceptual attribution of the gains to 'temporal knowledge' is not fully isolated from capacity effects, because the non-local teacher differs from the no-non-local teacher in both temporal modeling and parameter count. The benchmark numbers themselves are credible and likely reproducible, but the distinct contribution of temporal modeling needs one or two additional control experiments before the paper's core claim can be considered established.
major comments (2)
- [Section 4.8, Table 6] The central claim that the additional gain enabled by non-local blocks is due to temporal knowledge is confounded by capacity and by a change in the distillation target. In the no-non-local rows, the teacher target is the temporally averaged video feature v_n, not the per-frame feature f_nt used in Eq. (3), and the teacher also has fewer parameters. The 6.1-point top-1 gap between TKP-F without non-local blocks (68.9) and TKP-F with non-local blocks (75.0) could therefore come from the higher-capacity teacher, from the different target features, or from genuine temporal relations. A control using a capacity-matched teacher that is temporally blind (e.g., replacing non-local blocks with equal-capacity residual blocks that process each frame independently, or training the video network on temporally shuffled frames) is needed to attribute the gain to temporal modeling.
- [Section 4.3, Table 2] All MARS and Duke results are reported as single runs. The differences between TKP-F, TKP-D, and TKP are moderate (e.g., 75.0 vs. 75.6 top-1 on MARS; 63.1 vs. 64.2 vs. 65.1 mAP), and without run-to-run variance it is not possible to judge whether the complementarity of the two TKP losses is a genuine effect or stochastic noise. The authors should report the mean and standard deviation over at least three training runs, or provide a significance test, especially because the claim that the two transfer ways are 'complementary' in Section 4.3 rests on these small margins.
minor comments (6)
- [Section 3.3] There is a typo in the sentence 'in our emperiments, LF_TKP and LF_TKP are not back-propagated': 'emperiments' should be 'experiments', and the second loss symbol should be LD_TKP rather than LF_TKP.
- [Section 4.1 and 4.2] The iLIDS-VID protocol description is ambiguous: the text says only the first frames of videos from the first camera are used for training and testing, but TKP training requires video clips of T frames. Please clarify how the video clips are formed for the iLIDS-VID fine-tuning stage and how the gallery videos are used at test time.
- [Section 4.2] The statement about sampling '4 frames with a stride of 8 frames' and duplicating videos shorter than 32 frames is clear, but the choice of stride 8 and the duplication rule deserve a sentence of justification because they affect the temporal span of the clips and therefore the temporal knowledge available to the teacher.
- [Section 5] In the conclusion, 'start-of-the-art' should be 'state-of-the-art'.
- [Figure 2] The figure caption and legend refer to 'the process of TKP via features' and 'TKP via cross sample distances', which is helpful, but the red arrow for back-propagation should be labeled to indicate that it stops at the image network and does not enter the video network, to match the description in Section 3.3.
- [Section 4.5, Figure 4] In the discussion of the Duke gap, the text attributes the larger I2V-to-V2V gap to longer average video length. This is plausible, but the explanation would be strengthened by reporting the actual loss of information, e.g., the fraction of gallery videos where the first frame is occluded or blurred.
Circularity Check
No circularity: the TKP losses are training objectives and the reported Re-ID gains are independently measured empirical outcomes.
full rationale
The paper's derivation chain is a standard supervised training setup: define image features (Eq. 1) and video frame features (Eq. 2), define temporal knowledge propagation losses in terms of feature matching (Eq. 3) and cross-sample distance matching (Eq. 4), combine them with classification and triplet losses (Eq. 10), train on video clips, and then evaluate on held-out image-to-video retrieval. The reported accuracies in Table 2 are measured after training; they are not fed back into any equation as inputs. The TKP loss is an objective function, not a fitted parameter that is later renamed as a prediction. The claim that temporal knowledge is transferred is an interpretation of the distillation-style loss, and the ablations in Tables 6 and 7 provide empirical comparisons rather than circular justifications. There is no self-citation chain that supplies a load-bearing premise, no imported uniqueness theorem, and no ansatz smuggled in via citation to prior work. Even if the attribution of the improvement specifically to temporal modeling is underdetermined by the ablations, that is a question of experimental validity or confound control, not circularity. The derivation does not assume the results it claims to produce.
Assumptions & free parameters
free parameters (3)
- Video clip length T =
4
- Batch sampling parameters P and K =
P=4, K=4
- Triplet loss margin m =
not reported
assumptions (3)
- domain assumption Query images are drawn from the same distribution as training video frames (first-frame query protocol).
- domain assumption The video network's non-local blocks learn temporal knowledge that improves feature robustness, and this knowledge is captured in per-frame features f_nt.
- ad hoc to paper Gradients of the TKP loss must not reach the video network, or the non-local weights collapse to zero and temporal modeling is lost.
Cite this review
Pith. "Pith review of Temporal Knowledge Propagation for Image-to-Video Person Re-identification." pith.science (2026). https://pith.science/paper/DSGNWDN5
@misc{pith2026190803885,
author = {Pith},
title = {Pith review of: Temporal Knowledge Propagation for Image-to-Video Person Re-identification},
year = {2026},
howpublished = {\url{https://pith.science/paper/DSGNWDN5}},
note = {Machine review of arXiv:1908.03885}
}
read the original abstract
In many scenarios of Person Re-identification (Re-ID), the gallery set consists of lots of surveillance videos and the query is just an image, thus Re-ID has to be conducted between image and videos. Compared with videos, still person images lack temporal information. Besides, the information asymmetry between image and video features increases the difficulty in matching images and videos. To solve this problem, we propose a novel Temporal Knowledge Propagation (TKP) method which propagates the temporal knowledge learned by the video representation network to the image representation network. Specifically, given the input videos, we enforce the image representation network to fit the outputs of video representation network in a shared feature space. With back propagation, temporal knowledge can be transferred to enhance the image features and the information asymmetry problem can be alleviated. With additional classification and integrated triplet losses, our model can learn expressive and discriminative image and video features for image-to-video re-identification. Extensive experiments demonstrate the effectiveness of our method and the overall results on two widely used datasets surpass the state-of-the-art methods by a large margin. Code is available at: https://github.com/guxinqian/TKP
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Scalable person re- identification on supervised smoothed manifold
Song Bai, Xiang Bai, and Qi Tian. Scalable person re- identification on supervised smoothed manifold. In CVPR, 2017
work page 2017
-
[2]
Cristian Bucilu, Rich Caruana, and Alexandru Niculescu- Mizil. Model compression. In KDD, 2006
work page 2006
-
[3]
Video person re-identification with compet- itive snippet-similarity aggregation and co-attentive snippet embedding
Dapeng Chen, Hongsheng Li, Tong Xiao, Shuai Yi, and Xi- aogang Wang. Video person re-identification with compet- itive snippet-similarity aggregation and co-attentive snippet embedding. In CVPR, 2018
2018
-
[4]
Dark- rank: Accelerating deep metric learning via cross sample similarities transfer
Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. Dark- rank: Accelerating deep metric learning via cross sample similarities transfer. In AAAI, 2018
work page 2018
-
[5]
Dahjung Chung, Khalid Tahboub, and Edward J. Delp. A two stream siamese convolutional neural network for person re-identification. In ICCV, 2017
work page 2017
-
[6]
Learning spatiotemporal features with 3d convolutional networks
Tran Du, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. In ICCV, 2015
work page 2015
-
[7]
Christoph Feichtenhofer, Axel Pinz, and Richard P. Wildes. Spatiotemporal multiplier networks for video action recog- nition. In CVPR, 2017
work page 2017
-
[8]
Deep multi-metric learning for person re- identification
Yongxin Ge, Xinqian Gu, Min Chen, Hongxing Wang, and Dan Yang. Deep multi-metric learning for person re- identification. In ICME, 2018
work page 2018
Show all 45 references
-
[9]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016
2016
-
[10]
In defense of the triplet loss for person re-identification
Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. ArXiv:1703.07737, 2017
2017 arXiv
-
[11]
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NIPS workshop, 2014
2014
-
[12]
Interaction-and-aggregation network for person re-identification
Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, and Xilin Chen. Interaction-and-aggregation network for person re-identification. In CVPR, 2019
2019
-
[13]
Vrstc: Occlusion-free video person re-identification
Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, and Xilin Chen. Vrstc: Occlusion-free video person re-identification. In CVPR, 2019
2019
-
[14]
Learning euclidean-to-riemannian metric for point- to-set classification
Zhiwu Huang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Learning euclidean-to-riemannian metric for point- to-set classification. In CVPR, 2014
2014
-
[15]
3d convo- lutional neural networks for human action recognition
Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. 3d convo- lutional neural networks for human action recognition. In ICML, 2010
2010
-
[16]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015
2015
-
[17]
Joseph B. Kruskal. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 1964
1964
-
[18]
Large scale metric learning from equivalence constraints
Martin Kstinger, Martin Hirzer, Paul Wohlhart, Peter M Roth, and Horst Bischof. Large scale metric learning from equivalence constraints. In CVPR, 2012
2012
-
[19]
The approximation power of moving least- squares
David Levin. The approximation power of moving least- squares. Mathematics of Computation, 1998
1998
-
[20]
Diversity regularized spatiotemporal attention for video- based person re-identification
Shuang Li, Slawomir Bak, Peter Carr, and Xiaogang Wang. Diversity regularized spatiotemporal attention for video- based person re-identification. In CVPR, 2018
2018
-
[21]
Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z. Li. Per- son re-identification by local maximal occurrence represen- tation and metric learning. In CVPR, 2015
2015
-
[22]
Visualizing data using t-sne
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 2008
2008
-
[23]
Recurrent convolutional network for video-based per- son re-identification
Niall Mclaughlin, Jesus Martinez Del Rincon, and Paul Miller. Recurrent convolutional network for video-based per- son re-identification. In CVPR, 2016
2016
-
[24]
Beyond short snippets: Deep networks for video classification
Yue Hei Ng, Matthew Hausknecht, Sudheendra Vijaya- narasimhan, Oriol Vinyals, Rajat Monga, and George Toderici. Beyond short snippets: Deep networks for video classification. In CVPR, 2015
2015
-
[25]
Fitnets: Hints for thin deep nets
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015
2015
-
[26]
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, and Michael Bernstein. Imagenet large scale visual recognition challenge. IJCV, 2015
2015
-
[27]
Image de- formation using moving least squares
Scott Schaefer, Travis McPhail, and Joe Warren. Image de- formation using moving least squares. In SIGGRAPH, 2006
2006
-
[28]
Kot, and Gang Wang
Jianlou Si, Honggang Zhang, Chun-Guang Li, Jason Kuen, Xiangfei Kong, Alex C. Kot, and Gang Wang. Dual attention matching network for context-aware feature sequence based person re-identification. In CVPR, 2018
2018
-
[29]
Mask-guided contrastive attention model for person re-identification
Chunfeng Song, Yan Huang, Wanli Ouyang, and Liang Wang. Mask-guided contrastive attention model for person re-identification. In CVPR, 2018
2018
-
[30]
Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline)
Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). InECCV, 2018
2018
-
[31]
P2snet : Can an image match a video for person re-identification in an end-to-end way? TCSVT, 2017
Guangcong Wang, Jianhuang Lai, and Xiaohua Xie. P2snet : Can an image match a video for person re-identification in an end-to-end way? TCSVT, 2017
2017
-
[32]
Person re-identification by video ranking
Taiqing Wang, Shaogang Gong, Xiatian Zhu, and Shengjin Wang. Person re-identification by video ranking. In ECCV, 2014
2014
-
[33]
Non-local neural networks
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaim- ing He. Non-local neural networks. In CVPR, 2018
2018
-
[34]
Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning
Yu Wu, Yutian Lin, Xuanyi Dong, Yan Yan, Wanli Ouyang, and Yi Yang. Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning. In CVPR, 2018
2018
-
[35]
Top- push video-based person re-identification
Jinjie You, Ancong Wu, Xiang Li, and Wei-Shi Zheng. Top- push video-based person re-identification. In CVPR, 2016
2016
-
[36]
Hard-aware point-to-set deep metric for person re-identification
Rui Yu, Zhiyong Dou, Song Bai, Zhaoxiang Zhang, Yongchao Xu, and Xiang Bai. Hard-aware point-to-set deep metric for person re-identification. In ECCV, 2018
2018
-
[37]
Image-to-video per- son re-identification with temporally memorized similarity learning
Dongyu Zhang, Wenxi Wu, Hui Cheng, Ruimao Zhang, Zhenjiang Dong, and Zhaoquan Cai. Image-to-video per- son re-identification with temporally memorized similarity learning. TCSVT, 2017
2017
-
[38]
Multi-shot pedestrian re-identification via sequential decision making
Jianfu Zhang, Naiyan Wang, and Liqing Zhang. Multi-shot pedestrian re-identification via sequential decision making. In CVPR, 2018
2018
-
[39]
Aligne- dreid: Surpassing human-level performance in person re- identification
Xuan Zhang, Hao Luo, Xing Fan, Weilai Xiang, Yixiao Sun, Qiqi Xiao, Wei Jiang, Chi Zhang, and Jian Sun. Aligne- dreid: Surpassing human-level performance in person re- identification. ArXiv:1711.08184, 2017
2017 arXiv
-
[40]
Hospedales, and Huchuan Lu
Ying Zhang, Tao Xiang, Timothy M. Hospedales, and Huchuan Lu. Deep mutual learning. In CVPR, 2018
2018
-
[41]
Mars: A video benchmark for large-scale person re-identification
Liang Zheng, Zhi Bie, Yifan Sun, Jingdong Wang, Chi Su, Shengjin Wang, and Qi Tian. Mars: A video benchmark for large-scale person re-identification. In ECCV, 2016
2016
-
[42]
Scalable person re-identification: A benchmark
Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jing- dong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In ICCV, 2015
2015
-
[43]
From point to set: Extend the learning of distance metrics
Pengfei Zhu, Lei Zhang, Wangmeng Zuo, and David Zhang. From point to set: Extend the learning of distance metrics. In ICCV, 2013
2013
-
[44]
Learning hetero- geneous dictionary pair with feature projection matrix for pedestrian video retrieval via single query image
Xiaoke Zhu, Xiao-Yuan Jing, Fei Wu, Yunhong Wang, Wangmeng Zuo, and Wei-Shi Zheng. Learning hetero- geneous dictionary pair with feature projection matrix for pedestrian video retrieval via single query image. In AAAI, 2017
2017
-
[45]
Image to video person re-identification by learning heterogeneous dictionary pair with feature projection matrix
Xiaoke Zhu, Xiao-Yuan Jing, Xinge You, Wangmeng Zuo, Shiguang Shan, and Wei-Shi Zheng. Image to video person re-identification by learning heterogeneous dictionary pair with feature projection matrix. TIFS, 2018
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.