REVIEW 3 major objections 5 minor 37 references
Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that coding only camera identity into a ViT's side embeddings yields state-of-the-art visible-infrared person matching on LLCM, while adding domain labels sharply lowers accuracy, and that on IJB-MDF range-induced scale…
desk verdict The camera-only SIE finding on LLCM is intriguing, but the SIE-18 ablation is internally inconsistent as written, and the IJB-MDF benchmark is a useful but incomplete start. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Side Information Embedding (SIE) is a learnable embedding table $S\in\mathbb{R}^{N_D \times D}$; the row $S[i_{\text{dom}}]$ for the image's attribute index is added to the transformer input as $Z'_0 = Z_0 + \lambda S[i_{\text{dom}}]$, with $\lambda$ controlling the strength. The same mechanism is reused with different index sets: domain only (SIE-2 on LLCM, SIE-4 on IJB-MDF), camera only (SIE-9), camera plus domain (SIE-18), and domain plus range (SIE-8). The ablation across these tables is what carries the claim that camera identity is more informative than domain, and that range information is not well delivered through this mechanism.
What would settle it
Train two LLCM variants with exactly 18 side-embedding vectors: one with the paper's camera-plus-domain assignments and one with camera-only assignments formed by splitting each camera into two sub-indexes; if the camera-only 18-vector model keeps roughly 72% Rank-1 while the camera-plus-domain model drops to roughly 61%, the drop is caused by domain information, otherwise it is caused by vector count.
Extended reading notes
Core claim
The first central claim is that a ViT-B pretrained on a large visible-light person dataset and finetuned with a Side Information Embedding that indexes one of the nine LLCM cameras (SIE-9) reaches 72.8% Rank-1 for VIS-to-IR and 66.0% for IR-to-VIS, surpassing prior VI-ReID methods, and does so without any explicit domain embedding. The second central claim is that the same model, trained on one range of IJB-MDF, fails to match bodies at a different range: a short-range-trained model scores 52.1% on SWIR-30m probes against a VIS-500m gallery where a long-range-trained model scores 69.7%, and mixing ranges in training recovers only part of the gap. Encoding the range as a side embedding (SIE-8) does not help, and the authors take range-induced scale misalignment to be an underexplored occlusion modality for visible-infrared re-identification.
Load-bearing premise
The comparison takes the performance gap between SIE-9 and SIE-18 to reflect the type of encoded information, although the two configurations also differ in the number of learned side-embedding vectors (9 versus 18).
Editorial extensions
If this is right
- A visible-pretrained ViT can transfer to visible-infrared person re-identification without a domain-specific architecture, as long as camera identity is supplied at the input.
- Domain supervision is not automatically helpful in VI-ReID: on LLCM, adding domain labels on top of camera labels drops Rank-1 from 72.8% to 61.4% in the VIS-to-IR mode.
- Cross-range matching is a distinct failure mode from cross-spectral matching: training at one range and probing at another costs roughly twenty Rank-1 points on IJB-MDF.
- Mixing ranges during training reduces but does not eliminate the cross-range gap, and side-embedding range codes are not the fix.
Reading between the lines
- Because SIE-9 and SIE-18 differ in both the kind of information and the number of learned vectors (9 versus 18), the paper's headline conclusion that domain information is harmful is not fully isolated; a fair reading is that camera-indexed embeddings work well and the larger camera-plus-domain table does not.
- If each LLCM camera records in a single spectral domain, camera identity and domain label are confounded, so the SIE-9 model may be implicitly encoding domain through camera identity.
- The IJB-MDF protocols measure an operational scenario that existing VI-ReID benchmarks miss: long-range visible galleries with short-range infrared probes; a testable next step is whether simple multi-scale training or test-time scale augmentation recovers the lost Rank-1 without any side-embedding range code.
- A factored or continuous range encoding, separate from a discrete domain table, could be tested against SIE-8 to determine whether the failure is range information itself or the discrete joint table's capacity use.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper adapts a Vision Transformer pretrained on visible imagery to cross-spectral body recognition by adding Side Information Embeddings (SIE) that encode domain and/or camera information. On the LLCM dataset, the authors report that a camera-only configuration (SIE-9) achieves state-of-the-art Rank-1 and mAP results, while the joint domain-plus-camera configuration (SIE-18) degrades sharply, leading to the claim that camera information is more important than domain information. On the IJB-MDF dataset, the paper defines short-range, long-range, and mixed-range training sets and proposes four cross-spectral, cross-range test protocols, but reports results for only two of them, with no external baselines, and interprets performance gaps as evidence of range-induced occlusion sensitivity.
Significance. If the LLCM SIE-9 result is valid, the finding that camera identity alone is sufficient for strong visible-infrared matching would be a useful and non-obvious result for the VI-ReID community. The IJB-MDF protocols partly fill a real gap, since existing VI-ReID benchmarks lack range-induced scale variation. The paper also includes useful ablations (Table 2) and MDE visualizations. However, the headline claim depends on an ablation whose indexing appears internally inconsistent, and the validation protocol is not described, so the significance cannot be assessed as currently presented.
major comments (3)
- [Section 4.2.1 and Table 1] The definition of SIE-18 is internally inconsistent with the description of LLCM. Section 4.1 states that LLCM was captured using a 9-camera network, and Section 4.2.1 defines SIE-9 by assigning each of the 9 cameras an index 0–8. SIE-18 then maps 'the 9 VIS cameras' to indices 0–8 and 'the 9 IR cameras' to indices 9–17, implying that LLCM has 18 separate camera sensors. If LLCM actually has 9 cameras total, then SIE-18 either assigns two indices to some cameras or leaves indices unused, so the sharp drop in Table 1 (72.8/75.3 to 61.4/64.8 in VIS-to-IR Rank-1/mAP) does not isolate the effect of adding domain information. This is the load-bearing comparison for the central claim in Section 4.3.1 and the abstract, so the authors must clarify the true camera configuration and rerun or reinterpret SIE-18 accordingly.
- [Section 4.3.1 and Tables 1–2] No validation split or model-selection procedure is described. The paper appears to choose SIE-9 on the basis of test-set performance and then reports those same test numbers as the headline result. The authors should state whether hyperparameters and SIE configurations were selected on a held-out validation set, and if not, they should temper the state-of-the-art claim or add a validation-based selection protocol. Without this, the comparison among SIE-2, SIE-9, and SIE-18 is potentially overfit to the test set.
- [Sections 4.1 and 4.3.2, Tables 3–4] The IJB-MDF analysis is incomplete as a benchmark contribution. Four test protocols are proposed, but only the short-range and long-range protocols are reported; the text states that the mixed-range protocols are omitted 'for simplicity.' Additionally, no external baselines are provided, and only Rank-1 is reported without mAP or other standard retrieval metrics. The central interpretation that range-induced occlusions cause the degradation is asserted but never directly measured: the paper does not quantify occlusion rates, scale ratios, or detection quality. The authors should report all four protocols, add at least one existing or simple baseline, and provide quantitative evidence connecting range/scale to occlusion before claiming that the model is 'highly sensitive to scale and range variations.'
minor comments (5)
- [Table 1] There is a formatting error in the IRM row: '66.266.6' should be '66.2 66.6'.
- [Table 2] The baseline row label 'Baseline (no SIE, Triplet, no GeM Pool)' is confusing because the Triplet Loss column is marked with a check; rephrase as 'Baseline (no SIE, with Triplet, no GeM Pool)' for clarity.
- [Figures 2–5] The MDE visualizations would be more informative if they included quantitative separation metrics (e.g., inter-class vs. intra-class distance ratios) or at least pointed the reader to what compactness of clusters indicates; currently the interpretations are qualitative.
- [Section 4.3.2] The text says 'Red values indicate best performances in respective columns,' but the tables as rendered are monochrome; this should be clarified or the highlighting should be made visible in the final PDF.
- [References] Reference [15] is a preprint; if a peer-reviewed version is available, it should be cited instead. Also, the paper does not state whether code or trained models will be released, which would strengthen reproducibility.
Circularity Check
No significant circularity: the central claims are empirical ablation results; the only self-citation is a minor, non-load-bearing pretraining reference.
full rationale
The paper's load-bearing claims are empirical: on LLCM, the SIE-9 configuration outperforms SIE-18, and on IJB-MDF, adding range information via SIE-8 hurts performance. Neither claim is derived from an equation that contains its own conclusion. The SIE configurations are input choices, and the reported Rank-1 and mAP numbers come from training and testing on fixed public splits, so the conclusion that camera information is more important than domain information is an interpretation of an ablation rather than a fitted parameter renamed as a prediction. The SIE-9 versus SIE-18 gap may be confounded by the number of learned side-embedding vectors or by how the 18 modality-camera streams are indexed, and selecting the best configuration on the test set is a statistical concern, but these are correctness risks, not circularity. The only self-citation is the LUPerson-based pretraining described in [15] in Section 4.2; that reference supplies a pretraining pipeline and does not assume or contain the LLCM or IJB-MDF results, so it is not load-bearing circularity. No equation in the paper reduces to its own inputs, no uniqueness theorem is imported from the authors' prior work, and no known result is merely renamed. The score of 2 reflects only the minor, non-load-bearing self-citation; the derivation chain itself is self-contained against external benchmarks.
Assumptions & free parameters
free parameters (4)
- SIE scale lambda =
3
- triplet loss weight lambda_t =
1
- triplet margin m
- SIE configuration =
SIE-9 on LLCM, SIE-4 on IJB-MDF
assumptions (4)
- domain assumption LUPerson-pretrained ViT provides a strong starting point for cross-spectral body recognition.
- domain assumption Camera indices are available for gallery and probe images and carry useful information that generalizes across identities.
- domain assumption Scale variation from different capture ranges is usefully modeled as occlusion for body recognition.
- domain assumption YOLOv10 detections matched to face boxes by maximum IoU yield reliable identity labels for IJB-MDF body images.
Cite this review
Pith. "Pith review of Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF." pith.science (2026). https://pith.science/paper/XZYRY62I
@misc{pith2026250608953,
author = {Pith},
title = {Pith review of: Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF},
year = {2026},
howpublished = {\url{https://pith.science/paper/XZYRY62I}},
note = {Machine review of arXiv:2506.08953}
}
read the original abstract
Vision Transformers (ViTs) have demonstrated impressive performance across a wide range of biometric tasks, including face and body recognition. In this work, we adapt a ViT model pretrained on visible (VIS) imagery to the challenging problem of cross-spectral body recognition, which involves matching images captured in the visible and infrared (IR) domains. Recent ViT architectures have explored incorporating additional embeddings beyond traditional positional embeddings. Building on this idea, we integrate Side Information Embedding (SIE) and examine the impact of encoding domain and camera information to enhance cross-spectral matching. Surprisingly, our results show that encoding only camera information - without explicitly incorporating domain information - achieves state-of-the-art performance on the LLCM dataset. While occlusion handling has been extensively studied in visible-spectrum person re-identification (Re-ID), occlusions in visible-infrared (VI) Re-ID remain largely underexplored - primarily because existing VI-ReID datasets, such as LLCM, SYSU-MM01, and RegDB, predominantly feature full-body, unoccluded images. To address this gap, we analyze the impact of range-induced occlusions using the IARPA Janus Benchmark Multi-Domain Face (IJB-MDF) dataset, which provides a diverse set of visible and infrared images captured at various distances, enabling cross-range, cross-spectral evaluations.
Figures
Reference graph
Works this paper leans on
-
[1]
Minimum- distortion embedding.arXiv, 2021
Akshay Agrawal, Alnur Ali, and Stephen Boyd. Minimum- distortion embedding.arXiv, 2021. 5
work page 2021
-
[2]
From Cross-Modal to Mixed-Modal Visible-Infrared Re-Identification
Mahdi Alehdaghi, Rajarshi Bhattacharya, Pourya Shamsol- moali, Rafael MO Cruz, and Eric Granger. From cross- modal to mixed-modal visible-infrared re-identification. arXiv preprint arXiv:2501.13307, 2025. 2, 5, 6
work page Pith review arXiv 2025
-
[3]
Lihao Liu et al. Ao Wang, Hui Chen. Yolov10: Real-time end-to-end object detection.arXiv preprint arXiv:2405.14458, 2024. 4
arXiv 2024
-
[4]
Cuiqun Chen, Mang Ye, Meibin Qi, Jingjing Wu, Jianguo Jiang, and Chia-Wen Lin. Structure-aware positional trans- former for visible-infrared person re-identification.IEEE Transactions on Image Processing, 31:2352–2364, 2022. 2
work page 2022
-
[5]
Occlude them all: Occlusion-aware attention network for occluded person re-id
Peixian Chen, Wenfeng Liu, Pingyang Dai, Jianzhuang Liu, Qixiang Ye, Mingliang Xu, Qi’an Chen, and Rongrong Ji. Occlude them all: Occlusion-aware attention network for occluded person re-id. InProceedings of the IEEE/CVF international conference on computer vision, pages 11833– 11842, 2021. 3
work page 2021
-
[6]
Visible-infrared person re-identification via cross-modality interaction transformer
Yujian Feng, Jian Yu, Feng Chen, Yimu Ji, Fei Wu, Shang- don Liu, and Xiao-Yuan Jing. Visible-infrared person re-identification via cross-modality interaction transformer. IEEE Transactions on Multimedia, 25:7647–7659, 2022. 2
work page 2022
-
[7]
Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen. Unsu- pervised pre-training for person re-identification.Proceed- ings of the IEEE conference on computer vision and pattern recognition, 2021. 4
work page 2021
-
[8]
Deep spatial feature reconstruction for partial person re- identification: Alignment-free approach
Lingxiao He, Jian Liang, Haiqing Li, and Zhenan Sun. Deep spatial feature reconstruction for partial person re- identification: Alignment-free approach. InCVPR, pages 7073–7082, 2018. 3
work page 2018
Show all 37 references
-
[9]
Foreground-aware pyramid reconstruc- tion for alignment-free occluded person re-identification
Lingxiao He, Yinggang Wang, Wu Liu, He Zhao, Zhenan Sun, and Jiashi Feng. Foreground-aware pyramid reconstruc- tion for alignment-free occluded person re-identification. In ICCV, pages 8450–8459, 2019. 3
2019
-
[10]
Transreid: Transformer-based object re- identification
Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer-based object re- identification. InProceedings of the IEEE/CVF international conference on computer vision, pages 15013–15022, 2021. 3
2021
-
[11]
Partial person re-identification with part- part correspondence learning
Tianyu He, Xu Shen, Jianqiang Huang, Zhibo Chen, and Xian-Sheng Hua. Partial person re-identification with part- part correspondence learning. InCVPR, pages 9105–9115,
-
[12]
Instruct-reid: A multi-purpose person re-identification task with instructions
Weizhen He, Yiheng Deng, Shixiang Tang, Qihao Chen, Qingsong Xie, Yizhou Wang, Lei Bai, Feng Zhu, Rui Zhao, Wanli Ouyang, et al. Instruct-reid: A multi-purpose person re-identification task with instructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Patt...
2024
-
[13]
Adversarial cross- scale alignment pursuit for seriously misaligned person re- identification
Yuanhang He, Hua Yang, and Lin Chen. Adversarial cross- scale alignment pursuit for seriously misaligned person re- identification. InICIP, pages 2373–2377. IEEE, 2021. 2
2021
-
[14]
In de- fense of the triplet loss for person re-identification.arXiv preprint arXiv:1703.07737, 2017
Alexander Hermans, Lucas Beyer, and Bastian Leibe. In de- fense of the triplet loss for person re-identification.arXiv preprint arXiv:1703.07737, 2017. 3
2017 arXiv
-
[15]
Self-supervised learning of whole and component-based semantic representations for person re- identification.arXiv preprint arXiv:2311.17074, 2023
Siyuan Huang, Yifan Zhou, Ram Prabhakar, Xijun Liu, Yux- iang Guo, Hongrui Yi, Cheng Peng, Rama Chellappa, and Chun Pong Lau. Self-supervised learning of whole and component-based semantic representations for person re- identification.arXiv preprint arXiv:2311.17074, 2023. 4
2023 arXiv
-
[16]
Attentive part-aware networks for partial person re- identification
Lijuan Huo, Chunfeng Song, Zhengyi Liu, and Zhaoxiang Zhang. Attentive part-aware networks for partial person re- identification. InICPR, pages 3652–3659. IEEE, 2021. 2
2021
-
[17]
Exploring stronger transformer representation learning for occluded person re-identification.arXiv preprint arXiv:2410.15613,
Zhangjian Ji, Donglin Cheng, and Kai Feng. Exploring stronger transformer representation learning for occluded person re-identification.arXiv preprint arXiv:2410.15613,
-
[18]
Learning disentangled representation implicitly via trans- former for occluded person re-identification.IEEE Trans
Mengxi Jia, Xinhua Cheng, Shijian Lu, and Jian Zhang. Learning disentangled representation implicitly via trans- former for occluded person re-identification.IEEE Trans. Multimedia, 2022. 3
2022
-
[19]
Cross-modality transformer for visible-infrared person re-identification
Kongzhu Jiang, Tianzhu Zhang, Xiang Liu, Bingqiao Qian, Yongdong Zhang, and Feng Wu. Cross-modality transformer for visible-infrared person re-identification. InEuropean conference on computer vision, pages 480–496. Springer,
-
[20]
Iarpa janus benchmark multi-domain face
Nathan D Kalka, James A Duncan, Jeremy Dawson, and Charles Otto. Iarpa janus benchmark multi-domain face. In2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–9. IEEE,
-
[21]
Cross- modality transformer with modality mining for visible- infrared person re-identification.IEEE Transactions on Mul- timedia, 25:8432–8444, 2023
Tengfei Liang, Yi Jin, Wu Liu, and Yidong Li. Cross- modality transformer with modality mining for visible- infrared person re-identification.IEEE Transactions on Mul- timedia, 25:8432–8444, 2023. 2
2023
-
[22]
Self-supervised bodymap-to-appearance co-attention for partial person re- identification
Ci-Siang Lin and Yu-Chiang Frank Wang. Self-supervised bodymap-to-appearance co-attention for partial person re- identification. InICIP, pages 2299–2303. IEEE, 2021. 2
2021
-
[23]
Stnreid: Deep convolutional networks with pairwise spatial trans- former networks for partial person re-identification.IEEE Trans
Hao Luo, Wei Jiang, Xing Fan, and Chi Zhang. Stnreid: Deep convolutional networks with pairwise spatial trans- former networks for partial person re-identification.IEEE Trans. Multimedia, 22(11):2905–2913, 2020. 2
2020
-
[24]
Intriguing properties of vision transform- ers.Advances in Neural Information Processing Systems, 34: 23296–23308, 2021
Muhammad Muzammal Naseer, Kanchana Ranasinghe, Salman H Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transform- ers.Advances in Neural Information Processing Systems, 34: 23296–23308, 2021. 1
2021
-
[25]
Implicit discriminative knowl- edge learning for visible-infrared person re-identification
Kaijie Ren and Lei Zhang. Implicit discriminative knowl- edge learning for visible-infrared person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 393–402, 2024. 2, 5, 6
2024
-
[26]
Perceive where to focus: Learn- ing visibility-aware part-level features for partial person re- identification
Yifan Sun, Qin Xu, Yali Li, Chi Zhang, Yikang Li, Shengjin Wang, and Jian Sun. Perceive where to focus: Learn- ing visibility-aware part-level features for partial person re- identification. InCVPR, pages 393–402, 2019. 3
2019
-
[27]
Self-guided body part alignment with relation transformers for occluded person re-identification.IEEE Sig- nal Processing Letters, 28:1155–1159, 2021
Guanshuo Wang, Xiong Chen, Jialin Gao, Xi Zhou, and Shiming Ge. Self-guided body part alignment with relation transformers for occluded person re-identification.IEEE Sig- nal Processing Letters, 28:1155–1159, 2021. 3
2021
-
[28]
Feature erasing and diffusion net- work for occluded person re-identification
Zhikang Wang, Feng Zhu, Shixiang Tang, Rui Zhao, Lihuo He, and Jiangning Song. Feature erasing and diffusion net- work for occluded person re-identification. InCVPR, pages 4754–4763, 2022. 3
2022
-
[29]
Wrim-net: Wide-ranging in- formation mining network for visible-infrared person re- identification.arXiv preprint arXiv:2408.10624, 2024
Yonggan Wu, Ling-Chao Meng, Yuan Zichao, Sixian Chan, and Hong-Qiang Wang. Wrim-net: Wide-ranging in- formation mining network for visible-infrared person re- identification.arXiv preprint arXiv:2408.10624, 2024. 2, 4, 6
2024 arXiv
-
[30]
Occluded person re-identification with single-scale global representations
Cheng Yan, Guansong Pang, Jile Jiao, Xiao Bai, Xuetao Feng, and Chunhua Shen. Occluded person re-identification with single-scale global representations. InICCV, pages 11875–11884, 2021. 3
2021
-
[31]
Learning multi-granular hypergraphs for video-based person re-identification
Yichao Yan, Jie Qin, Jiaxin Chen, Li Liu, Fan Zhu, Ying Tai, and Ling Shao. Learning multi-granular hypergraphs for video-based person re-identification. InCVPR, pages 2899– 2908, 2020. 3
2020
-
[32]
Diverse embedding expan- sion network and low-light cross-modality benchmark for visible-infrared person re-identification
Yukang Zhang and Hanzi Wang. Diverse embedding expan- sion network and low-light cross-modality benchmark for visible-infrared person re-identification. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2153–2162, 2023. 2, 3, 4, 6
2023
-
[33]
Incremental generative occlusion adversarial suppression network for person reid.IEEE Trans
Cairong Zhao, Xinbi Lv, Shuguang Dou, Shanshan Zhang, Jun Wu, and Liang Wang. Incremental generative occlusion adversarial suppression network for person reid.IEEE Trans. Image Process., 30:4212–4224, 2021. 3
2021
-
[34]
Pyramidal person re-identification via multi-loss dynamic training
Feng Zheng, Cheng Deng, Xing Sun, Xinyang Jiang, Xi- aowei Guo, Zongqiao Yu, Feiyue Huang, and Rongrong Ji. Pyramidal person re-identification via multi-loss dynamic training. InCVPR, pages 8514–8522, 2019. 3
2019
-
[35]
Robust partial matching for person search in the wild
Yingji Zhong, Xiaoyu Wang, and Shiliang Zhang. Robust partial matching for person search in the wild. InCVPR, pages 6827–6835, 2020. 2, 3
2020
-
[36]
Random erasing data augmentation
Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. InAAAI Conf. Artif. Intell., pages 13001–13008, 2020. 4
2020
-
[37]
Occluded person re-identification
Jiaxuan Zhuo, Zeyu Chen, Jianhuang Lai, and Guangcong Wang. Occluded person re-identification. InICME, pages 1–6. IEEE, 2018. 3
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.