REVIEW 3 major objections 4 minor 33 references
Database-Agnostic Gait Enrollment using SetTransformers
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read The paper claims a SetTransformer that sees a probe embedding and its nearest gallery neighbors can decide known-vs-new enrollment without thresholds or retraining, transferring across datasets, recognition models, and gallery sizes.
desk verdict A useful problem framing and a clean evaluation protocol for open-set gait enrollment, but the paper's own cross-dataset numbers contradict the headline claim of database-agnostic performance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a SetTransformer, an attention-based permutation-invariant neural network that consumes an unordered set of embeddings. For a probe embedding $p$, the method selects its $K$ nearest gallery neighbors by embedding distance, and for each neighbor $g_k$ it forms an identity-average embedding $\overline{\mathrm{id}}_{g_k}=\frac{1}{|G_{g_k}|}\sum_{g_i\in G_{g_k}} g_i$, where $G_{g_k}$ gathers all gallery embeddings of that identity. The input set $\{p, g_k, \overline{\mathrm{id}}_{g_k}\}$ is passed through self-attention, and a two-layer MLP reads the output at the probe's position to produce the enrollment decision. The paper compares additive pairing, per-instance positional encodings, and per-identity positional encodings as ways to link each neighbor to its identity average, and finds that one shared position code per identity works best, because it lets the model attend to identity groups rather than isolated samples.
What would settle it
Run the enrollment model trained only on CASIA-B and PsyMo scenarios on a third gait database with different identities, capture conditions, and id:walk ratios; if MCC there is near zero, the 'database-agnostic' claim is false. The paper's Table 4 already provides a partial test, with cross-dataset MCC between $-0.007$ and $0.242$.
Extended reading notes
Core claim
The central claim is that gait enrollment can be decoupled from gait recognition: a recognition model produces embeddings, and a separate permutation-invariant SetTransformer makes the open-set decision by attending to the probe embedding together with its $K$ nearest gallery neighbors and the per-identity mean embedding of each neighbor. The paper presents this as a single framework that is agnostic to the recognition architecture and to the database, because it learns from the gallery context itself rather than from a fixed gallery size or identity set. On CASIA-B and PsyMo, using embeddings from three skeleton-based recognition models, the paper reports that the method achieves high enrollment accuracy in balanced and well-sampled scenarios, scales better with data than the logistic-regression baselines, and benefits from training on variable gallery configurations, larger $K$, and per-identity positional encodings. The paper also reports cross-dataset and cross-model transfer experiments; in those runs the numbers are much lower, with cross-dataset MCC between $-0.007$ and $0.242$.
Load-bearing premise
The load-bearing premise is that enrollment scenarios cut from CASIA-B and PsyMo represent other gait databases, so a model trained on those two transfers with no retraining; the paper's Table 4 cross-dataset MCCs, as low as $-0.007$ and at most $0.242$, show this premise is the point most likely to give way.
Editorial extensions
If this is right
- A deployed gait system could run recognition and enrollment with one shared embedding space: the recognition model identifies who the person is, and the SetTransformer decides whether that person is already known.
- Operator overhead falls because the enrollment decision no longer requires manually tuned similarity thresholds that need recalibration when the gallery changes.
- Performance increases with the number of neighbors $K$ and with per-identity context, so systems with more gallery evidence can buy accuracy by widening the context set.
- Training on variable gallery configurations, with different identity counts and walks per identity, is the mechanism behind generalization; fixed-gallery training is shown to be worse.
- Scenarios with many identities but very few walks per identity remain the hard regime, which tells data collectors that more walks per identity matter for enrollment.
Reading between the lines
- A direct extension would be to apply the same context-set formulation to open-set enrollment for other biometric embeddings, such as face or person re-identification, since the model only consumes embeddings and gallery structure; the paper does not test those modalities.
- The cross-dataset and cross-model tables suggest the honest scope is transfer within the distribution of controlled laboratory gait trips rather than transfer to arbitrary databases; a sharper name for the property would be gallery-context generalization rather than database agnosticism.
- A testable extension is to feed a small amount of unlabeled walks from the target database into the gallery context at inference time, letting the SetTransformer adapt its context statistics before enrollment rather than freezing the model entirely.
- The method could be compared against a train-free baseline that uses the ratio of the probe-to-nearest-neighbor distance to the probe-to-second-nearest-neighbor distance, which is a standard open-set heuristic; the paper only compares trained logistic baselines.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses open-set gait enrollment: deciding whether a probe gait embedding belongs to a known identity in a gallery or is a newcomer. The authors propose a SetTransformer-based binary classifier whose input is a probe embedding, its K nearest gallery neighbors, and per-identity average embeddings, with three variants for encoding neighbor-identity correspondence. They train and evaluate this model on enrollment scenarios constructed from CASIA-B and PsyMo at different identity-to-walk ratios, using embeddings from GaitPT, GaitFormer, and GaitGraph. The paper claims the method is database-agnostic and recognition-architecture-agnostic, requiring no task-specific thresholds or retraining, and reports within-dataset results, comparisons with logistic-regression baselines, and cross-dataset and cross-embedding transfer experiments.
Significance. If the central claims were established, the method would address a genuine deployment gap: automatic enrollment decisions in gait recognition systems, decoupled from the recognition backbone and applicable across databases without threshold tuning. The paper has strengths: it systematically varies id:walk ratios, compares against nontrivial baselines, and promises to release code and dataset scenarios. However, the headline claims of database-agnostic and architecture-agnostic behavior are not supported by the paper's own transfer experiments, and the within-dataset advantage over a simple baseline is inconsistent across ratios. The contribution is therefore potentially useful as a within-dataset enrollment model with known transfer limitations, but the current framing considerably overstates the evidence.
major comments (3)
- [§4, Table 4] The cross-dataset results directly contradict the 'database-agnostic' claim in the Abstract and Discussion. Table 4 reports MCCs between -0.007 and 0.242 across all transfer directions and embedding models, with GaitGraph PsyMo→CASIA-B at -0.007 (chance-level). The paper's own text in §4 acknowledges 'a clear domain gap.' Since the method is claimed to generalize 'across different settings of identities, samples per identities, gait recognition architectures, and datasets' (Discussion), the load-bearing assumption that enrollment scenarios from CASIA-B and PsyMo transfer to other databases is not merely unverified but contradicted by the reported numbers. To substantiate the claim, the authors would need a successful held-out third-dataset experiment (e.g., training on the union of CASIA-B and PsyMo and testing on a different gait database) or a substantial reframing of the contribution as within-dataset enrollment only.
- [§4, Table 3] The claim that the SetTransformer 'scales better with data compared to traditional approaches' is only supported at ID:Walk ratio 0.75. At ratio 0.25, the logistic regression on neighbor distances achieves MCC 0.844 versus the SetTransformer's 0.615; at ratio 0.5, the baseline MCC is 0.832 versus 0.830, essentially a tie. The proposed method clearly wins only in the 0.75 setting. The comparison should be reported across all ratios and datasets, with error bars or significance tests, before claiming a consistent scaling advantage. As written, Table 3 shows the baseline is competitive or superior in two of three conditions.
- [§4, Table 5 and §5] The 'model-agnostic' claim is also not supported by the cross-embedding transfer results. Table 5 reports near-zero MCCs in several scenarios, e.g., GaitGraph→GaitFormer on PsyMo (MCC 0.000), GaitFormer→GaitPT on PsyMo (MCC 0.023), and GaitGraph→GaitPT on PsyMo (MCC 0.025). If the enrollment model were truly independent of the recognition architecture, one would expect more consistent transfer across embedding spaces; the large variability (MCC from 0.000 to 0.400 in the same dataset) indicates strong sensitivity to the embedding source. The Discussion's claim that the method generalizes 'across ... gait recognition architectures' should be qualified accordingly.
minor comments (4)
- [§4] In the cross-dataset evaluation paragraph, the text states 'GaitGraph ones nearly collapse in the opposite direction (MCC of 0.007)', but Table 4 reports -0.007 for this entry. The sign is important because -0.007 indicates chance or slightly inverted prediction; please correct the inconsistency.
- [Tables 4 and 5 captions] Both Table 4 and Table 5 have the caption 'Cross-embedding enrollment performance', but Table 4 crosses datasets (train dataset → test dataset) while Table 5 crosses embedding models (train embedding → test embedding). Rename Table 4 to 'Cross-dataset enrollment performance' to avoid confusion.
- [§3.1] The construction of training and test scenarios should clarify how probe walks from 'new IDs' are chosen so that no identity overlaps between the gallery and probe subsets within an enrollment scenario; otherwise, the measured F1/MCC could be inflated by trivial identity leakage.
- [Figures 3-6] The figures report MCC without confidence intervals or repeated-seed variability. Given the large differences attributed to K, id:walk ratio, and training database variability, error bars or multiple-seed results would strengthen the claims and help readers assess stability.
Circularity Check
No circularity found: the enrollment classifier is trained from labeled gallery/probe contexts and is not derived from the quantity it predicts; self-citations are to tools, not to load-bearing unverified results.
full rationale
The paper's central contribution is a supervised SetTransformer that takes a probe embedding, K nearest-neighbor gallery embeddings, and identity-average embeddings as input and predicts an enrollment label. The predicted quantity (known vs. new identity) is not used to define the input features, the loss, or the training scenarios, so there is no self-definitional reduction: the model's output is not equal to its input by construction. The recognition embeddings from GaitPT, GaitFormer, and GaitGraph are used as fixed feature extractors, and the paper trains these architectures itself rather than relying on an unverified cited theorem; self-citations to GaitPT, GaitFormer, and PsyMo identify the authors' own models and dataset, but these are standard tool citations and are not the load-bearing justification for the enrollment claim. The K-NN selection and identity-average computation are explicit algorithmic preprocessing steps, not parameters fitted to the enrollment labels. The comparisons in Table 3 use independent logistic-regression baselines, and CASIA-B provides an external benchmark. The paper's own cross-dataset results (Table 4, MCC from -0.007 to 0.242) seriously weaken the 'database-agnostic' claim, but that is an empirical generalization failure, not circular reasoning: the claim is contradicted by evidence rather than being true by definition. No fitted parameter is renamed as a prediction, no uniqueness result is imported from the authors' prior work, and no known result is merely relabeled. Accordingly, the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- K (number of nearest neighbors) =
8 in main results, varied in experiments
- SetTransformer architecture hyperparameters =
not specified
- Training/test id:walk ratios =
0.25, 0.5, 0.75
assumptions (3)
- domain assumption Embeddings from pretrained gait recognition models are discriminative enough that nearest-neighbor distances in embedding space reflect identity relationships.
- domain assumption The mean embedding of an identity's gallery samples is a sufficient summary of that identity for enrollment decisions.
- ad hoc to paper Training enrollment scenarios built from CASIA-B and PsyMo generalize to unseen databases.
Cite this review
Pith. "Pith review of Database-Agnostic Gait Enrollment using SetTransformers." pith.science (2026). https://pith.science/paper/X5IIXUJN
@misc{pith2026250502815,
author = {Pith},
title = {Pith review of: Database-Agnostic Gait Enrollment using SetTransformers},
year = {2026},
howpublished = {\url{https://pith.science/paper/X5IIXUJN}},
note = {Machine review of arXiv:2505.02815}
}
read the original abstract
Gait recognition has emerged as a powerful tool for unobtrusive and long-range identity analysis, with growing relevance in surveillance and monitoring applications. Although recent advances in deep learning and large-scale datasets have enabled highly accurate recognition under closed-set conditions, real-world deployment demands open-set gait enrollment, which means determining whether a new gait sample corresponds to a known identity or represents a previously unseen individual. In this work, we introduce a transformer-based framework for open-set gait enrollment that is both dataset-agnostic and recognition-architecture-agnostic. Our method leverages a SetTransformer to make enrollment decisions based on the embedding of a probe sample and a context set drawn from the gallery, without requiring task-specific thresholds or retraining for new environments. By decoupling enrollment from the main recognition pipeline, our model is generalized across different datasets, gallery sizes, and identity distributions. We propose an evaluation protocol that uses existing datasets in different ratios of identities and walks per identity. We instantiate our method using skeleton-based gait representations and evaluate it on two benchmark datasets (CASIA-B and PsyMo), using embeddings from three state-of-the-art recognition models (GaitGraph, GaitFormer, and GaitPT). We show that our method is flexible, is able to accurately perform enrollment in different scenarios, and scales better with data compared to traditional approaches. We will make the code and dataset scenarios publicly available.
Figures
Reference graph
Works this paper leans on
- [1]
-
[2]
A. Catruna, A. Cosma, and E. Radoi. Gaitpt: Skeletons are all you need for gait recognition. In 2024 IEEE 18th Inter- national Conference on Automatic Face and Gesture Recog- nition (FG), pages 1–10. IEEE, 2024
work page 2024
-
[3]
H. Chao, Y . He, J. Zhang, and J. Feng. Gaitset: Regarding gait as a set for cross-view gait recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8126–8133, 2019
2019
- [4]
-
[5]
A. Cosma and E. Radoi. Learning gait representations with noisy multi-task learning. Sensors, 22(18):6803, 2022
work page 2022
-
[6]
A. Cosma and E. Radoi. Psymo: A dataset for estimat- ing self-reported psychological traits from gait. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2024
work page 2024
-
[7]
A. Cosma and I. E. Radoi. Wildgait: Learning gait represen- tations from raw surveillance streams.Sensors, 21(24):8387, 2021
work page 2021
-
[8]
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 8
arXiv 2010
Show all 33 references
-
[9]
C. Fan, J. Ma, D. Jin, C. Shen, and S. Yu. Skeletongait: Gait recognition using skeleton maps. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 1662–1669, 2024
2024
-
[10]
C. Fan, Y . Peng, C. Cao, X. Liu, S. Hou, J. Chi, Y . Huang, Q. Li, and Z. He. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 14225–14233, 2020
2020
-
[11]
Gimeno-G ´omez, A.-M
D. Gimeno-G ´omez, A.-M. Bucur, A. Cosma, C.-D. Mart´ınez-Hinarejos, and P. Rosso. Reading between the frames: Multi-modal depression detection in videos from non-verbal cues. In N. Goharian, N. Tonellotto, Y . He, A. Li- pani, G. McDonald, C. Macdonald, and I. Ounis, editors,...
-
[12]
Gunther, S
M. Gunther, S. Cruz, E. M. Rudd, and T. E. Boult. Toward open-set face recognition. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017
2017
-
[13]
J. Lee, Y . Lee, J. Kim, A. Kosiorek, S. Choi, and Y . W. Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In International con- ference on machine learning , pages 3744–3753. PMLR, 2019
2019
-
[14]
B. Lin, S. Zhang, M. Wang, L. Li, and X. Yu. Gaitgl: Learning discriminative global-local feature representations for gait recognition. arXiv preprint arXiv:2208.01380, 2022
2022 arXiv
-
[15]
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012– 10022, 2021
2021
-
[16]
Matthews
B. Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Structure, 405(2):442–451, 1975
1975
-
[17]
Mazzieri, J
R. Mazzieri, J. Pegoraro, and M. Rossi. Open-set gait recognition from sparse mmwave radar point clouds. arXiv preprint arXiv:2503.07435, 2025
2025 arXiv
-
[18]
Ni and B
Z. Ni and B. Huang. Open-set human identification based on gait radar micro-doppler signatures. IEEE sensors journal , 21(6):8226–8233, 2021
2021
-
[19]
Parashar, A
A. Parashar, A. Parashar, A. F. Abate, R. S. Shekhawat, and I. Rida. Real-time gait biometrics for surveillance applica- tions: A review. Image and Vision Computing, 138:104784, 2023
2023
-
[20]
J. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif., 10, 06 2000
2000
-
[21]
Y . Shu, Y . Shi, Y . Wang, T. Huang, and Y . Tian. P-odn: Prototype-based open deep network for open set recognition. Scientific reports, 10(1):7146, 2020
2020
-
[22]
Y .-F. Song, Z. Zhang, C. Shan, and L. Wang. Stronger, faster and more explainable: A graph convolutional baseline for skeleton-based action recognition. Inproceedings of the 28th ACM international conference on multimedia , pages 1625– 1633, 2020
2020
-
[23]
Y . Sun, D. Liang, X. Wang, and X. Tang. Deepid3: Face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873, 2015
2015 arXiv
-
[24]
Teepe, A
T. Teepe, A. Khan, J. Gilg, F. Herzog, S. H ¨ormann, and G. Rigoll. Gaitgraph: Graph convolutional network for skeleton-based gait recognition. In 2021 IEEE international conference on image processing (ICIP) , pages 2314–2318. IEEE, 2021
2021
-
[25]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing sys- tems, 30, 2017
2017
-
[26]
S. Vaze, K. Han, A. Vedaldi, and A. Zisserman. Open-set recognition: A good closed-set classifier is all you need? 2021
2021
-
[27]
Y . Yang, Y . Ge, B. Li, Q. Wang, Y . Lang, and K. Li. Multisce- nario open-set gait recognition based on radar micro-doppler signatures. IEEE Transactions on Instrumentation and Mea- surement, 71:1–13, 2022
2022
-
[28]
Y . Yang, C. Hou, Y . Lang, D. Guan, D. Huang, and J. Xu. Open-set human activity recognition based on micro-doppler signatures. Pattern Recognition, 85:60–69, 2019
2019
-
[29]
H. Yu, Y . Fan, K. Chen, H. Yan, X. Lu, J. Liu, and D. Xie. Unknown identity rejection loss: Utilizing unlabeled data for face recognition. In Proceedings of the IEEE/CVF interna- tional conference on computer vision workshops, pages 0–0, 2019
2019
-
[30]
S. Yu, D. Tan, and T. Tan. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In 18th international conference on pattern recognition (ICPR’06) , volume 4, pages 441–444. IEEE, 2006
2006
-
[31]
Zhang, A
H. Zhang, A. Li, J. Guo, and Y . Guo. Hybrid models for open set recognition. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 102–117. Springer, 2020
2020
-
[32]
Z. Zhu, X. Guo, T. Yang, J. Huang, J. Deng, G. Huang, D. Du, J. Lu, and J. Zhou. Gait recognition in the wild: A benchmark. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14789–14799, 2021. 9
2021
-
[2024]
Springer Nature Switzerland
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.