Pith. sign in

REVIEW 3 major objections 4 minor 33 references

Database-Agnostic Gait Enrollment using SetTransformers

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read The paper claims a SetTransformer that sees a probe embedding and its nearest gallery neighbors can decide known-vs-new enrollment without thresholds or retraining, transferring across datasets, recognition models, and gallery sizes.

desk verdict A useful problem framing and a clean evaluation protocol for open-set gait enrollment, but the paper's own cross-dataset numbers contradict the headline claim of database-agnostic performance. read the letter →

arxiv 2505.02815 v1 pith:X5IIXUJN submitted 2025-05-05 cs.CV

classification cs.CV
keywords open-setgaitenrollmentSetTransformerrecognitionembeddingsdatabase-agnosticthreshold-freeskeleton-basedgallerycontextid:walkratio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-set gait enrollment asks a surveillance or identification system to decide whether a new walking sample belongs to someone already in its gallery or to a person who should be enrolled as a new identity. The paper argues that this decision can be learned once and reused everywhere: it trains a SetTransformer on the probe embedding plus a small context set of nearest-neighbor gallery embeddings and their identity-average embeddings, and the model outputs an enrollment decision without any threshold that an operator must retune per database. This matters because classical approaches require manually tuned thresholds or per-dataset retraining, which does not scale when galleries and identity sets change. The paper tests the idea with skeleton-based embeddings from three recognition models on two datasets, across gallery sizes and id:walk ratios, and reports that the model outperforms logistic-regression baselines and improves with more neighbors, identity-linked positional encodings, and variable training databases.

What carries the argument

The machinery is a SetTransformer, an attention-based permutation-invariant neural network that consumes an unordered set of embeddings. For a probe embedding $p$, the method selects its $K$ nearest gallery neighbors by embedding distance, and for each neighbor $g_k$ it forms an identity-average embedding $\overline{\mathrm{id}}_{g_k}=\frac{1}{|G_{g_k}|}\sum_{g_i\in G_{g_k}} g_i$, where $G_{g_k}$ gathers all gallery embeddings of that identity. The input set $\{p, g_k, \overline{\mathrm{id}}_{g_k}\}$ is passed through self-attention, and a two-layer MLP reads the output at the probe's position to produce the enrollment decision. The paper compares additive pairing, per-instance positional encodings, and per-identity positional encodings as ways to link each neighbor to its identity average, and finds that one shared position code per identity works best, because it lets the model attend to identity groups rather than isolated samples.

What would settle it

Run the enrollment model trained only on CASIA-B and PsyMo scenarios on a third gait database with different identities, capture conditions, and id:walk ratios; if MCC there is near zero, the 'database-agnostic' claim is false. The paper's Table 4 already provides a partial test, with cross-dataset MCC between $-0.007$ and $0.242$.

Watch

Extended reading notes

Core claim

The central claim is that gait enrollment can be decoupled from gait recognition: a recognition model produces embeddings, and a separate permutation-invariant SetTransformer makes the open-set decision by attending to the probe embedding together with its $K$ nearest gallery neighbors and the per-identity mean embedding of each neighbor. The paper presents this as a single framework that is agnostic to the recognition architecture and to the database, because it learns from the gallery context itself rather than from a fixed gallery size or identity set. On CASIA-B and PsyMo, using embeddings from three skeleton-based recognition models, the paper reports that the method achieves high enrollment accuracy in balanced and well-sampled scenarios, scales better with data than the logistic-regression baselines, and benefits from training on variable gallery configurations, larger $K$, and per-identity positional encodings. The paper also reports cross-dataset and cross-model transfer experiments; in those runs the numbers are much lower, with cross-dataset MCC between $-0.007$ and $0.242$.

Load-bearing premise

The load-bearing premise is that enrollment scenarios cut from CASIA-B and PsyMo represent other gait databases, so a model trained on those two transfers with no retraining; the paper's Table 4 cross-dataset MCCs, as low as $-0.007$ and at most $0.242$, show this premise is the point most likely to give way.

Editorial extensions

If this is right

  • A deployed gait system could run recognition and enrollment with one shared embedding space: the recognition model identifies who the person is, and the SetTransformer decides whether that person is already known.
  • Operator overhead falls because the enrollment decision no longer requires manually tuned similarity thresholds that need recalibration when the gallery changes.
  • Performance increases with the number of neighbors $K$ and with per-identity context, so systems with more gallery evidence can buy accuracy by widening the context set.
  • Training on variable gallery configurations, with different identity counts and walks per identity, is the mechanism behind generalization; fixed-gallery training is shown to be worse.
  • Scenarios with many identities but very few walks per identity remain the hard regime, which tells data collectors that more walks per identity matter for enrollment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would be to apply the same context-set formulation to open-set enrollment for other biometric embeddings, such as face or person re-identification, since the model only consumes embeddings and gallery structure; the paper does not test those modalities.
  • The cross-dataset and cross-model tables suggest the honest scope is transfer within the distribution of controlled laboratory gait trips rather than transfer to arbitrary databases; a sharper name for the property would be gallery-context generalization rather than database agnosticism.
  • A testable extension is to feed a small amount of unlabeled walks from the target database into the gallery context at inference time, letting the SetTransformer adapt its context statistics before enrollment rather than freezing the model entirely.
  • The method could be compared against a train-free baseline that uses the ratio of the probe-to-nearest-neighbor distance to the probe-to-second-nearest-neighbor distance, which is a standard open-set heuristic; the paper only compares trained logistic baselines.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper addresses open-set gait enrollment: deciding whether a probe gait embedding belongs to a known identity in a gallery or is a newcomer. The authors propose a SetTransformer-based binary classifier whose input is a probe embedding, its K nearest gallery neighbors, and per-identity average embeddings, with three variants for encoding neighbor-identity correspondence. They train and evaluate this model on enrollment scenarios constructed from CASIA-B and PsyMo at different identity-to-walk ratios, using embeddings from GaitPT, GaitFormer, and GaitGraph. The paper claims the method is database-agnostic and recognition-architecture-agnostic, requiring no task-specific thresholds or retraining, and reports within-dataset results, comparisons with logistic-regression baselines, and cross-dataset and cross-embedding transfer experiments.

Significance. If the central claims were established, the method would address a genuine deployment gap: automatic enrollment decisions in gait recognition systems, decoupled from the recognition backbone and applicable across databases without threshold tuning. The paper has strengths: it systematically varies id:walk ratios, compares against nontrivial baselines, and promises to release code and dataset scenarios. However, the headline claims of database-agnostic and architecture-agnostic behavior are not supported by the paper's own transfer experiments, and the within-dataset advantage over a simple baseline is inconsistent across ratios. The contribution is therefore potentially useful as a within-dataset enrollment model with known transfer limitations, but the current framing considerably overstates the evidence.

major comments (3)
  1. [§4, Table 4] The cross-dataset results directly contradict the 'database-agnostic' claim in the Abstract and Discussion. Table 4 reports MCCs between -0.007 and 0.242 across all transfer directions and embedding models, with GaitGraph PsyMo→CASIA-B at -0.007 (chance-level). The paper's own text in §4 acknowledges 'a clear domain gap.' Since the method is claimed to generalize 'across different settings of identities, samples per identities, gait recognition architectures, and datasets' (Discussion), the load-bearing assumption that enrollment scenarios from CASIA-B and PsyMo transfer to other databases is not merely unverified but contradicted by the reported numbers. To substantiate the claim, the authors would need a successful held-out third-dataset experiment (e.g., training on the union of CASIA-B and PsyMo and testing on a different gait database) or a substantial reframing of the contribution as within-dataset enrollment only.
  2. [§4, Table 3] The claim that the SetTransformer 'scales better with data compared to traditional approaches' is only supported at ID:Walk ratio 0.75. At ratio 0.25, the logistic regression on neighbor distances achieves MCC 0.844 versus the SetTransformer's 0.615; at ratio 0.5, the baseline MCC is 0.832 versus 0.830, essentially a tie. The proposed method clearly wins only in the 0.75 setting. The comparison should be reported across all ratios and datasets, with error bars or significance tests, before claiming a consistent scaling advantage. As written, Table 3 shows the baseline is competitive or superior in two of three conditions.
  3. [§4, Table 5 and §5] The 'model-agnostic' claim is also not supported by the cross-embedding transfer results. Table 5 reports near-zero MCCs in several scenarios, e.g., GaitGraph→GaitFormer on PsyMo (MCC 0.000), GaitFormer→GaitPT on PsyMo (MCC 0.023), and GaitGraph→GaitPT on PsyMo (MCC 0.025). If the enrollment model were truly independent of the recognition architecture, one would expect more consistent transfer across embedding spaces; the large variability (MCC from 0.000 to 0.400 in the same dataset) indicates strong sensitivity to the embedding source. The Discussion's claim that the method generalizes 'across ... gait recognition architectures' should be qualified accordingly.
minor comments (4)
  1. [§4] In the cross-dataset evaluation paragraph, the text states 'GaitGraph ones nearly collapse in the opposite direction (MCC of 0.007)', but Table 4 reports -0.007 for this entry. The sign is important because -0.007 indicates chance or slightly inverted prediction; please correct the inconsistency.
  2. [Tables 4 and 5 captions] Both Table 4 and Table 5 have the caption 'Cross-embedding enrollment performance', but Table 4 crosses datasets (train dataset → test dataset) while Table 5 crosses embedding models (train embedding → test embedding). Rename Table 4 to 'Cross-dataset enrollment performance' to avoid confusion.
  3. [§3.1] The construction of training and test scenarios should clarify how probe walks from 'new IDs' are chosen so that no identity overlaps between the gallery and probe subsets within an enrollment scenario; otherwise, the measured F1/MCC could be inflated by trivial identity leakage.
  4. [Figures 3-6] The figures report MCC without confidence intervals or repeated-seed variability. Given the large differences attributed to K, id:walk ratio, and training database variability, error bars or multiple-seed results would strengthen the claims and help readers assess stability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the enrollment classifier is trained from labeled gallery/probe contexts and is not derived from the quantity it predicts; self-citations are to tools, not to load-bearing unverified results.

full rationale

The paper's central contribution is a supervised SetTransformer that takes a probe embedding, K nearest-neighbor gallery embeddings, and identity-average embeddings as input and predicts an enrollment label. The predicted quantity (known vs. new identity) is not used to define the input features, the loss, or the training scenarios, so there is no self-definitional reduction: the model's output is not equal to its input by construction. The recognition embeddings from GaitPT, GaitFormer, and GaitGraph are used as fixed feature extractors, and the paper trains these architectures itself rather than relying on an unverified cited theorem; self-citations to GaitPT, GaitFormer, and PsyMo identify the authors' own models and dataset, but these are standard tool citations and are not the load-bearing justification for the enrollment claim. The K-NN selection and identity-average computation are explicit algorithmic preprocessing steps, not parameters fitted to the enrollment labels. The comparisons in Table 3 use independent logistic-regression baselines, and CASIA-B provides an external benchmark. The paper's own cross-dataset results (Table 4, MCC from -0.007 to 0.242) seriously weaken the 'database-agnostic' claim, but that is an empirical generalization failure, not circular reasoning: the claim is contradicted by evidence rather than being true by definition. No fitted parameter is renamed as a prediction, no uniqueness result is imported from the authors' prior work, and no known result is merely relabeled. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. Its central claim rests on the discriminative quality of externally trained embeddings and on the transferability of lab-dataset scenarios, neither of which is established.

free parameters (3)
  • K (number of nearest neighbors) = 8 in main results, varied in experiments
    The SetTransformer context size is a user-chosen hyperparameter, not learned; results depend strongly on it (Figure 4).
  • SetTransformer architecture hyperparameters = not specified
    Number of attention heads, layers, MLP dimensions, dropout, and noise augmentation are not reported, so the model cannot be re-implemented exactly.
  • Training/test id:walk ratios = 0.25, 0.5, 0.75
    These scenario ratios are chosen by the authors to simulate imbalance; they are not derived from any theory and are not fitted to data.
assumptions (3)
  • domain assumption Embeddings from pretrained gait recognition models are discriminative enough that nearest-neighbor distances in embedding space reflect identity relationships.
    Assumed throughout Section 3.2 where K nearest neighbors and identity averages are used as context; if embeddings are poor (GaitGraph accuracy 0.304 on PsyMo in Table 2), enrollment context may be uninformative.
  • domain assumption The mean embedding of an identity's gallery samples is a sufficient summary of that identity for enrollment decisions.
    Used in Section 3.2 to define the identity embedding id_gk; no evidence is given that averaging is sufficient, and it discards within-identity variance.
  • ad hoc to paper Training enrollment scenarios built from CASIA-B and PsyMo generalize to unseen databases.
    This is the central premise of the 'database-agnostic' claim (Abstract, Section 5). Table 4 shows large domain gaps, so this axiom is not supported by the paper's own data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Database-Agnostic Gait Enrollment using SetTransformers." pith.science (2026). https://pith.science/paper/X5IIXUJN

@misc{pith2026250502815,
  author       = {Pith},
  title        = {Pith review of: Database-Agnostic Gait Enrollment using SetTransformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X5IIXUJN}},
  note         = {Machine review of arXiv:2505.02815}
}
read the original abstract

Gait recognition has emerged as a powerful tool for unobtrusive and long-range identity analysis, with growing relevance in surveillance and monitoring applications. Although recent advances in deep learning and large-scale datasets have enabled highly accurate recognition under closed-set conditions, real-world deployment demands open-set gait enrollment, which means determining whether a new gait sample corresponds to a known identity or represents a previously unseen individual. In this work, we introduce a transformer-based framework for open-set gait enrollment that is both dataset-agnostic and recognition-architecture-agnostic. Our method leverages a SetTransformer to make enrollment decisions based on the embedding of a probe sample and a context set drawn from the gallery, without requiring task-specific thresholds or retraining for new environments. By decoupling enrollment from the main recognition pipeline, our model is generalized across different datasets, gallery sizes, and identity distributions. We propose an evaluation protocol that uses existing datasets in different ratios of identities and walks per identity. We instantiate our method using skeleton-based gait representations and evaluate it on two benchmark datasets (CASIA-B and PsyMo), using embeddings from three state-of-the-art recognition models (GaitGraph, GaitFormer, and GaitPT). We show that our method is flexible, is able to accurately perform enrollment in different scenarios, and scales better with data compared to traditional approaches. We will make the code and dataset scenarios publicly available.

Figures

Figures reproduced from arXiv: 2505.02815 by the authors.

Figure 1
Figure 1. Overview of enrollment training and evaluation scenarios. Starting from an initial gallery of identity embeddings, probes are [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overall diagram of our model. A probe embedding at [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. The effect of increasing the number of neighbors [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Performance across ID: Walk ratios (0.25, 0.5, 0.75) on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 6
Figure 6. Figure 6: Enrollment MCC on CASIA-B and PsyMo for fixed [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 27 canonical work pages

  1. [1]

    Bashir, T

    K. Bashir, T. Xiang, and S. Gong. Gait recognition without subject cooperation. Pattern Recognition Letters , 31(13):2052–2060, 2010

  2. [2]

    Catruna, A

    A. Catruna, A. Cosma, and E. Radoi. Gaitpt: Skeletons are all you need for gait recognition. In 2024 IEEE 18th Inter- national Conference on Automatic Face and Gesture Recog- nition (FG), pages 1–10. IEEE, 2024

  3. [3]

    H. Chao, Y . He, J. Zhang, and J. Feng. Gaitset: Regarding gait as a set for cross-view gait recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8126–8133, 2019

  4. [4]

    Cosma, A

    A. Cosma, A. Catruna, and E. Radoi. Exploring self- supervised vision transformers for gait recognition in the wild. Sensors, 23(5):2680, 2023

  5. [5]

    Cosma and E

    A. Cosma and E. Radoi. Learning gait representations with noisy multi-task learning. Sensors, 22(18):6803, 2022

  6. [6]

    Cosma and E

    A. Cosma and E. Radoi. Psymo: A dataset for estimat- ing self-reported psychological traits from gait. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2024

  7. [7]

    Cosma and I

    A. Cosma and I. E. Radoi. Wildgait: Learning gait represen- tations from raw surveillance streams.Sensors, 21(24):8387, 2021

  8. [8]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 8

Show all 33 references
  1. [9]

    C. Fan, J. Ma, D. Jin, C. Shen, and S. Yu. Skeletongait: Gait recognition using skeleton maps. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 1662–1669, 2024

  2. [10]

    C. Fan, Y . Peng, C. Cao, X. Liu, S. Hou, J. Chi, Y . Huang, Q. Li, and Z. He. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 14225–14233, 2020

  3. [11]

    Gimeno-G ´omez, A.-M

    D. Gimeno-G ´omez, A.-M. Bucur, A. Cosma, C.-D. Mart´ınez-Hinarejos, and P. Rosso. Reading between the frames: Multi-modal depression detection in videos from non-verbal cues. In N. Goharian, N. Tonellotto, Y . He, A. Li- pani, G. McDonald, C. Macdonald, and I. Ounis, editors,...

  4. [12]

    Gunther, S

    M. Gunther, S. Cruz, E. M. Rudd, and T. E. Boult. Toward open-set face recognition. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017

  5. [13]

    J. Lee, Y . Lee, J. Kim, A. Kosiorek, S. Choi, and Y . W. Teh. Set transformer: A framework for attention-based permutation-invariant neural networks. In International con- ference on machine learning , pages 3744–3753. PMLR, 2019

  6. [14]

    B. Lin, S. Zhang, M. Wang, L. Li, and X. Yu. Gaitgl: Learning discriminative global-local feature representations for gait recognition. arXiv preprint arXiv:2208.01380, 2022

  7. [15]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012– 10022, 2021

  8. [16]

    Matthews

    B. Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Structure, 405(2):442–451, 1975

  9. [17]

    Mazzieri, J

    R. Mazzieri, J. Pegoraro, and M. Rossi. Open-set gait recognition from sparse mmwave radar point clouds. arXiv preprint arXiv:2503.07435, 2025

  10. [18]

    Ni and B

    Z. Ni and B. Huang. Open-set human identification based on gait radar micro-doppler signatures. IEEE sensors journal , 21(6):8226–8233, 2021

  11. [19]

    Parashar, A

    A. Parashar, A. Parashar, A. F. Abate, R. S. Shekhawat, and I. Rida. Real-time gait biometrics for surveillance applica- tions: A review. Image and Vision Computing, 138:104784, 2023

  12. [20]

    J. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif., 10, 06 2000

  13. [21]

    Y . Shu, Y . Shi, Y . Wang, T. Huang, and Y . Tian. P-odn: Prototype-based open deep network for open set recognition. Scientific reports, 10(1):7146, 2020

  14. [22]

    Y .-F. Song, Z. Zhang, C. Shan, and L. Wang. Stronger, faster and more explainable: A graph convolutional baseline for skeleton-based action recognition. Inproceedings of the 28th ACM international conference on multimedia , pages 1625– 1633, 2020

  15. [23]

    Y . Sun, D. Liang, X. Wang, and X. Tang. Deepid3: Face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873, 2015

  16. [24]

    Teepe, A

    T. Teepe, A. Khan, J. Gilg, F. Herzog, S. H ¨ormann, and G. Rigoll. Gaitgraph: Graph convolutional network for skeleton-based gait recognition. In 2021 IEEE international conference on image processing (ICIP) , pages 2314–2318. IEEE, 2021

  17. [25]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing sys- tems, 30, 2017

  18. [26]

    S. Vaze, K. Han, A. Vedaldi, and A. Zisserman. Open-set recognition: A good closed-set classifier is all you need? 2021

  19. [27]

    Y . Yang, Y . Ge, B. Li, Q. Wang, Y . Lang, and K. Li. Multisce- nario open-set gait recognition based on radar micro-doppler signatures. IEEE Transactions on Instrumentation and Mea- surement, 71:1–13, 2022

  20. [28]

    Y . Yang, C. Hou, Y . Lang, D. Guan, D. Huang, and J. Xu. Open-set human activity recognition based on micro-doppler signatures. Pattern Recognition, 85:60–69, 2019

  21. [29]

    H. Yu, Y . Fan, K. Chen, H. Yan, X. Lu, J. Liu, and D. Xie. Unknown identity rejection loss: Utilizing unlabeled data for face recognition. In Proceedings of the IEEE/CVF interna- tional conference on computer vision workshops, pages 0–0, 2019

  22. [30]

    S. Yu, D. Tan, and T. Tan. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In 18th international conference on pattern recognition (ICPR’06) , volume 4, pages 441–444. IEEE, 2006

  23. [31]

    Zhang, A

    H. Zhang, A. Li, J. Guo, and Y . Guo. Hybrid models for open set recognition. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pages 102–117. Springer, 2020

  24. [32]

    Z. Zhu, X. Guo, T. Yang, J. Huang, J. Deng, G. Huang, D. Du, J. Lu, and J. Zhou. Gait recognition in the wild: A benchmark. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14789–14799, 2021. 9

  25. [2024]

    Springer Nature Switzerland

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.