REVIEW 4 major objections 5 minor 40 references
With 10 samples per class, a text-to-motion diffusion prior can be distilled into a specialized action generator, raising median top-1 HAR accuracy from 63.1% to 86.2%.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 16:46 UTC pith:JW3B6DX7
load-bearing objection The CLIP-based kinetic-mining pipeline is a genuinely new way to turn a T2M prior into few-shot HAR augmentation, but the paper's own ablation undercuts its headline configuration and the gain over a plain pretrained MDM is only 2.3 pp. the 4 major comments →
Kinetic Mining in Context: Few-Shot Action Synthesis via Text-to-Motion Distillation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that a generalist diffusion motion prior, whose training objective favors broad artistic motion, can be repurposed for HAR's kinematically precise action classes by mining context. The method pairs each scarce target sample with semantically matched source captions via CLIP text embeddings, then learns frame-level correspondences through a contrastively trained module whose attention identifies the source window most kinematically consistent with the target action. That window is used as a pseudo-labeled training signal, with a similarity-based weight modulating its influence. The result is that the student remains specialized without collapsing: with 30 real samples
What carries the argument
The load-bearing component is the Mining In Context (MIC) module, an attention-enhanced bidirectional GRU that ingests frame-wise tokens from both the frozen teacher and the trainable student streams and produces context-aware latent vectors. A Soft Nearest Neighbors contrastive loss aligns latents of soft-positive pairs (target sample vs. retrieved source motion), so the attention distribution learns which frames of the long source motion are kinematically relevant. A max-cumulative-attention window extraction selects the matching subsequence, which is then used as a pseudo-label for the student's reconstruction and distillation losses, scaled by a dynamic window weight (dww) equal to the c
Load-bearing premise
The whole retrieval-and-mining chain assumes that captions whose text embeddings are similar to a target label describe motions whose kinematics are actually close to the target motion; when that fails (e.g., 'punch' vs. 'kick'), the mined windows are kinematically wrong and the filtering weight cannot recover.
What would settle it
Run KineMIC with the same hyperparameters on action classes that share high CLIP text similarity but different kinematics, such as punch vs. kick or wave vs. salute, using 10 samples per class. If median top-1 accuracy with augmentation drops below the real-data-only baseline (or shows no improvement), the semantic-proxy assumption fails; conversely, if dww filters bad windows successfully, the approach generalizes beyond the three easy actions tested.
If this is right
- If T2M priors can be specialized this cheaply, few-shot HAR augmentation no longer depends on training a GAN from scratch; a frozen generalist plus lightweight adaptation suffices.
- The +23.1 percentage-point gain over real-data-only training indicates that synthetic augmentation from a semantic-matched prior is viable even with only 10 samples per class.
- The framework's components transfer to any pair of source text-motion collection and target action-label domain sharing a text-embedding space, provided the mining filter (dww) is retained.
- The ablation that adding distillation alone hurts (83.78% vs. 85.21%) implies the teacher's generalist behavior must be filtered, not blindly imitated.
- KineMIC's dual conditioning enables a form of motion composition between action id and text prompt, as in the 'stretch while jumping' example.
Where Pith is reading between the lines
- Editorial: The decisive test is whether the semantic-proxy assumption holds on hard pairs (punch vs. kick, or actions with near-duplicate text embeddings but different kinematics). Running the same protocol on such classes would likely reveal that dww alone cannot recover when no retrieved window is kinematically correct.
- Editorial: The method's scalability to larger label sets is untested. With more action classes, the top-k retrieval per class could overlap heavily, and the contrastive batch may need harder negatives; a natural extension is to increase k or add class-aware negative sampling.
- Editorial: Because the pipeline already yields composition between action and text, one could test whether KineMIC can generate motions for unseen compositions of seen actions without retraining, which would extend its use beyond augmentation to interactive motion editing.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes KineMIC, a teacher-student framework that adapts a pre-trained text-to-motion (T2M) diffusion model (MDM) to a few-shot skeleton-based HAR task. The target action labels are matched to T2M source captions via CLIP text embeddings (soft positive search); a MIC module (attention-enhanced biGRU) aligns target and source motion latents with a contrastive loss; relevant source-motion windows are mined by attention and used as pseudo-training samples for the student, with a dynamic window weighting (dww) and a window-distillation loss. Experiments on three NTU RGB+D 120 actions with 10 samples per class report 86.2% median top-1 accuracy versus 63.1% for real-data-only training, approaching the 86.4% of prior GAN-based work. The paper includes ablations, diversity/multimodality metrics, and qualitative examples.
Significance. If the results hold, the paper demonstrates a meaningful new use of general T2M diffusion priors for few-shot HAR augmentation, offering an alternative to GAN-based data synthesis. The method is relatively simple, uses standard components, and the experimental setup is mostly described in sufficient detail to reproduce. The honest ablation study is a strength: it identifies the dynamic window weighting as the key component and reports that the distillation loss is harmful. However, the paper's central claim is weakened by a load-bearing inconsistency between the described complete loss (Eq. 7, which includes L_dist) and the ablation result that the best configuration excludes L_dist. The reported +23.1 pp gain is also measured against a weak real-data-only baseline; the improvement over simply using the pre-trained MDM as a generator is only about +2.3 pp. The paper's own stated limitation that semantic similarity is an unreliable proxy for kinematic relevance is not tested on hard cases, so the generality of the framework for arbitrary actions is unsubstantiated. The contribution is promising but needs revision to align claims with evidence.
major comments (4)
- [§5.2 Table 1 vs §5.3 Table 2 and Eq. (7)] The headline 'KineMIC 86.2' in Table 1 is not tied to a stated configuration. Table 2 shows that adding L_dist (Eq. 5) to the base model reduces accuracy from 85.21±2.24 to 83.78±2.38, and that the full Eq. (7) loss (L_dist + dww) gives 84.94±2.37, below dww-only at 86.41±0.95. If the 86.2% result comes from the dww-only configuration, then the paper's described 'complete multi-objective loss' and the title's 'distillation' mechanism are not what drive the gain; if it comes from the full Eq. (7) loss, it contradicts Table 2. Please state explicitly which loss configuration produced each row of Table 1 and revise the abstract/conclusions so that the described method matches the best evaluated configuration.
- [§4.5 and §5.3] The window-distillation loss L_dist is presented as a core component of the framework (Eq. 5 and Eq. 7), but the ablation shows it is net harmful for accuracy while increasing diversity/multimodality. The paper acknowledges this in the Table 2 discussion, but the method description, abstract, and contributions still frame distillation as central. The paper should be restructured so that the actual operating method is the best configuration (base + dynamic window weighting), with L_dist explicitly described as an optional/ablated variant. Otherwise the scientific claim is not internally consistent.
- [§6 and §4.2] The core assumption that CLIP text-space similarity is a proxy for kinematic relevance is acknowledged in the limitations as unreliable for pairs like punch vs. kick, yet all three evaluated actions (running on spot, side kick, stretch on self) are simple actions that the paper itself says a generalist T2M model can already generate. This means the evaluation does not test the framework under the failure mode the authors identify. To support the general claim of adapting T2M priors to arbitrary HAR actions, the paper should either include harder, semantically confusable action classes or explicitly restrict the claim to actions for which semantic retrieval is reliable.
- [§5.3, Table 2] The label 'dww only' is ambiguous. The dww score (Eq. 6) is computed from MIC latents z_T and z_P*, and the 'Base' row already includes the contrastive loss (Eq. 1) and target reconstruction (Eq. 3). If 'dww only' means 'Base + dww', this should be stated explicitly; as written, a reader could infer that the dynamic weighting works without the contrastive alignment, which would be circular because dww is a function of the aligned MIC embeddings.
minor comments (5)
- [Abstract] The phrase '+23.1% accuracy points improvement' should be '+23.1 percentage points' or '+23.1 pp' to avoid ambiguity with relative improvement.
- [Table 1 caption] Typo: 'T able' should be 'Table'.
- [Fig. 4 caption] Action labels are inconsistent: figure says 'Run on spot', 'Side kick', 'Stretch on self' while the text uses 'running on spot', 'side kick', 'stretch on self'. Please standardize.
- [§5.1] The notation D¯T is introduced but used inconsistently (e.g., '¯T ⊂ T' vs. 'D¯T'). Please use a single notation for the few-shot training subset.
- [§5.2] Table 1 reports median top-1 accuracy while Table 2 reports mean±std. The text should state this explicitly near the tables so readers do not compare the two tables directly as if they are the same statistic.
Circularity Check
No central derivation-equals-input circularity; the dww window-quality filter is self-referential because it is computed from the same MIC similarity that Eq. 1 was trained to maximize for soft-positive pairs.
specific steps
-
fitted input called prediction
[Sec. 4.3 Eq. (1) vs Sec. 4.5 Eq. (6); Sec. 4.4 window mining]
"MIC is trained to align z¯T_i with z¯P_j from soft-positive pairs sharing the same target class y. ... we compute a window quality score: dww = (1 + sim(z¯T, z^P*))/2 ... z^P* is computed with gradients detached, ensuring MIC remains trained exclusively by Eq. 1. The score dww scales both reconstruction (Eq. 4) and distillation (Eq. 5) losses, giving higher weight to more reliable prior windows."
The 'quality' score is the cosine similarity that Lcon (Eq. 1) was trained to maximize between target latents and soft-positive source latents. Soft positives are retrieved by CLIP semantics and can be kinematically wrong, as the paper concedes for 'punch' vs 'kick'; MIC's attention is trained to pick frames minimizing this alignment loss, so the mined-window latent z^P* inherits the same trained similarity. Thus dww is high by construction for any retrieved soft-pair, rather than measuring independent kinematic reliability. Using it to up/down-weight windows renames the contrastive training objective as a filter.
full rationale
No self-citations, imported uniqueness theorems, or ansatz-by-citation were found. The headline accuracy gain (+23.1 pp) is evaluated on held-out real NTU RGB+D 120 test samples, so the central performance claim is externally grounded and does not reduce to the training objectives. The paper's own ablations show the full Eq. 7 configuration underperforms dww-only (84.94±2.37 vs 86.41±0.95), and the configuration behind Table 1's 86.2 is not stated; that is a reproducibility/correctness concern, not a circularity. The only construction-level circularity is the dww score: it uses MIC embeddings that Eq. 1 explicitly trains to make soft-positive pairs similar, and the mining attention is trained toward the same objective, so Eq. 6 is the training objective renamed as a quality filter. This affects a supporting component of the framework, not the external benchmark, so the overall circularity score is moderate.
Axiom & Free-Parameter Ledger
free parameters (6)
- soft positive retrieval count k =
250
- contrastive temperature τ =
0.07
- LoRA rank/α/dropout =
16 / 32 / 0.1
- loss weights λ_T_rec, λ_con, λ_P*_rec, λ_dist =
1.0 each
- CFG guidance scale =
2.5
- training steps =
5000
axioms (4)
- domain assumption CLIP text embedding similarity between HAR label prompts and T2M captions is a reliable proxy for kinematic relevance (Sec. 4.2).
- domain assumption MDM pretrained on HumanML3D transfers to NTU skeletons after VIBE estimation, joint dropping, fps downsampling, and HumanML3D normalization (Sec. 5.1).
- domain assumption A frozen teacher GP guides LoRA fine-tuned student GT without collapse, preserving prior kinematics (Sec. 4.1).
- standard math Diffusion model forward/backward processes and contrastive (Soft Nearest Neighbor) losses are valid background (Eq. 1-5).
invented entities (2)
-
MIC module (attention-enhanced biGRU encoder)
no independent evidence
-
Learnable action embedding
no independent evidence
read the original abstract
The acquisition cost for large, annotated motion datasets remains a critical bottleneck for skeletal-based Human Activity Recognition (HAR). Although Text-to-Motion (T2M) generative models offer a compelling, scalable source of synthetic data, their training objectives, which emphasize general artistic motion, and dataset structures fundamentally differ from HAR's requirements for kinematically precise, class-discriminative actions. This disparity creates a significant domain gap, making generalist T2M models ill-equipped for generating motions suitable for HAR classifiers. To address this challenge, we propose KineMIC (Kinetic Mining In Context), a transfer learning framework for few-shot action synthesis. KineMIC adapts a T2M diffusion model to an HAR domain by hypothesizing that semantic correspondences in the text encoding space can provide soft supervision for kinematic distillation. We operationalize this via a kinetic mining strategy that leverages CLIP text embeddings to establish correspondences between sparse HAR labels and T2M source data. This process guides fine-tuning, transforming the generalist T2M backbone into a specialized few-shot Action-to-Motion generator. We validate KineMIC using HumanML3D as the source T2M dataset and a subset of NTU RGB+D 120 as the target HAR domain, randomly selecting just 10 samples per action class. Our approach generates significantly more coherent motions, providing a robust data augmentation source that delivers a +23.1% accuracy points improvement. Animated illustrations and supplementary materials are available at https://lucazzola.github.io/kinemic-page/.
Figures
Reference graph
Works this paper leans on
-
[1]
In: ICLR (2015)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
2015
-
[2]
In: ICCV (2021)
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., Hu, W.: Channel-wise topol- ogy refinement graph convolution for skeleton-based action recognition. In: ICCV (2021)
2021
-
[3]
In: CVPR (2022)
Chi, H.G., Ha, M.H., Chi, S., Lee, S.W., Huang, Q., Ramani, K.: InfoGCN: Repre- sentation learning for human skeleton-based action recognition. In: CVPR (2022)
2022
-
[4]
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recur- rent neural networks on sequence modeling. arXiv:1412.3555 (2014)
Pith/arXiv arXiv 2014
-
[5]
In: WACV (2022)
Degardin, B., Neves, J., Lopes, V., Brito, J., Yaghoubi, E., Proenca, H.: Generative adversarial graph convolutional networks for human action synthesis. In: WACV (2022)
2022
-
[6]
Expert Syst
Dentamaro, V., Gattulli, V., Impedovo, D., Manca, F.: Human activity recognition with smartphone-integrated sensors: A survey. Expert Syst. Appl. (2024)
2024
-
[7]
In: ECCV (2024)
Do, J., Kim, M.: Skateformer: Skeletal-temporal transformer for human action recognition. In: ECCV (2024)
2024
-
[8]
In: CVPR (2015)
Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: CVPR (2015)
2015
-
[9]
In: ACM MM (2022)
Duan,H.,Wang,J.,Chen,K.,Lin,D.:PYSKL:Towardsgoodpracticesforskeleton action recognition. In: ACM MM (2022)
2022
-
[10]
In: CVPR (2022)
Duan, H., Zhao, Y., Chen, K., Lin, D., Dai, B.: Revisiting skeleton-based action recognition. In: CVPR (2022)
2022
-
[11]
In: ICML (2019)
Frosst, N., Papernot, N., Hinton, G.: Analyzing and improving representations with the soft nearest neighbor loss. In: ICML (2019)
2019
-
[12]
In: WACV (2024)
Fukushi, K., Nozaki, Y., Nishihara, K., Nakahara, K.: Few-shot generative model for skeleton-based human action synthesis using cross-domain adversarial learning. In: WACV (2024)
2024
-
[13]
In: CVPR (2024)
Guo, C., Mu, Y., Javed, M.G., Wang, S., Cheng, L.: Momask: Generative masked modeling of 3d human motions. In: CVPR (2024)
2024
-
[14]
In: CVPR (2022)
Guo, C., Zou, S., Zuo, X., Wang, S., Ji, W., Li, X., Cheng, L.: Generating diverse and natural 3d human motions from text. In: CVPR (2022)
2022
-
[15]
In: ACM MM (2020)
Guo, C., Zuo, X., Wang, S., Zou, S., Sun, Q., Deng, A., Gong, M., Cheng, L.: Action2motion: Conditioned generation of 3d human motions. In: ACM MM (2020)
2020
-
[16]
In: ICLR (2022)
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: ICLR (2022)
2022
-
[17]
Sensors (Basel, Switzerland) (2023)
Hung-Cuong, N., Nguyen, T.H., Scherer, R., Le, V.H.: Deep learning for human activity recognition on 3d human skeleton: Survey and comparative study. Sensors (Basel, Switzerland) (2023)
2023
-
[18]
In: ECCV (2024) Kinetic Mining in Context 15
Khirodkar, R., Bagautdinov, T., Martinez, J., Zhaoen, S., James, A., Selednik, P., Anderson, S., Saito, S.: Sapiens: Foundation for human vision models. In: ECCV (2024) Kinetic Mining in Context 15
2024
-
[19]
In: CVPR (2020)
Kocabas, M., Athanasiou, N., Black, M.J.: VIBE: Video inference for human body pose and shape estimation. In: CVPR (2020)
2020
-
[20]
Leng, Z., Iyer, A., Plotz, T.: Scaling human activity recognition: A compar- ative evaluation of synthetic data generation and augmentation techniques. arXiv:2506.07612 (2025)
Pith/arXiv arXiv 2025
-
[21]
IEEE Trans
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L., Kot, A.C.: NTU RGB+D 120: A large-scale benchmark for 3d human activity understanding. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
2020
-
[22]
In: CVPR (2020)
Liu, Z., Zhang, H., Chen, Z., Wang, Z., Ouyang, W.: Disentangling and unifying graph convolutions for skeleton-based action recognition. In: CVPR (2020)
2020
-
[23]
Lupion, M., Cruciani, F., Cleland, I., Nugent, C., Ortigosa, P.: Data augmenta- tion for human activity recognition with generative adversarial networks. IEEE J. Biomed. Health Informatics (2024)
2024
-
[24]
In: ICCV (2019)
Mahmood, N., Ghorbani, N., Troje, N.F., Pons-Moll, G., Black, M.J.: AMASS: Archive of motion capture as surface shapes. In: ICCV (2019)
2019
-
[25]
In: ICCV (2021)
Petrovich, M., Black, M.J., Varol, G.: Action-conditioned 3d human motion syn- thesis with transformer VAE. In: ICCV (2021)
2021
-
[26]
In: ECCV (2022)
Petrovich, M., Black, M.J., Varol, G.: TEMOS: Generating diverse human motions from textual descriptions. In: ECCV (2022)
2022
-
[27]
In: ICPR (2020)
Plizzari, C., Cannici, M., Matteucci, M.: Spatial temporal transformer network for skeleton-based action recognition. In: ICPR (2020)
2020
-
[28]
In: ICML (2021)
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: ICML (2021)
2021
-
[29]
In: AISTATS (2007)
Salakhutdinov, R., Hinton, G.: Learning a nonlinear embedding by preserving class neighbourhood structure. In: AISTATS (2007)
2007
-
[30]
Sawdayee, H., Guo, C., Tevet, G., Zhou, B., Wang, J., Bermano, A.H.: Dance like a chicken: Low-rank stylization for human motion diffusion. arXiv:2503.19557 (2025)
Pith/arXiv arXiv 2025
-
[31]
In: ICLR (2024)
Shafir, Y., Tevet, G., Kapon, R., Bermano, A.H.: Human motion diffusion as a generative prior. In: ICLR (2024)
2024
-
[32]
In: CVPR (2016)
Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: NTU RGB+D: a large scale dataset for 3d human activity analysis. In: CVPR (2016)
2016
-
[33]
In: CVPR (2019)
Shi, L., Zhang, Y., Cheng, J., Lu, H.: Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: CVPR (2019)
2019
-
[34]
In: ICLR (2025)
Tevet, G., Raab, S., Cohan, S., Reda, D., Luo, Z., Peng, X.B., Bermano, A.H., van de Panne, M.: CLoSD: closing the loop between simulation and diffusion for multi-task character control. In: ICLR (2025)
2025
-
[35]
In: ICLR (2023)
Tevet, G., Raab, S., Gordon, B., Shafir, Y., Cohen-Or, D., Bermano, A.H.: Human motion diffusion model. In: ICLR (2023)
2023
-
[36]
In: CVPR (2025)
Wang, Y., Sun, Y., Patel, P., Daniilidis, K., Black, M.J., Kocabas, M.: PromptHMR: Promptable human mesh recovery. In: CVPR (2025)
2025
-
[37]
International Journal of Computer Vision (2025)
Wanyan, Y., Yang, X., Dong, W., Xu, C.: A comprehensive review of few-shot action recognition. International Journal of Computer Vision (2025)
2025
-
[38]
In: AAAI (2018)
Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: AAAI (2018)
2018
-
[39]
Zhang, Z., Wang, Y., Mao, W., Li, D., Zhao, R., Wu, B., Song, Z., Zhuang, B., Reid, I., Hartley, R.: Motion anything: Any to motion generation. arXiv:2503.06955 (2025)
Pith/arXiv arXiv 2025
-
[40]
Zhao, M., Liu, M., Ren, B., Dai, S., Sebe, N.: Modiff: Action-conditioned 3d motion generation with denoising diffusion probabilistic models. arXiv:2301.03949 (2023)
Pith/arXiv arXiv 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.