pith. sign in

arxiv: 2605.22715 · v2 · pith:6C5D5ZRAnew · submitted 2026-05-21 · 💻 cs.CV · cs.AI· cs.CL· cs.HC

AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild

Baiyu Chen , Zechen Li , Wilson Wongso , Lihuan Li , Xiachong Lin , Hao Xue , Benjamin Tag , Flora Salim This is my paper

Pith reviewed 2026-05-22 06:21 UTC · model grok-4.3

classification 💻 cs.CV cs.AIcs.CLcs.HC
keywords setup-agnostic modelingIMU simulationhuman motionzero-shot learningwearable sensorsactivity recognitionmotion captioninggraph encoder
0
0 comments X

The pith

AnyMo learns setup-independent motion representations by simulating IMU signals from dense body placements and aligning them with language models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces AnyMo to handle the fact that inertial signals from wearables vary strongly with body location, mounting, orientation, hardware, and sampling. It generates many plausible synthetic signals through physics-based simulation across numerous points on the body surface. These signals train a graph encoder to reconstruct full motion from partial masked views, after which the encoder produces motion tokens that are aligned with a large language model. The resulting system supports zero-shot activity recognition, retrieval between IMU and text, and motion captioning. A reader would care because this could let everyday wearables deliver consistent motion analysis without retraining for each new device or placement.

Core claim

AnyMo is a geometry-aware framework that uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion tokens, and aligns these tokens with an LLM for motion-language understanding, yielding gains of 11.7%/11.6%/22.6% in Accuracy/F1/R@2 on HAR, 15.9% and 28.6% MRR lifts in zero-shot retrieval, and 18.8% BERT-F1 in zero-shot captioning across 14 unseen datasets.

What carries the argument

Physics-grounded IMU simulation over dense body-surface placements that generates diverse synthetic signals used to pre-train a graph encoder on paired placement views and masked observations before tokenization and LLM alignment.

If this is right

  • Zero-shot activity recognition achieves 11.7% higher accuracy and 11.6% higher F1 across 14 previously unseen datasets.
  • Zero-shot cross-modal retrieval reaches 15.9% higher MRR for IMU-to-text and 28.6% higher MRR for text-to-IMU.
  • Zero-shot motion captioning from wearable IMU data improves by 18.8% in BERT-F1 score.
  • The same pre-trained tokens support generalist motion-language tasks without dataset-specific fine-tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same dense-placement simulation strategy could be adapted to other wearable modalities such as pressure or temperature sensors.
  • Combining the motion tokens with camera or audio streams might produce more robust real-time activity tracking systems.
  • Long-term deployment on consumer devices could reveal whether the learned representations remain stable over weeks of continuous use.

Load-bearing premise

Physics-grounded IMU simulation over dense body-surface placements generates diverse and plausible synthetic signals that successfully transfer to real-world wearable data across varying body locations, mounting positions, and hardware.

What would settle it

Measuring whether performance gains vanish when the trained model is tested on IMU data from a new body location, mounting orientation, or sensor hardware whose signal statistics were never included in the simulation.

Figures

Figures reproduced from arXiv: 2605.22715 by Baiyu Chen, Benjamin Tag, Flora Salim, Hao Xue, Lihuan Li, Wilson Wongso, Xiachong Lin, Zechen Li.

Figure 1
Figure 1. Figure 1: Method families for wearable human motion understanding and radar plot comparing the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Physics-grounded geometry￾aware motion simulation. To define a consistent in-surface direction, we set an anatomi￾cal axis ui from ci toward the centroid of its nearest available child segment in the body kinematic tree, or along the op￾posite direction from its nearest available parent when no child segment is available. For each vertex v ∈ Vi , we com￾pute a surface normal ni,v from the template mesh fac… view at source ↗
Figure 3
Figure 3. Figure 3: Details of (1) Geometry-Aware Pre-Training, (2) Full-Body IMU Tokenization and (3) [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Details of Contrastive Instruction Tuning (left) and inference phases (right) of AnyMo. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative Results of Wearable IMU Motion Caption Generation. We use green to highlight [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: UMAP visualization of paired real and synthetic IMU embeddings for ten activity categories. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: More details of Masked IMU Tokenization and Motion Language Model Pre-Training. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Prompt templates used for motion-language pre-training, instruction tuning, contrastive [PITH_FULL_IMAGE:figures/full_fig_p028_8.png] view at source ↗
read the original abstract

As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion tokens, and aligns these tokens with an LLM for motion-language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream datasets, cross-modal retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7\%/11.6\%/22.6\% on HAR, increases zero-shot IMU-to-text and text-to-IMU retrieval MRR by 15.9\% and 28.6\%, respectively, and improves zero-shot captioning BERT-F1 by 18.8\%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: https://baiyuchen.com/project/AnyMo.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces AnyMo, a geometry-aware framework for setup-agnostic human motion modeling from wearable IMUs. It generates synthetic IMU signals via physics-grounded simulation over dense body-surface placements, pre-trains a graph encoder on paired synthetic views and masked observations, tokenizes multi-position IMU data into full-body motion tokens, and aligns these with an LLM for motion-language tasks. The work evaluates on zero-shot activity recognition across 14 unseen datasets, cross-modal retrieval, and IMU motion captioning, reporting average gains of 11.7%/11.6%/22.6% on HAR metrics, 15.9%/28.6% MRR improvements on retrieval, and 18.8% BERT-F1 on captioning.

Significance. If the simulation-to-real transfer holds and the gains are attributable to the proposed components rather than other factors, AnyMo could provide a practical path toward generalist models for wearable motion understanding that generalize across body locations, mounting positions, and hardware. The use of dense physics-based synthetic data for pre-training and the multi-task zero-shot evaluation on unseen datasets represent a strength in addressing setup dependence, which is a known barrier in the field.

major comments (2)
  1. [§3.1] §3.1 (Physics-Grounded IMU Simulation): The central claim that dense body-surface physics simulation produces diverse, plausible signals that successfully transfer to real-world wearable data across varying placements and hardware lacks direct quantitative fidelity validation. No metrics such as signal distribution distances, noise spectra comparisons, or domain discrepancy measures (e.g., MMD or Wasserstein distance) between simulated and real traces at matched positions are reported. This is load-bearing because the observed gains on the 14 unseen datasets could arise from the graph encoder, tokenization, or LLM alignment rather than the simulation itself.
  2. [§5] §5 (Evaluation and Baselines): The abstract and results claim substantial improvements (e.g., 11.7% Accuracy on HAR) but provide insufficient details on baseline implementations, statistical significance testing, dataset characteristics, or explicit controls for simulation-to-real gap. Without these, it is difficult to attribute gains specifically to the geometry-aware components versus other modeling choices.
minor comments (2)
  1. [§3.2] Notation for graph encoder inputs and tokenization could be clarified with an explicit diagram or equation set showing how partial observations map to full-body tokens.
  2. [Figure 4] Figure captions for qualitative results should include more detail on the specific body placements and hardware variations shown.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below, clarifying our approach and outlining planned revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [§3.1] §3.1 (Physics-Grounded IMU Simulation): The central claim that dense body-surface physics simulation produces diverse, plausible signals that successfully transfer to real-world wearable data across varying placements and hardware lacks direct quantitative fidelity validation. No metrics such as signal distribution distances, noise spectra comparisons, or domain discrepancy measures (e.g., MMD or Wasserstein distance) between simulated and real traces at matched positions are reported. This is load-bearing because the observed gains on the 14 unseen datasets could arise from the graph encoder, tokenization, or LLM alignment rather than the simulation itself.

    Authors: We acknowledge that the manuscript does not include direct quantitative fidelity metrics such as MMD or Wasserstein distance between simulated and real IMU traces at matched positions. Our validation of the simulation relied primarily on the strong zero-shot generalization across 14 unseen real-world datasets, which we interpret as indirect evidence of successful transfer given the physics-grounded modeling. To directly address this concern and better isolate the simulation's contribution, we will add quantitative domain discrepancy analyses (including MMD and Wasserstein distances) as well as noise spectra comparisons for representative positions in the revised version. revision: yes

  2. Referee: [§5] §5 (Evaluation and Baselines): The abstract and results claim substantial improvements (e.g., 11.7% Accuracy on HAR) but provide insufficient details on baseline implementations, statistical significance testing, dataset characteristics, or explicit controls for simulation-to-real gap. Without these, it is difficult to attribute gains specifically to the geometry-aware components versus other modeling choices.

    Authors: We agree that additional experimental details would improve clarity and help attribute gains more precisely. In the revision, we will expand the evaluation section and supplementary material to provide: explicit descriptions of baseline implementations and adaptations, results of statistical significance testing (e.g., p-values across multiple runs), summary characteristics for all 14 datasets, and further controls/ablation studies that isolate the contribution of the geometry-aware simulation and pre-training from other modeling choices such as tokenization or LLM alignment. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical framework with independent evaluation

full rationale

The paper presents AnyMo as a framework that generates synthetic IMU signals via physics-grounded simulation over dense placements, pre-trains a graph encoder on paired views and masked observations, tokenizes multi-position data into motion tokens, and aligns them with an LLM. All reported gains (e.g., +11.7% Accuracy on HAR across 14 unseen datasets, improved retrieval MRR and captioning BERT-F1) are stated as outcomes of empirical evaluation on downstream tasks rather than any mathematical derivation, fitted parameter renamed as prediction, or self-citation chain. No equations appear in the provided text, and the central claims rest on external dataset performance instead of reducing to inputs by construction. This is a standard self-contained empirical contribution with no load-bearing circular steps.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated. The physics simulation and transfer assumption are implicit but not formalized.

pith-pipeline@v0.9.0 · 5833 in / 1242 out tokens · 39989 ms · 2026-05-22T06:21:37.514900+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.