{"id":"89475e3f-a3c1-4ac9-839f-3b5fbf0da649","arxiv_id":"2608.09152","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"LightAIR combines a frozen action-word codebook, null-space appearance projection, and a Riemannian-style gradient rectification to set new state-of-the-art results on text-based person anomaly search and four TIPR benchmarks.","lead":"This paper introduces LightAIR, a retrieval model that separates what a person is doing from how they look, using text-derived action anchors and projections that remove action information from appearance features. It reports top scores on person anomaly search and several text-to-image person retrieval benchmarks, and it claims a gradient correction step prevents the model from learning appearance-based shortcuts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed strict forward decoupling rests on a rank-1 action subspace: Eq. 7 only removes the component parallel to z_act, Appendix E's own failure cases show residual action leakage into appearance features, and no ablation tests a multi-dimensional action-basis projector.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the action subspace is assumed to be captured by a single reconstructed vector z_act, so the orthogonal projection in Eq. 7 is claimed to remove all action information from appearance features. This concern is genuinely central because both headline contributions, ONSP and GR, are built on that rank-1 projector. I do not see a more fundamental issue in the empirical claims: the tables and ablations are extensive, the improvements over CMP are consistent across PAB, Multi-Weather, UCC, and TIPR benchmarks, and the method is plausibly reproducible from the described pipeline, though no code or error bars are provided. The strongest additional evidence for the concern comes from the paper itself: Appendix E reports failure cases where appearance information still overshadows action features, directly contradicting the 'mathematically guaranteed' language used in the abstract and conclusion. This is an internal inconsistency rather than a disagreement with consensus, so it should weigh on the verdict. The proposed concrete test, a multi-basis projection plus a linear action probe on z_app, would settle whether the rank-1 assumption is the actual bottleneck or whether the empirical gains come from other components such as the AIO semantic anchors or the entropy-based gating. Since the reader already issued CONDITIONAL on essentially this basis, my stress-test does not change the verdict; it sharpens the condition by making the untested assumption explicit and pointing to the paper's own failure analysis as corroborating evidence.","tokens_in":1025,"tokens_out":1047,"duration_ms":53629,"concrete_test":"Retrain LightAIR with a multi-basis action projector Pi_act = D_K^T (D_K D_K^T + eps I)^{-1} D_K, where D_K contains the top-K activated codebook rows before softmax weighting, and compare PAB 0.1M R@1/mAP and Multi-Weather R@1 against the published rank-1 results. In the same run, train a linear probe on z_app to predict PAB action labels: under true decoupling, probe accuracy should be near chance; if it is substantially above chance, action information leaks into z_app and the 'strict forward decoupling' claim fails regardless of retrieval metrics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central guarantee, 'strict forward decoupling' via ONSP, depends on the assumption that the action subspace is fully captured by the single vector z_act. Eq. 4 defines z_act = D^T alpha / ||D^T alpha||_2, a normalized weighted combination of up to K codebook vectors, and Eq. 6 builds a rank-1 projector Pi_act = z_act z_act^T / (||z_act||^2 + eps). Eq. 7 then computes z_app = (I - Pi_act) v, which only removes the component of v parallel to this one direction. If TPAS action semantics occupy a multidimensional subspace (multiple independent posture/motion directions, or combinations of codebook atoms), z_app retains all action information orthogonal to z_act, and Eq. 9's projection onto the nullspace of Pi_act cannot restore separation. This is not merely a theoretical gap: the manuscript's own Appendix E, Figure 11(c), acknowledges exactly this failure mode, stating that 'weak local action features are easily overshadowed by explicit large-area appearance attributes' and that 'the current forward decoupling mechanism still faces the risk of interference from dominant appearance information.' That is internal evidence that the strict-decoupling claim is not achieved by construction. The ablations in Figure 4 (D#5, D#6) test replacing projection with subtraction and removing Top-K, but never replace the rank-1 projector with a multi-dimensional action-basis projector, so the load-bearing assumption is untested. Because both ONSP and GR inherit this rank-1 assumption, the claimed mathematical guarantee is the weakest link in the paper's central argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper targets Text-based Person Anomaly Search (TPAS), where a query describes both macro-level appearance and micro-level abnormal actions. It proposes LightAIR, combining three modules: an Action Inversion Operator (AIO) that reconstructs an action feature z_act from a frozen text-derived semantic codebook, an Orthogonal Null-Space Projection (ONSP) that obtains an appearance feature z_app by projecting the visual feature onto the null space of z_act, and a Gradient Rectification (GR) module that projects the Euclidean gradient onto a tangent space with an entropy-based adaptive weight. The method is evaluated on the PAB TPAS benchmark, its Multi-Weather and UCC variants, and four TIPR benchmarks, reporting state-of-the-art results, e.g., PAB 0.1M R@1 of 84.73% and mAP of 91.93%, and UCC R@1 of 62.36% with only 0.1M training data. Ablations are provided for each module across the TPAS settings.","tokens_in":36199,"tokens_out":8250,"duration_ms":80171,"significance":"If the reported results are reproducible, LightAIR would be a strong new state of the art for TPAS and also improves conventional TIPR benchmarks. The design avoids external pose estimators and the code is released, which are practical strengths. The ablations are reasonably complete at the module level and the reported gains over CMP are consistent across PAB, Multi-Weather, and UCC. However, the central mathematical claim of 'strict forward decoupling' is only established for the single direction defined by z_act; the paper's own failure cases in Appendix E acknowledge residual action–appearance interference, and the current experiments do not test a multi-dimensional action-subspace projector. The core idea is promising, but the decoupling guarantee and its empirical support need to be tightened before the headline claims are acceptable.","major_comments":[{"comment":"The claim that ONSP 'mathematically eliminates' action information from z_app is not established by the construction. The projector Pi_act in Eq. (6) is rank-1, built from the single vector z_act of Eq. (4), so Eq. (7) removes only the component of v parallel to z_act. If TPAS action semantics occupy more than one direction in the embedding space, the residual action information in z_app remains. This is not merely hypothetical: Appendix E, Figure 11(c) states that 'weak local action features are easily overshadowed by explicit large-area appearance attributes' and that 'the current forward decoupling mechanism still faces the risk of interference from dominant appearance information.' Please either restrict the claim to removal of the component along the estimated action direction, or provide evidence that a single direction suffices. A concrete test would be to train a linear probe on z_app to measure remaining action information, and to ablate a multi-dimensional action-basis projector built from the top-K codebook directions.","section":"§3.3, Eqs. (6)–(7); Abstract; Contributions"},{"comment":"The action discriminative loss L_cls aligns z_act with t_y = Phi_T(x_T), which is the feature of the full text query, not an action-only feature. Because the query text also describes appearance (e.g., clothing, background), this supervision can pull z_act toward appearance directions, weakening the claim that AIO extracts 'pure action features' and potentially reintroducing appearance into the action branch before ONSP operates. Please clarify whether t_y is restricted to action words (for example, by projecting the text feature onto the action codebook D) or provide an ablation that replaces the full-text feature with an action-only feature in L_cls.","section":"§3.2, Eq. (5); Algorithm 1, lines 6 and 10"},{"comment":"The text states that 'using multiple action bases' lowers Multi-Weather R@1 by 3.02 points, but the ablation D#6 is labeled 'w/o num_K' (removing Top-K sparsification). Removing Top-K changes z_act to a dense combination of codebook atoms, yet the projection operator in Eq. (6) remains rank-1. Thus the experiment does not test a multi-dimensional action-basis projector and cannot support the conclusion that a single action direction is sufficient. Please correct the description and add an explicit experiment with a projector onto a multi-dimensional action subspace, or discuss why the rank-1 projector is adequate despite the Appendix E failure cases.","section":"§4.3.2, Figure 4 and accompanying text"}],"minor_comments":[{"comment":"The two terms in the gradient decomposition are called 'orthogonal components,' but no orthogonality is shown; the second term, which involves the derivative of Pi_act, is not generally contained in the range of Pi_act. Please clarify the claim or soften the wording. Also, the symbol ×1 is used without definition.","section":"§3.4, Eq. (8)"},{"comment":"The term 'Riemannian gradient' is used for an orthogonal projection onto a linear subspace T_vM, but the manifold M is never formally defined. If the tangent space is just the null space of Pi_act, the analysis is a linear projection; please either define the manifold or use a more neutral term such as 'projected gradient.'","section":"§3.4, Eq. (9)"},{"comment":"The captions of the ablation figures do not state whether the reported numbers correspond to the 0.1M or 1M training setting; the main text refers to both. Please state the training data scale in each caption or in the surrounding text.","section":"§4.3, Figures 3–6"},{"comment":"The rows for CLIP and X-VLM without a #Data value are not directly comparable to the 0.1M and 1M rows. Please specify the training data used for those baseline rows or remove them from the comparison table.","section":"Table 2"},{"comment":"Reference [150] is listed as 'MRA (arXiv'25)' in Table 2, but the reference entry is identical to [59], the CMP ICCV'25 paper. Please correct the duplicate or replace it with the intended MRA reference.","section":"References"},{"comment":"The statement 'We equivalently implement Riemannian gradient rectification by applying a gradient clipping operation to Pi_act' is vague. Please specify how the clipping is applied to the projection operator and how it relates to Eq. (11).","section":"§4.1.2, Implementation Details"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of ACM MM and the empirical results are potentially interesting. The main concern is that the headline 'strict forward decoupling' guarantee is not supported by the rank-1 projection and is contradicted by the paper's own failure-case analysis; this should be addressed by either adding the missing multi-dimensional-basis experiments and residual-leakage probes or by rewriting the claims to match what is actually proved. The duplicate reference and the D#6 labeling issue should also be corrected during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: LightAIR is worth taking seriously. The reported gains on PAB, UCC, and the TIPR benchmarks are consistent and the component design is sensible. But the paper oversells the math. The \"strict forward decoupling\" claim doesn't survive contact with the equations, and the stress-test note is right about that.\n\nWhat's actually new: using a frozen semantic codebook to reconstruct an action feature from the visual embedding, then projecting the appearance feature into the null-space of that single action direction, and rectifying the gradient toward the tangent space. Individually the pieces have precedents (null-space debiasing, Riemannian gradient methods), but the specific combination for TPAS, without external pose estimators, is new. The ablations are reasonably thorough—11 variants across three settings—and the failure cases in Appendix E are a point in the paper's favor: the authors don't hide where the method breaks.\n\nThe main soft spot is the rank-1 action subspace. Eq. 7 removes only the component parallel to z_act. If actions span multiple directions, as they must for complex postures or interactions, the appearance feature still carries action information. The paper's own Fig 11(c) admits exactly this—\"weak local action features are easily overshadowed by explicit large-area appearance attributes\"—which undercuts the word \"strict\" in the contributions. The ablations never replace the rank-1 projector with a multi-basis version, so the load-bearing assumption is untested. That's a fixable gap, not a fatal one.\n\nAlso, the GR implementation is underspecified. Saying \"equivalent to gradient clipping\" and then having an ablation called \"w/o Stop-Gradient\" left me unsure what the network actually does in the backward pass. I would have liked error bars or multiple seeds, since the PAB gains over CMP are in the 1-2 point range. The L_cls loss borrows the same text encoder that appears at retrieval time, which is a mild circularity but not a dealbreaker.\n\nWho is this for? Anyone working on text-based person search, and to a lesser extent on feature decoupling via projection. It deserves a serious referee. I'd recommend major revision: soften the \"mathematically guaranteed\" language, add a multi-dimensional action basis experiment, spell out the gradient rectification, and make the code verifiable.\n\nYes, send it to peer review.","headline":"Solid empirical gains and a sensible new combination for TPAS, but the 'strict forward decoupling' claim is overstated: the projection is rank-1, the paper's own appendix shows residual leakage, and the missing multi-basis ablation leaves the core assumption untested.","tokens_in":36705,"tokens_out":3499,"would_cite":true,"duration_ms":33959,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LightAIR claims strict action-appearance decoupling via text anchors, null-space projection, and Riemannian gradient rectification, setting state-of-the-art scores on PAB and four TIPR benchmarks.","keywords":["Text-based Person Anomaly Search","Text-to-Image Person Retrieval","action-appearance decoupling","null-space projection","Riemannian gradient","shortcut learning","semantic codebook","hard negative samples"],"falsifier":"Compare LightAIR against a variant whose action subspace contains several orthogonal action directions rather than one; if the single-basis model already decouples perfectly, the multi-basis variant should perform identically, while a gain would indicate the core assumption is too strong. A direct measurement is to compute, for same-clothing/different-action pairs, the cosine similarity between $z_{\\text{app}}$ and the text feature of the action description, which should be near zero for both actions under strict decoupling.","tokens_in":35622,"feed_emoji":"🔍","tokens_out":8592,"duration_ms":78019,"temperature":0.7,"pith_summary":"Text-based Person Anomaly Search (TPAS) asks a model to retrieve a pedestrian by matching both static appearance and a fine-grained abnormal action. The paper tries to establish that this can be done without fragile external pose estimators: a lightweight action inversion operator uses a frozen codebook of action words to reconstruct a pure action vector from the visual feature, a null-space projection forces the appearance vector onto the orthogonal complement of that action direction, and a Riemannian gradient rectification confines backpropagation to the tangent space of the action semantic manifold so hard negatives cannot exploit appearance shortcuts. If the claim holds, the resulting LightAIR network is the top scorer on the PAB anomaly benchmark (84.73% R@1 and 91.93% mAP at 0.1M training pairs, improving to 85.49% and 92.20% at 1M) and also improves four standard text-to-image person retrieval benchmarks, including a large gain on the ultra-fine-grained UFine3C setting.","feed_headline":"Anomaly search hits 84.7% R@1 by decoupling action from appearance","feed_subtitle":"Text anchors, orthogonal projection, and Riemannian gradients stop appearance from masking behavior in person search.","key_machinery":"The load-bearing object is the orthogonal projection operator $\\Pi_{\\text{act}} = z_{\\text{act}} z_{\\text{act}}^{\\top}/(\\lVert z_{\\text{act}} \\rVert^2+\\epsilon)$, built from the single reconstructed action vector. It does double duty: in the forward pass it defines the null space $(I-\\Pi_{\\text{act}})v$ that yields the appearance feature, and in the backward pass it defines the tangent space onto which the Euclidean gradient is projected to obtain the Riemannian gradient. The action vector itself is produced by the Action Inversion Operator, which estimates sparse Top-K coefficients over a frozen codebook of action-word embeddings and reconstructs $z_{\\text{act}} = D^{\\top} \\boldsymbol{\\alpha} / \\lVert D^{\\top} \\boldsymbol{\\alpha} \\rVert_2$. An entropy-guided weight $\\omega = \\exp(-H/\\tau)$ interpolates between the Euclidean and Riemannian gradients during training, so early optimization explores freely and later updates respect the decoupling constraint.","core_discovery":"The paper's central claim is that action and appearance can be strictly decoupled in a shared image-text space using text priors alone. An action inversion operator maps the global visual feature $v$ to sparse coefficients over a frozen action-word codebook and reconstructs $z_{\\text{act}}$, then defines the appearance feature as $z_{\\text{app}} = (I-\\Pi_{\\text{act}})v$, an orthogonal projection onto the null space of $z_{\\text{act}}$. The paper shows that the Euclidean gradient of the contrastive loss with respect to $v$ splits into a tangent component and a harmful normal component, and that replacing it with the Riemannian gradient $g_{\\text{Riem}} = (I-\\Pi_{\\text{act}})g_{\\text{Euc}}$, blended by an entropy weight $\\omega = \\exp(-H/\\tau)$, blocks the normal component's shortcut. Empirically, this yields 84.73% R@1 and 91.93% mAP on PAB at 0.1M data, and 85.49% R@1 and 92.20% mAP at 1M, surpassing the CMP baseline, while the same pipeline improves all four TIPR benchmarks.","pith_inferences":["Beyond the paper: the whole decoupling scheme hinges on the action subspace being one-dimensional. If a pedestrian performs two simultaneous or sequential actions, a single $z_{\\text{act}}$ cannot span the action content, and the null-space projection would either leak action into appearance or discard part of the action; a multi-basis version of the codebook projection is the natural stress test.","Beyond the paper: the entropy-weighted Riemannian gradient rectification is a general anti-shortcut mechanism for any contrastive retrieval task with a weak semantic signal and a dominant distractor modality; the paper demonstrates it only for person search, but the gradient decomposition argument is task-agnostic.","Beyond the paper: because the codebook is frozen and built from training vocabulary, deployment to genuinely novel action words may require codebook extension; the UCC experiment shifts scene distribution, not action vocabulary, so it does not test vocabulary generalization."],"forward_implications":["External pose estimators become unnecessary for TPAS: action information is supplied by text anchors, so retrieval should survive occlusion, unusual poses, and low-resolution surveillance frames.","Hard-negative 'same appearance, different action' pairs no longer force appearance shortcuts, since the normal gradient component is blocked and updates stay on the tangent space.","The same decoupling machinery transfers to conventional TIPR, producing best reported averages on CUHK-PEDES, ICFG-PEDES, RSTPReid, and UFineBench, with an 8.69-point average Recall gain on UFine3C.","Out-of-distribution robustness improves: with only 0.1M training pairs LightAIR beats CMP trained on the full 1M data on the UCC set (62.36 vs 55.23 R@1; 51.53 vs 44.35 mAP)."],"supporting_citations":[{"why":"Defines the TPAS task and supplies the CMP baseline and the benchmark setting that LightAIR must beat.","marker":"[59]"},{"why":"Supplies the PAB dataset with normal and anomaly action image-text pairs used for TPAS training and testing.","marker":"[141]"},{"why":"Provides the null-space projection technique that the ONSP module adapts to force appearance features into the orthogonal complement of action features.","marker":"[139]"},{"why":"Supplies the InfoNCE contrastive loss and image-text matching loss that form the backbone of the training objective.","marker":"[72]"},{"why":"Gives the Riemannian gradient definition that the GR module uses to project Euclidean gradients onto the tangent space.","marker":"[140]"},{"why":"Provides the X2-VLM pretrained encoder weights used to initialize the visual and text encoders.","marker":"[146]"}],"fun_headline_variants":["LightAIR: action inversion + Riemannian gradients fix anomaly search","Decouple action from appearance with orthogonal null-space projection","Pure action features from text anchors beat appearance bias","Riemannian gradient blocks shortcut learning in person search","Text priors stop appearance from drowning action features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The decoupling argument rests on the assumption that the action content of an image is fully captured by the single reconstructed action vector $z_{\\text{act}}$, so the orthogonal complement $(I-\\Pi_{\\text{act}})v$ contains no action information and all appearance information.","fun_headline_variants_meta":{"raw":{"variants":["LightAIR: action inversion + Riemannian gradients fix anomaly search","Decouple action from appearance with orthogonal null-space projection","Pure action features from text anchors beat appearance bias","Riemannian gradient blocks shortcut learning in person search","Text priors stop appearance from drowning action features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000509,"raw_usage":{"total_tokens":2547,"prompt_tokens":1084,"completion_tokens":1463,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":700,"completion_tokens_details":{"reasoning_tokens":1387}},"tokens_in":700,"tokens_out":1463,"duration_ms":10983,"temperature":1.0,"reasoning_tokens":1387,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:31:52.262330+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare LightAIR against a variant whose action subspace contains several orthogonal action directions rather than one; if the single-basis model already decouples perfectly, the multi-basis variant should perform identically, while a gain would indicate the core assumption is too strong. A direct measurement is to compute, for same-clothing/different-action pairs, the cosine similarity between $z_{\\text{app}}$ and the text feature of the action description, which should be near zero for both actions under strict decoupling.","supporting_citations":[{"cited_title":"2008.Optimization algo- rithms on matrix manifolds","cited_arxiv_id":null,"evidence_quote":"Gives the Riemannian gradient definition that the GR module uses to project Euclidean gradients onto the tangent space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the X2-VLM pretrained encoder weights used to initialize the visual and text encoders."}],"review_version":1}