{"id":"63b84fe4-9b82-4c65-ae98-ca7d489af164","arxiv_id":"1908.03231","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Applying intrinsic and extrinsic sparse coding and dictionary learning to Kendall shape trajectories yields vector-space time-series that perform competitively on 3D action and 2D facial expression recognition.","lead":"This paper models faces and bodies as curved-space shape trajectories and uses sparse dictionary coding to turn them into ordinary vector time-series. The result is a competitive recognition pipeline for facial expressions, micro-expressions, and 3D actions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 11 is not derived from Eq. 10 because the encoding function F is undefined; if F is the weighted Karcher mean, Eq. 11 is a first-order relaxation, and the Eq. 13 dictionary update inherits the mismatch.","rationale":"The paper's advertised contribution is not just a recognition pipeline; it is a geometric coding method: sparse coding in Kendall shape spaces yields Euclidean time-series that are more discriminative because the dictionary operations are Riemannian. That claim requires a well-defined objective that relates the code w to a reconstruction on the manifold, and an implementable dictionary update. The paper gives Eq. (10) with an unspecified F, then jumps to Eq. (11). The only textual anchor for F is the Figure 2 caption, which names the weighted Karcher mean, and for that choice Eq. (11) is a tangent-space linearization, not an equality. This is exactly the kind of hidden assumption that can make a method work for reasons other than those stated: the learned dictionary may be minimizing a convenient proxy rather than the claimed geometric reconstruction. The missing D-update compounds the problem because one cannot even test the proxy independently. This is not an objection to the empirical comparisons; the ablation in Table 7 and the t-SNE evidence are real support, and the central claim may well survive. But the intrinsic branch of the paper is currently under-specified at its core, so the CONDITIONAL verdict is appropriate, and the proposed reconstruction-ratio check would settle whether the concern is substantive or benign.","tokens_in":23196,"tokens_out":9079,"duration_ms":98950,"concrete_test":"Pick one action dataset (e.g., UTKinect half-half) and, using the paper's clustering initialization and sparse-coding solver, learn a dictionary D with Eq. (13) exactly as the authors intend. For each training sample Z_i with learned code w_i, compute (a) R_i = ||Σ_j [w_i]_j log_{Z_i}(d_j)||^2 (Eq. 11) and (b) T_i = d_S(Z_i, μ_i)^2, where μ_i = argmin_μ Σ_j [w_i]_j d_S(μ,d_j)^2 is the weighted Karcher mean. Report the median T_i/R_i and the classification accuracy when codes are re-optimized by directly minimizing T_i (Riemannian gradient descent over w, dictionary fixed). If T_i/R_i is near 1 and accuracy is unchanged, the relaxation is benign; if T_i/R_i is large or accuracy drops, the intrinsic coding claim depends on an approximation that Eq. (10)/Fig. 2 do not state.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The intrinsic pipeline's central step is the passage from Eq. (10) to Eq. (11). Eq. (10) defines the sparse-coding loss as d_S(Z, F(D,w))^2, where F(D,w) is supposed to generate an approximated shape on S from the dictionary atoms and codes. However, F is never defined in the paper. Figure 2's caption says reconstruction is done by the weighted Karcher mean algorithm. If F is the weighted Karcher mean, then ||Σ_i [w]_i log_Z(d_i)||_Z^2 is not equal to d_S(Z, F(D,w))^2 except in a small-geodesic-distance, first-order sense: the norm of a weighted tangent sum equals the geodesic distance only to the exponential of that tangent vector, which generally differs from the Karcher mean. If instead F is defined implicitly as the exponential map of that tangent sum, it depends on the query Z, so Eq. (10) is not a global reconstruction objective. The dictionary-learning problem in Eq. (13) uses the same tangent-space functional, and the paper gives no update rule for D beyond 'optimizing D while fixing the sparse codes.' Thus the intrinsic method's geometric interpretation, and the claim that the resulting codes are discriminative because they encode Kendall geometry, rest on an unstated definition and an unvalidated approximation. The empirical results may still be useful, but the central claim is not yet fully supported as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to represent 2D facial-landmark and 3D-skeleton trajectories as trajectories in Kendall shape spaces and to apply sparse coding and dictionary learning (SCDL) to these manifold-valued data. Two formulations are developed: an extrinsic kernelized SCDL using the Procrustes Gaussian kernel (for 2D expressions) and an intrinsic SCDL using the logarithm map on the tangent space at each query shape (for 3D actions). The resulting sparse codes form Euclidean time series that are classified with either DTW+FTP+linear SVM or a Bi-LSTM. Experiments cover CK+, Oulu-CASIA, CASME II, Florence3D, UTKinect, MSR-Action3D, and NTU-RGB+D, and the paper reports competitive or state-of-the-art accuracies together with ablations on the Kendall representation, linear vs. nonlinear SCDL, and landmark detectors.","tokens_in":23510,"tokens_out":8113,"duration_ms":80312,"significance":"If the claims are fully supported, the contribution is practically valuable: it provides a view-invariant, geometry-aware representation of landmark trajectories that moves time-series classification into Euclidean space while retaining discriminative power. The strongest assets are the breadth of the experimental evaluation (seven datasets, two classifiers, ablations), the clear comparison with prior Riemannian approaches, and the attention to the subtle-deformation regime in micro-expression recognition. The Kendall geometry and the extrinsic kernelized SCDL derivation are standard and are built on correctly cited sources. The paper does not provide code or formal proofs, and the intrinsic method relies on an approximation that is not stated precisely; nevertheless, the empirical study is sufficiently extensive that the central idea deserves further scrutiny rather than rejection.","major_comments":[{"comment":"The encoding function F in Eq. (10) is never defined. If F is the weighted Karcher mean, as suggested by the caption of Figure 2, then the squared norm of the weighted tangent sum in Eq. (11) is not equal to d_S(Z, F(D,w))^2 except in a first-order or small-angle sense; the exponential of the weighted tangent vector is generally not the Karcher mean. If, instead, F is defined implicitly as the exponential map of the tangent sum, then the reconstruction in Eq. (10) depends on the query Z and is not a global objective. In either case, the passage from Eq. (10) to Eq. (11) is not a derivation, and the claim that the resulting codes preserve Kendall geometry is not supported as written. Please define F explicitly, state the approximation being made, and provide a validation (for example, a reconstruction-error experiment on held-out shapes) or a proof of equivalence under stated conditions.","section":"§4.2.1 (Eqs. 10-11)"},{"comment":"The intrinsic dictionary-learning problem is presented only as an objective; no update rule for the atoms D on the manifold is given beyond the phrase \"optimizing D while fixing the sparse codes.\" This is load-bearing because all reported intrinsic coding results depend on the learned dictionary, and an atom update in Kendall shape space requires either a geodesic gradient step or an explicit Procrustean re-estimation procedure. Please specify the exact update, state how the affine constraint on the codes is enforced during learning, and explain the convergence criterion used in the experiments.","section":"§4.2.2 (Eq. 13)"},{"comment":"The values of the kernel parameter σ, the sparsity parameter λ, the dictionary size N, and the Bi-LSTM hidden sizes are described only as \"chosen empirically\" and are not reported for any dataset. The selection procedure is also not described, so the reader cannot tell whether these hyperparameters were tuned on dedicated validation folds or on the test data. Since the comparisons in Tables 1, 3, 4, and 6 are the main evidence for the paper's claims, please report the chosen values per dataset and describe the validation protocol used for selection.","section":"§6 (Experimental Settings and Parameters)"}],"minor_comments":[{"comment":"The expansion of the reconstruction term appears to have a typographical error: the first line after the norm should read φ(z)^Tφ(z) - 2Σ_i [w]_i φ(d_i)^T φ(z) + ..., rather than having an extra equals sign before the negative term; please correct and re-check the algebra.","section":"§4.1.1 (Eq. 9)"},{"comment":"In the dictionary-update derivation, the expression contains \"IM−VA\" where \"A\" is likely meant to be \"W\"; please correct this and clarify why V = W^dagger follows, including the required invertibility assumptions.","section":"§4.1.2"},{"comment":"The notation \"diO*\" should be written as d_i O^* to make explicit that the optimal rotation is computed between the query Z and each atom d_i, and the angle in the formula is atom-dependent.","section":"§4.2.1 (Eq. 12)"},{"comment":"The exponential map formula is singular at θ = 0; please state the limiting behavior or define exp_Z(0) = Z explicitly.","section":"§3.1 (Eq. 3)"},{"comment":"The MSR-Action 3D row for Intrinsic SCDL (Bi-LSTM) reports 86.18 for both the Half-Half and the 3-Subsets protocols; please verify whether this duplication is correct or a rounding/typing artifact.","section":"Table 4"},{"comment":"The comparison in Table 3 says the result is state-of-the-art \"with the exception of [30]\", but the table lists two methods from [30] (STCLQP and DiSTLBP-IIP); please make the comparison statement precise.","section":"§6.1.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an extended version of the authors' CVPR 2018 paper on 3D action recognition. The new material consists mainly of the 2D facial-expression experiments and the extrinsic/intrinsic comparison, so the incremental contribution is moderate. If the authors can supply the missing dictionary-update algorithm and hyperparameter values, I would view the revised version as publishable; without those, the intrinsic experiments are not reproducible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this is a workmanlike extension of the authors' CVPR 2018 intrinsic SCDL work. What is actually new is the extrinsic kernel pipeline for 2D facial expression trajectories, including micro-expression recognition, plus a head-to-head intrinsic/extrinsic comparison on seven datasets. The experiments are extensive and the ablations are fair.\n\nThe main soft spot is the intrinsic formulation: Eq. 10 is written with an encoding function F that is never defined, and Eq. 11 quietly replaces the reconstruction distance with the norm of a weighted tangent sum. That is a first-order relaxation of the weighted Karcher mean, and the dictionary update in Eq. 13 inherits the mismatch. The stress-test is correct. It is not fatal to the empirical results, but it undercuts the claim that the codes are 'intrinsic' in a strict geometric sense. The paper also leaves the intrinsic atom update unspecified, gives no code or exact parameter values, and the 10-fold protocol on CK+ and Oulu-CASIA does not say whether folds are subject-disjoint, which matters for these benchmarks. The kernel positive definiteness for 3D is only checked empirically, which is acknowledged. All of these are fixable in revision.\n\nOn the credit side, applying kernel SCDL to dynamic 2D shape trajectories is a genuine extension of [26]'s static shape coding, and the micro-expression result using only landmarks is useful. The comparisons with state of the art are honestly reported, including where they lose. I would send this to a serious referee, expecting major revision.","headline":"Solid extension of the authors' earlier intrinsic SCDL work; the extrinsic 2D facial-expression pipeline is the new part, and the intrinsic formulation has a real but fixable gap.","tokens_in":24033,"tokens_out":2773,"would_cite":true,"duration_ms":28346,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse coding of shape trajectories in Kendall spaces yields vector-space time-series that are more discriminative than the raw data, the paper argues.","keywords":["Kendall shape space","shape trajectories","sparse coding","dictionary learning","facial expression recognition","micro-expression recognition","action recognition","Riemannian manifolds"],"falsifier":"Take a query shape and a learned dictionary, compute its intrinsic sparse code with Eq. (11), then reconstruct the shape as the weighted Karcher mean of the selected atoms and measure the geodesic distance between the reconstruction and the query; if that error is systematically large when the query is far from the chosen atoms, the tangent-space objective does not represent the true reconstruction loss the framework relies on.","tokens_in":23013,"feed_emoji":"📐","tokens_out":8413,"duration_ms":84735,"temperature":0.7,"pith_summary":"This paper tries to establish that sparse coding and dictionary learning applied to sequences of landmark configurations—faces in 2D, skeletons in 3D—produces time-series features that are easier to classify than the original shape trajectories. The key move is to treat each frame as a point in Kendall's shape space, the manifold that remains after removing translation, rotation, and scale, and to learn a Riemannian dictionary on that manifold. Coding a trajectory against this dictionary yields, at every frame, a sparse vector of coefficients; the whole trajectory becomes a Euclidean sparse time-series. The paper argues these coded series are more discriminative than raw data, and reports competitive accuracy on seven datasets covering macro- and micro-facial expressions and skeletal actions. If true, the practical payoff is that dynamic shape classification can be done in ordinary vector spaces with standard temporal tools, while keeping the view-invariance of the shape geometry.","feed_headline":"Sparse coding makes shape trajectories more discriminative","feed_subtitle":"Facial and skeletal sequences coded in Kendall shape spaces become vector time-series that rival state-of-the-art recognizers.","key_machinery":"The central object is the Kendall shape space, the manifold formed by landmark configurations modulo translation, rotation, and global scaling, together with the geodesic distance and logarithm/exponential maps induced by the Procrustes metric. On top of this manifold the paper builds Riemannian sparse coding and dictionary learning (SCDL): a dictionary of shapes is learned from training data, and each query shape is represented by a sparse weight vector. In the intrinsic version the weights act on tangent vectors obtained by logarithm-mapping every atom to the query's tangent space, so reconstruction error is measured at the query itself; in the extrinsic version the Procrustes Gaussian kernel embeds the 2D shape space into a Hilbert space where ordinary linear sparse coding applies. This machinery carries the argument because it converts nonlinear manifold-valued trajectories into Euclidean sparse time-series that can be fed to standard temporal classifiers.","core_discovery":"The paper's central claim is that Riemannian sparse coding and dictionary learning in Kendall shape spaces turns a shape trajectory into a sequence of sparse coefficient vectors that live in a vector space and are more discriminative than the original landmarks. For 2D facial sequences the coding is extrinsic: shapes are embedded in a reproducing kernel Hilbert space through the Procrustes Gaussian kernel, and sparse codes are computed there. For 3D skeletal sequences the coding is intrinsic: each shape is coded on its own tangent space, where the logarithm map sends dictionary atoms to tangent vectors at the query and the reconstruction cost is the norm of a weighted sum of those vectors. Class-specific dictionaries are learned, with atoms initialized by clustering and principal geodesic analysis, and the resulting sparse time-series are classified either by a DTW-FTP-SVM pipeline or by a bidirectional LSTM. The paper reports that extrinsic coding works better for subtle facial expressions, intrinsic coding works better for 3D actions, and both beat the raw-nonlinear baselines in ablation experiments on the action datasets.","pith_inferences":["Beyond the paper's own experiments, the extrinsic-versus-intrinsic split suggests a selection rule: use the kernel embedding for subtle, small deformations and per-query tangent coding for large, noisy skeletal datasets, a rule that could be tested by swapping the two coders across the same dataset.","A testable extension the paper only gestures at is to train a generative model on the Euclidean sparse codes and decode new trajectories through the weighted Karcher mean to synthesize novel actions or expressions.","Because the intrinsic objective is a first-order tangent approximation, one could audit the claim that codes are more discriminative by measuring exact geodesic reconstruction error between queries far from all dictionary atoms."],"forward_implications":["Dynamic shape classification can be moved off the manifold: once trajectories are coded, temporal modeling and classification run in Euclidean space with methods such as DTW, Fourier pyramids, SVMs, and Bi-LSTMs.","Because coding is done against class-specific Riemannian dictionaries, the concatenated sparse coefficients carry label information, and reported recognition rates on CK+, Oulu-CASIA, CASME II, Florence3D, UTKinect, MSR-Action 3D, and NTU-RGB+D are competitive with state-of-the-art.","Ablations show the full Kendall SCDL pipeline beats linear SCDL on the same features and beats raw normalized landmarks, so the nonlinear geometry of the shape space is doing real work.","On the large-scale NTU-RGB+D dataset the intrinsic SCDL representation improves over other Riemannian trajectory representations by large margins, suggesting the approach scales beyond small benchmarks."],"supporting_citations":[{"why":"Defines Kendall's shape space, the manifold geometry and Procrustes distance the coding operates on.","marker":"[39]"},{"why":"Supplies the extrinsic RKHS sparse coding and dictionary update via SVD and the Representer theorem, adapted here to Kendall space.","marker":"[25]"},{"why":"Provides the intrinsic SCDL formulation on tangent bundles using logarithm maps that the paper extends to Kendall shape trajectories.","marker":"[29]"},{"why":"Proves positive definiteness of the Procrustes Gaussian kernel used for the 2D extrinsic embedding.","marker":"[34]"},{"why":"Introduced skeletal shape trajectories in Kendall space and their elastic alignment, giving the trajectory representation this paper codes.","marker":"[4]"},{"why":"The authors' preliminary work on intrinsic SCDL for 3D action recognition, which this paper generalizes to 2D expressions.","marker":"[5]"},{"why":"Generalized sparse coding to nonlinear manifolds via positive definite kernels, including the 2D Kendall space, grounding the extrinsic comparison.","marker":"[26]"},{"why":"Supplies the Bayesian clustering of shapes used to initialize dictionary atoms and infer dictionary size.","marker":"[81]"}],"fun_headline_variants":["Sparse coding sharpens shape trajectories for expression and action recognition","Riemannian sparse coding boosts facial and skeletal recognition","Sparse dictionary learning lifts shape trajectories into vector spaces","Sparse coding of shape paths improves facial and action recognition","Sparse coding on shape manifolds sharpens expression and action recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The intrinsic coding rule assumes that combining small local displacements from a shape still measures the true curved distance to the reconstructed shape, an approximation that holds exactly only when dictionary shapes are close to the query.","fun_headline_variants_meta":{"raw":{"variants":["Sparse coding sharpens shape trajectories for expression and action recognition","Riemannian sparse coding boosts facial and skeletal recognition","Sparse dictionary learning lifts shape trajectories into vector spaces","Sparse coding of shape paths improves facial and action recognition","Sparse coding on shape manifolds sharpens expression and action recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000706,"raw_usage":{"total_tokens":3182,"prompt_tokens":942,"completion_tokens":2240,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":2158}},"tokens_in":558,"tokens_out":2240,"duration_ms":16996,"temperature":1.0,"reasoning_tokens":2158,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:20:12.071072+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a query shape and a learned dictionary, compute its intrinsic sparse code with Eq. (11), then reconstruct the shape as the weighted Karcher mean of the selected atoms and measure the geodesic distance between the reconstruction and the query; if that error is systematically large when the query is far from the chosen atoms, the tangent-space objective does not represent the true reconstruction loss the framework relies on.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Kendall's shape space, the manifold geometry and Procrustes distance the coding operates on."},{"cited_title":"Harandi, R","cited_arxiv_id":null,"evidence_quote":"Supplies the extrinsic RKHS sparse coding and dictionary update via SVD and the Representer theorem, adapted here to Kendall space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the intrinsic SCDL formulation on tangent bundles using logarithm maps that the paper extends to Kendall shape trajectories."},{"cited_title":"Jayasumana, M","cited_arxiv_id":null,"evidence_quote":"Proves positive definiteness of the Procrustes Gaussian kernel used for the 2D extrinsic embedding."},{"cited_title":"Harandi and M","cited_arxiv_id":null,"evidence_quote":"Generalized sparse coding to nonlinear manifolds via positive definite kernels, including the 2D Kendall space, grounding the extrinsic comparison."},{"cited_title":"Zhang, D","cited_arxiv_id":null,"evidence_quote":"Supplies the Bayesian clustering of shapes used to initialize dictionary atoms and infer dictionary size."}],"review_version":1}