{"id":"387854df-6fe5-42eb-b78d-035e12989902","arxiv_id":"2501.13829","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MV-GMN, a state-space model with graph convolution, reports state-of-the-art accuracies on NTU RGB+D and PKU-MMD action recognition benchmarks.","lead":"A new model called MV-GMN combines state-space (Mamba) layers with graph networks to recognize human actions from multiple camera views. It reports top accuracies on standard benchmarks while aiming to be cheaper than Transformer models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central 'linear inference complexity' claim is unsupported: Section 3.3 constructs dense rule edges with O(VT^2 + TV^2) entries and an exact KNN step with O((VT)^2) cost, so the full pipeline is not linear in view-time tokens.","rationale":"I read the paper as making an efficiency claim about the full MV-GMN pipeline, not just its Mamba layers; the abstract, Figure 1, and conclusion all frame linear inference complexity as the key advantage over Transformers. The weakest point is Section 3.3: the graph edge construction and propagation are part of every inference pass, and the text specifies dense rule edges and exact KNN comparisons without any complexity analysis. The reader's weakest_assumption correctly identified KNN construction; I partially agree and extend the same concern to the rule-based edges in Equation (17), which alone make the graph convolution quadratic in T for fixed V. This concern does not by itself disprove the reported accuracies, but it does mean the central complexity motivation is overstated as written. The results text also contains a table/text accuracy mismatch that should be fixed during revision. Since the reader already returned CONDITIONAL and the appropriate remedy is to supply a complete complexity analysis or qualify the claim, I recommend UNCHANGED.","tokens_in":13645,"tokens_out":7064,"duration_ms":65588,"concrete_test":"Run the full MV-GMN(View-Time) pipeline on NTU RGB+D 120 with V=3 and T=8, 16, 32, 64, keeping all other settings fixed, and record end-to-end inference time as well as the time spent in (a) building Erule and Eknn and (b) the graph convolution of Equation (21). Plot per-sample inference time against N=V*T on a log-log scale. If the slope is near 2 rather than 1, or if |Erule| follows V*T*(T-1)/2, the linear-complexity claim is false for the full model and must be revised to cover only the SSM scan; specifying an approximate KNN method and re-deriving its complexity would also settle the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim has two parts: SOTA-level accuracy and linear inference complexity. The accuracy part cannot be checked without code, but the complexity part can be checked against the method description and fails. In Section 3.3, Equation (17) connects each vertex to every other vertex at a different time within the same view, producing V*T*(T-1)/2 unordered rule edges; Equation (18) adds T*V*(V-1)/2 view edges. Graph propagation in Equation (21) is linear in the number of edges, so the rule-based GCN alone is O(V*T^2 + T*V^2), not O(V*T). Separately, Equation (19) defines KNN edges by comparing each vertex with all other vertices; unless an approximate neighbor-search method is supplied, this is O((V*T)^2) distance computations, and the manuscript does not state any approximation. Thus the abstract, Figure 1, and conclusion claim full-model linear complexity without accounting for graph construction or propagation. Table 5 reports a single 29.88ms inference point at roughly 48 view-time tokens, which cannot distinguish linear from quadratic scaling. A secondary inconsistency (the NTU-120 cross-view number is 96.7% in the abstract and Table 2 but 96.4% in the results text, with a 0.7% gain claim that does not match the table) should be corrected but is not the load-bearing issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MV-GMN, a multi-view action recognition model that combines a bidirectional state-space (Mamba) block with a graph convolutional network. Features from RGB and skeleton streams are fused via cross-attention, and view-time relationships are modeled through rule-based and KNN-based graph edges. The model is evaluated on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD, reporting state-of-the-art or competitive Top-1 accuracies, and the paper claims the full model has linear inference complexity.","tokens_in":14010,"tokens_out":6169,"duration_ms":50262,"significance":"If the reported accuracies are reproducible, MV-GMN is a competitive multi-view action recognition method and a useful early example of applying state-space models to multi-view multi-temporal fusion. The paper provides a substantial set of ablations (view/time priority, fusion strategies, KNN and block counts) and compares against several recent baselines. Its main weakness is that the central efficiency claim is not backed by a correct complexity analysis of the graph module, and several reported numbers are internally inconsistent. The contribution is therefore promising but needs a major revision before the claims can be accepted.","major_comments":[{"comment":"The claimed linear inference complexity is not supported. The rule-based edges in Eq. (17) generate O(VT^2) edges and Eq. (18) generates O(TV^2) edges; graph propagation in Eq. (21) is linear in the number of edges, so the rule-based GCN alone is O(VT^2 + TV^2). In addition, the KNN construction in Eq. (19) requires pairwise distance computation across all V×T vertices unless an approximate neighbor-search method is specified; with exact KNN this is O((VT)^2) and no approximation is described. Table 5 reports a single timing point at roughly 48 view-time tokens, which cannot distinguish linear from quadratic scaling. Please provide an asymptotic analysis that includes graph construction and propagation, or revise the complexity claim to apply only to the SSM sequence component.","section":"Section 3.3, Eqs. (17)–(19) and (21); Abstract; Figure 1; Conclusion"},{"comment":"The NTU RGB+D 120 cross-view accuracy is inconsistent across the paper: the abstract and Table 2 give 96.7%, while the results text states 96.4% and claims an improvement of 0.7% over Cheng et al. (2024). From Table 2, Cheng et al.'s cross-view accuracy is 95.6%, making the gap 1.1 percentage points if MV-GMN indeed scores 96.7%. These numbers must be reconciled, and all increment claims should be recomputed from the final tables.","section":"Section 4.1, Table 2; Abstract"},{"comment":"The ablation discussion for RGB-skeleton fusion contradicts Table 4. The text says 'the averaging method outperforms the linear approach by 2.2% and 0.9% in cross-subject scenarios', but Table 4 shows Linear Fusion with 95.1% (CS) and 95.8% (CV), both higher than Mean Fusion's 94.9% and 95.6%. Please correct either the text or the table so that the reported comparison matches the data.","section":"Section 4.2, Table 4 and prose"}],"minor_comments":[{"comment":"There are several typos and formatting errors: 'propsed' (Conclusion), 'acrion' (Section 3.3), 'Abaltion Studies' (Section 4.2 heading), 'M ambahaving' (Section 3.2), 'MV-GMA blocks' (Implementation Details), and 'T-KNN-GCT' (Section 4.2, should be T-KNN-GCN).","section":"Throughout"},{"comment":"The citation 'DeiTTouvron et al. (2021)' is malformed; the DeiT model should be attributed to Touvron et al. (2021). Similarly, 'Sparse-MSSTNetCheng et al. (2024)' is missing a space before the citation. Also, Implementation Details says 'Deit-B' but Table 4 lists 'Deit-S'; please clarify which backbone is used.","section":"References and model names"},{"comment":"The timing experiments would benefit from a description of the hardware and inference settings (e.g., batch size, whether the timing includes feature extraction or only the fusion module, and the number of view-time tokens V×T used). The current single number per model is insufficient to support scaling claims.","section":"Section 4.2, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a benchmark-oriented application paper. The novelty beyond existing Mamba and GCN methods is incremental, but the empirical results, if corrected and reproducible, could still be of interest. The main blocking issue is the unsupported linear-complexity claim, which is central to the paper's motivation. No code release is mentioned, which makes independent verification of the reported accuracies difficult; I would encourage the authors to provide code and detailed training configurations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first paper I've seen that applies Mamba-style SSMs to multi-view action recognition, and the design is a sensible mix of a bidirectional Mamba block with a GCN that fuses view and time tokens. The accuracy numbers, if they hold, are competitive, and the comparison tables are thorough. But the paper's central selling point—linear inference complexity—doesn't survive contact with its own method description.\n\nSection 3.3 constructs dense rule-based edges: Eq. (17) connects every pair of time steps within each view, which alone gives V×T(T−1)/2 edges, and Eq. (18) adds T×V(V−1)/2 view edges. Propagation is linear in edge count, but edge count is quadratic in V and T. The KNN step in Eq. (19) is worse: exact k-NN on all vertices is O((VT)^2) distance computations. The paper never mentions an approximate neighbor search. So the abstract's claim of \"only linear inference complexity\" is false for the described model. Table 5's single latency number at ~48 tokens can't distinguish linear from quadratic behavior.\n\nThere are also smaller consistency issues: the NTU-120 cross-view result is 96.4% in the text but 96.7% in the abstract and Table 2, and the prose claiming averaging beats linear fusion contradicts Table 4, where linear fusion is higher. The 0.7% gain over Cheng's model also doesn't match the table (95.6 to 96.7 is +1.1). These are fixable but need attention.\n\nWhat's genuinely new here is the four-scan bidirectional state space block and the graph construction for view-time tokens. That's a reasonable contribution, and the experiments cover three standard datasets with appropriate baselines. No code is provided, which is a reproducibility concern, but that's common for this type of paper.\n\nWho is this for? Anyone working on multi-view action recognition or efficient video backbones. The idea of using Mamba for view fusion is worth exploring further. But the current write-up has a load-bearing flaw, so I would not rely on the complexity claims or the exact numbers without verification.\n\nRecommendation: send to peer review—the novelty is real, and a referee can push for a corrected complexity analysis and fixed inconsistencies. If the authors can't reduce the graph construction cost or properly account for it, the paper's main argument collapses.","headline":"First Mamba-based multi-view action recognition architecture, with competitive accuracy numbers, but the headline linear-complexity claim is contradicted by the paper's own graph construction (rule-based edges are quadratic in V and T, and KNN is quadratic in VT).","tokens_in":14487,"tokens_out":3432,"would_cite":false,"duration_ms":28862,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper reports that its Multi-View Graph Mamba network (MV-GMN) outperforms existing state-of-the-art multi-view action-recognition models on NTU RGB+D 60/120 and PKU-MMD while keeping the linear inference complexity of state-space…","keywords":["multi-view action recognition","state space model","Mamba","graph convolutional network","RGB-skeleton fusion","bidirectional scanning","view-time graph","NTU RGB+D"],"falsifier":"Instrument the full model and double the number of viewpoints while holding the temporal length fixed: if the nearest-neighbor edge construction is exact, wall-clock inference time should grow roughly with the square of the token count, contradicting the linear-complexity claim; a profile of the graph-building step alone would show the same quadratic term.","tokens_in":13487,"feed_emoji":"🎥","tokens_out":7706,"duration_ms":63631,"temperature":0.7,"pith_summary":"The paper aims to establish that a state-space model can replace Transformer-based fusion for multi-view action recognition, combining RGB and skeleton data across viewpoints and time with lower computational cost. It introduces MV-GMN, a Multi-View Graph Mamba network whose blocks scan the view-time token grid in four directions and then propagate information over a graph built from explicit and learned edges. The authors report state-of-the-art top-1 accuracy on three benchmarks, including 97.3% and 96.7% on NTU RGB+D 120 in cross-subject and cross-view splits, and they attribute the efficiency gain to the linear inference complexity of the state-space core. A sympathetic reader would take the central contribution to be a practical demonstration that state-space-based fusion is competitive with attention-based fusion in this setting.","feed_headline":"Beats transformer fusion on multi-view action benchmarks","feed_subtitle":"MV-GMN merges RGB, skeleton, views, and time at linear inference cost.","key_machinery":"The load-bearing object is the Multi-View Graph Mamba block, made of a Bidirectional State Space Block and a GCN block. The state-space block flattens fused RGB-skeleton features in four orders and passes each through a 1D convolution followed by a selective state-space model (SSM/Mamba), giving the linear-cost recurrence; the GCN block creates a graph whose vertices are the $V \\times T$ view-time tokens, with rule-based edges connecting same-view tokens across time and same-time tokens across views, plus KNN edges connecting the most similar tokens, then propagates features with the symmetric normalized graph-convolution rule. The four scan orders are what let the SSM capture both view and time dependencies, and the graph edges are what let information move between any viewpoints at any moment.","core_discovery":"The central claim is that the Multi-View Graph Mamba network achieves state-of-the-art multi-modal, multi-view, multi-temporal action recognition while keeping the inference complexity of a state-space model. The architecture fuses skeleton-guided RGB features via cross-attention, forms a grid of $V \\times T$ view-time tokens, runs four bidirectional state-space scans (view-forward, view-backward, time-forward, time-backward), and then applies graph convolution over a combined edge set of time-consistent edges, view-consistent edges, and KNN-similarity edges. On NTU RGB+D 60 the View-Time variant reports 98.2% cross-subject and 99.7% cross-view; on NTU RGB+D 120 it reports 97.3% and 96.7%; on PKU-MMD it reports 98.0% and 99.1%.","pith_inferences":["The 'linear inference complexity' the paper advertises is a property of the state-space layers; the KNN graph construction over all $V \\times T$ vertices costs $O((VT)^2)$ distance computations with exact search, so the complexity claim should be read as covering the SSM core unless approximate neighbor search is used.","The same block design -- bidirectional selective scans over a view-time token grid plus graph edges -- is a template for other multi-view or multi-sensor sequence tasks, such as multi-camera video understanding or multi-view video prediction; testing it there would show whether the benefit transfers beyond action recognition.","Because view-prioritized ordering consistently beat time-prioritized ordering in the reported ablations, a natural next experiment is to vary the number of cameras or shuffle camera order at test time; the paper suggests that cross-view structure is the dependency the model learns first."],"forward_implications":["If MV-GMN's reported numbers hold, a state-space backbone can replace a Transformer for multi-view action recognition without sacrificing accuracy across the three tested benchmarks.","Because the SSM core processes tokens in a single pass, its per-token cost does not grow with the number of view-time tokens the way attention's pairwise comparisons do, which is the basis for the paper's linear-inference-complexity claim.","The ablation results indicate that both ingredients matter: KNN edges outperform rule-based edges alone, and adding the state-space block on top of KNN-GCN improves accuracy with fewer parameters than the Transformer-based version.","View-prioritized scanning consistently beats time-prioritized scanning, suggesting that ordering tokens by viewpoint before applying the bidirectional scan captures more of the cross-view dependencies."],"supporting_citations":[{"why":"Defines the selective state-space (Mamba) layer that the Bidirectional State Space Block is built on; supplies the linear-inference core.","marker":"Gu and Dao (2023)"},{"why":"Gives the graph-convolution propagation rule used to integrate view-time features.","marker":"Kipf and Welling (2016)"},{"why":"Provides the modified Sparse-MSSTNet skeleton encoder and is the strongest baseline MV-GMN is compared against.","marker":"Cheng et al. (2024)"},{"why":"Provides the frozen DeiT-B RGB encoder used for frame feature extraction.","marker":"Touvron et al. (2021)"},{"why":"Defines the NTU RGB+D 60 dataset used for evaluation.","marker":"Shahroudy et al. (2016)"},{"why":"Defines the NTU RGB+D 120 dataset whose cross-subject and cross-view splits the headline numbers come from.","marker":"Liu et al. (2019)"},{"why":"Defines the PKU-MMD dataset used as the third benchmark.","marker":"Liu et al. (2017)"}],"fun_headline_variants":["State-space model beats transformer baselines at linear inference cost","MV-GMN: efficient multi-view action recognition with state-space scans","Linear-complexity Mamba network tops multi-view action benchmarks","Four bidirectional scans enable fast multi-view action recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The strongest efficiency claim depends on the assumption that total inference time grows only linearly with the number of view-time tokens, yet the nearest-neighbor graph-building step compares every token with every other token, and the paper does not describe an approximate method to avoid that quadratic cost.","fun_headline_variants_meta":{"raw":{"variants":["State-space model beats transformer baselines at linear inference cost","MV-GMN: efficient multi-view action recognition with state-space scans","Linear-complexity Mamba network tops multi-view action benchmarks","Four bidirectional scans enable fast multi-view action recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000847,"raw_usage":{"total_tokens":3693,"prompt_tokens":961,"completion_tokens":2732,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":2664}},"tokens_in":577,"tokens_out":2732,"duration_ms":18403,"temperature":1.0,"reasoning_tokens":2664,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:33:19.188923+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Instrument the full model and double the number of viewpoints while holding the temporal length fixed: if the nearest-neighbor edge construction is exact, wall-clock inference time should grow roughly with the square of the token count, contradicting the linear-complexity claim; a profile of the graph-building step alone would show the same quadratic term.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the modified Sparse-MSSTNet skeleton encoder and is the strongest baseline MV-GMN is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the frozen DeiT-B RGB encoder used for frame feature extraction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the NTU RGB+D 60 dataset used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the NTU RGB+D 120 dataset whose cross-subject and cross-view splits the headline numbers come from."}],"review_version":1}