{"id":"06d4a069-8bae-45d5-8967-2533a8156f6b","arxiv_id":"2506.13322","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A multimodal few-shot action recognition framework that selects the more confident modality per sample, distills between modalities, and fuses with confidence-based weights, reporting strong results despite gaps in the derivation.","lead":"This paper proposes a few-shot action recognition method that picks the more reliable modality (RGB or optical flow) for each video using confidence scores, then distills knowledge between the two modalities and fuses them with adaptive weights. It reports large accuracy gains on four benchmarks, but the paper's math contains a defective loss term and an undefined 'free energy' quantity that the method depends on.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The ASI module's free-energy term F^m_i in Eq. 3 is never defined, so the central active-inference mechanism and the distillation grouping it controls are not computable as written.","rationale":"The reader's weakest assumption identifies the undefined free energy F in Eq. 3 as the point where the central mechanism loses support. I agree. Inspecting Eqs. 1-3, 5-8, and 10, Eq. 3 is the only place F appears, and no operational definition is supplied anywhere in the manuscript. Figure 2's 'free energy' curves are also unlabeled. This is not a stylistic omission: F determines which samples enter G^r and G^f, and Section 3.4 says distillation is applied to the selected samples in those groups, so the distillation loss and teacher/student direction depend on F. Without F the method cannot be reproduced from the text. The subsequent equations only define c^m_i, not F, and the later discussion indeed reduces to using c^m_i as a confidence weight in Eq. 6 and Eq. 8. The mislabeled Eq. 10, which as written computes sum p log p (negative entropy) rather than a cross-entropy loss, is a second correctness issue but is secondary to the undefined F. Overlap with AFMAR and the absence of released code, data, and error bars also weaken the empirical claim, but the undefined core module is the most load-bearing single concern. A corrected manuscript that defines F, fixes Eq. 10, and provides ablations isolating the grouping mechanism could merit a future review; the current verdict of REJECT is appropriate because the central claim is not supported as written.","tokens_in":12222,"tokens_out":3610,"duration_ms":38732,"concrete_test":"Obtain the authors' code or an explicit operational definition of F^m_i as a formula from p^m_i, distances, or logits, then run a three-way ablation: (a) group samples by the undefined Eq. 3 criterion if definable, (b) group by c^m_i from Eq. 5, and (c) use random balanced grouping, keeping all losses otherwise identical. If 1-shot and 5-shot accuracies on SSv2 and Kinetics-400 change by less than about 1 point across conditions, the F-driven active-inference grouping is not load-bearing and the reported gains do not validate the active-inference narrative; if the accuracy changes substantially, the missing definition prevents reproduction and the empirical claim remains unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest claim is that the ASI module actively selects the reliable modality per sample, and that this drives the reported gains. The selection rule is Eq. 3: G^m = { samples | F^m_i > F^n_i }, where F is described only as 'free energy derived from Eq. (1)'. But Eq. 1 is the POMDP variational free energy \\tilde F = D_KL(q(s,\\theta) || p(o,s,\\theta)); no formula maps the modality-specific posterior p^m_i(k|x^m_i) of Eq. 2 to a scalar F^m_i for a query sample. Figure 2 plots 'free energy' but gives no axis definition or equation. Consequently Eq. 3 cannot be evaluated, and the paper's active-inference grouping is non-operational. The rest of the method uses c^m_i = max_k p^m_i (Eq. 5) as the reliability signal in distillation (Eq. 6) and adaptive fusion (Eq. 8), so what remains is a confidence-weighted distillation and fusion scheme, not the claimed active-inference mechanism. Because Section 3.4 states that distillation is applied only to the 'selected samples in G^f and G^r', the undefined F also determines the teacher/student assignment and the training subset. If the authors in practice replace F by c^m_i, they should say so and justify max-posterior as a reliability measure; if F is a different quantity, it must be specified. As written, the reported improvements cannot be attributed to the central claimed mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AMFIR, a multimodal few-shot action recognition framework that combines RGB and optical flow modalities. The method has three main components: an Active Sample Inference (ASI) module that groups query samples into RGB-dominant and flow-dominant groups using a free-energy criterion; an Active Mutual Distillation (AMD) module that transfers knowledge from a reliable modality to an unreliable one; and an Adaptive Multimodal Inference (AMI) module that fuses modality-specific posteriors with confidence-based weights at meta-test time. Experiments on Kinetics-400, SSv2, HMDB51, and UCF101 report large gains over prior unimodal and multimodal baselines, with the authors attributing the gains to the ASI module's active modality selection.","tokens_in":12526,"tokens_out":3892,"duration_ms":40684,"significance":"The problem of multimodal few-shot action recognition is timely, and the paper addresses it with a plausible high-level pipeline: cross-modal distillation, confidence-based fusion, and per-sample modality selection. The experimental scope is broad, covering four benchmarks, and the ablation study in Table 2 shows that each component contributes to the final score. However, the central claimed mechanism is not operational as written: Eq. (3) depends on an undefined free-energy term, and the training objective in Eq. (10) is mathematically the negative entropy rather than cross-entropy, which would push predictions toward uniform distributions. These are load-bearing issues, not presentation problems. The paper also does not provide standard deviations or a confidence-only baseline, so the reported gains cannot currently be attributed to active inference rather than to simple confidence weighting. If the technical issues were corrected and the method re-evaluated, the framework could be a useful contribution, but the manuscript in its current form is not ready for publication.","major_comments":[{"comment":"The free-energy term F^m_i in Eq. (3) is never defined. The text says it is 'free energy derived from Eq. (1)', but Eq. (1) is the POMDP variational free energy and no formula maps the modality-specific posterior p^m_i of Eq. (2) to a scalar F^m_i for a query sample. Consequently, Eq. (3) cannot be evaluated, and since Section 3.4 applies distillation only to samples in G^r and G^f, the undefined quantity controls the teacher-student assignment and the training subset. Figure 2 plots 'free energy' without defining the axis or giving the computation. If F^m_i is meant to be c^m_i, the paper should state this explicitly and justify max-posterior as a reliability measure; otherwise, a concrete definition is required.","section":"§3.3, Eq. (3)"},{"comment":"Eq. (10) is not a cross-entropy loss. As written, L^m_ce = \\sum_i \\sum_k p^m_i(k) \\log p^m_i(k) is the negative Shannon entropy of the predicted posterior, and minimizing it drives p^m_i toward a uniform distribution over classes, which is the opposite of classification. The correct cross-entropy should involve the ground-truth label, e.g., -\\sum_i \\log p^m_i(y_i). Because Eq. (9) and Eq. (11) optimize this objective during meta-training, this is a load-bearing mathematical error that undermines the reported results.","section":"§3.6, Eq. (10)"},{"comment":"The method's 'active inference' mechanism is confounded with simple confidence weighting. Eq. (5) defines reliability c^m_i as the maximum of the modality-specific posterior, and Eq. (8) uses the same c^m_i to set the adaptive fusion weights. If F^m_i in Eq. (3) is also computed from the same posterior, then the same self-confidence measure determines the grouping, the distillation weights, and the final fusion. The paper needs an ablation that replaces the 'active inference' grouping with a trivial confidence-based grouping (e.g., assign each sample to the modality with higher c^m_i) to show that the claimed active-inference mechanism, rather than confidence weighting alone, produces the reported gains.","section":"§3.3–§3.5"},{"comment":"The experimental section reports only point estimates without standard deviations, confidence intervals, or statistical significance tests. Given that the reported margins over some baselines are small (e.g., UCF101 5-shot: 99.1 vs 99.0), the absence of variance estimates makes the strength of the comparison unclear. The paper should report mean and standard deviation over multiple random episodes or seeds, and should state the number of meta-test tasks used.","section":"§4.3, Table 1"}],"minor_comments":[{"comment":"The framework name is inconsistent: 'AMFIR' appears in the title and abstract, while 'AFMIR' appears in Table 1 and elsewhere, and the method name in Section 3 is also given as 'AFMIR' in one place. Please unify the nomenclature.","section":"Throughout"},{"comment":"The notation L^{m→L^n} in Eq. (9) and the text immediately after it appears to be a typo for L^{m→n}; please fix the superscript notation.","section":"§3.6, Eq. (9)"},{"comment":"Figure 2 plots 'free energy' but provides no axis labels, no units, and no equation for the plotted quantity; please define the vertical axis and state how the quantity is computed.","section":"§4.2, Figure 2"},{"comment":"The phrase 'reduce the thermal accuracy of Kinetics 5 by 7.06%' is unclear; it should likely read 'reduce the accuracy on Kinetics-400 5-shot by 7.06%.'","section":"§4.4"},{"comment":"Please provide details on the optical flow generation algorithm, backbone pretraining protocols, episode construction (number of tasks, class sampling), and whether the reported numbers are averaged over multiple runs; these details are needed to reproduce the results.","section":"§4.1"},{"comment":"The conclusion claims robustness to 'sensor noise, motion ambiguity, and extreme data scarcity', but none of the experiments in Section 4 introduce sensor noise or corrupted modalities; please either add such experiments or remove the claim.","section":"§5"}],"recommendation":"reject","confidential_remarks":"The paper has two load-bearing technical errors: an undefined free-energy quantity that drives the central grouping mechanism and a training loss that is negative entropy rather than cross-entropy. These are not presentation issues; they make the method as written non-computable and the optimization objective mathematically wrong. Even if the authors intended different formulas, the current text would need substantial rewriting and re-evaluation, which goes beyond a minor or major revision in this journal's normal scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the central mechanism doesn't compute as written—the free energy F in Eq. (3) is never defined, and Eq. (10) mislabels negative entropy as cross-entropy. Those are load-bearing flaws, not typos in the margins. The underlying idea—per-sample modal reliability weighting for few-shot action recognition—is sensible, and the experimental coverage is decent, but the paper can't be accepted in its current form.\n\nWhat's new: The paper proposes to use the model's own confidence (max softmax posterior) to weight RGB vs. flow per query sample, gate which samples receive distillation from which modality, and fuse adaptively at test time. That recipe isn't radically new—AFMAR is cited as already doing active modality selection, bidirectional distillation, and adaptive fusion—but the combination is a reasonable engineering contribution. The comparisons across SSv2, HMDB51, UCF101, and Kinetics are useful, and the ablations show each component contributes something.\n\nThe soft spots:\n1. Eq. (3) groups samples by F^m_i > F^n_i, but F^m_i never appears in any operational formula. The text says it's free energy derived from Eq. (1), yet Eq. (1) is a generic variational bound with no path from the modality posterior p^m_i to a scalar per-sample value. So the ASI module, which the paper credits for the gains, is not computable from the text.\n2. Eq. (10) defines L^m_ce as sum p log p. That's negative entropy. Minimizing it pushes predictions toward uniform, the opposite of classification. If that's really the loss, the reported results are impossible; if it's a typo, the correct loss with ground-truth labels needs to be stated.\n3. The 'reliability' c^m_i = max_k p^m_i is used to weight both distillation and fusion. That's a circularity: the model's own confidence defines reliability, so any overconfident error is self-reinforcing. Confidence weighting isn't wrong per se, but framing it as active inference adds little.\n4. No code, data, or error bars. The 99.1% 5-shot UCF101 number is striking enough to warrant reproducibility checks.\n\nOverlap with AFMAR is acknowledged but the differentiation is thin. I'm not convinced the active-inference framing buys anything beyond what the paper itself describes as AFMAR's existing mechanism.\n\nBottom line: the paper has a plausible empirical kernel, but the central math doesn't hold up as written. I'd desk-reject in its current state and invite a corrected version with F defined, the loss fixed, and code or seed-averaged numbers. Not something I'd cite yet, but worth a quick look if you're in the multimodal few-shot space.","headline":"Sensible multimodal fusion recipe undone by an undefined free-energy term and a mislabeled loss equation; as written, the central mechanism is not computable.","tokens_in":13069,"tokens_out":4229,"would_cite":false,"duration_ms":41370,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AMFIR, a framework that actively selects the more reliable of RGB or optical-flow modalities per query sample, claims substantial gains in few-shot action recognition across four benchmarks.","keywords":["few-shot action recognition","active inference","multimodal learning","knowledge distillation","RGB and optical flow","meta-learning","adaptive fusion","uncertainty estimation"],"falsifier":"Construct a test split where the RGB stream is made confidently wrong while optical flow remains correct (for example, by altering textures without changing motion), then check whether AMFIR still routes high-confidence RGB samples to the RGB-dominant group and follows the RGB branch in the final prediction.","tokens_in":12015,"feed_emoji":"🎬","tokens_out":8163,"duration_ms":74835,"temperature":0.7,"pith_summary":"The paper claims that few-shot action recognition benefits from deciding, per query video, which of two modalities—RGB frames or optical flow—is more reliable, rather than fusing them with fixed weights. It introduces AMFIR, a framework whose Active Sample Inference module scores each modality by the maximum value of its class-posterior distribution and uses that score to group samples, to weight a mutual distillation loss that strengthens the weaker modality, and to set adaptive fusion weights at test time. On SSv2, HMDB51, UCF101, and Kinetics-400, the method reports accuracy that exceeds prior unimodal and multimodal baselines, for example 70.6% and 92.3% on SSv2 and 94.9% and 99.1% on UCF101 for 1-shot and 5-shot. A sympathetic reading is that uncertainty-driven modality selection, not extra data, produces the gain. The components are simple enough that they can be ablated individually, and the paper does so.","feed_headline":"Active multimodal model hits 94.9% one-shot accuracy on UCF101","feed_subtitle":"The method picks each video's reliable modality and distills its knowledge into the weaker one.","key_machinery":"The load-bearing mechanism is the Active Sample Inference (ASI) module, built on the absolute certainty $c_i^m = \\max_k p_i^m(k \\mid x_i^m)$, where $p_i^m$ is the softmax over class prototypes for modality $m$ and query sample $i$. This scalar is computed separately for RGB and optical flow and does three jobs: it routes the query into an RGB-dominant or flow-dominant group, it weights the KL-divergence distillation loss $L^{m\\to n}$ that lets the reliable modality teach the unreliable one, and it produces the adaptive fusion weight $\\alpha_i^m = c_i^m/(c_i^r + c_i^f)$ for the final prediction. The paper's claim is that this confidence measure, taken from task-specific posterior distributions, captures which modality carries discriminative information for that particular sample in that particular episode.","core_discovery":"On the paper's own terms, the central discovery is that a single per-sample scalar—the maximum of a modality-specific softmax posterior over class prototypes—can serve as the organizing signal for multimodal few-shot learning. With only RGB and optical-flow streams, the framework uses this scalar to infer which modality is dominant for each query sample, transfers query-to-prototype similarity knowledge from the dominant to the weaker modality through a KL-divergence distillation, and fuses the final prediction using the same scalar as an adaptive weight. The authors report that the resulting AMFIR framework outperforms existing few-shot action recognition methods on all four tested benchmarks, with the largest reported gains on UCF101 and SSv2.","pith_inferences":["Inference: because the paper never gives an operational definition for the free energy $F_i^m$ used in the grouping equation, the active-inference grouping reduces in practice to a comparison of per-modality confidences; replacing the grouping rule with a simple threshold on $c_i^m$ would test whether the theoretical framing adds predictive power.","Inference: the same confidence-as-reliability proxy should transfer to other modality pairs, such as audio with video or skeleton with RGB, where posterior calibration patterns differ; a cross-modal test would reveal whether the method is truly modality-agnostic.","Inference: because the fusion weight is a per-sample normalized ratio of two confidences, the method implicitly assumes that the two modality streams are calibrated on comparable scales; per-dataset calibration curves would show whether the reported gains survive when the scales are mismatched."],"forward_implications":["On UCF101, the reported 99.1% 5-shot accuracy would leave that benchmark near saturation, so further progress would have to be demonstrated on harder datasets.","Because the teacher assignment is made per sample, a single episode can contain both RGB-teacher and flow-teacher samples, which a fixed teacher-student design cannot express.","The method adds no extra data or labels beyond the standard support and query episodes, so the reported gains, if real, come from the learning procedure rather than from additional supervision.","The three components—grouping, mutual distillation, and adaptive fusion—are modular enough to be inserted into an existing unimodal few-shot recognizer."],"supporting_citations":[{"why":"Supplies the active-inference formulation that the ASI module claims to instantiate.","marker":"[Tschantz et al., 2020]"},{"why":"The closest prior multimodal active-selection method and the main baseline to surpass.","marker":"[Wanyan et al., 2023]"},{"why":"Provides the Matching Net baseline and the RGB-stream few-shot formulation.","marker":"[Zhu and Yang, 2018]"},{"why":"Provides the TRX baseline using temporal-relational cross-transformers for few-shot action recognition.","marker":"[Perrett et al., 2021]"},{"why":"Provides the HyRSM baseline using hybrid relation guided set matching.","marker":"[Wang et al., 2022]"},{"why":"Provides the STRM baseline, which the authors also retrain on optical flow for comparison.","marker":"[Thatipelli et al., 2022]"},{"why":"Supplies the Kinetics-400 benchmark used in the evaluation.","marker":"[Kay et al., 2017]"},{"why":"Supplies the Something-Something V2 benchmark used in the evaluation.","marker":"[Goyal et al., 2017]"},{"why":"Prior modality-distillation approach that motivates the cross-modal distillation loss.","marker":"[Garcia et al., 2018]"},{"why":"MARS, which distills motion knowledge into an RGB stream, is a conceptual ancestor of the mutual distillation used here.","marker":"[Crasto et al., 2019]"}],"fun_headline_variants":["Active modality inference picks trusted stream for few-shot videos","Distilling reliable modality knowledge improves few-shot action recognition","Active multimodal distillation tunes per-sample modality weights","Which video modality to trust? Active inference and distillation","Active modality selection lifts one-shot accuracy to 94.9% on UCF101"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that when the model is very sure of one modality's answer, that modality is the right one to trust; a confidently wrong guess would mislead the grouping, the distillation, and the final fusion.","fun_headline_variants_meta":{"raw":{"variants":["Active modality inference picks trusted stream for few-shot videos","Distilling reliable modality knowledge improves few-shot action recognition","Active multimodal distillation tunes per-sample modality weights","Which video modality to trust? Active inference and distillation","Active modality selection lifts one-shot accuracy to 94.9% on UCF101"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000851,"raw_usage":{"total_tokens":3639,"prompt_tokens":826,"completion_tokens":2813,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":2732}},"tokens_in":442,"tokens_out":2813,"duration_ms":21631,"temperature":1.0,"reasoning_tokens":2732,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:04:15.877411+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a test split where the RGB stream is made confidently wrong while optical flow remains correct (for example, by altering textures without changing motion), then check whether AMFIR still routes high-confidence RGB samples to the RGB-dominant group and follows the RGB branch in the final prediction.","supporting_citations":[{"cited_title":"Hybrid relation guided set matching for few-shot action recognition","cited_arxiv_id":null,"evidence_quote":"Provides the HyRSM baseline using hybrid relation guided set matching."},{"cited_title":"Active exploration of multimodal complementarity for few-shot action recog- nition","cited_arxiv_id":null,"evidence_quote":"The closest prior multimodal active-selection method and the main baseline to surpass."},{"cited_title":"Compound memory networks for few-shot video classification","cited_arxiv_id":null,"evidence_quote":"Provides the Matching Net baseline and the RGB-stream few-shot formulation."},{"cited_title":"Temporal- relational crosstransformers for few-shot action recogni- tion","cited_arxiv_id":null,"evidence_quote":"Provides the TRX baseline using temporal-relational cross-transformers for few-shot action recognition."},{"cited_title":"Spatio-temporal relation modeling for few-shot action recognition","cited_arxiv_id":null,"evidence_quote":"Provides the STRM baseline, which the authors also retrain on optical flow for comparison."},{"cited_title":"The” something something” video database for learning and evaluating vi- sual common sense","cited_arxiv_id":null,"evidence_quote":"Supplies the Something-Something V2 benchmark used in the evaluation."},{"cited_title":"Modality distillation with multiple stream networks for action recognition","cited_arxiv_id":null,"evidence_quote":"Prior modality-distillation approach that motivates the cross-modal distillation loss."},{"cited_title":"Mars: Motion- augmented rgb stream for action recognition","cited_arxiv_id":null,"evidence_quote":"MARS, which distills motion knowledge into an RGB stream, is a conceptual ancestor of the mutual distillation used here."}],"review_version":1}