{"id":"e9f9279e-e116-4044-b817-f5527982a937","arxiv_id":"2505.12007","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MCO-E Net fuses event and RGB eye data via a jointly optimized Mamba and a heterogeneous MoE, achieving 91.3% WAR and 91.9% UAR on the SEE dataset.","lead":"This paper presents a deep network that combines event camera data with RGB eye images to recognize single-eye expressions, reporting the highest scores to date on the SEE benchmark. The network pairs a jointly optimized Mamba fusion module with a heterogeneous mixture-of-experts to handle low-light, overexposure, and high dynamic range.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim rests on hyperparameters selected directly on the SEE test set; without a validation split or error bars, the 4.4/4.2-point margin may be selection-induced.","rationale":"I read the paper as an empirical architecture contribution whose strongest claim is quantitative SOTA on the SEE benchmark. The central claim requires the final hyperparameter configuration to generalize, and the weakest point is exactly the evaluation protocol: hyperparameters are selected by ablating on the test set, with no validation split and no error bars. The reader's weakest_assumption identifies the same issue, and I agree with that diagnosis. The concern is concrete and testable: the final configuration beats its nearest ablated alternatives by only ~1.0–1.3 WAR, which is comparable to the standard error of a single test-set estimate, and selection among many configurations on the same test set can plausibly account for part or all of the claimed margin. I do not see an internal inconsistency in the architecture, and the paper does provide code and systematic ablations, which are positive evidence; however, code availability does not fix the test-set selection problem. The right disposition is to keep the reader's CONDITIONAL verdict: the work is publishable as an incremental architecture study, but the SOTA claim should be treated as conditional pending a proper validation-based selection protocol and variance reporting.","tokens_in":15218,"tokens_out":3298,"duration_ms":34971,"concrete_test":"Re-run model selection with a held-out validation split: split the 1,638 training sequences into ~80% training and ~20% validation, tune the SSM interaction matrices, Ne, TopK, and early-stopping epoch on validation WAR/UAR, then evaluate the single selected model once on the official 767-sequence test set. Repeat over 5 random seeds and report mean ± std. If the selected configuration's test margin over HI-Net (86.9 WAR, 87.7 UAR) falls below about 1 point or reverses, test-set selection is the likely cause of the reported gain. If the margin remains near 4.4/4.2 with small variance, the SOTA claim is materially supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that MCO-E Net surpasses prior SOTA by 4.4 WAR and 4.2 UAR on SEE (Section 4.2, Table 1). For this claim to hold, the reported configuration must generalize, but the configuration is chosen by test-set performance. Section 4.1 reports only a train/test split (1,638/767 sequences) and no validation set, and the final choices — interacting only the B and C SSM matrices, Ne=8, and TopK=2 — are the winners of Tables 3, 5, and 6, all of which report WAR/UAR on the same test split. The paper reports a single number per configuration without error bars or repeated seeds. With 767 test sequences, the standard error of a ~91% accuracy is about 1.0 percentage point, and the preferred configuration beats the closest ablated alternatives (Ne=5, k=3) by only 1.0–1.3 points while many configurations cluster between 89 and 91 WAR. Selecting among roughly 15 configurations on the same data can plausibly inflate the apparent gain by more than the remaining margin. The claimed SOTA advantage is therefore not yet established as a property of the architecture rather than of the selection procedure. This is a methodological objection, not an architectural one: the internal ablations are consistent and the components are plausible, but the evaluation protocol does not support the precise 4.4/4.2 claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MCO-E Net, a dual-modality network for single-eye expression recognition that combines RGB frames with event streams. The two main components are MCO-Mamba, which jointly optimizes and interacts the SSM parameters of the two modalities via a fusion function M and a cross-attention block, and HCE-MoE, which routes inputs through structurally heterogeneous experts (deep, attention, and focal) with an attention-based router. On the SEE benchmark, the authors report 91.3% WAR and 91.9% UAR, outperforming the previous best method by 4.4 and 4.2 percentage points, and they provide ablations of the two components, the interaction parameters, the number of experts, and the top-k routing. The code is released.","tokens_in":15488,"tokens_out":3760,"duration_ms":39839,"significance":"If the reported results are robust, this is a useful advance for event-assisted expression recognition, a niche but practically motivated task (driver monitoring, privacy-preserving HCI). The architectural ideas are plausible: collaborative SSM interaction is a reasonable extension of recent Mamba-based multimodal work, and heterogeneous experts with an attention router address a known limitation of homogeneous MoE layers. The paper ships code, reports systematic ablations, and the ablations are internally consistent with the design narrative. The central claim, however, is an empirical SOTA benchmark result, and its support depends entirely on the soundness of the evaluation protocol; the protocol as reported does not yet establish the claim.","major_comments":[{"comment":"The final configuration is selected by ablating directly on the same test split that is used for the headline SOTA comparison. Section 4.1 reports only a train/test split (1,638/767 sequences) and no validation set, and the chosen settings — interacting only the B and C SSM matrices, N_e=8, and TopK=2 — are the winners of Tables 3, 5, and 6, all of which report WAR/UAR on the test split. With 767 test sequences, the standard error of a ~91% accuracy is about 1.0 percentage point, and several alternative configurations are within 1.0-1.3 points of the chosen one (Table 5 A vs D; Table 6 B/C vs D). Selecting among roughly 15 configurations on the same data can plausibly inflate the apparent gain by more than the remaining margin. The claimed 4.4/4.2-point SOTA advantage is therefore not yet established as a property of the architecture rather than of the selection procedure. Please add a held-out validation split for model selection, or use nested cross-validation, and report the final configuration's performance on a truly unseen test set.","section":"Section 4.1, Tables 3, 5, 6"},{"comment":"The paper reports a single run per configuration and provides no error bars, confidence intervals, or significance tests. This matters because the central claim is a margin over prior SOTA, and many of the internal ablations differ by about one percentage point or less (e.g., Table 5: N_e=8 vs N_e=5 is 1.3 WAR; Table 6: k=2 vs k=3 is 1.0 WAR; Table 3: best B,C vs next-best A,C is 1.1 WAR). At 767 test sequences, these differences are within the expected sampling noise. Please report mean and standard deviation over at least three seeds for all tables, and preferably also confidence intervals or paired significance tests for the main comparison.","section":"Tables 1-6"},{"comment":"The strongest baseline, HI-Net, is the authors' own previous work (reference [14]), yet the manuscript does not state whether HI-Net was retrained under the exact same protocol (same pretraining, same event representation, same number of input frames, same optimizer and hyperparameters) or whether its numbers are taken from the original paper. Since the claimed SOTA advantage is precisely the margin over this baseline, the fairness of the comparison is load-bearing. Please clarify the protocol for all baselines and explicitly disclose the self-citation relationship in the text.","section":"Table 1, References [14]"}],"minor_comments":[{"comment":"The text says 'B is to keep only Focal Experts', but Table 4 lists row D as the Focal-Experts-only configuration; this is a typo that should be corrected.","section":"Section 4.5, Table 4"},{"comment":"The text states that single-parameter sharing 'drops dramatically', but rows A-D are all between 89.8 and 90.2 WAR, only 1.1-1.5 points below the 91.3 of the chosen configuration; the wording overstates the observed degradation.","section":"Section 4.4, Table 3"},{"comment":"There are minor language issues such as 'comptitive performance' in the abstract and 'surpass the current state-of-the-art methods' without an explicit subject; these should be polished.","section":"Abstract and Section 4.2"},{"comment":"The evaluation metric descriptions are slightly misleading: WAR is described as 'sample-size weighted recall' and as preventing majority-class dominance, but WAR is simply overall accuracy (weighted by class frequency); only UAR is class-balanced. Please rephrase for accuracy.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The core architectural work is plausible and the ablations are systematic, but the evaluation protocol does not yet support the central SOTA claim because hyperparameters are selected on the test set and no error bars are reported. This is fixable within the manuscript's scope by adding a validation split or nested cross-validation, reporting multiple seeds, and clarifying the HI-Net comparison. I do not see equation-level circularity or an irreparable flaw, so reject would be disproportionate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a decent, incremental paper. It combines two known ideas—cross-modal Mamba parameter interaction and a mixture of heterogeneous experts—into a single network for event-assisted single-eye expression recognition, and reports the best numbers on the SEE benchmark. The writing is clear and the authors release code, which helps. But I would not trust the precise 4.4/4.2 WAR/UAR margin until the evaluation protocol is fixed.\n\nWhat is genuinely new: the specific instantiation of MCO-Mamba (interacting the B and C state-space matrices across modalities, plus a gating fusion block) and a heterogeneous MoE with deep, attention, and focal experts. The ablations are internally consistent: removing either module hurts, and the component-wise contributions look sensible. That is real evidence the architecture does something.\n\nWhere it falls down: the final configuration is chosen from ablations run directly on the test set. Tables 3, 5, and 6 each show several configurations; the winners are (B,C), Ne=8, TopK=2. With 767 test sequences, a ~91% accuracy has a standard error near 1 point, and many nearby configurations sit within 1–1.5 points. Selecting the best from a handful of variants on the same test set can easily produce a spurious gain of that magnitude. No error bars or repeated seeds are reported, so we cannot tell whether the 4.4-point advantage over HI-Net is real or selection noise. The strongest baseline is also the authors' own prior work, which is fine but worth noting.\n\nThe paper does not do anything wrong beyond what is common in the field, but the central claim is worded too strongly. The architecture is plausible; the evidence is not yet convincing at that level of precision.\n\nWho it is for: people working on event-based facial or eye expression recognition, or on Mamba-based multimodal fusion. It is a reasonable reference for the design space. I would send it to peer review, but I would insist that the authors either provide a proper validation split or repeated runs with confidence intervals. Without that, the SOTA claim should be presented as preliminary.","headline":"A reasonable event-RGB fusion network, but the headline SOTA margin is likely inflated by test-set hyperparameter selection and missing error bars.","tokens_in":16073,"tokens_out":2736,"would_cite":false,"duration_ms":28331,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that jointly optimizing a Mamba state-space model with both RGB and event data, then routing through heterogeneous experts, sets a new state of the art on the SEE single-eye expression recognition benchmark.","keywords":["single-eye expression recognition","event camera","state space model","Mamba","mixture of experts","multimodal fusion","emotion recognition","illumination robustness"],"falsifier":"Run the same architecture with hyperparameters selected on a held-out validation split rather than the test set, or on a new collection of single-eye event-RGB recordings with different lighting and subjects; if the 4.4-point WAR margin shrinks or reverses, the paper's central claim of a new state of the art would not survive.","tokens_in":14972,"feed_emoji":"👁","tokens_out":4913,"duration_ms":46431,"temperature":0.7,"pith_summary":"The paper argues that a network combining RGB frames and event-camera streams can recognize single-eye expressions more reliably than prior methods, especially in low light, overexposure, and high dynamic range conditions. On the SEE benchmark it reports 91.3% WAR and 91.9% UAR, roughly 4.4 and 4.2 points above the previous best. This matters because single-eye expression recognition offers a privacy-preserving route to driver monitoring and close-range interaction, but has been fragile under difficult illumination. The proposed MCO-E Net attributes the gain to two components: a Mamba block whose state-space matrices are jointly optimized with both modalities, and a mixture-of-experts layer with structurally different experts and an attention-based router.","feed_headline":"Event plus RGB eye model hits 91.9 percent","feed_subtitle":"New fusion network claims state-of-the-art single-eye emotion recognition, with the biggest gains in poor lighting.","key_machinery":"The central machinery is the Multi-modal Collaborative Optimization Mamba (MCO-Mamba) and the Heterogeneous Collaborative and Expansion Mixture-of-Experts (HCE-MoE). In MCO-Mamba, the SSM parameter matrices B and C of each modality are produced by concatenating both modalities' features, applying a learned linear projection, and adding back the modality's own value; only these B and C matrices are interacted, while A and D are left untouched. A gated cross-attention block then lets RGB features retrieve event motion details and event features retrieve RGB static semantics. HCE-MoE routes each fused representation through a Top-K selection among eight experts of three structural types (deep feed-forward, multi-head self-attention, and small-kernel convolution), with routing weights produced by an attention-augmented router. The B-C interaction carries the cross-modal alignment; the heterogeneous experts carry the semantic diversification.","core_discovery":"The central claim is that the modality gap between RGB video and event streams can be narrowed inside a selective state-space model rather than after feature extraction. The paper's MCO-Mamba updates the SSM input and output matrices B and C for each modality using information from both modalities; this joint optimization, supported by a cross-attention interaction block, is what aligns the two distributions. The second claim is that a Mixture of Experts with heterogeneous architectures (deep, attention, focal) outperforms a homogeneous MoE, because different experts capture complementary aspects of eye expression semantics. On the SEE test set the full system reaches 91.3% WAR and 91.9% UAR, exceeding all compared RGB-only, event-only, and fused baselines, with the largest gains over the runner-up in low-light and HDR conditions.","pith_inferences":["A consequence the authors do not state is that their fusion recipe, especially the B-C-only interaction in a selective state-space model, may transfer to other event-RGB tasks where long sequences and modality gaps matter, such as action recognition or privacy-preserving monitoring.","Because the final hyperparameters were chosen by ablating directly on the test set, the reported margin may shrink when the configuration is selected on a held-out validation split; a validation-based protocol would settle this.","If the extreme-lighting robustness holds under independent evaluation, event-assisted eye-region recognition could become a practical alternative to full-face recognition for driver monitoring and eyewear devices where privacy and illumination robustness are priorities."],"forward_implications":["If the central claim is correct, event-assisted single-eye recognition becomes the state of the art on the SEE benchmark, with the strongest gains under overexposure, low-light, and high dynamic range conditions.","Interacting only the B and C state-space matrices, while leaving A and D untouched, is the configuration that yields the reported accuracy; other interaction patterns degrade performance.","Heterogeneous experts with an attention-based router contribute more than expert count alone, since changing the number of experts away from eight reduces accuracy.","Removing either MCO-Mamba or HCE-MoE drops WAR and UAR by about two points, indicating both components carry part of the reported gain."],"supporting_citations":[{"why":"Supplies the SEE dataset, the event-RGB task formulation, and the SEEN baseline that MCO-E Net must beat.","marker":"[40]"},{"why":"Provides the MSKD distillation baseline, the previous best eye-region event-RGB method in the comparison.","marker":"[33]"},{"why":"HI-Net is the nearest event-RGB interaction baseline, and the heatmap and accuracy comparisons target it directly.","marker":"[14]"},{"why":"Supplies the Mamba selective state-space architecture that MCO-Mamba builds on and modifies.","marker":"[11]"},{"why":"Supplies the bidirectional scanning design used in the joint optimization state-space updates.","marker":"[45]"},{"why":"Defines the sparsely-gated Mixture-of-Experts and Top-K routing that HCE-MoE adapts.","marker":"[28]"},{"why":"Motivates heterogeneous experts by showing the limits of homogeneous MoE, which HCE-MoE extends to structural heterogeneity.","marker":"[32]"},{"why":"Provides the voxel-grid conversion used to turn asynchronous event streams into tensor inputs.","marker":"[1]"}],"fun_headline_variants":["Event-RGB fusion via state-space model lifts eye expression scores","MCO-Mamba aligns event and RGB for sharper single-eye emotions","Heterogeneous MoE boosts single-eye emotion recognition in the dark","Joint SSM optimization yields 91.9% on single-eye expression","MCO-E Net event-assisted recognition excels in poor light"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the hyperparameters chosen by ablation on the SEE test set (interacting only B and C, eight experts, Top-K two) will keep working on unseen data, even though no held-out validation set or error bars are reported.","fun_headline_variants_meta":{"raw":{"variants":["Event-RGB fusion via state-space model lifts eye expression scores","MCO-Mamba aligns event and RGB for sharper single-eye emotions","Heterogeneous MoE boosts single-eye emotion recognition in the dark","Joint SSM optimization yields 91.9% on single-eye expression","MCO-E Net event-assisted recognition excels in poor light"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001078,"raw_usage":{"total_tokens":4483,"prompt_tokens":887,"completion_tokens":3596,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":3507}},"tokens_in":503,"tokens_out":3596,"duration_ms":27510,"temperature":1.0,"reasoning_tokens":3507,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:42:49.364647+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same architecture with hyperparameters selected on a held-out validation split rather than the test set, or on a new collection of single-eye event-RGB recordings with different lighting and subjects; if the 4.4-point WAR margin shrinks or reverses, the paper's central claim of a new state of the art would not survive.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MSKD distillation baseline, the previous best eye-region event-RGB method in the comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HI-Net is the nearest event-RGB interaction baseline, and the heatmap and accuracy comparisons target it directly."},{"cited_title":"Person Re-Identification without Identification via Event Anonymization","cited_arxiv_id":"2308.04402","evidence_quote":"Provides the voxel-grid conversion used to turn asynchronous event streams into tensor inputs."}],"review_version":1}