{"id":"d3f6e638-6ee5-4155-9d1a-b5e9c067249f","arxiv_id":"2507.20737","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"MMQ-Net combines masked modality queries, category queries, and interference queries in one transformer and reports 90 to 99 percent accuracy on emotion classification under missing physiological data.","lead":"This paper describes MMQ-Net, a neural network that fills in missing physiological signal channels and separates emotion-related information from noise to classify valence and arousal. A generalist would care because robust emotion recognition from imperfect wearable data could support mental health monitoring, but the paper's results would need independent verification.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reconstruction loss in Eq. (5) never supervises the values that modality queries are supposed to output for missing modalities, so the paper's central mechanism for handling incompleteness is untrained as written.","rationale":"The reader's weakest assumption correctly identifies a load-bearing gap in the paper's central argument. I considered other potential concerns, including the under-specified attention mask in Eq. (2), the unimplementable mutual-information term in Eq. (7), and the lack of code or error bars. These are serious reproducibility issues, but the most direct logical flaw is that Eq. (5) does not train the network to reconstruct missing modalities. If the reconstruction loss is insensitive to missing entries, then the claimed mechanism for robustness under high missing rates is not actually learned, and the experimental results, even if accurate, would not demonstrate the proposed contribution. This is an internal-consistency issue rather than a mere disagreement with field consensus. My proposed test is designed to settle the question: if the model trained under Eq. (5) reconstructs missing features no better than a baseline, the central claim fails; if an oracle-target loss is needed, the paper must document it and re-report results. Since the reader's verdict was already REJECT and this concern reinforces that conclusion, I recommend UNCHANGED.","tokens_in":8131,"tokens_out":2643,"duration_ms":38281,"concrete_test":"Re-run the DEAP experiments with a modified training protocol: randomly mask modalities in the input exactly as in the paper, but compute LR against the unmasked original feature vectors for all modalities, i.e., LR_oracle = || F_hat_i^M - F_i^M ||^2 on both available and masked entries. Also directly measure missing-modality reconstruction error: for held-out samples with modality k masked, compare F_hat_i^(k) to the ground-truth feature f_i^(k) against a mean-feature baseline. If the original Eq. (5) model does not reconstruct missing features better than baseline, or if the accuracy tables are unchanged by LR_oracle, then the paper's claim that modality queries reconstruct missing data is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that modality queries reconstruct missing data from incomplete signals (Sec. 2.3). However, Eq. (5) defines LR = (1 / sum_i a_i^T 1) * sum_i || a_i ⊙ F_hat_i^M - a_i ⊙ F_i^M ||^2. For any missing modality m, a_i,m = 0, so both the prediction and the target are multiplied by zero. The loss is therefore completely insensitive to the values produced for missing modalities. Nothing in the paper states that, during training, modalities are artificially dropped while the original unmasked features are retained as oracle targets; the experiment section only says missing rates are used to 'simulate real-world data conditions,' which is compatible with masking at both input and target. If no oracle-masking protocol exists, the model is never trained to output actual missing-modality features, and the reported robustness cannot be attributed to the proposed reconstruction mechanism. The ablation w/o LR in Table 3 only shows that removing LR hurts accuracy; it does not demonstrate that LR causes missing entries to be reconstructed. The same concern undermines the paper's stated contribution: the performance gains could come entirely from category queries attending to available modalities, with the reconstruction loss acting as an auxiliary regularizer rather than a missing-data imputer.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MMQ-Net, a transformer with three learnable query sets—modality queries, category queries, and interference queries—for emotion recognition from multimodal physiological signals with missing modalities and noise. It defines reconstruction, classification, and mutual-information losses, and reports accuracy on DEAP and MAHNOB-HCI across missing rates from 0.0 to 0.7, claiming consistent improvements over CCA, KCCA, DCCA, AE, SMIL, ShaSpe, and TAE. Ablation results show that removing either the reconstruction loss or the interference-reduction loss lowers accuracy. The central claims are that modality queries reconstruct missing data and that interference queries separate noise, but the manuscript provides no code, no detailed experimental protocol, and no direct validation of the interference mechanism.","tokens_in":8389,"tokens_out":7267,"duration_ms":90816,"significance":"The task is practically important, and the three-query decomposition is a plausible design direction. If the results were reproducible, a single architecture that robustly handles up to 70% missing physiological data would be a useful contribution. However, the reported evidence is not sufficient: the central reconstruction loss never supervises missing-modality outputs, the attention mask formula does not implement the claimed masking, the mutual-information loss is unspecified, and the experimental protocol lacks split definitions and error bars. The paper provides no code, machine-checked proofs, or parameter-free derivations, so the empirical tables are the sole support for the claims. The contribution may be salvageable in a substantially revised form, but as written it does not establish its claims.","major_comments":[{"comment":"The reconstruction loss LR only compares masked predictions ai ⊙ F_hat_i^M with masked targets ai ⊙ F_i^M; for missing modalities ai,m = 0, both sides vanish, so the loss provides no gradient signal for the values output by modality queries for missing modalities. The experimental section describes missing rates only as a simulation of real-world conditions and never states that, during training, available channels are artificially dropped while the original features are retained as oracle targets. As written, the modality-query reconstruction mechanism is therefore untrained, and the reported robustness cannot be attributed to it.","section":"§2.3, Eq. (5)"},{"comment":"The attention mask M = D + 1[a^T,1,1] is not a mask: it adds a positive constant to every position and leaves missing modalities with only the identity self-attention, rather than suppressing them. The text claims the mask ensures queries learn only from available modalities, but the formula does not implement this. A proper additive mask with -inf at missing tokens, or a clear derivation of why this formula masks missing positions, is needed.","section":"§2.3, Eq. (2)"},{"comment":"The interference-reduction loss I(y; F_I | F_C) is written as a conditional mutual-information term, but the paper gives no estimator, no discretization, no variational bound, and no statement of how it is differentiated during training. Consequently the third objective is not well-defined as an implementable loss, and the ablation of LI cannot be interpreted without knowing what was actually computed.","section":"§2.4, Eq. (7)"},{"comment":"The empirical protocol is under-specified. There is no description of train/validation/test splits, no indication of whether evaluation is subject-dependent or subject-independent, no standard deviations or significance tests, and no implementation details for the seven baselines. Hyperparameters are said to be selected by cross-validation, but it is not stated that the numbers in the tables come from an independent test set. Without these details, the reported 90-99.85% accuracies and the claimed improvements over baselines are not verifiable.","section":"§3.2 and Tables 1-2"},{"comment":"The paper's second contribution, interference reduction, is never directly evaluated. The experiments only vary the modality missing rate; no synthetic artifacts, movement corruption, or real artifact-containing test set is used. The ablation that removes LI shows an accuracy drop, but that could be due to regularization rather than to interference separation. The interference-query mechanism therefore lacks empirical support.","section":"§3.4 and Abstract"}],"minor_comments":[{"comment":"The caption contains a typo: 'Multi-model' should be 'Multi-modal'.","section":"Fig. 2 caption"},{"comment":"The notation 1[a^T,1,1] is ambiguous; the dimensions of the identity matrix and the resulting mask are not defined.","section":"§2.3, Eq. (2)"},{"comment":"The denominator sum_i a_i^T 1 can be zero when all modalities are missing; the paper should state how this case is handled.","section":"§2.4, Eq. (5)"},{"comment":"The list of frequency bands includes both alpha (8-10 Hz) and slow alpha (8-13 Hz); the overlap should be clarified.","section":"§2.2"},{"comment":"No code release or data-processing script is mentioned, which limits reproducibility of the preprocessing and feature-extraction steps.","section":"§3.2"}],"recommendation":"reject","confidential_remarks":"To the editor: I recommend rejection. The load-bearing problems—unsupervised missing-modality reconstruction, an incorrect attention mask formula, an undefined mutual-information loss, and an under-specified evaluation—mean the central claims are not supported as written. A future submission would need oracle-masked reconstruction, a corrected mask, a computable MI term, and a complete train/test protocol with variance estimates. This goes beyond a routine major revision because it requires re-deriving and re-running the experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBottom line: the paper's central mechanism for handling missing data is untrained as written. Eq. (5) computes the reconstruction loss only on available modalities: both the prediction and target are masked by a_i, so missing entries contribute zero. Nothing in the text says they artificially drop channels during training while keeping the original features as oracle targets. If that's the case, the modality queries are never supervised to output actual missing values, and the reported robustness gains must come from the category queries attending to available modalities, with LR acting as a regularizer rather than an imputer. The stress-test note is right.\n\nWhat's genuinely new: the three-query combination (modality, category, interference) in one transformer, with the three-term loss, is an architectural tweak I haven't seen in the cited baselines. The accuracy tables are new measurements. The authors also correctly identify a real problem—missing modalities and artifact interference in wearable affect monitoring.\n\nCredit where due: the ablation in Table 3 shows that removing LR hurts accuracy by 4-5 points, so the reconstruction loss is doing something. But that doesn't prove it reconstructs missing entries. The idea of using learnable query tokens to fill missing modalities is a reasonable extension of SMIL and TAE; it could work if the training protocol were specified and if the loss actually supervised imputation.\n\nSoft spots, in order: (1) the reconstruction supervision gap is load-bearing; (2) no error bars, no subject-split description, no baseline provenance, so the 99.85% DEAP accuracy is uninterpretable; (3) Eq. (2) attention mask is under-specified—D+1[a^T,1,1] doesn't make sense as an attention mask without more explanation; (4) the mutual information loss in Eq. (7) is not operationalized; I(y;F_I|F_C) for continuous features is not obviously computable; (5) the title promises artifact interference, but no motion/artifact experiments are shown. These are real concerns, but none are insurmountable—they are fixable with a full protocol and honest experiments.\n\nWho this is for: a reader working on missing-modality fusion in affective computing would find the architecture plausible and worth trying after the authors nail down the training protocol. Right now, the empirical claims outrun the evidence.\n\nMy take: I'd send this to peer review with a strong request for major revision—the question is meaningful and the architecture is plausible, but the paper as written doesn't support its central claim. I'd reject it in its current form, but I'd want to see the revision.","headline":"The reconstruction loss never supervises missing-modality outputs, so the paper's central mechanism for missing data is untrained as written, despite a plausible architecture and useful question.","tokens_in":8900,"tokens_out":2478,"would_cite":false,"duration_ms":27705,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MMQ-Net claims that one masked transformer keeps emotion recognition above 90 percent accuracy even when 70 percent of the physiological signals are missing.","keywords":["multi-modal emotion recognition","physiological signals","missing data","masked querying transformer","attention mask","DEAP","MAHNOB-HCI","EEG"],"falsifier":"Measure reconstruction error on zeroed-out entries for which ground-truth features are known: if the model's loss on those entries does not drop below a mean-fill baseline, the claimed reconstruction of missing modalities is not what drives the accuracy.","tokens_in":7904,"feed_emoji":"🧠","tokens_out":6682,"duration_ms":68794,"temperature":0.7,"pith_summary":"This paper tries to establish that a single transformer architecture, MMQ-Net, can recognize emotional valence and arousal from multi-modal physiological signals even when up to 70% of the signal modalities are missing and body-movement artifacts contaminate the rest. It reports that MMQ-Net outperforms seven comparison methods on DEAP and MAHNOB-HCI across all missing rates from 0 to 0.7, with the largest margins at the highest missing rates. The practical stake is real-time emotion monitoring for mental-health applications, where sensor dropouts and motion noise are unavoidable. If the results hold, the same design could serve as a front end for other incomplete multi-modal learning problems.","feed_headline":"Three-query transformer holds 90%+ accuracy with 70% of data missing","feed_subtitle":"Beats the next-best method by up to 6.8 points on DEAP and MAHNOB-HCI emotion sets.","key_machinery":"The central object is the masked multi-head attention block in the Multi-Masked Querying Transformer, defined by a softmax attention with a mask matrix M derived from the modality-presence indicator a. M allows each query to attend only to available modalities plus the category and interference queries. The mechanism does three jobs at once: learnable modality queries occupy slots of missing data, category queries collect emotional-state evidence, and interference queries absorb noise; the three output streams are then supervised by respective losses. The mask is the load-bearing element: without it, missing queries would contaminate attention; with it, the model is claimed to learn which modalities are absent and reconstruct them from the present ones.","core_discovery":"MMQ-Net's central claim is that missing-modality reconstruction and noise suppression can be unified in one masked multi-head attention block. The input sequence replaces absent modalities with learnable modality queries and appends a category query and an interference query; an attention mask built from the availability vector ensures that missing positions attend only to present data. The outputs split into reconstructed modality features, emotion features, and interference features, trained by three losses: feature-level reconstruction on present modalities, cross-entropy classification, and a mutual-information term that keeps emotion features label-relevant while decorrelating interference features from the label. On DEAP and MAHNOB-HCI, accuracy ranges from 90.06% to 99.85% depending on missing rate, exceeding the next best method by 2.37 to 6.76 percentage points.","pith_inferences":["We infer that the masked-query pattern is transferable to other partially observed multi-modal tasks, such as speech-emotion or human-activity recognition, since nothing in the design is physiological-signal-specific.","We note that because the reconstruction loss in Eq. (5) only uses available modalities, the attention mask may be acting mostly as a regularizer; a head-to-head comparison against oracle-supervised imputation would clarify.","We suggest a testable extension: hide a fixed fraction of present modalities during training with reconstruction targets, then report whether the held-out reconstruction error actually drops.","The near-99.85% accuracy at zero missing suggests the DEAP benchmark may be approaching saturation for this subject-dependent split, so future comparisons should include cross-subject protocols."],"forward_implications":["At a 0.7 missing rate, the method reports 90%+ accuracy on both datasets, implying it can be used when most modalities are absent.","The performance gap over the second-best method widens as missing rate increases, implying the benefits concentrate in data-scarce regimes.","Ablation dropping the reconstruction loss reduces accuracy by roughly 4 to 6 points at 30% missing, implying modality reconstruction contributes to classification.","Ablation dropping the interference loss also reduces accuracy, implying separation of noise from emotional features matters.","The same architecture and hyperparameters serve both DEAP and MAHNOB-HCI and both valence and arousal, implying the design transfers across datasets and targets."],"supporting_citations":[{"why":"TAE, the strongest previous incomplete-data transformer baseline that MMQ-Net must beat.","marker":"[2]"},{"why":"DEAP dataset, the first benchmark providing EEG and peripheral physiological signals.","marker":"[5]"},{"why":"SMIL, a baseline for severely missing modalities that MMQ-Net outperforms.","marker":"[13]"},{"why":"MAHNOB-HCI dataset, the second benchmark for evaluation.","marker":"[18]"},{"why":"The multi-head attention operation that the masked querying transformer builds on.","marker":"[20]"},{"why":"ShaSpe, a shared-specific feature baseline for missing-modality learning.","marker":"[21]"},{"why":"Mutual-cross-attention used to fuse DE and PSD features before the transformer.","marker":"[25]"}],"fun_headline_variants":["Neural net reads emotions from 70% missing bio-signals","Emotion AI robust to incomplete physiological data","Three-query network recognizes emotions from partial signals","MMQ-Net handles missing data, boosts emotion recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reconstruction loss is computed only on modalities that are present, so the stated training procedure never directly supervises the model on values it is supposed to invent; the paper does not describe an extra protocol that hides available modalities and scores the imputed entries against ground truth.","fun_headline_variants_meta":{"raw":{"variants":["Neural net reads emotions from 70% missing bio-signals","Emotion AI robust to incomplete physiological data","Three-query network recognizes emotions from partial signals","MMQ-Net handles missing data, boosts emotion recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000265,"raw_usage":{"total_tokens":1547,"prompt_tokens":822,"completion_tokens":725,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":661}},"tokens_in":438,"tokens_out":725,"duration_ms":8796,"temperature":1.0,"reasoning_tokens":661,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T13:18:34.205621+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure reconstruction error on zeroed-out entries for which ground-truth features are known: if the model's loss on those entries does not drop below a mean-fill baseline, the claimed reconstruction of missing modalities is not what drives the accuracy.","supporting_citations":[{"cited_title":"Neural Networks 172, 106111 (2024)","cited_arxiv_id":null,"evidence_quote":"TAE, the strongest previous incomplete-data transformer baseline that MMQ-Net must beat."},{"cited_title":"IEEE transactions on affective computing 3(1), 18–31 (2011)","cited_arxiv_id":null,"evidence_quote":"DEAP dataset, the first benchmark providing EEG and peripheral physiological signals."},{"cited_title":"In: Proceedings o f the AAAI Conference on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"SMIL, a baseline for severely missing modalities that MMQ-Net outperforms."},{"cited_title":"IEEE transactions on affective computing 3(1), 42–55 (2011)","cited_arxiv_id":null,"evidence_quote":"MAHNOB-HCI dataset, the second benchmark for evaluation."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni tion","cited_arxiv_id":null,"evidence_quote":"ShaSpe, a shared-specific feature baseline for missing-modality learning."},{"cited_title":"In: International Conference on M edical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"Mutual-cross-attention used to fuse DE and PSD features before the transformer."}],"review_version":1}