{"id":"3d752ada-3b18-4d25-8a9c-e6fff223fa25","arxiv_id":"2507.06821","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HeLo fuses physiological and behavioral features with optimal transport and label-correlation-driven attention, reporting the best average rank on DMER and WESAD emotion distribution benchmarks.","lead":"A new neural network architecture, HeLo, combines cross-attention, optimal transport, and learnable label correlations to predict a person's mixture of emotions from physiological and behavioral signals. The authors report state-of-the-art results on two public emotion datasets, DMER and WESAD, in both subject-dependent and subject-independent settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Printed forward pass has undefined tensor shapes (Eqs. 6, 9) and a degenerate ground-truth correlation (Eq. 7), so the reported gains cannot be traced to the described method without a corrected specification or verified code.","rationale":"The reader's weakest assumption identifies the dimensional mismatches in Eqs. (6) and (9) as the key reproducibility risk. I agree that these are the most load-bearing issues because the empirical claim of superiority can only be attributed to the described architecture if that architecture is actually specified. The reader did not flag Eq. (7), where the ground-truth label correlation as written degenerates to a matrix of ones, which makes the correlation-constrained loss meaningless. This reinforces the same conclusion: the text does not define a valid training objective or a valid forward pass. I do not see a more fundamental problem with the experimental design; the reported rank improvements are consistent and the protocol is standard, though error bars are missing. Because the concerns are correctable through a revised manuscript or verified code, the existing CONDITIONAL verdict is appropriate. I would keep the verdict unchanged rather than escalate to rejection, since the released code may well resolve the ambiguity, and the paper's contribution is plausible if the hidden projections are made explicit.","tokens_in":24162,"tokens_out":11561,"duration_ms":109589,"concrete_test":"Clone the released repository (github.com/kaio-99/HeLo) and execute its forward pass on a dummy batch with the dataset-specific feature dimensions (DMER: EEG 90, GSR 28, PPG 27, Video 768; WESAD: ECG 73, EMG 14, EDA 4, ACC 12). Insert shape assertions after Eq. (6) and Eq. (9). If the code runs, record the actual operand order/projections used and compare them to the paper; if it raises a shape error, the paper's method is not implementable as stated. Additionally, print M_gt for one batch from Eq. (7); if it is all ones, the correlation constraint is degenerate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that HeLo, as specified in Sec. 3, achieves the best average rank on DMER and WESAD. This claim can only be evaluated if the forward pass is implementable from the text. It is not. In Eq. (6), x_Phy ∈ R^{2C×d} is multiplied by T ∈ R^{2C×2C} via 'matrix multiplication'; this requires d=2C, which is never stated or implied by the data descriptions (DMER features are 90/28/27/768 and WESAD 12/73/4/14). In Eq. (9), Q_L(K_m)^T is l×C' and M_L is l×l; adding them requires C'=l, yet no projection of the multi-modal features to l tokens is specified. In Eq. (7), M_gt is computed with L ∈ R^{l×1}: for nonnegative label intensities, L_i·L_j/(|L_i||L_j|) = 1 for all i,j, making the correlation-constrained loss vacuous (or undefined when an intensity is zero). These are not typos in one equation; they affect the three core modules. If the released code silently inserts transposes or projections to make shapes work, then the paper's equations are not the method that produced Tables 1-2, and the empirical superiority claim is unattributable. If the code does not insert such corrections, the forward pass cannot be constructed. Either way, the central claim is unsupported by the text as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HeLo, a multi-modal emotion distribution learning framework that fuses physiological signals (EEG, GSR, PPG in DMER; ECG, EMG, EDA in WESAD) with behavioral signals (video in DMER; ACC in WESAD) through three modules: cross-attention-based physiological fusion (CAPF), optimal-transport-based heterogeneity mining (OTHM), and label-correlation-driven cross-attention (LCDCA). The central claim, stated in the abstract and Section 4.4, is that HeLo achieves the best average rank across six distribution metrics on the DMER and WESAD datasets in both subject-dependent and subject-independent protocols, outperforming ten comparison methods including PT-SVM, AA-KNN, SA-CPNN, Conformer, MAET, CARAT, LDL-LRR, TLRLDL, CAD, and EmotionDict. The paper includes ablation studies on components, modalities, attention heads, transformer depth, feature visualizations, and model efficiency. The code is released at a GitHub link.","tokens_in":24595,"tokens_out":7506,"duration_ms":79647,"significance":"If the reported results hold and the architecture is implementable as described, the paper would be a useful contribution to multi-modal emotion distribution learning: it combines optimal transport for modality alignment with label-correlation-guided attention, evaluates on two public datasets under two protocols with six metrics, and provides extensive ablations and qualitative visualizations. The released code is a positive aspect that can support reproducibility. I find no circularity in the derivation: L_CC is an ordinary supervised target computed from ground-truth label distributions, not an assumption baked into the architecture. However, the significance is currently undercut by specification errors in the core equations (Eqs. (6), (7), and (9)), which prevent reproduction of the forward pass from the text alone, and by the absence of any statistical assessment of the reported superiority.","major_comments":[{"comment":"As written, Eq. (6) computes x_Phy ⊗ T, where x_Phy ∈ R^{2C×d} is multiplied on the right by T ∈ R^{2C×2C}. This matrix product is only defined if d = 2C, which is never stated and is inconsistent with the dataset feature dimensions reported in the supplementary material (e.g., 90 EEG, 28 GSR, 27 PPG, and 768 video features, with an embedding size of 128 given in Sec. 4.3). If the intended operation is instead T ⊗ x_Phy, the expression must be written with that order and with the projection from the raw behavioral features to R^{2C×d} explicitly defined. Without this correction, the OTHM module cannot be implemented from the text, and the reported gains cannot be attributed to the described mechanism.","section":"Sec. 3.2, Eq. (6)"},{"comment":"In Eq. (9), Q_L(K_m)^T is an l × C' matrix, where C' is the number of tokens in the projected multi-modal features x_m, while M_L is l × l. Adding these two matrices requires C' = l, but no projection from the multi-modal representation to exactly l tokens is described anywhere in Section 3. The text must specify the token count and the linear projections used to obtain K_m and V_m, otherwise the LCDCA forward pass is not reproducible from the paper.","section":"Sec. 3.3, Eq. (9)"},{"comment":"With L ∈ R^{l×1}, the entries L_i and L_j in Eq. (7) are nonnegative scalar label intensities. For any pair of nonzero intensities, the cosine similarity L_i · L_j / (||L_i|| · ||L_j||) equals 1, and it is undefined when either intensity is zero. The ground-truth correlation matrix M_gt is therefore either the all-ones matrix or undefined, which makes the correlation-constrained loss L_CC vacuous and contradicts the meaningful label correlations displayed in Figure 9 (e.g., 'afraid' correlated with 'nervous' and 'scared'). A meaningful ground-truth label correlation must be defined differently, for example through co-occurrence statistics over training samples or through correlations of full label-distribution vectors.","section":"Sec. 3.3, Eq. (7)"},{"comment":"The empirical claim of superiority rests entirely on point estimates. Tables 1 and 2 report single numbers per metric without standard deviations, confidence intervals, or significance tests. This is especially problematic in the subject-independent DMER results, where the margins over the second-best method are small (e.g., Chebyshev 0.0882 vs. 0.0925, Cosine 0.9148 vs. 0.9036). Moreover, the ablations over attention heads and transformer depth (Figures 6-7) appear to be conducted on the test set, because no validation split is described in Sec. 4.3. Please report variance across subject folds, perform paired significance tests across subjects, and state explicitly how hyperparameters were selected.","section":"Sec. 4.3-4.5, Tables 1-2"}],"minor_comments":[{"comment":"The text says the methods are compared 'on both the DMER and AMIGOS datasets,' but the experiments use DMER and WESAD; AMIGOS is not mentioned anywhere else in the paper.","section":"Sec. 4.4"},{"comment":"The ranks for Conformer (1.2581) and TLRLDL (1.3271) are both printed as 5; the correct ranks are 4 and 5 (or ties must be handled explicitly).","section":"Table 1, DMER Canberra row"},{"comment":"The ranks for TLRLDL (0.8537) and EmotionDict (0.8492) are both printed as 5; the correct ranks are 4 and 5.","section":"Table 1, DMER Intersection row"},{"comment":"The caption states that the learned label correlation M_L is visualized 'in Figure 7,' but the figure is numbered Figure 9.","section":"Sec. 4.6, Figure 9 caption"},{"comment":"The heading 'Muli-modal Emotion Recognition' contains a typo; it should be 'Multi-modal Emotion Recognition.'","section":"Sec. 2.2 heading"},{"comment":"The minimization in Eq. (5) is written as 'min_{f∈Π(u,v)}' but the objective depends on T, not f; the optimization variable should be T to match the text and the definition of Π(u,v).","section":"Sec. 3.2, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The core equations in Section 3 contain shape and definition errors that make the method unimplementable as written. Since the authors release code, these issues may be fixable by correcting equations and adding the missing projections, but the current text does not describe the method that produced Tables 1-2. The reported benchmark results also lack any statistical grounding. I would not recommend acceptance until the specification is corrected and the empirical evaluation is strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible empirical paper with strong results on two EDL benchmarks, but the method as printed cannot be implemented. Eq. (6) multiplies x_Phy (2C×d) by T (2C×2C) without stating d=2C; Eq. (9) adds M_L (l×l) to Q_L K_m^T (l×C') without C'=l; and Eq. (7) computes the ground-truth correlation between scalars L_i and L_j, which gives 1 whenever both are positive and undefined for zeros. That last one makes the correlation-constrained loss degenerate — it pushes all label embeddings to correlate perfectly, not to learn meaningful emotion relationships. These aren't cosmetic typos; they touch the three core modules, so the reported gains can't be traced to the described architecture until the equations are corrected and the released code is verified against them.\n\nWhat's genuinely good: the combination — cross-attention for physiological fusion, Sinkhorn OT for physiological-behavioral alignment, learnable label embeddings with correlation-driven attention — is a sensible search of the EDL design space. The ablation suite is thorough: component removals, per-modality removals, query-modality choice, attention heads, transformer depth, efficiency counts. Results consistently beat EmotionDict, the closest prior multi-modal EDL method, on both datasets and both protocols. The code link is a plus, though I couldn't verify it in the time available.\n\nSoft spots beyond the equations: no error bars or significance tests; hyperparameters are tuned via ablations on the same datasets rather than a held-out split; and a couple of rank ties in Tables 1–2 suggest the tables were assembled with care but should be double-checked. If the corrected code reproduces the numbers, this is a useful subfield contribution. As printed, it's not reproducible from the text.\n\nBottom line: it deserves peer review, but only if the authors provide a corrected specification and working code, and the referee checks that the reported numbers come from that code. I'd read a revised version before citing it.","headline":"Strong EDL results, but the printed forward pass doesn't type-check and the ground-truth correlation in Eq. (7) is degenerate; needs corrected equations and verified code before the claims can be trusted.","tokens_in":25060,"tokens_out":3889,"would_cite":false,"duration_ms":40980,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that cross-attention fusion, optimal-transport alignment of physiological and behavioral features, and label-correlation-driven attention yield the best average rank across six emotion metrics on DMER and WESAD.","keywords":["emotion distribution learning","multi-modal fusion","optimal transport","cross-attention","label correlation","physiological signals","emotion recognition"],"falsifier":"A reader could settle reproducibility by implementing the forward pass exactly as printed: multiplying the physiological representation of shape $2C \\times d$ by the transport matrix $T$ of shape $2C \\times 2C$ fails unless $d = 2C$ or a projection is inserted, and adding $M_L$ of shape $l \\times l$ to $Q_L K_m^T$ of shape $l \\times C'$ fails unless $C' = l$; if the published equations cannot be executed without hidden transposes or projections, the central empirical claim is not reproducible from the text. Running the released code and comparing its tensor shapes and its subject-independent WESAD average rank (reported as $1.16$) against the tables would settle whether the results correspond to the described architecture.","tokens_in":24012,"feed_emoji":"🎭","tokens_out":15872,"duration_ms":134476,"temperature":0.7,"pith_summary":"Emotions rarely arrive one at a time: a person can feel nervous and afraid together, with each feeling at its own intensity. This paper is trying to establish that a learned model can predict such mixed emotion distributions better when it does three things at once: fuse the physiological signals (EEG, GSR, and PPG on DMER; ECG, EMG, and EDA on WESAD) using cross-attention; align the fused physiological representation with the behavioral data (facial video or accelerometer) through an optimal-transport matching flow; and use a learnable label embedding, whose correlation matrix is regularized toward the true label-correlation structure, to steer a final cross-attention layer. The claim is that this combination, called HeLo, attains the best average rank over six distribution-quality measures on two public datasets, DMER and WESAD, in both subject-dependent and subject-independent protocols. Accurate mixed-emotion prediction matters because it lets human-computer interaction respond to the mixture and intensity of real emotional experience rather than picking one dominant label.","feed_headline":"HeLo outranks 10 baselines in mixed-emotion learning","feed_subtitle":"Fusing EEG, GSR and PPG with facial video and label correlations gives the best average ranking on DMER and WESAD.","key_machinery":"The load-bearing objects are three. (1) Cross-attention physiological fusion (CAPF): with EEG as the query and GSR and PPG as key–value pairs, multi-head cross-attention with residual connections produces the fused physiological representation, and making EEG the query lets the neural signal anchor the fusion. (2) The optimal-transport heterogeneity mining module (OTHM): a discrete Kantorovich optimal-transport problem defines the Wasserstein distance between the physiological and behavioral representations, and its optimal matching flow $T \\in \\mathbb{R}^{2C \\times 2C}$, solved by a batch-wise Sinkhorn algorithm, is multiplied with the physiological representation before a transformer encoder, so the transport plan itself acts as a cross-modal correlation map that reduces distributional disparity. (3) The label correlation-driven cross-attention (LCDCA): a learnable label embedding $x_L \\in \\mathbb{R}^{l \\times d}$ yields a learnable label-correlation matrix $M_L$ via cosine similarity, pulled toward the ground-truth label-correlation matrix $M_{gt}$ by an $\\ell^2$ penalty; $M_L$ is then added inside the softmax logits of a cross-attention layer whose queries come from the label embedding and whose keys and values come from the multi-modal features, so semantic correlations among emotions directly shape the attended representation.","core_discovery":"On the paper's own terms, the discovery is that explicit handling of two kinds of structure — cross-modal heterogeneity and cross-label correlation — pays off in emotion distribution learning. The authors propose HeLo, in which EEG is used as the query modality to attend over GSR and PPG features, giving a fused physiological representation $x_{Phy}$; a discrete optimal-transport matching flow $T \\in \\mathbb{R}^{2C \\times 2C}$, computed by a batch-wise Sinkhorn solver of the Kantorovich formulation, then aligns $x_{Phy}$ with the behavioral representation $x_v$, and the transported features concatenated with the transformer-encoded behavioral features form the multi-modal representation. A learnable label embedding $x_L$ produces a label-correlation matrix $M_L$, trained by an $\\ell^2$ loss toward the ground-truth correlation matrix $M_{gt}$ computed from the label distributions, and the final cross-attention adds $M_L$ into the attention logits, $x_o = \\mathrm{softmax}((Q_L K_m^T + M_L)/\\sqrt{d}) V_m$, so that attention is steered by the semantic relationships among emotions. The paper reports that this model records the best average rank across the six metrics on both datasets in both protocols (best on all six metrics except KL on DMER under subject dependence and KL on WESAD under subject independence), and that its ablations show every module and every modality contributes positively.","pith_inferences":["Editorial inference: because the optimal-transport module is stated generically over two representation sets, the same align-then-fuse pattern could transfer to other heterogeneous modality pairs, such as text with images or audio with physiology, wherever a transport plan between two feature clouds is computable.","Editorial inference: the label-correlation matrix is added directly into the attention logits, effectively acting as a differentiable prior over label semantics, so the same mechanism could be grafted onto any label-distribution prediction head, not just the emotion setting.","Editorial inference: the paper's learned correlation matrices vary across subjects and trials, which suggests treating the correlation matrix as a subject descriptor for personalization, or as a signal for detecting trials where the emotion labels are ambiguous."],"forward_implications":["If HeLo's results hold, emotion distribution learning can treat label correlations as an explicit inductive bias: the learned correlation matrices cluster 'afraid' with 'nervous' and 'scared' in the paper's visualization, matching human intuition about which emotions co-occur.","The optimal-transport matching flow works as an align-then-fuse operator for physiology and behavior, which suggests transport plans can serve as interpretable cross-modal correlation maps rather than only as distance penalties inside a loss.","The paper reads its subject-independent (leave-one-subject-out) results as evidence that the aligned features carry partly subject-invariant information, which matters for deploying emotion models on users never seen during training.","The ablation studies imply that no modality and no module is redundant: removing any of them degrades all six distribution metrics, so the reported gains come from the full composition of the three modules.","The reported efficiency numbers (about 11.7 million FLOPs and 4.0 million parameters) place HeLo below three of the four deep baselines it is compared against, so the claimed accuracy does not obviously come at high compute cost."],"supporting_citations":[{"why":"Supplies the DMER dataset (EEG, GSR, PPG, and facial video with PANAS-based emotion distributions) on which the central comparisons are run.","marker":"[40]"},{"why":"Supplies the WESAD dataset (ECG, EMG, EDA, and accelerometer) used for the second set of comparisons.","marker":"[28]"},{"why":"EmotionDict is the strongest multi-modal emotion-distribution baseline the method must beat, and its average-rank reporting sets the evaluation convention HeLo follows.","marker":"[23]"},{"why":"Defines the six distribution metrics and the subject-dependent and subject-independent protocols that the experimental comparison adopts.","marker":"[30]"},{"why":"Establishes label distribution learning and supplies two traditional LDL baselines (PT-SVM and AA-KNN) together with the evaluation metrics.","marker":"[7]"},{"why":"CARAT is the multi-modal multi-label baseline that motivates the modality-to-label dependency design which the label-correlation stage extends.","marker":"[26]"},{"why":"Provides the batch-wise Sinkhorn solver that the optimal-transport heterogeneity mining module uses to compute the matching flow.","marker":"[4]"},{"why":"Gives the discrete Kantorovich formulation through which the Wasserstein distance and transport plan between modality representations are defined.","marker":"[16]"}],"fun_headline_variants":["HeLo fuses modalities and label correlations for better emotion learning","HeLo exploits optimal transport and label correlation for mixed-emotion learning","HeLo: Top average rank on DMER and WESAD with heterogeneous fusion","HeLo: Label correlation improves multi-modal emotion distribution","HeLo: Heterogeneous fusion plus label correlation outranks baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"That the architecture actually runs as described: the printed formulas leave the tensor dimensions of the transport multiplication and of the label-correlation addition unspecified, so the entire reported comparison rests on an unstated projection or transpose that a reader cannot verify from the equations alone.","fun_headline_variants_meta":{"raw":{"variants":["HeLo fuses modalities and label correlations for better emotion learning","HeLo exploits optimal transport and label correlation for mixed-emotion learning","HeLo: Top average rank on DMER and WESAD with heterogeneous fusion","HeLo: Label correlation improves multi-modal emotion distribution","HeLo: Heterogeneous fusion plus label correlation outranks baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001442,"raw_usage":{"total_tokens":5876,"prompt_tokens":1078,"completion_tokens":4798,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":694,"completion_tokens_details":{"reasoning_tokens":4707}},"tokens_in":694,"tokens_out":4798,"duration_ms":56337,"temperature":1.0,"reasoning_tokens":4707,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:53:48.369393+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle reproducibility by implementing the forward pass exactly as printed: multiplying the physiological representation of shape $2C \\times d$ by the transport matrix $T$ of shape $2C \\times 2C$ fails unless $d = 2C$ or a projection is inserted, and adding $M_L$ of shape $l \\times l$ to $Q_L K_m^T$ of shape $l \\times C'$ fails unless $C' = l$; if the published equations cannot be executed without hidden transposes or projections, the central empirical claim is not reproducible from the text. Running the released code and comparing its tensor shapes and its subject-independent WESAD average rank (reported as $1.16$) against the tables would settle whether the results correspond to the described architecture.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DMER dataset (EEG, GSR, PPG, and facial video with PANAS-based emotion distributions) on which the central comparisons are run."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the WESAD dataset (ECG, EMG, EDA, and accelerometer) used for the second set of comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"EmotionDict is the strongest multi-modal emotion-distribution baseline the method must beat, and its average-rank reporting sets the evaluation convention HeLo follows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the six distribution metrics and the subject-dependent and subject-independent protocols that the experimental comparison adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes label distribution learning and supplies two traditional LDL baselines (PT-SVM and AA-KNN) together with the evaluation metrics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CARAT is the multi-modal multi-label baseline that motivates the modality-to-label dependency design which the label-correlation stage extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the batch-wise Sinkhorn solver that the optimal-transport heterogeneity mining module uses to compute the matching flow."}],"review_version":1}