{"id":"3d4b65cc-ee82-4a35-aac9-4832c8636254","arxiv_id":"2601.00181","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Using only past turns, ERC accuracy saturates within 10–30 preceding utterances; hierarchical encoding and SenticNet add little once context is present, and Sad turns show the largest context benefit and fewer left-periphery markers.","lead":"An NLP study tests which design choices actually matter for emotion recognition in conversation (ERC), on the IEMOCAP corpus, under a strictly past-only (causal) setup with 10-seed statistics. It finds that conversational context dominates accuracy but saturates within roughly 10–30 prior turns, that hierarchical encoders and an affective lexicon add little once context is used, and that sad utterances are the most context-dependent and show reduced left-periphery discourse m","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported headline F1 likely inflated by per-seed best-K selection; needs a fixed-K replication","rationale":"The reader's weakest assumption (per-seed best-K selection inflates headline numbers) is precisely the load-bearing concern I identify. Table 2's fixed-K values peak around 80.6 for 4-way, while the headline is 82.69; Appendix B.2 reveals that 82.69 is the mean over seeds of each seed's best K. This is a textbook selection-on-test-estimate bias. Because the paper's abstract and §3.9/§4 use these numbers to claim competitiveness with (and in some cases superiority to) bidirectional systems, the central claim depends on this selection procedure being honest. The paper's own disclaimer that the sweep is for analysis rather than model selection is undercut by Appendix B.2 using 'best K per seed' for its primary pooling comparison and by the main text reporting these sweep-selected numbers as the model's headline results. If the true fixed-K operating point is ~80.5, the 82.69 vs 81.54/81.4 comparison against HFFN/HCAM no longer shows an advantage. The reader's other concerns (missing MELD experiments contradicting abstract, missing tables, DM count discrepancy) are also valid but secondary: the MELD claim is in the abstract only and could be removed without changing the IEMOCAP saturation finding; missing tables are presentation issues; the DM count inconsistency affects a supporting analysis. The correct verdict remains CONDITIONAL: the saturation methodology and fixed-K Table 2 are solid, but the headline claim needs either a validation-based fixed-K report or a clear honesty caveat that the reported numbers are sweep-maxima, not deployable operating points. I would not move to REJECT because the central saturation finding is independently supported by the fixed-K Table 2, which is not affected by selection bias; only the headline comparison is.","tokens_in":14460,"tokens_out":1999,"duration_ms":18179,"concrete_test":"Rerun or re-analyze the IEMOCAP sweep under a single fixed-K rule that is selected on validation (e.g., choose K once on Session 1 validation, then evaluate on Session 5 test for all seeds), and report mean±std over seeds at that K. If the resulting 4-way WF1 is within 0.5 points of 82.69, the headline survives; if it lands near 80.5–81.0 (consistent with Table 2 fixed-K values), the 'competitive with bidirectional systems' claim overstates the causal model's operating-point performance. Also recompute the DM association with an explicit subsetting rule that reconciles the 5,286 vs 8,955 counts.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that simple past-only models are competitive with bidirectional systems rests on the headline numbers 82.69% 4-way / 67.07% 6-way WF1. Per Appendix B.2, these are means over seeds of each seed's best K from the sweep over K∈{0..200}, i.e., 10 separate selections. This is a selection-on-test-estimate procedure: even with no true context effect, each seed's maximum over ~200 correlated configurations is biased upward relative to any fixed-K operating point, and the variance of the reported mean is understated. The abstract and §3.9 present these numbers against fixed-configuration baselines (HCAM 81.4, DialogueRNN 76.2, DAG-ERC 68.03) without multiplicity correction. Since the paper's own Table 2 shows fixed-K 4-way FLAT mean peaking around 80.6 (K=30/50) and K=200 dropping to 79.59, the 82.69 headline is about 2 points above the best fixed-K value in the table, well beyond the ±0.50 seed std reported in B.2. This gap is exactly the magnitude that would make the difference versus HCAM/DAG-ERC flip. The paper's claim that the sweep is 'used strictly for analysis' (footnote 1) is contradicted by Appendix B.2 reporting turn-level pooling comparisons at 'best K per seed' and by the main text using these numbers for the headline comparison. A second, separate concern is a numerical inconsistency in the DM analysis: the abstract/main text say 5,286 DM occurrences, but Appendix A Table 5 totals 8,955; no subsetting rule is given. This does not directly threaten the recognition claim but undermines the DM finding's reproducibility. The most load-bearing issue is the selection bias in the headline F1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies text-only emotion recognition in conversation (ERC) on IEMOCAP under a strictly causal (past-only) setting. Using Sentence-RoBERTa features, it sweeps the number of preceding turns K, compares flat versus hierarchical utterance encoders, pooling variants, and SenticNet fusion across 10 seeds, and reports three recognition findings: conversational context dominates and saturates quickly; hierarchical encoding helps only without context; SenticNet provides no improvement. It also reports headline weighted F1 scores of 82.69% (4-way) and 67.07% (6-way), and a complementary discourse-marker analysis of 5,286 occurrences reporting emotion-specific left-periphery patterns, especially reduced left-periphery marker usage in Sad utterances. The abstract additionally claims cross-dataset validation on MELD.","tokens_in":14723,"tokens_out":7085,"duration_ms":66104,"significance":"If the central claims hold, the paper gives practical guidance for causal ERC: simple flat encoders with a modest past-only window can capture most of the available signal, and discourse-marker position is interpretably associated with emotion. The paper has genuine strengths: 10-seed evaluations with paired significance tests, a speaker-disjoint split, and a fixed-K evaluation table (Table 2) that enables like-for-like comparisons. The discourse-marker analysis is theory-grounded and uses a defined marker inventory. However, the headline performance numbers are not fixed-K operating points but per-seed best-K maxima from a sweep, so the 'strong performance' and cross-dataset claims are not supported as stated. The fixed-K saturation finding remains valuable if reported honestly.","major_comments":[{"comment":"The abstract and §3.9 report 82.69% (4-way) and 67.07% (6-way) as the central performance claim. Appendix B.2 identifies these as 'Turn-level (best K per seed)', i.e., for each random seed the maximum over the K sweep is selected before averaging. This is a selection-on-test estimate: with K∈{0,...,200}, the per-seed maximum is biased upward relative to any fixed-K operating point, and the quoted seed standard deviation understates the variability of the selected maximum. The paper's own fixed-K Table 2 peaks at 80.66 (4-way, K=50) and 64.77 (6-way, K=30), about 2.0 and 2.3 points below the headline. Because the baselines in Tables 3–4 are fixed-configuration numbers, the headline comparison is not like-for-like and can flip conclusions (e.g., HCAM 81.4 vs. the fixed-K 4-way result 80.66). Please report the fixed-K comparison as the headline, or make explicit that the per-seed best-K val","section":"§3.9, Appendix B.2"},{"comment":"The abstract states that the study is conducted 'with cross-dataset validation on MELD' and that hierarchical sentence representations 'show a clear advantage on MELD'. No MELD experiment, table, or result appears anywhere in the body; MELD is mentioned only as future work in §5. A cross-dataset generalization claim with no supporting evidence is not assessable. Either add the MELD experiments and report their results, or remove/caveat all MELD claims from the abstract and from the stated findings.","section":"Abstract, §1.1, §3.5"},{"comment":"The interaction claim at the start of §4.1 — 'In the best-performing configurations at K=0, hierarchical encoding exceeds flat encoding' — is contradicted by the fixed-K results in Table 2. At K=0, 4-way FLAT(mean) is 64.94±0.77 versus HIER(mean) 64.52±1.01; with wmean_pos the values are 64.80±0.87 versus 64.03±1.15. The 6-way row likewise favors flat (52.35±1.36 vs. 51.54±1.04). The missing 'Table??' referenced in §4.1 does not resolve this. Unless the claim refers to an unreported configuration or a different metric, the direction is opposite to what is stated. Please reconcile or correct the RQ2 finding and the abstract's corresponding claim.","section":"§4.1 and Table 2"},{"comment":"The text repeatedly states that the discourse-marker analysis used 5,286 occurrences, but Table 5 lists markers whose counts sum to 8,955. No inclusion/exclusion criterion is given to explain the subset. This discrepancy propagates into all reported proportions (e.g., 21.9% left-periphery for Sad), chi-square tests, and mixed-effects models. Specify the filtering rule and report counts consistently in the text, tables, and abstract.","section":"§3.2, Abstract, Appendix A Table 5"}],"minor_comments":[{"comment":"There are unresolved 'Table??' cross-references, including the table of best sweep configurations referenced in §3.6 and the K=0 hierarchical comparison referenced in §4.1. These should be completed or removed.","section":"§3.6, §4.1"},{"comment":"Several cells report a mean without a standard deviation (e.g., 4-way FLAT K=200: 79.59; 6-way HIER wmean_pos_rev K=200: 62.49). If these are single-seed runs or partial results, say so explicitly; otherwise report the full mean±std.","section":"Table 2"},{"comment":"Mixed-effects models are listed among the DM statistical tests, but no mixed-effects estimates are reported; the results shown are ANOVA, chi-square, and post-hoc comparisons. Either add the mixed-model results or remove that claim.","section":"§3.2"},{"comment":"The saturation statement ('roughly 90% of the gain...') depends on the definition of the sweep range and on which K values were actually evaluated. State the full set of swept K values and confirm whether saturation is computed on evaluated points or interpolated, so the 90% figure is reproducible.","section":"§3.7"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid fixed-K ablation core and a genuinely useful saturation analysis, but the current presentation overstates the headline and contains internal contradictions between the text and Table 2, as well as an unsupported MELD claim. I believe the fixed-K findings are salvageable and the discourse-marker analysis could be valuable, but the selection issue, the K=0 hierarchical claim, and the MELD statements must be resolved before the paper can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core of this paper is the controlled ablation work on IEMOCAP: 10 seeds, paired tests, fixed hyperparameters, strictly past-only context, and a clean sweep over K. That part is done carefully, and the main findings hold up on reading. Context saturates quickly—most of the gain is in the last 10–30 turns, and the aggregate curve in Table 2 supports that. The emotion-specific pattern (Sad gains far more from context than Angry) is also visible in the per-class analysis, and the lexical fusion null result is credible. The discourse-marker analysis is a nice addition, and the Sad left-periphery finding is a real, if modest, empirical observation.\n\nThe soft spots are real but concentrated. The headline numbers (82.69/67.07) come from taking each seed's best K from a sweep over K∈{0..200}, then averaging those per-seed maxima. That is selection on the test estimate, and it biases the reported mean upward relative to any fixed-K operating point. Table 2 shows the best fixed-K 4-way number is around 80.6–80.7, so the 2-point gap is exactly the kind of inflation that could flip the comparison against HCAM or DAG-ERC. The paper's footnote says the sweep is for analysis only, but Appendix B.2 explicitly reports 'best K per seed' as the headline configuration. That tension needs to be resolved—either report a prespecified K or correct for the selection. This is the load-bearing issue, and it is fixable: give a fixed-K result (say K=30 or K=50) as the headline and treat the sweep as descriptive.\n\nSecond, the abstract claims a 'clear advantage on MELD,' but there is no MELD experiment in the body. MELD appears only in the limitations section as future work. That claim has to be removed or the experiment added.\n\nSmaller problems: the DM count discrepancy (5,286 in the abstract vs. 8,955 in the inventory table) with no stated subsetting rule; the 'Table??' placeholders in §3.6 and §4.1; and §4.1's claim that hierarchical beats flat at K=0 contradicts Table 2, where FLAT is 64.94 and HIER is 64.52. These are minor but should be cleaned up.\n\nThe DM analysis is a correlation, and the causal story about Sad and left-periphery markers is speculative. The paper itself mostly says this is a hypothesis, so that is acceptable.\n\nWho this is for: ERC practitioners who want a design rule (short past-only window, flat encoder, no lexicon) and anyone working on discourse markers and emotion. The methods section is a good model for how to run controlled ablations. I would send it to peer review, with the explicit caveat that the headline numbers need a fixed-K replication and the MELD claim needs to be addressed. It is a solid paper with two fixable problems, not a rejection.","headline":"Solid ablation study with a real saturation finding, but the headline F1 numbers are likely inflated by per-seed best-K selection and the MELD claim is unsupported in the text.","tokens_in":15382,"tokens_out":745,"would_cite":true,"duration_ms":9396,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Conversational context dominates emotion recognition, but its benefit saturates within the most recent 10–30 preceding turns; under strictly past-only access, a simple flat encoder reaches 82.69% 4-way and 67.07% 6-way weighted F1 on IEMOCA","keywords":["emotion recognition in conversation","past-only causal context","context saturation","IEMOCAP","discourse markers","left periphery","flat vs hierarchical encoding","affective lexicon"],"falsifier":"Compute the mean weighted F1 across the 10 seeds for a single fixed context length (say K=30) chosen once on the validation set, and compare it to the reported 82.69% (4-way) and 67.07% (6-way) and to the bidirectional baselines. If the fixed-K number drops materially or falls below those baselines, the 'competitive without future turns' claim is falsified. Separately, verify whether any MELD experiment exists in the paper or supplement to support the abstract's MELD claim.","tokens_in":14161,"feed_emoji":"🗨️","tokens_out":11848,"duration_ms":107661,"temperature":0.7,"pith_summary":"This paper asks which modeling choices actually matter for emotion recognition in conversation (ERC) and whether gains come from conversational history or from architecture. Under a strictly causal (past-only) setting, it claims that conversational context is the dominant factor and that performance saturates quickly: roughly 90% of the attainable gain is captured within the most recent 10–30 preceding turns. The paper further claims that simple models—a flat utterance encoder plus a recurrent context layer—reach 82.69% weighted F1 on 4-way IEMOCAP and 67.07% on 6-way, competitive with or better than reported bidirectional systems that use future turns. It also reports a small but reliable linguistic association: sad utterances use fewer left-periphery discourse markers, which aligns with sadness benefiting most from added context. If correct, this would simplify real-time ERC systems and connect recognition accuracy to interpretable discourse cues for emotion-conditioned generation.","feed_headline":"A 10-30 turn past window captures most emotion-recognition gains","feed_subtitle":"Simple past-only model hits 82.69% 4-way F1, matching bidirectional systems on IEMOCAP.","key_machinery":"The key machinery is the controlled context-length sweep: K, the number of strictly preceding turns, is varied from 0 to 200, and saturation is defined as the smallest K at which 90% of the maximum improvement within the sweep is reached. Utterances are embedded once by a fixed pretrained sentence encoder (either flat whole-utterance or hierarchical sentence-then-aggregate), and a unidirectional recurrent layer processes the K preceding embeddings to produce the prediction. The discourse-marker analysis operationalizes 'left periphery' as normalized position < 0.15 within the utterance, 'right periphery' as > 0.85, and links these to emotion labels via chi-square and mixed-effects tests.","core_discovery":"The central discovery is the steep and early saturation of conversational context: sweeping K from 0 to 200 strictly preceding turns shows that the large majority of the performance gain is already realized with the 10 to 30 immediately preceding turns, depending on the label set. Under strictly past-only access, the paper's simple models—a flat pooled sentence encoder feeding a single-layer recurrent context encoder—reach 82.69% weighted F1 on 4-way IEMOCAP and 67.07% on 6-way, matching or exceeding several bidirectional systems. Hierarchical sentence-level encoding helps only when no context is present (K=0) and its benefit disappears once turn-level context is available; an external affec","pith_inferences":["A natural extension is to test whether the 10–30 turn saturation transfers to other corpora; if it does, long-context designs in ERC could be replaced by short windows with little loss.","The Sad–left-periphery link suggests a concrete generation experiment: deleting left-periphery markers from otherwise neutral utterances and testing whether human raters perceive increased sadness would validate the production cue.","Because the abstract's MELD advantage is not reported in the body, a direct replication of the IEMOCAP ablation on MELD is needed before the claimed cross-dataset generalization can be accepted."],"forward_implications":["Real-time ERC systems can use a short past-only window (10–30 turns) with a flat encoder and match or exceed more complex bidirectional systems, eliminating the need for future-turn access.","Hierarchical sentence modeling is unnecessary once context is available; simpler flat encoding suffices, reducing computational cost and engineering complexity.","Adding an external affective lexicon to modern pretrained encoders does not help ERC, suggesting such resources are redundant when strong pretrained representations are used.","Emotion-specific context dependence (Sad gains ~22 percentage points, Angry only ~8–9) implies models and evaluation should report per-class context sensitivity; generation systems can use reduced left-periphery discourse markers as a style cue for sadness.","The weak identifiability of the 6-way taxonomy in text-only settings (e.g., Happy–Excited confusion) suggests that fine-grained label sets may need prosodic inputs or a revised label hierarchy."],"fun_headline_variants":["Past 30 turns capture 90% of emotion recognition gains","Simple causal model matches bidirectional ERC on IEMOCAP","Sadness depends more on conversational context than other emotions","Turn-level context eliminates need for hierarchical utterance encoding","External affective lexicon adds nothing to pretrained ERC encoders"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The reported headline accuracies are the mean over seeds of each seed's best context length from a sweep over K in 0..200; the claim that a simple past-only model is competitive assumes this per-seed best-K selection does not inflate the mean relative to a fixed-K operating point.","fun_headline_variants_meta":{"raw":{"variants":["Past 30 turns capture 90% of emotion recognition gains","Simple causal model matches bidirectional ERC on IEMOCAP","Sadness depends more on conversational context than other emotions","Turn-level context eliminates need for hierarchical utterance encoding","External affective lexicon adds nothing to pretrained ERC encoders"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1451,"prompt_tokens":854,"completion_tokens":597,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":516}},"tokens_in":598,"tokens_out":597,"duration_ms":6045,"temperature":1.0,"reasoning_tokens":516,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T13:08:25.462834+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the mean weighted F1 across the 10 seeds for a single fixed context length (say K=30) chosen once on the validation set, and compare it to the reported 82.69% (4-way) and 67.07% (6-way) and to the bidirectional baselines. If the fixed-K number drops materially or falls below those baselines, the 'competitive without future turns' claim is falsified. Separately, verify whether any MELD experiment exists in the paper or supplement to support the abstract's MELD claim.","supporting_citations":[],"review_version":1}