{"id":"50f37ec4-8ad6-47c0-98f5-3a67832a89da","arxiv_id":"2607.26967","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Pair-level judgement consistently outperforms dialogue-level generation for LLM-based ECPEC because models recognize pairs under explicit queries but fail at set-level discovery and shared-threshold decisions.","lead":"Pair-level yes/no judgement beats full-dialogue generation for LLM emotion-cause pair extraction in every controlled comparison the authors ran. The gap is mostly about forcing the model to invent complete pair sets at once, not about missing the relations themselves.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"Unequal LoRA capacity and schedules across paradigms remain the main threat to attributing the JUD>GEN gap purely to task formulation.","rationale":"The paper’s strongest evidence is broad and concrete: JUD>GEN across 18 settings (Table 2), high recoverable-omission rates (Figure 3), and a large same-checkpoint recall jump when candidate-set prediction is split into pair queries (Figure 4, final step). Those results make a real methodological contribution. The load-bearing vulnerability is exactly the one the reader named—paradigm-specific LoRA hyperparameters in Table 11—so the gap and the bottleneck diagnosis are not yet cleanly isolated from optimization/capacity. §4.2 helps on mechanism but does not fully close the hole, because GEN is an untrained probe on a gold-target joint checkpoint. No stronger internal inconsistency (e.g., metric leakage or contradictory tables) is apparent; single-seed reporting is a secondary weakness already noted. Reader’s CONDITIONAL verdict with medium correctness risk is therefore appropriate; this pass does not move it. Matched-budget multi-seed reruns are the decisive check.","tokens_in":18307,"tokens_out":699,"duration_ms":38974,"concrete_test":"Re-fine-tune GEN, SEL, and JUD on LLaMA-3.1-8B (and one other backbone) with identical LoRA r/α/dropout, the same number of epochs or matched optimizer-step and token budgets, and the same seed set (e.g., 3 seeds). Recompute Table 1 local/full F1 and the 18 ΔF1 cells of Table 2. If any JUD−GEN gap falls below ~1–2 F1 or loses consistent sign, or if §4.2’s set→pair recall jump shrinks sharply under a GEN-inclusive joint tune, the pure task-formulation claim and set-level-bottleneck diagnosis weaken.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that task formulation (not model capacity or optimization) drives the paradigm gap: JUD beats GEN in all 18 backbone–input–scope comparisons, and GEN’s misses are a set-level discovery/return bottleneck rather than a recognition failure (§3–§4; 92.7%–98.1% pair recall under explicit queries). That attribution requires the controlled paradigms to differ only in inference granularity. Appendix B.4 / Table 11 instead assigns JUD higher LoRA rank/α (often r=16, α=32, dropout 0.05, 2 epochs) than GEN/SEL (r=8, α=16, dropout 0, 6 epochs) on the same backbones. Effective adapter capacity and optimization trajectory are therefore confounded with formulation. If GEN is under-fit or capacity-limited relative to JUD, both the 18-way gap and the §4 “omissions arise during set-level prediction” story are partly training artifacts. The same-checkpoint progressive probe in §4.2 partially isolates query format, but dialogue-level GEN is only an “additional probe” on a model jointly tuned on gold-target formats, so the early steps of that ladder still mix format mismatch with true set-prediction difficulty. Matched-budget re-training is required before the pure-formulation claim is secure.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper argues that how ECPEC is posed to LLMs—dialogue-level generation (GEN), target-level selection (SEL), or pair-level judgement (JUD)—substantially changes performance. Under controlled fine-tuning, JUD beats GEN in all 18 backbone–input–scope comparisons on ECF, ConvECPE, and RECCON. Progressive probes show that under explicit pair queries the same backbone recovers 92.7%–98.1% of gold pairs, so GEN’s misses are attributed mainly to set-level discovery and return rather than inability to recognize relations. JUD’s rankings are more reliable than a shared Yes/No threshold (ISR ≫ RSR); a lightweight auxiliary retriever that re-checks the top rejected boundary candidate then yields +0.50–1.46 F1 at 1.49× GEN-Full inference cost. The authors conclude that task decomposition and candidate scope are critical for LLM-based ECPEC.","tokens_in":18745,"tokens_out":1333,"duration_ms":29102,"significance":"If the paradigm gap is truly driven by formulation rather than training confounds, the paper gives a clear, actionable account of how to use LLMs for ECPEC: prefer pair-level queries, keep a local candidate window, and treat shared thresholds as the remaining bottleneck. The multi-dataset, multi-backbone, multimodal, and local/full-scope design, plus the ROR/ISR/RSR diagnostics and efficiency table, are concrete contributions that go beyond another leaderboard entry. The progressive decomposition in §4.2 and the ranking-vs-threshold taxonomy in §5.2 are especially useful for the broader community working on structured extraction with LLMs. Credit is due for reporting inference cost carefully (Table 8) and for a diagnosis-guided remedy rather than an ad-hoc system.","major_comments":[{"comment":"Appendix B.4 / Table 11 assigns unequal LoRA capacity and schedules by paradigm (JUD often r=16, α=32, dropout 0.05, 2 epochs vs GEN/SEL r=8, α=16, dropout 0, 6 epochs). The central claim that task formulation alone drives the 18-way JUD>GEN gap (§3, abstract) therefore confounds granularity with effective adapter capacity and optimization trajectory. Matched-budget re-training (same r/α/dropout/epochs, or equal trainable parameters and wall-clock) on at least one backbone and one dataset is needed before the pure-formulation attribution is secure. Without it, both the gap and the §4 “set-level bottleneck” diagnosis remain partly training artifacts.","section":"Appendix B.4 / Table 11; §3"},{"comment":"§4.2’s progressive ladder is the strongest internal evidence that set-level prediction is the dominant omission source, but dialogue-level GEN is only an “additional probe” on a checkpoint jointly fine-tuned on the three gold-target formats. Early steps of the ladder therefore mix true set-prediction difficulty with format mismatch relative to the training mixture. Please either (i) report the same ladder on a GEN-only fine-tuned checkpoint, or (ii) clearly qualify that the 30.83–55.04 point jump isolates pair vs set prediction only under the joint gold-target training regime, not under the GEN training used in Table 1.","section":"§4.2; Figure 4"},{"comment":"All main results are single-run with seed 13 (Appendix B.4). On ConvECPE the paradigm spread is 18–28 F1 points, so the qualitative ordering is unlikely to reverse, but the remedy gains of +0.50–1.46 F1 (Table 7) and several close GEN/SEL/JUD comparisons in Table 1 sit inside typical run-to-run noise for LoRA fine-tunes. At minimum, report 3 seeds for JUD-Local vs GEN-Local on one dataset and for the retriever delta; otherwise the small remedy claim is under-supported relative to the paper’s emphasis on it.","section":"Table 1; Table 7; Appendix B.4"}],"minor_comments":[{"comment":"Table 1: JUD-Full on ConvECPE has very high recall (79.48) and low precision (33.27), yielding F1 below SEL-Full; a one-sentence discussion of when full-scope JUD hurts would help readers choose configurations.","section":"Table 1; §5.2"},{"comment":"Figure 1(b) is referenced as showing unrealized separation potential but is hard to read without axis labels or a short caption definition of the shared boundary; align the caption with the ISR/RSR definitions in §5.2.","section":"Figure 1"},{"comment":"Local window size 4 is justified by Table 10 (87–94% of gold pairs) but is never ablated (e.g., window 2 vs 4 vs 6). A short ablation would strengthen the “controlled candidate scope” recommendation.","section":"§3.1; Table 10"},{"comment":"Prompts in Appendix A use [EMOTION LABEL SET] and [SCOPE INSTRUCTION] placeholders; stating the exact label inventories per dataset would aid reproducibility.","section":"Appendix A"},{"comment":"Typos/style: “JUDtheoretically” (missing space, §2); “we choice the RECCON-DD” → “choose” (B.1); occasional missing spaces before citations.","section":"§2; Appendix B.1"}],"recommendation":"major_revision","confidential_remarks":"The skeptic’s LoRA-capacity concern is load-bearing and correctly located; I would not accept without matched-budget controls or a clear qualification that weakens the “pure formulation” claim. Novelty is solid as a controlled paradigm study rather than another agent system. Fit for a serious CL venue is good if the training confound is fixed. No integrity red flags."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing: under matched backbones and scopes, pair-level yes/no consistently beats dialogue-level generation on ECPEC (all 18 backbone–input–scope cells on ECF, plus the three-dataset table). That is not a huge conceptual leap—prior LLM-ECPEC already mixed generation, target selection, and pair checks—but the controlled distillation into GEN/SEL/JUD, the progressive-decomposition ladder, and the ranking-vs-threshold breakdown are new and cleanly reported.\n\nWhat they do well is the diagnosis. Many pairs GEN drops are still recoverable under explicit queries; same-checkpoint probes push pair recall to ~93–98% once you stop asking for a full set at once. GEN omissions worsen with more gold pairs, multi-cause targets, and distance. JUD’s rankings look better than its shared 0.5 Yes-threshold (ISR vs RSR gap is the right framing). Local scope is the practical sweet spot; the small boundary retriever (+0.5–1.5 F1 at 1.49× GEN time) is a modest, honest fix rather than a claim of solving the problem. Baselines, efficiency table, and multimodal checks are adequate for a methods paper. Citations cover the recent LLM-ECPEC line without obvious gaps.\n\nSoft spots, in proportion: single seed (13), no error bars. More importantly, Table 11 gives JUD higher LoRA rank/α (and different epochs/dropout) than GEN/SEL on the main backbones. That confounds “task formulation” with capacity and schedule. The §4.2 same-checkpoint ladder partially isolates query format, but dialogue GEN is only an extra probe on a model tuned for gold-target formats, so the early steps still mix mismatch with true set difficulty. Matched-budget re-runs would lock the claim. Remedy gains are small and one-candidate-local by design; free choices (window=4, threshold=0.5) are reasonable but fixed.\n\nCentral argument still holds directionally: decomposition and candidate scope matter more than people were treating them. This is for people doing LLM structured extraction in dialogue, not a general reasoning paper. I’d bring it to reading group, cite the paradigm gap and ISR/RSR framing if I work nearby, and send it to referees—ask them to demand matched adapters and multi-seed numbers, not desk-reject.","headline":"Solid controlled finding that pair judgement beats full-dialogue generation for LLM ECPEC; the set-return bottleneck story is useful, but unequal LoRA setups weaken pure-formulation attribution.","tokens_in":19339,"tokens_out":591,"would_cite":true,"duration_ms":19631,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"How you ask an LLM to find emotion–cause pairs matters more than the model: judging one pair at a time beats generating the full set.","keywords":["emotion-cause pair extraction","conversation","large language models","task formulation","pair-level judgement","dialogue-level generation","candidate scope","decision boundary"],"falsifier":"Retrain GEN, SEL, and JUD with matched LoRA rank, alpha, dropout, epochs, and effective capacity on the same backbone and data; if the JUD-over-GEN F1 gap disappears or shrinks sharply across the same 18 settings, the paradigm diagnosis is confounded by optimization rather than task formulation.","tokens_in":19213,"feed_emoji":"💬","tokens_out":1008,"duration_ms":23576,"temperature":0.7,"pith_summary":"This paper asks a practical question about using large language models for emotion–cause pair extraction in conversation: does it matter whether the model is asked to dump every pair from a whole dialogue, pick causes for one target utterance, or answer yes/no on a single candidate pair? Across controlled comparisons, pair-level judgement consistently beats dialogue-level generation. The authors show the gap is not mainly that models cannot see the relations—when queried pair by pair they recognize the vast majority of gold links—but that generating a complete set at once is hard, especially when dialogues have many pairs, targets have multiple causes, or causes sit farther away. Judgement eases that discovery burden, yet a single shared yes/no threshold still fails to turn good rankings into clean sets. A light auxiliary retriever that rechecks only ambiguous boundary cases improves F1 a little on three datasets without blowing up cost, arguing that task decomposition and candidate scope are the levers that actually unlock LLMs on this task.","feed_headline":"Pair-by-pair judgement beats full-set generation for LLMs","feed_subtitle":"Models spot most emotion–cause links when asked directly, but miss them when forced to list every pair at once.","key_machinery":"Three controlled inference paradigms by granularity—dialogue-level generation (GEN), target-level selection (SEL), and pair-level judgement (JUD)—plus local versus full candidate scope, used to isolate how task decomposition and candidate range expose or hide pair recognition; a diagnosis-guided auxiliary retriever then re-examines only the highest-ranked rejected boundary candidate.","core_discovery":"Task formulation substantially shapes LLM performance on emotion–cause pair extraction in conversation: pair-level judgement outperforms dialogue-level generation in all 18 controlled backbone–input–scope comparisons, because models can recognize most gold relations under explicit pair queries (about 92.7%–98.1% recall) yet struggle to discover and return complete pair sets in one generative pass; rankings under judgement are more reliable than binary decisions from a shared threshold.","pith_inferences":["The same generation-versus-judgement split may show up in other multi-instance extraction tasks (e.g., event argument sets, multi-hop relation bundles) where models recognize items better than they exhaustively list them.","If matched-capacity training still leaves a large gap, prompting and decoding strategies that force complete set coverage become higher priority than bigger backbones alone.","Target-specific thresholds or calibrated scores could capture much of the Ideal-vs-Realized separability gap the paper measures without full pairwise re-inference.","Multimodal evidence did not close the paradigm gap here, suggesting formulation may dominate modality fusion for this family of conversational pair tasks."],"forward_implications":["For LLM-based ECPEC, defaulting to pair-level judgement over local candidates (self plus up to four preceding utterances) is a stronger practical baseline than one-shot full-dialogue generation.","Gains from finer decomposition come mainly from removing set-level discovery burden, not from giving the model new causal knowledge.","Shared binary thresholds leave separable rankings unrealized; adapting or correcting decision boundaries is a direct path to better F1.","Restricting candidate scope keeps most gold pairs while cutting the O(N²) cost of full judgement toward roughly linear time.","A small boundary retriever can trade a modest compute increase (~1.49× baseline time) for consistent F1 lifts without rewriting the main paradigm."],"fun_headline_variants":["Pair judgement beats full-set generation for LLM emotion-cause extraction","LLMs recognize emotion-cause links but miss full pair sets in generation","Task formulation drives LLM gains on conversational emotion-cause pairs","Explicit pair queries recover 92.7–98.1% of emotion-cause relations","Judgement rankings outperform shared-threshold decisions on ECPEC"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The claim that the performance gap is purely about how the task is posed rests on treating separately fine-tuned adapters for each paradigm as fair apples-to-apples comparisons, even though their training capacity and schedules differ.","fun_headline_variants_meta":{"raw":{"variants":["Pair judgement beats full-set generation for LLM emotion-cause extraction","LLMs recognize emotion-cause links but miss full pair sets in generation","Task formulation drives LLM gains on conversational emotion-cause pairs","Explicit pair queries recover 92.7–98.1% of emotion-cause relations","Judgement rankings outperform shared-threshold decisions on ECPEC"]},"model":"grok-4.5","effort":"low","cost_usd":0.002604,"raw_usage":{"total_tokens":1005,"prompt_tokens":802,"num_sources_used":0,"completion_tokens":76,"cost_in_usd_ticks":26044000,"prompt_tokens_details":{"text_tokens":802,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":127,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":802,"tokens_out":76,"duration_ms":3511,"temperature":1.0,"reasoning_tokens":127,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T15:29:38.883745+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Retrain GEN, SEL, and JUD with matched LoRA rank, alpha, dropout, epochs, and effective capacity on the same backbone and data; if the JUD-over-GEN F1 gap disappears or shrinks sharply across the same 18 settings, the paradigm diagnosis is confounded by optimization rather than task formulation.","supporting_citations":[],"review_version":1}