{"id":"80fd50c5-3ebc-45e9-9272-6f825b9fd758","arxiv_id":"2411.14880","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using hierarchy-aware learned prototypes as the verbalizer improves implicit discourse relation recognition over manual-connective verbalizers on English benchmarks and enables zero-shot transfer to six languages.","lead":"This paper replaces manually chosen connective words with learned prototype vectors as the verbalizer for implicit discourse relation recognition, and reports improved accuracy on PDTB-2 and PDTB-3 plus zero-shot cross-lingual gains on TED-MDB. A generalist reader might care because understanding discourse relations underpins summarization, question answering, and event extraction, and the method targets low-resource languages.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-shot cross-lingual result is underspecified: target prototypes are trained with the same supervised losses as Section 3, so target-language labels may be used; if so, Table 6 does not demonstrate zero-shot transfer.","rationale":"I agree with the reader that the zero-shot cross-lingual training procedure is the weakest point. The monolingual method is internally coherent: the three contrastive losses are well-specified, the ablations show each term contributes, and the reported gains over prior systems, though lacking error bars, are plausible. I do not see an internal inconsistency in the monolingual prototype-verbalizer construction. The cross-lingual section, however, is genuinely ambiguous: Section 6.2 says target prototypes are trained with the same methodology as Section 3, which requires labels, yet the paper calls the setting zero-shot. If target labels were used, the central cross-lingual claim collapses into a supervised transfer result. This is exactly the reader's weakest_assumption, so I mark agreement as 'agree.' I do not move the verdict because the conditional verdict already captures this uncertainty: the paper should clarify and re-run the experiment in a truly zero-shot setting before the cross-lingual claim can be accepted. The concrete test I propose would settle the ambiguity by checking the training pipeline and, if needed, rerunning without any target-side supervision.","tokens_in":12656,"tokens_out":4641,"duration_ms":50135,"concrete_test":"Inspect the TED-MDB splits and training script used for Section 6 to check whether target-language labels are used when estimating or aligning target prototypes. Then re-run the cross-lingual experiment in a strictly zero-shot configuration: freeze the source-trained encoder and source prototypes, obtain target predictions by projecting target instances into the source prototype space, and perform no target-side gradient updates or target-label-based prototype estimation. If Macro-F1 on Table 6 falls to the XLMR-base baseline level, the claimed zero-shot advantage is not supported; if it remains, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest novel claim is zero-shot cross-lingual IDRR, but Section 6.2 says target-language prototypes are 'trained by using the same methodology we described in section 3.' That methodology requires gold labels: Eq. 3 uses y_k for supervised contrastive pairs, Eq. 4 pairs each instance with its class prototype c_i, and Eq. 5 requires knowing the parent class of each prototype. If TED-MDB target-language labels participate anywhere in this training or prototype estimation, the setting is supervised cross-lingual transfer, not zero-shot. The paper never states whether target-side labels are used, how target prototypes are initialized, or how the parallel alignment would supply supervision without target labels. This is load-bearing because the cross-lingual experiment is the main extension beyond the monolingual prototype verbalizer, and the interpretation of Table 6 changes completely based on this undisclosed detail.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a prompt-based method for implicit discourse relation recognition (IDRR) that replaces manual connective verbalizers with learned hierarchical prototype vectors. On the [MASK] token hidden state, the model applies three contrastive losses: instance-instance, instance-prototype, and prototype-prototype, incorporating the PDTB sense hierarchy. Experiments on PDTB-2 and PDTB-3 report improvements over prior systems at both top and second level, with ablations showing all losses contribute. The method is then extended to a claimed zero-shot cross-lingual setting using PDTB-3 as source and TED-MDB target languages (German, Lithuanian, Polish, Portuguese, Russian, Turkish), reporting Macro-F1 gains over a vanilla XLM-R baseline and the prior zero-shot system of Kurfalı and Östling (2019).","tokens_in":12852,"tokens_out":1988,"duration_ms":20873,"significance":"If the claims are fully supported, the paper would make a useful contribution: it addresses a known weakness of manual verbalizers in prompt-based IDRR, introduces a hierarchical contrastive objective that appears to improve over several strong baselines, and tackles the underexplored problem of low-resource cross-lingual discourse relation recognition. The monolingual experimental setup is standard (PDTB-2/3 with the usual train/dev/test split), and the ablation study is informative. However, the paper's most distinctive extension, the zero-shot cross-lingual claim, is insufficiently specified to verify; the current description leaves open the possibility that target-language labels are used, which would change the interpretation of the reported results. The paper also lacks significance tests or multiple runs, and several rare classes receive zero F1, weakening the strength of the comparative claims. The prototype analysis in Section 5.2 is a useful qualitative sanity check but does not substitute for a rigorous evaluation.","major_comments":[{"comment":"The zero-shot cross-lingual setting is underspecified in a load-bearing way. The paper states that target-language prototypes are 'trained by using the same methodology we described in section 3', but that methodology uses gold labels in Eq. (3) (supervised contrastive pairs via y_k), Eq. (4) (pairing each instance with its class prototype c_i), and Eq. (5) (parent-class knowledge). If TED-MDB target-language labels participate in prototype training or in the source-target alignment stage, the setting is supervised or semi-supervised cross-lingual transfer, not zero-shot. The paper never states whether target-side labels are used, how target prototypes are initialized in a zero-shot scenario, or how the parallel alignment supplies supervision without target labels. Since the cross-lingual experiment is the main extension beyond the monolingual prototype verbalizer, the interpretation of Table 6 changes completely depending on this undisclosed detail. The authors must clarify the exact data flow and, if they intend a true zero-shot claim, describe how target prototypes are obtained without target-language labels.","section":"Section 6.2, Eqs. (3)-(5)"},{"comment":"Related to the previous point, the cross-lingual experiment lacks basic methodological details needed to assess the result: how the language-specific templates are constructed (only one German example is shown in Figure 6), whether the same random initialization is used for target prototypes, whether the source-target prototype alignment is trained jointly with the source model or after it, and how the approximately 200 implicit relations per TED-MDB language are split for testing. Without these details, the reader cannot reproduce the experiment or determine what is actually being transferred.","section":"Section 6.2 and Table 6"},{"comment":"The claim that the method 'demonstrate[s] better performance than previous systems for both Level-1 and Level-2' is overstated because several rare classes receive zero F1. In Table 4, our method obtains 0.00 F1 for Cont.Pragmatic cause and Exp.List, and the same is true for second-level classes in the compared systems. Zero F1 on an entire class means the model never predicts that class, which is a qualitative failure not reflected in the aggregate Macro-F1. The paper should report class-level results with confidence intervals or at least acknowledge this limitation in the main text rather than presenting the aggregate results as uniformly better.","section":"Section 5, Tables 1-4"},{"comment":"All reported numbers come from a single run, and no significance tests, standard deviations, or error bars are provided. Given that the improvements over the strongest baselines are often small (e.g., 72.47 vs. 71.70 accuracy on PDTB-2 top level, and 75.37 vs. 74.36 on PDTB-3), the reader cannot tell whether the differences are meaningful. The authors should report multiple seeds and significance tests, or at minimum provide a clear justification for why a single run is sufficient in this setting.","section":"Section 4.3 and Tables 1-2"}],"minor_comments":[{"comment":"The title contains a typo ('V erbalizer'), and the abstract has a grammatical error in 'These advancement validate'; these should be corrected.","section":"Title and Abstract"},{"comment":"The description of Kurfalı and Östling (2019) as 'the first study on a zero-shot transfer learning' is not supported by a detailed comparison; the paper should clarify what exactly that prior work does and how the proposed method differs beyond the prototype verbalizer.","section":"Section 2.3"},{"comment":"The label 'Asychronous' is misspelled in Figure 4; it should be 'Asynchronous'. The figures also lack a clear legend explaining whether the percentages are averaged over the top-ten neighbors of each prototype, which is only described in the text.","section":"Figure 4 and 5"},{"comment":"The construction of language-specific templates is described only through a single German example in Figure 6; the paper should state whether templates for all six target languages are manually designed and whether the 'label information' injected in the template is translated or kept in English.","section":"Section 6.1"}],"recommendation":"major_revision","confidential_remarks":"The monolingual contribution is plausible and the reported numbers are competitive, but the paper's flagship zero-shot cross-lingual claim is not verifiable as written because the methodology does not specify whether target-language labels are used. This is a correctness issue that affects the central claim of Section 6, and it should be resolved before publication. The lack of significance testing and the zero-F1 classes in Table 4 are secondary but should also be addressed. The paper is within the scope of the journal and the underlying idea is worth pursuing, but the current manuscript is not ready for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the monolingual half of this paper is a real, if modest, contribution; the zero-shot cross-lingual half does not support its own claim as written. The central problem is Section 6.2, which says target prototypes are 'trained by using the same methodology we described in section 3' — that methodology uses gold labels in all three losses. If TED-MDB labels are used to train or estimate target prototypes, Table 6 is supervised transfer, not zero-shot. The paper never states whether target labels are used. That detail is load-bearing because the cross-lingual result is the stated main extension.\n\nWhat is genuinely new: applying the prototypical verbalizer idea (Cui et al.) to implicit discourse relations, plus a hierarchy-aware prototype-prototype loss. The monolingual tables show consistent gains over prior systems on PDTB-2 and PDTB-3, and the ablations show each loss contributes, with the prototype-prototype loss most important. The nearest-neighbor analysis of learned prototypes is a nice diagnostic, and the observation that some 'Conjunction' examples could plausibly be 'Synchrony' is honest and worth following up.\n\nWhere it is soft: the zero-shot ambiguity above is the main one. Also, there are no significance tests or error bars, and several rare classes get 0 F1 in Table 4 — that suggests the macro-F1 is carried by a handful of classes. The TED-MDB test sets are only about 200 instances per language, so the cross-lingual differences are probably not stable. The baselines for cross-lingual are weak: vanilla fine-tuning and Kurfalı & Östling from 2019. No code or data is released, which makes it hard to check the prototype training details. These are not fatal for the monolingual story, but they lower confidence.\n\nBottom line: this paper deserves a serious referee, but the authors need to clarify the zero-shot procedure, add variance reporting or significance tests, and ideally release code. If the zero-shot claim is actually supervised transfer, the paper should say so and reframe the contribution. I would send it back for major revision.\n\nFor a reading group, it's a maybe — useful for people working on discourse or prompt-based methods, but not a must-read.","headline":"Monolingual prototypical verbalizer results are solid and honestly analyzed, but the zero-shot cross-lingual claim is underspecified to the point of being unsupported as written.","tokens_in":13354,"tokens_out":1981,"would_cite":false,"duration_ms":18521,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing hand-chosen connective verbalizers with learned, hierarchy-organized prototype vectors improves implicit discourse relation recognition on English PDTB-2 and PDTB-3, and enables zero-shot transfer to six low-resource languages.","keywords":["implicit discourse relation recognition","verbalizer","prototype learning","prompt-based tuning","contrastive learning","sense hierarchy","zero-shot cross-lingual transfer","PDTB"],"falsifier":"Hold the prompt and the encoder fixed, and replace the learned prototypes with the per-class average [MASK] hidden state of the training instances; if top-level accuracy on PDTB-2 does not drop materially below 72.47, the learned prototype machinery is not what carries the gain.","tokens_in":12452,"feed_emoji":"🧩","tokens_out":13977,"duration_ms":103112,"temperature":0.7,"pith_summary":"Implicit discourse relations are the connections between text spans that are not marked by an explicit connective word, and recognizing them is hard. Recent prompt-based methods turn the task into a cloze exercise where the model fills in a connective and a hand-built verbalizer maps that connective to a sense label, but most connectives are ambiguous, mapping to several senses. The paper tries to replace that manual verbalizer with a set of learned prototype vectors, one per sense, organized according to the three-level sense hierarchy of the Penn Discourse TreeBank, and trained with three contrastive losses. It reports that this hierarchical prototype verbalizer improves over competitive baselines on PDTB-2 and PDTB-3 at both the top and second level, and that the same method transfers zero-shot to six languages with limited data. If correct, the work offers a way to remove a manually designed, ambiguity-prone component from discourse relation systems and to extend them to languages without annotated data.","feed_headline":"Learned prototypes outperform manual connectives as verbalizers","feed_subtitle":"A contrastive prototype verbalizer lifts top-level accuracy on PDTB-3 to 75.4% and extends zero-shot to six languages.","key_machinery":"The hierarchical prototype verbalizer: a set of learnable prototype vectors, one for each discourse sense at each level of the PDTB sense hierarchy, used in place of manually selected connective words to map model outputs to labels. The [MASK] hidden state of a RoBERTa encoder is the instance representation, projected by a linear layer into the prototype space, and inference is a softmax over similarities to prototypes. The space is shaped by three contrastive losses: an instance-instance loss that pulls same-sense instances together and pushes different-sense ones apart, an instance-prototype loss that makes each prototype the center of its class, and a prototype-prototype loss that keeps each level-2 prototype close to its level-1 parent; the template additionally injects the list of all level-1 and level-2 labels into the input.","core_discovery":"The paper's central claim is that the verbalizer of a prompt-based implicit discourse relation system can be learned rather than hand-designed, and that learning it as a hierarchy of prototypes outperforms the manual connective-based verbalizers used by previous systems. Concretely, the [MASK] token's last-layer hidden state is projected into a low-dimensional space to represent each input instance; a set of prototype vectors, one per level-1 and level-2 sense, is learned in the same space. Three contrastive losses shape the space: instances of the same sense are pulled together and different senses pushed apart, each training instance is pulled toward its class prototypes and pushed from others, and each level-2 prototype is pulled toward its level-1 parent while different prototypes are pushed apart. At inference the predicted sense is the class whose prototype is most similar to the instance representation. On PDTB-3 the system reaches 75.37 accuracy and 71.19 Macro-F1 at the top level, and on PDTB-2 72.47 accuracy and 69.66 Macro-F1, both above the listed prior systems; the same prototype alignment, trained on English PDTB-3, improves Macro-F1 over baselines on German, Lithuanian, Polish, Portuguese, Russian, and Turkish.","pith_inferences":["A direct extension would apply the same hierarchy-aware prototype verbalizer to other tasks with a fixed label taxonomy, such as fine-grained sentiment or event-relation classification, where manual label words are equally ambiguous.","The observed mismatches between nearest neighbors and gold labels suggest the prototype space could be used as an annotation-aiding tool to surface plausible missing senses in datasets like PDTB-2.","The cross-lingual part implicitly assumes that the parallel TED-MDB corpus provides label equivalence between English and the target languages; if that assumption holds, the same alignment could be tested on other parallel corpora, such as TED-CDB for Chinese, to bootstrap discourse parsing in more languages.","The three losses are summed with equal weight; tuning their relative weights or adding a separate temperature for the prototype-prototype loss might push the reported numbers higher, and such tuning is a natural next test."],"forward_implications":["On both PDTB-2 and PDTB-3, the hierarchical prototype verbalizer improves top-level and second-level accuracy and Macro-F1 over every listed baseline, with the largest relative gains on Macro-F1.","Ablations show all three contrastive losses contribute, and the prototype-to-prototype loss matters more than the instance-to-instance loss, indicating that encoding the sense hierarchy itself carries much of the benefit.","The same approach transfers zero-shot to six target languages in TED-MDB, improving Macro-F1 by roughly 8% over the previous zero-shot transfer system on Turkish and by nearly 10% over a vanilla XLM-R baseline on Portuguese.","Learned prototypes are interpretable: the ten nearest neighbors of each second-level prototype are mostly from the same class, and some mismatches correspond to plausible unannotated secondary senses in the data."],"supporting_citations":[{"why":"Proposes the prototypical verbalizer for prompt-based few-shot tuning, which this paper adapts by organizing prototypes according to the sense hierarchy.","marker":"Cui et al. (2022)"},{"why":"Defines the supervised contrastive loss that underlies the instance-instance and instance-prototype objectives.","marker":"Khosla et al. (2020)"},{"why":"Provides the hierarchical contrastive learning of discourse relational senses that the prototype-prototype and instance-instance losses build on, and serves as a key baseline.","marker":"Long and Webber (2022)"},{"why":"Prompt-based connective prediction system whose manual verbalizer is the main target of the paper's critique and a baseline it outperforms.","marker":"Zhou et al. (2022)"},{"why":"ConnPrompt, a connective-cloze prompt baseline whose manual verbalizer the paper compares against on PDTB-3.","marker":"Xiang et al. (2022b)"},{"why":"DiscoPrompt, a prompt-tuning baseline injecting label dependencies, which the prototype verbalizer surpasses on both PDTB versions.","marker":"Chan et al. (2023)"},{"why":"The prior zero-shot transfer system for implicit discourse relations, used as the baseline for the cross-lingual experiments.","marker":"Kurfalı and Östling (2019)"},{"why":"The PDTB-3 corpus and annotation manual that define the sense hierarchy and supply the labels used for the prototype structure.","marker":"Webber et al. (2019)"}],"fun_headline_variants":["Hierarchical prototypes beat manual verbalizers for implicit discourse relations","Learn prototypes as verbalizers, not manual connectives, for discourse relations","Prototype verbalizer lifts discourse relation accuracy to 75.4%","Zero-shot cross-lingual discourse relations via prototype verbalizers","Prototype-based verbalizer improves implicit discourse relation recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that, after training, the model's internal representation of a candidate relation is organized so that the closest learned prototype is a better predictor of the correct relation than the hand-picked connective words used by previous verbalizers.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical prototypes beat manual verbalizers for implicit discourse relations","Learn prototypes as verbalizers, not manual connectives, for discourse relations","Prototype verbalizer lifts discourse relation accuracy to 75.4%","Zero-shot cross-lingual discourse relations via prototype verbalizers","Prototype-based verbalizer improves implicit discourse relation recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0005,"raw_usage":{"total_tokens":2451,"prompt_tokens":951,"completion_tokens":1500,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":1413}},"tokens_in":567,"tokens_out":1500,"duration_ms":9212,"temperature":1.0,"reasoning_tokens":1413,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:45:41.755404+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold the prompt and the encoder fixed, and replace the learned prototypes with the per-class average [MASK] hidden state of the training instances; if top-level accuracy on PDTB-2 does not drop materially below 72.47, the learned prototype machinery is not what carries the gain.","supporting_citations":[],"review_version":1}