Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read Replacing hand-chosen connective verbalizers with learned, hierarchy-organized prototype vectors improves implicit discourse relation recognition on English PDTB-2 and PDTB-3, and enables zero-shot transfer to six low-resource languages.

desk verdict Monolingual prototypical verbalizer results are solid and honestly analyzed, but the zero-shot cross-lingual claim is underspecified to the point of being unsupported as written. read the letter →

arxiv 2411.14880 v1 pith:RBR2UGYW submitted 2024-11-22 cs.CL

classification cs.CL
keywords implicitdiscourserelationrecognitionverbalizerprototypelearningprompt-basedtuningcontrastivesensehierarchyzero-shotcross-lingualtransferPDTB
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Implicit discourse relations are the connections between text spans that are not marked by an explicit connective word, and recognizing them is hard. Recent prompt-based methods turn the task into a cloze exercise where the model fills in a connective and a hand-built verbalizer maps that connective to a sense label, but most connectives are ambiguous, mapping to several senses. The paper tries to replace that manual verbalizer with a set of learned prototype vectors, one per sense, organized according to the three-level sense hierarchy of the Penn Discourse TreeBank, and trained with three contrastive losses. It reports that this hierarchical prototype verbalizer improves over competitive baselines on PDTB-2 and PDTB-3 at both the top and second level, and that the same method transfers zero-shot to six languages with limited data. If correct, the work offers a way to remove a manually designed, ambiguity-prone component from discourse relation systems and to extend them to languages without annotated data.

What carries the argument

The hierarchical prototype verbalizer: a set of learnable prototype vectors, one for each discourse sense at each level of the PDTB sense hierarchy, used in place of manually selected connective words to map model outputs to labels. The [MASK] hidden state of a RoBERTa encoder is the instance representation, projected by a linear layer into the prototype space, and inference is a softmax over similarities to prototypes. The space is shaped by three contrastive losses: an instance-instance loss that pulls same-sense instances together and pushes different-sense ones apart, an instance-prototype loss that makes each prototype the center of its class, and a prototype-prototype loss that keeps each level-2 prototype close to its level-1 parent; the template additionally injects the list of all level-1 and level-2 labels into the input.

What would settle it

Hold the prompt and the encoder fixed, and replace the learned prototypes with the per-class average [MASK] hidden state of the training instances; if top-level accuracy on PDTB-2 does not drop materially below 72.47, the learned prototype machinery is not what carries the gain.

Watch

Extended reading notes

Core claim

The paper's central claim is that the verbalizer of a prompt-based implicit discourse relation system can be learned rather than hand-designed, and that learning it as a hierarchy of prototypes outperforms the manual connective-based verbalizers used by previous systems. Concretely, the [MASK] token's last-layer hidden state is projected into a low-dimensional space to represent each input instance; a set of prototype vectors, one per level-1 and level-2 sense, is learned in the same space. Three contrastive losses shape the space: instances of the same sense are pulled together and different senses pushed apart, each training instance is pulled toward its class prototypes and pushed from others, and each level-2 prototype is pulled toward its level-1 parent while different prototypes are pushed apart. At inference the predicted sense is the class whose prototype is most similar to the instance representation. On PDTB-3 the system reaches 75.37 accuracy and 71.19 Macro-F1 at the top level, and on PDTB-2 72.47 accuracy and 69.66 Macro-F1, both above the listed prior systems; the same prototype alignment, trained on English PDTB-3, improves Macro-F1 over baselines on German, Lithuanian, Polish, Portuguese, Russian, and Turkish.

Load-bearing premise

The method assumes that, after training, the model's internal representation of a candidate relation is organized so that the closest learned prototype is a better predictor of the correct relation than the hand-picked connective words used by previous verbalizers.

Editorial extensions

If this is right

  • On both PDTB-2 and PDTB-3, the hierarchical prototype verbalizer improves top-level and second-level accuracy and Macro-F1 over every listed baseline, with the largest relative gains on Macro-F1.
  • Ablations show all three contrastive losses contribute, and the prototype-to-prototype loss matters more than the instance-to-instance loss, indicating that encoding the sense hierarchy itself carries much of the benefit.
  • The same approach transfers zero-shot to six target languages in TED-MDB, improving Macro-F1 by roughly 8% over the previous zero-shot transfer system on Turkish and by nearly 10% over a vanilla XLM-R baseline on Portuguese.
  • Learned prototypes are interpretable: the ten nearest neighbors of each second-level prototype are mostly from the same class, and some mismatches correspond to plausible unannotated secondary senses in the data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would apply the same hierarchy-aware prototype verbalizer to other tasks with a fixed label taxonomy, such as fine-grained sentiment or event-relation classification, where manual label words are equally ambiguous.
  • The observed mismatches between nearest neighbors and gold labels suggest the prototype space could be used as an annotation-aiding tool to surface plausible missing senses in datasets like PDTB-2.
  • The cross-lingual part implicitly assumes that the parallel TED-MDB corpus provides label equivalence between English and the target languages; if that assumption holds, the same alignment could be tested on other parallel corpora, such as TED-CDB for Chinese, to bootstrap discourse parsing in more languages.
  • The three losses are summed with equal weight; tuning their relative weights or adding a separate temperature for the prototype-prototype loss might push the reported numbers higher, and such tuning is a natural next test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a prompt-based method for implicit discourse relation recognition (IDRR) that replaces manual connective verbalizers with learned hierarchical prototype vectors. On the [MASK] token hidden state, the model applies three contrastive losses: instance-instance, instance-prototype, and prototype-prototype, incorporating the PDTB sense hierarchy. Experiments on PDTB-2 and PDTB-3 report improvements over prior systems at both top and second level, with ablations showing all losses contribute. The method is then extended to a claimed zero-shot cross-lingual setting using PDTB-3 as source and TED-MDB target languages (German, Lithuanian, Polish, Portuguese, Russian, Turkish), reporting Macro-F1 gains over a vanilla XLM-R baseline and the prior zero-shot system of Kurfalı and Östling (2019).

Significance. If the claims are fully supported, the paper would make a useful contribution: it addresses a known weakness of manual verbalizers in prompt-based IDRR, introduces a hierarchical contrastive objective that appears to improve over several strong baselines, and tackles the underexplored problem of low-resource cross-lingual discourse relation recognition. The monolingual experimental setup is standard (PDTB-2/3 with the usual train/dev/test split), and the ablation study is informative. However, the paper's most distinctive extension, the zero-shot cross-lingual claim, is insufficiently specified to verify; the current description leaves open the possibility that target-language labels are used, which would change the interpretation of the reported results. The paper also lacks significance tests or multiple runs, and several rare classes receive zero F1, weakening the strength of the comparative claims. The prototype analysis in Section 5.2 is a useful qualitative sanity check but does not substitute for a rigorous evaluation.

major comments (4)
  1. [Section 6.2, Eqs. (3)-(5)] The zero-shot cross-lingual setting is underspecified in a load-bearing way. The paper states that target-language prototypes are 'trained by using the same methodology we described in section 3', but that methodology uses gold labels in Eq. (3) (supervised contrastive pairs via y_k), Eq. (4) (pairing each instance with its class prototype c_i), and Eq. (5) (parent-class knowledge). If TED-MDB target-language labels participate in prototype training or in the source-target alignment stage, the setting is supervised or semi-supervised cross-lingual transfer, not zero-shot. The paper never states whether target-side labels are used, how target prototypes are initialized in a zero-shot scenario, or how the parallel alignment supplies supervision without target labels. Since the cross-lingual experiment is the main extension beyond the monolingual prototype verbalizer, the interpretation of Table 6 changes completely depending on this undisclosed detail. The authors must clarify the exact data flow and, if they intend a true zero-shot claim, describe how target prototypes are obtained without target-language labels.
  2. [Section 6.2 and Table 6] Related to the previous point, the cross-lingual experiment lacks basic methodological details needed to assess the result: how the language-specific templates are constructed (only one German example is shown in Figure 6), whether the same random initialization is used for target prototypes, whether the source-target prototype alignment is trained jointly with the source model or after it, and how the approximately 200 implicit relations per TED-MDB language are split for testing. Without these details, the reader cannot reproduce the experiment or determine what is actually being transferred.
  3. [Section 5, Tables 1-4] The claim that the method 'demonstrate[s] better performance than previous systems for both Level-1 and Level-2' is overstated because several rare classes receive zero F1. In Table 4, our method obtains 0.00 F1 for Cont.Pragmatic cause and Exp.List, and the same is true for second-level classes in the compared systems. Zero F1 on an entire class means the model never predicts that class, which is a qualitative failure not reflected in the aggregate Macro-F1. The paper should report class-level results with confidence intervals or at least acknowledge this limitation in the main text rather than presenting the aggregate results as uniformly better.
  4. [Section 4.3 and Tables 1-2] All reported numbers come from a single run, and no significance tests, standard deviations, or error bars are provided. Given that the improvements over the strongest baselines are often small (e.g., 72.47 vs. 71.70 accuracy on PDTB-2 top level, and 75.37 vs. 74.36 on PDTB-3), the reader cannot tell whether the differences are meaningful. The authors should report multiple seeds and significance tests, or at minimum provide a clear justification for why a single run is sufficient in this setting.
minor comments (4)
  1. [Title and Abstract] The title contains a typo ('V erbalizer'), and the abstract has a grammatical error in 'These advancement validate'; these should be corrected.
  2. [Section 2.3] The description of Kurfalı and Östling (2019) as 'the first study on a zero-shot transfer learning' is not supported by a detailed comparison; the paper should clarify what exactly that prior work does and how the proposed method differs beyond the prototype verbalizer.
  3. [Figure 4 and 5] The label 'Asychronous' is misspelled in Figure 4; it should be 'Asynchronous'. The figures also lack a clear legend explaining whether the percentages are averaged over the top-ten neighbors of each prototype, which is only described in the text.
  4. [Section 6.1] The construction of language-specific templates is described only through a single German example in Figure 6; the paper should state whether templates for all six target languages are manually designed and whether the 'label information' injected in the template is translated or kept in English.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central claim is evaluated on held-out external benchmark splits, and the self-citation to the authors' prior work is transparent and not load-bearing.

full rationale

I find no circular step that reduces a claimed prediction to the paper's own inputs by construction. The monolingual experiments train prototypes and the encoder on PDTB-2/PDTB-3 Sections 2-20 and evaluate on held-out Sections 21-22, so the reported gains are measured against unseen test data rather than being definitionally implied by the training objective. The prototype verbalizer is a new mechanism replacing manual verbalizers, and although the instance-instance contrastive loss follows the authors' earlier Long and Webber (2022) framework, that prior work is also listed as a baseline and outperformed, so the self-citation is not load-bearing. The cross-lingual section is genuinely underspecified: Section 6.2 says target-language prototypes are 'trained by using the same methodology we described in section 3,' and that methodology uses gold labels in Equations 3-5; if target-language labels participate in prototype training, the 'zero-shot' description would be inaccurate. However, that is an experimental-validity concern about an undisclosed protocol detail, not a circularity: no equation or construction shows the reported cross-lingual scores being forced by the target labels or by a fitted parameter renamed as a prediction. The paper is therefore self-contained against external benchmarks, and I assign a low score reflecting only the minor, non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or linguistic entities. The free parameters are standard hyperparameters, and the assumptions are typical for NLP methods of this kind, though the zero-shot cross-lingual assumption about label correspondence is not fully evidenced.

free parameters (4)
  • temperature tau = 0.1
    Used in all three contrastive loss equations (Eq. 3-5) and in the inference similarity; chosen without reported sensitivity analysis.
  • prototype dimension = 128
    Dimensionality of the projected instance vectors and prototypes; set in Section 4.3 without justification.
  • learning rate = 5e-5
    Adam optimizer learning rate reported in Section 4.3.
  • batch size = 196
    Batch size used for training, reported in Section 4.3, affecting the in-batch contrastive pair construction.
assumptions (4)
  • domain assumption The PDTB sense hierarchy is a reliable organization of discourse relations and can act as a structural prior for contrastive learning.
    Invoked in Section 3.2 where father-child prototype pairs are defined according to the sense hierarchy.
  • domain assumption RoBERTa and XLM-RoBERTa [MASK] hidden states carry sufficient discourse information after fine-tuning.
    The instance representation v is defined as a linear projection of the [MASK] hidden state (Eq. 2), assuming this vector is informative enough for relation classification.
  • domain assumption TED-MDB is a parallel corpus whose English labels can be used to align target-language prototypes for zero-shot transfer.
    Section 6.2 assumes source and target prototypes of the same class can be pulled together, which requires some label correspondence between languages.
  • ad hoc to paper The random initialization of prototypes combined with contrastive losses converges to meaningful class centers.
    The paper states prototypes are randomly initialized and trained with contrastive losses (Section 3.2); no proof or analysis is given that this procedure yields unbiased class centers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition." pith.science (2026). https://pith.science/paper/RBR2UGYW

@misc{pith2026241114880,
  author       = {Pith},
  title        = {Pith review of: Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBR2UGYW}},
  note         = {Machine review of arXiv:2411.14880}
}
read the original abstract

Implicit discourse relation recognition involves determining relationships that hold between spans of text that are not linked by an explicit discourse connective. In recent years, the pre-train, prompt, and predict paradigm has emerged as a promising approach for tackling this task. However, previous work solely relied on manual verbalizers for implicit discourse relation recognition, which suffer from issues of ambiguity and even incorrectness. To overcome these limitations, we leverage the prototypes that capture certain class-level semantic features and the hierarchical label structure for different classes as the verbalizer. We show that our method improves on competitive baselines. Besides, our proposed approach can be extended to enable zero-shot cross-lingual learning, facilitating the recognition of discourse relations in languages with scarce resources. These advancement validate the practicality and versatility of our approach in addressing the issues of implicit discourse relation recognition across different languages.

Figures

Figures reproduced from arXiv: 2411.14880 by the authors.

Figure 1
Figure 1. The manual verbalizer and the labels. the filled connectives are mapped to corresponding labels via the mannually-designed verbalizer. However, since most connectives are not only used for one sense, many connectives they selected can be connected with several labels instead of only one labels. In the given example depicted in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. In order to create the prompt in￾put, the input x is added a template T(·) = “The connective between Arg1 and Arg2 is [MASK]”. The verbalizer maps labels to connectives that can indicate the labels. For ex￾amples, The discourse relation “Asynchronous” is mapped as the connective “previously”. 3.2 Our Approach [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The hidden states of [MASK] token represent instances and project them to another embedding space for prototype learning. Three contrastive learning losses adjust the distances among prototypes, the distances among instances, and the distances between prototypes and instances based on the sense hierarchy. Finally, we calculate the similarity scores of query and prototypes during inference. where ci represents the pr… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Label distribution of the top ten nearest neigh [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 3
Figure 3. Figure 3: Average Cosine Distances between the proto [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 6
Figure 6. Figure 6: One example on constructing our language [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Prototype Alignment between the source and [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Lingual Implicit Discourse Relation Recognition with Multi-Label Hierarchical Learning

    cs.CL 2025-08 conditional novelty 5.0 of 10

    HArch, a hierarchical multi-task model, is the first to recognize implied discourse relations with multi-label sense distributions in four languages, and it outperforms few-shot LLM prompting.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [3]

    A regulariza- tion approach for incorporating event knowledge and coreference relations into neural discourse parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 2976– 2987, Hong Kong, China. Association for Comp...

  2. [5]

    In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 2418–2431, Online and Punta Cana, Dominican Republic

    Not just classification: Recog- nizing implicit discourse relation on joint modeling of classification and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 2418–2431, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Prannay Khosla, Piotr Teterwak, Chen Wang,...

  3. [9]

    CoRR, abs/1907.11692

    Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. Wanqiu Long and Bonnie Webber

  4. [11]

    In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 2793–2803, On- line

    TED-CDB: A large-scale Chinese discourse relation dataset on TED talks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 2793–2803, On- line. Association for Computational Linguistics. Linh The Nguyen, Ngo Van Linh, Khoat Than, and Thien Huu Nguyen

  5. [12]

    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2804–2819, Online

    QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2804–2819, Online. Association for Computational Linguistics. Lianhui Qin, Zhisong Zhang, and Hai Zhao

  6. [13]

    In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguistics: Main V olume, pages 255–269, Online

    Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguistics: Main V olume, pages 255–269, Online. Association for Computa- tional Linguistics. Wei Shi and Vera Demberg. 2019a. Learning to explici- tate connective...

  7. [14]

    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Online

    AutoPrompt: Elic- iting Knowledge from Language Models with Auto- matically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Online. Association for Computational Linguistics. Jake Snell, Kevin Swersky, and Richard S. Zemel

  8. [15]

    A Label Dependence-aware Sequence Generation Model for Multi-level Implicit Discourse Relation Recognition

    The penn discourse treebank 3.0 annota- tion manual. Changxing Wu, Liuwen Cao, Yubin Ge, Yang Liu, Min Zhang, and Jinsong Su. 2021a. A label dependence- aware sequence generation model for multi-level im- plicit discourse relation recognition. arXiv preprint arXiv:2112.11740. Changxing Wu, Liuwen Cao, Yubin Ge, Yang Liu, Min Zhang, and Jinsong Su. 2021b. ...

Show all 17 references
  1. [16]

    A label dependence- aware sequence generation model for multi-level im- plicit discourse relation recognition. In AAAI. Wei Xiang, Bang Wang, Lu Dai, and Yijun Mo. 2022a. Encoding and fusing semantic connection and lin- guistic evidence for implicit discourse relation recog- n...

  2. [17]

    In Findings of the Association for Computational Linguistics: EMNLP 2022 , pages 3848–3858, Abu Dhabi, United Arab Emirates

    Prompt-based connective pre- diction method for fine-grained implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: EMNLP 2022 , pages 3848–3858, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics. Ran Z...

  3. [2016]

    In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 1224–1233, Austin, Texas

    Recognizing implicit discourse relations via repeated reading: Neural net- works with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 1224–1233, Austin, Texas. Association for Computational Linguistics. ...

  4. [2017]

    In Proceedings of the 2017 Conference on Empirical Methods in Natu- ral Language Processing, pages 1299–1308, Copen- hagen, Denmark

    Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natu- ral Language Processing, pages 1299–1308, Copen- hagen, Denmark. Association for Computational ...

  5. [2019]

    CoRR, abs/1911.02116

    Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. Ganqu Cui, Shengding Hu, Ning Ding, Longtao Huang, and Zhiyuan Liu

  6. [2020]

    arXiv preprint arXiv:2004.11362

    Supervised contrastive learning. arXiv preprint arXiv:2004.11362. Diederick P Kingma and Jimmy Ba

  7. [2021]

    In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 1275–1283, Punta Cana, Dominican Republic

    CV AE-based re-anchoring for implicit discourse re- lation classification. In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 1275–1283, Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics. Tianyu Gao, Adam Fisch, and Danqi Chen

  8. [2022]

    In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 10704– 10716, Abu Dhabi, United Arab Emirates

    Facilitating contrastive learning of discourse relational senses by exploiting the hierarchy of sense relations. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 10704– 10716, Abu Dhabi, United Arab Emirates. Associa- tion fo...

  9. [2023]

    ArXiv, abs/2305.03973

    Discoprompt: Path prediction prompt tuning for implicit discourse relation recogni- tion. ArXiv, abs/2305.03973. Jifan Chen, Qi Zhang, Pengfei Liu, and Xuanjing Huang. 2016a. Discourse relations detection via a mixed generative-discriminative framework. In AAAI. Jifan Chen, Qi...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.