REVIEW 4 major objections 4 minor 1 cited by
Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition
T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Replacing hand-chosen connective verbalizers with learned, hierarchy-organized prototype vectors improves implicit discourse relation recognition on English PDTB-2 and PDTB-3, and enables zero-shot transfer to six low-resource languages.
desk verdict Monolingual prototypical verbalizer results are solid and honestly analyzed, but the zero-shot cross-lingual claim is underspecified to the point of being unsupported as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The hierarchical prototype verbalizer: a set of learnable prototype vectors, one for each discourse sense at each level of the PDTB sense hierarchy, used in place of manually selected connective words to map model outputs to labels. The [MASK] hidden state of a RoBERTa encoder is the instance representation, projected by a linear layer into the prototype space, and inference is a softmax over similarities to prototypes. The space is shaped by three contrastive losses: an instance-instance loss that pulls same-sense instances together and pushes different-sense ones apart, an instance-prototype loss that makes each prototype the center of its class, and a prototype-prototype loss that keeps each level-2 prototype close to its level-1 parent; the template additionally injects the list of all level-1 and level-2 labels into the input.
What would settle it
Hold the prompt and the encoder fixed, and replace the learned prototypes with the per-class average [MASK] hidden state of the training instances; if top-level accuracy on PDTB-2 does not drop materially below 72.47, the learned prototype machinery is not what carries the gain.
Extended reading notes
Core claim
The paper's central claim is that the verbalizer of a prompt-based implicit discourse relation system can be learned rather than hand-designed, and that learning it as a hierarchy of prototypes outperforms the manual connective-based verbalizers used by previous systems. Concretely, the [MASK] token's last-layer hidden state is projected into a low-dimensional space to represent each input instance; a set of prototype vectors, one per level-1 and level-2 sense, is learned in the same space. Three contrastive losses shape the space: instances of the same sense are pulled together and different senses pushed apart, each training instance is pulled toward its class prototypes and pushed from others, and each level-2 prototype is pulled toward its level-1 parent while different prototypes are pushed apart. At inference the predicted sense is the class whose prototype is most similar to the instance representation. On PDTB-3 the system reaches 75.37 accuracy and 71.19 Macro-F1 at the top level, and on PDTB-2 72.47 accuracy and 69.66 Macro-F1, both above the listed prior systems; the same prototype alignment, trained on English PDTB-3, improves Macro-F1 over baselines on German, Lithuanian, Polish, Portuguese, Russian, and Turkish.
Load-bearing premise
The method assumes that, after training, the model's internal representation of a candidate relation is organized so that the closest learned prototype is a better predictor of the correct relation than the hand-picked connective words used by previous verbalizers.
Editorial extensions
If this is right
- On both PDTB-2 and PDTB-3, the hierarchical prototype verbalizer improves top-level and second-level accuracy and Macro-F1 over every listed baseline, with the largest relative gains on Macro-F1.
- Ablations show all three contrastive losses contribute, and the prototype-to-prototype loss matters more than the instance-to-instance loss, indicating that encoding the sense hierarchy itself carries much of the benefit.
- The same approach transfers zero-shot to six target languages in TED-MDB, improving Macro-F1 by roughly 8% over the previous zero-shot transfer system on Turkish and by nearly 10% over a vanilla XLM-R baseline on Portuguese.
- Learned prototypes are interpretable: the ten nearest neighbors of each second-level prototype are mostly from the same class, and some mismatches correspond to plausible unannotated secondary senses in the data.
Reading between the lines
- A direct extension would apply the same hierarchy-aware prototype verbalizer to other tasks with a fixed label taxonomy, such as fine-grained sentiment or event-relation classification, where manual label words are equally ambiguous.
- The observed mismatches between nearest neighbors and gold labels suggest the prototype space could be used as an annotation-aiding tool to surface plausible missing senses in datasets like PDTB-2.
- The cross-lingual part implicitly assumes that the parallel TED-MDB corpus provides label equivalence between English and the target languages; if that assumption holds, the same alignment could be tested on other parallel corpora, such as TED-CDB for Chinese, to bootstrap discourse parsing in more languages.
- The three losses are summed with equal weight; tuning their relative weights or adding a separate temperature for the prototype-prototype loss might push the reported numbers higher, and such tuning is a natural next test.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a prompt-based method for implicit discourse relation recognition (IDRR) that replaces manual connective verbalizers with learned hierarchical prototype vectors. On the [MASK] token hidden state, the model applies three contrastive losses: instance-instance, instance-prototype, and prototype-prototype, incorporating the PDTB sense hierarchy. Experiments on PDTB-2 and PDTB-3 report improvements over prior systems at both top and second level, with ablations showing all losses contribute. The method is then extended to a claimed zero-shot cross-lingual setting using PDTB-3 as source and TED-MDB target languages (German, Lithuanian, Polish, Portuguese, Russian, Turkish), reporting Macro-F1 gains over a vanilla XLM-R baseline and the prior zero-shot system of Kurfalı and Östling (2019).
Significance. If the claims are fully supported, the paper would make a useful contribution: it addresses a known weakness of manual verbalizers in prompt-based IDRR, introduces a hierarchical contrastive objective that appears to improve over several strong baselines, and tackles the underexplored problem of low-resource cross-lingual discourse relation recognition. The monolingual experimental setup is standard (PDTB-2/3 with the usual train/dev/test split), and the ablation study is informative. However, the paper's most distinctive extension, the zero-shot cross-lingual claim, is insufficiently specified to verify; the current description leaves open the possibility that target-language labels are used, which would change the interpretation of the reported results. The paper also lacks significance tests or multiple runs, and several rare classes receive zero F1, weakening the strength of the comparative claims. The prototype analysis in Section 5.2 is a useful qualitative sanity check but does not substitute for a rigorous evaluation.
major comments (4)
- [Section 6.2, Eqs. (3)-(5)] The zero-shot cross-lingual setting is underspecified in a load-bearing way. The paper states that target-language prototypes are 'trained by using the same methodology we described in section 3', but that methodology uses gold labels in Eq. (3) (supervised contrastive pairs via y_k), Eq. (4) (pairing each instance with its class prototype c_i), and Eq. (5) (parent-class knowledge). If TED-MDB target-language labels participate in prototype training or in the source-target alignment stage, the setting is supervised or semi-supervised cross-lingual transfer, not zero-shot. The paper never states whether target-side labels are used, how target prototypes are initialized in a zero-shot scenario, or how the parallel alignment supplies supervision without target labels. Since the cross-lingual experiment is the main extension beyond the monolingual prototype verbalizer, the interpretation of Table 6 changes completely depending on this undisclosed detail. The authors must clarify the exact data flow and, if they intend a true zero-shot claim, describe how target prototypes are obtained without target-language labels.
- [Section 6.2 and Table 6] Related to the previous point, the cross-lingual experiment lacks basic methodological details needed to assess the result: how the language-specific templates are constructed (only one German example is shown in Figure 6), whether the same random initialization is used for target prototypes, whether the source-target prototype alignment is trained jointly with the source model or after it, and how the approximately 200 implicit relations per TED-MDB language are split for testing. Without these details, the reader cannot reproduce the experiment or determine what is actually being transferred.
- [Section 5, Tables 1-4] The claim that the method 'demonstrate[s] better performance than previous systems for both Level-1 and Level-2' is overstated because several rare classes receive zero F1. In Table 4, our method obtains 0.00 F1 for Cont.Pragmatic cause and Exp.List, and the same is true for second-level classes in the compared systems. Zero F1 on an entire class means the model never predicts that class, which is a qualitative failure not reflected in the aggregate Macro-F1. The paper should report class-level results with confidence intervals or at least acknowledge this limitation in the main text rather than presenting the aggregate results as uniformly better.
- [Section 4.3 and Tables 1-2] All reported numbers come from a single run, and no significance tests, standard deviations, or error bars are provided. Given that the improvements over the strongest baselines are often small (e.g., 72.47 vs. 71.70 accuracy on PDTB-2 top level, and 75.37 vs. 74.36 on PDTB-3), the reader cannot tell whether the differences are meaningful. The authors should report multiple seeds and significance tests, or at minimum provide a clear justification for why a single run is sufficient in this setting.
minor comments (4)
- [Title and Abstract] The title contains a typo ('V erbalizer'), and the abstract has a grammatical error in 'These advancement validate'; these should be corrected.
- [Section 2.3] The description of Kurfalı and Östling (2019) as 'the first study on a zero-shot transfer learning' is not supported by a detailed comparison; the paper should clarify what exactly that prior work does and how the proposed method differs beyond the prototype verbalizer.
- [Figure 4 and 5] The label 'Asychronous' is misspelled in Figure 4; it should be 'Asynchronous'. The figures also lack a clear legend explaining whether the percentages are averaged over the top-ten neighbors of each prototype, which is only described in the text.
- [Section 6.1] The construction of language-specific templates is described only through a single German example in Figure 6; the paper should state whether templates for all six target languages are manually designed and whether the 'label information' injected in the template is translated or kept in English.
Circularity Check
No significant circularity: the central claim is evaluated on held-out external benchmark splits, and the self-citation to the authors' prior work is transparent and not load-bearing.
full rationale
I find no circular step that reduces a claimed prediction to the paper's own inputs by construction. The monolingual experiments train prototypes and the encoder on PDTB-2/PDTB-3 Sections 2-20 and evaluate on held-out Sections 21-22, so the reported gains are measured against unseen test data rather than being definitionally implied by the training objective. The prototype verbalizer is a new mechanism replacing manual verbalizers, and although the instance-instance contrastive loss follows the authors' earlier Long and Webber (2022) framework, that prior work is also listed as a baseline and outperformed, so the self-citation is not load-bearing. The cross-lingual section is genuinely underspecified: Section 6.2 says target-language prototypes are 'trained by using the same methodology we described in section 3,' and that methodology uses gold labels in Equations 3-5; if target-language labels participate in prototype training, the 'zero-shot' description would be inaccurate. However, that is an experimental-validity concern about an undisclosed protocol detail, not a circularity: no equation or construction shows the reported cross-lingual scores being forced by the target labels or by a fitted parameter renamed as a prediction. The paper is therefore self-contained against external benchmarks, and I assign a low score reflecting only the minor, non-load-bearing self-citation.
Assumptions & free parameters
free parameters (4)
- temperature tau =
0.1
- prototype dimension =
128
- learning rate =
5e-5
- batch size =
196
assumptions (4)
- domain assumption The PDTB sense hierarchy is a reliable organization of discourse relations and can act as a structural prior for contrastive learning.
- domain assumption RoBERTa and XLM-RoBERTa [MASK] hidden states carry sufficient discourse information after fine-tuning.
- domain assumption TED-MDB is a parallel corpus whose English labels can be used to align target-language prototypes for zero-shot transfer.
- ad hoc to paper The random initialization of prototypes combined with contrastive losses converges to meaningful class centers.
Cite this review
Pith. "Pith review of Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition." pith.science (2026). https://pith.science/paper/RBR2UGYW
@misc{pith2026241114880,
author = {Pith},
title = {Pith review of: Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/RBR2UGYW}},
note = {Machine review of arXiv:2411.14880}
}
read the original abstract
Implicit discourse relation recognition involves determining relationships that hold between spans of text that are not linked by an explicit discourse connective. In recent years, the pre-train, prompt, and predict paradigm has emerged as a promising approach for tackling this task. However, previous work solely relied on manual verbalizers for implicit discourse relation recognition, which suffer from issues of ambiguity and even incorrectness. To overcome these limitations, we leverage the prototypes that capture certain class-level semantic features and the hierarchical label structure for different classes as the verbalizer. We show that our method improves on competitive baselines. Besides, our proposed approach can be extended to enable zero-shot cross-lingual learning, facilitating the recognition of discourse relations in languages with scarce resources. These advancement validate the practicality and versatility of our approach in addressing the issues of implicit discourse relation recognition across different languages.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 1 Pith paper
-
Multi-Lingual Implicit Discourse Relation Recognition with Multi-Label Hierarchical Learning
HArch, a hierarchical multi-task model, is the first to recognize implied discourse relations with multi-label sense distributions in four languages, and it outperforms few-shot LLM prompting.
Reference graph
Works this paper leans on
-
[3]
A regulariza- tion approach for incorporating event knowledge and coreference relations into neural discourse parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pages 2976– 2987, Hong Kong, China. Association for Comp...
work page 2019
-
[5]
Not just classification: Recog- nizing implicit discourse relation on joint modeling of classification and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natu- ral Language Processing, pages 2418–2431, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Prannay Khosla, Piotr Teterwak, Chen Wang,...
work page 2021
-
[9]
Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. Wanqiu Long and Bonnie Webber
arXiv 1907
-
[11]
TED-CDB: A large-scale Chinese discourse relation dataset on TED talks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 2793–2803, On- line. Association for Computational Linguistics. Linh The Nguyen, Ngo Van Linh, Khoat Than, and Thien Huu Nguyen
work page 2020
-
[12]
QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2804–2819, Online. Association for Computational Linguistics. Lianhui Qin, Zhisong Zhang, and Hai Zhao
work page 2020
-
[13]
Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguistics: Main V olume, pages 255–269, Online. Association for Computa- tional Linguistics. Wei Shi and Vera Demberg. 2019a. Learning to explici- tate connective...
work page 2019
-
[14]
AutoPrompt: Elic- iting Knowledge from Language Models with Auto- matically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Online. Association for Computational Linguistics. Jake Snell, Kevin Swersky, and Richard S. Zemel
work page 2020
-
[15]
The penn discourse treebank 3.0 annota- tion manual. Changxing Wu, Liuwen Cao, Yubin Ge, Yang Liu, Min Zhang, and Jinsong Su. 2021a. A label dependence- aware sequence generation model for multi-level im- plicit discourse relation recognition. arXiv preprint arXiv:2112.11740. Changxing Wu, Liuwen Cao, Yubin Ge, Yang Liu, Min Zhang, and Jinsong Su. 2021b. ...
Show all 17 references
-
[16]
A label dependence- aware sequence generation model for multi-level im- plicit discourse relation recognition. In AAAI. Wei Xiang, Bang Wang, Lu Dai, and Yijun Mo. 2022a. Encoding and fusing semantic connection and lin- guistic evidence for implicit discourse relation recog- n...
2022
-
[17]
In Findings of the Association for Computational Linguistics: EMNLP 2022 , pages 3848–3858, Abu Dhabi, United Arab Emirates
Prompt-based connective pre- diction method for fine-grained implicit discourse relation recognition. In Findings of the Association for Computational Linguistics: EMNLP 2022 , pages 3848–3858, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics. Ran Z...
2022
-
[2016]
In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 1224–1233, Austin, Texas
Recognizing implicit discourse relations via repeated reading: Neural net- works with multi-level attention. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 1224–1233, Austin, Texas. Association for Computational Linguistics. ...
2016
-
[2017]
In Proceedings of the 2017 Conference on Empirical Methods in Natu- ral Language Processing, pages 1299–1308, Copen- hagen, Denmark
Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In Proceedings of the 2017 Conference on Empirical Methods in Natu- ral Language Processing, pages 1299–1308, Copen- hagen, Denmark. Association for Computational ...
2017
-
[2019]
CoRR, abs/1911.02116
Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. Ganqu Cui, Shengding Hu, Ning Ding, Longtao Huang, and Zhiyuan Liu
1911 arXiv
-
[2020]
arXiv preprint arXiv:2004.11362
Supervised contrastive learning. arXiv preprint arXiv:2004.11362. Diederick P Kingma and Jimmy Ba
2004 arXiv
-
[2021]
In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 1275–1283, Punta Cana, Dominican Republic
CV AE-based re-anchoring for implicit discourse re- lation classification. In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 1275–1283, Punta Cana, Dominican Republic. Asso- ciation for Computational Linguistics. Tianyu Gao, Adam Fisch, and Danqi Chen
2021
-
[2022]
In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 10704– 10716, Abu Dhabi, United Arab Emirates
Facilitating contrastive learning of discourse relational senses by exploiting the hierarchy of sense relations. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 10704– 10716, Abu Dhabi, United Arab Emirates. Associa- tion fo...
2022
-
[2023]
ArXiv, abs/2305.03973
Discoprompt: Path prediction prompt tuning for implicit discourse relation recogni- tion. ArXiv, abs/2305.03973. Jifan Chen, Qi Zhang, Pengfei Liu, and Xuanjing Huang. 2016a. Discourse relations detection via a mixed generative-discriminative framework. In AAAI. Jifan Chen, Qi...
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.