BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction

Taifeng Wang; Wei Chu; Weipeng Huang; Xingyi Cheng

REVIEW 4 major objections 5 minor 19 references

BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports a joint entity-relation extraction system that reaches F1 0.876 with a single model and 0.892 with a four-model ensemble by layering BERT, weakly supervised NER pretraining, soft label embedding, and a global relation…

desk verdict A useful competition report with one small genuine mechanism, but the NER-pretraining gain has a concrete data-leakage risk that should be addressed before trusting the ablation. read the letter →

arxiv 1908.05908 v2 pith:QW3SNYCZ submitted 2019-08-16 cs.CL

Weipeng Huang , Xingyi Cheng , Taifeng Wang , Wei Chu This is my paper · ORCID

classification cs.CL

keywords jointentity-relationextractionmulti-headselectionBERTsoftlabelembeddingweaklysupervisedNERpretrainingschema-basedinformationSKEdatasetmodelensemble

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a competition report claiming that joint entity-relation extraction can be made substantially stronger by taking the multi-head selection framework and adding three modifications: BERT as the feature extractor with an extra semantic pretraining task, NER pretraining on about six million weakly labeled Baidu Baike sentences where article titles act as pseudo entity labels, and soft label embeddings computed from CRF logits instead of hard decoded tags. The claimed payoffs are F1 0.876 for a single model and F1 0.892 for an ensemble of four variants on the SKE test set, results that placed first and second in the 2019 Language and Intelligence Challenge. A sympathetic reader would take this as evidence that the hard-label BERT baseline is not saturated: the ablations in Table 1 show each component adding a small positive increment, and the gains compounding when combined. The findings matter because one entity participating in multiple triplets and overlapping entity spans are exactly the cases where earlier tagging-based joint models fail, and the two cheap tricks (weak pretraining and soft labels) are transferable to other extraction tasks.

What carries the argument

The load-bearing object is the multi-head selection relation classifier mounted on top of BERT and a CRF entity tagger. Each token gets a subject-oriented vector $h_i^s$ and an object-oriented vector $h_i^o$, and for each ordered pair $(i,j)$ a multi-sigmoid layer predicts all relations $r_{i,j}$, which is what lets one entity anchor multiple triplets and lets spans overlap. The named new mechanism is soft label embedding, formally $h_i = \sum \operatorname{softmax}(s(X,i)) \cdot \mathbf{M} / N$, which replaces the hard CRF-decoded tag with a softmax-weighted mixture of label embeddings so the connection between entity recognition and relation extraction remains differentiable. The auxiliary sentence-level global relation prediction uses the [CLS] representation with a multi-sigmoid layer, and the combined training loss is $L = L_{ner} + L_{rel} + L_{global\,rel}$. Together these mechanisms convert the SKE extraction problem into a per-token-pair classification problem that the reported ablations show responds positively to each added component.

What would settle it

Running the full pipeline on the SKE development set with and without Baidu Baike NER pretraining, repeated across several random seeds, could falsify the pretraining claim if the F1 difference falls within seed noise. Replacing soft label embeddings with hard decoded labels while keeping BERT and the auxiliary tasks fixed would similarly test whether soft labels carry the relation-branch gain claimed in Table 1.

Watch

Extended reading notes

Core claim

The central discovery is that the multi-head selection architecture, originally built around BiLSTM and hard label embeddings, can be pushed to new precision by replacing the encoder with BERT and by making the NER-to-relation information path softer and richer. The NER branch is a linear-chain CRF over BERT character representations; the relation branch predicts, for every ordered token pair $(i,j)$, a multi-sigmoid distribution over all relation types, so a single token can be part of several triplets at once. Soft label embedding replaces the non-differentiable argmax of Viterbi decoding with a probability-weighted average of label embeddings, keeping the entity-to-relation transfer differentiable and avoiding the propagation of hard tagging errors. The paper also adds two auxiliary signals: a semantic-enhanced BERT pretraining objective that includes previous-sentence and document-level prediction, and a sentence-level global relation prediction from the [CLS] vector. On SKE, the baseline BERT model with hard labels scores F1 0.864; each modification raises it, and the full single model reaches F1 0.876, while ensembling four variants with XGBoost over hand-designed triplet features reaches F1 0.892 on test set 1 and 0.8924 on test set 2.

Load-bearing premise

The load-bearing premise is that treating a Baidu Baike article title as a pseudo entity label for the entity mentioned in the article provides useful NER supervision; if those titles are often not actual entity mentions, or if their types match SKE's types only accidentally, the reported NER-pretraining gain could be dataset-specific.

Editorial extensions

If this is right

The ablations in Table 1 imply that no single modification dominates: each raised F1 by roughly 0.2–0.5 points over the hard-label BERT baseline, and the full combination is what produced the 0.876 score.
The ensemble result implies that the four base models make complementary errors, because combining them via XGBoost over triplet-level features gained 1.6 F1 points beyond the best single model.
Because relations are predicted per ordered token pair with multi-sigmoid heads, the method can handle one entity in multiple triplets and overlapping entity spans, the two cases the paper identifies as hard for tagging-based joint models.
The preprocessing and postprocessing rules for incomplete book-title entities and date entities were applied consistently to dev and test, so the reported gains reflect the model and the rules working together rather than test-only cleanup.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The weak-supervision result suggests a concrete transfer test: crawl an encyclopedia corpus in another language, use article titles as pseudo entity labels, and check whether the same NER pretraining gain appears on a non-Chinese schema-based IE benchmark; the paper's mechanism predicts it would, but the paper does not test this.
The soft label embedding gain could be partly a smoothing or regularization effect rather than a semantic one; comparing softmax-weighted embeddings with temperature-scaled or uniformly weighted label embeddings would separate the two.
One ensemble feature, whether a candidate triplet appears in the training set, may reward memorization; ablating that feature from the XGBoost model would clarify how much of the final gain is genuine generalization.
If title-pseudo-label pretraining transfers, then building NER supervision for new relation schemas could be automated by collecting encyclopedia articles whose titles match the target entity types, lowering annotation cost for future extraction tasks.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a competition write-up from the 2019 Language and Intelligence Challenge, and it reads like one. The model is BERT inserted into Bekoulis et al.'s multi-head selection framework, plus three tweaks: an enhanced BERT pretraining objective, pseudo-label NER pretraining on 6M Baidu Baike sentences, and soft label embedding. The only genuinely new piece is soft label embedding (Eq. 3): taking CRF logits, softmaxing them, and using the weighted sum as input to the relation branch instead of hard decoded tags. That is a small but reasonable idea, and Table 1 shows it adding a small positive increment. The paper is honest about being an extension of an existing framework, is clearly written, and the math is straightforward. I have no issue with the multi-head selection baseline or the ensemble recipe.

The largest soft spot is a leakage risk in the NER pretraining step. The SKE dataset is itself extracted from Baidu Baike and Baidu News Feeds, and the pretraining corpus is 6M sentences auto-crawled from Baidu Baike. No deduplication against SKE is reported. If even a small fraction of test sentences appear in the pretraining corpus, the test-1 gain from NER pretraining could be memorization rather than transfer. This is testable: the authors should report overlap statistics and rerun the ablation after exact and fuzzy dedup. As written, the NER pretraining contribution is suspect, and since the 'Baseline+all' result builds on it, the headline 0.876 inherits some doubt.

Second, there are no error bars or multiple seeds. The ablation differences are all in the 0.002 to 0.012 range on test-1, and without variance we cannot tell which differences are meaningful. The paper also delegates the enhanced BERT objective to a separate paper (ref 19) and does not release code or models, so reproducing the central result from the text alone would take meaningful effort.

That said, the paper is not fatally flawed. The core framework is sound, the soft label embedding is a real contribution, and the SKE benchmark is external, so there is no direct fitting to the test labels. The leakage concern is a red flag, not a smoking gun.

This paper is for people working on joint entity-relation extraction, especially for Chinese text, and for anyone building on multi-head selection. It is a useful recipe paper, not a foundational one. If I were the area chair, I would send it to review and explicitly ask for deduplication analysis and at least seed-variance numbers. Without those, the NER pretraining claim should not be accepted as solid.

Referee Report

4 major / 5 minor

Summary. The paper describes a joint entity-relation extraction system for the 2019 Language and Intelligence Challenge. The authors replace the BiLSTM encoder of Bekoulis et al.'s multi-head selection model with BERT and add three modifications: a semantic-enhanced BERT pretraining objective (Section 3.2), NER pretraining on roughly 6M Baidu Baike sentences using article titles as pseudo labels (Section 3.3), and soft label embedding that feeds CRF logits rather than decoded tags into the relation classifier (Section 3.4). Ablations on the SKE benchmark (Table 1) show each modification improving F1 over a BERT hard-label baseline, with a best single-model testset-1 F1 of 0.876 and an XGBoost ensemble of four variants reaching 0.892 on testset-1 and 0.8924 on testset-2.

Significance. If the reported results are reliable, the paper documents a strong practical configuration for Chinese joint extraction and a useful idea in soft label embedding; the differentiable, probability-weighted transmission from NER to relation classification is a clean and plausible improvement over hard label embeddings. The paper also has the merit of reporting ablations on a single external benchmark rather than fitting a target result, and the ensemble design with XGBoost over triplet samples is specific and implemented. However, the significance is substantially discounted by the absence of released code or models, single-run numbers without error bars, and the unresolved overlap between the NER pretraining corpus and SKE, which directly affects the attribution of the NER-pretraining gain. The contribution is therefore conditional on a leakage analysis and reproducibility checks.

major comments (4)

[§3.3, §4.1–4.2, Table 1] The paper states that SKE sentences are extracted from Baidu Baike and Baidu News Feeds, while the extra NER pretraining corpus is auto-crawled from Baidu Baike, but no overlap removal or overlap statistics are reported. Because the same source domain is used, SKE sentences, including those in testset-1, could appear verbatim or near-verbatim in the pretraining corpus, and the reported testset-1 gain from 0.864 to 0.868 attributed to NER pretraining could then be due to memorization instead of transferable entity recognition. Please report exact and near-duplicate overlap between the pretraining corpus and the SKE train/dev/test splits, and show that the NER-pretraining gain survives after removing overlapping sentences or that the gain is absent on the deduplicated subset.
[§3.2, Table 1] The 'Enhanced BERT' component is not self-contained: the semantic-enhanced pretraining objective is only referenced to the authors' companion paper [19], with no definition of the auxiliary task, its training data, or a model release. Since Table 1 attributes a 0.007 testset-1 F1 improvement to this component, the reader cannot evaluate or reproduce it from the manuscript. Please either specify the semantic-enhanced task and its pretraining setup in sufficient detail, or explicitly state that this component is exactly the model of [19] and make that model available.
[§4.2, Table 1] All ablations report a single run, and the component increments are as small as 0.002–0.004 F1 on testset-1, for example 'Baseline+Global Predicate Prediction' at 0.866 versus baseline 0.864. Without multiple seeds or confidence intervals, the claim that every listed modification contributes a positive increment is not statistically supported, and the relative ordering of the weaker components could be noise. Please report the mean and standard deviation over at least three random seeds for the baseline and each variant, or otherwise provide significance evidence.
[§3.4, Eq. (3)] The soft label embedding formula as written is ambiguous: it lacks an explicit summation index and divides by N, the number of entity types, which is not part of a standard convex combination of label embeddings. Please rewrite the equation with an explicit sum over tag classes and justify or remove the 1/N factor; if the intended operation is a softmax-weighted sum without the 1/N factor, the current formula misleads reproduction of a central claimed contribution.

minor comments (5)

[Abstract, §3.3, Keywords] The phrase 'weekly supervised learning' should be 'weakly supervised learning'; the same typo appears in the keywords and in Section 3.3.
[§4.3] The sentence 'We contact the sentence and the triplet to train an NLI model' should likely read 'concatenate'; the formulation as written is unclear.
[§4.1] The hyperparameter paragraph lists max sequence length, optimizer, learning rate, and dropout, but it does not state the random seed or the exact number of training epochs used for the reported runs; please add these details.
[§4.3, Figure 5] The precision-recall curve in Figure 5 would be more informative if the PR curves of the individual base models were also plotted, since the reader currently cannot see how the ensemble changes the precision/recall tradeoff relative to its components.
[§2] The claim that the multi-turn QA framework of Li et al. (2019) 'is also of low computational efficiency' is made without a comparison of inference cost to the proposed model; please add a quantitative or cited basis for this statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-level circularity: headline F1 scores are measured on the external SKE benchmark and no fitted parameter is renamed as a prediction.

full rationale

The paper's central claims are empirical F1 numbers on the SKE test sets. Each row of Table 1 is a held-out measurement of a distinct model variant (baseline, +enhanced BERT, +NER pretraining, +soft label embedding, +global predicate prediction, +all); none of these rows is derived algebraically from the model definition or from the ablation inputs. Equations (1)-(6) define a loss/architecture, and the reported outputs are evaluated against ground-truth triples, so there is no step where a predicted quantity is substituted back into the input that produced it. The NER pretraining with Baidu Baike titles as pseudo-labels is weak supervision; the reported test-1 F1 gain is a test-set measurement, not a construction. The possibility that the Baidu Baike pretraining corpus overlaps the SKE corpus (also Baidu Baike/News) is a data-contamination validity risk, not a circular reduction. The only self-referential element is the 'semantic-enhanced BERT' contribution: Section 3.2 says 'We pre-train BERT by combining MLM, NSP and the semantic-enhanced task together [19]', and Ref. [19] shares an author with this paper. That is a missing-specification/independence concern for reproducibility, but the F1 gain attributed to Enhanced BERT is still measured in this paper's Table 1 rather than imported from the citation. No derivation step reduces by construction to its own input, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper contributes empirical components on top of a standard framework; its main claim depends on standard machine learning assumptions rather than invented quantities. The explicit hyperparameters are hand-set, and the weakest assumptions concern pseudo-label quality and representation transfer.

free parameters (6)

learning rate = 2e-5
Chosen by hand for Adam optimization; standard for BERT fine-tuning but affects the final F1.
dropout probability = 0.1
Chosen by hand and applied during training; affects both NER and relation branches.
max sequence length = 128
Chosen for efficiency; sentences longer than 128 are split by punctuation, which changes the available context for extraction.
relation classifier layers = 2
Number of fully connected layers in the relation classification branch, set in Section 3.5 without a sweep.
global relation classifier layers = 1
Number of fully connected layers in the auxiliary global relation prediction branch, set in Section 3.5 without a sweep.
training epochs = 3
The reported convergence point; no early stopping or epoch selection details are given.

assumptions (5)

standard math The linear-chain CRF score model in Equation 1 is a valid conditional model for sequence labeling.
Used without proof in Section 3.3 as a standard formulation.
domain assumption BERT representations transfer to Chinese character-level NER and relation extraction after fine-tuning.
Taken as given from prior work in Section 3.2; the paper does not test this premise independently.
domain assumption Baidu Baike article titles are useful pseudo entity labels for NER pretraining.
Introduced in Section 3.3; if titles are not entity mentions, the claimed NER pretraining gain is not general.
domain assumption Softmax of CRF logits captures label uncertainty useful for the relation branch.
Core assumption of soft label embedding in Section 3.4; no theoretical or analytical justification is given.
domain assumption SKE ground truth labels are correct and consistent with the predefined schema.
Necessary for all F1 measurements in Section 4; the paper relies on the challenge organizers' annotation quality.

how reviews work

0 comments

Cite this review

Pith. "Pith review of BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction." pith.science (2026). https://pith.science/paper/QW3SNYCZ

@misc{pith2026190805908,
  author       = {Pith},
  title        = {Pith review of: BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QW3SNYCZ}},
  note         = {Machine review of arXiv:1908.05908}
}

read the original abstract

In this paper, we report our method for the Information Extraction task in 2019 Language and Intelligence Challenge. We incorporate BERT into the multi-head selection framework for joint entity-relation extraction. This model extends existing approaches from three perspectives. First, BERT is adopted as a feature extraction layer at the bottom of the multi-head selection framework. We further optimize BERT by introducing a semantic-enhanced task during BERT pre-training. Second, we introduce a large-scale Baidu Baike corpus for entity recognition pre-training, which is of weekly supervised learning since there is no actual named entity label. Third, soft label embedding is proposed to effectively transmit information between entity recognition and relation extraction. Combining these three contributions, we enhance the information extracting ability of the multi-head selection model and achieve F1-score 0.876 on testset-1 with a single model. By ensembling four variants of our model, we finally achieve F1 score 0.892 (1st place) on testset-1 and F1 score 0.8924 (2nd place) on testset-2.

Figures

Figures reproduced from arXiv: 1908.05908 by the authors.

**Figure 2.** Examples: entity contained in book title marks can be song or book, its relation to a person can be singer or writer [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Overall Framwork: BERT-Based Multi-Head Selection [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Crawled corpus from Baidu Baike. Extra Corpus for NER Pretraining Previous works show that introducing extra data for distant supervised learning usually boost the model performance. For this task, we collect a large-scale Baidu Baike corpus (about 6 million sentences…

**Figure 5.** Figure 5: Precision-Recall Curve 4.3 Model Ensemble We select the following four variant model to further conduct model ensembling. The ensemble model is XGBoost binary classifier, which is very fast during training. Since the base models are trained on the training set, we per…

**Figure 6.** Figure 6: Examples the model fails to predict. 5 Conclusion In this paper, we report our solution to the information extraction task in 2019 Language and Intelligence Challenge. We first analyze the problem and find that most entities are involved in multiple triplets. To solve …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 16 canonical work pages

[19]

Symmetric Regularization based BERT for Pair-wise Semantic Reasoning

Cheng X, Xu W, Chen K, et al. Symmetric Regularization based BERT for Pair- wise Semantic Reasoning. arXiv preprint arXiv:1909.03405, 2019

work page Pith review arXiv 1909
[1]

Bert: Pre-training of deep bidirectional transformers for lan- guage understanding

Devlin, Jacob, et al. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805 (2018)

arXiv 2018
[2]

Hendrickx, Iris, et al. ”Semeval-2010 task 8: Multi-way classiﬁcation of semantic relations between pairs of nominals.” Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. Association for Compu- tational Linguistics, 2009

work page 2010
[3]

Relation classiﬁcation via convolutional deep neural network[J]

Zeng D, Liu K, Lai S, et al. Relation classiﬁcation via convolutional deep neural network[J]. 2014

work page 2014
[4]

Semantic relation classiﬁcation via convolutional neural networks with simple negative sampling[J]

Xu K, Feng Y, Huang S, et al. Semantic relation classiﬁcation via convolutional neural networks with simple negative sampling[J]. arXiv preprint arXiv:1506.07650, 2015

arXiv 2015
[5]

”Kernel methods for relation extraction.” Journal of machine learning research 3.Feb (2003): 1083-1106

Zelenko, Dmitry, Chinatsu Aone, and Anthony Richardella. ”Kernel methods for relation extraction.” Journal of machine learning research 3.Feb (2003): 1083-1106

work page 2003
[6]

Miwa, Makoto, et al. ”A rich feature vector for protein-protein interaction extraction from multiple corpora.” Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1. Association for Computational Linguistics, 2009

work page 2009
[7]

Chan, Yee Seng, and Dan Roth. ”Exploiting syntactico-semantic structures for re- lation extraction.” Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 2011

work page 2011

Show all 19 references

[8]

Yu, Xiaofeng, and Wai Lam. ”Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach.” Proceedings of the 23rd Inter- national Conference on Computational Linguistics: Posters. Association for Com- putational Linguistics, 2010

2010
[9]

”Modeling joint entity and relation extraction with table representation.” Proceedings of the 2014 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP)

Miwa, Makoto, and Yutaka Sasaki. ”Modeling joint entity and relation extraction with table representation.” Proceedings of the 2014 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP). 2014

2014
[10]

”Incremental joint extraction of entity mentions and relations.” Proceedings of the 52nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers)

Li, Qi, and Heng Ji. ”Incremental joint extraction of entity mentions and relations.” Proceedings of the 52nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). Vol. 1. 2014

2014
[11]

Gupta, Pankaj, Hinrich Schutze, and Bernt Andrassy. ”Table ﬁlling multi-task recurrent neural network for joint entity and relation extraction.” Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 2016

2016
[12]

Zheng, Suncong, et al. ”Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme.” Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017

2017
[13]

Zeng, Xiangrong, et al. ”Extracting relational facts by an end-to-end neural model with copy mechanism.” Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018

2018
[14]

”Entity-Relation Extraction as Multi-Turn Question Answering.” arXiv preprint arXiv:1905.05529 (2019)

Li, Xiaoya, et al. ”Entity-Relation Extraction as Multi-Turn Question Answering.” arXiv preprint arXiv:1905.05529 (2019)

2019 arXiv
[15]

”Joint entity recognition and relation extraction as a multi- head selection problem.” Expert Systems with Applications 114 (2018): 34-45

Bekoulis, Giannis, et al. ”Joint entity recognition and relation extraction as a multi- head selection problem.” Expert Systems with Applications 114 (2018): 34-45

2018
[16]

”Conditional ran- dom ﬁelds: Probabilistic models for segmenting and labeling sequence data.” (2001)

Laﬀerty, John, Andrew McCallum, and Fernando CN Pereira. ”Conditional ran- dom ﬁelds: Probabilistic models for segmenting and labeling sequence data.” (2001). 12 Weipeng Huang, Xingyi Cheng, Taifeng Wang, Wei Chu

2001
[17]

”End-to-end relation extraction using lstms on sequences and tree structures.” arXiv preprint arXiv:1601.00770 (2016)

Miwa, Makoto, and Mohit Bansal. ”End-to-end relation extraction using lstms on sequences and tree structures.” arXiv preprint arXiv:1601.00770 (2016)

2016 arXiv
[18]

”Xgboost: A scalable tree boosting system.” Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining

Chen, Tianqi, and Carlos Guestrin. ”Xgboost: A scalable tree boosting system.” Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. ACM, 2016

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [19]

Symmetric Regularization based BERT for Pair-wise Semantic Reasoning

Cheng X, Xu W, Chen K, et al. Symmetric Regularization based BERT for Pair- wise Semantic Reasoning. arXiv preprint arXiv:1909.03405, 2019

work page Pith review arXiv 1909

[2] [1]

Bert: Pre-training of deep bidirectional transformers for lan- guage understanding

Devlin, Jacob, et al. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805 (2018)

arXiv 2018

[3] [2]

Hendrickx, Iris, et al. ”Semeval-2010 task 8: Multi-way classiﬁcation of semantic relations between pairs of nominals.” Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. Association for Compu- tational Linguistics, 2009

work page 2010

[4] [3]

Relation classiﬁcation via convolutional deep neural network[J]

Zeng D, Liu K, Lai S, et al. Relation classiﬁcation via convolutional deep neural network[J]. 2014

work page 2014

[5] [4]

Semantic relation classiﬁcation via convolutional neural networks with simple negative sampling[J]

Xu K, Feng Y, Huang S, et al. Semantic relation classiﬁcation via convolutional neural networks with simple negative sampling[J]. arXiv preprint arXiv:1506.07650, 2015

arXiv 2015

[6] [5]

”Kernel methods for relation extraction.” Journal of machine learning research 3.Feb (2003): 1083-1106

Zelenko, Dmitry, Chinatsu Aone, and Anthony Richardella. ”Kernel methods for relation extraction.” Journal of machine learning research 3.Feb (2003): 1083-1106

work page 2003

[7] [6]

Miwa, Makoto, et al. ”A rich feature vector for protein-protein interaction extraction from multiple corpora.” Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1. Association for Computational Linguistics, 2009

work page 2009

[8] [7]

Chan, Yee Seng, and Dan Roth. ”Exploiting syntactico-semantic structures for re- lation extraction.” Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 2011

work page 2011