Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Transfer Learning for Relation Extraction via Relation-Gated Adversarial Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Relation-gated adversarial learning lets relation extractors transfer across domains with different label sets.

desk verdict Plausible partial-DA recipe for relation extraction, with a useful new dataset, but the core equations are sloppy and the most relevant baselines are missing. read the letter →

arxiv 1908.08507 v1 pith:BS6MH56Z submitted 2019-08-22 cs.LG cs.CLstat.ML

classification cs.LGcs.CLstat.ML
keywords relationextractiontransferlearningdomainadaptationpartialadversarialdistantsupervisiongatenegative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes relation-gated adversarial learning (R-Gated), a domain adaptation method for relation extraction that transfers labeled source data to an unlabeled target domain even when the source has extra relation types the target lacks. It argues that a single adversarial alignment is not enough: source classes and instances differ in how transferable they are, so the model learns to downweight non-transferable source data at both the relation-category level and the instance level. The gate mechanism balances these two granularities per target sentence. If correct, this makes it practical to adapt relation extractors from general knowledge-base text to specialised domains and to clean noisy distant-supervision labels by fine-tuning.

What carries the argument

The load-bearing mechanism is the relation-gate defined by \(\$\alpha$ = \$\sigma$(W_r F_t(x))\), which mixes two weight sources. Category weights average the frozen source classifier's predictions over all target sentences, estimating how useful each source relation class is for the target. Instance weights take \(w_i = 1 - D(f)\), where \(D\) is a pretrained auxiliary domain discriminator; source instances that the discriminator can easily separate from the target get low weight. The total source weight \($w_i^{{total}}$ = \$\alpha$ $w_i^{{instance}}$ + (1-\$\alpha$) $w_j^{{category}}$\) is fed into a reweighted adversarial domain loss, and the gate's \(\$\alpha$\) lets different target relations rely on different granularities.

What would settle it

Run the same R-Gated pipeline with a source classifier whose softmax is deliberately made miscalibrated on the target distribution (for instance, by temperature scaling that makes it confident on outlier classes), and check whether the category weights change in the wrong direction and partial-DA F1 drops. A more direct version: on a synthetic partial-DA benchmark with known target label distribution, compare category weights from Eq. 2 with the true class frequencies; if they diverge substantially on outlier classes while performance holds, the mechanism is not doing what the paper claims.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that partial domain adaptation for relation extraction can be handled by weighting source training data with a relation-gate that combines category-level weights, computed as the average of a frozen source classifier's soft predictions over target data, with instance-level weights derived from an auxiliary domain discriminator. The gate predicts a per-target-sentence mixing coefficient \(\$\alpha$ = \$\sigma$(W_r F_t(x))\) that decides whether category or instance evidence matters more. Empirically, this yields average F1 of 58.92 on ACE05 partial domain adaptation versus 56.68 for the adversarial baseline, top-500 average precision of 0.80 on Wiki-NYT versus 0.74, and fine-tuning with 25% of target labels matching models trained from scratch with roughly ten times more target data.

Load-bearing premise

The whole method assumes the frozen source classifier's averaged predictions over unlabeled target data faithfully indicate which source relation types are actually useful in the target domain; if that classifier is systematically confident about the wrong classes on target text, the category weights will down-weight the very relations that should transfer.

Editorial extensions

If this is right

  • A relation extractor pretrained on a large general corpus can be adapted to a narrower target domain with no target labels, provided the target's relation types are a subset of the source's.
  • Adding a small amount of labeled target data, around 25% of the target set, matches training from scratch with roughly ten times more data.
  • The same weighting mechanism can reduce the impact of wrong distant-supervision labels in the target by relying on higher-quality source labels.
  • Because the gate is learned per target sentence, the method can automatically decide which relations need fine-grained instance weighting and which need coarse category weighting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is that the category weights depend on the calibration of the frozen source classifier, so temperature scaling or entropy regularisation on target predictions could make the gate more robust.
  • A testable extension is to condition \(\alpha\) on the source class label as well as the target sentence, since the current gate cannot distinguish two source relations that are both similar to the target sentence but differ in their transferability.
  • The instance-weight formula is essentially a measure of domain similarity, so the method could be applied to other sequence-labeling tasks, such as named entity recognition or event extraction, where partial label-space overlap is common.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes R-Gated, a relation-gated adversarial learning framework for domain adaptation in relation extraction. It combines three components: an instance encoder, an adversarial domain discriminator, and a relation-gate that balances category-level weights (computed by averaging a frozen source classifier's softmax outputs over target data, Eq. 2) and instance-level weights (derived from an auxiliary domain discriminator, Eq. 4). The method is evaluated on ACE05 for normal and partial domain adaptation, and on a new Wiki-NYT distant-supervision dataset for unsupervised and supervised adaptation. The authors report F1 gains over CNN+DANN on ACE05 partial DA and top-500 precision improvements on Wiki-NYT, with ablations, parameter analysis, and case studies.

Significance. If the empirical claims are supported, the work addresses a genuine gap—partial domain adaptation for relation extraction—and the idea of combining category and instance weights with a learned relation gate is a reasonable contribution. The paper includes ablations, parameter analysis, and t-tests, and it explicitly notes that the weights are fixed after computation. However, the significance is undercut by the absence of the two partial-DA baselines whose ideas the method builds on, by the malformed instance-weight equation, and by the lack of aggregate validation of the category-weight mechanism. These issues must be resolved before the headline claim can be accepted.

major comments (4)
  1. [§3.4, Eq. (4)] The instance-weight formula is algebraically inconsistent as displayed: if the left-hand side is read literally, it is (1 / (D(F_s(x)) D(F_t(x)))) + 1, which cannot equal the stated result 1 - D(f); the intended expression is unrecoverable from the text. Because Eq. (4) defines the instance-weight mechanism that, together with Eq. (2), drives the reported gains, this must be stated unambiguously for the method to be reproducible.
  2. [§4.3–§4.4, Tables 1–2] The headline claim that the approach 'outperforms previous domain adaptation methods regarding partial domain adaptation' (Abstract) is not supported by the experiments, because the only adversarial DA baseline is CNN+DANN. The two partial-DA methods whose weighting ideas the paper adopts—PADA (Cao et al., 2018) and IWAN (Zhang et al., 2018a)—are described in Section 2 as potentially transferable to RE but are never evaluated on ACE05 or Wiki-NYT. Without these comparisons, the experiments can at most support superiority over CNN+DANN, not over 'previous domain adaptation methods' as claimed.
  3. [§3.4, Eq. (2)] The category-weight mechanism is load-bearing: the ablation in Table 3 shows that removing category weights changes average top-500 precision from 0.80 to 0.74 on Wiki-NYT. Yet the only evidence that Eq. (2) correctly identifies outlier classes is the selected examples in Table 4. Because the frozen source classifier is not calibrated on the shifted target distribution, averaging its softmax outputs could mis-weight classes; the paper should provide an aggregate comparison of category weights for known shared vs. outlier classes (e.g., across random target-class subsets) and a sensitivity or calibration analysis. Without this, the mechanism claimed to drive the partial-DA gain remains an unvalidated assumption.
  4. [§3.4, Eqs. (5)–(6)] The relation-gate α is computed as σ(W_r F_t(x)) in Eq. (6), but the argument x is not specified: Eq. (5) uses α as a scalar applied to each source instance, whereas the motivation in Section 3.4, Figure 5(a), and Table 4 treat α as a per-relation quantity. This ambiguity affects the interpretation of the ablation study (w/o gate, w/o category, w/o instance) and of the development-set tuning described in Section 4.1. Please clarify whether α is a per-instance or per-relation variable and how it is trained or inferred.
minor comments (5)
  1. [Figure 2 caption] The caption contains the typo 'fowardprop backprop'; it should read 'forward propagation/backpropagation'.
  2. [Abstract and §4.4] The text uses 'distance supervised' in several places; the standard term is 'distant supervision' or 'distantly supervised'.
  3. [§4.1] The statement that the development set is used to 'fine-tune hyper-parameters such as α' conflicts with α being computed from a learned weight matrix W_r in Eq. (6); please reconcile whether α is a learned output or a free hyper-parameter.
  4. [§4.4] The new Wiki-NYT dataset is not released and its construction is described only as 'We filter 60 shared relations'; for reproducibility, provide a detailed construction protocol and make the dataset available to reviewers.
  5. [§2] The claim that the approach is 'the first partial DA work in RE even in NLP' is broader than the evidence presented; consider softening it, as no partial-DA NLP baselines are compared.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are measured on held-out target data and the gating weights are derived from unsupervised predictions, not from the test labels or evaluation metric.

full rationale

The derivation chain is self-contained in the empirical sense required here. The paper pretrains a source encoder and classifier (Eq. 1), computes category weights by averaging the frozen source classifier's softmax outputs over unlabeled target data (Eq. 2), computes instance weights from an auxiliary discriminator (Eqs. 3-4), combines them through a relation-gate (Eqs. 5-6), and then trains the target encoder with adversarial objectives (Eq. 7). The reported results—F1 on ACE05 and top-k precision on Wiki-NYT—are computed on held-out target test data, which are never used in Eq. 2 or in the weight formulas. The category weights are an unsupervised heuristic, not a fit to target labels or to the evaluation metric. The relation-gate parameter α is learned, but no equation identifies α or the weights with the held-out F1 or precision; the ablation study reports empirical differences rather than definitional ones. The authors cite their own prior relation-extraction work (e.g., Zhang et al. 2018b and 2019) only as background on existing RE models; those citations are not load-bearing for the transfer-learning claim, and there is no imported uniqueness theorem or ansatz smuggled through self-citation. A residual concern that Eq. 2's averaged softmax weights may be miscalibrated under domain shift is a robustness and correctness issue, not a circularity issue, because the paper does not define its outcome in terms of those weights. Therefore no circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No scientific constants are fit; the only "free" object is a learned gate weight matrix, which is trained on data rather than chosen to match an external benchmark. The listed axioms are the domain assumptions that make the weighting scheme meaningful: unlabeled subset target space, source-classifier-based category weights, discriminator-based instance transferability, and convergence of adversarial training. None of these is proven; the first is a problem definition, and the final three are empirical assumptions common to domain adaptation.

assumptions (4)
  • domain assumption The target label space is a subset of the source label space, and target training data are unlabeled.
    Problem setup in Section 3.1; this defines the partial domain adaptation setting and justifies the need for category and instance weights.
  • domain assumption Averaged soft predictions of the frozen source classifier over target data estimate the usefulness of each source relation class.
    Used to compute category weights in Eq. 2 (Section 3.4). No calibration or sensitivity analysis is given.
  • domain assumption The auxiliary domain discriminator's confidence indicates whether a source instance is transferable, with perfectly separable source instances receiving near-zero weight.
    Basis for instance weights in Eq. 3-4 (Section 3.4). Eq. 4 as printed does not establish the stated relationship.
  • domain assumption Adversarial training with a gradient reversal layer converges to a useful feature alignment.
    Standard assumption inherited from Ganin et al. 2016; invoked in Eq. 7 and the training procedure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transfer Learning for Relation Extraction via Relation-Gated Adversarial Learning." pith.science (2026). https://pith.science/paper/BS6MH56Z

@misc{pith2026190808507,
  author       = {Pith},
  title        = {Pith review of: Transfer Learning for Relation Extraction via Relation-Gated Adversarial Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BS6MH56Z}},
  note         = {Machine review of arXiv:1908.08507}
}
read the original abstract

Relation extraction aims to extract relational facts from sentences. Previous models mainly rely on manually labeled datasets, seed instances or human-crafted patterns, and distant supervision. However, the human annotation is expensive, while human-crafted patterns suffer from semantic drift and distant supervision samples are usually noisy. Domain adaptation methods enable leveraging labeled data from a different but related domain. However, different domains usually have various textual relation descriptions and different label space (the source label space is usually a superset of the target label space). To solve these problems, we propose a novel model of relation-gated adversarial learning for relation extraction, which extends the adversarial based domain adaptation. Experimental results have shown that the proposed approach outperforms previous domain adaptation methods regarding partial domain adaptation and can improve the accuracy of distance supervised relation extraction through fine-tuning.

Figures

Figures reproduced from arXiv: 1908.08507 by the authors.

Figure 1
Figure 1. Knowlege transfer for RE from the general [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our approach. The parameters of the instance encoder for the source ( [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Unsupervised adaptation results. 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 Recall 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Precision CNN PCNN Rank+ExATT PCNN+R-Gated+100% [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Supervised adaptation results. Supervised Adaptation. Supervised Adapta￾tion does require labeled target data; however, the target labels might be few or noisy. In this set￾ting, we fine-tune our model with target labels. We report the results of our approach and vari￾…
Figure 5
Figure 5. Figure 5: Parameter analysis reuslts. formance degrades when we remove ”category weights” or ”instance weights.” This is reasonable because different weights have different effects in de-emphasizing those outlier classes or instances. Precision Top 100 Top 200 Top 500 Ave. PCNN+…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 35 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  4. [4]

    Zhangjie Cao, Mingsheng Long, Jianmin Wang, Michael I Jordan, and MOE KLiss. 2017. Partial transfer learning with selective adversarial networks. arXiv preprint arXiv:1707.07901

  5. [5]

    Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang. 2018. Partial adversarial domain adaptation. In Proceddings of ECCV, volume 1, page 4

  6. [6]

    Qingchao Chen, Yang Liu, Zhaowen Wang, Ian Wassell, and Kevin Chetty. 2018. Re-weighted adversarial adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7976--7985

  7. [7]

    Zihang Dai, Lei Li, and Wei Xu. 2016. Cfo: Conditional focused neural question answering with large-scale knowledge bases

  8. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

Show all 43 references
  1. [9]

    Lisheng Fu, Thien Huu Nguyen, Bonan Min, and Ralph Grishman. 2017. Domain adaptation for relation extraction with domain adversarial neural network. In Proceedings of IJCNLP, volume 2, pages 425--429

  2. [10]

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Fran c ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. JMLR, 17(1):2096--2030

  3. [11]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In processing of NIPS, pages 2672--2680

  4. [12]

    Zhengqiu He, Wenliang Chen, Zhenghua Li, Meishan Zhang, Wei Zhang, and Min Zhang. 2018. See: Syntax-aware entity embedding for neural relation extraction. In Proceedings of AAAI

  5. [13]

    Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao, et al. 2017. Distant supervision for relation extraction with sentence-level attention and entity descriptions. In AAAI, pages 3060--3066

  6. [14]

    Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of ACL, volume 1, pages 2124--2133

  7. [15]

    Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of ACL, pages 1003--1011. Association for Computational Linguistics

  8. [16]

    Ndapandula Nakashole, Martin Theobald, and Gerhard Weikum. 2011. Scalable knowledge harvesting with high precision and high recall. In Proceedings of WSDM, pages 227--236. ACM

  9. [17]

    Minh Luan Nguyen, Ivor W Tsang, Kian Ming A Chai, and Hai Leong Chieu. 2014. Robust domain adaptation for relation extraction via clustering consistency. In Proceedings of ACL, volume 1, pages 807--817

  10. [18]

    Thien Huu Nguyen and Ralph Grishman. 2014. Employing word representations and regularization for domain adaptation of relation extraction. In Proceedings of ACL, volume 2, pages 68--74

  11. [19]

    Thien Huu Nguyen and Ralph Grishman. 2016. Combining neural networks and log-linear models to improve relation extraction. In Proceedings of IJCAI Workshop DLAI

  12. [20]

    Thien Huu Nguyen, Barbara Plank, and Ralph Grishman. 2015. Semantic representations for domain adaptation: A case study on the tree kernel-based method for relation extraction. In Proceedings of ACL, volume 1, pages 635--644

  13. [21]

    Sinno Jialin Pan, Qiang Yang, et al. 2010. A survey on transfer learning. TKDE, 22(10):1345--1359

  14. [22]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP, pages 1532--1543

  15. [23]

    Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365

  16. [24]

    Barbara Plank and Alessandro Moschitti. 2013. Embedding semantic similarity in tree kernels for domain adaptation of relation extraction. In Proceedings of ACL, volume 1, pages 1498--1507

  17. [25]

    Pengda Qin, Weiran Xu, and William Yang Wang. 2018. Dsgan: Generative adversarial training for distant supervision relation extraction. In Proceedings of ACL

  18. [26]

    Anthony Rios, Ramakanth Kavuluru, and Zhiyong Lu. 2018. Generalizing biomedical relation classification with neural adversarial domain adaptation. Bioinformatics, 1:9

  19. [27]

    Michael T Rosenstein, Zvika Marx, Leslie Pack Kaelbling, and Thomas G Dietterich. 2005. To transfer or not to transfer. In NIPS 2005 workshop on transfer learning, volume 898, pages 1--4

  20. [28]

    Darsh J Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018. Adversarial domain adaptation for duplicate question detection. arXiv preprint arXiv:1809.02255

  21. [29]

    Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein distance guided representation learning for domain adaptation. In Proceddings of AAAI

  22. [30]

    Daniil Sorokin and Iryna Gurevych. 2017. Context-aware representations for knowledge base relation extraction. In Proceedings of EMNLP, pages 1784--1789

  23. [31]

    Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceddings of CVPR, volume 1, page 4

  24. [32]

    Ximei Wang, Liang Li, Weirui Ye, Mingsheng Long, and Jianmin Wang. 2019. Transferable attention for domain adaptation

  25. [33]

    Fei Wu and Daniel S Weld. 2010. Open information extraction using wikipedia. In Proceedings of ACL, pages 118--127. Association for Computational Linguistics

  26. [34]

    Hai Ye, Wenhan Chao, Zhunchen Luo, and Zhoujun Li. 2017. Jointly extracting relations with class ties via effective deep ranking. In Proceedings of ACL, volume 1, pages 1810--1820

  27. [35]

    Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of EMNLP, pages 1753--1762

  28. [36]

    Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING, pages 2335--2344

  29. [37]

    Wenyuan Zeng, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2017. Incorporating relation paths in neural relation extraction. In Proceddings of EMNLP

  30. [38]

    Xiangrong Zeng, Shizhu He, Kang Liu, and Jun Zhao. 2018. Large scaled relation extraction with reinforcement learning. In Processings of AAAI, volume 2, page 3

  31. [39]

    Dongxu Zhang and Dong Wang. 2015. Relation classification via recurrent neural network. arXiv preprint arXiv:1508.01006

  32. [40]

    Jing Zhang, Zewei Ding, Wanqing Li, and Philip Ogunbona. 2018 a . Importance weighted adversarial nets for partial domain adaptation. In Proceedings of CVPR, pages 8156--8164

  33. [41]

    Ningyu Zhang, Shumin Deng, Zhanlin Sun, Guanying Wang, Xi Chen, Wei Zhang, and Huajun Chen. 2019. Long-tail relation extraction via knowledge graph embeddings and graph convolution networks. arXiv preprint arXiv:1903.01306

  34. [42]

    Ningyu Zhang, Shumin Deng, Zhanling Sun, Xi Chen, Wei Zhang, and Huajun Chen. 2018 b . Attention-based capsule networks with dynamic routing for relation extraction. In Proceedings of EMNLP

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.