Pith. sign in

REVIEW 4 major objections 7 minor 28 references

Open Event Extraction from Online Text using a Generative Adversarial Network

T0 review · 4 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An adversarial generator-discriminator pair extracts open-domain event quadruples from tweets and news articles without supervision, beating Bayesian baselines LEM and DPEMM on all three test sets.

desk verdict New adversarial architecture for open event extraction, but the headline F-measure claim is not yet reproducible given the subjective evaluation and query-built Google dataset. read the letter →

arxiv 1908.09246 v1 pith:5WFOHVR7 submitted 2019-08-25 cs.CL cs.IRcs.LG

Rui Wang , Deyu Zhou , Yulan He This is my paper · ORCID
classification cs.CLcs.IRcs.LG
keywords open-domaineventextractiongenerativeadversarialnetworksunsupervisedlearningquadrupleDirichletpriorTwitterdetectionnewstraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that open-domain event extraction can be recast as an adversarial learning problem: a generator maps a per-document event mixture drawn from a Dirichlet prior into four word distributions - entity, location, keyword, and date - and a discriminator tries to separate the resulting fake document from the real one. The authors introduce the Adversarial-neural Event Model (AEM) and test it on two Twitter datasets and a news-article dataset, reporting that it beats K-means, LEM, and DPEMM on all three, with the largest gains on long news text, where F-measure rises by about 15 points over the Bayesian baseline LEM. If this holds, the payoff is an unsupervised extractor that produces human-readable event quadruples from both short and long text without requiring a single event per document or slow Gibbs sampling.

What carries the argument

The load-bearing mechanism is the generator's learned projection from the event-mixture simplex to event-component word distributions. AEM represents an event as a quadruple $\langle e, l, k, d \rangle$, draws a document's event mixture from $\vec{\theta} \sim \mathrm{Dir}(\vec{\theta}|\vec{\alpha})$, and transforms it through a layer-normalized LeakyReLU hidden layer and four softmax subnets, one per component, whose outputs are concatenated into a generated document. The discriminator, made Lipschitz-continuous by spectral normalization and trained with a gradient-penalty regularized loss, both supplies the learning signal and provides low-dimensional features for event visualization. At inference time, event extraction is a single forward pass of the generator on a one-hot event-seed vector $\vec{s}_t$.

What would settle it

Hold out several events with known descriptions from the Google dataset, train AEM on the rest, then check whether each one-hot event seed's output distributions top the words, people, and locations from its known event; if they do not, the event readout is refuted. A cheaper check is to feed two-blend event seeds and confirm that the output distributions interpolate smoothly rather than jumping between unrelated word sets.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is a working recipe for unsupervised structured event extraction: represent each document as a TF-IDF-weighted concatenation of entity, location, keyword, and date distributions; train a generator that takes a document-event distribution $\vec{\theta}$ drawn from a Dirichlet prior and outputs those four multinomial distributions; train a discriminator, regularized by spectral normalization and a gradient penalty, to distinguish original documents from generated ones; then read out events by feeding one-hot event-seed vectors into the trained generator. The extracted event for seed $\vec{s}_t$ is the quadruple of component distributions $G(\vec{s}_t) = [\vec{\varphi}_t^e; \vec{\varphi}_t^l; \vec{\varphi}_t^k; \vec{\varphi}_t^d]$. Across the FSD, Twitter, and Google news datasets, this recipe produces higher precision, recall, and F-measure than K-means, LEM, and DPEMM, with the clearest margin on long news articles.

Load-bearing premise

The load-bearing assumption is that the generator, trained only on random blends of events, will still return a correct single-event word distribution when it is handed a vector that selects exactly one event and nothing else; nothing in the paper tests that extrapolation.

Editorial extensions

If this is right

  • Event extraction no longer requires the single-event-per-document assumption, so long news articles can contribute to several event components at once.
  • Because inference is a forward pass rather than Gibbs sampling, AEM trains and runs in a fraction of the time of the Bayesian baselines, making larger corpora practical.
  • The discriminator's learned features give a label-free way to cluster and visualize documents by event, which the paper demonstrates with t-SNE plots.
  • The same architecture transfers across text types by re-slotting the quadruple, such as using organization, location, person, and keyword for news articles where date information is absent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: train on a mixture of continuous and one-hot event vectors and check whether extracted event quality improves, since the one-hot readout is an extrapolation the paper never validates.
  • A streaming consequence the paper only lists as future work: retrain or fine-tune AEM on rolling windows and treat the generator's event distributions as an evolving event inventory.
  • A label-free metric the paper does not report: cluster documents in discriminator feature space across time slices and measure cluster stability.
  • A likely transfer to neighbouring problems: the four-slot decomposition could be reused for jointly extracting product, sentiment, and reviewer aspects from reviews.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes AEM, an unsupervised GAN-based model for open-domain event extraction. Each document is represented as a concatenation of four TF-IDF weighted multinomial distributions over entities, locations, keywords, and dates; a generator maps a Dirichlet-sampled event distribution into these four word distributions, and a discriminator with spectral normalization and gradient penalty distinguishes reconstructed documents from real ones. At inference, one-hot event seeds are fed to the generator to obtain per-event quadruple distributions. Experiments compare AEM against K-means, LEM, and DPEMM on FSD, Twitter, and Google news datasets, claiming superior F-measure everywhere, with a 15.4-point improvement over LEM on Google. The paper also reports parameter sensitivity, t-SNE visualization, and faster training time.

Significance. If the empirical claims held, AEM would be a useful contribution: it is one of the first GAN formulations for open event extraction, it does not assume that each document is generated from a single event, and its neural inference is much faster than Gibbs sampling. The architecture and objective are described in enough detail to reimplement, and the method is genuinely unsupervised. However, the evaluation does not currently support the headline claim: the matching protocol is subjective and unreported, the Google dataset is constructed from query words that coincide with event keywords, and the one-hot readout step is an out-of-distribution extrapolation with no validation. The weaknesses are empirical rather than logical, so the central claim should be treated as conditional until these evaluation issues are addressed.

major comments (4)
  1. [§4.2, Table 1] The evaluation protocol is not reproducible. Precision and recall are defined via two subjective questions on page 7, but there is no deterministic matching rule, no list of gold quadruples, no number of annotators, and no inter-annotator agreement. Since the central claim that AEM outperforms all baselines rests entirely on these scores, the protocol must be operationalized (e.g., exact-match or F1 over gold quadruple elements, or release of the manually scored event lists) before the comparison can be verified.
  2. [§4.1, Google dataset] The Google dataset is assembled by retrieving documents that contain the target event's own query words (e.g., 'malaysia', 'airline', 'search', 'plane' for MH370). This makes the event-defining keywords artificially frequent in the corpus, so a model that outputs frequent terms for a seed can appear to match the gold events. The comparison with LEM/DPEMM on this dataset is therefore confounded by dataset construction; the authors should either construct the dataset by thread or date linkage without using the event keywords as queries, or demonstrate that the baseline methods are not disadvantaged by the construction.
  3. [§3.4, Eq. (14)] During training (Algorithm 1, line 6), θ is drawn continuously from a Dirichlet distribution, so one-hot vectors lie on the boundary of the training support and were never seen by the generator. The paper provides no experiment showing that G extrapolates coherently to these boundary inputs. If it does not, the per-event distributions φ are artifacts of out-of-distribution evaluation. Please validate by (a) comparing one-hot readout against the generator output for θ concentrated near each vertex, or (b) using a validated readout such as the mean θ of documents assigned to each event.
  4. [§4.2, Figure 3 and Table 1] All results are reported from what appears to be a single run, with no error bars, variance, or significance tests. The parameter sensitivity study on the Google dataset shows that AEM's F-measure varies between 85.7% and 96.7% depending on architecture, so the margin over LEM (72.4%) may depend on favorable settings. Report multiple restarts and a statistical comparison, especially for the headline 15-point improvement.
minor comments (7)
  1. [Abstract] The abstract states 'an increase of 15% in F-measure', but Table 1 reports 87.8% versus 72.4%, which is a 15.4 percentage-point improvement; please clarify whether the claim is absolute or relative.
  2. [Figure 1 caption] The caption contains a typo: 'Adverarial-neural Event Model' should be 'Adversarial-neural Event Model'.
  3. [§3.3] The text refers to 'Jansen-Shannon divergence'; the correct name is Jensen-Shannon divergence. Also, 'lipschitz' should be capitalized as Lipschitz.
  4. [§4.2] There is a typo: 'AEM outpoerforms both LEM and DPEMM' should read 'AEM outperforms both LEM and DPEMM'.
  5. [Figure 3 caption] The caption says 'The vertical axis represents methods/parameter settings' while the text says the horizontal axis denotes the performance value; please clarify which axis is which, since the figure appears to have performance on the horizontal axis.
  6. [Table 2] The table formatting is garbled, with repeated phrases (e.g., the 'US debt ceiling' row repeats location tokens), and should be cleaned so that the extracted quadruples are readable.
  7. [§4.1] The event number E is set to 25, 25, and 35 for the three datasets without a sensitivity analysis with respect to E, while DPEMM learns the number of events automatically; please discuss how the choice of E affects the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: AEM is trained end-to-end on document reconstruction and evaluated against external event annotations; no derivation reduces to its inputs.

full rationale

AEM's training objective (Eqs. 11-13) is a standard adversarial loss between reconstructed document vectors and real document vectors, and the generator is not fitted to any gold event quadruple. Event extraction at inference is a readout of generator outputs at one-hot seeds (Eq. 14), and those outputs are compared with human-annotated events in Table 1; the target F-measure is not used as a training signal. The baselines LEM and DPEMM are prior work by overlapping authors, but they serve only as comparison systems, not as justification of AEM's correctness, and no uniqueness theorem or design choice is imported from them. The Google dataset is built by retrieving articles with event-related words, which may ease extraction, but this affects external validity rather than circularity. The one-hot event seed readout is an unvalidated extrapolation from the Dirichlet training support, but it is not defined in terms of the evaluation output. Hence no circular step is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

AEM introduces no new physical or semantic entity. Its main unstated costs are the preset event count, the unspecified Dirichlet concentration, and the unvalidated one-hot seed readout that turns the continuous latent space into discrete event distributions.

free parameters (5)
  • Event number E = 25 (FSD), 25 (Twitter), 35 (Google)
    Preset per dataset with no data-driven selection; the number of output events depends directly on it.
  • Dirichlet concentration alpha
    Hyperparameter of the prior in Equation 1; its value is never stated in the paper and is chosen by hand.
  • Gradient penalty coefficient lambda = 10
    Set to 10 following WGAN-GP, but still a hand-chosen trade-off in the objective in Equation 13.
  • Generator hidden units H = 200 default
    Default hidden size is 200; sensitivity is only tested at 100, 150, and 200 on the Google dataset.
  • Discriminator iterations nd = 5 default
    Default is 5; sensitivity is tested at 5, 7, and 10 on the Google dataset only.
assumptions (4)
  • domain assumption A document's event content is fully captured by TF-IDF-weighted multinomial distributions over NER/POS-selected entities, locations, keywords, and dates (Section 3.1).
    If the NER/POS pipeline misses or mislabels event elements, the document representation loses information that no downstream model can recover.
  • domain assumption An event is representable as a quadruple <entity, location, keyword, date>, or <organization, location, person, keyword> for news (Section 3, Section 4.1).
    The whole extraction target is defined as this fixed tuple, which may not cover events that lack one of these components.
  • ad hoc to paper Feeding one-hot event seeds into the generator trained on continuous Dirichlet samples yields valid per-event distributions (Section 3.4, Equation 14).
    One-hot vectors are vertices of the simplex and were not seen during training; no experiment validates this extrapolation.
  • domain assumption The four event components are conditionally independent given the event distribution, implemented as four separate linear subnets (Equations 4-7).
    Real events may have dependencies between, for example, a location and a keyword, which this factorization ignores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Open Event Extraction from Online Text using a Generative Adversarial Network." pith.science (2026). https://pith.science/paper/5WFOHVR7

@misc{pith2026190809246,
  author       = {Pith},
  title        = {Pith review of: Open Event Extraction from Online Text using a Generative Adversarial Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5WFOHVR7}},
  note         = {Machine review of arXiv:1908.09246}
}
read the original abstract

To extract the structured representations of open-domain events, Bayesian graphical models have made some progress. However, these approaches typically assume that all words in a document are generated from a single event. While this may be true for short text such as tweets, such an assumption does not generally hold for long text such as news articles. Moreover, Bayesian graphical models often rely on Gibbs sampling for parameter inference which may take long time to converge. To address these limitations, we propose an event extraction model based on Generative Adversarial Nets, called Adversarial-neural Event Model (AEM). AEM models an event with a Dirichlet prior and uses a generator network to capture the patterns underlying latent events. A discriminator is used to distinguish documents reconstructed from the latent events and the original documents. A byproduct of the discriminator is that the features generated by the learned discriminator network allow the visualization of the extracted events. Our model has been evaluated on two Twitter datasets and a news article dataset. Experimental results show that our model outperforms the baseline approaches on all the datasets, with more significant improvements observed on the news article dataset where an increase of 15\% is observed in F-measure.

Figures

Figures reproduced from arXiv: 1908.09246 by the authors.

Figure 1
Figure 1. The framework of the Adverarial-neural Event Model (AEM), [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Visualization of the ten randomly selected events on each dataset. Each point denotes a document. Different color denotes different events. 0 20 40 60 80 100 Precision K-means LEM DPEMM AEM-n5 AEM-n7 AEM-n10 AEM-h100 AEM-h150 AEM-h200 AEM-layer3 AEM-layer4 AEM-layer5 60.0% 71.4% 29.7% 85.7% 85.7% 88.6% 88.6% 88.6% 85.7% 85.7% 91.4% 85.7% 0 20 40 60 80 100 Recall K-means LEM DPEMM AEM-n5 AEM-n7 AEM-n10 AEM-h100 AEM-h… view at source ↗
Figure 3
Figure 3. Comparison of methods and parameter settings, [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of training time of models. 5 Conclusions and Future Work In this paper, we have proposed a novel ap￾proach based on adversarial training to extract the structured representation of events from on￾line text. The experimental comparison with the state-of-the-…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 19 canonical work pages

  1. [1]

    Hamed Abdelhaq, Christian Sengstock, and Michael Gertz. 2013. Eventweet: Online localized event detection from twitter. Proceedings of the VLDB Endowment, 6(12):1326--1329

    work page 2013

  2. [2]

    Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting city traffic events from social streams. ACM Transactions on Intelligent Systems and Technology (TIST), 6(4):43

    work page 2015

  3. [3]

    Martin Arjovsky, Soumith Chintala, and L \'e on Bottou. 2017. Wasserstein gan. arXiv preprint arXiv:1701.07875

    arXiv 2017

  4. [4]

    Edward Benson, Aria Haghighi, and Regina Barzilay. 2011. Event discovery in social media feeds. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 389--398. Association for Computational Linguistics

    work page 2011

  5. [5]

    William Fedus, Ian Goodfellow, and Andrew M Dai. 2018. Maskgan: Better text generation via filling in the \_. arXiv preprint arXiv:1801.07736

    arXiv 2018

  6. [6]

    Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A Smith. 2010. Part-of-speech tagging for twitter: Annotation, features, and experiments. Technical report, Carnegie-Mellon Univ Pittsburgh Pa School of Computer Science

    work page 2010

  7. [7]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems, pages 2672--2680

    2014

  8. [8]

    Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. 2017. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems, pages 5769--5779

    work page 2017

Show all 28 references
  1. [9]

    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  2. [10]

    Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. 2018. The gan landscape: Losses, architectures, regularization, and normalization. arXiv preprint arXiv:1807.04720

    2018 arXiv

  3. [11]

    Yan Li and Jieping Ye. 2018. Learning adversarial networks for semi-supervised text classification via policy gradient. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1715--1723. ACM

    2018

  4. [12]

    Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming-Ting Sun. 2017. Adversarial ranking for language generation. In Advances in Neural Information Processing Systems, pages 3155--3165

    2017

  5. [13]

    Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579--2605

    2008

  6. [14]

    Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957

    2018 arXiv

  7. [15]

    Sandeep Panem, Manish Gupta, and Vasudeva Varma. 2014. Structured information extraction from natural disaster events on twitter. In Proceedings of the 5th International Workshop on Web-scale Knowledge Representation Retrieval & Reasoning, pages 1--8. ACM

    2014

  8. [16]

    Sasa Petrovic, Miles Osborne, Richard McCreadie, Craig Macdonald, Iadh Ounis, and Luke Shrimpton. 2013. Can twitter replace newswire for breaking news? In Seventh international AAAI conference on weblogs and social media

    2013

  9. [17]

    Ana-Maria Popescu, Marco Pennacchiotti, and Deepa Paranjpe. 2011. Extracting events and event descriptions from twitter. In Proceedings of the 20th international conference companion on World wide web, pages 105--106. ACM

    2011

  10. [18]

    Pengda Qin, Weiran Xu, and William Yang Wang. 2018. Dsgan: Generative adversarial training for distant supervision relation extraction. arXiv preprint arXiv:1805.09929

    2018 arXiv

  11. [19]

    Alan Ritter, Oren Etzioni, Sam Clark, et al. 2012. Open domain event extraction from twitter. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1104--1112. ACM

    2012

  12. [20]

    Ke Wang and Xiaojun Wan. 2018. Sentigan: Generating sentimental texts via mixture adversarial networks. In IJCAI, pages 4446--4452

    2018

  13. [21]

    Yu Wang, David Fink, and Eugene Agichtein. 2015. Seeft: Planned social event discovery and attribute extraction by fusing twitter and web content. In ICWSM, pages 483--492

    2015

  14. [22]

    Chaolun Xia, Jun Hu, Yan Zhu, and Mor Naaman. 2015. What is new in our city? a framework for event extraction using social media posts. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 16--32. Springer

    2015

  15. [23]

    Yuichi Yoshida and Takeru Miyato. 2017. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv:1705.10941

    2017 arXiv

  16. [24]

    Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI, pages 2852--2858

    2017

  17. [25]

    Daojian Zeng, Yuan Dai, Feng Li, R Simon Sherratt, and Jin Wang. 2018. Adversarial learning for distant supervised relation extraction. Computers, Materials & Continua, 55(1):121--136

    2018

  18. [26]

    Tongtao Zhang and Heng Ji. 2018. Event extraction with generative adversarial imitation learning. arXiv preprint arXiv:1804.07881

    2018 arXiv

  19. [27]

    Deyu Zhou, Liangyu Chen, and Yulan He. 2014. A simple bayesian modelling approach to event extraction from twitter. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 700--705

    2014

  20. [28]

    Deyu Zhou, Xuan Zhang, and Yulan He. 2017. Event extraction from twitter using non-parametric bayesian mixture model with word embeddings. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, vol...

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.