Pith. sign in

REVIEW 4 major objections 5 minor 23 references

Automatically Identifying Comparator Groups on Twitter for Digital Epidemiology of Pregnancy Outcomes

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Twitter timelines can be scanned automatically to find users reporting a term pregnancy and a normal-weight baby from the same pregnancy, at a user-level F1-score of 0.933.

desk verdict The reported 0.933 F1 is real on the tested sample, but that sample is preselected by the same regexes the pipeline depends on, so the deployment claim needs an end-to-end evaluation. read the letter →

arxiv 1908.06015 v1 pith:O43QNRRY submitted 2019-08-16 cs.CL cs.SI

Ari Z. Klein , Abeselom Gebreyesus , Graciela Gonzalez-Hernandez This is my paper · ORCID
classification cs.CLcs.SI
keywords digitalepidemiologypregnancyoutcomescomparatorgroupsTwitterminingnaturallanguageprocessingtermnormalbirthweightcase-controlstudies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Pregnancy-outcome studies on Twitter need control groups: users whose pregnancies did not end in miscarriage, stillbirth, preterm birth, or low birthweight. This paper claims those comparators can be assembled automatically by detecting users who report, in their own tweets, that their pregnancy reached at least 37 weeks (term) and that their baby was born weighing at least 5 lb 8 oz (normal birthweight), with both reports tied to the same pregnancy. The pipeline — regular-expression retrieval, supervised tweet classification, and a 50-day same-pregnancy window — reached user-level precision 0.947, recall 0.920, and F1-score 0.933 on a held-out test set of 170 users. If correct, this replaces the prior practice of selecting controls simply because no adverse outcome was detected in their timeline, a practice that silently let adverse outcomes into the comparison group.

What carries the argument

The load-bearing device is the temporal same-pregnancy link: a user enters the comparator group only if the classifier marks one 'term pregnancy' tweet positive and one 'normal birthweight' tweet positive within 50 days of each other — the rule that separates one completed pregnancy from reports scattered across two pregnancies or borrowed from another person's baby. Around it sits the rest of the machinery: hand-crafted regular expressions, augmented with automatically generated lexical variants such as misspellings and abbreviations, that retrieve candidate tweets from a database of users who announced pregnancy; pre-filtering rules that drop bot-like accounts; normalization of matched phrases into '_term_' and '_normalbirthweight_' tokens; and the majority-voting ensemble of a decision tree, logistic regression, and a support vector machine with radial-basis kernel, trained on 1,851 annotated tweets using word n-grams and word clusters as features.

What would settle it

Take a random sample of users the pipeline places in the comparator group and verify their outcomes independently — by survey, by a second annotator reading the full timeline, or by linked birth records — and count how many actually had a full-term birth at normal weight in the same pregnancy; the paper's own test-set comparison already shows that a 125-day window changes the numbers, so the same-pregnancy rule is directly testable.

Watch

Extended reading notes

Core claim

The discovery is that a user-level judgment — 'this person reported a term pregnancy and a normal-weight baby for the same pregnancy' — can be automated end to end from raw tweets. Tweet classification alone, with a majority-voting ensemble of a decision tree, logistic regression, and a support vector machine over n-gram and word-cluster features, reaches an F1-score of 0.950. Adding the 50-day constraint lifts user-level performance to precision 0.947, recall 0.920, and F1 0.933, up from 0.910 before the constraint. The same-pregnancy link is the point: users fail the test not only when a tweet is about someone else's baby or a preterm '36 weeker', but also when a term tweet and a birthweight tweet belong to different pregnancies. The paper further reports that a 125-day window evaluated on the test set yields F1 0.967, and that a majority-class baseline nearly ties the ensemble, evidence that the high-precision retrieval patterns rather than the classifier carry most of the signal.

Load-bearing premise

The evaluation assumes the annotators' guidelines plus the 50-day same-pregnancy rule define what a true 'term pregnancy with normal birthweight' report is, and the paper offers no external check — clinical records, surveys, or registries — that these tweet-based labels correspond to real pregnancy outcomes.

Editorial extensions

If this is right

  • Comparator groups can grow much larger, because the pipeline removes the manual timeline-verification step that had capped the earlier case-control study at one control per case.
  • Excluding users who reported a miscarriage, stillbirth, preterm birth, or low birthweight makes the comparison group a cleaner baseline for studying those very outcomes.
  • A 125-day same-pregnancy window evaluated on the test set raises recall to 0.956 and F1 to 0.967, and the paper's learning curve projects further gains from more annotated data, so the deployed system may beat the reported 0.933.
  • The pipeline is intended to be run against a constantly growing database of pregnancy-announcing users, so comparator cohorts can accumulate prospectively rather than being assembled by hand.
  • The majority-class baseline (ZeroR) nearly ties the ensemble at the user level (F1 0.934 vs 0.933), and the paper's confusion-matrix analysis attributes the tiny gap to one-user differences — yet it will deploy the ensemble because its tweet-level accuracy is expected to keep improving.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because 88% of the annotated tweets were positive, the regular-expression retrieval appears to be doing most of the hard filtering; a deployment that loosens the query patterns to gain recall would shift the discrimination burden onto the tweet classifier, whose reported F1 may not survive that shift.
  • The pipeline's polarity is easily reversed: analogous patterns for '36 weeker', NICU stays, or birth weights under 5 lb 8 oz could turn the comparator finder into a case finder for preterm or low-birthweight outcomes, giving both arms of a case-control study from one system.
  • The 50-day window was tuned on cross-validation predictions, while the paper's own test-set comparison shows 125 days performs better (F1 0.967); because a '38 weeks' tweet and a birth announcement are physiologically constrained to be roughly zero to six weeks apart, the window's calibration against true gestational timing remains an open question.
  • The gold standard has no external anchor: a follow-up validation that surveys a sample of identified users or links them to birth records would reveal whether 'term plus normal birthweight' tweets are genuine pregnancy outcomes or truthful statements that occasionally describe a different pregnancy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an NLP pipeline for automatically identifying Twitter users who report that their pregnancy reached term and that their baby was born at a normal weight, with the goal of selecting comparator groups for observational studies of adverse pregnancy outcomes. The pipeline consists of handcrafted regular expressions for tweet retrieval, pre-filtering rules to remove non-personal accounts, tweet-level classification (ZeroR, J48, logistic regression, SVM, and a majority-voting ensemble), and a 50-day temporal threshold to link a term-pregnancy tweet and a normal-birthweight tweet to the same pregnancy. The authors annotated 2,361 tweets from 847 users, split by user into training and held-out test sets, and report a user-level F1-score of 0.933 for the ensemble classifier after applying the 50-day threshold.

Significance. If the reported performance held in deployment, the pipeline would be a valuable tool for constructing larger and more valid comparator groups in social-media-based pregnancy outcome research, a field that currently relies on weak exclusion-based controls. The paper has notable strengths: the train/test split is performed at the user level, inter-annotator agreement is reported (Cohen's κ = 0.80), and the temporal threshold is selected on training predictions rather than on the test set. However, the significance is strongly tempered by the fact that the evaluation population is limited to users who already matched both the term-pregnancy and normal-birthweight regular expressions, and by a post-hoc test-set evaluation of a different threshold. The near-tie with the ZeroR baseline also raises the question of how much of the reported performance is due to the supervised classifiers versus the handcrafted retrieval and thresholding rules.

major comments (4)
  1. [Data Collection; Results, Classification] The test set is drawn exclusively from the 847 users who already had at least one tweet matching a 'term pregnancy' regex and at least one matching a 'normal birthweight' regex. Therefore, the reported user-level F1 of 0.933 is a conditional measure: it estimates performance only on users for whom regex retrieval already succeeded. In the deployment population of all pregnancy announcers, recall is the product of regex recall, pre-filtering recall, classifier recall, and the temporal-threshold recall; the paper only measures the latter two. Because the regexes are explicitly designed as high-precision patterns, they likely miss paraphrased or non-lexical reports, and no experiment quantifies this loss. The conclusion that the pipeline 'will be deployed to identify large comparator groups' requires an end-to-end evaluation on a random sample of the underlying database, or at least an explicit statement of the unmeasured components and a conservative estimate of deployment performance.
  2. [Discussion] The paragraph reporting that 'in evaluating a higher, 125-day threshold directly on the annotated test set' F1 improved to 0.967 is a post-hoc use of the test set for model selection. These numbers are not unbiased estimates of performance for an a priori chosen threshold; presenting them as evidence that 'increasing the temporal threshold will improve user-level performance' overstates what the test set can support. Either pre-specify the threshold, use nested cross-validation on the training set, or clearly label this analysis as exploratory and separate from the main held-out evaluation.
  3. [Results, Classification] The reported composition of the test set is arithmetically inconsistent: '137 users reporting a term pregnancy and normal birthweight for the same pregnancy, 13 users reporting a term pregnancy and normal birthweight for different pregnancies, and 80 users did not post at least one...' sums to 230 users, not the stated 170. The confusion matrices in Figure 2 appear to correspond to 137 true-positive users and 33 true-negative users, totaling 170. Please reconcile these numbers and precisely define the user-level ground-truth categories, including how users without at least one positive term tweet and one positive normal-birthweight tweet are labeled.
  4. [Classification, Table 4] The ZeroR baseline achieves a user-level F1 of 0.934 after the 50-day threshold, essentially matching the ensemble's 0.933, with the ensemble's advantage limited to precision (0.947 versus 0.941). Because ZeroR predicts every tweet as positive, the post-threshold ZeroR rule is almost equivalent to 'user has a term regex hit and a birthweight regex hit within 50 days.' This suggests that the handcrafted regexes, pre-filtering, and temporal threshold account for nearly all of the pipeline's performance, and the supervised classifiers provide little measurable benefit on this test set. To support the abstract's claim that the approach uses supervised machine learning 'as a basis' for detection, the authors should report confidence intervals or a significance test of the F1 difference and show that the learned classifiers contribute beyond the regex-intersection rule.
minor comments (5)
  1. [Abstract; Conclusion] Please qualify the F1-score of 0.933 as measured on users who already matched both the term-pregnancy and normal-birthweight regular expressions, to avoid implying end-to-end performance on the full pregnancy-announcer database.
  2. [Table 4; Figure 2] Define the superscripts 1 and 2 in Table 4 in the caption (before and after the 50-day threshold), and reformat Figure 2's confusion matrices with explicit row and column labels so that predicted and actual classes are unambiguous.
  3. [Results, Annotation] The paper reports κ = 0.80 on 2,170 overlapping annotations out of 2,361 tweets; please clarify how the remaining 191 tweets were handled (single annotator, adjudication, or exclusion).
  4. [Classification] State the search range and selection criterion used to choose the 50-day threshold on the cross-validated training predictions (e.g., the threshold maximizing user-level F1), so that readers can assess the risk of threshold overfitting.
  5. [Introduction] When describing the database of 'more than 100,000 users who have announced their pregnancy,' specify that the deployment denominator for comparator selection is this population, and briefly note that the current evaluation addresses only the subpopulation that matches the retrieval patterns.

Circularity Check

1 steps flagged · score 2.0 of 10

Headline user-level F1 is a legitimate held-out evaluation; the only circular step is the post-hoc 125-day threshold fitted directly on the test set and then offered as a predicted improvement.

  1. fitted input called prediction [Discussion, paragraph on the 125-day threshold]
    "However, in evaluating a higher, 125-day threshold directly on the annotated test set, we found that recall increased from 0.920 to 0.956, with an increase in precision (from 0.947 to 0.978) and F1-score (from 0.933 to 0.967) as well. Thus, improving the performance of tweet-level classification and increasing the temporal threshold will improve user-level performance."

    Unlike the 50-day threshold, which was selected using cross-validated predictions on the training set, the 125-day threshold is evaluated directly on the held-out test set. Reporting the resulting metrics and then asserting that 'increasing the temporal threshold will improve user-level performance' makes the improvement a consequence of having fitted the threshold to the test labels rather than an independent prediction. The improved F1 is therefore a test-set fit presented as a forward-looking result, though it is peripheral to the headline 50-day evaluation.

full rationale

The paper's central claim is an empirical performance measurement, not a mathematical derivation: a stratified held-out test set of 510 tweets from 170 users is used to compute tweet- and user-level precision/recall/F1, with SVM hyperparameters and the 50-day same-pregnancy threshold selected on training folds via cross-validation. Human annotation, not the classifier, defines the gold standard, and the handcrafted regexes are only a retrieval filter, so the headline estimate is not equivalent to the model's inputs by construction. Self-citations (cohort database, birth-defect corpus, medication classifier) supply background or data infrastructure but are not the load-bearing justification for the reported F1. The main caveat is external validity: because the evaluation population is restricted to users whose tweets already match the term and normal-birthweight regexes, the reported recall is conditional on regex retrieval and does not estimate deployment recall over all pregnancy announcers. That is a sampling/generalizability limitation, not circularity, so it does not raise the score. The one genuine circular step is the 125-day threshold experiment in the Discussion: this threshold is evaluated directly on the annotated test set and its improved metrics are then used to claim that increasing the temporal threshold 'will improve user-level performance,' which is fitting a parameter to the test set and presenting the result as a prediction. This does not affect the headline 50-day result, so the overall circularity is minor.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a manually annotated gold standard, a 50-day temporal rule, and a regex-preselected user population. No new theoretical entities are introduced; the main assumptions are about label validity, population representativeness, and generalization of the supervised classifiers.

free parameters (3)
  • Temporal proximity threshold (days) = 50 days
    Chosen from 10-fold cross-validation predictions on the training set; determines whether a user's positive term and normal-birthweight tweets refer to the same pregnancy. Central to user-level F1.
  • SVM cost parameter c = 128
    Set by 10-fold cross-validation over the training set for the RBF SVM classifier in the ensemble.
  • SVM negative-class weight = 8.5
    Set by 10-fold cross-validation to counter the class imbalance between positive and negative tweets.
assumptions (4)
  • domain assumption Manual annotation is a valid gold standard for detecting reports of term pregnancy and normal birthweight.
    The labels are annotator judgments on tweet text; there is no verification against medical records or other outcome data.
  • ad hoc to paper A 50-day gap between a positive term tweet and a positive normal-birthweight tweet indicates the same pregnancy.
    The threshold is an empirical choice tuned on training data, not derived from gestational timing or external data.
  • domain assumption Users who matched both term and NB regex patterns represent the population for comparator selection.
    All training and test users came from regex-retrieved tweets; deployment will encounter all pregnancy-announcing users, not just those who match these patterns.
  • domain assumption Tweet text features generalize across users.
    The model is evaluated on one held-out split; no external cohort or temporal validation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automatically Identifying Comparator Groups on Twitter for Digital Epidemiology of Pregnancy Outcomes." pith.science (2026). https://pith.science/paper/O43QNRRY

@misc{pith2026190806015,
  author       = {Pith},
  title        = {Pith review of: Automatically Identifying Comparator Groups on Twitter for Digital Epidemiology of Pregnancy Outcomes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O43QNRRY}},
  note         = {Machine review of arXiv:1908.06015}
}
read the original abstract

Despite the prevalence of adverse pregnancy outcomes such as miscarriage, stillbirth, birth defects, and preterm birth, their causes are largely unknown. We seek to advance the use of social media for observational studies of pregnancy outcomes by developing a natural language processing pipeline for automatically identifying users from which to select comparator groups on Twitter. We annotated 2361 tweets by users who have announced their pregnancy on Twitter, which were used to train and evaluate supervised machine learning algorithms as a basis for automatically detecting women who have reported that their pregnancy had reached term and their baby was born at a normal weight. Upon further processing the tweet-level predictions of a majority voting-based ensemble classifier, the pipeline achieved a user-level F1-score of 0.933, with a precision of 0.947 and a recall of 0.920. Our pipeline will be deployed to identify large comparator groups for studying pregnancy outcomes on Twitter.

Figures

Figures reproduced from arXiv: 1908.06015 by the authors.

Figure 1
Figure 1. Automatic NLP pipeline for detecting women who have reported in tweets that their pregnancy had reached term (term) and their baby was born at a normal birthweight (NB), from which to select comparator groups for observational studies of pregnancy outcomes on Twitter. Results Annotation Two professional annotators manually annotated 2361 tweets by 847 users who posted at least one tweet matching a “term pregnancy” q… view at source ↗
Figure 2
Figure 2. Confusion matrices for the user-level performance of the ZeroR and majority voting-based ensemble classifiers, before and after applying the 50-day threshold between a user’s “term pregnancy” and “normal birthweight” tweets that the classifier predicted as “positive.” Error analysis of the ensemble classifier reveals that six of the seven false positive users stem from false positive tweets, and that three of the si… view at source ↗
Figure 3
Figure 3. Precision, recall, and F1-scores for the majority voting-based ensemble classifier, with incremental training sizes. Conclusion In this paper, we presented an NLP pipeline for automatically detecting women who have reported on Twitter that their pregnancy had reached term and their baby was born at a normal weight. Given the performance of the pipeline on a held-out test set (an F1-score of 0.933), it will be deploy… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

  1. [1]

    A system atic review to calculate background miscarriage rates using life table analysis

    Ammon Avalos L, Galindo C, Li DK. A system atic review to calculate background miscarriage rates using life table analysis. Birth Defects Res A Clin Mol Teratol. 2012;94(6):417-423

    work page 2012

  2. [2]

    Deaths: final data for 2016

    Xu J, Murphy SL, Kochanek K, Bastian B, Arias E. Deaths: final data for 2016. Natl Vital Stat Rep. 2018;67(5)

    work page 2016

  3. [3]

    Estimated pregnancy rates and rates of pregnancy outcomes for the United States, 1990-2008

    Ventura SJ, Curtin SC, Abma JC. Estimated pregnancy rates and rates of pregnancy outcomes for the United States, 1990-2008. Natl Vital Stat Rep. 2012;60(7)

    work page 1990

  4. [4]

    Update on overall prevalence of major birth defects—Atlanta, Georgia, 1978-2005

    Rynn L, Cragan J, Correa A. Update on overall prevalence of major birth defects—Atlanta, Georgia, 1978-2005. MMWR Morb Mortal Wkly Rep. 2008;57(1):1-5

    work page 1978

  5. [5]

    Fetal and perinatal mortality: United States, 2013

    MacDorman MF, Gregory ECW. Fetal and perinatal mortality: United States, 2013. Natl Vital Stat Rep. 2015;64(8)

    work page 2013

  6. [6]

    Effects of maternal age and age -specific preterm birth rates on overall preterm birth rates —United States, 2007 and 2014

    Ferré C, Callaghan W, Olson C, Sharma A, Barfield W. Effects of maternal age and age -specific preterm birth rates on overall preterm birth rates —United States, 2007 and 2014. MMWR Morb Mortal Wkly Rep. 2016;65(43):1181-1184. 0.88 0.89 0.90 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1.00 20% 40% 60% 80% 100% Incremental Training of a Majority Voting-Ba...

    work page 2007

  7. [7]

    Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000

    Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, Rudan I, Campbell H, Cibulskis R, Li M, Mathers C, Black RE. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379(9832):2151-2161

    work page 2010

  8. [8]

    Epidemiology and the medical causes of miscarriage

    Regan L, Rai R. Epidemiology and the medical causes of miscarriage. Baillieres Best Pract Res Clin Obstet Gynaecol. 2000;14(5):839-854

    work page 2000

Show all 23 references
  1. [9]

    Comparison of the aetiology of stillbirth over five decades in a single centre: a retrospective study

    Wou K, Ouellet MP, Chen MF, Brown RN. Comparison of the aetiology of stillbirth over five decades in a single centre: a retrospective study. BMJ Open. 2014;4(6):e004635

    2014

  2. [10]

    BMJ 2017;357:j2249

    Feldkamp ML, Carey JC, Byrne JLB, Krikov S , Botto LD Etiology and clinical presentation of birth defects: population based study. BMJ 2017;357:j2249

    2017

  3. [11]

    Spontaneous preterm birth, a clinical dilemma: etiologic, pathophysiologic and genetic heterogeneities and racial disparity

    Menon R. Spontaneous preterm birth, a clinical dilemma: etiologic, pathophysiologic and genetic heterogeneities and racial disparity. Acta Obstet Gynecol Scand. 2008;87(6):590-600

    2008

  4. [12]

    Social Media Use in 2018

    Smith A, Anderson M. Social Media Use in 2018. Pew Research Center Report. 2018. https://www.pewinternet.org/2018/03/01/social-media-use-in-2018/ (accessed 2019 June 17)

    2018

  5. [13]

    Social media mining for birth defects research: a rule -based, bootstrapping approach to collecting data for rare health -related events on Twitter

    Klein AZ, Sarker A, Cai H, Weissenbacher D, Gonzalez -Hernandez G. Social media mining for birth defects research: a rule -based, bootstrapping approach to collecting data for rare health -related events on Twitter. J Biomed Inform. 2018;87:68-78

    2018

  6. [14]

    Discovering cohorts of pregnant women from social media for safety surveillance and analysis

    Sarker A, Chandrashekar P, Magge A, Cai H, Klein A, Gonzalez G. Discovering cohorts of pregnant women from social media for safety surveillance and analysis. J Med Internet Res. 2017;19(10):e361

    2017

  7. [15]

    Pharmacoepidemiologic evaluation of birth defects from health -related postings in social media during pregnancy

    Golder S, Chiuve S, Weissenbacher D, Klein A, O’Connor K, Bland M, Malin M, Bhattacharya M, Scarazinni LJ, Gonzalez-Hernandez G. Pharmacoepidemiologic evaluation of birth defects from health -related postings in social media during pregnancy. Drug Saf. 2019;42(3):389-400

    2019

  8. [16]

    Detecting personal medication intake in Twitter: an annotated corpus and baseline classification system

    Klein AZ, Sarker A, Rouhizadeh M, O’Connor K, Gonzalez G. Detecting personal medication intake in Twitter: an annotated corpus and baseline classification system. In: Proceedings of the BioNLP 2017 Workshop; 2017 Aug 4; Vancouver, Canada. Association for Computational Linguist...

    2017

  9. [17]

    A systematic review of pregnancy exposure registries: examination of protocol -specified pregnancy outcomes, target sample size, and comparator selection

    Gelperin K, Hammad H, Leishear K, Bird ST, Taylor L, Hampp C, Sahin L. A systematic review of pregnancy exposure registries: examination of protocol -specified pregnancy outcomes, target sample size, and comparator selection. Pharmacoepidemiol Drug Saf. 2017;26(2):208-214

    2017

  10. [18]

    Risk of adverse birth outcome and miscarriage in pregnant users of non -steroidal anti -inflammatory drugs: population based observational and case -control study

    Nielsen GL, S ørenson HT, Larsen H, Pedersen L. Risk of adverse birth outcome and miscarriage in pregnant users of non -steroidal anti -inflammatory drugs: population based observational and case -control study. BMJ. 2001;322:266

    2001

  11. [19]

    An unsupervised and c ustomizable misspelling generator for mining n oisy health-related text sources

    Sarker A, Gonzalez -Hernandez G. An unsupervised and c ustomizable misspelling generator for mining n oisy health-related text sources. J Biomed Inform. 2018;88:98-107

    2018

  12. [20]

    Top Names of the Period 2010 -2018

    Social Security Administration. Top Names of the Period 2010 -2018. 2019. https://www.ssa.gov/oact/babynames/decades/names2010s.html

    2010

  13. [21]

    An algorithm for suffix stripping

    Porter MF. An algorithm for suffix stripping. Program. 1980;14(3):130 -137

    1980

  14. [22]

    Part-of-speech tagging for Twitter: word clusters and other advances

    Owoputi O, O’Connor B, Dyer C, Gimpel K, Schneider N. Part-of-speech tagging for Twitter: word clusters and other advances. 2012. http://www.cs.cmu.edu/~ark/TweetNLP/owoputi+etal.tr12.pdf

    2012

  15. [23]

    Understanding interobserver agreement: the kappa statistic

    Viera AJ, Garret JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360-363

    2005

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.