Pith. sign in

REVIEW 3 major objections 5 minor 47 references

On the efficacy of old features for the detection of new bots

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read Cheap, decade-old Twitter features still distinguish newly evolved bots, including coordinated teams, the paper claims.

desk verdict A useful, plainly written empirical comparison showing cheap timeline/profile features still work on recent coordinated-bot datasets, but the 'novel bot' claim needs a temporal or fully held-out evaluation. read the letter →

arxiv 2506.19635 v1 pith:3Y324AOR submitted 2025-06-24 cs.CR cs.AIcs.SI

classification cs.CRcs.AIcs.SI
keywords socialbotdetectionTwitterfeaturesetstimelinefeaturescoordinatedbotsBotometerlow-costmachinelearningclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tests whether cheap, well-known Twitter account features can detect bots that are newer and more sophisticated than the bots those features were originally designed to catch. It trains five standard classifiers on four feature sets, including Botometer's CAP score, profile-only features, timeline-only features, and the proportion of tweets sent from unofficial clients, then evaluates them on five training sets built from six recent datasets. The central finding is that profile and especially timeline features separate novel bots from humans, including bots that act in coordinated teams, with area-under-the-curve values often above 0.85 and sometimes above 0.9. The authors argue this matters because it suggests a general-purpose, low-cost first-stage bot detector remains viable even as bots evolve.

What carries the argument

The central objects are two 'old' feature sets inherited from earlier fake-follower detection: Class A, computed only from account profile fields such as friends and followers counts, account age, and the presence of a name, image, bio, or URL; and Class B, computed from the last 400 tweets, namely the rates of posts containing hashtags, URLs, and mentions, plus the retweet rate. These are contrasted with Botometer's CAP score, a calibrated bot probability derived from more than 1,000 features, and with the percentage of tweets posted from unofficial Twitter clients. The load-bearing mechanism is that timeline content statistics compress enough behavioral signal to separate evolved bots from humans without requiring the friend-and-follower graphs that are expensive to collect.

What would settle it

A strict temporal holdout experiment: train the same five classifiers with Class A and Class B features on accounts labeled in 2018-2019, then evaluate on a dataset of accounts newly labeled after 2020; if ROC-AUC for the timeline features drops to near chance on the new accounts, the paper's central claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a set of simple features obtained from the user's timeline are effective in distinguishing novel social bots, whether the bots act individually, like those in the Vendor dataset, or in teams, like the retweeters in Rtbust and the low-value-stock promoters in Stock. Timeline features reach ROC-AUC values above 0.85 on the coordinated-bot datasets, and on the easier self-declared-bot datasets both profile and timeline features exceed 0.95, typically matching or beating Botometer's CAP score when the same classifiers are used. The Twitter-client heuristic, by contrast, works only on self-declared bots and collapses on fake-follower and coordinated-bot datasets. The authors read the results as evidence that detection can keep pace with bot evolution without expensive network or relationship features.

Load-bearing premise

The reasoning assumes that test scores on these particular 2018-2019 datasets tell us how well the features will spot bots that appear later.

Editorial extensions

If this is right

  • On these datasets, timeline features rival or beat the Botometer CAP score when both are fed to the same classifiers, suggesting that expensive feature engineering is not always necessary.
  • A two-stage detection pipeline becomes plausible: cheap profile and timeline features skim likely bots first, and specialized coordination or network detectors then inspect only the shortlist.
  • The Twitter-client heuristic should be treated as a weak signal that is useful only for naive, self-declared bots, not for sophisticated or coordinated accounts.
  • Because Class A and Class B features need only profile data and up to 400 recent tweets, they can be gathered at far lower API cost than relationship-based features, making large-scale screening more practical.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The strongest untested implication is temporal transfer; because the paper uses 10-fold cross-validation within each dataset, the decisive next experiment is a strict time-split evaluation that trains on pre-2020 accounts and tests on accounts labeled later.
  • Inference: If timeline statistics are the signal, bot operators could respond by randomizing hashtag, mention, URL, and retweet rates to imitate human timelines, so the durability of Class B features should be monitored on fresh datasets.
  • Inference: A practical pipeline suggested by the results is to use timeline features as a low-cost pre-filter and then apply expensive coordination analysis only to flagged accounts, though the paper does not test this end-to-end.
  • Inference: Combining timeline features with sequence-based encodings of account behavior could yield both individual bot scores and team similarity measures, potentially turning the first-stage detector into a group detector; the paper gestures in this direction but does not implement it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper compares four feature sets for Twitter bot detection: the Botometer CAP_UNI* score, profile-based Class A features, timeline-based Class B features, and the proportion of tweets from unofficial clients. Using five classifiers and 10-fold cross-validation on five constructed training sets drawn from six datasets (Celebrity, Verified, Botwiki, Vendor, Stock, Rtbust), it reports that Class A and especially Class B features perform well, including on coordinated-bot datasets, while the Twitter-client feature performs poorly. The authors conclude that cheap, old features can help detect novel bots and propose that such classifiers could serve as a first-stage screening tool before specialized coordinated-behavior analysis.

Significance. If the result held as stated, it would be practically useful: profile and timeline features are inexpensive to collect and could support general-purpose first-stage bot detection. The paper has real strengths: it uses publicly available datasets, standard tooling with Weka default settings, reports several metrics (Balanced Accuracy, precision, recall, MCC, PR-AUC, ROC-AUC), and includes robustness checks over timeline length and a feature-importance analysis. However, the evaluation protocol does not establish the central 'novel bot' claim, and the Botometer arm is affected by label-derived adjustments and optimistic threshold selection. The contribution is therefore best described as an exploratory benchmark with a generalization claim that currently outruns the evidence.

major comments (3)
  1. [4.1, 6.2, Table 11]
  2. [4.4, 6.3]
  3. [6.4, Figure 1]
minor comments (5)
  1. [Table 11]
  2. [Section 5.5]
  3. [Section 6.3]
  4. [Abstract and Introduction]
  5. [Section 4.4]

Circularity Check

2 steps flagged · score 4.0 of 10

Botometer baseline is partly circular: CAP* embeds the dataset's label prior and Table 11's threshold is fit on the same accounts; the central Class A/B finding is independent and the 'novel bot' claim is an external-validity gap, not circularity.

  1. fitted input called prediction [Section 4.1, CAP* definition]
    "However, if one knows the background level of bots in the sample (and this is our case, since the datasets are tagged), CAP can be adjusted with a corrective factor: CAP∗=CAP P d(Bot)/0.15 where P d(Bot) is the actual probability that in the domain d under investigation there is a specific number of bots. We know in advance the nature of our datasets, so it is possible to compute P d(Bot)."

    P d(Bot) is the proportion of bot labels in the dataset, so the CAP* feature is defined in terms of the target labels. In the 10-fold CV experiments, the same full-dataset P d(Bot) is applied to training and test folds, meaning the CAP* value of a test account is computed from a prior that includes that test account's own label. The Botometer-arm scores therefore partly measure the classifier's access to the label distribution through the feature, rather than the standalone discriminating power of Botometer's 1,000 features. The paper presents CAP* as one of the four feature sets for detecting novel bots, so this evaluation is partially self-referential.

  2. fitted input called prediction [Section 6.2, Table 11]
    "Once having CAP_UNI* for every account in the training sets, we measured the Botometer's performances by applying a threshold-based rule: if CAP_UNI*≥th, then the account is tagged as a bot. Then, we evaluated the performances of the rule varying the threshold in the interval [0, ...1]. Finally, for each training set, we selected the threshold that gave the best Balanced Accuracy."

    The threshold th is chosen to maximize Balanced Accuracy on the very accounts whose scores are then reported in Table 11. The rule-based Botometer results are therefore in-sample fitted values, not held-out predictions; the reported precision, recall, and balanced accuracy are near-optimal by construction on those same accounts. This makes the comparison with the 10-fold CV results of the learning models unequal, although it does not affect the paper's central Class A/B finding.

full rationale

The central claim about Class A and Class B features is not circular: those features are computed from account profiles/timelines without using the bot/human labels, and 10-fold cross-validation is a legitimate separability test within each labeled dataset. The paper's admission that Botwiki bots are 'very easy to recognise' weakens the 'novel bots' inference but is an honest dataset characterization, not a circular derivation. The acknowledged lack of testing on Sayyadiharikandeh et al.'s datasets is an external-validity gap, not a circularity. No load-bearing self-citation or imported uniqueness theorem is present. The partial circularity is confined to the Botometer arm: CAP* is rescaled by the actual label prevalence in the sample, and the threshold used for the rule-based Botometer comparison is fitted on the full data. Both steps inflate the Botometer baseline's reported performance but do not drive the article's principal conclusion that cheap timeline/profile features separate the tested bot and human accounts.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central comparative claim rests on trusted dataset labels, the validity of Botometer's CAP formulation, default Weka parameters, and the Bovet-Makse client signal; the Botometer comparison additionally embeds the true bot prevalence as a fitted prior and an in-sample threshold.

free parameters (3)
  • CAP_UNI* prevalence correction P_d(Bot) = Dataset-specific: 0.128 (Celebrity-Botwiki), 0.251 (Verified-Botwiki), 0.102 (Verified-Vendor), 0.591 (Stock), 0.496…
    Section 4.1: CAP is reweighted by the ratio P_d(Bot)/0.15 using the actual bot proportion in each labeled training set, embedding label-distribution knowledge into the Botometer feature.
  • Botometer decision threshold = 0.015, 0.025, 0.025, 0.4, 0.02 per training set
    Section 6.2, Table 11: threshold chosen per training set to maximize Balanced Accuracy on the same data, an optimistic in-sample fit for the Botometer rule-based baseline.
  • Minimum tweets per account = 400
    Section 3: accounts with fewer than 400 tweets are excluded, shrinking datasets (e.g., Vendor from 1088 to 217 accounts) and potentially biasing toward active accounts; Figure 1 shows this choice materially changes results for Verified-Vendor.
assumptions (4)
  • domain assumption The a priori labels in the six datasets are correct and the datasets represent 'novel' or 'evolved' bots.
    Section 3: the study relies on public bot-repository datasets with trusted labels; however, Botwiki bots are self-declared and Vendor fake followers are from 2019, so representativeness of 'evolved' bots is assumed.
  • domain assumption The Botometer CAP formulation (prior P(Bot)=0.15 and Bayes correction) is a valid score for bot probability.
    Section 4.1: the paper adopts CAP* from cited literature without independently validating the calibration.
  • domain assumption Weka default hyperparameters are adequate for fair comparison across feature sets.
    Section 4.4: all classifiers use default Weka parameters; no tuning or hyperparameter search is reported.
  • domain assumption The proportion of tweets from unofficial clients is a meaningful automation signal, per Bovet and Makse.
    Section 4.3: the official-client list is taken from the cited paper and not re-validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the efficacy of old features for the detection of new bots." pith.science (2026). https://pith.science/paper/3Y324AOR

@misc{pith2026250619635,
  author       = {Pith},
  title        = {Pith review of: On the efficacy of old features for the detection of new bots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3Y324AOR}},
  note         = {Machine review of arXiv:2506.19635}
}
read the original abstract

For more than a decade now, academicians and online platform administrators have been studying solutions to the problem of bot detection. Bots are computer algorithms whose use is far from being benign: malicious bots are purposely created to distribute spam, sponsor public characters and, ultimately, induce a bias within the public opinion. To fight the bot invasion on our online ecosystem, several approaches have been implemented, mostly based on (supervised and unsupervised) classifiers, which adopt the most varied account features, from the simplest to the most expensive ones to be extracted from the raw data obtainable through the Twitter public APIs. In this exploratory study, using Twitter as a benchmark, we compare the performances of four state-of-art feature sets in detecting novel bots: one of the output scores of the popular bot detector Botometer, which considers more than 1,000 features of an account to take a decision; two feature sets based on the account profile and timeline; and the information about the Twitter client from which the user tweets. The results of our analysis, conducted on six recently released datasets of Twitter accounts, hint at the possible use of general-purpose classifiers and cheap-to-compute account features for the detection of evolved bots.

Figures

Figures reproduced from arXiv: 2506.19635 by the authors.

Figure 1
Figure 1. ROC-AUC by varying the minimum number of posts in the account timelines, per training set. [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 35 canonical work pages

  1. [1]

    Mustafaraj, P

    E. Mustafaraj, P. T. Metaxas, From obscurity to prominence in minutes: Political speech and real-time search, in: Web Science: Extending the Frontiers of Society On-Line, 2010

  2. [2]

    Sharma, E

    K. Sharma, E. Ferrara, Y. Liu, Identifying coordinated accounts in disinformation campaigns (2020). arXiv:2008.11308

  3. [3]

    Caldarelli, R

    G. Caldarelli, R. Nicola, F. D. Vigna, M. Petrocchi, F. Saracco, The role of bot squads in the political propaganda on Twitter, Communications Physics 3 (2019) 1–15

  4. [4]

    Flow of online misinformation during the peak of the COVID-19 pandemic in Italy

    G. Caldarelli, R. D. Nicola, M. Petrocchi, M. Pratelli, F. Saracco, Analysis of online misinformation during the peak of the COVID-19 pandemics in italy, CoRR abs/2010.01913 (2020).arXiv:2010. 01913. URLhttps://arxiv.org/abs/2010.01913

  5. [5]

    Ferrara, O

    E. Ferrara, O. Varol, C. Davis, F. Menczer, A. Flammini, The rise of social bots, Commun. ACM 59 (7) (2016) 96–104.doi:10.1145/2818717. URLhttp://doi.acm.org/10.1145/2818717

  6. [6]

    Cresci, R

    S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, M. Tesconi, The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race, in: Proceedings of the 26th International Conference on World Wide Web Companion (WWW’17), ACM, 2017, pp. 963–972

  7. [7]

    Sonic Research Group at Nortwestern Univ., Episode 5: The bits and bots of the web with fil menczer, online: https://tinyurl.com/y49gmchk (2020)

  8. [8]

    Bovet, H

    A. Bovet, H. Makse, Influence of fake news in Twitter during the 2016 US presidential election, Nature Communications 10 (7) (2019)

Show all 47 references
  1. [9]

    K. Yang, O. Varol, P. Hui, F. Menczer, Scalable and generalizable social bot detection through data selection, in: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty- Second Innovative Applications of Artificial Intelligence Conference, IAAI 20...

  2. [10]

    Varol, E

    O. Varol, E. Ferrara, C. A. Davis, F. Menczer, A. Flammini, Online human-bot interactions: Detection, estimation, and characterization, in: Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, Montréal, Québec, Canada, May 15-18, 2017., 201...

  3. [11]

    K. Yang, O. Varol, C. Davis, E. Ferrara, A. Flammini, F. Menczer, Arming the public with artificial intelligence to counter social bots, Human Behavior and Emerging Technologies 48–61

  4. [12]

    Cresci, R

    S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, M. Tesconi, Fame for sale: efficient detection of fake twitter followers, Decision Support Systems 80 (2015) 56–71

  5. [13]

    Vico, La Scienza Nuova, 1744

    G. Vico, La Scienza Nuova, 1744

  6. [14]

    Yardi, D

    S. Yardi, D. M. Romero, G. Schoenebeck, danah boyd, Detecting spam in a twitter network, First Monday 15 (1) (2010)

  7. [15]

    Cresci, A decade of social bot detection, Commun

    S. Cresci, A decade of social bot detection, Commun. ACM 63 (10) (2020) 72–83. 22

  8. [16]

    Viswanath, M

    B. Viswanath, M. A. Bashir, M. B. Zafar, S. Bouget, S. Guha, K. P. Gummadi, A. Kate, A. Mislove, Strength in numbers: Robust tamper detection in crowd computations, in: Proceedings of the 2015 ACM on Conference on Online Social Networks, ACM, 2015, pp. 113–124

  9. [17]

    Cresci, R

    S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, M. Tesconi, DNA-inspired online behavioral modeling and its application to spambot detection, IEEE Intelligent Systems 31 (5) (2016) 58–64

  10. [18]

    R. Yu, X. He, Y. Liu, GLAD: Group anomaly detection in social media analysis, ACM Transactions on Knowledge Discovery from Data (TKDD) 10 (2) (2015) 1–22

  11. [19]

    Cresci, R

    S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, M. Tesconi, Social fingerprinting: detection of spambot groups through dna-inspired behavioral modeling, IEEE Transactions on Dependable and Secure Computing 15 (4) (2018) 561–576

  12. [20]

    Sayyadiharikandeh, O

    M. Sayyadiharikandeh, O. Varol, K. Yang, A. Flammini, F. Menczer, Detection of novel social bots by ensembles of specialized classifiers, in: M. d’Aquin, S. Dietze, C. Hauff, E. Curry, P. Cudré-Mauroux (Eds.), CIKM ’20: The 29th ACM International Conference on Information and ...

  13. [21]

    R. J. Schuchard, A. T. Crooks, Insights into elections: An ensemble bot detection coverage framework applied to the 2018 U.S. midterm elections, PLOS ONE 16 (1) (2021) 1–19.doi:10.1371/journal. pone.0244309. URLhttps://doi.org/10.1371/journal.pone.0244309

  14. [22]

    Chavoshi, H

    N. Chavoshi, H. Hamooni, A. Mueen, Debot: Twitter bot detection via warped correlation, in: 2016 IEEE 16th International Conference on Data Mining (ICDM), 2016, pp. 817–822.doi:10.1109/ICDM. 2016.0096

  15. [23]

    Beskow, K

    D. Beskow, K. Carley, H. Bisgin, A. Hyder, C. Dancy, R. Thomson, Introducing Bothunter: A tiered approach to detection and characterizing automated activity on Twitter, in: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior R...

  16. [24]

    Ferrara, Disinformation and social bot operations in the run up to the 2017 french presidential election, First Monday 22 (8) (2017)

    E. Ferrara, Disinformation and social bot operations in the run up to the 2017 french presidential election, First Monday 22 (8) (2017). URLhttps://firstmonday.org/ojs/index.php/fm/article/view/8005

  17. [25]

    Pedregosa, et al., Scikit-learn: Machine learning in python, Journal of Machine Learning Research 12 (85) (2011) 2825–2830

    F. Pedregosa, et al., Scikit-learn: Machine learning in python, Journal of Machine Learning Research 12 (85) (2011) 2825–2830. URLhttp://jmlr.org/papers/v12/pedregosa11a.html

  18. [26]

    El-Mawass, P

    N. El-Mawass, P. Honeine, L. Vercouter, Similcatch: Enhanced social spammers detection on twitter using markov random fields, Information Processing & Management 57 (6) (2020) 102317.doi:https: //doi.org/10.1016/j.ipm.2020.102317. URLhttp://www.sciencedirect.com/science/articl...

  19. [27]

    P. Hui, K. Yang, C. Torres-Lugo, F. Menczer, Botslayer: DIY real-time influence campaign detection, in: M. D. Choudhury, R. Chunara, A. Culotta, B. F. Welles (Eds.), Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, ICWSM 2020, Held Virtually...

  20. [28]

    Giatsoglou, D

    M. Giatsoglou, D. Chatzakou, N. Shah, A. Beutel, C. Faloutsos, A. Vakali, ND-Sync: Detecting syn- chronized fraud activities, in: PAKDD, Springer, 2015

  21. [29]

    Jiang, P

    M. Jiang, P. Cui, C. Faloutsos, Suspicious behavior detection: Current trends and future directions, IEEE Intelligent Systems 31 (1) (2016) 31–39

  22. [30]

    Q. Cao, X. Yang, J. Yu, C. Palow, Uncovering large groups of active malicious accounts in online social networks, in: ACM SIGSAC Conference on Computer and Communications Security, ACM, 2014, pp. 477–488

  23. [31]

    K. Yang, O. Varol, C. A. Davis, E. Ferrara, A. Flammini, F. Menczer, Arming the public with AI to counter social bots, CoRR abs/1901.00912 (2019). URLhttp://arxiv.org/abs/1901.00912 23

  24. [32]

    Mazza, S

    M. Mazza, S. Cresci, M. Avvenuti, W. Quattrociocchi, M. Tesconi, RTbust: Exploiting temporal patterns for botnet detection on Twitter, in: 11th ACM Conference on Web Science, 2019, pp. 183– 192.doi:10.1145/3292522.3326015

  25. [33]

    Cresci, F

    S. Cresci, F. Lillo, D. Regoli, S. Tardelli, M. Tesconi, Cashtag piggybacking: Uncovering spam and bot activity in stock microblogs on Twitter, ACM Trans. Web 13 (2) (Apr. 2019).doi:10.1145/3313184. URLhttps://doi.org/10.1145/3313184

  26. [34]

    Breiman, Random forests, Machine Learning 45 (1) (2001) 5–32.doi:10.1023/A:1010933404324

    L. Breiman, Random forests, Machine Learning 45 (1) (2001) 5–32.doi:10.1023/A:1010933404324

  27. [35]

    Bovet, F

    A. Bovet, F. Morone, H. A. Makse, Validation of twitter opinion trends with national polling aggregates: Hillary clinton vs donald trump, Scientific Reports 8 (1) (2018) 8673

  28. [36]

    CE, Basic principles of ROC analysis, Seminars in Nuclear Medicine 8 (4) (1978) 283–298

    M. CE, Basic principles of ROC analysis, Seminars in Nuclear Medicine 8 (4) (1978) 283–298

  29. [37]

    G. M. Weiss, F. Provost, Learning when training data are costly: The effect of class distribution on tree induction, Journal of artificial intelligence research 19 (2003) 315–354

  30. [38]

    Saito, M

    T. Saito, M. Rehmsmeier, The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets, PLOS ONE 10 (3) (2015) 1–21.doi:10.1371/ journal.pone.0118432. URLhttps://doi.org/10.1371/journal.pone.0118432

  31. [39]

    S. K. Pal, S. Mitra, Multilayer perceptron, fuzzy sets, and classification, IEEE Transactions on Neural Networks 3 (5) (1992) 683–697.doi:10.1109/72.159058

  32. [40]

    W. W. Cohen, Fast effective rule induction, in: A. Prieditis, S. Russell (Eds.), Machine Learning Proceedings 1995, Morgan Kaufmann, San Francisco (CA), 1995, pp. 115 – 123.doi:https://doi. org/10.1016/B978-1-55860-377-6.50023-2. URLhttp://www.sciencedirect.com/science/article...

  33. [41]

    G. H. John, P. Langley, Estimating continuous distributions in bayesian classifiers, in: Proceedings of theEleventhConferenceonUncertaintyinArtificialIntelligence, UAI’95, MorganKaufmannPublishers Inc., San Francisco, CA, USA, 1995, p. 338–345

  34. [42]

    Breiman, Random forests, Machine Learning 45 (2001) 5–32

    L. Breiman, Random forests, Machine Learning 45 (2001) 5–32

  35. [43]

    I. H. Witten, E. Frank, M. A. Hall, Data mining: practical machine learning tools and techniques, 3rd Edition, Morgan Kaufmann, Elsevier, 2011

  36. [44]

    D. Aha, D. Kibler, M. Albert, Instance-based learning algorithms, Machine Learning 6 (1991) 37–66

  37. [45]

    Ozenne, F

    B. Ozenne, F. Subtil, D. Maucort-Boulch, The precision–recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases, Journal of Clinical Epidemiology 68 (8) (2015) 855–859.doi:https://doi.org/10.1016/j.jclinepi.2015.02.010

  38. [46]

    J. T. Kent, Information gain and a general measure of correlation, Biometrika 70 (1) (1983) 163–173

  39. [47]

    T. M. Mitchell, et al., Machine learning (1997). 24

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.