Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Using predicted hate-crime probabilities to pick which police reports to read, 200 manual annotations estimate Sweden's 2022 hate crime total at 6,051 (SE 548), and reveal thousands missed by police flags.

desk verdict A promising but not-yet-sound application of prediction-powered sampling to hate crime estimation; the H2P2 estimator is misdefined, the proof of its unbiasedness fails, and the reported SE likely understates variance. read the letter →

arxiv 2505.04643 v1 pith:MW6YU6SV submitted 2025-05-05 cs.CL cs.LG

Hannes Waldetoft , Jakob Torgander , M{aa}ns Magnusson This is my paper · ORCID
classification cs.CLcs.LG MSC 62D0568T50
keywords prediction-poweredestimationHansen-Hurwitzestimatorprobability-proportional-to-sizesamplinghatecrimestatisticsfinitepopulationtransformertextclassificationhighlyimbalanceddataofficial
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Estimating how many of Sweden's roughly 1.46 million 2022 police reports are hate crimes is hard because true positives are around 0.4 percent of the population and expert annotation is expensive. This paper shows that a transformer text classifier's predicted probabilities can serve as an auxiliary variable in survey sampling: draw reports with probability proportional to the predicted hate-crime probability, then use the Hansen-Hurwitz estimator. With only 200 manually read reports, the authors estimate 6,051 hate crimes (SE 548), a precision that would require roughly 28,000 reports under simple random sampling. The same machinery yields stratified and difference estimates and an estimated 2,260 to 4,452 hate crimes that police flags missed. The paper's conclusion is that, when a labeled training set exists, prediction-powered finite-population estimation can make statistics on rare text events cheaper and less dependent on police annotation.

What carries the argument

The H2P2 estimator (Hansen-Hurwitz prediction-powered estimator), a named estimator at the paper's core, is probability-proportional-to-size sampling with replacement: each report's inclusion probability is proportional to the classifier's predicted hate-crime probability $\hat p_i$, and the total is recovered by $\hat t_{HH}=\frac{1}{n}\sum_{i\in S} y_i/\pi_i$. The predicted probabilities act as a cheap auxiliary variable correlated with the rare target $y_i$, so the variance formula $V(\hat t_{HH})=\frac{1}{n}\sum_{i\in S}\pi_i(y_i/\pi_i - t)^2$ converts classifier quality directly into sampling efficiency. The supporting mechanism is stratification-by-prediction, which splits the population into predicted-positive and predicted-negative strata and uses the difference estimator inside the near-empty predicted-negative stratum to avoid the bimodal sampling distribution of simple random sampling.

What would settle it

Run the paper's Monte Carlo setup on the incitement-against-ethnic-group subset while forcing the smallest predicted probabilities to zero: if the H2P2 estimate drops below the known true total, the positivity assumption is falsified. A field check is to manually annotate the 2022 reports with the lowest predicted probabilities; finding genuine hate crimes there at a rate the $n=200$ sample could rarely hit would mean the headline total of 6,051 is not dependable.

Watch

Extended reading notes

Core claim

The paper's central claim is that model predictions for a rare binary attribute can be treated as a size measure in probability-proportional-to-size sampling, producing unbiased finite-population estimators for text corpora. Its H2P2 estimator uses the Hansen-Hurwitz total $\hat t_{HH}=\frac{1}{n}\sum_{i\in S} y_i/\pi_i$ with draw probabilities $\pi_i$ proportional to the predicted probability $\hat p_i$; Proposition 1 shows the variance tends to zero as the classifier's cross-entropy loss tends to zero, so classifier quality and annotation effort are directly interchangeable. Applied to all 1,463,762 Swedish police reports from 2022, the best domain-adapted transformer classifier with a manually annotated sample of $n=200$ gives an estimated 6,051 hate crimes (SE 548), a design effect of 0.0068 against simple random sampling, and an estimated police under-flagging of 2,260 to 4,452 hate crimes. Stratification-by-prediction and difference estimators give alternative totals of 4,618 (SE 263) and 6,193 (SE 1,220).

Load-bearing premise

Every real hate crime has to get a positive predicted probability from the classifier, and the sampling probabilities have to be normalized so that even the least likely reports can be drawn; if some true hate crimes get a zero or near-zero score, the estimate is biased downward or its variance explodes.

Editorial extensions

If this is right

  • The H2P2 estimate implies that official hate crime totals based only on police-flagged reports miss an estimated 2,260 to 4,452 hate crimes in 2022, and this under-reporting can be quantified without annotating the full corpus.
  • With labeled training data from previous years, an annual Swedish hate crime estimate requires about 200 expert annotations instead of the roughly 28,000-report simple random sample that would give the same precision, shortening and cheapening the production cycle.
  • The better the classifier, the smaller the sampling variance, with the limiting case that a perfect classifier makes the estimator's variance zero; investment in classification quality and in annotation are interchangeable.
  • In 2022 the transformer classifier was correct in roughly 62 percent (CI 0.55 to 0.69) of cases where it disagreed with police flags, so the model can serve as a higher-quality auxiliary variable than the police annotation.
  • Stratification-by-prediction with difference estimation in the zero-stratum is the more practical route for domain sub-estimates such as antisemitic hate crimes, because pure SRS in a nearly empty stratum gives a bimodal distribution and an understated variance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the 2022 result rests on one sample of 200 author-annotated reports, the gap between the H2P2 estimate (6,051) and the stratified SRS estimate (4,618) is a sign that estimator choice and sampling variability matter at this scale; a replication with a larger annotated sample would settle which total is closer.
  • The thick right tail that the paper documents for H2P2 suggests a concrete safeguard for production use: manually check the highest-probability reports not selected by the PPS draw, or add a deterministic high-probability stratum, to protect against low-probability false negatives.
  • The same machinery transfers to other rare-event text populations—fraud reports, domestic violence, medical symptom mentions—wherever a fine-tuned classifier outputs probabilities and a rare binary label is costly to read.
  • Swapping the Hansen-Hurwitz estimator for a without-replacement Horvitz-Thompson version, which the paper lists as future work, plus calibrating predicted probabilities to known marginal totals, could reduce the tail variance without changing the core idea.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes using transformer-encoder predicted probabilities as an auxiliary variable for finite-population estimation of rare binary outcomes in text. Three estimators are presented: Hansen-Hurwitz with PPS sampling (H2P2), stratified-by-prediction with SRS, and stratified difference estimation. The method is applied to Swedish police reports to estimate the 2022 hate crime total and police under-reporting. Training BERT and RoBERTa models on 2007-2021 reports, the authors validate on a known-label 'incitement against ethnic group' subset with Monte Carlo and then report an H2P2 estimate of 6,051 hate crimes (SE 548) from n=200 manual annotations, with a design effect of 0.0068 relative to SRS.

Significance. If the theoretical claims hold, the practical contribution is substantial: reducing tens of thousands of manual annotations to a few hundred is highly valuable for official statistics and for text corpora generally. The Monte Carlo results on a real known-total corpus (SE 102 vs 600 for SRS, DEFF 0.029) provide genuine empirical support for the efficiency of the implemented procedure, and the concrete Swedish hate-crime application is timely and policy-relevant. The central limitation is that the estimator's theoretical foundation, as written, needs repair before those results can be interpreted as unbiased; once corrected, the approach could be a useful template for prediction-powered finite-population inference.

major comments (3)
  1. [§2.1, Eq. (1)] The definition π_i = p_i / Σ_{j∈S} p_j uses the random sample S in the denominator, but Hansen-Hurwitz unbiasedness requires fixed first-draw probabilities π_i = p_i / Σ_{j∈U} p_j. With a sample-dependent denominator, the expectation E[y_i / π_i] does not equal the population total, and the variance expression in Eq. (1) is not the Hansen-Hurwitz variance; this invalidates the sentence 'The HH estimator is unbiased' as applied to the proposed procedure. Please either define the inclusion probabilities with the fixed population sum or provide a separate unbiasedness proof for the estimator actually implemented.
  2. [§2.1, Proposition 1] The proof of Proposition 1 contains invalid equalities: it treats the random sample set S+ as having |S+| = t (the population total of positives), and it replaces population sums with sample sums in the variance decomposition. Perfect classification (p_i → 1 for positives, p_i → 0 for negatives) also cannot be reconciled with the earlier assumption p_i ∈ (0,1), and the limiting argument does not establish V(ŷ_HH) → 0 for fixed n. As written, the proposition does not support the efficiency claim; it should be restated and proved for the true Hansen-Hurwitz variance under fixed inclusion probabilities.
  3. [§4.1.1 and §4.2.2] Unbiasedness also requires p_i > 0 for every unit with y_i = 1. The paper's own evidence indicates this assumption is questionable for the 2022 application: the model struggles with Quran-burning and anti-Sámi cases (§4.1), the estimated full-set F1 is 0.80 (§4.1.1), and the single n=200 sample found zero positives in the stratum where both police and model predicted 'not hate crime' (§4.2.2). If true positives with very small predicted probabilities exist, the 6,051 estimate is biased downward and the reported SE 548 understates the variance. Please add diagnostics (e.g., the distribution of p_i among known positives in held-out data, sensitivity analyses under assumed numbers of undetected positives in the zero-predicted stratum) or temper the unbiasedness and efficiency claims accordingly.
minor comments (5)
  1. [§4.1.1 and Appendix B] The delta-method standard error for the full-set F1 treats the observed zero false negatives in the sampled stratum as exactly zero variance; this ignores the sampling zero-inflation and likely understates the uncertainty in F1.
  2. [Table 3 caption] The caption contains a typo: 'Stata 1' should be 'Stratum 1'.
  3. [References] Reference [40] is a placeholder ('To Do') and must be replaced with a complete citation.
  4. [§2.1] The variance notation mixes sample and population sums; using S in the population variance formula is confusing even after fixing the denominator issue.
  5. [Appendix A] The text says 'Hansen-Hurwits' but should say 'Hansen-Hurwitz'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the estimators are standard design-based survey estimators using classifier predictions as auxiliary variables, and the efficiency claims are validated with known-label Monte Carlo data and manual annotation.

full rationale

The paper's derivation chain is not circular. The H2P2 estimator is the classical Hansen-Hurwitz estimator applied with predicted probabilities as size measures; its unbiasedness is imported from standard sampling theory (Cochran [1]; Hansen and Hurwitz [2]), not from the fitted model. Proposition 1, stating that variance goes to zero as cross-entropy loss goes to zero, is an asymptotic property of PPS sampling with a perfect size variable, not a definitional equivalence. The Monte Carlo experiment in Section 4.2.1 uses a held-out incitement-against-ethnic-group subset with known labels, and the 2022 application in Section 4.2.2 uses a manually annotated sample, so the headline estimate (6,051, SE 548) is not the classifier's own output renamed as a prediction. The full-population F1 estimate in Section 4.1.1 uses model predictions only to define sampling strata, while manual labels supply the inference; that is a valid design-based procedure. Self-citation [57] appears only for error-analysis context and future-work suggestions, not as a load-bearing justification of the estimator. The 'H2P2' label is an application of a known estimator, not a renaming of a known empirical pattern. Two correctness concerns (not circularity) should be flagged: Section 2.1 defines pi_i = p_i / sum_{j in S} p_j with a random sample denominator, whereas the Hansen-Hurwitz theorem requires a fixed population denominator sum_{j in U} p_j; and the reference list contains a placeholder entry, '[40] To Do. Reference we need to fix. Not at all, 2099.', which is a missing-reference flag but is not used to justify the method. Neither issue makes the derivation circular, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The estimators rest on classical survey-sampling theory. The novel ingredient, using classifier probabilities as a size measure, rests on domain assumptions about model coverage of the target population and about treating unannotated reports as negatives. No new entities are postulated.

assumptions (4)
  • standard math The Hansen-Hurwitz estimator is unbiased when inclusion probabilities are fixed and proportional to a known positive size measure.
    Invoked in Section 2.1 to claim unbiasedness of H2P2; classical result from Cochran (1977).
  • domain assumption Predicted probabilities p_i are treated as a fixed auxiliary variable, with the population-level sum used implicitly as the PPS normalizer.
    Section 2.1 writes the normalizer as the sample sum, which is not a fixed design quantity; the intended population normalizer must be assumed for unbiasedness.
  • domain assumption Unannotated police reports can be labeled as non-hate crimes in training without materially biasing the classifier.
    Section 3.2.1 labels 50,000 random 'other crimes' as y=0 and argues the label noise is small; this is load-bearing for classifier quality.
  • domain assumption The classifier trained on 2007-2021 flagged and keyword-selected reports transfers to the full 2022 population.
    Full-population F1 is estimated at 0.80 in Section 4.1.1, so transfer is imperfect and this assumption is only partially satisfied.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation." pith.science (2026). https://pith.science/paper/MW6YU6SV

@misc{pith2026250504643,
  author       = {Pith},
  title        = {Pith review of: Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MW6YU6SV}},
  note         = {Machine review of arXiv:2505.04643}
}
read the original abstract

Estimating population parameters in finite populations of text documents can be challenging when obtaining the labels for the target variable requires manual annotation. To address this problem, we combine predictions from a transformer encoder neural network with well-established survey sampling estimators using the model predictions as an auxiliary variable. The applicability is demonstrated in Swedish hate crime statistics based on Swedish police reports. Estimates of the yearly number of hate crimes and the police's under-reporting are derived using the Hansen-Hurwitz estimator, difference estimation, and stratified random sampling estimation. We conclude that if labeled training data is available, the proposed method can provide very efficient estimates with reduced time spent on manual annotation.

Figures

Figures reproduced from arXiv: 2505.04643 by the authors.

Figure 1
Figure 1. Number of police reports filed for the period 2007-2022, the number of confirmed hate [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Histogram of simulating 10 000 H2P2 estimates, SRS without stratification, SbP using SRS, SbP with difference estimation in the zero-stratum within the subset of incitement against ethnic group with n = 500. The red line is the true total. We can summarise the results as the H2P2 estimator is a suitable choice if statistical efficiency is important. However, if the setting is more like that of SNCCP, in which the ma… view at source ↗
Figure 3
Figure 3. shows the distribution of the auxiliary variable, pˆ, on the observations in the sample used for the Hansen-Hurwits estimator, along with the distribution of pˆ in all observations from 2022. It is clear that the sample increases the probability of a true hate crime being included in the sample, comparing to taking an SRS [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Design-Based Prediction-Powered Inference for Spatial Data

    stat.ME 2026-08 accept novelty 6.0 of 10

    Design-based prediction-powered inference for spatial data with misspecified sampling weights leaves a non-vanishing spatial remainder, so coverage can fall as labels accumulate.

Reference graph

Works this paper leans on

57 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    John Wiley & Sons, 3 edition, 1977

    William Gemmell Cochran.Sampling techniques. John Wiley & Sons, 3 edition, 1977

    work page 1977

  2. [2]

    On the theory of sampling from finite populations

    Morris H Hansen and William N Hurwitz. On the theory of sampling from finite populations. The Annals of Mathematical Statistics, 14(4):333–362, 1943

    work page 1943

  3. [3]

    A generalization of sampling without replacement from a finite universe.Journal of the American Statistical Association, 47(260):663–685, 1952

    Daniel G Horvitz and Donovan J Thompson. A generalization of sampling without replacement from a finite universe.Journal of the American Statistical Association, 47(260):663–685, 1952

    work page 1952

  4. [4]

    Model assisted survey sampling

    Carl-Erik Särndal, Bengt Swensson, and Jan Wretman. Model assisted survey sampling. Springer New York, NY, 1 edition, 2003

    work page 2003

  5. [5]

    A Horvitz–Thompson-type estimator of species richness.Environmetrics, 22(7):901–910, 2011

    Steen Magnussen. A Horvitz–Thompson-type estimator of species richness.Environmetrics, 22(7):901–910, 2011

    work page 2011

  6. [6]

    Estimating population treatment effects from a survey subsample

    Kara E Rudolph, Iván Díaz, Michael Rosenblum, and Elizabeth A Stuart. Estimating population treatment effects from a survey subsample. American Journal of Epidemiology, 180(7):737–748, 2014

    work page 2014

  7. [7]

    High- resolution population estimation using household survey data and building footprints.Nature communications, 13(1):1330, 2022

    Gianluca Boo, Edith Darin, Douglas R Leasure, Claire A Dooley, Heather R Chamberlain, Attila N Lázár, Kevin Tschirhart, Cyrus Sinai, Nicole A Hoff, Trevon Fuller, et al. High- resolution population estimation using household survey data and building footprints.Nature communications, 13(1):1330, 2022

    work page 2022

  8. [8]

    Some Thoughts on Official Statistics and its Future (with discus- sion)

    Yves Tillé, Marc Debusschere, Henri Luomaranta, Martin Axelson, Eva Elvers, Anders Holm- berg, and Richard Valliant. Some Thoughts on Official Statistics and its Future (with discus- sion). Journal of Official Statistics, 38(2):557–598, 2022

    work page 2022

Show all 57 references
  1. [9]

    Statistics Canada’s Quality Assurance Framework

    Statistics Canada. Statistics Canada’s Quality Assurance Framework. https://www150. statcan.gc.ca/n1/en/catalogue/12-586-X, 2017. Statistics Canada, Ottawa

    2017

  2. [10]

    Lag om den officiella statistiken [Offi- cial Statistics Act]

    Riksdagen [The Swedish Parliament]. Lag om den officiella statistiken [Offi- cial Statistics Act]. https://www.riksdagen.se/sv/dokument-och-lagar/dokument/ svensk-forfattningssamling/lag-200199-om-den-officiell_sfs-2001-99/ , 2001. SFS 2001:99

    2001

  3. [11]

    Regulation (EC) No 223/2009 of the European Parliament and of the Council of 11 March 2009 on European Statistics, 2009

    European Parliament and Council. Regulation (EC) No 223/2009 of the European Parliament and of the Council of 11 March 2009 on European Statistics, 2009. OJ L 87, p 164–173. 15

    2009

  4. [12]

    The National Academies Press, Washington, DC, 2024

    National Academies of Sciences, Engineering, and Medicine.Principles and Practices for a Federal Statistical Agency: Eighth Edition. The National Academies Press, Washington, DC, 2024

    2024

  5. [13]

    Politics as Usual? Measuring Populism, Nationalism, and Authoritarianism in US Presidential Campaigns (1952–2020) with Neural Language Models

    Bart Bonikowski, Yuchen Luo, and Oscar Stuhler. Politics as Usual? Measuring Populism, Nationalism, and Authoritarianism in US Presidential Campaigns (1952–2020) with Neural Language Models. Sociological Methods & Research, 51(4):1721–1787, 2022

    1952

  6. [14]

    The augmented social scientist: Using sequential transfer learning to annotate millions of texts with human-level accuracy.Sociological Methods & Research, 53(3):1167–1200, 2024

    Salomé Do, Étienne Ollion, and Rubing Shen. The augmented social scientist: Using sequential transfer learning to annotate millions of texts with human-level accuracy.Sociological Methods & Research, 53(3):1167–1200, 2024

    2024

  7. [15]

    A deep language model for symptom extraction from clinical text and its application to extract COVID-19 symptoms from social media

    Xiao Luo, Priyanka Gandhi, Susan Storey, and Kun Huang. A deep language model for symptom extraction from clinical text and its application to extract COVID-19 symptoms from social media. IEEE Journal of Biomedical and Health Informatics, 26(4):1737–1748, 2021

    2021

  8. [16]

    Emergingindustry classification based on BERT model.Information Systems, 128:102484, 2025

    BaochengYang, BingZhang, KevinCutsforth, ShanfuYu, andXiaowenYu. Emergingindustry classification based on BERT model.Information Systems, 128:102484, 2025

    2025

  9. [17]

    Innovations in public health surveillance: An overview of novel use of data and analytic methods

    Heather Rilkoff, Shannon Struck, Chelsea Ziegler, Laura Faye, Dana Paquette, and David Buckeridge. Innovations in public health surveillance: An overview of novel use of data and analytic methods. Canada Communicable Disease Report, 50(3-4):93, 2024

    2024

  10. [18]

    Anmälda brott 2023 slutlig statistik [Reported Crimes 2023 Final Statistics].https://bra.se/download/18

    Brottsförebyggande rådet [Swedish National Council for Crime Prevention] . Anmälda brott 2023 slutlig statistik [Reported Crimes 2023 Final Statistics].https://bra.se/download/18. 3d22500318e5f70da2846d9/1711454523087/Statistikrapport_anmalda_2023.pdf, 2024. Accessed: 2024-08-25

    2023

  11. [19]

    Active Learning Literature Survey

    Burr Settles. Active Learning Literature Survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison, 2009

    2009

  12. [20]

    Jon Lundgren and Aravella Lejonstad. Polisanmälda hatbrott 2022 - En sammanställning av de ärenden som hatbrottsmarkerats av polisen [Hate crimes in police reports 2022 - a summary of reports marked as hate crimes by the police]. Brottsförebyggande rådet [Swedish National Coun...

    2022

  13. [21]

    Prediction-powered inference.Science, 382(6671):669–674, 2023

    Anastasios N Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I Jordan, and Tijana Zrnic. Prediction-powered inference.Science, 382(6671):669–674, 2023

    2023

  14. [22]

    Naoki Egami, Musashi Hinck, Brandon Stewart, and Hanying Wei. Using imperfect surrogates for downstream inference: Design-based supervised learning for social science applications of large language models.Advances in Neural Information Processing Systems, 36, 2024

    2024

  15. [23]

    Natural language processing

    Jacob Eisenstein. Natural language processing. MIT Press, 2019

    2019

  16. [24]

    A survey on sentiment analysis methods, applications, and challenges

    Mayur Wankhade, Annavarapu Chandra Sekhara Rao, and Chaitanya Kulkarni. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7):5731–5780, 2022

    2022

  17. [25]

    Analytical mapping of opinion mining and sentiment analysis research during 2000–2015.Information Processing & Manage- ment, 53(1):122–150, 2017

    Rajesh Piryani, Devaraj Madhavi, and Vivek Kumar Singh. Analytical mapping of opinion mining and sentiment analysis research during 2000–2015.Information Processing & Manage- ment, 53(1):122–150, 2017. 16

    2000

  18. [26]

    Text classification algorithms: A survey.Information, 10(4):150, 2019

    Kamran Kowsari, Kiana Jafari Meimandi, Mojtaba Heidarysafa, Sanjana Mendu, Laura Barnes, and Donald Brown. Text classification algorithms: A survey.Information, 10(4):150, 2019

    2019

  19. [27]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  20. [28]

    Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  21. [29]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  22. [30]

    SciBERT: A pretrained language model for scientific text

    Iz Beltagy, Kyle Lo, and Arman Cohan. SciBERT: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676, 2019

    1903 arXiv

  23. [31]

    Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. How to fine-tune bert for text classi- fication? In Chinese computational linguistics: 18th China national conference, CCL 2019, Kunming, China, October 18–20, 2019, proceedings 18, pages 194–206. Springer, 2019

    2019

  24. [32]

    A survey on text classification algorithms: From text to predictions.Information, 13(2):83, 2022

    Andrea Gasparetto, Matteo Marcuzzo, Alessandro Zangari, and Andrea Albarelli. A survey on text classification algorithms: From text to predictions.Information, 13(2):83, 2022

    2022

  25. [33]

    Chatgpt for Text Annotation? Mind the Hype!SocArXiv preprint, 2023

    Etienne Ollion, Rubing Shen, Ana Macanovic, and Arnault Chatelain. Chatgpt for Text Annotation? Mind the Hype!SocArXiv preprint, 2023

    2023

  26. [34]

    A survey of text classification with transformers: How wide? how large? how long? how accurate? how expensive? how safe? IEEE Access, 12:6518–6531, 2024

    John Fields, Kevin Chovanec, and Praveen Madiraju. A survey of text classification with transformers: How wide? how large? how long? how accurate? how expensive? how safe? IEEE Access, 12:6518–6531, 2024

    2024

  27. [35]

    Are chatbots reliable text annotators? sometimes.PNAS nexus, 4(4):pgaf069, 2025

    Ross Deans Kristensen-McLachlan, Miceal Canavan, Marton Kárdos, Mia Jacobsen, and Lene Aarøe. Are chatbots reliable text annotators? sometimes.PNAS nexus, 4(4):pgaf069, 2025

    2025

  28. [36]

    From words to watts: Benchmarking the energy costs of large language model inference

    Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. From words to watts: Benchmarking the energy costs of large language model inference. In2023 IEEE High Performance Extreme...

    2023

  29. [37]

    Machine learning in official statistics

    Martin Beck, Florian Dumpert, and Joerg Feuerhake. Machine learning in official statistics. arXiv preprint arXiv:1812.10422, 2018

    2018 arXiv

  30. [38]

    Model-assisted survey regression estimation with the lasso

    Kelly S McConville, F Jay Breidt, Thomas CM Lee, and Gretchen G Moisen. Model-assisted survey regression estimation with the lasso. Journal of Survey Statistics and Methodology, 5(2):131–158, 2017

    2017

  31. [39]

    Brottsbalken [Swedish Penal Code], 1962

    Sveriges Riksdag. Brottsbalken [Swedish Penal Code], 1962. SFS 1962:700

    1962

  32. [40]

    Reference we need to fix

    To Do. Reference we need to fix. Not at all, 2099. 17

  33. [41]

    Carina Djärv and Anna Gavell Frenzel. Polisens hatbrottsmarkering - Kvalitetsgranskning av polisens hatbrottsmarkering samt jämförelse mellan statistik baserad på polisens hatbrotts- markering och Brås hatbrottsstatistik [Police hate crime flagging - Quality Control of police ...

    2018

  34. [42]

    Playing with Words at the National Library of Sweden - Making a Swedish BERT.arXiv preprint arXiv:2007.01658, 2020

    Martin Malmsten, Love Börjeson, and Chris Haffenden. Playing with Words at the National Library of Sweden - Making a Swedish BERT.arXiv preprint arXiv:2007.01658, 2020

    2007 arXiv

  35. [43]

    Don’t stop pretraining: Adapt language models to domains and tasks

    SuchinGururangan, AnaMarasović, SwabhaSwayamdipta, KyleLo, IzBeltagy, DougDowney, and Noah A Smith. Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964, 2020

    2004 arXiv

  36. [44]

    Learning from positive and unlabeled data: A survey.Machine Learning, 109(4):719–760, 2020

    Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey.Machine Learning, 109(4):719–760, 2020

    2020

  37. [45]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. InInternational Conference on Learning Representations (ICLR), 2019

    2019

  38. [46]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  39. [47]

    Deep Learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org

    2016

  40. [48]

    LEGAL-BERT: The muppets straight out of law school

    Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion An- droutsopoulos. LEGAL-BERT: The muppets straight out of law school. arXiv preprint arXiv:2010.02559, 2020

    2010 arXiv

  41. [49]

    Downstream task performance of BERT models pre-trained using automatically de-identified clinical data

    Thomas Vakili, Anastasios Lamproudis, Aron Henriksson, and Hercules Dalianis. Downstream task performance of BERT models pre-trained using automatically de-identified clinical data. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4245– 4252, 2022

    2022

  42. [50]

    Should you mask 15% in masked language modeling?arXiv preprint arXiv:2202.08005, 2022

    Alexander Wettig, Tianyu Gao, Zexuan Zhong, and Danqi Chen. Should you mask 15% in masked language modeling?arXiv preprint arXiv:2202.08005, 2022

    2022 arXiv

  43. [51]

    Spanbert: Improving pre-training by representing and predicting spans.Transactions of the association for computational linguistics, 8:64–77, 2020

    Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. Spanbert: Improving pre-training by representing and predicting spans.Transactions of the association for computational linguistics, 8:64–77, 2020

    2020

  44. [52]

    Albert: A lite bert for self-supervised learning of language representations

    Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019

    1909 arXiv

  45. [53]

    Pmi-masking: Principled masking of correlated spans.arXiv preprint arXiv:2010.01825, 2020

    Yoav Levine, Barak Lenz, Opher Lieber, Omri Abend, Kevin Leyton-Brown, Moshe Tennen- holtz, and Yoav Shoham. Pmi-masking: Principled masking of correlated spans.arXiv preprint arXiv:2010.01825, 2020

    2010 arXiv

  46. [54]

    How to train BERT with an academic budget

    Peter Izsak, Moshe Berchansky, and Omer Levy. How to train BERT with an academic budget. arXiv preprint arXiv:2104.07705, 2021. 18

    2021 arXiv

  47. [55]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877– 1901, 2020

    1901

  48. [56]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  49. [57]

    not hate crime

    Holli Sargeant, Hannes Waldetoft, and Måns Magnusson. Classifying Hate: Legal and Ethical Evaluations of ML-Assisted Hate Crime Classification and Estimation in Sweden. InProceed- ings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’25, New Yor...

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.