Pith. sign in

REVIEW 3 major objections 4 minor 65 references

Political Leaning and Politicalness Classification of Texts

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that political leaning classifiers trained on a single text domain consistently fail on out-of-distribution data, and that models fine-tuned on a combined 12-dataset corpus with leave-one-out-tuned hyperparameters set a…

desk verdict A genuinely useful benchmark resource with an overstated SOTA claim; the validation protocol leaks one target domain into model selection. read the letter →

arxiv 2507.13913 v1 pith:7D2EZXXU submitted 2025-07-18 cs.CL cs.AI

Matous Volf (1) , Jakub Simko (2) ((1) DELTA High school of computer science , economics , Pardubice , Czechia , (2) Kempelen Institute of Intelligent Technologies , Bratislava , Slovakia) This is my paper
classification cs.CLcs.AI MSC 68T5068T07
keywords politicalleaningclassificationpoliticalnessout-of-distributiongeneralizationleave-one-outbenchmarkingtransformerfine-tuningmulti-datasettrainingmediabiastext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Political leaning classifiers are usually trained and tested on a single dataset, and this paper argues that the practice is the root of their failure on new text: across every model tested, F1 drops consistently and markedly on datasets that were not in the training mix. To confront that, the authors assembled 12 public political-leaning datasets, created a new binary politicalness (political vs. non-political) dataset by extending 18 existing corpora, and benchmarked both existing and newly trained transformer models under leave-one-in and leave-one-out protocols. The leave-one-out runs confirm that single-dataset models collapse on unfamiliar text, while models fine-tuned on the combined corpus with hyperparameters tuned against a held-out dataset perform better on every evaluated dataset and set what the authors call a new state of the art. They also find politicalness classification to be substantially easier than leaning classification, with the zero-shot natural-language-inference classifier Political DEBATE reaching about 90% F1 on a balanced aggregate. The practical point is a recipe: diverse combined training data plus out-of-distribution-aware tuning produces text classifiers that are more likely to work on the messy, future text a real application would encounter.

What carries the argument

The central mechanism is the leave-one-out benchmark turned into a training signal: fine-tune the model on samples from every dataset except one, validate and select checkpoints purely on examples from the left-out dataset, and repeat with each dataset left out. The configuration that scores best on held-out domains, found by leaving out Article bias prediction and then confirmed across all folds, is applied to the full corpus to produce the final models. The second essential piece is the compiled corpus itself: twelve political-leaning datasets unified to left/center/right, with a multiplier to rebalance the underrepresented center examples, and 28 source datasets merged into a single politicalness benchmark, with the Webis datasets excluded from training folds because their heavy intersection with Article bias prediction would cause label spillover.

What would settle it

Run the two final models on a freshly collected, independently labeled corpus of political and non-political texts created after the paper's datasets, for example news and social posts from 2026; if their F1 falls to the level of single-dataset models, roughly 40 to 50 on unseen data, rather than staying near the reported 87 for political leaning, the leave-one-out benchmark overstates real-world generalization. In parallel, hand-label a random sample of texts from datasets the paper marks entirely non-political, such as IMDB, Yelp, or Recipes; a material share of political reviews would falsify the politicalness labels.

Watch

Extended reading notes

Core claim

The paper's central claim is that existing political leaning classifiers are siloed solutions that perform poorly on out-of-distribution texts, and that this gap can be substantially closed by the combination of a diverse training corpus and a training protocol that optimizes for out-of-distribution performance. The authors demonstrate the failure mode first: every existing model and every benchmark model fine-tuned on one dataset scores consistently lower on datasets it never trained on, with the drop at least 7 F1 points and often much larger. They then show the remedy: fine-tuning POLITICS and DeBERTa V3 large on the full 12-dataset corpus, with hyperparameters selected by the leave-one-out benchmark, yields models that score the highest F1 on every evaluated dataset (Table 9), averaging 87.2 for DeBERTa V3 large. The same protocol applied to the 28-source politicalness corpus shows that the binary political/non-political task is much easier, with base transformers already above 90% F1 on a balanced aggregate, and with Political DEBATE being the strongest existing filter. On the paper's terms, this establishes a new state of the art for political leaning classification and the first large-scale benchmark and models for politicalness.

Load-bearing premise

The state-of-the-art claim rests on the assumption that leave-one-out scores on existing datasets faithfully predict performance on genuinely new text, and the politicalness benchmark further assumes that entire genres such as reviews, recipes, and dialogues are universally non-political.

Editorial extensions

If this is right

  • If the claim is right, any future political leaning classifier should be evaluated on at least one held-out domain, since single-dataset F1 substantially overstates real-world performance.
  • A single model trained on the combined corpus can be deployed across news, social media, and other text types without per-domain fine-tuning, which is the practical use case the paper points to for aggregator and media-audit tools.
  • Politicalness filtering is accurate enough at around 90% F1 on balanced data to serve as a practical sieve in front of a leaning classifier, so non-political inputs no longer have undefined behavior.
  • The continued-pretraining advantage of POLITICS over newer general-purpose architectures suggests that domain-specific pretraining on political text is a more direct route to transferable political classifiers than scaling the base model alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Taken as a recipe, the leave-one-out tuning protocol is a form of domain-generalization training, and the same procedure could plausibly improve other text classifiers that suffer from distribution shift, such as sentiment or stance detection, whenever several heterogeneous datasets are available.
  • The politicalness benchmark's measured difficulty depends on the genre-wide labeling assumption; auditing samples from datasets like IMDB, Yelp, and Recipes would show how much headroom remains once genuinely political reviews are accounted for.
  • The consistently lower out-of-distribution scores on article-level three-class datasets such as Qbias suggest that genuinely neutral or center text, not just domain shift, remains the hard core of the task.
  • The authors' own caveat that generalization can only truly be shown on future text implies a direct extension: re-running the released checkpoints when a new political dataset appears, and comparing their F1 against the reported 87.2 baseline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper compiles and unifies twelve political-leaning datasets and eighteen politicalness datasets (sixteen of which are relabeled by topic inspection or implicit assumptions), evaluates eight existing leaning models and three existing politicalness models, and conducts leave-one-in and leave-one-out benchmarks using BERT, RoBERTa, DeBERTa V3, and POLITICS as base models. The main contributions claimed are: (1) a demonstration that single-dataset models degrade substantially out-of-distribution, (2) a new politicalness dataset covering diverse text types, and (3) two newly trained models (POLITICS-based and DeBERTa V3 large) that reportedly establish a new state of the art for political leaning classification after training on the combined data with leave-one-out-optimized hyperparameters.

Significance. If the claims held in full, the paper would be a valuable empirical contribution: it provides a broad benchmark, a new politicalness dataset, released code and models, and a systematic measurement of cross-dataset transfer that aligns with prior work. The finding that all tested models, including the new ones, perform consistently worse on unseen datasets is an honest and useful result. The strengths are the scope of the benchmark, the dataset-intersection analysis, and the public release of resources. However, the headline claims of 'enhanced generalization capabilities' and 'new state-of-the-art' are only partially supported, because the validation protocol and the evaluation setup for the final models do not cleanly separate in-sample target selection from model selection, and because Table 9 is an in-distribution evaluation rather than a cross-domain test. The significance therefore depends on whether these claims are appropriately qualified.

major comments (3)
  1. [§3.5.2, Table 7] The hyperparameter optimization for both POLITICS (Optuna search) and DeBERTa V3 large (manual tuning) explicitly used Article bias prediction as the left-out dataset for validation, and the resulting hyperparameters were then frozen for every other leave-one-out fold and for the final all-data models. Consequently, the Article bias prediction row in Table 7 is not an out-of-distribution estimate, and the reported average left-out F1 mixes one in-sample target with nine out-of-sample targets. This compromises the leave-one-out benchmark as a clean estimate of generalization to an arbitrary unseen domain and weakens the abstract's claim of 'enhanced generalization capabilities.' Please either conduct a nested validation in which hyperparameters are selected without using the target fold, or explicitly report the results with this caveat and soften the generalization claim accordingly.
  2. [§4.5, Table 9, and §8 Conclusion] Table 9 evaluates the final models on held-out 15% slices of the same datasets that were used for training (with only the two Webis datasets excluded), so it is an in-distribution benchmark, not a cross-domain test. The sentence 'They establish the new state-of-the-art performance across all the evaluated datasets' is defensible only as an in-distribution claim, and the conclusion's 'our newly trained models based on POLITICS and DeBERTa large establish a new state-of-the-art performance for political leaning classification' overstates the evidence, particularly in light of the paper's own §6 admission that extensive testing on truly unseen data was not carried out. The claims in the abstract, conclusion, and §4.5 should be reworded to specify that the state-of-the-art result is on the collected benchmark datasets and that cross-domain generalization remains an open question.
  3. [§3.1.2, Table 2] For sixteen of the eighteen politicalness datasets, the binary political/non-political labels were created by the authors rather than obtained from explicit annotations. In particular, datasets marked 'implicit' (Yelp review full, IMDB, Recipes, Dialogsum) are assigned entirely to the non-political class without per-text inspection, and 'topics' datasets are labeled by inspecting topic category names rather than the text content. If a nontrivial fraction of these texts concern political subjects, the politicalness benchmark in Table 8 and the politicalness filter models are trained and evaluated on systematically noisy labels, and the near-ceiling F1 scores may be inflated. The paper should provide evidence of label quality, such as a manual validation sample or an inter-annotator agreement study, and should discuss the potential impact of this label noise on the reported politicalness results.
minor comments (4)
  1. [§3.5.2] The 'center multiplier' used to balance the center class is described only verbally ('we have added a multiplier... We adjust it to even out the class distribution'). The exact multiplier value or the balancing algorithm should be reported to make the training setup reproducible.
  2. [Table 4] The table mixes models that support the center class with models that do not, and although the text notes that models with different class counts should be compared separately, the 'overall average' column is still likely to be misinterpreted. A footnote restating the random-baseline accuracy for 2-class vs. 3-class problems would improve clarity.
  3. [§3.5.2] The choice of Article bias prediction as the left-out validation dataset is justified after the fact, but the sentence explaining the reasons currently appears after the description of both hyperparameter searches. Moving this justification to the first mention would make the protocol easier to follow.
  4. [§3.2] The text refers to the 'Levenstein distance'; the standard spelling is 'Levenshtein distance.'

Circularity Check

3 steps flagged · score 5.0 of 10

Leave-one-out 'generalization' scores are partially fitted: hyperparameters targeted the Article bias prediction fold and checkpoints were selected on each left-out validation set, making the OOD F1 means ceiling estimates; Political DEBATE also filters CommonCrawl and is then scored on the filtered data. The Table 9 SOTA is in-distribution and the qualitative OOD drop is independent content.

  1. fitted input called prediction [§3.5.2 (Leave-one-out benchmark); results: Table 7 (§4.4.1), Table 9 (§4.5)]
    "Both in the hyperparameter search for POLITICS and while tweaking DeBERTa large, we have chosen the Article bias prediction to be the left-out dataset and aimed for the best performance when evaluating the model on it. ... Using the selected optimized hyperparameters with POLITICS and DeBERTa large, we have conducted the whole leave-one-out benchmark again, leaving a different dataset at a time."

    The optimized models are defined by hyperparameters H that were selected by aiming for the best performance on the Article bias prediction fold (F*). The leave-one-out benchmark is then re-run with H frozen and F* included among the folds; Table 7 reports F*'s left-out F1 (63.1 for POLITICS, 64.3 for DeBERTa large) and the averaged left-out F1 (64.1/66.2) is offered as evidence that the setup generalizes to 'different text styles or topics from a different time period.' By construction, F*'s left-out score is the value H was chosen to optimize — a validation-selected quantity reported as an out-of-distribution prediction, not a clean estimate for an arbitrary unseen domain.

  2. fitted input called prediction [§3.5.2 (Leave-one-out benchmark); results: Tables 7–8 (§4.4)]
    "The most interesting evaluation is then on this left-one-out dataset, because it shows how well the model handles not just unseen individual texts, but also different text styles or topics from a different time period. This is why the validation set in this benchmark consists purely of examples from the left-out dataset, so that the best model checkpoint can be selected based on that performance."

    For every leave-one-out fold, the model checkpoint is chosen as the one with the best F1 on the left-out dataset's own validation slice, and the value reported in Tables 7–8 is that selected checkpoint's score on the left-out test slice. The quantity advertised as measuring how well the model handles 'unseen individual texts' and 'different text styles' is therefore a maximum-over-checkpoints statistic of target-domain validation performance: the supposedly unseen fold is used as model-selection data.

1 more flagged steps
  1. self definitional [§3.1.1 (Data preprocessing); results: Table 12 (Appendix B.2), Table 5 (§4.2.2)]
    "On this dataset, it achieved an accuracy of only 33 % – lower than a randomly guessing model would. We have investigated the individual examples the model has identified as non-political and confirmed that most of them in fact were. ... we have utilized the model as a filter and removed just the non-political texts. To mitigate the removal of false negatives (actually political texts), we have only considered the text non-political if the model was very confident – with the logit score above 0.99."

    CommonCrawl was originally labeled entirely political. Political DEBATE's confident non-political predictions (logit > 0.99) were used to delete texts from it before the remaining texts entered the politicalness collection. The same model is then scored against that filtered dataset: Table 12 reports Political DEBATE at 98.8 F1 on CommonCrawl, and the headline aggregate 90.4 F1 (Table 5) includes this slice. Because the composition of the evaluation set was determined by the model's own outputs — the texts the model firmly called non-political were removed — the reported accuracy is, to that extent, the model confirming its own filter decisions, not an independent measurement on that domain.

full rationale

Overall: this is an empirical benchmark paper with no derivation chain, no self-citations (no cited work is by Volf or Simko), no imported uniqueness theorems, and no ansatz-by-citation. The three listed steps are genuine but partial reductions. Steps 1–2 compromise the headline OOD generalization estimate: the Article bias prediction left-out score reduces to the hyperparameter-tuning target, and every left-out score is checkpoint-selected against the target fold's validation set. However the central finding does not collapse: dropping the tuned fold changes the means by at most 0.2 F1 (POLITICS 64.1→64.2; DeBERTa large 66.2→66.4), nine of ten folds are hyperparameter-honest, the trained-vs-left-out drop is consistent across all models and protocols and agrees with the cited external literature (Cohen and Ruths 2021; Yan et al. 2017), and the paper itself concedes the drop persists (§8). Step 3 is self-confirming on only one of the thirty aggregate slices. Table 9's 'new state-of-the-art across all the evaluated datasets' is an in-distribution measurement on held-out slices of the training collection — not a fit masquerading as a prediction, though the abstract/conclusion wording ('enhanced generalization capabilities') outruns what the protocol cleanly establishes; the paper concedes in §6 that extensive OOD testing is future work. The author-assigned politicalness labels (16 datasets, e.g. Yelp/IMDB marked entirely non-political) are a label-reliability/correctness risk, not circularity under the review rules.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rest on assembled datasets, human-inspected labels, and hand-picked preprocessing and training parameters. The main free parameters are the per-dataset length cutoffs, the 100k downsampling cap, the 0.99 politicalness filter threshold, the 10k training sample size, and the center-class multiplier. The key domain assumptions are the 3-class leaning scheme, the topic-based politicalness definition, and the trustworthiness of AllSides-derived labels.

free parameters (5)
  • text length lower bounds per dataset = not specified numerically
    Section 3.1: for each dataset 'we have set an appropriate lower bound and dropped all examples that were shorter.' The bounds are chosen by hand; the exact values are not given in the paper, so this preprocessing choice is a free parameter affecting the data distribution.
  • 100,000 downsampling cap and systematic sampling by text length = 100,000 rows per large dataset
    Section 3.1: 'BIGNEWSBLN, CommonCrawl news articles, Media political stance, Political tweets and Yelp review full have been sampled to 100 000 datapoints while ensuring an even class distribution.' This cap was chosen to maintain balance across datasets and affects which examples are used.
  • politicalness filter logit threshold 0.99 for CommonCrawl news articles = 0.99
    Section 3.1.1: 'we have only considered the text non-political if the model was very confident with the logit score above 0.99.' This threshold is a hand-picked parameter that determines which CommonCrawl examples are removed before training the political leaning classifiers.
  • training sample size of 10,000 per dataset for optimized models = 10,000
    Section 3.5.2: '10 000 has turned out to be a good sample size to be taken from each dataset for training' chosen to allow class balancing with the center multiplier. This is a manual choice affecting model training.
  • center class multiplier = not specified numerically
    Section 3.5.2: 'For this we have added a multiplier which increases the number of taken center leaning examples from the dataset that have them. We adjust it to even out the class distribution.' The exact multiplier value is not reported.
assumptions (5)
  • domain assumption The 3-class left/center/right labeling is a sufficient simplification of political leaning for the benchmark.
    Section 1: 'Political leaning is in the context of this analysis categorized into three classes: left, center and right. This is a simplification of the real problem.' The authors state this is a limitation, yet the entire benchmark and SOTA claim rest on this reduction.
  • domain assumption Politicalness is adequately defined as 'main topic is politics' rather than 'expresses a political opinion'.
    Section 1: 'A text is political if its main topic is politics. Otherwise, it is non-political.' The authors acknowledge that this definition lets in Wikipedia-style neutral descriptions of political topics, which is a known weakness for the constructed dataset.
  • domain assumption The AllSides media-level annotations are trustworthy and transferable to individual articles.
    Section 3.1.1: the authors rely on AllSides ratings and cite Baly et al. 2020 that individual labels differ from media leaning in only 3.11 percent of cases. This assumption is load-bearing for most of the news datasets.
  • domain assumption The 'politicalness' labels derived from topic inspection and genre assumptions (e.g., Yelp reviews as non-political) are accurate enough for training.
    Section 3.1.2: 'When the texts in the dataset were labeled by topics, we have carefully inspected the text contents in each topic in order to either assign the correct label, or discard the topic.' Several genres are labeled implicitly as entirely non-political, which could introduce label noise.
  • domain assumption The string-slice body overlap test is a sufficient estimator of dataset intersection.
    Section 3.2: 'A simpler algorithm has been chosen: slicing a middle part (50 characters or less if the body is shorter) out of the first dataset's row's body and testing, whether the second dataset's row's body contains it.' The authors acknowledge this is only an estimate, but they use it to exclude datasets from benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Political Leaning and Politicalness Classification of Texts." pith.science (2026). https://pith.science/paper/7D2EZXXU

@misc{pith2026250713913,
  author       = {Pith},
  title        = {Pith review of: Political Leaning and Politicalness Classification of Texts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7D2EZXXU}},
  note         = {Machine review of arXiv:2507.13913}
}
read the original abstract

This paper addresses the challenge of automatically classifying text according to political leaning and politicalness using transformer models. We compose a comprehensive overview of existing datasets and models for these tasks, finding that current approaches create siloed solutions that perform poorly on out-of-distribution texts. To address this limitation, we compile a diverse dataset by combining 12 datasets for political leaning classification and creating a new dataset for politicalness by extending 18 existing datasets with the appropriate label. Through extensive benchmarking with leave-one-in and leave-one-out methodologies, we evaluate the performance of existing models and train new ones with enhanced generalization capabilities.

Figures

Figures reproduced from arXiv: 2507.13913 by the authors.

Figure 1
Figure 1. The distribution of the body word count val [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The AllSides media bias chart. Version 10.1, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The progression of the F1 score and loss on the validation set across training steps when fine-tuning DeBERTa V3 large for the leave-one-out benchmark with 10 000 examples from each dataset, leaving out and validating on the Article bias prediction dataset. The F1 score peaks at around 7 000 steps – at the same point as the loss drops for the last time before starting to dramatically increase, indicating overfitting… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The resulting confusion matrix of Political [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrices of POLITICS and DeBERTa V3 large with optimized hyperparameters (test set size: [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 43 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. http://arxiv.org/abs/1907.10902 Optuna: A next-generation hyperparameter optimization framework . CoRR, abs/1907.10902

    arXiv 2019

  4. [4]

    amrrs. 2022. https://www.kaggle.com/datasets/nulldata/medium-post-titles Medium post titles

    work page 2022

  5. [5]

    Dimosthenis Antypas, Asahi Ushio, Jose Camacho-Collados, Vitor Silva, Leonardo Neves, and Francesco Barbieri. 2022. https://aclanthology.org/2022.coling-1.299/ T witter topic classification . In Proceedings of the 29th International Conference on Computational Linguistics, pages 3386--3400, Gyeongju, Republic of Korea. International Committee on Computati...

    work page 2022

  6. [6]

    Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. 2021. https://doi.org/10.18653/v1/2021.acl-long.7 PENS : A dataset and generic framework for personalized news headline generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pr...

    doi:10.18653/v1/2021.acl-long.7 2021

  7. [7]

    Ramy Baly, Giovanni Da San Martino, James Glass, and Preslav Nakov. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.404 We can detect your bias: Predicting the political ideology of news articles . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4982--4991, Online. Association for Computational Li...

    doi:10.18653/v1/2020.emnlp-main.404 2020

  8. [8]

    Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the international AAAI conference on web and social media, volume 14, pages 830--839

    work page 2020

Show all 65 references
  1. [9]

    Shengtao Bian. 2022. https://huggingface.co/datasets/Shengtao/recipe Recipe

    2022

  2. [10]

    Michael Burnham. 2024. https://doi.org/10.1017/psrm.2024.35 Stance detection: a practical guide to classifying political beliefs in text . Political Science Research and Methods, pages 1--18

    2024 doi

  3. [11]

    Michael Burnham, Kayla Kahn, Ryan Yank Wang, and Rachel X Peng. 2024. Political debate: Efficient zero-shot and few-shot classifiers for political text. arXiv preprint arXiv:2409.02078

    2024 arXiv

  4. [12]

    Wei-Fan Chen, Khalid Al Khatib, Henning Wachsmuth, and Benno Stein. 2020. https://doi.org/10.18653/v1/2020.nlpcss-1.16 Analyzing political bias and unfairness in news articles at different levels of granularity . In Proceedings of the Fourth Workshop on Natural Language Proces...

    2020 doi

  5. [13]

    Wei-Fan Chen, Henning Wachsmuth, Khalid Al-Khatib, and Benno Stein. 2018. https://doi.org/10.18653/v1/W18-6509 Learning to flip the bias of news headlines . In Proceedings of the 11th International Conference on Natural Language Generation, pages 79--88, Tilburg University, Th...

    2018 doi

  6. [14]

    Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. 2021. https://doi.org/10.18653/v1/2021.findings-acl.449 D ialog S um: A real-life scenario dialogue summarization dataset . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5062--5074, Online...

    2021 doi

  7. [15]

    Raviv Cohen and Derek Ruths. 2021. https://doi.org/10.1609/icwsm.v7i1.14434 Classifying political orientation on twitter: It’s not easy! Proceedings of the International AAAI Conference on Web and Social Media, 7(1):91--99

    2021 doi

  8. [16]

    Dmitry Demidov. 2023. https://www.researchgate.net/publication/377074277_Political_Bias_of_News_Content_Classification_based_on_Individual_Articles_and_Media Political bias of news content: Classification based on individual articles and media

    2023

  9. [17]

    Tu My Doan and Jon Atle Gulla. 2022. https://doi.org/https://doi.org/10.1016/j.osnem.2022.100208 A survey on political viewpoints identification . Online Social Networks and Media, 30

    2022

  10. [18]

    Cristina Espa \ n a-Bonet. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.787 Multilingual coarse political stance classification of media. the editorial line of a C hat GPT and bard newspaper . In Findings of the Association for Computational Linguistics: EMNLP 2023, p...

    2023 doi

  11. [19]

    Rama Rohit Reddy Gangula, Suma Reddy Duggenpudi, and Radhika Mamidi. 2019. https://doi.org/10.18653/v1/W19-4809 Detecting political bias in news articles using headline attention . In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks ...

    2019 doi

  12. [20]

    gptmurdock. 2024. https://huggingface.co/gptmurdock/classifier-main_subjects_politics Classifier main subject politics

    2024

  13. [21]

    Yupeng Gu, Ting Chen, Yizhou Sun, and Bingyu Wang. 2017. Ideology detection for twitter users via link analysis. In Social, Cultural, and Behavioral Modeling, pages 262--268, Cham. Springer International Publishing

    2017

  14. [22]

    Fabian Haak and Philipp Schaer. 2023. https://doi.org/10.1145/3578503.3583628 Qbias - a dataset on media bias in search queries and query suggestions . In Proceedings of the 15th ACM Web Science Conference 2023, WebSci '23, pages 239--244, New York, NY, USA. Association for Co...

    2023

  15. [23]

    Andrew Halterman. 2025. https://doi.org/10.1017/pan.2024.31 Synthetically generated text for supervised text analysis . Political Analysis, pages 1--14

    2025 doi

  16. [24]

    Michael Heseltine and Bernhard Clemm von Hohenberg. 2024. https://doi.org/10.1177/20531680241236239 Large language models as a substitute for human experts in annotating political text . Research & Politics, 11(1)

    2024 doi

  17. [25]

    Adrien Heymans. 2022. https://huggingface.co/datasets/adrienheymans/imdb-movie-genres Imdb movie genres

    2022

  18. [26]

    Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. 2024. Bridging language and items for retrieval and recommendation. arXiv preprint arXiv:2403.03952

    2024 arXiv

  19. [27]

    Bj rn H yland, Jean-Fran c ois Godbout, Emanuele Lapponi, and Erik Velldal. 2014. https://doi.org/10.3115/v1/W14-2516 Predicting party affiliations from E uropean parliament debates . In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Sci...

    2014 doi

  20. [28]

    Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, and Philip Resnik. 2014. https://doi.org/10.3115/v1/P14-1105 Political ideology detection using recursive neural networks . In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P...

    2014 doi

  21. [29]

    Christopher Jones. 2024. https://huggingface.co/datasets/cajcodes/political-bias Political bias dataset: A synthetic dataset for bias detection and reduction

    2024

  22. [30]

    Kornraphop Kawintiranon and Lisa Singh. 2022. https://aclanthology.org/2022.lrec-1.801/ P oli BERT weet: A pre-trained language model for analyzing political content on T witter . In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 7360--7367, ...

    2022

  23. [31]

    Yujian Liu, Xinliang Frederick Zhang, David Wegsman, Nicholas Beauchamp, and Lu Wang. 2022. https://doi.org/10.18653/v1/2022.findings-naacl.101 POLITICS : Pretraining with same-story article comparison for ideology prediction and stance detection . In Findings of the Associati...

    2022 doi

  24. [32]

    Giuliano Lorenzoni, Ivens Portugal, Paulo Alencar, and Donald Cowan. 2024. http://arxiv.org/abs/2501.00241 Exploring variability in fine-tuned models for text classification with distilbert

    2024 arXiv

  25. [33]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. https://aclanthology.org/P11-1015/ Learning word vectors for sentiment analysis . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: H...

    2011

  26. [34]

    Rishabh Misra. 2022. News category dataset. arXiv preprint arXiv:2209.11429

    2022 arXiv

  27. [35]

    Rishabh Misra and Jigyasa Grover. 2021. Sculpting Data for ML: The first act of Machine Learning. Misra, Rishabh and Grover, Jigyasa

    2021

  28. [36]

    Narayana. 2024. https://www.kaggle.com/datasets/nbandhi/political-podcasts-listing-with-audio-links Political podcasts listing with audio links

    2024

  29. [37]

    Jyoti Shankar Nayak. 2024 a . https://huggingface.co/datasets/JyotiNayak/political_ideologies Gpt-4 political ideologies

    2024

  30. [38]

    Jyoti Shankar Nayak. 2024 b . https://huggingface.co/JyotiNayak/political_ideologies_detection_roberta_finetuned Political ideologies roberta fine-tuned

    2024

  31. [39]

    Ren \'a ta N \'e meth. 2023. A scoping review on the use of natural language processing in research on political polarization: trends and research prospects. Journal of computational social science, 6(1):289--313

    2023

  32. [40]

    Mary Newhauser. 2022. https://huggingface.co/m-newhauser/distilbert-political-tweets Distilbert political tweets

    2022

  33. [41]

    Renáta Németh. 2022. https://doi.org/10.1007/s42001-022-00196-2 A scoping review on the use of natural language processing in research on political polarization: trends and research prospects . Journal of Computational Social Science, 6

    2022 doi

  34. [42]

    Oscar Palmqvist. 2024. https://huggingface.co/oscpalML/DeBERTa-political-classification Deberta political classification

    2024

  35. [43]

    Bo Pang and Lillian Lee. 2005. https://doi.org/10.3115/1219840.1219855 Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales . In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics ( ACL `05...

    2005

  36. [44]

    Open Phi. 2023. https://huggingface.co/datasets/open-phi/textbooks Textbooks

    2023

  37. [45]

    Daniel Preo t iuc-Pietro, Ye Liu, Daniel Hopkins, and Lyle Ungar. 2017. https://doi.org/10.18653/v1/P17-1068 Beyond binary labels: Political ideology prediction of T witter users . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volu...

    2017 doi

  38. [46]

    Bucket Research. 2023. https://huggingface.co/bucketresearch/politicalBiasBERT Politicalbiasbert

    2023

  39. [47]

    Premtim Sahitaj. 2024. https://huggingface.co/premsa/political-bias-prediction-allsides-DeBERTa Political bias prediction allsides deberta

    2024

  40. [48]

    Harshal Shrimali. 2024. https://huggingface.co/harshal-11/DistillBERT-Political-Finetune Distilbert political fine-tune

    2024

  41. [49]

    Emily Silcock, Abhishek Arora, Luca D'Amico-Wong, and Melissa Dell. 2024. http://arxiv.org/abs/2406.09490 Newswire: A large-scale structured database of a century of historical news

    2024 arXiv

  42. [50]

    Maximilian Splieth \"o ver, Maximilian Keiff, and Henning Wachsmuth. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.152 No word embedding model is perfect: Evaluating the representation accuracy for social bias in the media . In Findings of the Association for Computati...

    2022 doi

  43. [51]

    Peter Stefanov, Kareem Darwish, Atanas Atanasov, and Preslav Nakov. 2020. https://doi.org/10.18653/v1/2020.acl-main.50 Predicting the topical stance and political leaning of media using tweets . In Proceedings of the 58th Annual Meeting of the Association for Computational Lin...

    2020 doi

  44. [52]

    Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. http://arxiv.org/abs/1905.05583 How to fine-tune BERT for text classification? CoRR, abs/1905.05583

    2019 arXiv

  45. [53]

    Peter Szemraj. 2023. https://huggingface.co/datasets/pszemraj/goodreads-bookgenres Goodreads book genres

    2023

  46. [54]

    Petter Törnberg. 2024. https://doi.org/10.1177/08944393241286471 Large language models outperform expert coders and supervised classifiers at annotating political social media messages . Social Science Computer Review

    2024 doi

  47. [55]

    Jacob van Steyn. 2023. https://huggingface.co/datasets/Jacobvs/PoliticalTweets Political tweets

    2023

  48. [56]

    Alejandro Vélez Castañeda. 2024. https://huggingface.co/jhonalevc1995/BERT-political_bias-finetune Bert political bias finetune

    2024

  49. [57]

    Webhose.io. n.d. https://github.com/Webhose/free-news-datasets Free news datasets

  50. [58]

    Fagan, Bryan D

    Christina Wolbrecht, Brooke Shannon, E.J. Fagan, Bryan D. Jones, Frank R. Baumgartner, Sean M. Theriault, Derek A. Epp, Cheyenne Lee, and Miranda E. Sullivan. 2023 a . Policy agendas project: Democratic party platform

    2023

  51. [59]

    Fagan, Bryan D

    Christina Wolbrecht, Brooke Shannon, E.J. Fagan, Bryan D. Jones, Frank R. Baumgartner, Sean M. Theriault, Derek A. Epp, Cheyenne Lee, and Miranda E. Sullivan. 2023 b . Policy agendas project: Republican party platform

    2023

  52. [60]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 arXiv

  53. [61]

    Felix Ming Fai Wong, Chee Wei Tan, Soumya Sen, and Mung Chiang. 2016. https://doi.org/10.1109/TKDE.2016.2553667 Quantifying political leaning from tweets, retweets, and retweeters . IEEE Transactions on Knowledge & Data Engineering, 28(08):2158--2172

    2016

  54. [62]

    Zhiping Xiao, Weiping Song, Haoyan Xu, Zhicheng Ren, and Yizhou Sun. 2020. https://doi.org/10.1145/3394486.3403275 Timme: Twitter ideology-detection via multi-task multi-relational embedding . In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discover...

    2020

  55. [63]

    Hao Yan, Allen Lavoie, and Sanmay Das. 2017. https://api.semanticscholar.org/CorpusID:31418886 The perils of classifying political orientation from text . In LINKDEM@IJCAI

    2017

  56. [64]

    Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. http://arxiv.org/abs/1509.01626 Character-level convolutional networks for text classification . CoRR, abs/1509.01626

    2015 arXiv

  57. [65]

    Maayan Zhitomirsky-Geffet, Esther David, Moshe Koppel, and Hodaya Uzan. 2016. https://doi.org/10.1108/OIR-06-2015-0211 Utilizing overtly political texts for fully automatic evaluation of political leaning of online news websites . Online Information Review, 40:362--379

    2016 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.