Using Word Embeddings to Examine Gender Bias in Dutch Newspapers, 1950-1990
Pith reviewed 2026-05-24 18:53 UTC · model grok-4.3
The pith
Word embeddings trained separately on six Dutch newspapers from 1950 to 1990 reveal gender bias shifting toward men overall while diverging by newspaper ideology.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By comparing local changes in word embedding models trained on newspapers with divergent ideological backgrounds, the study demonstrates clear differences in gender bias and changes within and between newspapers over time. In relation to themes such as sexuality and leisure, the bias moves toward women, whereas generally the bias shifts in the direction of men, despite growing female employment numbers and feminist movements. Even though Dutch society became less stratified ideologically through depillarization, an increasing divergence in gender bias is found between religious and social-democratic newspapers on the one hand and liberal newspapers on the other.
What carries the argument
Separately trained word embedding models on ideologically divergent newspaper corpora, compared via local changes in associations between gender terms and other words.
Load-bearing premise
Local changes in word associations from separately trained embedding models on different newspaper corpora directly and comparably reflect real differences in gender bias in the underlying texts rather than corpus statistics or training artifacts.
What would settle it
A manual review of gender associations in random samples of articles from the same newspapers that shows no matching patterns of shift or divergence by ideology would indicate the embeddings are not tracking the claimed bias differences.
read the original abstract
Contemporary debates on filter bubbles and polarization in public and social media raise the question to what extent news media of the past exhibited biases. This paper specifically examines bias related to gender in six Dutch national newspapers between 1950 and 1990. We measure bias related to gender by comparing local changes in word embedding models trained on newspapers with divergent ideological backgrounds. We demonstrate clear differences in gender bias and changes within and between newspapers over time. In relation to themes such as sexuality and leisure, we see the bias moving toward women, whereas, generally, the bias shifts in the direction of men, despite growing female employment number and feminist movements. Even though Dutch society became less stratified ideologically (depillarization), we found an increasing divergence in gender bias between religious and social-democratic on the one hand and liberal newspapers on the other. Methodologically, this paper illustrates how word embeddings can be used to examine historical language change. Future work will investigate how fine-tuning deep contextualized embedding models, such as ELMO, might be used for similar tasks with greater contextual information.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper trains separate word embedding models on six Dutch national newspapers (1950-1990) spanning different ideological backgrounds and compares gender bias via local changes in word associations. It claims directional shifts (bias moving toward women for sexuality/leisure themes but toward men generally) and increasing divergence between religious/social-democratic versus liberal papers despite depillarization, while illustrating embeddings for historical language analysis.
Significance. If the cross-model comparisons hold, the work offers a methodological illustration of embeddings for detecting historical bias trends in ideologically distinct corpora. No machine-checked proofs, reproducible code, or parameter-free derivations are described, so the significance rests entirely on the empirical claims about bias trajectories.
major comments (1)
- [Abstract and Methods] Abstract and Methods (implied): The central claim requires comparing cosine similarities or nearest-neighbor relations between gender terms and target words across six independently trained embedding models. No alignment procedure (Procrustes, CCA, or shared anchors) is reported. Because skip-gram/CBOW spaces are defined only up to orthogonal transformation, reported 'local changes' and 'increasing divergence' between newspapers can be artifacts of random initialization or vocabulary differences rather than textual bias shifts. This is load-bearing for all temporal and cross-paper findings.
minor comments (1)
- [Abstract] The abstract provides no description of the bias metric, statistical tests, error bars, corpus construction details, or model hyperparameters, making it impossible to assess whether reported directional shifts are supported.
Simulated Author's Rebuttal
We thank the referee for identifying this methodological concern, which is central to the validity of our cross-model comparisons. We address it directly below.
read point-by-point responses
-
Referee: [Abstract and Methods] Abstract and Methods (implied): The central claim requires comparing cosine similarities or nearest-neighbor relations between gender terms and target words across six independently trained embedding models. No alignment procedure (Procrustes, CCA, or shared anchors) is reported. Because skip-gram/CBOW spaces are defined only up to orthogonal transformation, reported 'local changes' and 'increasing divergence' between newspapers can be artifacts of random initialization or vocabulary differences rather than textual bias shifts. This is load-bearing for all temporal and cross-paper findings.
Authors: We agree that the absence of an alignment procedure is a substantive limitation for any claims involving comparisons across the six independently trained models. The manuscript as submitted does not describe or apply such a step (e.g., Procrustes, CCA, or anchor-based alignment), so the reported shifts and divergence could indeed be influenced by rotational differences or vocabulary mismatches. In the revised manuscript we will (1) implement Procrustes alignment of all six embedding spaces to a common reference space using a fixed set of high-frequency shared anchor words, (2) document the alignment procedure and the choice of anchors, and (3) re-compute the gender-bias measures and divergence statistics on the aligned spaces. We will also add a sensitivity check comparing results before and after alignment. This revision will be reflected in both the Methods and Results sections. revision: yes
Circularity Check
No significant circularity in the paper's methodological derivation.
full rationale
The paper trains independent embedding models on separate historical newspaper corpora and extracts association-based gender bias measures directly from those models. These operations are applied to external text data with no reported fitting of parameters that are subsequently renamed as predictions, no self-definitional reductions, and no load-bearing self-citations that justify uniqueness or ansatzes. The derivation chain is self-contained as a descriptive measurement technique and does not reduce any claimed result to its own inputs by construction.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Word embeddings trained on newspaper text capture semantic associations that reflect gender bias in the source material.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.