Pith. sign in

REVIEW 3 major objections 6 minor 19 references

Web2Wiki: Characterizing Wikipedia Linking Across the Web

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Across 90.8 million web links, the paper finds English Wikipedia is used mainly to delegate explanations—background context—rather than to cite evidence, making Wikipedia web-scale knowledge infrastructure.

desk verdict A genuinely useful web-scale measurement paper whose headline 95% figure is softer than it looks, but the dataset and the topic/placement analyses carry it. read the letter →

arxiv 2505.15837 v1 pith:ZAECKVVQ submitted 2025-05-17 cs.SI cs.CYcs.DL

Veniamin Veselovsky , Tiziano Piccardi , Ashton Anderson , Robert West , Akhil Arora This is my paper · ORCID
classification cs.SIcs.CYcs.DL
keywords Wikipediahyperlinkanalysisweb-scalemeasurementdelegationvsevidenceknowledgeinfrastructurewebpagesegmentationreferencelinkingCommonCrawl
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Using a public archive of 2.7 billion web pages, the paper builds the Web2Wiki dataset of 90.8 million links to Wikipedia, reaching 1.68% of web domains. For English Wikipedia it claims the dominant linking pattern is delegation: 95% of sampled links point readers to Wikipedia for background explanation rather than to support a claimed fact. It also finds that 91% of links appear in main page content, and that news, science, and society sites link Wikipedia far more often than business and shopping sites, which tend to relegate links to boilerplate. If these figures hold, Wikipedia functions on the web mainly as background knowledge infrastructure—a shared explanatory layer—rather than as a source of proof. The released dataset lets others test and extend that picture.

What carries the argument

The load-bearing mechanism is the Web2Wiki dataset: a regex-based pass over all <a> tags in a web archive, producing link–domain–webpage records for 90.8 million Wikipedia links. Three instruments carry the analysis: Wikipedia's internal in-degree, computed from an XML dump, serves as the importance baseline; a rule-based page segmenter, iteratively developed on 500 inspected pages, labels each link's location as boilerplate, main content, or user contributions (main content precision 0.91, recall 0.96); and a two-category manual taxonomy distinguishes 'delegation' (link offers background explanation) from 'evidence' (link supports a fact or supplies sourced media). The hand annotation of 500 sampled webpages is what produces the 95/5 split that the paper generalizes to all English main-content links.

What would settle it

Take a fresh, domain-stratified sample of several thousand Wikipedia links from the released Web2Wiki dataset, have independent annotators classify each as delegation or evidence, and measure inter-annotator agreement. If the delegation share falls well below 95%, or if evidence links dominate on high-link domains such as news and science, the global generalization fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that Wikipedia's external web footprint is enormous but overwhelmingly explanatory rather than evidential. Extracting every hyperlink to Wikipedia from a 2.7-billion-page public web archive yields 90,805,367 links, of which 48,829,702 point to English Wikipedia from 14,462,267 distinct webpages across 940,239 domains; 80% of those links target only 9% of English articles, and 58% of English articles receive at least one external link. External linking correlates with Wikipedia's own in-degree ($r = 0.5$, $p < 0.001$) but differs by topic from both Wikipedia-internal linking and Reddit sharing. Within pages, 91% of links sit in main content, 7% in boilerplate, and 2% in user-generated sections. Manual annotation of 500 randomly sampled links places 95% in a 'delegation' category—sending readers elsewhere for context—and 5% in an 'evidence' category, from which the paper estimates 46.4 million delegation links and 2.4 million evidence links among English main-content references.

Load-bearing premise

The 95% delegation figure rests on 500 manually annotated webpages being representative of all 48.8 million English Wikipedia links; if that sample skews toward explanation-heavy pages, the headline split does not generalize.

Editorial extensions

If this is right

  • If the 95/5 split holds, the practical value of Wikipedia to the web is mostly as a shared explanatory layer: sites offload definitions and background to it, so its quality affects comprehension across millions of pages.
  • News, science, and society sites are the main carriers of this infrastructure, while business and shopping sites concentrate their links in boilerplate, so Wikipedia's reach is not uniform across web genres.
  • External web linking tracks Wikipedia's internal importance ($r = 0.5$) more closely than Reddit's sharing patterns, so social media should not be treated as a proxy for how the wider web uses Wikipedia.
  • The Web2Wiki dataset makes it possible to quantify Wikipedia's external reach article by article, enabling gap and bias audits that go beyond Wikipedia's own internal link network.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the delegation/evidence split is likely not uniform across languages; the released multilingual data could be tested for higher evidence shares in legal, medical, or other contexts with stronger attribution norms.
  • Beyond the paper: if delegation dominates, Wikipedia's influence on public understanding flows through unseen background links, implying that pageview metrics substantially understate Wikipedia's reach.
  • Beyond the paper: the same link-extraction and two-category coding approach could be applied to other reference sites, such as dictionaries or specialized encyclopedias, to test whether explanatory delegation is a general web pattern or specific to Wikipedia.
  • Beyond the paper: pairing this single web-archive snapshot with later crawls could measure how quickly Wikipedia links propagate through news cycles and whether the evidence share rises around controversial or rapidly developing topics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Web2Wiki, a dataset of Wikipedia links extracted from the February 2021 Common Crawl dump, containing 90,805,367 links to Wikipedia articles across 1.68% of crawled domains, with a detailed analysis of the 48.8 million English Wikipedia links. The authors address three questions: which articles are linked (compared with Wikipedia internal in-degree and Reddit citations via ORES topics), where links appear (domain topics via Homepage2Vec and in-page segmentation into boilerplate, main content, and user-generated sections), and why links are made (manual annotation into delegation versus evidence). The headline findings are that Wikipedia links are concentrated in news and science sites, that 91% of links appear in main content, and that 95% of links serve explanatory delegation rather than evidence, supporting the interpretation of Wikipedia as background knowledge infrastructure. The dataset and code are promised for public release.

Significance. If the findings hold, this is the first web-scale characterization of how Wikipedia is referenced across the general Web, going beyond the existing platform-specific studies on Reddit and search engines. The paper's strengths include the large-scale extraction from Common Crawl, the use of external baselines (Wikipedia in-degree, ORES, Homepage2Vec, Reddit) that avoid circularity, the bootstrapped confidence intervals in the topic comparisons, and the planned public release of a reusable dataset and code. The qualitative conclusion that Wikipedia predominantly functions as a delegation target rather than an evidence source is plausibly correct and would be a meaningful contribution to the field. However, the quantitative support for the two headline percentages (91% main-content and 95% delegation) is weaker than the paper presents: the main-content figure depends on a rule-based segmenter with low recall on user-generated content, and the delegation figure comes from a 500-item manual annotation with no reported reliability or sampling detail. These issues are fixable but require additional evidence or a recalibration of the claims.

major comments (3)
  1. [Section 6 / Appendix C] The 95% delegation versus 5% evidence estimate is load-bearing for the abstract, introduction, and conclusion, but the generalization from the manual annotation to all 48.8 million English links is not supported as reported. Section 6 says the authors 'randomly sampled 500 webpages that link to Wikipedia,' while Appendix C says they sampled '500 Web links'; these are different sampling units. Moreover, the text generalizes the resulting proportions specifically to 'the 48.8 million Wikipedia links in the main content,' even though the annotation sample is not described as restricted to main content. The paper reports no confidence intervals, no inter-annotator agreement, no codebook excerpt, and no stratification by segment or domain tail, yet it multiplies the 95/5 split to obtain 46.4 million delegation and 2.4 million evidence links. Appendix C itself concedes that intent is difficult to infer from links and that the taxonomy was reframed as a relationship classification. The authors should clarify the sampling unit, report reliability and confidence intervals, and either restrict the generalization claim to the population actually sampled or temper the headline claim.
  2. [Section 5 / Appendix B / Table 4] The claim that 91% of Wikipedia links appear in main content relies on the rule-based segmentation whose evaluation in Appendix B reports recall of only 0.57 for user-generated content and precision of 0.65 for boilerplate. With user-generated content recall at 0.57, the measured 2% share for this segment is likely an underestimate, and the main-content share of 91% may be correspondingly overstated. The paper does not report a sensitivity analysis or discuss how misclassification would affect Table 2. Additionally, Table 2's column header 'Webpages (M)' is inconsistent with the values: 43.7 million main-content webpages exceeds the 14.46 million total linking webpages reported in Table 1, so the column must refer to links rather than webpages. The authors should correct the table label and provide a robustness check on the segment percentages given the measured precision/recall.
  3. [Introduction / Section 6 / Appendix C] There is a mismatch between the paper's framing of RQ3 ('Why do people reference Wikipedia articles?') and the operational definition in Appendix C, which states that the authors moved away from intent inference and instead classified the 'relationship between Wikipedia and the site that invokes it.' The main text nonetheless presents delegation and evidence as motivations and uses this to support the conclusion that Wikipedia serves as background knowledge infrastructure. The authors should consistently describe the taxonomy as a functional or relational classification of link usage, not as a direct measurement of author intent, and should state the associated limitations explicitly in the main text rather than only in an appendix.
minor comments (6)
  1. [Abstract / throughout] The dataset name is inconsistently capitalized as 'Web2Wiki' in the abstract and 'WEB2WIKI' in the body; please standardize.
  2. [Section 3] The text 'we use Wikipedia in degrees' should read 'Wikipedia in-degrees.'
  3. [Section 6] The phrase 'bucket almost all the Wikipedia sharing links (98%)' is awkward; consider 'classify 98% of the links.'
  4. [Section 4 / Figure 1] The x-axis label 'Difference in probabilities' is vague; it should state that the difference is Web (or Reddit) probability minus Wikipedia-internal probability.
  5. [Section 4] The Pearson correlation r=0.5 is quoted without clarifying whether the correlation is computed on raw counts, log-counts, or probability vectors; please specify the exact variables.
  6. [Section 7] In the phrase 'Raphael’s School of Athens frequently appear,' the singular/plural agreement should be fixed ('frequently appears').

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 95% delegation estimate is a manual-annotation sample proportion, not an input-derived identity, and external baselines/models are independent.

full rationale

The reported findings are not derived from their own conclusions. The 90.8M-link count and 1.68%-of-domains statistic come directly from Common Crawl HTML parsing; the article-importance analysis uses Wikipedia in-degrees computed from an XML dump plus ORES topics; the where-analysis uses the externally published Homepage2Vec model and hand-defined segmentation rules evaluated with precision/recall; and the why-analysis uses manual annotation of 500 links/webpages, with the 95%/5% split being the annotated sample proportion rather than a fitted parameter or a definitional identity. No equation in the paper reduces to its output, and no fitted value is renamed as a prediction. The authors' self-citations (e.g., Homepage2Vec, prior Wikipedia-reading studies) are contextual or methodological and are not the load-bearing target of the present claims; Homepage2Vec in particular is an independent pretrained classifier, not an artifact of this paper. The sampling, annotation-reliability, and unit-of-analysis concerns about the 500-link generalization are validity and statistical-inference issues, not circularity. The derivation chain is therefore self-contained with respect to the circularity definitions: score 0.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No numerical parameters are fitted to the Web2Wiki data in this paper. The analysis uses pre-trained classifiers (ORES, Homepage2Vec), an external baseline (Wikipedia in-degree), and author-defined rules; each is an input, not a fitted constant.

assumptions (6)
  • domain assumption Common Crawl's February 2021 snapshot represents the surface Web for this analysis
    A single crawl is used throughout, and the paper acknowledges that login-gated platforms such as Reddit, Facebook, and Twitter are excluded (Section 3, Appendix A).
  • domain assumption Wikipedia internal in-degree is a valid baseline for article importance
    Used in Section 4 to interpret which articles are disproportionately linked; the paper cites prior work but does not validate the baseline for this purpose.
  • domain assumption ORES article topic classifications are sufficiently accurate
    All topic-level comparisons in Section 4 rely on ORES 64-category labels; no accuracy evaluation is reported in this paper.
  • domain assumption Homepage2Vec topic scores represent website categories
    Section 5 compares Wikipedia-linking domains to the general web using Homepage2Vec predictions; no in-paper accuracy check is provided.
  • domain assumption Rule-based segment assignments capture boilerplate, main content, and user contributions
    Section 5 and Appendix B report precision and recall for the rules, with user-generated recall 0.57 and boilerplate precision 0.65, so the segment proportions carry measurement error.
  • domain assumption Manual labels on 500 webpages generalize to all links for motivation
    Section 6 extrapolates the 95% delegation and 5% evidence split to 48.8M links without confidence intervals or inter-annotator agreement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Web2Wiki: Characterizing Wikipedia Linking Across the Web." pith.science (2026). https://pith.science/paper/ZAECKVVQ

@misc{pith2026250515837,
  author       = {Pith},
  title        = {Pith review of: Web2Wiki: Characterizing Wikipedia Linking Across the Web},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZAECKVVQ}},
  note         = {Machine review of arXiv:2505.15837}
}
read the original abstract

Wikipedia is one of the most visited websites globally, yet its role beyond its own platform remains largely unexplored. In this paper, we present the first large-scale analysis of how Wikipedia is referenced across the Web. Using a dataset from Common Crawl, we identify over 90 million Wikipedia links spanning 1.68% of Web domains and examine their distribution, context, and function. Our analysis of English Wikipedia reveals three key findings: (1) Wikipedia is most frequently cited by news and science websites for informational purposes, while commercial websites reference it less often. (2) The majority of Wikipedia links appear within the main content rather than in boilerplate or user-generated sections, highlighting their role in structured knowledge presentation. (3) Most links (95%) serve as explanatory references rather than as evidence or attribution, reinforcing Wikipedia's function as a background knowledge provider. While this study focuses on English Wikipedia, our publicly released Web2Wiki dataset includes links from multiple language editions, supporting future research on Wikipedia's global influence on the Web.

Figures

Figures reproduced from arXiv: 2505.15837 by the authors.

Figure 1
Figure 1. Comparison of proportion of Wikipedia-internal [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison between websites that link to Wikipe [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparison of where Wikipedia links are likely to [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages

  1. [1]

    For most authors... (a) Would answering this research question advance sci- ence without violating social contracts, such as violat- ing privacy norms, perpetuating unfair profiling, exac- erbating the socio-economic divide, or implying disre- spect to societies or cultures? Yes. (b) Do your main claims in the abstract and introduction accurately reflect ...

    work page

  2. [2]

    Additionally, if your study involves hypotheses testing... (a) Did you clearly state the assumptions underlying all theoretical results? N/A (b) Have you provided justifications for all theoretical re- sults? N/A (c) Did you discuss competing hypotheses or theories that might challenge or complement your theoretical re- sults? N/A (d) Have you considered ...

  3. [3]

    (a) Did you state the full set of assumptions of all theoret- ical results? N/A (b) Did you include complete proofs of all theoretical re- sults? N/A

    Additionally, if you are including theoretical proofs... (a) Did you state the full set of assumptions of all theoret- ical results? N/A (b) Did you include complete proofs of all theoretical re- sults? N/A

  4. [4]

    Additionally, if you ran machine learning experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? N/A (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? N/A (c) Did you report error bars (...

    work page

  5. [5]

    Additionally, if you are using existing assets (e.g., code, data, models) or curating/releasing new assets, without compromising anonymity... (a) If your work uses existing assets, did you cite the cre- ators? Yes (b) Did you mention the license of the assets? Yes (c) Did you include any new assets in the supplemental material or as a URL? No. The collect...

    work page

  6. [6]

    no- follow

    Additionally, if you used crowdsourcing or conducted research with human subjects, without compromising anonymity... (a) Did you include the full text of instructions given to participants and screenshots? N/A (b) Did you describe any potential participant risks, with mentions of Institutional Review Board (IRB) ap- provals? N/A (c) Did you include the es...

    work page 2010

  7. [14]

    Sample 1000 random Wikipedia links

    work page

  8. [15]

    Label each link as belonging to one of the three classes, namely boilerplate, main content, or user generated

    work page

Show all 19 references
  1. [16]

    Learn the rules using the 1000 Wikipedia links (labeled in Step 2) and define a rule-based classification model

  2. [17]

    Sample 500 random Wikipedia links and apply the learned rules

  3. [18]

    Refine the rules based on false positives and negatives obtained on the 500 links (sampled in Step 4)

  4. [19]

    blogroll

    Repeat until convergence. Using the aforementioned iterative process, we learn structural rules that can be grouped into two broad HTML categories, namely tag-based and attribute-based. Then, for each Wikipedia link we iterate through its ancestors in the DOM tree to determine...

  5. [2006]

    In Inter- national workshop on algorithms and models for the web- graph, 59–71

    Approximating PageRank from in-degree. In Inter- national workshop on algorithms and models for the web- graph, 59–71. Springer. Garcia-Duran, A.; Arora, A.; and West, R. 2022. Efficient Entity Candidate Generation for Low-Resource Languages. In Proceedings of the Thirteenth L...

    2022

  6. [2012]

    University of Mas- sachusetts, Amherst, Tech

    Wikilinks: A large-scale cross-document coreference corpus labeled via links to Wikipedia. University of Mas- sachusetts, Amherst, Tech. Rep. UM-CS-2012, 15. Spitkovsky, V . I.; and Chang, A. X. 2012. A cross-lingual dictionary for english wikipedia concepts. In LREC, 3168– 31...

    2012

  7. [2015]

    In Proceedings of the International AAAI Conference on Web and Social Me- dia, volume 9, 493–502

    Misalignment between supply and demand of quality content in peer production communities. In Proceedings of the International AAAI Conference on Web and Social Me- dia, volume 9, 493–502. WMF, M. 2022. A New Research Roadmap For Addressing Knowledge Gaps. Available: https://di...

    2022

  8. [2018]

    This is primarily because of the following three requirements

    to perform webpage segmentation, here we rely on a simple and intuitive rule-based approach that we propose to segment a webpage into three classes, namely boilerplate, main content, and user generated. This is primarily because of the following three requirements. First, owin...

  9. [2019]

    The sum of all human knowledge

    Why the World Reads Wikipedia: Beyond English Speakers. In WSDM, 618–626. Lugeon, S.; Piccardi, T.; and West, R. 2022. Home- page2Vec: Language-Agnostic Website Embedding and Classification. In Proceedings of the International AAAI Conference on Web and Social Media , volume 1...

    2022

  10. [2023]

    Journal of Economics & Manage- ment Strategy, 32(3): 657–669

    Wikipedia matters. Journal of Economics & Manage- ment Strategy, 32(3): 657–669. Holloway, T.; Bozicevic, M.; and B¨orner, K. 2007. Analyz- ing and visualizing the semantic coverage of Wikipedia and its authors. Complexity, 12(3): 30–40. Kohlsch¨utter, C.; Fankhauser, P.; and ...

    2007

  11. [2024]

    In Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Process- ing, 22796–22819

    Entity Insertion in Multilingual Linked Corpora: The Case of Wikipedia. In Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Process- ing, 22796–22819. Forte, A.; Andalibi, N.; Gorichanaz, T.; Kim, M. C.; Park, T.; and Halfaker, A. 2018. Information...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.