Pith. sign in

REVIEW 4 major objections 5 minor 55 references

Not Here, Go There: Analyzing Redirection Patterns on the Web

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that half of all redirecting URIs in the archived web end in errors, that canonical HTTP-to-HTTPS redirects dominate, and that nearly half of custom 404 pages are soft 404s returning HTTP 200.

desk verdict Large, reproducible redirect measurement whose soft-404 headline rests on an unstated 'contains 404' rule; the rest of the descriptive findings are solid and worth citing. read the letter →

arxiv 2507.22019 v1 pith:V3OC2WPH submitted 2025-07-29 cs.DL cs.IRcs.NI

Kritika Garg , Sawood Alam , Dietrich Ayala , Michele C. Weigle , Michael L. Nelson This is my paper
classification cs.DLcs.IRcs.NI
keywords URIredirectionredirectchainscanonicalsoft404sinkwebarchivinglinkrotHTTPstatuscodes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish, at web scale, what actually happens when a URI redirects: whether the chain reaches a live page, where it fails, and where the traffic lands. Using 11 million redirecting URIs sampled from a large web archive and re-crawled in 2023 with a ten-hop limit, the authors find that about half of the chains terminate successfully and half end in errors, with 0.06% exceeding ten hops. They also show that canonical redirects—mostly HTTP to HTTPS transitions—dominate the landscape, that a small set of 'sink' URIs absorb a disproportionate share of redirect traffic, and that among 62,000 custom 404 pages, nearly half are 'soft 404s' returning HTTP 200 instead of 404. The result matters because redirect behavior determines whether preserved links, search-engine value, and user navigation actually reach the intended content.

What carries the argument

The load-bearing object is the redirect chain, recorded hop by hop by a web crawler following each URI up to ten hops and labelling the final HTTP status as success, error, or indeterminate. The core classification device is the SURT (Sort-friendly URI Reordering Transform), a canonicalized key for URIs: when the SURT forms of source and target match exactly, the redirect is labelled canonical; otherwise it is non-canonical. Redirects the crawler cannot follow—because of invalid Location headers or client-side meta-refresh redirects—are labelled invalid and set aside as indeterminate. Sink URIs are then computed by aggregating source URIs and source domains by their shared target URI. Together these mechanisms carry the analysis: SURT enables the canonical/non-canonical split, the hop-by-hop crawl logs produce the success/error counts, and sink aggregation exposes the traffic-convergence patterns.

What would settle it

Re-run the same ten-hop redirect crawl on an independent, live-web sample—for example, a random sample of registered domains or a fresh URL index rather than an archive's historical index—and compare the success/error split and the fraction of custom 404s returning 200; if the 50/50 split and the 47% soft-404 rate do not approximately reproduce, the archive-derived dataset cannot be treated as representative of the broader web.

Watch

Extended reading notes

Core claim

In the paper's own terms, the central discovery is a quantitative map of redirect behaviour in the archived web. Following 9.5 million terminating redirect chains up to ten hops, the authors find a near-exact 50/50 split between successful (2xx) and failed terminations, with 0.06% of chains exceeding ten hops and 0.42% exceeding four hops. Canonical redirects account for roughly two-thirds of terminating chains (6 million of 9.5 million), and HTTP-to-HTTPS transitions alone make up about 4.6 million cases; these canonical chains resolve to a 2xx status only 48.70% of the time. Non-canonical redirects resolve to 2xx more often (about 73%) but frequently land on root pages, parked pages, or soft-error pages rather than the originally linked content. The authors further identify 'sink' URIs that collect redirects from many sources, and they find that of 62,000 custom 404 target URIs, only 46% actually return 404 while 47% return 200, meaning broken links are often masked as successes.

Load-bearing premise

The whole measurement rests on the assumption that URIs first archived between 1996 and 2021 and drawn from one archive's index are representative of redirect behavior on the web at large; if the sample skews toward abandoned or low-cost sites, the error and soft-404 rates will be overstated.

Editorial extensions

If this is right

  • A five-hop redirect cap—already the recommended practice—covers all but 0.42% of chains, so crawlers can budget resources with that bound in mind.
  • Because 47% of custom 404 pages return HTTP 200, link-rot and SEO audits that rely on status codes alone will systematically undercount broken pages; final page content must be inspected.
  • The concentration of traffic into sink URIs (login pages, custom 404s, parked domains, homepages) means web archives can waste storage on duplicates; archive crawlers could detect and deduplicate sinks.
  • The 13.22% of redirects with indeterminate termination (invalid or client-side redirects) form a class that crawler designs must handle explicitly rather than treating as ordinary 3xx chains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 50/50 success/error split probably reflects the age and provenance of archive-derived URIs; a sample of freshly created live-web URIs would likely show a healthier split, so the paper's numbers are best read as a lower bound on current redirect health rather than a universal snapshot.
  • Sink analysis suggests a cheap content-decay detector: a redirect landing on a known sink page (login wall, custom 404, conglomerate homepage) can flag link rot without parsing page text; this heuristic could be tested against manual content review.
  • The affinity of soft 404s with custom error URIs implies a crawl-time heuristic—if the target path contains '404' or 'error' but returns 200, flag it as a likely soft 404—whose false-positive rate could be measured in a follow-up study.
  • The Rickrolling sink and affiliate-marketing sinks show that deliberate redirects can be distinguished from accidental ones by sink concentration and source-domain relationships, which could be used to surface prank or affiliate networks at scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper analyzes a sample of 11 million unique redirecting URIs drawn from Internet Archive TimeMaps first archived between 1996 and 2021. The authors re-crawled these URIs in September 2023 with Heritrix, following up to 10 redirect hops, and report aggregate outcomes: roughly half of the URIs terminate in a 2xx success and half in an error; most redirects are one-hop; about 6 million canonical and 3.5 million non-canonical redirects are identified via SURT exact matching; and a small set of ``sink'' URIs account for large numbers of source URIs. The paper also claims to identify 62,000 custom 404 target URIs, of which 47% are soft 404s returning HTTP 200. The manuscript presents flow diagrams for canonical and non-canonical redirects, path-depth changes, domain-change statistics, and qualitative case studies of sink URIs including consolidation, affiliate marketing, login pages, loopback sinks, and a Rickroll sink.

Significance. If the quantitative claims hold, the paper would be a useful large-scale descriptive reference for redirect behavior in the archived web: it documents the prevalence of HTTP-to-HTTPS canonicalization, the dominance of single-hop redirects, the existence of high-degree sink URIs, and the frequency of invalid redirects. The public dataset and transparent crawl procedure are strengths. However, the headline 50/50 success/error split depends on how invalid redirects and >10-hop chains are counted, and the soft-404 finding rests on an unstated and demonstrably unreliable URI-string heuristic. Because the abstract and conclusions repeatedly emphasize the soft-404 result, that claim must be re-derived with a content-based validation before the paper's central contributions can be accepted.

major comments (4)
  1. [§5.3.8 and Tables 3–5] The 62,000 custom-404 count and the 47% soft-404 figure are not supported by a reproducible detection rule. The text says only that certain sinks ``contain the characters '404' in their URIs,'' and no algorithm is given for identifying the 62,000 targets. Table 5 directly contradicts a literal ``contains 404'' definition: http://chaturbate.com/affiliates/in/grq0/KZiNo/?track=404exit is an affiliate link that returns 429, not a 404 page, and https://www.wp.pl/?404&src01=99f53 has ``404'' as a query parameter on a portal page. The soft-404 share is therefore unverified and likely inflated. The authors must either (a) specify and validate a string-based rule that excludes such false positives, or (b) re-run the soft-404 analysis using page content (e.g., title/text containing ``Not Found'') to confirm that a target is actually an error page.
  2. [§3.2 and Figure 2] The denominator behind the headline 50% success / 50% error split is unclear. Section 3.2 reports 9.5 million URIs terminating within 10 redirects, 5.4 million of them successful, and then states that ``5.5 million URIs resulted in an error, including 1.5 million invalid redirects.'' Those two statements cannot both describe the 9.5-million dataset, and the abstract's 50/50 ratio matches the original 11 million if the 1.5 million invalid redirects and 6,068 over-10-hop URIs are counted as errors. In the 9.5-million final dataset the success share is about 57%, not 50%. Also, Figure 2's caption says ``3 million reach the second stage (R2),'' but the §3.2 numbers imply 11M − 6.9M ≈ 4.1M reach R2. The paper should state explicitly which denominator is used for each percentage and reconcile the R2/R3 counts.
  3. [§3.3 and Figure 1] The canonical/non-canonical split of 6 million versus 3.5 million depends entirely on ``exact match'' of the SURT transform, but the paper never specifies what normalization the SURT form applies. Figure 1's blue SURT strings show the same key for http://ecogeneration.com.au/, https://ecogeneration.com.au/, and https://www.ecogeneration.com.au/, which means the implementation strips both scheme and ``www''. The standard SURT transform does not remove ``www''; the background text in Section 2 also says SURT handles www, trailing slashes, and schemes, which is inconsistent with standard definitions. If a custom canonicalization was used, it should be precisely defined; otherwise the canonical/non-canonical classification is not reproducible and the reported ratio cannot be interpreted.
  4. [§3, first paragraph and §7] The paper generalizes to ``the web'' (title, abstract, conclusions) from a sample of URIs first archived by the Internet Archive between 1996 and 2021. The paper's own sink analysis shows heavy representation of .work TLDs, free hosting providers, expired domains, and abandoned content, which suggests the sample may overrepresent low-maintenance and low-cost sites. A quantitative statement about selection bias, or at least an explicit limitations paragraph acknowledging that the 50% error rate and soft-404 proportions may not generalize to the live web at large, is needed before the headline claims can be accepted as global web phenomena.
minor comments (5)
  1. [§3.1] The subtraction 11.7 million − 744,244 gives about 10.956 million, but the text reports 10,975,138; please check the arithmetic or clarify the rounding.
  2. [§3.2] The sentence ``only 0.42% of the redirects exceeded four hops without termination'' should define the population precisely (all URIs, or only those that reach the fourth hop) so the claimed support for a five-hop crawl cap is unambiguous.
  3. [Table 5] The caption says the table shows ``how some of these custom error pages are soft 404s,'' but the table includes a 429 status and a URI whose ``404'' is a query parameter; either annotate the detection rule or exclude rows that are not error pages.
  4. [§2, Figure 1] The description of SURT as removing www and scheme differences is non-standard; please add a precise definition or a citation to the SURT specification and state the exact normalization used in the crawler.
  5. [§3.2 and Figure 2] Figure 2's caption says 6068 redirects were still redirecting at hop 10, while Section 3.2 mentions ``6,000 URIs encountered more than 10 consecutive redirects''; please align these numbers and use a consistent count.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's headline measurements are direct observations of crawl logs, with no fitted parameters and no load-bearing self-citation chain.

full rationale

The study is observational: it re-crawls redirecting URIs and reports distributions of status codes, hop counts, canonical/non-canonical splits, and sink frequencies. None of these quantities is derived from an earlier fitted value or from a self-referential definition; the 50% success/error split and the status-code distributions are direct measurements of the crawl outcome. The sink category is introduced after observing that multiple source URIs converge on common targets, and the reported tables are empirical frequencies rather than predictions, so the category is descriptive rather than circular. Self-citations to the authors' earlier dataset construction [19,46] are used transparently as provenance for the sample, and the paper re-crawls the URIs itself in June and September 2023, so the central claims do not reduce to the cited work. The soft-404 analysis (Section 5.3.8) relies on an unstated operationalization that appears to equate 'custom 404 URI' with target URIs containing the substring '404', which can admit false positives such as the chaturbate affiliate link in Table 5. That is a measurement-validity concern about the soft-404 estimate, not a circular derivation: the terminating status codes are independently observed, and the paper makes no fitted quantity that is renamed as a prediction. No circular step can be exhibited under the required standard.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no fitted model parameters and no mathematical axioms. Its quantitative claims are conditioned on the crawl configuration (Heritrix, 10-hop cap), the SURT-based canonical classification, and target-URI string heuristics for custom 404s; these assumptions are listed above. The sink URI is a data-derived category, not an unobserved entity, so no invented entities are needed.

assumptions (4)
  • domain assumption The Internet Archive Zipnum sample of 27.3M URIs is representative enough to support conclusions about web redirect practices.
    The dataset is drawn from IA's index of URIs first archived between 1996 and 2021 (Section 3). The paper's title and conclusions generalize to 'the web', but this is a convenience sample of archived URIs, not a random sample of all web redirects.
  • domain assumption SURT exact-match equivalence is a valid operational definition of canonical redirect.
    Section 3.3 classifies redirects as canonical iff SURT forms of source and target URIs match. This equates different schemes, www prefixes, trailing slashes, and case changes, but may not capture all semantically canonical redirects and may include case-only changes.
  • domain assumption Heritrix's HTTP-only redirect following (up to 10 hops, without JavaScript execution) is a sufficient proxy for real-world redirect behavior.
    Section 3.1 describes the crawl; Section 5.3.7 acknowledges browsers execute JavaScript and trigger additional redirects, so the measured hop counts are a conservative floor, but the quantitative comparisons rest on crawler-visible HTTP redirects.
  • ad hoc to paper Custom 404 targets can be identified from target URI strings containing '404'.
    Section 5.3.8 and Table 5 enumerate 'custom error pages' that contain '404' in the URI. The paper never states this detection rule explicitly, and Table 5 includes a non-404 page (chaturbate affiliate URL returning 429), so the soft-404 estimates inherit this heuristic's errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Not Here, Go There: Analyzing Redirection Patterns on the Web." pith.science (2026). https://pith.science/paper/V3OC2WPH

@misc{pith2026250722019,
  author       = {Pith},
  title        = {Pith review of: Not Here, Go There: Analyzing Redirection Patterns on the Web},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V3OC2WPH}},
  note         = {Machine review of arXiv:2507.22019}
}
read the original abstract

URI redirections are integral to web management, supporting structural changes, SEO optimization, and security. However, their complexities affect usability, SEO performance, and digital preservation. This study analyzed 11 million unique redirecting URIs, following redirections up to 10 hops per URI, to uncover patterns and implications of redirection practices. Our findings revealed that 50% of the URIs terminated successfully, while 50% resulted in errors, including 0.06% exceeding 10 hops. Canonical redirects, such as HTTP to HTTPS transitions, were prevalent, reflecting adherence to SEO best practices. Non-canonical redirects, often involving domain or path changes, highlighted significant web migrations, rebranding, and security risks. Notable patterns included "sink" URIs, where multiple redirects converged, ranging from traffic consolidation by global websites to deliberate "Rickrolling." The study also identified 62,000 custom 404 URIs, almost half being soft 404s, which could compromise SEO and user experience. These findings underscore the critical role of URI redirects in shaping the web while exposing challenges such as outdated URIs, server instability, and improper error handling. This research offers a detailed analysis of URI redirection practices, providing insights into their prevalence, types, and outcomes. By examining a large dataset, we highlight inefficiencies in redirection chains and examine patterns such as the use of "sink" URIs and custom error pages. This information can help webmasters, researchers, and digital archivists improve web usability, optimize resource allocation, and safeguard valuable online content.

Figures

Figures reproduced from arXiv: 2507.22019 by the authors.

Figure 1
Figure 1. A snippet of a TimeMap retrieved from the IA’s CDX Server. These fields represent the SURT (canonicalized URI) (blue text), the datetime, the original URI (red text), the MIME type of the original document, the HTTP response code, and the length of the response record. Source URI Target URI Type of redirect http://blogs.nasa.gov/ https://blogs.nasa.gov/ HTTP to HTTPS Redirect http://ecogeneration.com.au/ https://www… view at source ↗
Figure 2
Figure 2. Journey of 11 million URIs through multiple redirects, labeled as R (initial redirects) to R4+ (redirection chains [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Distribution of the terminated status codes of the 11 million redirecting URIs [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Invalid redirect; a cURL request to a URI results in a 301 redirect with an invalid location header [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Invalid redirect; a cURL request to a URI results in a 301 redirect without a location header, relying instead on an [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 10
Figure 10. Figure 10: These often involve legacy pages that are no longer [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]
Figure 6
Figure 6. Figure 6: The flow of canonical redirects from their initial redirect statuses to their final terminating statuses. The left side [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Prevalence of different types of canonical redirects [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: The flow of non-canonical redirects from their initial redirecting statuses to their final terminating statuses. Two main [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: The flow of URLs transitioning from source path depths to target path depths during non-canonical redirection. Each [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Distribution of non-canonical redirects classified by changes in URI path depth. Redirects are divided into three [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Distribution of 3.5 million non-canonicalized redirections by host and domain changes. The majority (65%) of [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: A screenshot from the music video of “Never [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 49 canonical work pages

  1. [1]

    Teru Agata, Yosuke Miyata, Emi Ishita, Atsushi Ikeuchi, and Shuichi Ueda. 2014. Life Span of Web Pages: A Survey of 10 Million Pages Collected in 2001. In Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2014). IEEE, London, UK, 463–464. https://doi.org/10.1109/JCDL.2014.6970226

    work page arXiv 2014

  2. [2]

    Weigle, Michael L

    Sawood Alam, Kritika Garg, Michele C. Weigle, Michael L. Nelson, Mark Graham, and Dietrich Ayala. 2023. TrendMachine: A Temporal Webpage Resilience Portal. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL) . IEEE Press, Santa Fe, New Mexico, USA, 93–97. https://doi.org/10.1109/JCDL57899. 2023.00023

    work page arXiv 2023

  3. [3]

    Alkwai, Michael L

    Lulwah M. Alkwai, Michael L. Nelson, and Michele C. Weigle. 2017. Comparing the Archival Rate of Arabic, English, Danish, and Korean Language Web Pages. ACM Transactions on Information Systems 36, 1, Article 1 (June 2017), 34 pages. https://doi.org/10.1145/3041656

    work page doi:10.1145/3041656 2017

  4. [4]

    Nelson, Robert Sanderson, and Herbert Van de Sompel

    Ahmed AlSum, Michael L. Nelson, Robert Sanderson, and Herbert Van de Sompel

    work page

  5. [5]

    Internet Archive. 2024. Heritrix 3 Wiki. GitHub Wiki. https://github.com/ internetarchive/heritrix3/wiki

    work page 2024

  6. [6]

    Broder, Ravi Kumar, and Andrew Tomkins

    Ziv Bar-Yossef, Andrei Z. Broder, Ravi Kumar, and Andrew Tomkins. 2004. Sic transit gloria telae: towards an understanding of the web’s decay. In Proceedings of the 13th International Conference on World Wide Web (New York, NY, USA) (WWW ’04). Association for Computing Machinery, New York, NY, USA, 328–337. https://doi.org/10.1145/988672.988716

    work page arXiv 2004

  7. [7]

    Tim Berners-Lee, Robert Cailliau, Ari Luotonen, Henrik Frystyk Nielsen, and Arthur Secret. 1994. The World-Wide Web. Commun. ACM 37, 8 (August 1994), 76–82. https://doi.org/10.1145/179606.179671

    work page arXiv 1994

  8. [8]

    Fielding, and Lawrence Masinter

    Tim Berners-Lee, Roy T. Fielding, and Lawrence Masinter. 2005. Uniform Resource Identifier (URI): Generic Syntax, RFC 3986. https://www.rfc-editor.org/rfc/ rfc3986

    work page 2005

Show all 55 references
  1. [9]

    Nelson, and Michele C

    Haley Bragg, Himarsha Jayanetti, Michael L. Nelson, and Michele C. Weigle

  2. [10]

    Li Chang, Hsu-Chun Hsiao, Wei Jeng, Tiffany Hyun-Jin Kim, and Wei-Hsi Lin

  3. [11]

    Athena Chapekis, Samuel Bestvater, Emma Remy, and Gonzalo Rivero. 2024. When Online Content Disappears. https://www.pewresearch.org/data-labs/ 2024/05/17/when-online-content-disappears/

    2024

  4. [12]

    Junghoo Cho and Hector Garcia-Molina. 2000. The Evolution of the Web and Implications for an Incremental Crawler. In Proceedings of the 26th International Conference on Very Large Data Bases (VLDB ’00) . Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 200–209

    2000

  5. [13]

    Baeldung Contributors. 2024. Redirection Status Codes - Complete Guide. https: //www.baeldung.com/cs/redirection-status-codes

    2024

  6. [14]

    MDN Web Docs. 2024. HTTP Redirections. Mozilla Developer Network. https: //developer.mozilla.org/en-US/docs/Web/HTTP/Redirections Not Here, Go There: Analyzing Redirection Patterns on the Web Websci ’25, May 20–24, 2025, New Brunswick, NJ, USA

    2024

  7. [15]

    MDN Web Docs. 2024. HTTP Status Codes. Mozilla Developer Network. https: //developer.mozilla.org/en-US/docs/Web/HTTP/Status

    2024

  8. [16]

    Dennis Fetterly, Mark Manasse, Marc Najork, and Janet Wiener. 2003. A large- scale study of the evolution of web pages. In Proceedings of the 12th International Conference on World Wide Web (Budapest, Hungary) (WWW ’03). Association for Computing Machinery, New York, USA, 669–...

    2003

  9. [17]

    Roy Fielding and Julian Reschke. 2014. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content, RFC 7231. https://www.rfc-editor.org/rfc/rfc7231

    2014

  10. [18]

    Kritika Garg. 2024. Analyzing Redirects and Getting Rickrolled Along the Way. https://ws-dl.blogspot.com/2024/10/2024-10-22-analyzing-redirects-and.html

    2024

  11. [19]

    Weigle, and Michael L

    Kritika Garg, Sawood Alam, Michele C. Weigle, and Michael L. Nelson. 2025. Longitudinal Sampling of URLs From the Wayback Machine . Technical Report arXiv:2507.14752. arXiv

    2025 arXiv

  12. [20]

    Wendy Hall and Thanassis Tiropanis. 2012. Web evolution and Web Science. Computer Networks 56, 18 (2012), 3859–3865. https://doi.org/10.1016/j.comnet. 2012.10.004

    2012 doi

  13. [21]

    Helge Holzmann, Wolfgang Nejdl, and Avishek Anand. 2016. The Dawn of Today’s Popular Domains: A Study of the Archived German Web over 18 Years. In Proceedings of the 16th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2016). ACM, New Jersey, Newark, USA, 73–82. https:...

    2016

  14. [22]

    International Organization for Standardization. 2017. Information and documen- tation — WARC file format. https://www.iso.org/standard/68004.html

    2017

  15. [23]

    Internet Archive. 2023. Not Your Parents’ Web Dataset. https://archive.org/ details/not-your-parents-web

    2023

  16. [24]

    Ilya Kreymer Jack Cushman. 2017. Thinking like a hacker: Security Considera- tions for High-Fidelity Web Archives. http://labs.rhizome.org/presentations/ security.html

    2017

  17. [25]

    Jones, Herbert Van de Sompel, Harihar Shankar, Martin Klein, Richard Tobin, and Claire Grover

    Shawn M. Jones, Herbert Van de Sompel, Harihar Shankar, Martin Klein, Richard Tobin, and Claire Grover. 2016. Scholarly Context Adrift: Three out of Four URI References Lead to Changed Content. PLOS ONE 11, 12 (2016), e0167475. https://doi.org/10.1371/journal.pone.0167475

    2016 doi

  18. [26]

    Alkwai, Sawood Alam, Michael L

    Mat Kelly, Lulwah M. Alkwai, Sawood Alam, Michael L. Nelson, Michele C. Weigle, and Herbert Van de Sompel. 2017. Impact of URI Canonicalization on Memento Count. In Proceedings of the 17th ACM/IEEE Joint Conference on Digital Libraries (JCDL ’17). IEEE Press, Toronto, Ontario,...

    2017

  19. [27]

    Alkwai, Sawood Alam, Michael L

    Mat Kelly, Lulwah M. Alkwai, Sawood Alam, Michael L. Nelson, Michele C. Weigle, and Herbert Van de Sompel. 2017. Impact of URI Canonicalization on Memento Count. Technical Report arXiv:1703.03302. arXiv

    2017 arXiv

  20. [28]

    Martin Klein, Herbert Van de Sompel, Robert Sanderson, Harihar Shankar, Lyud- mila Balakireva, Ke Zhou, and Richard Tobin. 2014. Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot. PLOS ONE 9, 12 (2014), e115253. https://doi.org/10.1371/journal.pone.0115253

    2014 doi

  21. [29]

    Jeffery Kline, Edward Oakes, and Paul Barford. 2019. A URL-based Analysis of WWW Structure and Dynamics. InProceedings of the Network Traffic Measurement and Analysis Conference (TMA). IEEE Press, Paris, France, 81–800. https://doi. org/10.23919/TMA.2019.8784665

    2019

  22. [30]

    Wallace Koehler. 1999. An Analysis of Web Page and Web Site Constancy and Permanence. Journal of the American Society for Information Science 50, 2 (1999), 162–180. https://doi.org/10.1002/(SICI)1097-4571(1999)50:2<162::AID- ASI7>3.0.CO;2-B

    1999 doi

  23. [31]

    Martin Koop, Erik Tews, and Stefan Katzenbeisser. 2020. In-depth Evaluation of Redirect Tracking and Link Usage. Proceedings on Privacy Enhancing Technologies 2020, 4 (2020), 394–413. https://doi.org/10.2478/popets-2020-0079

    2020 doi

  24. [32]

    John Kurkowski. 2024. tldextract. https://pypi.org/project/tldextract/ Python package for extracting Top-Level Domain (TLD) from URLs

    2024

  25. [33]

    Taehyung Lee, Jinil Kim, Jin Wook Kim, Sung-Ryul Kim, and Kunsoo Park

  26. [34]

    Jonathan Leitschuh. 2019. Zoom Zero Day: 4+ Million Webcams & maybe an RCE? Just get them to visit your website! https://infosecwriteups.com/zoom- zero-day-4-million-webcams-maybe-an-rce-just-get-them-to-visit-your- website-ac75c83f4ef5

    2019

  27. [35]

    Moz Contributors. 2024. Canonicalization. https://moz.com/learn/seo/ canonicalization

    2024

  28. [36]

    Michael L. Nelson. 2021. Not Your Parents’ Web: The Scope and Archiving of the Modern Web. https://ws-dl.blogspot.com/2021/10/2021-10-20-not-your- parents-web-scope.html

    2021

  29. [37]

    Maile Ohye and Joachim Kupke. 2012. The Canonical Link Relation. https: //www.rfc-editor.org/info/rfc6596

    2012

  30. [38]

    Reddit users. 2022. Pinger.pl: A Polish Personal Blogging Platform - Discussion on ArchiveTeam. https://www.reddit.com/r/Archiveteam/comments/scp03k/ pingerpl_a_polish_personal_blogging_platform/

    2022

  31. [39]

    Panda, and Seema Verma

    Simple Sharma, Supriya P. Panda, and Seema Verma. 2022. Role and Analy- sis of Various SEO Strategies to Improve Website Ranking. In Proceedings of the International Conference on Machine Learning, Big Data, Cloud and Paral- lel Computing (COM-IT-CON) , Vol. 1. IEEE Press, Far...

    2022

  32. [40]

    Kushagra Singh, Gurshabad Grover, and Varun Bansal. 2020. How India Censors the Web. InProceedings of the 12th ACM Conference on Web Science (Southampton, United Kingdom) (WebSci ’20). Association for Computing Machinery, New York, USA, 21–28. https://doi.org/10.1145/3394231.3397891

    2020

  33. [41]

    2014–2021

    Webrecorder Software, Rhizome, and Contributors. 2014–2021. Zipnum Sharded Index. pywb 2.7 Documentation. https://pywb.readthedocs.io/en/latest/manual/ indexing.html#zipnum-sharded-index

    2014

  34. [42]

    Southern

    Matt G. Southern. 2020. Google Recommends Less Than 5 Hops Per Redirect Chain. Search Engine Journal. https://www.searchenginejournal.com/googles- john-mueller-recommends-less-than-5-hops-per-redirect-chain/344664/

    2020

  35. [43]

    Thompson

    Henry S. Thompson. 2024. Improved Methodology for Longitudinal Web Analyt- ics Using Common Crawl. In Proceedings of the 16th ACM Web Science Conference (Stuttgart, Germany) (WebSci ’24). Association for Computing Machinery, New York, USA, 59–69. https://doi.org/10.1145/361441...

    2024

  36. [44]

    Nelson, and Robert Sanderson

    Herbert Van de Sompel, Michael L. Nelson, and Robert Sanderson. 2013. HTTP framework for time-based access to resource states – Memento, Internet RFC

    2013

  37. [45]

    Xiaozhe Wang, Ajith Abraham, and Kate A. Smith. 2005. Intelligent Web Traffic Mining and Analysis. The Journal of Network and Computer Applications 28, 2 (April 2005), 147–165. https://doi.org/10.1016/j.jnca.2004.01.006

    2005 doi

  38. [46]

    Michele C. Weigle. 2024. Some URLs are Immortal, Most are Not. https://ws- dl.blogspot.com/2024/09/2024-09-20-some-urls-are-immortal-most.html

    2024

  39. [47]

    World Wide Web Consortium (W3C). 2008. H76: Using the title attribute of the iframe element. https://www.w3.org/TR/WCAG20-TECHS/H76.html

    2008

  40. [48]

    Huanwei Wu. 2011. Search Engine Optimization of E-Commerce Websites. In Proceedings of the International Conference on Management and Service Science . IEEE, Wuhan, China, 1–3. https://api.semanticscholar.org/CorpusID:35218016

    2011

  41. [49]

    Sonya Zhang and Neal Cabage. 2017. Search Engine Optimization: Comparison of Link Building and Social Sharing. Journal of Computer Information Systems 57 (2017), 148 – 159. https://api.semanticscholar.org/CorpusID:63144097

    2017

  42. [50]

    Zittrain, John Bowers, and Clare Stanton

    Jonathan L. Zittrain, John Bowers, and Clare Stanton. 2021. The Paper of Record Meets an Ephemeral Web: An Examination of Linkrot and Content Drift within The New York Times. SSRN Electronic Journal (2021), 1–13. https://doi.org/10. 2139/ssrn.3833133

    2021

  43. [2009]

    In Proceedings of the 18th International Conference on World Wide Web (Madrid, Spain) (WWW ’09)

    Detecting Soft Errors by Redirection Classification. In Proceedings of the 18th International Conference on World Wide Web (Madrid, Spain) (WWW ’09). Association for Computing Machinery, New York, USA, 1119–1120. https: //doi.org/10.1145/1526709.1526886

  44. [2013]

    In Proceedings of the 22nd International Conference on World Wide Web (WWW ’13 Companion)

    Archival HTTP redirection retrieval policies. In Proceedings of the 22nd International Conference on World Wide Web (WWW ’13 Companion) . Association for Computing Machinery, Rio de Janeiro, Brazil, 1051–1058. https://doi.org/10. 1145/2487788.2488117

  45. [2017]

    In Proceedings of the 26th International Conference on World Wide Web (Perth, Australia) (WWW ’17)

    Security Implications of Redirection Trail in Popular Websites Worldwide. In Proceedings of the 26th International Conference on World Wide Web (Perth, Australia) (WWW ’17) . International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1491–...

  46. [2023]

    In Proceedings of ACM/IEEE Joint Conference on Digital Libraries (JCDL)

    Less than 4% of Archived Instagram Account Pages for the Disinformation Dozen are Replayable. In Proceedings of ACM/IEEE Joint Conference on Digital Libraries (JCDL) . IEEE Press, Santa Fe, New Mexico, USA, 102–106. https: //doi.org/10.1109/JCDL57899.2023.00025

  47. [7089]

    http://tools.ietf.org/html/rfc7089

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.