pith. sign in

arxiv: 2406.06958 · v2 · submitted 2024-06-11 · 💻 cs.CR · cs.CY· cs.MA· cs.NI· cs.SI

Towards Multi-Stakeholder Vulnerability Notifications in the Ad-Tech Supply Chain

Yash Vekaria (1) , Rishab Nithyanand (2) , Zubair Shafiq (1) ((1) University of California , Davis , (2) University of Iowa) This is my paper

Pith reviewed 2026-05-24 00:19 UTC · model grok-4.3

classification 💻 cs.CR cs.CYcs.MAcs.NIcs.SI
keywords dark poolingvulnerability notificationsad-tech supply chainmulti-stakeholderonline advertisingad inventory fraudsupply chain vulnerabilities
0
0 comments X

The pith

Notifications to ad-networks reduce dark pooling vulnerabilities in the ad-tech supply chain.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines whether notifying different parties in the opaque ad-tech supply chain can fix issues like dark pooling, in which low-quality publishers mix their ad inventory with higher-quality sites to mislead advertisers. It created and operated the first automated pipeline that detects these vulnerabilities and sends notifications to publishers, ad-networks, and advertisers. A nine-month study found that the notifications lowered dark pooling rates, with the largest drops occurring when ad-networks were the target. Responses were statistically similar whether the notifications came from academics or activists. This points to a practical way to address fraud when multiple parties must coordinate despite conflicting incentives.

Core claim

The authors implemented the first online advertising supply chain vulnerability notification pipeline to systematically evaluate the responsiveness of various stakeholders in ad-tech supply chain, including publishers, ad-networks, and advertisers to vulnerability notifications by academics and activists. Our nine-month long automated multi-stakeholder notification study shows that notifications are an effective method for reducing dark pooling vulnerabilities in the online advertising ecosystem, especially when targeted towards ad-networks. Further, the sender reputation does not impact responses to notifications from activists and academics in a statistically different way.

What carries the argument

An automated multi-stakeholder vulnerability notification pipeline that detects dark pooling and contacts publishers, ad-networks, and advertisers.

If this is right

  • Notifications reduce dark pooling most effectively when sent to ad-networks.
  • Academic and activist senders receive statistically equivalent responses.
  • The method supports industry-scale efforts to combat ad inventory fraud.
  • Multi-stakeholder notifications are feasible in supply chains with misaligned incentives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The results suggest similar notification approaches could be tested in other complex digital supply chains.
  • Ad-networks may act as high-impact intervention points in ecosystems with interdependent roles.
  • Longer-term tracking could reveal whether reductions persist after the notification period ends.

Load-bearing premise

Reductions in dark pooling can be attributed to the notifications rather than external factors, and the automated pipeline detects vulnerabilities and measures responses without major error or bias.

What would settle it

A comparison showing no reduction in dark pooling after notifications in a controlled setting, or direct evidence of large errors in the vulnerability detection or response tracking.

Figures

Figures reproduced from arXiv: 2406.06958 by (2) University of Iowa), Davis, Rishab Nithyanand (2), Yash Vekaria (1), Zubair Shafiq (1) ((1) University of California.

Figure 1
Figure 1. Figure 1: Stakeholders in the ad-tech supply chain [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Our threat model representing ad-tech supply chain [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An overview of the timeline of the the notification campaign. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: An illustration of the measured remediation metrics before and after notifications in different rounds to different [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Categorization of notification responses by sender reputation under different themes for [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
read the original abstract

Online advertising relies on a complex and opaque supply chain that involves multiple stakeholders, including advertisers, publishers, and ad-networks, each with distinct and sometimes conflicting incentives. Recent research has demonstrated the existence of ad-tech supply chain vulnerabilities such as dark pooling, where low-quality publishers bundle their ad inventory with higher-quality ones to mislead advertisers. We investigate the effectiveness of vulnerability notification campaigns aimed at mitigating dark pooling. Prior research on vulnerability notifications have primarily explored single-stakeholder contexts, leaving multi-stakeholder scenarios understudied. There is limited attention to complex multi-stakeholder supply chain ecosystems such as ad-tech supply chain, where resolving vulnerabilities often requires coordinated action across entities with misaligned incentives and interdependent roles. We address this gap by implementing the first online advertising supply chain vulnerability notification pipeline to systematically evaluate the responsiveness of various stakeholders in ad-tech supply chain, including publishers, ad-networks, and advertisers to vulnerability notifications by academics and activists. Our nine-month long automated multi-stakeholder notification study shows that notifications are an effective method for reducing dark pooling vulnerabilities in the online advertising ecosystem, especially when targeted towards ad-networks. Further, the sender reputation does not impact responses to notifications from activists and academics in a statistically different way. Overall, our research fosters industry-scale solution to combat ad inventory fraud and fosters future research on feasibility of multi-stakeholder vulnerability notifications in other supply chain ecosystems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript describes the design and deployment of an automated multi-stakeholder vulnerability notification pipeline targeting dark pooling in the ad-tech supply chain. It reports results from a nine-month observational study claiming that notifications reduce dark pooling, with greater effectiveness when directed at ad-networks than at publishers or advertisers, and that sender reputation (academic vs. activist) produces no statistically distinguishable difference in responses.

Significance. If the causal attribution and measurement validity can be established, the work would be the first systematic empirical evaluation of multi-stakeholder notifications in a complex, incentive-misaligned supply chain and could inform scalable industry responses to ad inventory fraud.

major comments (3)
  1. [Study methods] The description of the nine-month automated study supplies no information on the detection pipeline used to identify dark pooling before and after notifications (metrics, thresholds, validation against ground truth, or false-positive rates).
  2. [Results and attribution analysis] No control conditions, baseline trends, statistical tests, or adjustment for confounding factors (market changes, platform policy shifts, or detection artifacts) are reported to support the claim that observed reductions are attributable to the notifications.
  3. [Stakeholder response measurement] The measurement of stakeholder responses, handling of non-responses, and criteria for classifying a notification as effective lack any description of selection criteria, response coding, or inter-rater reliability, undermining the central effectiveness claim.
minor comments (1)
  1. [Abstract] The abstract states quantitative conclusions without accompanying sample sizes, effect magnitudes, or confidence intervals.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive feedback, which identifies key areas where additional detail and analysis would strengthen the manuscript. We respond to each major comment below, indicating planned revisions.

read point-by-point responses

  1. Referee: [Study methods] The description of the nine-month automated study supplies no information on the detection pipeline used to identify dark pooling before and after notifications (metrics, thresholds, validation against ground truth, or false-positive rates).

    Authors: We agree that the manuscript would benefit from greater detail on the detection pipeline. The revised version will add a dedicated subsection describing the metrics and thresholds used to identify dark pooling, the validation approach against available ground truth samples, and estimated false-positive rates derived from manual review of a detection subset. revision: yes

  2. Referee: [Results and attribution analysis] No control conditions, baseline trends, statistical tests, or adjustment for confounding factors (market changes, platform policy shifts, or detection artifacts) are reported to support the claim that observed reductions are attributable to the notifications.

    Authors: The study is observational by design, as randomized controlled interventions are impractical in a live commercial supply chain. In revision we will incorporate baseline trend analysis, before-after statistical tests, and explicit discussion of potential confounders. We cannot retroactively introduce control conditions that were not part of the original protocol. revision: partial

  3. Referee: [Stakeholder response measurement] The measurement of stakeholder responses, handling of non-responses, and criteria for classifying a notification as effective lack any description of selection criteria, response coding, or inter-rater reliability, undermining the central effectiveness claim.

    Authors: We will expand the methods section to specify the selection criteria for notifications, the rule-based coding scheme for responses, and the handling of non-responses. Inter-rater reliability assessment is not applicable because response classification followed deterministic, predefined rules rather than subjective human judgment. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical observational study with external measurements

full rationale

The paper reports results from a nine-month automated notification study measuring stakeholder responses in the ad-tech supply chain. No mathematical derivations, fitted parameters, predictions, or self-citations are described in the provided text that reduce the central claim to its own inputs by construction. The study relies on external observations of dark pooling reductions and responses, which are independent of any internal redefinition or self-referential fitting. This matches the default case of a self-contained empirical paper.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on domain assumptions about stakeholder responsiveness and accurate vulnerability detection in a complex ecosystem; no free parameters or invented entities are introduced as this is an empirical evaluation study.

axioms (1)
  • domain assumption Stakeholders in the ad-tech supply chain will respond to vulnerability notifications in measurable ways despite misaligned incentives
    Invoked in the abstract when describing the gap and the need for coordinated action across entities.

pith-pipeline@v0.9.0 · 5822 in / 1160 out tokens · 21352 ms · 2026-05-24T00:19:16.951940+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Understanding Data Collection, Brokerage, and Spam in the Lead Marketing Ecosystem

    cs.CR 2026-04 unverdicted novelty 8.0

    An empirical study of over 100 lead-generation websites and 200 controlled contacts shows sensitive health data shared with more than 70 parties, leading to thousands of immediate marketing calls and texts, with evide...

Reference graph

Works this paper leans on

78 extracted references · 78 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://newormedia .com/blog/ 2023-advertising-spend-2023-benchmarks/, 2023

    2023 advertising spend & benchmarks. https://newormedia .com/blog/ 2023-advertising-spend-2023-benchmarks/, 2023

    work page 2023

  2. [2]

    https://fraudblocker .com/ad-fraud-data-facts, 2023

    Juniper research. https://fraudblocker .com/ad-fraud-data-facts, 2023

    work page 2023

  3. [3]

    V oelker

    Paul Pearce, Vacha Dave, Chris Grier, Kirill Levchenko, Saikat Guha, Damon McCoy, Vern Paxson, Stefan Savage, and Geoffrey M. V oelker. Characterizing large-scale click fraud in zeroaccess. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security , CCS ’14, page 141–152, New York, NY , USA, 2014. Association for Computing Machinery

    work page 2014

  4. [4]

    The economics of click fraud

    Nir Kshetri. The economics of click fraud. IEEE Security & Privacy, 8(3):45–53, 2010

    work page 2010

  5. [5]

    Impression fraud in on-line advertising via {Pay-Per-View} networks

    Kevin Springborn and Paul Barford. Impression fraud in on-line advertising via {Pay-Per-View} networks. In 22nd USENIX Security Symposium, pages 211–226, 2013

    work page 2013

  6. [6]

    Affiliate crookies: Characterizing affiliate marketing abuse

    Neha Chachra, Stefan Savage, and Geoffrey M V oelker. Affiliate crookies: Characterizing affiliate marketing abuse. In Proceedings of the 2015 Internet Measurement Conference , pages 41–47, 2015

    work page 2015

  7. [7]

    Understanding fraudulent activities in online ad exchanges

    Brett Stone-Gross, Ryan Stevens, Apostolis Zarras, Richard Kem- merer, Chris Kruegel, and Giovanni Vigna. Understanding fraudulent activities in online ad exchanges. In Proceedings of the 2011 ACM SIGCOMM conference on IMC , pages 279–294, 2011

    work page 2011

  8. [8]

    Vekaria, R

    Y . Vekaria, R. Nithyanand, and Z. Shafiq. The inventory is dark and full of misinformation: Understanding ad inventory pooling in the ad-tech supply chain. In 2024 IEEE Symposium on Security and Privacy (SP), pages 7–7, Los Alamitos, CA, USA, may 2024. IEEE Computer Society

    work page 2024

  9. [9]

    Markatos, and Nicolas Kourtellis

    Emmanouil Papadogiannakis, Panagiotis Papadopoulos, Evangelos P. Markatos, and Nicolas Kourtellis. Who funds misinformation? a systematic analysis of the ad-related profit routines of fake news sites. In Proceedings of ACM WWW 2023 , pages 2765–2776, 2023

    work page 2023

  10. [10]

    Placement laundering and the complexities of attribution in online advertising

    Jeffery Kline, Aaron Cahn, and Paul Barford. Placement laundering and the complexities of attribution in online advertising. arXiv preprint arXiv:2208.07310, 2022

    work page arXiv 2022

  11. [11]

    https://www .linkedin.com/pulse/programmatic-ad-fraud- ongoing-fight-between-adtech-players-novikova/, 2023

    Programmatic ad fraud: The ongoing fight between adtech players and bad actors. https://www .linkedin.com/pulse/programmatic-ad-fraud- ongoing-fight-between-adtech-players-novikova/, 2023

    work page 2023

  12. [12]

    https://iabeurope.kinsta.cloud/news-blog/driving-transparency- across-the-digital-advertising-supply-chain/

    Driving transparency across the digital advertising supply chain. https://iabeurope.kinsta.cloud/news-blog/driving-transparency- across-the-digital-advertising-supply-chain/

    work page

  13. [13]

    https://iabtechlab .com/wp- content/uploads/2022/04/Ads.txt-1.1.pdf, 2022

    IAB ads.txt Specifications Version 1.1. https://iabtechlab .com/wp- content/uploads/2022/04/Ads.txt-1.1.pdf, 2022

    work page 2022

  14. [14]

    https://iabtechlab .com/wp-content/ uploads/2019/07/Sellers.json Final.pdf, 2019

    IAB sellers.json Specifications. https://iabtechlab .com/wp-content/ uploads/2019/07/Sellers.json Final.pdf, 2019

    work page 2019

  15. [15]

    https://iabtechlab .com/ads-cert/, 2022

    Ads.cert 2.0. https://iabtechlab .com/ads-cert/, 2022

    work page 2022

  16. [16]

    IAB: https://www .iab.com/guidelines/ openrtb/, 2022

    OpenRTB Guidelines. IAB: https://www .iab.com/guidelines/ openrtb/, 2022

    work page 2022

  17. [17]

    A longitudinal analysis of the ads

    Muhammad Ahmad Bashir, Sajjad Arshad, Engin Kirda, William Robertson, and Christo Wilson. A longitudinal analysis of the ads. txt standard. In Proceedings of the Internet Measurement Conference, pages 294–307, 2019

    work page 2019

  18. [18]

    The devil is in the details: Analyzing the lucrative ad fraud patterns of the online ad ecosystem

    Emmanouil Papadogiannakis, Nicolas Kourtellis, Panagiotis Pa- padopoulos, and Evangelos P Markatos. The devil is in the details: Analyzing the lucrative ad fraud patterns of the online ad ecosystem. arXiv preprint arXiv:2306.08418 , 2023

    work page arXiv 2023

  19. [19]

    The Three Deadly Sins of Ads

    S Tingleff. The Three Deadly Sins of Ads. Txt and How Publishers Can Avoid Them. https://iabtechlab .com/blog/the-three-deadly-sins- of-ads-txt-and-how-publishers-can-avoid-them, 2019

    work page 2019

  20. [20]

    Establishing trust in online advertising with signed transactions

    Antonio Pastor, Rub ´en Cuevas, ´Angel Cuevas, and Arturo Azcorra. Establishing trust in online advertising with signed transactions. IEEE Access, 9, 2020

    work page 2020

  21. [21]

    Accountability in a smoke-filled room: The inadequacy of self regulation within the internet behavioral advertising industry

    Julia Zukina. Accountability in a smoke-filled room: The inadequacy of self regulation within the internet behavioral advertising industry. Brook. J. Corp. Fin. & Com. L. , 7:277, 2012

    work page 2012

  22. [22]

    Protecting privacy online: Is self-regulation working? Journal of Public Policy & Marketing , 19(1):20–26, 2000

    Mary J Culnan. Protecting privacy online: Is self-regulation working? Journal of Public Policy & Marketing , 19(1):20–26, 2000

    work page 2000

  23. [23]

    The law and policy of online privacy: Regulation, self-regulation, or co-regulation

    Dennis D Hirsch. The law and policy of online privacy: Regulation, self-regulation, or co-regulation. Seattle UL Rev., 34:439, 2010

    work page 2010

  24. [24]

    Hey, you have a problem: On the feasibility of {Large-Scale} web vulnerability notification

    Ben Stock, Giancarlo Pellegrino, Christian Rossow, Martin Johns, and Michael Backes. Hey, you have a problem: On the feasibility of {Large-Scale} web vulnerability notification. In 25th USENIX Security Symposium (USENIX Security 16) , pages 1015–1032, 2016

    work page 2016

  25. [25]

    You’ve got vulnerability: Exploring effective vulnerability notifications

    Frank Li, Zakir Durumeric, Jakub Czyz, Mohammad Karami, Michael Bailey, Damon McCoy, Stefan Savage, and Vern Paxson. You’ve got vulnerability: Exploring effective vulnerability notifications. In 25th USENIX Security Symposium , pages 1033–1050, 2016

    work page 2016

  26. [26]

    Remedying web hijacking: Notification effectiveness and webmaster comprehension

    Frank Li, Grant Ho, Eric Kuan, Yuan Niu, Lucas Ballard, Kurt Thomas, Elie Bursztein, and Vern Paxson. Remedying web hijacking: Notification effectiveness and webmaster comprehension. In Proceed- ings of the 25th International Conference on World Wide Web , pages 1009–1019, 2016

    work page 2016

  27. [27]

    Understanding the role of sender reputation in abuse reporting and cleanup

    Orcun Cetin, Mohammad Hanif Jhaveri, Carlos Ga ˜n´an, Michel van Eeten, and Tyler Moore. Understanding the role of sender reputation in abuse reporting and cleanup. Journal of Cybersecurity, 2(1):83–98, 2016

    work page 2016

  28. [28]

    Make notifications great again: learning how to notify in the age of large-scale vulnerability scanning

    Orcun Cetin, Carlos Ganan, Maciej Korczynski, and Michel van Eeten. Make notifications great again: learning how to notify in the age of large-scale vulnerability scanning. In Workshop on the Economics of Information Security (WEIS) , 2017

    work page 2017

  29. [29]

    Tell me you fixed it: Evaluating vulnerability notifications via quarantine networks

    Orc ¸un C ¸ etin, Carlos Ga ˜n´an, Lisette Altena, Samaneh Tajal- izadehkhoob, and Michel Van Eeten. Tell me you fixed it: Evaluating vulnerability notifications via quarantine networks. In 2019 IEEE European Symposium on Security and Privacy (EuroS&P) , pages 326–339. IEEE, 2019

    work page 2019

  30. [30]

    Snail mail beats email any day: on effective operator security notifications in the internet

    Max Maass, Marc-Pascal Clement, and Matthias Hollick. Snail mail beats email any day: on effective operator security notifications in the internet. In Proceedings of the 16th International Conference on Availability, Reliability and Security , pages 1–13, 2021

    work page 2021

  31. [31]

    Effective notification campaigns on the web: A matter of trust, framing, and support

    Max Maass, Alina St ¨over, Henning Prid ¨ohl, Sebastian Bretthauer, Dominik Herrmann, Matthias Hollick, and Indra Spiecker. Effective notification campaigns on the web: A matter of trust, framing, and support. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2489–2506, 2021

    work page 2021

  32. [32]

    Sleeping giants and indirect boycotts against the far right in united states of america

    Claudia Pereira Ferraz. Sleeping giants and indirect boycotts against the far right in united states of america. Aurora., 14(40):28–47, 2021

    work page 2021

  33. [33]

    Activism, advertising, and far-right media: The case of sleeping giants

    Joshua A Braun, John D Coakley, and Emily West. Activism, advertising, and far-right media: The case of sleeping giants. Media and Communication, 7(4):68–79, 2019

    work page 2019

  34. [34]

    https://checkmyads .org/, 2022

    Check My Ads Institute. https://checkmyads .org/, 2022

    work page 2022

  35. [35]

    Enhancing user transparency in online ads ecosystem with site selfdisclosures, 2018

    Lukasz Olejnik. Enhancing user transparency in online ads ecosystem with site selfdisclosures, 2018

    work page 2018

  36. [36]

    Do malware reports expedite cleanup? an experimental study

    Marie Vasek and Tyler Moore. Do malware reports expedite cleanup? an experimental study. USENIX Association, 2012

    work page 2012

  37. [37]

    Fixing https misconfigurations at scale: An experiment with security notifications

    Eric Zeng, Frank Li, Emily Stark, Adrienne Porter Felt, and Parisa Tabriz. Fixing https misconfigurations at scale: An experiment with security notifications. 2019

    work page 2019

  38. [38]

    Didn’t you hear me?—towards more successful web vulnerability notifications

    Ben Stock, Giancarlo Pellegrino, Frank Li, Michael Backes, and Christian Rossow. Didn’t you hear me?—towards more successful web vulnerability notifications. 2018. 14

    work page 2018

  39. [39]

    Exit from hell? reducing the impact of {Amplification}{DDoS} attacks

    Marc K ¨uhrer, Thomas Hupperich, Christian Rossow, and Thorsten Holz. Exit from hell? reducing the impact of {Amplification}{DDoS} attacks. In 23rd USENIX security symposium , pages 111–125, 2014

    work page 2014

  40. [40]

    The matter of heartbleed

    Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Pax- son, Michael Bailey, et al. The matter of heartbleed. In Proceedings of the 2014 conference on internet measurement conference , pages 475–488, 2014

    work page 2014

  41. [41]

    Best practices for notification studies for security and privacy issues on the internet

    Max Maass, Henning Prid ¨ohl, Dominik Herrmann, and Matthias Hollick. Best practices for notification studies for security and privacy issues on the internet. In Proceedings of the 16th International Conference on Availability, Reliability and Security , pages 1–10, 2021

    work page 2021

  42. [42]

    Let me out! evaluating the effectiveness of quarantining compromised users in walled gardens

    Orc ¸un C ¸ etin, Carlos Gan ´an, Lisette Altena, Samaneh Tajal- izadehkhoob, and Michel van Eeten. Let me out! evaluating the effectiveness of quarantining compromised users in walled gardens. In SOUPS, pages 251–263, 2018

    work page 2018

  43. [43]

    The role of web hosting providers in detecting compromised websites

    Davide Canali, Davide Balzarotti, and Aur ´elien Francillon. The role of web hosting providers in detecting compromised websites. In Proceedings of the 22nd international conference on World Wide Web, pages 177–188, 2013

    work page 2013

  44. [44]

    a how website owners face privacy issues: Thematic analysis of responses from a covert notification study reveals diverse circumstances and challenges

    Alina St ¨over, Nina Gerber, Henning Prid ¨ohl, Max Maass, Sebastian Bretthauer, I Spiecker, M Hollick, and D Herrmann. a how website owners face privacy issues: Thematic analysis of responses from a covert notification study reveals diverse circumstances and challenges. Proc Priv Enhanc Technol , 2023

    work page 2023

  45. [45]

    Feasibility of large-scale vulnerability notifications after gdpr

    Wissem Soussi, Maciej Korczynski, Sourena Maroofi, and Andrzej Duda. Feasibility of large-scale vulnerability notifications after gdpr. In 2020 IEEE European Symposium on Security and Privacy Work- shops (EuroS&PW), pages 532–537. IEEE, 2020

    work page 2020

  46. [46]

    Comparing large-scale privacy and security notifications

    Christine Utz, Matthias Michels, Martin Degeling, Ninja Marnau, and Ben Stock. Comparing large-scale privacy and security notifications. Proceedings on Privacy Enhancing Technologies , 2023

    work page 2023

  47. [47]

    {ZMap}: fast internet-wide scanning and its security applications

    Zakir Durumeric, Eric Wustrow, and J Alex Halderman. {ZMap}: fast internet-wide scanning and its security applications. In 22nd USENIX Security Symposium , pages 605–620, 2013

    work page 2013

  48. [48]

    Reflections on the engineering and operation of a large-scale embedded device vulnerability scanner

    Ang Cui and Salvatore J Stolfo. Reflections on the engineering and operation of a large-scale embedded device vulnerability scanner. In Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security , pages 8–18, 2011

    work page 2011

  49. [49]

    The dagstuhl beginners guide to re- producibility for experimental networking research

    Vaibhav Bajpai, Anna Brunstrom, Anja Feldmann, Wolfgang Kellerer, Aiko Pras, Henning Schulzrinne, Georgios Smaragdakis, Matthias W¨ahlisch, and Klaus Wehrle. The dagstuhl beginners guide to re- producibility for experimental networking research. ACM SIGCOMM Computer Communication Review , 49(1):24–30, 2019

    work page 2019

  50. [50]

    Extortion or expansion? an investigation into the costs and consequences of icann’s gtld exper- iments

    Shahrooz Pouryousef, Muhammad Daniyal Dar, Suleman Ahmad, Phillipa Gill, and Rishab Nithyanand. Extortion or expansion? an investigation into the costs and consequences of icann’s gtld exper- iments. In PAM 2020, March 30–31, 2020, Proceedings 21 , pages 141–157. Springer, 2020

    work page 2020

  51. [51]

    https://easylist .to/easylist/easylist.txt

    Easylist. https://easylist .to/easylist/easylist.txt

    work page

  52. [52]

    Misinformation domains

    Jana Lasser. Misinformation domains. https://github .com/JanaLasser/ misinformation domains/blob/main/data/clean/disinformation domains clean.csv, 2022

    work page 2022

  53. [53]

    Misinformation domains

    DeSmog. Misinformation domains. https://www .desmog.com/ climate-disinformation-database/, 2022

    work page 2022

  54. [54]

    The consumption of pink slime journalism: Who, what, when, where, and why? 2023

    Ryan Moore, Ross Dahlke, Priyanjana Bengani, and Jeffrey Hancock. The consumption of pink slime journalism: Who, what, when, where, and why? 2023

    work page 2023

  55. [55]

    Fakenews dataset

    Mark Allen and Chris. Fakenews dataset. https://github .com/ marktron/fakenews/blob/master/fakenews, 2022

    work page 2022

  56. [56]

    https://github .com/Lucetia/piracy/tree/master/docs

    Piracy websites. https://github .com/Lucetia/piracy/tree/master/docs

    work page

  57. [57]

    https:// sanctionssearch.ofac.treas.gov/Default.aspx

    Ofac of the us department of the treasury, sanctions list. https:// sanctionssearch.ofac.treas.gov/Default.aspx

    work page

  58. [58]

    Tranco: A research-oriented top sites ranking hardened against manipulation

    Victor Le Pochat, Tom Van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczy ´nski, and Wouter Joosen. Tranco: A research-oriented top sites ranking hardened against manipulation. Network and Dis- tributed Systems Security (NDSS) Symposium , 2019

    work page 2019

  59. [59]

    https://github .com/ duckduckgo/tracker-radar/tree/main/entities

    Github: duckduckgo tracker-radar/entities. https://github .com/ duckduckgo/tracker-radar/tree/main/entities. Accessed: 2022-05-17

    work page 2022

  60. [60]

    Apopha- nies or epiphanies? how crawlers impact our understanding of the web

    Syed Suleman Ahmad, Muhammad Daniyal Dar, Muhammad Fareed Zaffar, Narseo Vallina-Rodriguez, and Rishab Nithyanand. Apopha- nies or epiphanies? how crawlers impact our understanding of the web. In Proceedings of WWW 2020 , pages 271–280, 2020

    work page 2020

  61. [61]

    Beyond clicktivism: What makes digitally native activism effective? an exploration of the sleeping giants movement

    Yevgeniya Li, Jean-Gr ´egoire Bernard, and Markus Luczak-Roesch. Beyond clicktivism: What makes digitally native activism effective? an exploration of the sleeping giants movement. Social media+ society, 7(3):20563051211035357, 2021

    work page 2021

  62. [62]

    Using thematic analysis in psychology

    Virginia Braun and Victoria Clarke. Using thematic analysis in psychology. Qualitative research in psychology , 3(2):77–101, 2006

    work page 2006

  63. [63]

    Successful qualitative research: A practical guide for beginners

    Victoria Clarke and Virginia Braun. Successful qualitative research: A practical guide for beginners. Successful qualitative research, pages 1–400, 2013

    work page 2013

  64. [64]

    Difference-in-differences with multiple time periods

    Brantly Callaway and Pedro HC Sant’Anna. Difference-in-differences with multiple time periods. Journal of econometrics, 225(2):200–230, 2021

    work page 2021

  65. [65]

    Inference with difference-in- differences and other panel data

    Stephen G Donald and Kevin Lang. Inference with difference-in- differences and other panel data. The review of Economics and Statistics, 89(2):221–233, 2007

    work page 2007

  66. [66]

    Glowing in the dark: Uncovering {IPv6} address dis- covery and scanning strategies in the wild

    Hammas Bin Tanveer, Rachee Singh, Paul Pearce, and Rishab Nithyanand. Glowing in the dark: Uncovering {IPv6} address dis- covery and scanning strategies in the wild. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 6221–6237, 2023

    work page 2023

  67. [67]

    Experimental and quasi- experimental designs for research

    Donald T Campbell and Julian C Stanley. Experimental and quasi- experimental designs for research . Ravenio books, 2015

    work page 2015

  68. [68]

    Design of experiments: a realistic approach

    Virgil L Anderson and Robert A McLean. Design of experiments: a realistic approach. CRC Press, 2018

    work page 2018

  69. [69]

    Research design and issues of validity

    Marilynn B Brewer and William D Crano. Research design and issues of validity. Handbook of research methods in social and personality psychology, pages 3–16, 2000

    work page 2000

  70. [70]

    https://www .dhs.gov/sites/default/files/publications/ CSD-MenloPrinciplesCORE-20120803 1.pdf, 2012

    Menlo report. https://www .dhs.gov/sites/default/files/publications/ CSD-MenloPrinciplesCORE-20120803 1.pdf, 2012

    work page 2012

  71. [71]

    https://www .hhs.gov/ohrp/regulations-and-policy/ belmont-report/read-the-belmont-report/index .html, 1979

    Belmont report. https://www .hhs.gov/ohrp/regulations-and-policy/ belmont-report/read-the-belmont-report/index .html, 1979

    work page 1979

  72. [72]

    Auditing algorithms: Research methods for detecting discrimination on internet platforms

    Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: con- verting critical concerns into productive inquiry , 22(2014), 2014

    work page 2014

  73. [73]

    Selling off privacy at auction

    Lukasz Olejnik, Tran Minh-Dung, and Claude Castelluccia. Selling off privacy at auction. 2013

    work page 2013

  74. [74]

    Inferring tracker- advertiser relationships in the online advertising ecosystem using header bidding

    John Cook, Rishab Nithyanand, and Zubair Shafiq. Inferring tracker- advertiser relationships in the online advertising ecosystem using header bidding. arXiv preprint arXiv:1907.07275 , 2019

    work page arXiv 1907

  75. [75]

    The effects of naming and shaming on public support for compliance with international agree- ments: an experimental analysis of the paris agreement

    Dustin Tingley and Michael Tomz. The effects of naming and shaming on public support for compliance with international agree- ments: an experimental analysis of the paris agreement. International Organization, 76(2):445–468, 2022

    work page 2022

  76. [76]

    Companies inadvertently fund online misinformation despite consumer backlash

    Waqas Ahmad, Anupam Sen, Charles Eesley, et al. Companies inadvertently fund online misinformation despite consumer backlash. Nature, 630:123–131, 2024

    work page 2024

  77. [77]

    Small world with high risks: A study of security threats in the npm ecosystem

    Thomas Zimmermann, Premkumar Devanbu, and Christian Bird. Small world with high risks: A study of security threats in the npm ecosystem. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis , pages 9–20, 2019

    work page 2019

  78. [78]

    Open banking apis: Security risks, classification and countermeasures

    Abdel Rahman Abdou, Ashraf Matrawy, and Ioannis Lambadaris. Open banking apis: Security risks, classification and countermeasures. IEEE Access, 8:152273–152284, 2020. Appendix 15 0 2 4 6 8 10 12 Counts of responded entities Actionable details requested Forwarded to right department Collab. resolution performed Fixed the reported issues Contacted responsib...

    work page 2020