Pith.

Computers and Society · arXiv:2611.16800 v1 · submitted 12 Nov 2026 · 33 references

A Code-Level Audit of X

Luka Bekavac, Simon Mayer

The paper says

“The transparency regime of recommender systems on Very Large Online Platforms (VLOPs) is undergoing significant change under the EU Digital Services Act (DSA).”

more of the abstract

The DSA requires platforms to disclose information about their recommendation pipelines and grants vetted researchers access to non-public data, which may extend to recommender code. However, it is not known to what extent such code releases enable substantive and meaningful third-party auditing of a production recommender system. We address this question by proposing a code-level audit methodology for recommender systems and applying it to the X platform, which has voluntarily open-sourced its recommender pipeline. Our methodology is a backwards-walk traversal of the repository, tracing each decision-relevant component from the user-facing timeline back to its upstream producers, checking that it is reachable from the served pipeline, and classifying each under a four-tier inspectability scheme. Applying this procedure, we identify 13 transparency dead-ends on the live serving path: models, services, label lists, and configuration values that the public code depends on but the release does not contain. We find that the components that decide which posts are filtered and how they are ranked fall into the two least inspectable tiers, a pattern that intensifies with the 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking. We further show that the release keeps 2023 code that the 2025 pipeline no longer reaches, with 36 parameter references that would pass for dead-ends without a reachability check. Finally, we discuss which additional artifacts would be required for meaningful auditing.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.

A well-executed code audit that delivers a clear methodology and a concrete dead-ends inventory; the central finding is real but the stress-test concern about conflating interface/implementation gaps is a fair nuance, not a flaw. This line is wrong

4 major objections · 10 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 33 references say.

The paper says, at §3.2, §6, Table 3

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.95):

“Our reachability check is static as well: it follows the pipelines that the release registers, which need not match what X deploys.”

Pith says: Objection 1 of 4 · major

The central empirical claim is the identification of 13 'transparency dead-ends on the live serving path' (Table 3). However, the methodology's reachability check (§3.2, step 3) determines which code is on the live serving path by following pipelines registered in the repository.

more of objection 1

As the authors acknowledge (§8, b215), this is a static analysis that may not match what X actually deploys. The paper does not provide independent verification that the registered pipelines correspond to production pipelines. This is load-bearing because the classification of components as dead-ends depends on the premise that the code paths traced are actually served. Without corroborating evidence (e.g., from network observations, documentation, or regulatory disclosures), the inventory conflates 'absent from the release' with 'absent from the production system'. The authors should either provide additional evidence that the registered pipelines match production, or reframe the dead-ends as 'dead-ends in the release relative to the registered pipelines' and discuss the implications of this gap more explicitly.

The paper says, at §6, Table 3, §C.1.2

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The dead-ends inventory (Table 3) lists 'Gold, Gray, and Legacy verification flags' as Class IV dead-ends because the upstream producer is not in the release. However, as noted in §C.1.2, the code shows these flags are copied from upstream service responses.

more of objection 2

The flags themselves are present and used in the code; what is missing is the producer of those flags. This is a different kind of gap from, say, the missing ranking model weights (Navi, Phoenix). The paper's classification scheme distinguishes Class III (interface present, implementation missing) from Class IV (referenced by name only), but the headline count of '13 transparency dead-ends' treats them uniformly. The authors should clarify whether the count conflates different types of gaps and whether the practical audit implications differ between a missing implementation (Class III) and a missing producer of a value that is otherwise present and used (Class IV for the flags). This does not undermine the overall finding but affects how the inventory should be interpreted.

The paper says, at §5.1.1, §C.4.3, §C.9.1

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.90):

“The release supports the claim that the label does not act on ranking, but whether the logged counts feed advertising, profiling, or the training of unreleased models cannot be established from the code.”

Pith says: Objection 3 of 4 · major

The discussion of the political inclination label (§5.1.1) is careful to note that the code shows it is 'used only for metrics tracking' and does not affect recommendations. However, the paper then raises concerns about GDPR and DSA implications based on the possibility that logged counts could feed advertising, profiling, or training of unreleased models.

more of objection 3

While the authors frame this as an open question requiring further artifacts, the presentation could be read as implying that the label is more consequential than the code evidence supports. The authors should more sharply distinguish between what the code shows (the label is not used for ranking) and what cannot be ruled out (downstream use of logs). The current framing is balanced but could be tightened to avoid giving the impression that the code itself reveals a privacy violation.

The paper says, at §8, b215, b216, b217, b218, b219

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.93):

“Our reachability check is static as well: it follows the pipelines that the release registers, which need not match what X deploys.”

Pith says: Objection 4 of 4 · major

The limitations section is admirably thorough, but several of the acknowledged limitations are so fundamental that they call into question the strength of the central claims. Specifically: (1) the reachability check may not match production (b215); (2) complete coverage is unrealistic (b216); (3) the repository is a snapshot that may not reflect current production (b217-b218); (4) a newer repository exists but was not analyzed (b219).

more of objection 4

These limitations collectively mean that the paper's findings are about a specific commit of a repository that may not correspond to what X currently runs. The authors should consider whether the conclusions need to be tempered to reflect that the audit is of the release, not of the production system. The current framing (e.g., '13 transparency dead-ends on the live serving path') implies a stronger connection to production than the limitations warrant. I recommend rewording to consistently refer to 'dead-ends in the released code relative to the registered pipelines' and adding a sentence in the conclusion that explicitly restates this scope.

10 minor objections

The paper says, at §1, abstract

Pith says: minor objection 1 of 10

The abstract states 'We further show that the release keeps 2023 code that the 2025 pipeline no longer reaches, with 36 parameter references that would pass for dead-ends without a reachability check.' This is a nice finding, but the number 36 appears only in the abstract and conclusion; it would be helpful to have a table or explicit list in §6.2 showing which parameters are orphaned.

The paper says, at §3.3, Table 2

Pith says: minor objection 2 of 10

The four-tier inspectability classification is a useful contribution, but the boundary between Class II and Class III could be clarified with additional examples. For instance, the 'Grok annotations' are listed as Class II/III in Table 2 because the logic is visible but the model is external. It would help to have a decision rule for when a component is Class II vs. Class III (e.g., is the presence of a default value the distinguishing factor?).

show 8 more

The paper says, at §C.4.1, Table 4

Pith says: minor objection 3 of 10

Table 4 lists the 13 Grok annotation features. It would be helpful to indicate which of these are used by the ranking model (as inputs) versus only by filters or logging. The text mentions that the political inclination label is metrics-only, but the table does not distinguish usage.

The paper says, at §5.1.2

Pith says: minor objection 4 of 10

The discussion of monetized amplification is informative, but the claim that 'the explicit 2023 boosts show that the paid badge was wired to reach' could be misinterpreted. The 2023 boosts were for Blue-verified authors, which at that time included both legacy verified and paid subscribers. The paper should clarify that the 2023 code does not distinguish between paid and legacy verification for the boost, so the boost applied to all verified accounts, not just paid ones.

The paper says, at §7, Table 3

Pith says: minor objection 5 of 10

The discussion of required artifacts (model cards, parameter dumps, reachability manifest) is valuable. However, the paper could more explicitly connect each artifact to the specific dead-end it would resolve. Table 3 already does this partially, but a cross-reference in §7 would strengthen the argument.

The paper says, at §2.4

Pith says: minor objection 6 of 10

The paper states that code audits are rare and cites only one study (Weber and Kosterich 2017). While this is accurate for recommender systems, there is a growing literature on code-level audits of AI systems more broadly (e.g., audits of facial recognition, hiring algorithms). The authors might consider citing a few examples from adjacent domains to contextualize the novelty of their approach.

The paper says, at §C.9.5

Pith says: minor objection 7 of 10

The appendix reference for orphaned pipelines is thorough, but the paper would benefit from a diagram or table summarizing which pipelines are live vs. orphaned in the 2025 release. Figure 1 shows the high-level architecture but does not indicate which components are orphaned.

The paper says, at §1, b254

Pith says: minor objection 8 of 10

The sentence labeled b254 ('Black-Box Access is Insufficient for Rigorous AI Audits') appears to be a citation title rather than a claim of the paper. The paper should ensure that all citations are properly formatted and that quoted titles are clearly attributed.

The paper says, at §8, b219

Pith says: minor objection 9 of 10

The paper notes that X published a separate repository in January 2026 that was not analyzed. Given that the paper is submitted in November 2026, this is a significant limitation. The authors should explain why the newer repository was not incorporated, even as a comparison point. The current justification (analysis began before the new release) is reasonable but could be expanded.

The paper says, at §C.6.5

Pith says: minor objection 10 of 10

The reference to 'screenshot engagement action definition' is interesting but the connection to the main argument is unclear. Consider moving to a footnote or removing if not essential.

What would settle it

For each of the 13 dead-ends in Table 3, determine whether the missing artifact is (i) an implementation that could be replaced without changing the interface (e.g., the evergreen table producer), or (ii) a parameter or model whose absence makes the component's behavior indeterminate (e.g., ranking weights). If the majority of dead-ends fall into category (i), then the headline claim that the release 'does not show what X's recommender does' is overstated, because the interface alone may suffice for many audit questions. Re-classify each dead-end as 'interface sufficient' or 'behavior indeterminate' and recompute the count of critical gaps.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees 7 s · 25,316 tokens read, 850 written
  3. Editor wrote the verdict above 8 s · 26,274 tokens read, 631 written
  4. Referee major revision: 4 major objections, 10 minor 16 s · 25,699 tokens read, 2,308 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This paper does something genuinely useful: it proposes a backwards-walk code audit methodology with a four-tier inspectability classification, applies it to X's 2025 open-source repository, and produces a concrete inventory of 13 transparency dead-ends. The methodology is clearly described, the code claims are pinned to a specific commit and archived in Software Heritage, and the authors are upfront about limitations. The finding that the components deciding what users see fall into the two least inspectable tiers (Class III Interface only, Class IV Opaque) is well-supported and important, especially given the Grok integration. The separation of live from orphaned code (36 parameter references in unreachable 2023 pipelines) is a nice methodological contribution that prevents false positives.

The soft spots are mostly acknowledged by the authors. The load-bearing assumption is that the registered pipelines match production (b215). The stress-test note raises a fair point: the dead-ends inventory treats 'absent from the release' as equivalent to 'absent from the production system,' but some dead-ends are architectural separations of concern (interface vs. implementation) that are standard in distributed systems. For example, the Gold/Gray/Legacy verification flags are listed as Class IV dead-ends because the upstream producer is not in the release, but the flags themselves are present and used. The paper's classification already distinguishes Class III from Class IV, but the headline count of 13 dead-ends lumps them together. This is a nuance, not a flaw—the authors are transparent about what each dead-end means—but it means the inventory should be read as a map of where provenance breaks down in the release, not necessarily as 13 equivalent transparency failures. The paper's argument does not depend on treating all 13 as equivalent; it depends on the pattern that the most consequential components (ranking models, Grok annotations, evergreen selection) are in the least inspectable tiers, and that pattern holds.

Who this is for: researchers and regulators working on DSA compliance, algorithm auditing methodology, and platform transparency. It deserves a serious referee. The methodology is generalizable, the empirical contribution is solid, and the limitations are honestly stated. I would accept peer review and would cite this if I were working on code-level auditing or DSA transparency.

Recommendation: Accept with minor revisions. The authors should add a sentence or two in Section 6 clarifying that the dead-ends inventory includes both missing implementations and missing interfaces, and that the audit significance varies by case. The stress-test concern is already partially addressed by the classification scheme, but making it explicit would strengthen the paper.

The strongest objection

The paper's central empirical contribution is the dead-ends inventory (Table 3), which lists 13 components that 'the public code depends on but the release does not contain.' The methodology identifies these by static reachability from registered pipelines. However, the paper cannot verify that these pipelines are the ones X actually deploys (b215). More critically, even if the pipelines match, the dead-ends are defined as dependencies that are absent from the release, not as dependencies that are absent from the production system. For example, the 'Gold, Gray, and Legacy verification flags' are listed as Class IV dead-ends because the upstream producer is not in the release, but the code shows these flags are copied from upstream service responses (C.1.2). The release does not contain the producer, but the flags themselves are present and used. The dead-ends inventory thus conflates two distinct gaps: (a) components whose implementation is missing but whose interface is present (Class III), and (b) components that are referenced by name only with no interface (Class IV). The paper's classification scheme already distinguishes these, but the headline count of '13 transparency dead-ends' treats them uniformly as gaps. The load-bearing question is whether the absence of the implementation (e.g., the producer of the evergreen table) is a transparency dead-end in the same sense as the absence of the ranking model weights. The paper's argument that the release 'does not show what X's recommender does' depends on treating all 13 as equivalent failures, but some are architectural separations of concerns (interface vs. implementation) that are standard in distributed systems, not necessarily transparency failures. The concrete test would be to check whether the missing artifacts are truly opaque or whether the interface alone suffices for meaningful auditing in each case.

The reader's weakest_assumption correctly identifies that the reachability check may not match production (b215). However, the paper explicitly acknowledges this limitation (Section 8). The more subtle concern is that the dead-ends inventory conflates missing implementations with missing parameters/models, and the paper's argument that the release is insufficient for auditing depends on treating all 13 as equivalent. The paper's own classification (Class II, III, IV) already distinguishes these, but the headline count and the discussion in Section 6 treat them uniformly. This is not a fatal flaw—the paper is transparent about what each dead-end means—but it is a load-bearing concern because the central claim (that the release has 13 transparency dead-ends on the live serving path) could be misinterpreted as 13 points where auditing is impossible, when in fact some are architectural separations that are standard in distributed systems. The concrete test would clarify the severity distribution. The verdict should remain UNCHANGED because the paper's methodology and findings are sound; the concern is about framing and interpretation, not correctness. The reader's verdict of ACCEPT is appropriate.

The referee's report

If the methodology and findings hold, this paper makes a significant contribution by providing a systematic, generalizable method for code-level auditing of recommender systems, a domain where such audits are extremely rare. The empirical application to X's repository is timely given the DSA's transparency requirements and the industry trend toward hybrid code/LLM recommender architectures. The dead-ends inventory provides concrete evidence of transparency gaps that can inform regulatory data access requests. The paper also contributes a conceptual framework (four-tier inspectability classification) that could be adopted by other auditors and regulators. The methodology is designed to be platform-agnostic and applicable under DSA data access provisions, increasing its potential impact.

Pith's reading of the paper (a machine's summary, not the authors')

The paper develops a systematic method to audit the source code of a large platform's recommender system, using X's voluntarily open-sourced repository as a test case. The method traces each decision from the user's timeline backwards through the code to identify where the public code depends on components that are not released: models, services, label lists, and configuration values. These are called transparency dead-ends. The audit finds 13 such dead-ends on the live serving path. The components that filter posts and rank them are classified as either Interface Only (the call is visible but the implementation is external) or Opaque (referenced by name only). The 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking moves more components into these low-inspectability tiers. The paper also finds that the repository contains 2023 code that the 2025 pipeline no longer reaches, with 36 parameter references that would appear to be dead-ends without a reachability check. The authors discuss what additional artifacts would be needed for meaningful auditing, such as model cards for LLM components, parameter dumps, and a reachability manifest.

Core claim. The central claim is that applying the proposed backwards-walk audit methodology to X's 2025 open-source repository identifies 13 transparency dead-ends on the live serving path: models, services, label lists, and configuration values that the public code depends on but the release does not contain. The components that decide which posts are filtered and how they are ranked fall into the two least inspectable tiers (Class III Interface only and Class IV Opaque), a pattern that intensifies with the 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking.

The premise it rests on. The weakest assumption is that the reachability check (step 3 of the methodology) correctly identifies which code is on the live serving path by following the pipelines that the release registers. The authors acknowledge this is a static analysis that may not match what X actually deploys (Section 8, b215). The methodology assumes that the registered pipelines in the repository correspond to the production serving path, but X could deploy different pipelines or configurations not reflected in the open-source commit. This is a load-bearing premise because the entire classification of components and identification of dead-ends depends on knowing which code actually runs.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

The transparency regime of recommender systems on Very Large Online Platforms (VLOPs) is undergoing significant change under the EU Digital Services Act (DSA). The DSA requires platforms to disclose information about their recommendation pipelines and grants vetted researchers access to non-public data, which may extend to recommender code. However, it is not known to what extent such code releases enable substantive and meaningful third-party auditing of a production recommender system. We address this question by proposing a code-level audit methodology for recommender systems and applying it to the X platform, which has voluntarily open-sourced its recommender pipeline. Our methodology is a backwards-walk traversal of the repository, tracing each decision-relevant component from the user-facing timeline back to its upstream producers, checking that it is reachable from the served pipeline, and classifying each under a four-tier inspectability scheme. Applying this procedure, we identify 13 transparency dead-ends on the live serving path: models, services, label lists, and configuration values that the public code depends on but the release does not contain. We find that the components that decide which posts are filtered and how they are ranked fall into the two least inspectable tiers, a pattern that intensifies with the 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking. We further show that the release keeps 2023 code that the 2025 pipeline no longer reaches, with 36 parameter references that would pass for dead-ends without a reachability check. Finally, we discuss which additional artifacts would be required for meaningful auditing.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 0.97

“We find that the components that decide which posts are filtered and how they are ranked fall into the two least inspectable tiers, a pattern that intensifies with the 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.67 · human participants 0.05 · controlled comparison 0.04 · randomized assignment 0.01 · states quantitative result 0.98 · compares with prior results 0.10 · mentions code or data availability 0.12

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
The transparency regime of recommender systems on Very Large Online Platforms (VLOPs) is undergoing significant change under the EU Digital Services Act (DSA). background 1.00 0.04 0.02 asserted 1.00
The DSA requires platforms to disclose information about their recommendation pipelines and grants vetted researchers access to non-public data, which may extend to recommender code. background 1.00 0.04 0.02 asserted 0.62
However, it is not known to what extent such code releases enable substantive and meaningful third-party auditing of a production recommender system. gap 1.00 0.05 0.04 asserted 0.77
We address this question by proposing a code-level audit methodology for recommender systems and applying it to the X platform, which has voluntarily open-sourced its recommender pipeline. contribution 1.00 0.87 0.05 asserted 0.95
Our methodology is a backwards-walk traversal of the repository, tracing each decision-relevant component from the user-facing timeline back to its upstream producers, checking that it is reachable from the served pipeline, and classifying each under a four-tier inspectability scheme. method 1.00 0.85 0.04 asserted 0.98
Applying this procedure, we identify 13 transparency dead-ends on the live serving path: models, services, label lists, and configuration values that the public code depends on but the release does not contain. result 1.00 0.97 0.34 asserted 1.00
We find that the components that decide which posts are filtered and how they are ranked fall into the two least inspectable tiers, a pattern that intensifies with the 2025 integration of the proprietary Grok language model into moderation, annotation, and ranking. result 0.97 0.96 0.46 asserted 1.00
We further show that the release keeps 2023 code that the 2025 pipeline no longer reaches, with 36 parameter references that would pass for dead-ends without a reachability check. result 1.00 0.96 0.09 asserted 1.00
Finally, we discuss which additional artifacts would be required for meaningful auditing. future work 0.93 0.73 0.05 unclear 0.80

What kind of paper the classifier reports this is: method or tool 0.82 · evidence from analysis of existing data 0.74

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 7,110 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (1)
Figure 1 of the paper
Figure 1. High-level architecture of X’s Home Timeline recommendation pipeline. Source:

Figures are the authors', reproduced from the paper.

References (33)
  1. [1] Jack Bandy. 2021. Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits.Proc. ACM Hum.-Comput. Interact.5, CSCW1, Article 74 (April 2021), 34 pages. doi:10.1145/3449148 on Pithdoi:10.1145/3449148
  2. [2] Luka Bekavac, Jannis Strecker-Bischoff, Kimberly Garcia, Simon Mayer, and Aurelia Tamò-Larrieux. 2026. Scrutinizing Systemic Risks in Personalized Recommender Systems Through Sock-Puppet Auditing of VLOPs.ACM Trans. Recomm. Syst.(Jan. 2026). doi:10.1145/3795516 Just Accepted on Pithdoi:10.1145/3795516
  3. [3] Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin Von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau,... on Pith
  4. [4] Giulio Corsi. 2024. Evaluating Twitter’s algorithmic amplification of low-credibility content: an observational study.EPJ Data Science13, 1 (2024), on Pith
  5. [5] Roberto Di Cosmo and Stefano Zacchiroli. 2023.The Software Heritage Open Science Ecosystem. Springer International Publishing, 33–61. doi:10.1007/978-3-031-36060-2_2 on Pithdoi:10.1007/978-3-031-36060-2_2
  6. [6] Alicia DeVrio and David Gray Widder. 2026. No More Plaudits for Algorithm Audits: A Provocation. InProceedings of the Ninth AAAI/ACM Conference on AI, Ethics, and Society (AIES ’26), Vol. 9. 818–830. doi:10.1609/aies.v9i2.43185 on Pithdoi:10.1609/aies.v9i2.43185
  7. [7] European Parliament and Council of the European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (Gener... on Pith
  8. [8] European Parliament and Council of the European Union. 2022. Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services (Digital Services Act). Official Journal of the European Union, L 277. 24 Bekavac and Mayer on Pith
  9. [9] Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. 2023. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models. InProceedings of the 61st Annual Meeting of the Association for Computationa... on Pithdoi:10.18653/v1/2023.acl-long.656
  10. [10] Maurizio Ferrari Dacrema, Simone Boglio, Paolo Cremonesi, and Dietmar Jannach. 2021. A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research.ACM Transactions on Information Systems39, 2 (Jan. 2021), 1–49. doi:10.1145/3434185 on Pithdoi:10.1145/3434185
  11. [11] Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. Bias and Fairness in Large Language Models: A Survey.Computational Linguistics50, 3 (2024), 1097–1179. doi:10.1162/coli_a_00524 on Pithdoi:10.1162/coli_a_00524
  12. [12] Balázs Hidasi and Ádám Tibor Czapp. 2023. The Effect of Third Party Implementations on Reproducibility. InProceedings of the 17th ACM Conference on Recommender Systems(Singapore, Singapore)(RecSys ’23). Association for Computing Machinery, New York, NY, USA, 272–282. doi:10.11... on Pithdoi:10.1145/3604915.3609487
  13. [13] Balázs Hidasi and Ádám Tibor Czapp. 2023. Widespread Flaws in Offline Evaluation of Recommender Systems. InProceedings of the 17th ACM Conference on Recommender Systems (RecSys ’23). ACM, 848–855. doi:10.1145/3604915.3608839 on Pithdoi:10.1145/3604915.3608839
  14. [14] Ferenc Huszár, Sofia Ira Ktena, Conor O’Brien, Luca Belli, Andrew Schlaikjer, and Moritz Hardt. 2022. Algorithmic amplification of politics on Twitter.Proceedings of the National Academy of Sciences119, 1 (2022), e2025334119. doi:10.1073/pnas.2025334119 on Pithdoi:10.1073/pnas.2025334119
  15. [15] Joshua A. Kroll, Joanna Huey, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu. 2017. Accountable Algorithms. University of Pennsylvania Law Review165, 3 (2017), 633–705 on Pith
  16. [16] Deepak Kumar, Yousef Anees AbuHashem, and Zakir Durumeric. 2024. Watch Your Language: Investigating Content Moderation with Large Language Models. InProceedings of the International AAAI Conference on Web and Social Media, Vol. 18. 865–878. doi:10.1609/icwsm.v18i1.31358 on Pithdoi:10.1609/icwsm.v18i1.31358
  17. [17] Paddy Leerssen. 2020. The Soap Box as a black box: Regulating transparency in social Media recommender Systems.SSRN Electronic Journal(Jan. 2020). doi:10.2139/ssrn.3544009 on Pithdoi:10.2139/ssrn.3544009
  18. [18] doi:10.1140/epjds/s13688-024-00456-3 on Pithdoi:10.1140/epjds/s13688-024-00456-3
  19. [19] Zachary C. Lipton. 2017. The Mythos of Model Interpretability. (2017). arXiv:1606.03490 [cs.LG] https://arxiv.org/abs/1606.03490 on PithPith's reviewarXiv
  20. [20] Anna-Katharina Meßmer and Martin Degeling. 2023. Auditing Recommender Systems – Putting the DSA into practice with a risk-scenario-based approach. arXiv:2302.04556 [cs.CY] https://arxiv.org/abs/2302.04556 on PithPith's reviewarXiv
show 13 more
  1. [21] Smitha Milli, Micah Carroll, Yike Wang, Sashrika Pandey, Sebastian Zhao, and Anca D. Dragan. 2025. Engagement, user satisfaction, and the amplification of divisive content on social media.PNAS Nexus4, 3 (2025), pgaf062. doi:10.1093/pnasnexus/pgaf062 on Pithdoi:10.1093/pnasnexus/pgaf062
  2. [22] Marçal Mora-Cantallops, Salvador Sánchez-Alonso, Elena García-Barriocanal, and Miguel-Angel Sicilia. 2021. Traceability for Trustworthy AI: A Review of Models and Tools.Big Data and Cognitive Computing5, 2 (2021), 20. doi:10.3390/bdcc5020020 on Pithdoi:10.3390/bdcc5020020
  3. [23] Matej Mosnar, Adam Skurla, Branislav Pecher, Matus Tibensky, Jan Jakubcik, Adrian Bindas, Peter Sakalik, and Ivan Srba. 2025. Revisiting Algorithmic Audits of TikTok: Poor Reproducibility and Short-term Validity of Findings. InProceedings of the 48th International ACM SIGIR Co... on Pithdoi:10.1145/3726302.3730293
  4. [24] Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019. Social data: biases, methodological pitfalls, and ethical boundaries. Frontiers in Big Data2 (July 2019), 13. doi:10.3389/fdata.2019.00013 on Pithdoi:10.3389/fdata.2019.00013
  5. [25] Cecilia Panigutti, Delia Fano Yela, Lorenzo Porcaro, Astrid Bertrand, and Josep Soler Garrido. 2025. How to investigate algorithmic-driven risks in online platforms and search engines? A narrative review through the lens of the EU Digital Services Act. InProceedings of the 202... on Pithdoi:10.1145/3715275.3732052
  6. [26] Pearl Pu, Li Chen, and Rong Hu. 2012. Evaluating recommender systems from the user’s perspective: survey of the state of the art.User Modeling and User-Adapted Interaction22, 4–5 (2012), 317–355. doi:10.1007/s11257-011-9115-7 on Pithdoi:10.1007/s11257-011-9115-7
  7. [27] Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing. InProceedin... on Pithdoi:10.1145/3351095.3372873
  8. [28] Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing Algorithms: Research Methods for Detecting Discrimi- nation on Internet Platforms. InData and Discrimination: Converting Critical Concerns into Productive Inquiry, Preconference of the 64... on Pith
  9. [29] Lukas Seiling, Iglesias Keller Clara, Jakob Ohme, De Vreese Claes, and Ulrike Klinger. 2025. Data Access for Researchers under the Digital Services Act. (Sept. 2025). doi:10.34669/wi.wpp/14 on Pithdoi:10.34669/wi.wpp/14
  10. [30] Jatinder Singh, Jennifer Cobbe, and Chris Norval. 2019. Decision Provenance: Harnessing Data Flow for Accountable Systems.IEEE Access7 (2019), 6562–6574. doi:10.1109/ACCESS.2018.2887201 on Pithdoi:10.1109/access.2018.2887201
  11. [31] Aleksandra Urman, Mykola Makhortykh, and Aniko Hannak. 2025. WEIRD Audits? Research Trends, Linguistic and Geographical Disparities in the Algorithm Audits of Online Platforms - A Systematic Literature Review. InProceedings of the 2025 ACM Conference on Fairness, Accountabilit... on Pithdoi:10.1145/3715275.3732026
  12. [32] Matthew Weber and Allie Kosterich. 2017. Coding the News: The role of computer code in filtering and distributing news.Digital Journalism6 (09 2017), 1–20. doi:10.1080/21670811.2017.1366865 on Pithdoi:10.1080/21670811.2017.1366865
  13. [33] Jinyi Ye, Luca Luceri, and Emilio Ferrara. 2025. Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’25). ACM, 2349–2... on Pithdoi:10.1145/3715275.3732159

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment