Pith. sign in

REVIEW 2 major objections 5 minor 33 references

The reported Power of Noise in RAG is not a general benefit of random documents; it appears, weakens, or vanishes with small changes to prompts and decoding limits.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 07:03 UTC pith:SNA674O7

load-bearing objection Clean reproduction that shows the Power-of-Noise effect is mostly an artifact of short decoding limits, rigid extraction prompts, and older model interfaces—not a general RAG property. the 2 major comments →

arxiv 2607.03615 v2 pith:SNA674O7 submitted 2026-07-03 cs.IR

The Powerless Noise: How Experimental Settings Shape the Reported Power of Noise

Micha{\l} Mazuryk , Fleur Dolmans , Louis Gehringer , Ina Klaric , Jia-Huei Ju , Mohammad Aliannejadi This is my paper
classification cs.IR
keywords Retrieval-Augmented GenerationNoise in RAGPrompt CompositionRetrieval StrategyInference configurationDecoding limitsExact-match evaluationInstruction templates
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper re-runs the main experiments behind the claim that adding random, irrelevant documents can raise question-answering accuracy in retrieval-augmented generation. Under the original setup—earlier models, a rigid extraction prompt with a NO-RES option, and a 15-token generation cap—the authors recover the same pattern: random documents can help when the gold passage sits near the query. They then systematically change model family, numerical precision, chat-style instruction templates, output length limits, and task wording. Once those knobs move toward ordinary modern practice, baseline accuracy rises and the noise advantage shrinks or disappears; error analysis attributes much of the original swing to truncated answers, placeholder text, and forced refusals. The practical message is that inference design can manufacture an apparent retrieval phenomenon, so RAG claims about noise need to be stress-tested under relaxed, realistic generation settings.

Core claim

The Power-of-Noise pattern is reproducible under the original constrained inference regime, but it is highly sensitive to prompt formulation and decoding limits; combined with large shares of truncation and malformed generations among errors, the original accuracy gains cannot be confirmed as a general benefit of noisy retrieval under the conditions tested.

What carries the argument

Controlled Random–Near ablations (gold document nearest the query, random passages added) that step from the original Base setup through chat templates, a 100-token generation limit, and revised task instructions (FinalP01/FinalP02), paired with a catalogue of output-error patterns (truncation, NO-RES, underscores, newlines) that explain accuracy swings.

Load-bearing premise

That scoring an answer as correct only when a fixed gold string appears remains a fair accuracy measure after longer, freer generations—so residual noise trends still reflect model use of context rather than leftover exact-match brittleness.

What would settle it

On the same Natural Questions open split, re-evaluate FinalP01/FinalP02 runs with a semantic or multi-reference judge instead of exact string match; if a clear accuracy rise with added random documents reappears under that judge for modern instruction-tuned models, the artifact claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Claims that random corpus noise improves RAG accuracy should be re-checked under chat templates and longer decoding before being treated as design advice.
  • Restrictive extraction prompts and short generation caps can systematically understate performance when only the gold document is present.
  • Exact-match evaluation on truncated or placeholder outputs can invent a noise benefit that vanishes once those failures are reduced.
  • Model architecture and instruction tuning change whether any noise-plus pattern appears at all.
  • RAG benchmarks need explicit reporting of prompt templates and max-token settings as first-class experimental factors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Many published RAG robustness results that fix a short max_new_tokens budget may be partly measuring refusal and truncation rates rather than retrieval quality.
  • A useful follow-up is to hold the FinalP01-style prompt fixed and sweep only generation length and stop sequences to map the exact boundary where the noise-plus curve collapses.
  • Retriever design may matter less for headline accuracy than inference hygiene when evaluation is exact-match and models are instruction-tuned.
  • Error-pattern dashboards (truncation, forced abstention, format garbage) could become standard diagnostics alongside accuracy in noise-in-RAG studies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper reproduces Cuconasu et al.'s Power-of-Noise finding for RAG and tests whether the reported benefit of random documents is robust. Under the original setup (4-bit Llama2/MPT, raw extraction prompt, 15-token limit), the authors recover the original pattern: random documents improve accuracy in the Near placement (Tables 1–3). They then ablate precision, modern instruction-tuned models (Llama3, Mistral, Falcon3, Qwen2.5), chat templates, generation length (15→100), and task-instruction wording (prompt01/prompt02). Across these interventions the noise benefit appears, weakens, or vanishes; error analysis (Figure 2, Appendix B) attributes much of the original gap to truncation, NO-RES refusals, and malformed placeholders. The authors conclude that the effect is an artifact of inference configuration rather than a general property of noisy retrieval, and release code.

Significance. If the result holds, it is a useful corrective for the RAG literature: a high-visibility claim that random noise can help is shown to be highly sensitive to prompt formulation and decoding limits. The contribution is primarily empirical and methodological—clean factorial ablations, multi-model coverage, explicit error-pattern rates, and public code—rather than a new theory of attention or retrieval. That is appropriate for a reproduction/robustness study and is of clear practical value to anyone designing or evaluating RAG systems. Strengths include successful numerical reproduction of the original experiments, controlled isolation of inference factors, and transparent diagnosis of failure modes that exact-match scoring would otherwise hide.

major comments (2)
  1. §3.4.4 / §3.5.4 and Tables 5–6: the paper retains exact-match-on-answer-string evaluation for comparability, but under Max (100 tokens) and chat templates many correct paraphrases or longer extractions will still be scored wrong. The authors acknowledge this limitation, and FinalP01 already shows flat or slightly negative noise trends across five models, so residual under-counting is unlikely to reverse the headline claim. Still, a short soft-match or LLM-as-judge sanity check on a subset of FinalP01/FinalP02 outputs would make the evaluation-artifact concern fully load-bearing-free rather than secondary.
  2. §4.2–4.3 and Figure 2: Llama3 under Base produces ~73.6% placeholder/underscore outputs at 0-DOC, driving the anomalous 14.62% accuracy that then rises with random documents. The paper correctly treats this as an artifact, but the main text could more sharply separate “true noise benefit” from “artifact-driven accuracy recovery” when summarizing Llama3, so readers do not over-weight that single Base curve as evidence of architecture-dependent Power-of-Noise.
minor comments (5)
  1. §3.4.4: the Δ(%) formula is written as (reproduced−original)/reproduced; confirm this is intentional (vs. the more common /original) and state it once in the caption of Table 1 so readers do not misread the signed percentages.
  2. Table 2 / Table 4: yellow highlighting for “strong Power-of-Noise” is useful but not defined numerically in the caption; a one-line rule (e.g., accuracy rising with # random docs relative to 0-DOC) would help.
  3. Figure 2: error-pattern rates are conditioned on incorrect answers; a companion panel or appendix table with absolute rates over all generations would make the contribution of truncation/NO-RES to overall accuracy clearer.
  4. §3.5.2: prompt01 vs prompt02 differ in both constraint removal and fallback wording; a brief note that the two changes are confounded would prevent over-interpreting FinalP02’s lower absolute accuracy as pure phrasing sensitivity.
  5. Typos / polish: “NON-RES” vs “NO-RES” appears inconsistently in §5; “Wwho” in Appendix B.1 is a query typo from the dataset but could be marked as such; a few long sentences in the Discussion could be tightened.

Circularity Check

0 steps flagged

No significant circularity: purely empirical reproduction and ablations with independent interventions.

full rationale

This paper is an empirical reproduction and robustness study of Cuconasu et al.'s Power-of-Noise claim. It does not derive a theoretical quantity, fit a free parameter that is later re-presented as a prediction, or import a uniqueness theorem from overlapping authors. The original Power-of-Noise numbers are treated as external benchmarks that the authors re-measure under the original setup (Tables 1–3); their extensions (precision, modern LLMs, chat templates, max tokens, revised prompts FinalP01/FinalP02) are independent experimental interventions whose outcomes are reported as measured accuracy and error-pattern rates (Tables 4–6, Figure 2, Appendix Table 7). Exact-match evaluation is retained for comparability, not redefined in terms of the noise effect. Self-citations are limited to standard methodological references and do not load-bear the central claim. No step reduces by construction to its inputs; circularity score is therefore 0.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

Empirical reproduction paper. Almost all load-bearing content is experimental design choices and standard IR/LLM evaluation assumptions rather than free parameters or invented physical entities. The ledger therefore lists the few hand-chosen experimental knobs and the domain assumptions required for the accuracy numbers to be interpretable.

free parameters (2)
  • max_new_tokens=100
    Chosen by the authors as a ‘relaxed’ generation budget; not derived from data or theory, yet used to define the Max and Final configurations that eliminate the noise effect.
  • prompt01 / prompt02 wording
    Two hand-crafted instruction variants that remove the five-token and NO-RES constraints; their exact phrasing is free and affects absolute accuracy levels.
axioms (3)
  • domain assumption Exact-match presence of any gold answer string constitutes a valid binary accuracy measure for open-domain QA.
    Stated and retained in Sections 3.4.4 and 3.5.4 for comparability; the paper itself notes paraphrase and partial-span failures.
  • domain assumption NQ-open queries plus the 20 Dec 2018 English Wikipedia dump (100-word passages) form a representative testbed for RAG noise effects.
    Inherited from Cuconasu et al. and used for all experiments (Section 3.1).
  • domain assumption Greedy decoding (temperature 0) and the listed context windows are sufficient to expose configuration sensitivity.
    Used throughout the extended experiments (Section 3.5.1).

pith-pipeline@v1.1.0-grok45 · 22907 in / 2384 out tokens · 66141 ms · 2026-07-13T07:03:34.354468+00:00 · methodology

0 comments
read the original abstract

Recent work has suggested that adding irrelevant documents to the input of retrieval-augmented generation (RAG) systems can improve question-answering performance, a phenomenon referred to as the Power of Noise. This motivated investigations into the role of noise in information retrieval. In this paper, we reproduce the main findings of Cuconasu et al. and evaluate the robustness of the effect under extended experimental settings. We first confirm that the phenomenon holds under the original setup, which uses earlier-generation LLMs, restrictive prompting and constrained decoding settings. We subsequently introduce a series of extensions to investigate the underlying causes of the noise effect, examining the authors' original design choices including the use of different models, instruction prompting, and relaxed output length constraints. Across these ablations, the Power-of-Noise pattern proves highly sensitive to inference configuration: it can appear, weaken, or disappear under small changes to prompt formulation and decoding limits. Combined with our error analysis, which shows substantial contributions from truncation and malformed generations, this variance indicates that the original effect cannot be robustly confirmed as a general benefit of noisy retrieval under these experimental conditions. More broadly, our work highlights the importance of carefully scrutinizing inference design in retrieval-augmented generation systems. Our code is available at https://github.com/ina0105/The-Power-of-Noise-Reproduction.

Figures

Figures reproduced from arXiv: 2607.03615 by Fleur Dolmans, Ina Klaric, Jia-Huei Ju, Louis Gehringer, Micha{\l} Mazuryk, Mohammad Aliannejadi.

Figure 1
Figure 1. Figure 1: Prompt structure listed for the original prompt. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Error-pattern rates (% incorrect answers that con [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 12 linked inside Pith

  1. [1]

    Varshney, and Reza Zafarani

    Reyhaneh Abdolazimi, Shengmin Jin, Pramod K. Varshney, and Reza Zafarani

  2. [2]

    arXiv:2410.06348 [cs.LG]

    Harnessing the Power of Noise: A Survey of Techniques and Applications. arXiv:2410.06348 [cs.LG]

    Pith/arXiv arXiv

  3. [3]

    Chen Amiraz, Florin Cuconasu, Simone Filice, and Zohar Karnin. 2025. The Distracting Effect: Understanding Irrelevant Passages in RAG. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 18228–18258. doi:10. 18653/v1/2025.acl-long.892

    2025

  4. [4]

    Shuyang Cao, Karthik Radhakrishnan, David Rosenberg, Steven Lu, Pengxiang Cheng, Lu Wang, and Shiyue Zhang. 2025. Evaluating the Retrieval Robustness of Large Language Models. arXiv:2505.21870 [cs.CL]

    Pith/arXiv arXiv 2025

  5. [5]

    Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. 2025. Un- leashing the potential of prompt engineering for large language models.Patterns 6, 6 (June 2025), 101260. doi:10.1016/j.patter.2025.101260

    doi:10.1016/j.patter.2025.101260 2025

  6. [6]

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. Benchmarking large language models in retrieval-augmented generation. InProceedings of the Thirty- Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelli...

    doi:10.1609/aaai.v38i16.29728 2024

  7. [7]

    Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The Power of Noise: Redefining Retrieval for RAG Systems. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2024). ACM, 719–729. d...

    doi:10.1145/3626772.3657834 2024

  8. [8]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. LLM.int8(): 8-bit matrix multiplication for transformers at scale. InProceedings of the 36th International Conference on Neural Information Processing Systems(New Orleans, LA, USA)(NIPS ’22). Curran Associates Inc., Red Hook, NY, USA, Article 2198, 15 pages

    2022

  9. [9]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI]

    Pith/arXiv arXiv 2024

  10. [10]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. arXiv:2210.17323 [cs.LG]

    Pith/arXiv arXiv 2023

  11. [11]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Infor- mation Retrieval with Contrastive Learning. arXiv:2112.09118 [cs.IR]

    Pith/arXiv arXiv 2022

  12. [12]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. arXiv:2310.068...

    Pith/arXiv arXiv 2023

  13. [13]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural Questions: A Benchmark for Question Answering Research.Tr...

    doi:10.1162/tacl_a_00276 2019

  14. [14]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. InProceedings of the 34th International Conference on Neural Information Processing Systems (NI...

    2020

  15. [15]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts.Transactions of the Association for Computational Linguistics 12 (2024), 157–173. doi:10.1162/tacl_a_00638

    doi:10.1162/tacl_a_00638 2024

  16. [16]

    2023.Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs

    MosaicML NLP Team. 2023.Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs. https://www.databricks.com/blog/mpt-7b Accessed: 2003-05-05

    2023

  17. [17]

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2025. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. arXiv:2402.07927 [cs.AI]

    Pith/arXiv arXiv 2025

  18. [18]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. arXiv:2310.11324 [cs.CL]

    Pith/arXiv arXiv 2024

  19. [19]

    Jiuding Sun, Chantal Shaib, and Byron C Wallace. 2023. Evaluating the zero-shot robustness of instruction-tuned language models.arXiv preprint arXiv:2306.11270 (2023)

    Pith/arXiv arXiv 2023

  20. [20]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony...

    Pith/arXiv arXiv 2023

  21. [21]

    Falcon LLM TII UAE. 2024. The Falcon 3 Family of Open Models. https:// huggingface.co/blog/falcon3

    2024

  22. [22]

    Cunxiang Wang, Sirui Cheng, Qipeng Guo, Yuanhao Yue, Bowen Ding, Zhikun Xu, Yidong Wang, Xiangkun Hu, Zheng Zhang, and Yue Zhang. 2023. Evaluating open-QA evaluation. InProceedings of the 37th International Conference on Neu- ral Information Processing Systems(New Orleans, LA, USA)(NIPS ’23). Curran Associates Inc., Red Hook, NY, USA, Article 3367, 30 pages

    2023

  23. [23]

    Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, and Sercan O Arik. 2025. Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vienna, Austria, 3...

    doi:10.18653/v1/ 2025

  24. [24]

    Zhepei Wei, Wei-Lin Chen, and Yu Meng. 2025. InstructRAG: In- structing Retrieval-Augmented Generation via Self-Synthesized Rationales. arXiv:2406.13629 [cs.CL]

    Pith/arXiv arXiv 2025

  25. [25]

    Jinyang Wu, Shuai Zhang, Feihu Che, Mingkuan Feng, Pengpeng Shao, and Jianhua Tao. 2025. Pandora’s Box or Aladdin’s Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguis...

    2025

  26. [26]

    doi:10.18653/v1/2025.acl-long.250

    doi:10.18653/v1/2025.acl-long.250 2025

  27. [27]

    Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. 2025. Con- trollable chain-of-thought compression in llms.arXiv preprint arXiv:2502.12067 (2025)

    arXiv 2025

  28. [28]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han

  29. [29]

    InProceedings of the 40th International Conference on Machine Learning(Honolulu, Hawaii, USA)(ICML’23)

    SmoothQuant: accurate and efficient post-training quantization for large language models. InProceedings of the 40th International Conference on Machine Learning(Honolulu, Hawaii, USA)(ICML’23). JMLR.org, Article 1585, 13 pages

  30. [30]

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang Ren, Xinlong Yang, Yong Li, Zhiying Xu, and Zipeng Zhang. 2025. Qwen2.5-1M Techni...

    Pith/arXiv arXiv 2025

  31. [31]

    Shuangfei Zhai, Tatiana Likhomanenko, Etai Littwin, Dan Busbridge, Jason Rama- puram, Yizhe Zhang, Jiatao Gu, and Josh Susskind. 2023. Stabilizing transformer training by preventing attention entropy collapse. InProceedings of the 40th In- ternational Conference on Machine Learning(Honolulu, Hawaii, USA)(ICML’23). JMLR.org, Article 1709, 34 pages

    2023

  32. [32]

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Guoyin Wang, and Fei Wu. 2026. Instruction Tuning for Large Language Models: A Survey.ACM Comput. Surv.58, 7, Article 169 (Jan. 2026), 36 pages. doi:10.1145/3777411

    doi:10.1145/3777411 2026

  33. [33]

    NO-RES"Pattern Response starts with or equals

    Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Gong, and Xing Xie. 2024. PromptRo- bust: Towards Evaluating the Robustness of Large Language Models on Ad- versarial Prompts. InProceedings of the 1st ACM Workshop on Large AI Sys- tems and Models with Privacy and Safety Analysis(Salt Lake City...

    doi:10.1145/3689217.3690621 2024