Paper Citation Record · LEDGER

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

As of 12 August 2026, this Paper Citation Record lists 36 of 36 outbound references and 11 inbound Pith citation observations for arXiv:2501.10711.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2501.10711 v5

Coverage vector

measured 36 of 36 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-10T19:04:26.024098Z

measured 47 of 47 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-12T06:34:41.77262+00:00

measured 11 of 11 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-06T23:25:46.325101Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-03T08:17:45.313006Z

Reference resolution

36 of 36 outbound references displayed

verified exact0
verified fuzzy13
unresolved21
parse uncertain0
malformed identifier0
metadata mismatch2

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 34896d21-bd5d-4b53-8da1-2aee4c39f741 · outbound

This paper cites Program Synthesis with Large Language Models.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Program Synthesis with Large Language Models

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.901736Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 307726f7-db38-4df1-aa70-44029ce64160 · outbound

This paper cites CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.915749Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d7fb5baa-a027-43a4-b79a-749c7f0ae816 · outbound

This paper cites On the Impacts of Contexts on Repository-Level Code Generation.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility On the Impacts of Contexts on Repository-Level Code Generation

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.919849Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8ca4f069-33a3-4768-8b75-1b2ff9c01810 · outbound

This paper cites Execution-based Evaluation for Data Science Code Generation Models.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Execution-based Evaluation for Data Science Code Generation Models

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.923680Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 392285b3-6eaf-4498-b896-a7b3e5ae6a71 · outbound

This paper cites RepoQA: Evaluating Long Context Code Understanding.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility RepoQA: Evaluating Long Context Code Understanding

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.946456Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation da9dd4fe-2709-43c9-adad-ac262a9c5c87 · outbound

This paper cites The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.950216Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b0985be0-94b7-4b92-a3e6-5fa8d74508c8 · outbound

This paper cites RunBugRun -- An Executable Dataset for Automated Program Repair.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility RunBugRun -- An Executable Dataset for Automated Program Repair

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.961358Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f4957f80-38f9-426e-86cb-37a07e3b7a1a · outbound

This paper cites BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.964826Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 841a835e-fbcf-427d-a7d2-e0249605e945 · outbound

This paper cites ACM Comput.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility ACM Comput

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.334240Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation e25778f0-993d-4db0-91f9-63d64667c6ec · outbound

This paper cites In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022

Reference 22

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.324022Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 5538b631-824e-4882-be67-1604f0ec4d8a · outbound

This paper cites AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.974940Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 641b435f-c8d6-4d26-aeaf-5bf8f6da3200 · outbound

This paper cites Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.978610Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 41fb8a0d-318b-45b7-95ab-53e4389c1f5e · outbound

This paper cites Benchmarking TPU, GPU, and CPU Platforms for Deep Learning.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Benchmarking TPU, GPU, and CPU Platforms for Deep Learning

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.986001Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6c0761f5-e342-418f-a39e-575223e02759 · outbound

This paper cites Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H

Reference 27

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.312967Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 8871a011-9d4f-4647-b182-379e83234441 · outbound

This paper cites Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.993387Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b5a330f3-866f-42e8-8471-0b3810afb14f · outbound

This paper cites CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.997038Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c618f911-c028-42e7-98f4-592bb192423a · outbound

This paper cites HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility HumanEvo: An Evolution-aware Benchmark for More Realistic Evaluation of Repository-level Code Generation

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:26.001569Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4bbc60dd-80a8-42d0-82c1-d7fec200b412 · outbound

This paper cites 0 1000 2000 3000 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 Citations Figure 10: Citation Distribution of Benchmarks Coding Task.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility 0 1000 2000 3000 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 Citations Figure 10: Citation Distribution of Benchmarks Coding Task

Reference 32

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.301471Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 7144b574-bcae-4541-bb62-9410db5821aa · outbound

This paper cites For example, Hu- manEval (Chen et al., 2021a) and MBPP (Austin et al., 2021)), class-level (i.e., a class with mul- tiple function units of code.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility For example, Hu- manEval (Chen et al., 2021a) and MBPP (Austin et al., 2021)), class-level (i.e., a class with mul- tiple function units of code

Reference 33

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.290186Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation e886fbfb-412f-468a-8bd7-cdf61fa6333e · outbound

This paper cites "" if len(dict.keys()) == 0: return False else: state =.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility "" if len(dict.keys()) == 0: return False else: state =

Reference 35

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.268643Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation e2308edd-b6d1-4238-a348-5085acc9db28 · outbound

This paper cites assert upper_ctr('PYthon') == 1.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility assert upper_ctr('PYthon') == 1

Reference 36

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.258180Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation a0345340-8e1f-40de-b16c-c684c63cdb84 · outbound

This paper cites CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis

Reference 294

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.953770Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b87d9451-8b50-4d17-aa66-36f687992189 · outbound

This paper cites RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale

Reference 516

Resolution: metadata mismatch; local_arxiv, observed 2026-08-10T19:04:26.196889Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 6e9b65b5-40ce-4d6e-bbf6-3c801369c065 · outbound

This paper cites The Impact of Reasoning Step Length on Large Language Models.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility The Impact of Reasoning Step Length on Large Language Models

Reference 1442

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.931274Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 24b41b94-ab75-4ba6-a038-b2b2c6862b19 · outbound

This paper cites Gorilla: Large Language Model Connected with Massive APIs.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Gorilla: Large Language Model Connected with Massive APIs

Reference 1569

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.957389Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9220f3cd-78d5-4726-b8e8-7a63997dcb24 · outbound

This paper cites VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation

Reference 2014

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.982237Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8db7c4e9-ea1e-4336-94e0-7ecb703eb50f · outbound

This paper cites Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

Reference 2017

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:26.005585Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7457eae2-0314-4877-87a8-8e5fa287585b · outbound

This paper cites Dianshu Liao, Shidong Pan, Xiaoyu Sun, Xiaoxue Ren, Qing Huang, Zhenchang Xing, Huan Jin, and Qinying Li.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Dianshu Liao, Shidong Pan, Xiaoyu Sun, Xiaoxue Ren, Qing Huang, Zhenchang Xing, Huan Jin, and Qinying Li

Reference 2018

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.345346Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation c77e4a68-9820-48cf-837b-9db14ee61242 · outbound

This paper cites an unresolved cited work.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Unresolved cited work

Reference 2019

Resolution: unresolved; raw_fallback, observed 2026-08-10T19:04:26.384535Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation bc2a4932-c16a-47b3-844d-063958e4e60f · outbound

This paper cites In Proceedings of the 28th International Confer- ence on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020 , pages 26–38.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility In Proceedings of the 28th International Confer- ence on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020 , pages 26–38

Reference 2020

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.394682Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 5439fe26-5714-40ce-9e05-b0c5bb2310f4 · outbound

This paper cites measure the ability of these models to synthesize short Python programs from natural language descriptions.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility measure the ability of these models to synthesize short Python programs from natural language descriptions

Reference 2021

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.279136Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 9e39f514-2992-4811-9d48-abf6cf58bbcd · outbound

This paper cites SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities

Reference 2022

Resolution: metadata mismatch; local_arxiv, observed 2026-08-10T19:04:26.181938Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation cc6295b4-01b7-4629-b9f5-3c910c840f3c · outbound

This paper cites In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023, pages 1430–.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023, pages 1430–

Reference 2023

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.356005Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation b154828c-19f1-4b79-84bf-0bb7265d9243 · outbound

This paper cites Long Code Arena: a Set of Benchmarks for Long-Context Code Models.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Long Code Arena: a Set of Benchmarks for Long-Context Code Models

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-10T19:04:25.910216Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 43d11c34-c450-4efa-a791-c2a118d34726 · outbound

This paper cites Lakshya A.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Lakshya A

Reference 5445

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.375676Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 35c6a7bb-d39d-49b3-a7d7-83b04d35f4e5 · outbound

This paper cites Ramakrishna Bairi, Atharv Sonwane, Aditya Kanade, Vageesh D.

Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility Ramakrishna Bairi, Atharv Sonwane, Aditya Kanade, Vageesh D

Reference 8474

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T19:04:26.366160Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Pith citing papers

Observation beede259-c224-4d6f-b773-850b7dbae6e6 · inbound

Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation cites this paper.

Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 23

Resolution: verified exact; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 3973a4fa-89d9-446d-a73e-f298d673980b · inbound

A Conceptual Framework for AI Capability Evaluations cites this paper.

A Conceptual Framework for AI Capability Evaluations Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-06T23:25:46.325101Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 58917dc5-a8db-4654-8d3a-2a65a00bfc8e · inbound

Establishing Best Practices for Building Rigorous Agentic Benchmarks cites this paper.

Establishing Best Practices for Building Rigorous Agentic Benchmarks Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-06T20:24:20.949000Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6a12dde6-efe5-4dfd-a678-8026e8bc1c33 · inbound

Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models cites this paper.

Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 2

Resolution: metadata mismatch; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation fda34f98-829b-403e-be0e-948799f4c56c · inbound

Guidelines for Empirical Studies in Software Engineering involving Large Language Models cites this paper.

Guidelines for Empirical Studies in Software Engineering involving Large Language Models Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 19

Resolution: verified exact; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 65779e12-7cd7-49cc-bd71-1f333fd5523c · inbound

Guidelines for Empirical Studies in Software Engineering involving Large Language Models cites this paper.

Guidelines for Empirical Studies in Software Engineering involving Large Language Models Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 19

Resolution: verified exact; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 6631b1be-bbee-457c-9245-9f7bb63248dd · inbound

Knowledge-Graph-Driven Data Synthesis for Low-Resource Software Development: A HarmonyOS Case Study cites this paper.

Knowledge-Graph-Driven Data Synthesis for Low-Resource Software Development: A HarmonyOS Case Study Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 5

Resolution: metadata mismatch; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 0061e5a0-8657-4136-9935-52d0364e35f4 · inbound

Edit, But Verify: An Empirical Audit of Instructed Code-Editing Benchmarks cites this paper.

Edit, But Verify: An Empirical Audit of Instructed Code-Editing Benchmarks Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 4

Resolution: metadata mismatch; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 8513986c-d294-41b6-a051-13b72123b60d · inbound

Flaws in the LLM Automation Narrative cites this paper.

Flaws in the LLM Automation Narrative Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 24

Resolution: verified exact; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation cc18cf3f-27f6-45f0-a7e8-ac73c421df8c · inbound

AlgoBench: Benchmarking Algorithmic Adaptation in Code Generation cites this paper.

AlgoBench: Benchmarking Algorithmic Adaptation in Code Generation Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 4

Resolution: metadata mismatch; arxiv_id, observed 2026-07-07T03:18:46.014814Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation eb8b0150-4c21-4608-81bf-219d2a213365 · inbound

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing cites this paper.

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-03T01:30:13.354497Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.