Typed states for the displayed outbound observations.
Source: paper_references, paper_reference_links, observed 2026-08-15T20:41:09.087338Z
Paper Citation Record · LEDGER
As of 19 August 2026, this Paper Citation Record lists 61 of 61 outbound references and 0 inbound Pith citation observations for arXiv:2505.12331.
A citation records a reference. It does not transfer a finding from one paper to another.
Typed states for the displayed outbound observations.
Source: paper_references, paper_reference_links, observed 2026-08-15T20:41:09.087338Z
One-hop event checks from named stored sources.
Source: scholarly_work_events, retraction_status_cache, observed 2026-08-18T06:34:40.430872+00:00
Pith citing papers itemized under the disclosed page cap.
Source: paper_references, paper_reference_links
A source-named dated measurement, never combined with another source.
Source: cited_works
61 of 61 outbound references displayed
External citation measurements
No source-named external measurement is stored.
Observation 421c77fd-dd77-4d41-8c94-8255de3985fa · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Github copilot.https://copilot.github.com, 2021
Reference 1
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 9420a2ab-6a59-4c0a-ad9c-199c6e3e0c0d · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Cursor: The ai-powered code editor.https://cursor.so, 2023
Reference 2
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 37a4dd52-72cb-4904-9535-2acb97562cc9 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Unresolved cited work
Reference 3
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation ffd97942-0d28-4e3f-8692-fa5f96fc972c · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Codelmsec benchmark: Systematically evaluating and finding security vulnerabilities in black-box code language models
Reference 4
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation f4362cce-2d41-45cc-9720-d8e55ab24d87 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs DynaCode: A Dynamic Complexity-Aware Code Benchmark for Evaluating Large Language Models in Code Generation
Reference 5
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 92931cb0-3c53-4536-bab2-27d63c5c177c · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Humanevo: An evolution-aware benchmark for more realistic evaluation of repository-level code generation
Reference 6
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation be0cf7b6-15bc-4966-9283-e7b95304764b · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Codeif-bench: Evaluating instruction-following capabilities of large language models in interactive code generation.arXiv preprint arXiv:2503.22688, 2025
Reference 7
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 19ebd076-1d70-427d-912f-77193fece6a3 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
Reference 8
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 0fcf29d9-48f7-44a6-bf52-a8bd0d4fd83b · outbound
OSS-Bench: Benchmark Generator for Coding LLMs SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Reference 9
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation be14a1d2-d442-401c-a987-89953bee9903 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica
Reference 10
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 1f926510-652a-45e2-81e3-6297ad342f41 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Unresolved cited work
Reference 11
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 5aabb163-a1e4-4c76-8171-de64910dd20d · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving
Reference 12
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 4f899914-f0ea-4ffe-be68-ef5008893aec · outbound
OSS-Bench: Benchmark Generator for Coding LLMs SecRepoBench: Benchmarking LLMs for secure code generation in real-world repositories, 2025
Reference 13
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 175ab0dc-7966-411f-b6c0-7ddeb40f37a1 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs CWEval: Outcome-driven evaluation on functionality and security of LLM code generation, 2025
Reference 14
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation d267b0db-c1a3-4ef8-b916-cecb179222a6 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Reference 15
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 8097f06e-e716-4baa-a9f5-f247dd6ddd16 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Codearena: Inspecting and improving code quality metrics using minecraft
Reference 16
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 838d148b-e976-4a6a-8eae-01547b8c5a40 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs CodeElo: Benchmarking competition-level code generation of LLMs with human-comparable elo ratings, 2025
Reference 17
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation f7e97651-5964-4935-a92f-b6670019008f · outbound
OSS-Bench: Benchmark Generator for Coding LLMs ComplexCodeEval: A benchmark for evaluating large code models on more complex code
Reference 18
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation d54edff5-10a6-4393-a1e0-24f00abd278f · outbound
OSS-Bench: Benchmark Generator for Coding LLMs PythonSaga: Redefining the benchmark for code generating LLMs, 2024
Reference 19
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation b56dff38-2269-4311-9d09-2d1988dba53b · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmark
Reference 20
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 5d8a4b47-36f1-4ff6-8f67-cc13055f3cd5 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan
Reference 21
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation f99279ee-a255-4536-9367-6778d381a976 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs ClassEval: A manually-crafted benchmark for evaluating LLMs on class-level code generation, 2023
Reference 22
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 2cbeb47d-f80b-4c4f-86fb-534d76cc5f87 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs HumanEval-XL: A multilingual code generation benchmark for cross-lingual natural language generalization, 2024
Reference 23
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 996c0b14-7950-4375-9405-0055fdad5d9e · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x
Reference 24
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 25f43123-634e-4ac1-998e-991732e8da59 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs {AddressSanitizer}: A fast address sanity checker
Reference 25
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 74e819d9-d3f8-47ba-9843-c946e1a8f32b · outbound
OSS-Bench: Benchmark Generator for Coding LLMs php-src: The php interpreter.https://github.com/php/php-src, 2025
Reference 26
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation b09990d6-ebf5-434e-9d9b-0c5d93593900 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Richard Hipp
Reference 27
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 6e377465-2058-4c79-ac59-b801173648b5 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs https://testing.googleblog.com/2020/08/ code-coverage-best-practices.html, 2020
Reference 28
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 88dc70a5-32bc-440f-abeb-a04a7fac21bc · outbound
OSS-Bench: Benchmark Generator for Coding LLMs libclang: C interface to the clang library
Reference 29
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation d1450324-a4f6-45df-a943-c1b8d6b4b472 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Unresolved cited work
Reference 30
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation 1978e6ba-1cb1-4ecc-bc52-25c65abf6ff3 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs difflib — helpers for computing deltas between objects
Reference 31
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation cd2d49f0-5dc8-4fae-98e3-42c38be67156 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs gpt-o1 model.https://platform.openai.com/docs/models/o1, 2025
Reference 32
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation ab3f1842-84bd-444f-b18d-d960ecb00322 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs o3-mini model.https://platform.openai.com/docs/models/o3-mini, 2025
Reference 33
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation c5d584d7-b454-482d-bd96-1006e253fffb · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Claude 3.7 sonnet.https://www.anthropic.com/claude/sonnet, 2025
Reference 34
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 5ac39b2b-67e2-42b1-a827-d6712bfb99da · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Claude 3.5 haiku.https://www.anthropic.com/claude/haiku, 2025
Reference 35
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation ecb39fce-e3dc-4376-b46c-2dc6b95967f7 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Gemini 2.5 flash.https://developers.generativelanguage.google/, 2025
Reference 36
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 614a4c67-f17b-462e-afc7-18d9722d4bb9 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Llama 3.3 70b instruct (fp16)
Reference 37
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 3e19ca2f-c7eb-4839-9edb-39d90a5268ed · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Codellama 70b instruct (fp16)
Reference 38
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation d97379d9-4aec-40a6-af08-2b98a80f12bc · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Qwen 2.5 coder 32b instruct (fp16)
Reference 39
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 16c0acef-3449-42ba-8439-7306efa9a235 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Qwen 3.0 30b-a3b fp16.https://ollama.com/library/qwen3:30b-a3b-fp16, 2025
Reference 40
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 36fdb130-8cbe-47e1-ad88-2d20377819a9 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Qwen 3 8b fp16.https://ollama.com/library/qwen3:8b-fp16, 2025
Reference 41
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation ed415a7d-841a-4574-9f59-df739406e666 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Gemma 3 27b-it fp16.https://ollama.com/library/gemma3:27b-it-fp16, 2025
Reference 42
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation f14f87fd-c5ed-4fa4-9171-ec76af6b46ed · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Qwen 2.5 coder 14b instruct (fp16)
Reference 43
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 2e09b457-c428-4822-8ef3-01b67b7c49c2 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Deepseek coder v2 16b lite instruct (fp16)
Reference 44
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 2651a8ef-fb93-47ae-a2dc-546876a5fc10 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Starcoder2-15b-instruct-v0.1 (fp16)
Reference 45
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 065a418e-8f61-4bad-9e28-8c86c4b37fce · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Phi-4 14b fp16.https://ollama.com/library/phi4:14b-fp16, 2025
Reference 46
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation e8a8fc19-035d-41bc-9fb8-27aa8ccfdf02 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Mistral 7b instruct (fp16)
Reference 47
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 56c62f9f-203a-46fc-bb32-8424f361b066 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Codegemma 7b instruct (fp16)
Reference 48
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 51c00e5f-a141-4348-b0c7-1541b8d6090b · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Openai: Advances in safe and beneficial ai.https://openai.com, 2025
Reference 49
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 1074e2ff-727f-496b-800a-30dcd2e6b1b3 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Anthropic: Building reliable, steerable ai systems
Reference 50
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation faa5be50-d86b-462e-97bc-7f4e4e873a70 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Google: Organizing the world’s information.https://www.google.com, 2025
Reference 51
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation b029a42d-0535-4919-b5d9-939351530fc5 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Deepseek: Developer of high-performance open-source llms
Reference 52
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation c12238fa-e33f-4420-ac49-bcf5605bd33e · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Alibaba group: Global trade and technology.https://www.alibaba.com, 2025
Reference 53
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation c9f9aa06-2519-4215-acfd-e96bd8cb4b54 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Meta: Bringing the metaverse and social technology together
Reference 54
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation cded8149-9ca2-4f3e-80e8-5a24d16e8f85 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Microsoft: Empowering every person and organization
Reference 55
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 1ab6e7d1-79b9-4dc6-904f-a9092340bc36 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Bigcode: Open and responsible development of code llms
Reference 56
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 2c5f0684-a7fa-4f47-ab91-988aacd0d958 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Mistral ai: Frontier ai in your hands.https://mistral.ai, 2025
Reference 57
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 77e66e68-6bcb-4ffd-a89b-8398e909a334 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Ollama: Get up and running with large language models locally
Reference 58
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
Observation 6376ec4b-63b3-4a79-bb6d-4e07616eb100 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs SPoC: Search-based Pseudocode to Code
Reference 59
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation c79ca41e-c891-4d6a-b74f-17d8593de675 · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Evaluating Large Language Models Trained on Code
Reference 60
Source-reported events for the cited work
Unavailable: canonical work link unavailable.
Observation ca45235d-09f5-492e-9102-3bbbf33dedfd · outbound
OSS-Bench: Benchmark Generator for Coding LLMs Fuzzing the PHP Interpreter via Dataflow Fusion
Reference 61
Source-reported events for the cited work
No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.
No inbound Pith citation observations are available.