Paper Citation Record · LEDGER

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI

As of 22 August 2026, this Paper Citation Record lists 26 of 26 outbound references and 2 inbound Pith citation observations for arXiv:2501.15627.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2501.15627 v2

Coverage vector

measured 26 of 26 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-10T14:11:16.960772Z

measured 28 of 28 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-22T06:32:14.747728+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-15T20:18:43.057747Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: pith, observed 2026-08-05T12:24:02.722405Z

Reference resolution

26 of 26 outbound references displayed

verified exact0
verified fuzzy3
unresolved22
parse uncertain0
malformed identifier0
metadata mismatch1

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation e71574fb-b897-40ff-8d9f-a3458143e4f2 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 1

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.272660Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation e2d90d53-12a0-45b0-b1c6-1a4a5ee49ab4 · outbound

This paper cites B., Mann, B., Ryder, N., et al.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI B., Mann, B., Ryder, N., et al

Reference 2

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T14:11:17.262917Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation d99a925f-c492-4f43-8b98-82f9547d4415 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 3

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.251934Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 71d960eb-2ca7-49a1-874d-c956f056d2a5 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 4

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.238508Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 5072a14a-2ad9-4b90-b39d-977e5b9573a6 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 5

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.226327Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 39c04393-42f3-4f9b-9f54-84de34b4a8c6 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 6

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.215528Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 2823e9a8-2701-4776-8b1b-d0b5fffb7f50 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 7

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.205479Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 2fc2b982-75bf-42bd-90ae-647afd7d4b2d · outbound

This paper cites ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.895743Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1c9655e3-f3d6-4d10-9b1d-dd4b0b1244e7 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 9

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.195411Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation a1f0bde9-7a6e-460f-a158-51a2fe9c4775 · outbound

This paper cites I., & Mitchell, T.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI I., & Mitchell, T

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T14:11:17.184951Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 97099659-ae3e-483b-8fc3-141d589a4dda · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.905833Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 19b0b44a-e66a-4cf5-ab76-617a81c09dcd · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 12

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.167158Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation dd4ac28e-f27f-44cd-a2de-df6488633e1b · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 13

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.156506Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 35456de7-2428-4833-8abe-1b84011777e8 · outbound

This paper cites Direct speech-to-speech translation with a sequence-to-sequence model.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Direct speech-to-speech translation with a sequence-to-sequence model

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.915616Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation de65e791-2b36-486d-ba3d-ba130315b2eb · outbound

This paper cites FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.919552Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4879da6c-4d51-425d-bcca-ed2ea660295d · outbound

This paper cites MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.923011Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f0177ed7-f761-4ff3-b7e7-1eddb433ec6c · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 17

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.146946Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 69eddccf-a3ef-4517-a999-dfc9532d3cac · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 18

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.136544Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 02311cf2-784f-431e-a237-615d4c8fb339 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 19

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.124286Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 36318977-ea7c-4ee8-a667-b60999375aad · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 20

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.112994Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 5686f064-125e-41de-88c2-a0ffc791202b · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.100887Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 6924ce18-cc38-4f58-bb96-05578bb0eda7 · outbound

This paper cites an unresolved cited work.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Unresolved cited work

Reference 22

Resolution: unresolved; raw_fallback, observed 2026-08-10T14:11:17.090510Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 540962e0-6f97-456a-b433-f6464c81b3bc · outbound

This paper cites MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.950061Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b84227a2-2fc1-4ad0-859c-3f2bff1c6329 · outbound

This paper cites Learning to Reason with LLMs.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Learning to Reason with LLMs

Reference 24

Resolution: verified fuzzy; raw_fallback, observed 2026-08-10T14:11:17.080099Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 75f288d7-23ac-4a58-80da-1a4ca3ef2fae · outbound

This paper cites From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams

Reference 25

Resolution: metadata mismatch; local_arxiv, observed 2026-08-10T14:11:16.999605Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation eb125f94-3cf4-43b8-82cf-be3025d6a083 · outbound

This paper cites Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI.

HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI Understanding and Benchmarking Artificial Intelligence: OpenAI's o3 Is Not AGI

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-10T14:11:16.960772Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation ae28873d-c04b-4a9a-975d-5ca945b11b12 · inbound

Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard cites this paper.

Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI

Reference 60

Resolution: unresolved; no resolver link, observed 2026-08-15T20:18:43.057747Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4ad69974-7d04-4d41-9bf4-127080bd50a7 · inbound

Reinforcement Learning for Machine Learning Engineering Agents cites this paper.

Reinforcement Learning for Machine Learning Engineering Agents HardML: A Benchmark For Evaluating Data Science And Machine Learning knowledge and reasoning in AI

Reference 26

Resolution: verified exact; local_arxiv, observed 2026-08-05T12:24:02.725488Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.