Paper Citation Record · LEDGER

Aligning Black-box Language Models with Human Judgments

As of 9 August 2026, this Paper Citation Record lists 44 of 44 outbound references and 0 inbound Pith citation observations for arXiv:2502.04997.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2502.04997 v1

Coverage vector

measured 44 of 44 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-08T20:45:46.434071Z

measured 44 of 44 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-09T06:31:02.800959+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

44 of 44 outbound references displayed

verified exact3
verified fuzzy2
unresolved39
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 4483f975-684c-4823-a6f1-580ead32d782 · outbound

This paper cites online" 'onlinestring :=.

Aligning Black-box Language Models with Human Judgments online" 'onlinestring :=

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.245898Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a1da4f4e-ef0a-4d90-8a36-63ef976f40ad · outbound

This paper cites write newline.

Aligning Black-box Language Models with Human Judgments write newline

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.251217Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 93990ef9-6206-4c1d-9299-5ab10a4df999 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 3

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.140190Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 0f59a00b-bb6a-4a42-b6cc-1335c515cae0 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 4

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.126706Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation d9f24b30-7e2e-45d4-8b78-2e170cd8f140 · outbound

This paper cites LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks.

Aligning Black-box Language Models with Human Judgments LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.264867Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2e6a1d5a-7e15-4314-908e-6217626a7550 · outbound

This paper cites Language Models are Few-Shot Learners.

Aligning Black-box Language Models with Human Judgments Language Models are Few-Shot Learners

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.269564Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d141ca34-f822-48da-8bea-8b7a98572119 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 7

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.113710Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation fbff8ad6-8164-4a0c-857f-3db836924bc3 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.278677Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 622ef66a-81aa-4255-bf52-020aafbd6f7a · outbound

This paper cites Training Verifiers to Solve Math Word Problems.

Aligning Black-box Language Models with Human Judgments Training Verifiers to Solve Math Word Problems

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.283385Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 90649c17-889e-47d0-b6c5-4760afbe896e · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.287833Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 254441ba-aa16-4be5-9dac-dd0bdcad13f0 · outbound

This paper cites The Llama 3 Herd of Models.

Aligning Black-box Language Models with Human Judgments The Llama 3 Herd of Models

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.292008Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5aa9b96f-c02a-4d9e-a43a-ab4988b7f0ff · outbound

This paper cites Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, and Henning Wachsmuth.

Aligning Black-box Language Models with Human Judgments Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Benno Stein, and Henning Wachsmuth

Reference 12

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T20:45:47.100523Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 86ab0998-45d3-4b91-a598-1b287848c2a9 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.300819Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cf4e09c5-8204-4563-add3-4cc4a5f5c0bb · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.304959Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 99d68814-44f6-464a-aeb0-a2aff91eade3 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 15

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.085821Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation f06b3004-d2b2-4e73-8dd0-1c7123850d1a · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.313150Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a71a05ad-45d4-480d-9d0d-c68b76c11208 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.317112Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 82745105-bcef-4e67-97a7-f12fc685acbb · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 18

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.062614Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation a338e2ff-0fb5-421f-8db3-b07b930aa97e · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 19

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.049538Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 82cff5a0-1a12-4155-b359-df1a249e99cb · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 20

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.036428Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation f4dff6d2-90ac-417d-aac5-4378cb9e6f98 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:47.022738Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation d8cd5b4c-db56-4737-9dd3-7c9460c3acd8 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.337470Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 52fe548e-7fff-4ad9-8c59-fea8efc72f2b · outbound

This paper cites Jackson and Samuel Messick.

Aligning Black-box Language Models with Human Judgments Jackson and Samuel Messick

Reference 23

Resolution: verified fuzzy; raw_fallback, observed 2026-08-08T20:45:47.008645Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation f3fca404-8e47-4d62-b12a-8392ce254678 · outbound

This paper cites Mixtral of Experts.

Aligning Black-box Language Models with Human Judgments Mixtral of Experts

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.345893Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a1919819-2f27-428a-8ac0-b4e7979c6ef1 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.350732Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3feaad36-890d-4b79-ba74-fe87121fbcc3 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.354777Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 388a265d-6e1e-4e32-98a7-d39cfb7941f4 · outbound

This paper cites Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment.

Aligning Black-box Language Models with Human Judgments Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.358882Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 35cb33a1-32a8-4b63-bb6c-cd9e976f61eb · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.364356Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 12311ef5-463a-4ee5-bac2-c3cc8ca55ef1 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 29

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:46.986303Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation cdd79145-d381-4534-806d-679522d6871b · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 30

Resolution: verified exact; doi, observed 2026-08-08T20:45:46.503759Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 05435747-f404-4008-a800-993c5aaf28ba · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 31

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:46.973000Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation eedd96ea-abe7-4b26-8ace-32fac4e5a6fd · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.381451Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 86c3da3e-3219-4f24-99df-5cbc9e41eda6 · outbound

This paper cites Schoonees, Michel Van de Velden, and Patrick J.

Aligning Black-box Language Models with Human Judgments Schoonees, Michel Van de Velden, and Patrick J

Reference 33

Resolution: verified exact; doi, observed 2026-08-08T20:45:46.480160Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 1c9658af-d635-4474-8a35-a7f2fb7d51a3 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.389924Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 10260a13-2b83-484f-a1f7-109376bdedc8 · outbound

This paper cites ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning.

Aligning Black-box Language Models with Human Judgments ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.394502Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 70b19d40-96e2-4285-b35c-c635bd64dbc0 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 36

Resolution: verified exact; raw_fallback, observed 2026-08-08T20:45:46.803135Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation e4d132d8-f832-42b3-a66b-1aa75b111845 · outbound

This paper cites Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models.

Aligning Black-box Language Models with Human Judgments Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.404348Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 096b69a6-1795-4ca5-990c-0896087c9502 · outbound

This paper cites Large Language Models are not Fair Evaluators.

Aligning Black-box Language Models with Human Judgments Large Language Models are not Fair Evaluators

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.408949Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ac8ee51a-41a3-4b1b-87e6-4d869c392b52 · outbound

This paper cites Style Over Substance: Evaluation Biases for Large Language Models.

Aligning Black-box Language Models with Human Judgments Style Over Substance: Evaluation Biases for Large Language Models

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.413279Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ad7cfba8-1b75-4f1e-ae68-23e7c00c00c7 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-08T20:45:46.417726Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bb051c28-8f83-4cc8-91e0-e0f03f417a23 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 41

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:46.951570Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 23dc06c6-6fbc-4fc7-b80f-0b99b109b789 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 42

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:46.938246Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 2566f1ff-3245-4fa0-a4d9-18fb10246fed · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 43

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:46.925107Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 9632e177-9f44-490e-ad70-27ea20562360 · outbound

This paper cites an unresolved cited work.

Aligning Black-box Language Models with Human Judgments Unresolved cited work

Reference 44

Resolution: unresolved; raw_fallback, observed 2026-08-08T20:45:46.911916Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Pith citing papers

No inbound Pith citation observations are available.