Paper Citation Record · LEDGER

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets

As of 15 August 2026, this Paper Citation Record lists 30 of 30 outbound references and 3 inbound Pith citation observations for arXiv:1908.07898.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
1908.07898 v2

Coverage vector

measured 30 of 30 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-14T11:58:39.838934Z

measured 33 of 33 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-14T06:32:32.682623+00:00

measured 3 of 3 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-14T05:07:56.650660Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-15T12:40:00.372749Z

Reference resolution

30 of 30 outbound references displayed

verified exact0
verified fuzzy5
unresolved25
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 6d104b89-504e-4d32-9adc-ce3e45b2bfee · outbound

This paper cites URL: " 'urlintro :=.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets URL: " 'urlintro :=

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.223300Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0c16e7fb-cb2f-4c4b-8de4-cdd2c023c31a · outbound

This paper cites write newline.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets write newline

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.265154Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b4a1420f-ec87-454a-b3b8-d6f83b28dfe3 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 3

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:41.080364Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation e6b05cec-44a6-49df-874b-f0d196235b49 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 4

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.994053Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation db0c3402-dae8-4fd4-9157-1f566056a2fa · outbound

This paper cites Bowman, G.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Bowman, G

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.282819Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 043bb611-d933-497e-8370-6dbb32f1c17b · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 6

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.948475Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 96100969-729a-406f-b3fe-3b9e19f91030 · outbound

This paper cites BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.291912Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 341cb92c-ea9b-4aa9-ba96-09274e580387 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 8

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.933973Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 0b6b63c9-a1de-42c4-9cde-06d8b1b7a1cf · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.301451Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 57f58460-ebb3-408a-9c7e-06622afe05d3 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 10

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.909387Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation cc388608-c23e-4b0b-b2ed-419ac34b493f · outbound

This paper cites Annotation Artifacts in Natural Language Inference Data.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Annotation Artifacts in Natural Language Inference Data

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.372952Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1cf3c79f-0ea2-46ad-9f75-4597530c5acd · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 12

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.804277Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 4345fe9a-644a-4b3f-9773-be0a53fbb534 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 13

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.765066Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 00c20e69-1f44-4b2d-ac62-6baf2b0093d6 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.389135Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 68458838-1ada-4ac4-bc68-a6e7a4c6c48f · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 15

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.739925Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 980ad2e9-b649-4206-9c16-d0f560d98428 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.400126Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8e40f513-0b19-49b2-be31-214cfbd3c3a1 · outbound

This paper cites Hypothesis Only Baselines in Natural Language Inference.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Hypothesis Only Baselines in Natural Language Inference

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.436117Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f056bcc5-1662-40b9-84c1-e5e6dc066592 · outbound

This paper cites Rajpurkar, J.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Rajpurkar, J

Reference 18

Resolution: verified fuzzy; raw_fallback, observed 2026-08-14T11:58:40.589092Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation e1cae967-837b-4eb8-aa3e-4421bac2f4bf · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.663956Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 949e4dfb-f978-43bd-9831-5478f8ea895e · outbound

This paper cites Richardson, C.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Richardson, C

Reference 20

Resolution: verified fuzzy; raw_fallback, observed 2026-08-14T11:58:40.562633Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 6d0514c5-982b-48ed-b40d-cf7ed15fe128 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.418487Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 9e082994-bd51-428b-869c-163d84d52823 · outbound

This paper cites Schwartz, M.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Schwartz, M

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.678236Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8ad3088b-67b7-4680-a081-e35e6e8757a1 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 23

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.353618Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation fdb51d34-139f-4e98-a4a8-5e80879cec5d · outbound

This paper cites A Corpus for Reasoning About Natural Language Grounded in Photographs.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets A Corpus for Reasoning About Natural Language Grounded in Photographs

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.738160Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation eea18006-bf87-42a7-a35a-99503ac0e7e9 · outbound

This paper cites Talmor, J.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Talmor, J

Reference 25

Resolution: verified fuzzy; raw_fallback, observed 2026-08-14T11:58:40.293511Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 0bfcf4aa-9f0d-4290-a633-361c222f613f · outbound

This paper cites Trischler, T.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Trischler, T

Reference 26

Resolution: verified fuzzy; raw_fallback, observed 2026-08-14T11:58:40.243698Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation ea544555-6673-45be-af9f-17c722c290b6 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 27

Resolution: unresolved; raw_fallback, observed 2026-08-14T11:58:40.077569Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 1be7d6a7-0869-466b-90f0-b7878f99b53e · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.809743Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a2587623-1a7c-489b-9323-578de30b5d2e · outbound

This paper cites Zaidan and Chris Callison-Burch.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Zaidan and Chris Callison-Burch

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-14T11:58:40.051424Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

Observation 2a6f416f-86da-4f40-b0ee-175a35954a09 · outbound

This paper cites an unresolved cited work.

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Unresolved cited work

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-14T11:58:39.838934Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 9d60a1ba-a982-4825-9936-ee64dccd14d8 · inbound

An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction cites this paper.

An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-14T05:07:56.650660Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3d97bd7f-daf8-4e7e-8dd3-0d97340ee51e · inbound

From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set cites this paper.

From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-12T14:27:57.352788Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7fee0b59-ef37-4b88-97fb-c69429fc8d58 · inbound

From Pre-trained Models to Large Language Models: A Comprehensive Survey of AI-Driven Psychological Computing cites this paper.

From Pre-trained Models to Large Language Models: A Comprehensive Survey of AI-Driven Psychological Computing Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets

Reference 9

Resolution: verified exact; arxiv_id, observed 2026-05-15T12:40:00.374963Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.