Pith. sign in

Paper Citation Record · LEDGER

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs

As of 18 August 2026, this Paper Citation Record lists 54 of 54 outbound references and 0 inbound Pith citation observations for arXiv:2507.15715.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2507.15715 v2

Coverage vector

measured 54 of 54 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-06T15:29:42.348441Z

measured 54 of 54 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-18T06:34:40.430872+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

54 of 54 outbound references displayed

  • verified exact4
  • verified fuzzy9
  • unresolved40
  • parse uncertain0
  • malformed identifier1
  • metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 27c96f69-cd38-4040-a144-6b5fa93f0f7c · outbound

This paper cites Kurtz, Edwin A.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Kurtz, Edwin A

Reference 1

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.780248Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.327954Z digest=sha256:42a528091a0e684c077a8c64773519d69cf5ae07f7832031aee974255cc9ee74

Observation e857067f-c455-44c1-8017-c896aefbafe8 · outbound

This paper cites Henneken, Carolyn S.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Henneken, Carolyn S

Reference 2

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.590827Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.353537Z digest=sha256:b7fcc100a975f022577ac5cf0275609cc7c915676cc3ecafc4c6daa0986db401

Observation 16e9845e-ce20-4707-ab47-19d22826bbe0 · outbound

This paper cites OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.422914Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.422914Z digest=sha256:93cd95538e0acd9641a4e9c7f671e26cc167fbe112ad829dd5f63d11192c2b86

Observation 7b3b0184-0ec3-48f0-bbe2-5efbd72a176d · outbound

This paper cites Search B ots: User engagement with ChatBots during collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Search B ots: User engagement with ChatBots during collaborative search

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.501898Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.501898Z digest=sha256:8524367a7a6394efed3830bde261997138a6a9b254305155c56352a7ba12c376

Observation 9b256564-1ae3-4dc9-ac4e-53ef4e34de2a · outbound

This paper cites Embedding search into a conversational platform to support collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Embedding search into a conversational platform to support collaborative search

Reference 5

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.620210Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.620210Z digest=sha256:2d751a472c1c21331643960649317c272264a3504b8440fa6cf7f1449ff75fc4

Observation 857cfe90-fb18-485b-8def-5517ff44a169 · outbound

This paper cites The effects of system initiative during conversational collaborative search.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The effects of system initiative during conversational collaborative search

Reference 6

Resolution
verified exact
doi, observed 2026-08-06T15:29:43.102800Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:37.707455Z digest=sha256:329db29d23ca98401be57f262eb17458e948ceede1500a1d366171d39f0c6829

Observation f7e1caf3-44f0-4c07-86a0-d19a3b528224 · outbound

This paper cites Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.798046Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.798046Z digest=sha256:cffc05ce1f100db0c31453ae400f325c01f33a00263b6bd110337c5351598185

Observation 874cf6a4-dc96-4f38-9d90-68df6f0ff5e0 · outbound

This paper cites Bowman and George Dahl.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Bowman and George Dahl

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.857577Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.857577Z digest=sha256:00e0aef4a55dc2821e19d9cd40defdd9ab0a6879fea4f7e6df7640145cc0b2e5

Observation 49e8f034-f8d8-4ee5-8e89-cc9f404679d3 · outbound

This paper cites Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services

Reference 9

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:37.975077Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:37.975077Z digest=sha256:0918c8a85fc653e97d116a12134be1275dabc2ad1f6c2b8bd51daab8aa18eb77

Observation 74b9716d-7221-49e5-aa41-abd3c7533141 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 10

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:46.374142Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:38.086420Z digest=sha256:42e8d2d949d18d40e2556fa904c9f7d254dffe1ee82e6c90b7598bc6d3e1d3f9

Observation 043b933d-e42e-4251-ad1e-b29e00ad287c · outbound

This paper cites Fabbri, Wojciech Kry \'s ci \'n ski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Fabbri, Wojciech Kry \'s ci \'n ski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.206098Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.206098Z digest=sha256:a2331c8536cb9c51b9566263f6755ce0c46b74bd65ca6e6d5fcab83e18859b73

Observation 6570ef50-b6b5-4676-b30c-279824ba7822 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 12

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.333778Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.333778Z digest=sha256:b8e821d13e21be723e9c1a3718c47aaa1cd0501a603c5a04df9a232ddfcc6c10

Observation 266e21d3-c56b-44e6-9fb6-0e17a90ecfb5 · outbound

This paper cites Enabling large language models to generate text with citations.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Enabling large language models to generate text with citations

Reference 13

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.435601Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.435601Z digest=sha256:b5da7bef7f6612b9b4a5d5c335a21562f6b6e1e9b45e3eb6798dae095b217b95

Observation 456fe5da-355b-4c50-8d98-531913e588bb · outbound

This paper cites What can large language models do in chemistry? A comprehensive benchmark on eight tasks.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs What can large language models do in chemistry? A comprehensive benchmark on eight tasks

Reference 14

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:46.087564Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:38.538154Z digest=sha256:5fc1f93cd6ba2cc482373bc7f6b267204ed96d3c116a85dae9d9d444cd46f84a

Observation 29d230c2-2941-40ff-8f67-5c12f3144f92 · outbound

This paper cites Measuring mathematical problem solving with the math dataset.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Measuring mathematical problem solving with the math dataset

Reference 15

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.632350Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.632350Z digest=sha256:dea5668d5a4aa8d3a7ac1eeb3e9e52c0474387c05665e0576756b0e9a80ef9aa

Observation bf012bce-e408-49c0-86dd-64eb54026ca1 · outbound

This paper cites Iyer, Mikaeel Yunus, Charles O’Neill, Christine Ye, Alina Hyk, Kiera McCormick, Ioana Ciucă, John F.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Iyer, Mikaeel Yunus, Charles O’Neill, Christine Ye, Alina Hyk, Kiera McCormick, Ioana Ciucă, John F

Reference 16

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.737079Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.737079Z digest=sha256:50d4fca1828a3b9184215bc10f230e5c0920017a5333ec9f453c6f62ab0b9415

Observation 13f31780-8ae0-4420-8746-a8aeabd65f56 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 17

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.791678Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.791678Z digest=sha256:597ba61212fd7a5b70869e2a018bbe51b05bb2e5183cb56653caf07de7455b27

Observation 7eb8b2d2-134b-47ff-aa79-48e18b31b379 · outbound

This paper cites On scientific understanding with artificial intelligence.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs On scientific understanding with artificial intelligence

Reference 18

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:38.893061Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:38.893061Z digest=sha256:3167b9d296d05332c44291758b4aba69b0b02dbe4b554cca19f8124922a2fe54

Observation f701a313-32ec-485f-a132-4e78c62403b5 · outbound

This paper cites Wikibench: Community-driven data curation for AI evaluation on W ikipedia.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Wikibench: Community-driven data curation for AI evaluation on W ikipedia

Reference 19

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.005780Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.005780Z digest=sha256:47c350d1c476af534c78d8f71feaf6a91c5a34e10e22d3b8dcce15b10fdc484f

Observation 690c2d37-96c5-4c25-842a-3e23044ae119 · outbound

This paper cites Chapter 8 - interviews and focus groups.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Chapter 8 - interviews and focus groups

Reference 20

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.128709Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.128709Z digest=sha256:01b3be0cc02f4d2d9d4e7964d2817d1d693d0877ae32cbc727593f392cf47ba5

Observation 0218d1d6-2b04-4562-bc11-17f6776dba69 · outbound

This paper cites H alu E val: A large-scale hallucination evaluation benchmark for large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs H alu E val: A large-scale hallucination evaluation benchmark for large language models

Reference 21

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.222337Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.222337Z digest=sha256:503f61393eeab946fde020c6f4723c8aca1fe7eb5fce91faab666155f7b03309

Observation 04ef06a9-e284-49af-b016-443301616f0d · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 22

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:45.873366Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:39.305544Z digest=sha256:f89bd15bac789b8c80a4ae2032cbb682cdf56e8c3f6d5937999fb56cf95785df

Observation 64e7b96d-dc3b-424a-bdb2-ee5f5deb5e4b · outbound

This paper cites Rethinking Model Evaluation as Narrowing the Socio-Technical Gap.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Rethinking Model Evaluation as Narrowing the Socio-Technical Gap

Reference 23

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.408794Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.408794Z digest=sha256:1ec855e8c06196f895f6cf875c4c0bb0e934305a7b4175a63763c67e72660c31

Observation a92f3370-7547-4c92-97ad-5a0957c5af95 · outbound

This paper cites Vera Liao, Daniel Gruen, and Sarah Miller.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Vera Liao, Daniel Gruen, and Sarah Miller

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.516481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.516481Z digest=sha256:b923586d7c0f48e96983d6b8c57a8b0d4716fab0d5756a298e7a0cca188a2465

Observation a8b993fe-9673-47a8-aa2b-6b9d3d39ce79 · outbound

This paper cites S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models

Reference 25

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.620818Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.620818Z digest=sha256:a19335ae2e2c2fdbcd5d7f89b84d1bd69a7c36ca767d1459d0c89ae92534a385

Observation f96673d3-1578-4deb-aa67-acf7f07d0710 · outbound

This paper cites Mathewson, Jaylen Pittman, and Richard Evans.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Mathewson, Jaylen Pittman, and Richard Evans

Reference 26

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.788439Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.788439Z digest=sha256:00f7a2d3ca2afebecd6d2e8f33c4eaaeebb4c14d055d6dd5213e12e7088c485e

Observation 23dff6a3-e837-4a45-ab51-6548962615c2 · outbound

This paper cites Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, and Casey Dugan

Reference 27

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:39.917055Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:39.917055Z digest=sha256:cf1aaff7b3da489e711a91e4294574b9eea7f80e611db0ef5c2edb403bf7135f

Observation 0dc0bdb0-4283-4bf8-a8fe-4d6e40d4b2e3 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 28

Resolution
unresolved
raw_fallback, observed 2026-08-06T15:29:45.676716Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.039345Z digest=sha256:39746ecaf8a60f0cad5a66b9fdd0737d5cefff38847b9c09849cdd489c72245b

Observation a529e512-12a9-4686-aae0-4702e446ffb3 · outbound

This paper cites Rodriguez Mendez, Thang Bui, Alyssa Goodman, Alberto Accomazzi, Jill Naiman, Jesse Cranney, Kevin Schawinski, and Roberta Raileanu.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Rodriguez Mendez, Thang Bui, Alyssa Goodman, Alberto Accomazzi, Jill Naiman, Jesse Cranney, Kevin Schawinski, and Roberta Raileanu

Reference 29

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.182493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.182493Z digest=sha256:e2694c837fb1560456a09efb37c458435b4880ac00a314454d1e7e8b7e4901e3

Observation 2ed222ae-9815-4590-82e8-c0b06816356b · outbound

This paper cites emr QA : A large corpus for question answering on electronic medical records.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs emr QA : A large corpus for question answering on electronic medical records

Reference 30

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.252325Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.252325Z digest=sha256:b70fdfdbcd2ce8a3449dce405e11f5aaef85674b56db3e9edc1b22997b741e0f

Observation 340bbaff-2ae5-4607-9b65-9f4e20bfe3f5 · outbound

This paper cites B io R ead: A new dataset for biomedical reading comprehension.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs B io R ead: A new dataset for biomedical reading comprehension

Reference 31

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.468781Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.375448Z digest=sha256:60aadc3c8227affe4eda2ebe6412d2653ddde510bc7b91c16c856747b5671869

Observation e74bfa0e-789f-4503-9ab5-483edc04a799 · outbound

This paper cites B io MRC : A dataset for biomedical machine reading comprehension.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs B io MRC : A dataset for biomedical machine reading comprehension

Reference 32

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.489331Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.489331Z digest=sha256:58b7a63a51b7fa8174889b24a35f36899fc55eb875e51bfa96e684e3766d7b3c

Observation c7ff9d22-4084-4f89-bdd1-f30853b47950 · outbound

This paper cites Exploring temperature effects on large language models across various clinical tasks.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Exploring temperature effects on large language models across various clinical tasks

Reference 33

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.268357Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:40.584441Z digest=sha256:9b00c76fabd6fee4fd9f8dd6de396ba6721acabf11fba037f552eebb48ea82a2

Observation 3314d704-20c7-4a91-8284-519ee72dda91 · outbound

This paper cites Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă, and UniverseTBD.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Smith, Huiling Liu, Kevin Schawinski, Kartheik Iyer, Ioana Ciucă, and UniverseTBD

Reference 34

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.688976Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.688976Z digest=sha256:8132ba9a277c71ad7897a40e9b832d83dfa302f5685cecdd960b6fbb5dcb0f8c

Observation 4fca715d-5d5d-4316-a7e7-c0c8c5af6b18 · outbound

This paper cites Bender, Alex Hanna, and Amandalynne Paullada.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Bender, Alex Hanna, and Amandalynne Paullada

Reference 35

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.862940Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.862940Z digest=sha256:fb17728bf147ae70e712486aa8f5df39f5443891937fe7307d27fcb660f828ea

Observation db18018f-e43b-4cf7-8f0a-1e5d630aa9bd · outbound

This paper cites The effect of sampling temperature on problem solving in large language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The effect of sampling temperature on problem solving in large language models

Reference 36

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:40.975379Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:40.975379Z digest=sha256:bb86674a582e3aa6de7faa97709fbfd4fea874168c0d6e9b7afd35a2dca4867f

Observation 44a2d16e-6a6f-4b4e-b5e5-73306be032ec · outbound

This paper cites Improving evidence retrieval for automated explainable fact-checking.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Improving evidence retrieval for automated explainable fact-checking

Reference 37

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.860122Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.040347Z digest=sha256:6ca50798a562caacd847f78f9101d3be7a021ffa3a16fb236bd6000199748822

Observation 187fe219-bbef-4d96-8112-24a864213317 · outbound

This paper cites an unresolved cited work.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Unresolved cited work

Reference 38

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.097841Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.097841Z digest=sha256:cad92cc7e59139fcd0e039ed90b58335d24b9c08b5c6dc7d06e989319fd0f713

Observation 71985320-c1e8-4ecf-a5dc-8468c119c5d6 · outbound

This paper cites The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism

Reference 39

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.138123Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.138123Z digest=sha256:6cd7fb1198b2d49e92d4018c199a49d72ba80f6c3fb0d3efa3e894d72c4a2acd

Observation 3aa554d2-c2d9-4573-ac49-0400e7b08b79 · outbound

This paper cites Scieval: a multi-level large language model evaluation benchmark for scientific research.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Scieval: a multi-level large language model evaluation benchmark for scientific research

Reference 40

Resolution
malformed identifier
no resolver link, observed 2026-08-06T15:29:41.201818Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.201818Z digest=sha256:47a045a0a14fd2082abddfd4ac0e6aceb142e9764165c5da5d90a0166ab0da9a

Observation 5dd25c18-98f3-4b28-91af-f17b7e0950cd · outbound

This paper cites Nestor, Ali Soroush, Pierre A.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Nestor, Ali Soroush, Pierre A

Reference 41

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.679016Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.320038Z digest=sha256:3812cbe463261e1996088d76279df9e9d8b92c2d4a013ba2839713ae04607b36

Observation 6d21ebef-9c75-4bf2-8a6d-fe3d9e583d95 · outbound

This paper cites FEVER : a large-scale dataset for fact extraction and VER ification.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs FEVER : a large-scale dataset for fact extraction and VER ification

Reference 42

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.418093Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.418093Z digest=sha256:b4cd82c8402435653391e0b3bc6bfd75152ce454d65e6706abab21fa8a58a523

Observation c21746c8-56d1-4ecc-b42f-5a61afa09ddd · outbound

This paper cites Group chat ecology in enterprise instant messaging: How employees collaborate through multi-user chat channels on slack.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Group chat ecology in enterprise instant messaging: How employees collaborate through multi-user chat channels on slack

Reference 43

Resolution
verified exact
doi, observed 2026-08-06T15:29:42.523254Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.479622Z digest=sha256:97d64b343a382ba2a70f543a4e430665a2c97b2409349fbac498804c47cad859

Observation 643ed9c8-a501-4895-bf7a-81bef5ad4cce · outbound

This paper cites Evaluating large language models on academic literature understanding and review: An empirical study among early-stage scholars.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Evaluating large language models on academic literature understanding and review: An empirical study among early-stage scholars

Reference 44

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.581642Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.581642Z digest=sha256:1a8b2fe07acabbf666ffa607ded044ac023254f929643acbd3e6dba5aa2e2579

Observation 194c1acc-3bff-422b-b52c-c2e22d164b3c · outbound

This paper cites as an ai language model, i cannot.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs as an ai language model, i cannot

Reference 45

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.655835Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.655835Z digest=sha256:96a9f9d8ae66359d4b8841cbfdd1d1d343f27d531f934b6c15388a7b407c1843

Observation b5cda476-4e8d-475d-a1c2-9704ddabbf1e · outbound

This paper cites Designing an Evaluation Framework for Large Language Models in Astronomy Research.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Designing an Evaluation Framework for Large Language Models in Astronomy Research

Reference 46

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.746197Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.746197Z digest=sha256:8f6acc62bed61e1d7a197124112db90c282c84056caaf45dd588471d250b987b

Observation 347b5cc6-f90b-465a-87da-5b86b931a88a · outbound

This paper cites C-pack: Packed resources for general chinese embeddings.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs C-pack: Packed resources for general chinese embeddings

Reference 47

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.830639Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.830639Z digest=sha256:5be71387ef0c85149f8da6c5911713dd5222cb08bec0839788a3ca65b5c06cbb

Observation 844b0afc-4d00-46e9-a8b5-eb4d4912d025 · outbound

This paper cites AI as an active writer: Interaction strategies with generated text in human- AI collaborative fiction writing.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs AI as an active writer: Interaction strategies with generated text in human- AI collaborative fiction writing

Reference 48

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:45.034922Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:41.891915Z digest=sha256:a33a6ba1fbe77448200f4621b76d64968493d73abf935903cb52d767b0687b27

Observation 01bbfc41-c32a-45ea-bb2a-0b9b7c493064 · outbound

This paper cites User-Controlled Knowledge Fusion in Large Language Models: Balancing Creativity and Hallucination.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs User-Controlled Knowledge Fusion in Large Language Models: Balancing Creativity and Hallucination

Reference 49

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:41.990196Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:41.990196Z digest=sha256:a31336d4e8edee221f32a5455b260e32345ab19c63c3fddb820c568b97579a3f

Observation b2bde327-f37d-4162-9d84-bf16ed863ac6 · outbound

This paper cites Weinberger, and Yoav Artzi.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Weinberger, and Yoav Artzi

Reference 50

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.830578Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.057541Z digest=sha256:c03d567951d458e97bac08424b52a6aaa49978912797cb3ab856b36a8aed35ed

Observation 40a5499e-16cf-4445-8e8f-cd3dde3432f5 · outbound

This paper cites WildChat : 1 M C hat GPT interaction logs in the wild.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs WildChat : 1 M C hat GPT interaction logs in the wild

Reference 51

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.598764Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.106650Z digest=sha256:1cde283011ebed7c6b3825c9426d9443b2f160ffa5191f1a609058825929c9b5

Observation 67552ce0-5813-426e-a34d-c33fa787ac2b · outbound

This paper cites Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications

Reference 52

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:42.197073Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:42.197073Z digest=sha256:e88833e8a7da6283359f86f141bcedd64ade06a416d39c697ecc43b4243a90c7

Observation e98031c2-bf97-47b0-a7f9-a425c93788b5 · outbound

This paper cites Navigating the grey area: How expressions of uncertainty and overconfidence affect language models.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Navigating the grey area: How expressions of uncertainty and overconfidence affect language models

Reference 53

Resolution
unresolved
no resolver link, observed 2026-08-06T15:29:42.298617Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T15:29:42.298617Z digest=sha256:e16dec5087a2d5f9c215a3ab9367fb857e1477a76a3348549caf022432a97bcd

Observation fe456e7f-3a03-4119-ac9b-93293c239cc1 · outbound

This paper cites Relying on the unreliable: The impact of language models ' reluctance to express uncertainty.

From Queries to Criteria: Understanding How Astronomers Evaluate LLMs Relying on the unreliable: The impact of language models ' reluctance to express uncertainty

Reference 54

Resolution
verified fuzzy
raw_fallback, observed 2026-08-06T15:29:44.375318Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

source=arxiv_source observed=2026-08-06T15:29:42.348441Z digest=sha256:6ef76e8243c6226d4b9471e3bd2a394f53b9241630f62a95d1286de589aaba92

Pith citing papers

No inbound Pith citation observations are available.