Paper Citation Record · LEDGER

Large Language Models for IT Automation Tasks: Are We There Yet?

As of 11 August 2026, this Paper Citation Record lists 35 of 35 outbound references and 0 inbound Pith citation observations for arXiv:2505.20505.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2505.20505 v1

Coverage vector

measured 35 of 35 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-07T13:58:24.398450Z

measured 35 of 35 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-11T06:34:44.6726+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

35 of 35 outbound references displayed

verified exact1
verified fuzzy3
unresolved23
parse uncertain0
malformed identifier8
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 75baa306-a754-48cc-a577-66a628bdd8e7 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 1

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.726709Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 28adf6ff-5581-4807-9032-04eedf2e196a · outbound

This paper cites Posts relying heavily on obscure, deprecated, or highly specialized external collections were typically excluded.

Large Language Models for IT Automation Tasks: Are We There Yet? Posts relying heavily on obscure, deprecated, or highly specialized external collections were typically excluded

Reference 2

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T13:58:27.633261Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 21d5b4ea-7853-44b8-8cad-b60babc80b2d · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 3

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.354544Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 00c2ebca-2ee8-44ac-8aef-39fb141dd32c · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 4

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.221044Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 7f113197-b26f-4629-b739-b3bb92e8c82b · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 5

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.138910Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation d6eede5c-936a-4ea7-8c00-141eee005559 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 6

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.029149Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 3a2ae625-1487-4b1e-a090-e0572a6c4aa9 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 7

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.521777Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 3d5db91f-5d34-488c-9cbe-480fd5831df3 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 8

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:27.430756Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 94b8ac9f-b2fa-40ff-954a-45c47d1349a6 · outbound

This paper cites A.3 Manual Curation Criteria The set of posts selected via stratified sampling (ini- tially 200 posts) underwent manual review.

Large Language Models for IT Automation Tasks: Are We There Yet? A.3 Manual Curation Criteria The set of posts selected via stratified sampling (ini- tially 200 posts) underwent manual review

Reference 13

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T13:58:26.917522Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation d9cbee04-2c80-4a8f-bf4f-914932558329 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 14

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:26.819638Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation c0a0de8c-8e78-419d-bbb8-e11e64e4b5a9 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 15

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:26.737094Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation ff47b632-73dc-4769-8490-f5725914d44f · outbound

This paper cites upward trend.

Large Language Models for IT Automation Tasks: Are We There Yet? upward trend

Reference 16

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:26.646689Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 0190151b-06a9-4235-808d-dd9c316a1a2a · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 17

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:26.538827Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 1427b0ff-f714-453d-84a3-fab17ec5a93a · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 18

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:26.433879Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 0c2d0b5a-6165-40d4-b568-1e8edb97745a · outbound

This paper cites All solutions should be self-contained and well-documented.

Large Language Models for IT Automation Tasks: Are We There Yet? All solutions should be self-contained and well-documented

Reference 19

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:26.330455Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation e681226d-e32b-46c4-b07b-9374971857e2 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 25

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.712958Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation a1578c7f-e128-4098-b568-58a8e5fb6a5c · outbound

This paper cites Table 8: Level 3 TELeR (Santu and Feng, 2023) Prompt Structure for IT Automation Tasks.

Large Language Models for IT Automation Tasks: Are We There Yet? Table 8: Level 3 TELeR (Santu and Feng, 2023) Prompt Structure for IT Automation Tasks

Reference 26

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:25.585825Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation c970bd44-7c0e-40fa-86c7-598e5ed7abfa · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 27

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.465983Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 36831de7-7a1d-4270-85bf-f6d2fce78133 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 28

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.332044Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation c69ee08a-61e3-4524-9f24-75d50f37d4d7 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 29

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.237063Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 1ca7606e-f85d-4908-ba2c-2b2a3efe8f18 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 30

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.144868Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation d8a06a25-eafb-47e1-8b55-5d727cfa28f0 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 31

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.046560Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation c2ff49e4-02ea-4335-81a5-be5f3db823d5 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 32

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:24.959527Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 8e7312b6-ba40-4c09-98fa-386106913376 · outbound

This paper cites prompt Transform the following Stack Overflow post into an Ansible playbook solution.

Large Language Models for IT Automation Tasks: Are We There Yet? prompt Transform the following Stack Overflow post into an Ansible playbook solution

Reference 33

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:24.859053Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 054032f1-304c-4522-80b9-33a5d4631caa · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 34

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:26.218704Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 74466a0d-a846-4023-ae4f-c0c0ec6ef127 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 35

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:26.147640Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation c0f82e97-4e2b-460c-8345-1b9e306305f7 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 36

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:26.010309Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation d7bd8df9-b22f-48b4-83af-8191ee48bb82 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 37

Resolution: malformed identifier; raw_fallback, observed 2026-08-07T13:58:25.924914Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation ac621e45-1f68-49fc-a8e4-890d6fbf79b3 · outbound

This paper cites an unresolved cited work.

Large Language Models for IT Automation Tasks: Are We There Yet? Unresolved cited work

Reference 38

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:58:25.832690Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 93e89e34-a75e-47d3-a57c-68c821452bb7 · outbound

This paper cites • Carefully examine the constraint and runtime environment, and select modules appropriate for both.

Large Language Models for IT Automation Tasks: Are We There Yet? • Carefully examine the constraint and runtime environment, and select modules appropriate for both

Reference 39

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T13:58:24.787789Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation 6fc12e42-c7b0-4833-93ec-d768a2b38783 · outbound

This paper cites {{ lookup ( ' env ', ' username ') }}.

Large Language Models for IT Automation Tasks: Are We There Yet? {{ lookup ( ' env ', ' username ') }}

Reference 40

Resolution: verified exact; raw_fallback, observed 2026-08-07T13:58:24.683697Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-11T06:34:44.6726+00:00.

Observation ee1829f7-b026-4a3a-a39a-b6f312bd1bb2 · outbound

This paper cites Code Llama: Open Foundation Models for Code.

Large Language Models for IT Automation Tasks: Are We There Yet? Code Llama: Open Foundation Models for Code

Reference 2020

Resolution: unresolved; no resolver link, observed 2026-08-07T13:58:21.640426Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cdeb6428-0f1a-45a5-9e4a-7ced476d04cf · outbound

This paper cites Evaluating Large Language Models Trained on Code.

Large Language Models for IT Automation Tasks: Are We There Yet? Evaluating Large Language Models Trained on Code

Reference 2021

Resolution: unresolved; no resolver link, observed 2026-08-07T13:58:21.505733Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 242ee573-5491-4171-99f5-dbf3aff4dd33 · outbound

This paper cites StarCoder 2 and The Stack v2: The Next Generation.

Large Language Models for IT Automation Tasks: Are We There Yet? StarCoder 2 and The Stack v2: The Next Generation

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-07T13:58:21.565010Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 64ccc9b5-6d1c-4bd1-89e0-935453735973 · outbound

This paper cites Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task.

Large Language Models for IT Automation Tasks: Are We There Yet? Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-07T13:58:21.731545Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.