Paper Citation Record · LEDGER

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training

As of 13 August 2026, this Paper Citation Record lists 22 of 22 outbound references and 0 inbound Pith citation observations for arXiv:2411.16618.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2411.16618 v1

Coverage vector

measured 22 of 22 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-12T12:57:10.419757Z

measured 22 of 22 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-13T06:32:02.005865+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

22 of 22 outbound references displayed

verified exact3
verified fuzzy0
unresolved19
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation ff0d57aa-701c-4fbf-aedf-f332e042d0b8 · outbound

This paper cites Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.321304Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9271ea2d-d0e9-4d74-b19a-98f26e76fc9b · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.326876Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b3b9bfb0-e2ac-4639-a6d2-2e4feb79b6d5 · outbound

This paper cites Longformer: The Long-Document Transformer.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Longformer: The Long-Document Transformer

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.336499Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 49ff6eed-1157-4894-99bb-9f2ab2f67ebf · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.341379Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 551661b6-d13e-40fc-a21d-a897fb70bab3 · outbound

This paper cites Generating Long Sequences with Sparse Transformers.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Generating Long Sequences with Sparse Transformers

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.346238Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 070c0b76-0fe4-4e14-80eb-ba30aa2faa8a · outbound

This paper cites What Does BERT Look At? An Analysis of BERT's Attention.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training What Does BERT Look At? An Analysis of BERT's Attention

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.351093Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3dbc6d13-94be-4939-b0d5-1a794a06b99d · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 8

Resolution: unresolved; raw_fallback, observed 2026-08-12T12:57:10.825050Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation c2e66699-5ecc-49b3-b376-9b33c823dad8 · outbound

This paper cites Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.359611Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 52d0a0bb-d9bb-45e6-b558-d2a755f33054 · outbound

This paper cites SciREX: A Challenge Dataset for Document-Level Information Extraction.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training SciREX: A Challenge Dataset for Document-Level Information Extraction

Reference 10

Resolution: verified exact; local_arxiv, observed 2026-08-12T12:57:10.678044Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 4d30b2fb-8644-4ecb-8fbd-54f3341020c2 · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.368060Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 79638ee5-813e-41f8-b6ae-9b88577f9e76 · outbound

This paper cites Gomez, ukasz Kaiser, and Illia Polosukhin.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Gomez, ukasz Kaiser, and Illia Polosukhin

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.372035Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 958ed193-c0d6-4a4b-bb17-ab735a2d82b5 · outbound

This paper cites A Multiscale Visualization of Attention in the Transformer Model.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training A Multiscale Visualization of Attention in the Transformer Model

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.376013Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2448f413-d812-478f-bd11-ec3a95001274 · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.380153Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a65310ea-0f9e-4cc8-ab62-352413e48a05 · outbound

This paper cites Should You Mask 15% in Masked Language Modeling?.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Should You Mask 15% in Masked Language Modeling?

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.384081Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 94d40095-8429-49a0-b4be-b51c93d4345c · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 16

Resolution: unresolved; raw_fallback, observed 2026-08-12T12:57:10.800997Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 637cc336-870a-4a0d-9e29-cc623f14fe94 · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 17

Resolution: unresolved; raw_fallback, observed 2026-08-12T12:57:10.786158Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 20d5abe3-3def-41f0-9e3e-4f7b4f6a96fe · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 18

Resolution: verified exact; doi, observed 2026-08-12T12:57:10.461026Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 81b72c04-b3dd-41d0-b00b-401ac3c85016 · outbound

This paper cites Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.401528Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f9cbdf35-174c-4d91-806a-255a43e5e568 · outbound

This paper cites an unresolved cited work.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Unresolved cited work

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.406041Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 410806e6-78d3-45f8-bbca-b66f97020879 · outbound

This paper cites Importance of Data Loading Pipeline in Training Deep Neural Networks.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training Importance of Data Loading Pipeline in Training Deep Neural Networks

Reference 21

Resolution: verified exact; local_arxiv, observed 2026-08-12T12:57:10.522219Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 5737a762-c682-46d8-bc9d-61255d7f0a5c · outbound

This paper cites online" 'onlinestring :=.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training online" 'onlinestring :=

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.414994Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 29fb1313-c0a5-4ee7-ace3-fc748c20d238 · outbound

This paper cites write newline.

StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Training write newline

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-12T12:57:10.419757Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.