Pith. sign in

REVIEW 4 cited by

A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.01976 v3 pith:7GLS5HEJ submitted 2024-07-02 cs.CL cs.AIcs.MM

Jinghui Lu , Haiyang Yu , Yanjie Wang , Yongjie Ye , Jingqun Tang , Ziwei Yang , Binghong Wu , Qi Liu
show 4 more authors
Hao Feng Han Wang Hao Liu Can Huang
This is my paper · ORCID
classification cs.CLcs.AIcs.MM
keywords laytextllmtextllmsdocumentlanguagelargelayouttasks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recently, many studies have demonstrated that exclusively incorporating OCR-derived text and spatial layouts with large language models (LLMs) can be highly effective for document understanding tasks. However, existing methods that integrate spatial layouts with text have limitations, such as producing overly long text sequences or failing to fully leverage the autoregressive traits of LLMs. In this work, we introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM)} for document understanding. LayTextLLM projects each bounding box to a single embedding and interleaves it with text, efficiently avoiding long sequence issues while leveraging autoregressive traits of LLMs. LayTextLLM not only streamlines the interaction of layout and textual data but also shows enhanced performance in KIE and VQA. Comprehensive benchmark evaluations reveal significant improvements of LayTextLLM, with a 15.2% increase on KIE tasks and 10.7% on VQA tasks compared to previous SOTA OCR-based LLMs. All resources are available at https://github.com/LayTextLLM/LayTextLLM.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth

    cs.LG 2026-05 conditional novelty 5.0 of 10

    OCR tools can be ranked without ground-truth labels by measuring how much a multimodal LLM must correct each tool's output.

  2. Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning

    cs.CL 2025-05 conditional novelty 5.0 of 10

    CAR routes each query to either a short answer or full reasoning based on the perplexity of the model's draft answer, improving accuracy and cutting token use on VQA, KIE, and math/common sense benchmarks.

  3. LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data

    cs.CV 2026-01 reject novelty 4.0 of 10

    A lightweight RGB-D cross-attention network is proposed for rail defect detection, but the SOTA accuracy and generalization claims are internally inconsistent and the implementation is not public.

  4. Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method

    cs.CV 2026-01 reject novelty 3.0 of 10

    Fine-tuning Stable Diffusion with DreamBooth-style knowledge and hypernetwork-guided crack control maps can synthesize substation meter defect images that boost a YOLOv8 defect detector's mAP when added to the training set.

Pith tools