REVIEW 3 major objections 3 minor 43 references
PaDoc: Layout-Grounded Parallel Decoding for Document Parsing
T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read PaDoc shows that document parsing can instead branch off a shared layout stream, cutting the decoding path from total content length to the longest layout-content path.
desk verdict PaDoc is a solid, honestly-reported efficiency paper for document parsing; the load-bearing independence assumption is clearly stated but untested, which is the right thing to ask for in revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is layout-grounded branching, expressed as an ancestor-attention graph in which the image is the root, layout nodes $B_1,\dots,B_N$ form a chain, and each content node $Y_k$ has ancestors $\{X, B_{\le k}\}$. Training realizes this graph as packed variable-length causal attention: each content branch attends to the shared image and layout prefix plus its own tokens, but never to sibling contents or future layout, so a single next-token objective trains the whole parallel factorization. At inference, the decoder forks a content stream whenever a region-closing layout token appears, and the serving runtime treats the layout stream and all content streams as concurrent requests whose token-identical multimodal and layout prefixes are reused through automatic prefix caching.
What would settle it
Parse pages that contain explicit cross-region dependencies, such as a table caption referring to a later table or a paragraph continuing across two regions, with PaDoc and with a sequential baseline: if the parallel model makes errors that the sequential model avoids exactly on those pages, the sufficiency assumption is violated. A direct statistical check would estimate the conditional mutual information $I(Y_k; B_{>k}, Y_{<k} \mid X, B_{\le k})$ on a labeled corpus; values clearly separated from zero would overturn the factorization.
Extended reading notes
Core claim
The paper's central discovery is that the exact document-parsing distribution $p^\star(b_{1:N}, y_{1:N} \mid x)$ can be rewritten as $\prod_{k=1}^{N} p^\star(b_k \mid x, b_{<k}) \prod_{k=1}^{N} p^\star(y_k \mid x, b_{\le k})$, provided a region's content is conditionally independent of other regions' contents and of future layout once the page image and the region's own layout prefix are known. Under that assumption, the next layout region $B_{k+1}$ and the current region's content $Y_k$ are both functions of the same prefix $(X, B_{\le k})$, so they can be decoded concurrently, and the logical decoding depth becomes $D_{\mathrm{PaDoc}} = \max_{1\le k\le N}\{\ell(B_{\le k})+\ell(Y_k)\}$ instead of $\ell(B_{1:N})+\sum_{k=1}^{N}\ell(Y_k)$. PaDoc realizes this with a single multimodal LLM: an ancestor-attention mask makes standard next-token training reproduce the factorization, while at serving time the layout stream and content branches execute as independent vLLM requests that reuse a shared image and layout prefix through the key–value cache. The measured consequence is a 67.4–118% throughput gain and a 39.2–54.9% P95 latency reduction over the same-backbone sequential baseline, with an OmniDocBench Overall of 94.24 and the best Text Edit and Formula CDM among end-to-end parsers.
Load-bearing premise
The load-bearing premise is that a region's content carries no information from other regions' contents or from later layout once the full page image and that region's own layout are known; if a document has a caption that refers to a later table or a paragraph whose wording depends on the previous region, this premise fails.
Editorial extensions
If this is right
- Pages with many spatially separate regions gain the most, because the serial cost of summing all region lengths is replaced by the length of the single longest layout-content path.
- Parallel decoding does not require a separate layout detector or a draft model; the same MLLM both predicts layout and generates all region contents, so deployment stays a single model with standard serving infrastructure.
- Because every content branch keeps the full page image as prefix, the approach preserves page-level context that crop-based two-stage parsers lose, including reading order and cross-region references.
- The controlled same-backbone comparison indicates the factorization itself, not a bigger model, is what drives the efficiency gain, and quality remains competitive despite the parallel structure.
Reading between the lines
- If region-specific content sufficiency holds for other visual document types, the same fork-and-branch decoding could accelerate layout-grounded tasks such as form understanding, slide parsing, or screenshot-to-code, where detected regions provide natural fork points.
- The reported wall-clock gains are likely a conservative estimate of the structural speedup, because the request-level vLLM realization pays admission and scheduling overhead; a fused implementation of the ancestor mask could move closer to the ideal critical-path ratio, though unequal branch lengths would still cap the gain.
- Throughput plateaus after C64 in the paper's measurements, suggesting that beyond that concurrency the bottleneck shifts from decoding depth to scheduler capacity and memory bandwidth, so multi-GPU deployments would need balanced branch scheduling to keep the same relative advantage.
- The same-backbone quality tables show PaDoc trades some recall for much higher precision in layout localization, which suggests the parallel training objective may induce a more conservative region predictor, a behavioral difference worth testing beyond aggregate F1.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. PaDoc proposes a layout-grounded parallel decoder for end-to-end document parsing. The paper derives a prefix-conditioned factorization from a region-sufficiency conditional-independence assumption, implements it with ancestor attention and packed variable-length attention, and serves the resulting branches as concurrent vLLM requests with shared-prefix cache reuse. On OmniDocBench v1.6 Full it reports layout F1 91.1, end-to-end Overall 94.24, and on a 384-page single-A800 workload 67.4%-118% throughput gains and 39.2%-54.9% P95 latency reductions relative to a same-backbone Sequential SFT baseline. The appendix contains derivations, a custom PageIoU protocol, controlled same-backbone quality comparisons, serving details, and a training-backend ablation.
Significance. The central theoretical reduction is valid given Assumption 1, and the measured efficiency gains are presented with a controlled same-backbone comparison and an explicit distinction between ideal structural depth (Appendix E.2) and wall-clock results. The release of code, the same-backbone Sequential SFT baseline, and the transparent PageIoU protocol are concrete strengths. The remaining risk is empirical: Assumption 1 is not tested, and the paper's own Related Work identifies cross-region dependencies that could violate it. If the assumption fails on a substantial page population, the parallel branches no longer sample the full sequential distribution, so the quality-equivalence claim needs a targeted probe before the method can be taken as a drop-in replacement for sequential parsing.
major comments (3)
- [Problem Formulation (Eqs. 5-10)] Assumption 1 (Eq. 5) is load-bearing: it is the only premise that turns Eq. (10) into an exact factorization of p*(b,y|x), yet it is not tested anywhere in the paper. The aggregate comparison in Appendix D.1, Table 10 (Overall -0.345 vs. Sequential SFT) is reassuring but is a macro-average that can hide systematic degradation on pages with cross-region references, captions tied to tables, or section hierarchy; the paper's own Related Work names precisely these dependencies as context that independent recognition 'may weaken.' I request a targeted probe: report PaDoc vs. Sequential SFT quality on a stratified subset of cross-reference-heavy pages, or an evaluation that conditions on the presence of such dependencies, and/or a direct conditional-independence test using held-out prefixes. Until such evidence exists, the claim that parallel decoding 'preserves competitive quality' should be scoped to the aggregate benchmark.
- [Appendix E.3 (Serving envelope)] The headline speedup vs. Sequential SFT may be confounded by a serving-configuration asymmetry. Appendix E.3 states that the standard scheduler envelope is 256 sequences and 32,768 batched tokens per iteration, while the evaluated PaDoc configuration allows 384 sequences and a 24,576-token budget. Table 3 attributes the 67.4%-118% throughput gain to PaDoc's parallel decoding, but if the Sequential SFT baseline runs under the standard envelope rather than the same 384/24,576 envelope, part of the gain could come from the scheduler settings rather than from layout-grounded branching. Please either run Sequential SFT under the identical PaDoc envelope, or report PaDoc under the standard envelope, for the same-backbone comparison.
- [Experiments / Layout Protocol (Table 1, Appendix B)] The headline layout result uses the newly defined PageIoU countable-acceptable protocol rather than an official metric of OmniDocBench. Because external two-stage and end-to-end parsers are compared under this custom protocol, their relative ordering may depend on the category mapping in Table 4 and on the acceptable-coverage elasticity in Eq. (26). I do not object to a transparent custom metric, but the paper should make clear whether the same external prediction outputs would also rank similarly under the benchmark's official layout metric; otherwise the claim that PaDoc is 'top-tier' in layout accuracy is metric-dependent. At minimum, report the published official layout F1 for the external systems and PaDoc under identical conditions.
minor comments (3)
- [Appendix E.3 (Validity statistics)] The validity filter treats truncated outputs as invalid, and the appendix reports that 20-25 Sequential SFT pages per setting reached the generation limit while all PaDoc pages completed normally. Since the wall-clock denominator includes work on invalid pages, the valid-page throughput comparison combines generation-capacity effects with decoding parallelism; please report raw completion rates and throughput computed with a lenient validity definition as a sensitivity check.
- [Training Paradigm (Eq. 16 and following paragraph)] The branch-start token mechanism and the handling of variable-length regions in the packed sequence are described at a high level; a small worked token-sequence example in the main text or appendix would improve reproducibility.
- [Tables 1 and 2] The tables report a single run per system; small differences such as PaDoc vs. Qianfan-OCR (0.34 Overall points) are not accompanied by variance estimates, so the ranking should be interpreted with caution.
Circularity Check
No significant circularity: the prefix-conditioned factorization follows from an explicit assumption via the conditional-mutual-information chain rule, and all efficiency claims are measured wall-clock rather than derived from the ideal critical-path formula.
full rationale
PaDoc's derivation is self-contained. Assumption 1 (Eq. 5) is an explicit statistical premise, and the step to the prefix-conditioned form (Eq. 8), identity (Eq. 9), and factorization (Eq. 10) is a direct chain-rule/weak-union argument given in Appendix A.1; it is a mathematical implication, not a fitted input renamed as a prediction. The decoding-depth expression DPaDoc (Eq. 14) is explicitly labeled an ideal structural quantity, and the paper's actual efficiency claims rest on measured throughput and P95 latency against a same-backbone Sequential SFT baseline (Table 3), not on Eq. 14. No constants are fit to OmniDocBench to produce the quality results, and the benchmark comparisons are external. The nearest potential self-citations (Qwen3-VL checkpoint, InternVL3.5 baselines) are ordinary references to external models and published results, and they do not carry the derivation. Assumption 1 is untested and could be violated by cross-region dependencies such as table-caption relations or reading-order context, but an untested premise is a correctness risk, not circularity.
Assumptions & free parameters
free parameters (1)
- Serving scheduler envelope for PaDoc =
384 sequences; 24,576 token budget per iteration
assumptions (1)
- domain assumption Region-specific content sufficiency: I(Yk; B<k, B>k, Y<k | X, Bk) = 0 for each region k.
Cite this review
Pith. "Pith review of PaDoc: Layout-Grounded Parallel Decoding for Document Parsing." pith.science (2026). https://pith.science/paper/ERLA7VLV
@misc{pith2026260806146,
author = {Pith},
title = {Pith review of: PaDoc: Layout-Grounded Parallel Decoding for Document Parsing},
year = {2026},
howpublished = {\url{https://pith.science/paper/ERLA7VLV}},
note = {Machine review of arXiv:2608.06146}
}
read the original abstract
End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence. This formulation forces independent regions onto a decoding path whose length grows with the total content, whereas crop-based two-stage parsers expose region-level parallelism at the cost of repeated visual prefills and fragmented page context. To retain full-page context while removing dependencies, we propose PaDoc, a layout-grounded parser that treats the predicted layout as a branching structure over a shared page representation. Under a region-sufficiency assumption, we derive a prefix-conditioned factorization in which the layout stream and regional content branches advance concurrently, reducing the decoding depth to the longest layout-content path. We realize this factorization within a single MLLM: packed variable-length ancestor attention preserves the visibility under standard next-token training, while masked parallel decoding creates branches that the evaluated vLLM backend serves as concurrent requests with cache-resident shared-prefix reuse. On OmniDocBench Full, PaDoc attains an Overall layout F1 of 91.1 and, among end-to-end parsers, a top-tier Overall score of 94.24 together with the best Text Edit (0.038) and Formula CDM (95.59). On a 384-page subset and one A800 GPU, it is the fastest end-to-end parser at five concurrency levels, improving valid-page throughput by 67.4-118% and reducing P95 latency by 39.2-54.9% relative to a same-backbone Sequential SFT baseline. Code is available at https://github.com/Longin-Yu/Padoc
Figures
Reference graph
Works this paper leans on
-
[1]
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations , author =. 2024 , journal =
work page 2024
-
[2]
InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency , author =. 2025 , journal =
work page 2025
- [3]
- [4]
-
[5]
dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model , author =. 2025 , journal =
work page 2025
- [6]
- [7]
-
[8]
APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding , author =. 2024 , journal =
work page 2024
Show all 43 references
-
[9]
arXiv preprint arXiv:2601.21957 , year=
PaddleOCR-VL-1.5: Towards a Multi-Task 0.9 B VLM for Robust In-the-Wild Document Parsing , author=. arXiv preprint arXiv:2601.21957 , year=
-
[10]
arXiv preprint arXiv:2607.04884 , year=
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better , author=. arXiv preprint arXiv:2607.04884 , year=
-
[11]
arXiv preprint arXiv:2607.11562 , year=
MonkeyOCRv2: A Visual-Text Foundation Model for Document AI , author=. arXiv preprint arXiv:2607.11562 , year=
-
[12]
5: A decoupled vision-language model for efficient high-resolution document parsing , author=
Mineru2. 5: A decoupled vision-language model for efficient high-resolution document parsing , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) , pages=
2026
-
[13]
Information and Control , volume=
A definition of conditional mutual information for arbitrary ensembles , author=. Information and Control , volume=. 1978 , publisher=
1978
-
[14]
American mathematical society translations , volume=
General formulation of Shannon's main theorem in information theory , author=. American mathematical society translations , volume=. 1963 , publisher=
1963
-
[15]
arXiv preprint arXiv:2410.12628 , year=
Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception , author=. arXiv preprint arXiv:2410.12628 , year=
-
[16]
arXiv preprint arXiv:2503.17213 , year=
PP-DocLayout: A unified document layout detection model to accelerate large-scale data construction , author=. arXiv preprint arXiv:2503.17213 , year=
-
[17]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
DocBed: A multi-stage OCR solution for documents with complex layouts , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[18]
Findings of the Association for Computational Linguistics: ACL 2025 , pages=
Dolphin: Document image parsing via heterogeneous anchor prompting , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=
2025
-
[19]
arXiv preprint arXiv:2602.05384 , year=
Dolphin-v2: Universal Document Parsing via Scalable Anchor Prompting , author=. arXiv preprint arXiv:2602.05384 , year=
-
[20]
arXiv preprint arXiv:2602.12957 , year=
HSD: Training-Free Acceleration for Document Parsing Vision-Language Model with Hierarchical Speculative Decoding , author=. arXiv preprint arXiv:2602.12957 , year=
-
[21]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
Spts v2: single-point scene text spotting , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2023 , publisher=
2023
-
[22]
European Conference on Computer Vision (ECCV) , year =
OCR-Free Document Understanding Transformer , author =. European Conference on Computer Vision (ECCV) , year =
-
[23]
International Conference on Machine Learning , pages=
Pix2struct: Screenshot parsing as pretraining for visual language understanding , author=. International Conference on Machine Learning , pages=. 2023 , organization=
2023
-
[24]
International Conference on Learning Representations , volume=
Nougat: Neural optical understanding for academic documents , author=. International Conference on Learning Representations , volume=
-
[25]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[26]
arXiv preprint arXiv:2502.18443 , year=
olmocr: Unlocking trillions of tokens in pdfs with vision language models , author=. arXiv preprint arXiv:2502.18443 , year=
-
[27]
arXiv preprint arXiv:2603.13398 , year=
Qianfan-OCR: A Unified End-to-End Model for Document Intelligence , author=. arXiv preprint arXiv:2603.13398 , year=
-
[28]
Proceedings of the 29th symposium on operating systems principles , pages=
Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=
-
[29]
Advances in neural information processing systems , volume=
Sglang: Efficient execution of structured language model programs , author=. Advances in neural information processing systems , volume=
-
[30]
International Conference on Machine Learning , pages=
Fast inference from transformers via speculative decoding , author=. International Conference on Machine Learning , pages=. 2023 , organization=
2023
-
[31]
arXiv preprint arXiv:2401.10774 , year=
Medusa: Simple llm inference acceleration framework with multiple decoding heads , author=. arXiv preprint arXiv:2401.10774 , year=
-
[32]
arXiv preprint arXiv:2402.02057 , year=
Break the sequential dependency of llm inference using lookahead decoding , author=. arXiv preprint arXiv:2402.02057 , year=
-
[33]
arXiv preprint arXiv:2401.15077 , year=
Eagle: Speculative sampling requires rethinking feature uncertainty , author=. arXiv preprint arXiv:2401.15077 , year=
-
[34]
arXiv preprint arXiv:2404.19737 , year=
Better & faster large language models via multi-token prediction , author=. arXiv preprint arXiv:2404.19737 , year=
-
[35]
International Conference on Learning Representations , volume=
Skeleton-of-thought: Prompting llms for efficient parallel generation , author=. International Conference on Learning Representations , volume=
-
[36]
arXiv preprint arXiv:2502.11517 , year=
Learning to keep a promise: Scaling language model decoding parallelism with learned asynchronous decoding , author=. arXiv preprint arXiv:2502.11517 , year=
-
[37]
arXiv preprint arXiv:2508.08895 , year=
Aspd: Unlocking adaptive serial-parallel decoding by exploring intrinsic parallelism in llms , author=. arXiv preprint arXiv:2508.08895 , year=
-
[38]
Advances in Neural Information Processing Systems , volume=
Sprint: Enabling interleaved planning and parallelized execution in reasoning models , author=. Advances in Neural Information Processing Systems , volume=
-
[39]
Advances in Neural Information Processing Systems , volume=
Hogwild! inference: Parallel llm generation via concurrent attention , author=. Advances in Neural Information Processing Systems , volume=
-
[40]
International Conference on Learning Representations , volume=
Deft: Decoding with flash tree-attention for efficient tree-structured llm inference , author=. International Conference on Learning Representations , volume=
-
[41]
5-Pro: Pushing the limits of data-centric document parsing at scale , author=
MinerU2. 5-Pro: Pushing the limits of data-centric document parsing at scale , author=. arXiv preprint arXiv:2604.04771 , year=
-
[42]
arXiv preprint arXiv:2603.13032 , year=
Multimodal OCR: Parse anything from documents , author=. arXiv preprint arXiv:2603.13032 , year=
-
[43]
arXiv preprint arXiv:2010.11929 , year=
An image is worth 16x16 words: Transformers for image recognition at scale , author=. arXiv preprint arXiv:2010.11929 , year=
2010 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.