{"id":"7ac49e38-dfa1-4fb8-912d-5c61c31d420c","arxiv_id":"2607.22679","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DOSA's tree-guided self-regressive chunking improves multi-page document hierarchy parsing by up to 4 F1 and 19 TEDS over prior methods on DocHieNet.","lead":"This paper introduces DOSA, a model that reads long documents chunk by chunk and uses a partially built semantic tree of the document to decide what context to keep when reading the next chunk. The system reports higher accuracy on document-structure benchmarks, including a 4-point F1 and 19-point tree-similarity gain on the hardest multi-page benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tree-guided context is applied only at inference, never during training, so the mechanism driving the central claim may be untested.","rationale":"The reader's weakest assumption (ground-truth reading order) is explicitly disclosed and the paper still shows DOSA beating DHFormer without it (+1.91 F1, +7.48 TEDS). The more damaging issue is that the published results and ablations never train with the tree-guided context that defines the method. The paper describes DOSA as a self-regressive framework where the partial semantic tree conditions subsequent prediction, but the training protocol in Appendix A.4 contradicts this: each chunk is trained in isolation, so the model never learns to exploit the context that §4.2 and §6.6 claim is responsible for the gains. The position-encoding normalization (relative page numbers per chunk) further means that when context from earlier chunks is appended at inference, the page-coordinate signals are normalized on different scales, so the model cannot reliably infer the true page distance. This is not a minor implementation detail; it threatens the internal consistency of the claimed contribution. Because the empirical numbers may still be reproducible even with this mismatch, the conditionality of the verdict is unchanged, but the central mechanistic claim requires re-training and re-evaluation to be supported.","tokens_in":813,"tokens_out":872,"duration_ms":122688,"concrete_test":"Retrain DOSA using ground-truth partial trees to provide tree-guided context during training (i.e., each training example is a chunk concatenated with its rightmost-branch context, with context objects labeled as possible parents/predecessors). Then re-run the §6.6 context-selection ablation and the DocHieNet comparison. If retrained performance is materially better, the current single-stage training under-sells the mechanism; if materially worse, the reported inference-only context is not a learned behavior and Table 3's gains are confounded. A supplementary check: on DocHieNet test documents, measure the fraction of correct parent predictions that come from context objects; if it is near zero, the tree-guided context is not being used as claimed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central mechanism — tree-guided context selection — is only applied at inference, never in training. Appendix A.4 states \"During training, each chunk is treated as an individual training unit,\" while §4.2 says selected context objects are merged \"for inference.\" Consequently, the parent/predecessor classifier (Eq. 3) is trained with a candidate set limited to N in-chunk objects, but at evaluation the softmax runs over N + window_size candidates that include rightmost-branch context objects. This is a train/test shift the paper never addresses. It is compounded by §5.2's position encoding: page_no is normalized by \"the total page count of the chunk,\" so context objects from a previous chunk carry page indices normalized to their original chunk, which are numerically incompatible with the current chunk's normalization when concatenated. The model therefore cannot correctly perceive absolute page distance between context and current objects. The claimed benefit of tree-guided context (Table 3: +2.36 F1, +17.38 TEDS over No Context) might instead reflect longer input sequences, attention dilution, or coincidental alignment. The self-regressive aspect of the framework is thus not actually trained, and the published numbers do not validate the stated mechanism.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DOSA, a chunk-based framework for document structure analysis that incrementally constructs a semantic tree and uses the tree's rightmost branches as context for processing later chunks. The central claim is that this tree-guided, self-regressive context selection enables efficient modeling of long-range cross-page dependencies, yielding large gains over prior work on DocHieNet (about 4 F1 points and 20 TEDS points over DHFormer) and outperforming LLM baselines. The paper includes a formal proof of the rightmost-branch sufficiency invariant, ablation studies isolating the contribution of tree-guided context, feature embedding ablations, and a comparison with LLMs.","tokens_in":12371,"tokens_out":4251,"duration_ms":46774,"significance":"If the reported results hold, DOSA offers a practical alternative to full-document attention for document hierarchy parsing, with a compact 33M-parameter model that outperforms both specialized document models and large multimodal LLMs. The paper ships a formal proof of the context-selection invariant and a clear ablation structure, and it evaluates on five benchmarks including the challenging DocHieNet. These are real strengths. However, the headline numbers rest on a single training run with an oracle reading order, and the main mechanism—tree-guided context—is only applied at inference, never during training, which raises a train/test shift that the paper does not address. The significance is therefore conditional on closing that gap and providing variance estimates.","major_comments":[{"comment":"The central mechanism—tree-guided context selection—is applied only at inference. Appendix A.4 states that during training each chunk is treated as an individual unit, while §4.2 merges context objects from previous chunks only for inference. Consequently, the relation classifier in Eq. (3) is trained with a softmax over the N in-chunk candidates but evaluated over N+window_size candidates that include context objects. The model has never been trained to discriminate or select among out-of-chunk candidates, so the gains in Table 3 may reflect a different task at inference rather than the learned mechanism. Please either train with context (e.g., by sampling prior chunks as context) or provide analysis demonstrating that the classifier generalizes to out-of-chunk candidates (e.g., per-candidate-type accuracy, or a comparison with a model trained with context).","section":"§4.2, §5.3, Appendix A.4"},{"comment":"The position encoding is normalized per chunk: page_no is relative to the chunk and is normalized by the total page count of that chunk. When context objects from a previous chunk are concatenated with the current chunk, their normalized page coordinates are on different scales (different denominators and reset page indices). The model therefore cannot correctly perceive absolute page distance between context and current objects, undermining the claimed cross-page dependency modeling. Please use absolute page numbers or a consistent normalization (e.g., by document length or a fixed maximum) and justify the choice.","section":"§5.2"},{"comment":"All reported results come from a single training run with a fixed random seed. The headline claim—a gain of ~4 F1 and ~20 TEDS over DHFormer on DocHieNet—has no variance estimate. Given the sensitivity to window size shown in Table 7 and the modest size of some ablations, a single run is insufficient to establish that the improvements are not partly due to seed noise. Please report mean and standard deviation over at least three seeds for the main results and key ablations.","section":"Appendix A.3 and Tables 3–4"},{"comment":"The window size reporting is inconsistent: §6.1 states the context window size is 8, while Appendix A.3 says DocHieNet uses 16 and Table 3's sequential-window strategy uses 16. This makes it difficult to interpret the main results and ablations. Please state clearly which window size is used for each dataset and each reported table, and reconcile the main-text default with the appendix.","section":"§6.1, §6.6, Appendix A.3, A.6"},{"comment":"The headline comparison provides DOSA with ground-truth reading order at inference. Table 4 shows that with inferred ordering the TEDS advantage over DHFormer shrinks from ~19 to ~7.5 points. This is disclosed in the text, but the abstract and conclusion emphasize the 'nearly 20 TEDS' gain without this qualification. Since reading-order prediction is part of the task, the practical advantage is materially smaller; please clarify in the abstract and conclusion that the headline margin uses the oracle ordering.","section":"§6.4, Table 4"}],"minor_comments":[{"comment":"The arg max over a softmax expression is visually awkward; consider defining the probability p_ij and then stating the prediction as arg max_j p_ij.","section":"Eq. (3)"},{"comment":"The normalization of the position vector is described in prose; please give the exact formula (e.g., page_no / P_c, x1/W, y1/H) to avoid ambiguity.","section":"§5.2"},{"comment":"The text says context is selected based on model predictions to avoid oracle information, but §6.4 says ground-truth ordering is used at inference. Clarify which parts of the input are oracle (ordering) and which are predicted (tree structure).","section":"Appendix A.4"},{"comment":"Some references lack venue or publication details (e.g., Longformer, LongLoRA, RAG-QA Arena). Please ensure all entries are complete.","section":"References"},{"comment":"The LLM baseline versions (Gemini-2.5-Pro, GPT-5.2) are not dated; given rapid model evolution, specify the exact access date and version to allow reproducibility.","section":"Table 2"},{"comment":"The sequential-window baseline retrieves the last 16 objects, but the default window size in §6.1 is 8. For a controlled comparison, all context strategies should use the same window size.","section":"§6.6, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the inference-only application of tree-guided context is valid and load-bearing. The paper's mechanism claim is not fully supported by the current experiments because the model is never trained with context. This is fixable with additional experiments or analysis, so I am not recommending rejection, but the revision should directly address the train/test shift, the position-normalization incompatibility, and the lack of variance estimates. The single-run reporting and window-size inconsistency further weaken the empirical case as currently written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is worth your time: DOSA parses long documents chunk-by-chunk and uses the partially built semantic tree—specifically its rightmost branch—as structural context for later chunks. That partial-tree-as-context mechanism is new, and the rightmost-branch invariant is a nice formal observation. The model itself is straightforward: multimodal feature fusion, a transformer encoder, two relation heads. It beats DHFormer by ~4 F1 and ~19 TEDS on DocHieNet, and also beats the GPT-5.2 / Gemini-2.5-Pro baselines. The ablations are mostly honest—they show reading-order dependence and per-feature contributions, and the appendix includes an error-accumulation study and a qualitative window-size analysis. The paper is clearly written and the authors acknowledge limitations like error propagation and missing citation relations.\n\nThe soft spots are real, and the stress-test note lands. The tree-guided context is applied only at inference, never during training. Appendix A.4 says each chunk is treated as an individual training unit, while §4.2 merges selected context objects 'for inference.' The relation classifier in §5.3 is trained with a candidate set limited to N in-chunk objects, but at evaluation it runs softmax over N plus window-size candidates, which include rightmost-branch objects. That is a train/test shift the paper never discusses. Compounding it, §5.2 normalizes page_no by the total page count of the chunk, so context objects from a previous chunk carry page indices normalized to their source chunk, which are numerically incompatible with the current chunk's normalization when concatenated. The model cannot correctly perceive absolute page distance between context and current objects. The claimed benefit of tree-guided context (Table 3: +2.36 F1, +17.38 TEDS) could partly come from longer sequences or a more permissive candidate set rather than the stated mechanism. There is also no variance reporting—all numbers are from a single run with a fixed seed—and the window size is 8 in §6.1 but 16 in A.3. The headline comparison uses ground-truth reading order; with inferred order TEDS drops 11.69 points, still beating DHFormer but a much smaller margin.\n\nThis paper is for document-AI researchers working on long-document structure parsing or RAG chunking. The idea is interesting and the engineering is clean, but the central mechanism needs to be trained with merged context and the position encoding needs to be fixed before the claims can be trusted. It deserves a serious referee because the approach is novel and the issues are addressable. I would not cite the headline numbers yet, but I would ask the authors to retrain with context and re-run the ablations. If they do that, the paper could be solid.","headline":"Tree-guided context is a genuinely clever idea, but it is only wired into inference, so the central claim about what drives the gains is not actually validated.","tokens_in":729,"tokens_out":1410,"would_cite":false,"duration_ms":51156,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A chunk-by-chunk, tree-guided framework reconstructs multi-page document hierarchies more accurately and cheaply than full-document attention or long-context LLMs.","keywords":["document structure analysis","semantic tree","page-object relations","long-document modeling","tree-guided context selection","self-regressive decoding","multimodal feature fusion","hierarchy reconstruction"],"falsifier":"Find a single ground-truth document where an object in a later chunk has a parent that is already present in the partial tree but does not lie on the rightmost branch; such a counterexample would refute the sufficiency proof in Appendix A.1, and the framework's advertised gains would fail on layouts that violate the invariant's depth-first reading-order assumption.","tokens_in":11988,"feed_emoji":"🌳","tokens_out":11044,"duration_ms":109712,"temperature":0.7,"pith_summary":"Long multi-page documents encode structure through relations among page objects—headers, tables, paragraphs—and the paper's goal is to reconstruct the full semantic tree without paying the cost of whole-document attention. The proposed method, DOSA, reads a document chunk-by-chunk, predicts parent and predecessor relations within each chunk, and grows a semantic tree incrementally. Its key claim is that the rightmost branch of the partial tree is sufficient context for the next chunk, so attention stays small while global coherence is preserved. On the hardest benchmark this yields roughly 4 F1 and 19 TEDS (tree-edit-distance) points over the prior state of the art, and it beats long-context LLMs while training on a single 24 GB GPU. If correct, the paper shows that structural context selection—not longer windows or bigger models—is what makes hierarchy reconstruction scale.","feed_headline":"Tree-guided chunking beats full-document attention on page hierarchies","feed_subtitle":"A 33M-parameter model improves multi-page hierarchy F1 by 4 points and tree-edit-distance similarity by ~20 over prior state of the art.","key_machinery":"The rightmost-branch sufficiency invariant—proved in the appendix—states that a later chunk's object can only have a parent on the rightmost branch of the partial tree. This invariant is the engine of the framework: it justifies feeding only a fixed-size window of structural context into the transformer, avoiding full-document attention and attention dilution. A branch-expanding soft window adds robustness by pulling in objects from the second- and third-rightmost branches when the window is not filled, mitigating the error propagation inherent in self-regressive decoding.","core_discovery":"The central discovery is the rightmost-branch sufficiency invariant: because a depth-first traversal of the partial semantic tree matches document reading order, any page object in a later chunk that connects to an already-seen parent must attach to the tree's rightmost branch. This turns long-range cross-page dependency modeling into a sequence of bounded-size, in-chunk predictions, with a soft window across neighboring branches absorbing earlier errors. The in-chunk model fuses visual, semantic, category, and sizing features per object and predicts parent and predecessor jointly; the tree is updated online and guides the next chunk. Across five benchmarks, DOSA consistently improves over p","pith_inferences":["The paper's error-accumulation analysis suggests a confidence-aware or dynamically sized context window could recover some of the ~12 TEDS points lost when reading order is inferred rather than given—an extension the authors do not pursue.","The rightmost-branch invariant is specific to tree-structured relations; extending DOSA to citation or reference graphs would require generalizing 'rightmost branch' to a frontier set, something the paper's limitations section leaves open.","Real deployments without a ground-truth reading order would need a reliable ordering predictor or joint training that up-weights ordering loss, since the ablations show ordering errors damage tree similarity far more than F1 (11.69 vs 1.07 points)."],"forward_implications":["The same chunked, tree-guided decoding scheme applies to any task whose ground truth is a rooted ordered tree with a linear reading order—outlines, table-of-contents structures, or XML/JSON document hierarchies.","A roughly 20-point TEDS gain on the hardest multi-page benchmark means the parsed tree can be used to set semantically meaningful chunk boundaries for retrieval-augmented generation and document retrieval, improving coherence downstream.","At around 33M parameters, about a second per document on a single L4 GPU, and with per-chunk latency amortizable across parallel documents, the framework is a practical production option for long-document understanding.","Because DOSA beats frontier long-context LLMs on structured reconstruction, the paper implies that explicit structural decoding outperforms generative prompting for hierarchy recovery, pointing future systems toward structure-aware inference."],"fun_headline_variants":["Rightmost-branch sufficiency: long-range document relations made local","Chunk-wise tree building lifts multi-page hierarchy F1 by 4","DOSA: self-regressive tree guidance for document structure","Self-regressive chunk predictions build document semantic trees","Tree-guided chunking beats full-document attention"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline comparison assumes ground-truth reading order is available at inference to define chunk boundaries and place predicted nodes; when the model infers order itself, the TEDS advantage on the hardest benchmark drops from roughly 19 to 7 points.","fun_headline_variants_meta":{"raw":{"variants":["Rightmost-branch sufficiency: long-range document relations made local","Chunk-wise tree building lifts multi-page hierarchy F1 by 4","DOSA: self-regressive tree guidance for document structure","Self-regressive chunk predictions build document semantic trees","Tree-guided chunking beats full-document attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00103,"raw_usage":{"total_tokens":4157,"prompt_tokens":708,"completion_tokens":3449,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":452,"completion_tokens_details":{"reasoning_tokens":3367}},"tokens_in":452,"tokens_out":3449,"duration_ms":23043,"temperature":1.0,"reasoning_tokens":3367,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:22:02.405366+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a single ground-truth document where an object in a later chunk has a parent that is already present in the partial tree but does not lie on the rightmost branch; such a counterexample would refute the sufficiency proof in Appendix A.1, and the framework's advertised gains would fail on layouts that violate the invariant's depth-first reading-order assumption.","supporting_citations":[],"review_version":1}