REVIEW 4 major objections 7 minor 13 references
Orthogonal Hierarchical Decomposition for Structure-Aware Table Understanding with Large Language Models
T0 review · 4 major / 7 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read This paper argues that complex tables should be represented as two orthogonal hierarchical trees—one over columns, one over rows—rather than as flattened text or normalized grids, and that this representation yields large, consistent gains
desk verdict OHD's orthogonal row/column decomposition is a fresh take on table representation, but the 'consistent outperformance' claim is contradicted by its own HiTab results and the LLM-dependent steps are insufficiently specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Orthogonal Tree Induction (OTI), which factorizes a table into a column tree T_col and a row tree T_row via the Recent Spatial-Semantic Subsumption relation: an edge between header h_i (parent) and h_j (child) requires both spatial containment (h_j's span inside h_i's, and h_i above h_j) and an LLM-evaluated semantic predicate P_semantic(h_i, h_j) confirming a logical subsumption. A conflict set detects headers that are spatially nested but semantically parallel, and a dynamic anchoring rule assigns each data cell to the correct header based on row boundaries. The dual-pathway association protocol then reconstructs each cell's context as S_d = Φ_pre(d) ⊕ (Φ_attr(d) ⇒ d), and a final LLM sema
What would settle it
Build a benchmark of complex tables with annotated ground-truth logical hierarchies (which header subsumes which, and which header owns each data cell), run OTI to reconstruct trees, and measure tree edit distance or subsumption accuracy against the gold hierarchy; if the trees frequently disagree with the gold relations, the claimed mechanism cannot be the source of the reported QA gains.
Extended reading notes
Core claim
The central claim is that decomposing an irregular table into two orthogonal trees—one for column headers and one for row headers—preserves the multi-level semantic lineage of every data cell, and that feeding this representation to an LLM outperforms flat serialization, schema-aligned programmatic modeling, and geometry-only topology reconstruction. The decomposition is guided by a semantic-spatial synergy principle: an edge between headers is induced only when spatial containment and LLM-verified semantic subsumption agree, and data cells are anchored to leaf headers through a conflict-resolution rule that corrects misleading spatial layouts. On AITQA the full framework reaches 69.34 exact
Load-bearing premise
The framework assumes the LLM's semantic judgments about header subsumption and its final arbitration are reliable enough that the induced trees capture the true logical hierarchy rather than reproducing the same ambiguities the method is meant to remove.
Editorial extensions
If this is right
- If OHD is correct, flat linearization is a structural bottleneck, not just a formatting choice: information needed for multi-hop table reasoning is lost before the LLM sees the table.
- Table understanding can benefit from a symbolic pre-pass that builds orthogonal structures using LLM judgments, suggesting a hybrid neuro-symbolic pipeline rather than pure end-to-end prompting.
- Because the row and column trees are induced symmetrically, the same machinery applies to tables where the primary hierarchy is in rows or in columns, or both.
- The large ablation gap between lineage-based representations and Markdown/HTML suggests that any structure-preserving serialization that keeps ancestor chains for every cell could yield similar gains.
Reading between the lines
- The paper does not report the prompts or failure rates of the LLM semantic predicate and the final arbitrator; a natural extension is to benchmark these calls against gold subsumption labels and measure how tree error propagates to QA accuracy.
- Since the framework depends on pre-identified cell roles (column header, row header, data), one could test whether the method degrades gracefully when role classification is noisy or must itself be inferred.
- The orthogonal decomposition suggests a formal analogy to Cartesian products of partial orders: if the table's logical structure factorizes, then a product-of-trees representation could be the canonical input format for LLMs over semi-structured documents.
- The curated HiTab subset is capped at 50×50 cells; extending the decomposition to page-scale tables with thousands of cells is an open scaling question the paper leaves implicit.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Orthogonal Hierarchical Decomposition (OHD), a framework for structure-aware complex table understanding with LLMs. OHD first induces a column tree and a row tree via Orthogonal Tree Induction (OTI), which uses spatial containment plus an LLM-based semantic predicate (Eq. 2). It then reconstructs each data cell's semantic lineage through a dual-pathway association protocol and uses an LLM arbitrator (Section 3.4) to merge the two orthogonal views into a single prompt. The framework is evaluated on AITQA and HiTab with Qwen2-72B and TableLLaMA-7B backbones, reporting Exact Match (EM) and an LLM-based holistic score, plus ablations. The abstract and Section 5.4 claim that OHD consistently outperforms existing representation paradigms across metrics.
Significance. If the empirical claims hold, OHD offers a genuinely different approach to table representation: instead of linearizing or normalizing the grid, it factorizes the table into orthogonal hierarchical trees and uses semantic verification to correct layout ambiguities. This is a plausible and potentially impactful direction for irregular tables with multi-level headers. The paper also provides a full ablation breakdown and, in Appendix B, the exact LLM-evaluation prompt, which are useful for reproducibility. The weaknesses are concentrated in the empirical support for the headline claim: the results contain an internal reversal, the HiTab subset is not adequately justified, and the core LLM calls are underspecified. The idea is worth pursuing, but the paper in its current form overstates its evidence.
major comments (4)
- [Abstract; §5.4; Table 1] The central claim that OHD 'consistently outperforms existing representation paradigms' is contradicted by Table 1 on the full HiTab benchmark: Ours (TableLLaMA-7B) achieves EM 63.62 vs. TableLLaMA-7B 64.71, and LLM Eval Avg. 65.97 vs. 66.99. This is not a marginal difference and it is in the same backbone configuration. The claim should be revised to reflect the settings where OHD is better, or the authors must explain why this specific comparison should be discounted. As written, the headline is not supported by the paper's own data.
- [§5.1; Table 1; Table 5] The 'HiTab Subset' is introduced by a 50×50 cell-size cutoff, but the paper reports no information about how many tables are retained, how the distribution of questions changes, or whether the subset is easier/different from the full HiTab in any systematic way. Since the subset results are more favorable to OHD, the selection rule could bias the comparison. The authors should report the filtering statistics, justify the cutoff, and confirm that all baselines are evaluated on exactly the same filtered subset. Without this, the 'consistent' outperformance claim rests on an incomparable benchmark variant.
- [§3.2, Eq. (2); §3.4] The load-bearing components — the LLM semantic predicate P_semantic and the final LLM arbitration prompt I — are not specified. The paper gives no prompt text, model version, decoding parameters, or failure-rate analysis for these calls. Since OTI's tree construction depends on P_semantic's correctness, and the final representation depends on the arbitrator, the method is not reproducible and its core mechanism is a black box. Providing these prompts and an error analysis (e.g., on a sample of header pairs) is essential to support the claim that OHD removes structural ambiguity rather than encoding the same ambiguities through LLM judgment.
- [§5.4; Table 2] No error bars, significance tests, or multi-seed results are reported for any experiment. This is especially problematic given the reversed result on HiTab and the ablation asymmetry: w/o TP(Tcol) drops 20.19 EM on AITQA but only 3.20 on HiTab, while w/o TP(Trow) drops 8.60 and 3.60 respectively. The conclusion that the column tree is 'the primary bottleneck' is drawn from a single dataset without variance estimates. The authors should provide at least three seeds (or bootstrap confidence intervals) and use appropriate significance testing for the main comparisons and key ablations.
minor comments (7)
- [§4, Question 1] Typo: 'municipalites' should be 'municipalities'.
- [§3.3, Eqs. (5) and (7)] Equations (5) and (7) are identical; renumber or differentiate them.
- [Algorithm 1, line 10] The function Arbitrate(L_d, P_sem) is called but never defined in the text; please define it or connect it to Eq. (4).
- [Table 6] The column header 'GPT' should be 'GPT-4' for consistency with Tables 3–5.
- [§5.3] 'three diverse backends' — these are evaluator models, not backends; consider rephrasing to 'three diverse LLM evaluators'.
- [Appendix B] The sentence 'The specific prompt used for this evaluation is detailed below' appears twice in succession; remove the duplicate.
- [General] No code or data-processing pipeline is released; a public repository would materially aid verifiability, especially given the custom tree construction and LLM calls.
Circularity Check
No significant circularity: the central claim is an empirical comparison on external benchmarks, with EM as an objective metric; the LLM calls are part of the method, not a fitted input renamed as a prediction.
full rationale
The paper's core claim is that the OHD representation improves complex-table QA, supported by Exact Match and LLM-evaluator scores on two external benchmarks, AITQA and HiTab. No parameter is fit to the test sets, and EM is not definitionally tied to the method's construction. The LLM-based semantic predicate in Eq. (2) and the final LLM arbitrator in Section 3.4 are used to build the representation, not to score it; the evaluator prompt in Appendix B is applied uniformly to baselines and ablations. The only mild concern is that Qwen2-72B appears both as a backbone and as one of three LLM evaluators, but the paper reports EM as the primary objective metric and shows cross-evaluator agreement across three models, and no equation or fitted value makes the reported gains true by construction. The ablation results, including the asymmetric drop for w/o TP(Tcol), are empirical observations rather than analytic consequences. The paper is therefore not circular; concerns about unspecified prompts, missing error bars, and the HiTab subset curation are correctness/reproducibility risks, not circularity.
Assumptions & free parameters
free parameters (3)
- HiTab subset size cutoff 50×50 =
50
- LLM semantic predicate configuration =
not specified
- Arbitration prompt I =
not specified
assumptions (5)
- domain assumption Any complex table can be decomposed into two orthogonal trees that faithfully represent all semantic dependencies.
- domain assumption Spatial containment plus semantic subsumption is sufficient to determine parent–child relations among headers.
- domain assumption Each data cell has a unique nearest parent header; footnotes and auxiliary remarks never belong under data cells.
- ad hoc to paper The LLM semantic predicate and the final arbitrator are accurate enough not to introduce systematic errors.
- domain assumption The chosen LLM evaluators (Qwen2-72B, DeepSeek-v3, GPT-4) with the stated prompt provide a valid semantic correctness measure.
invented entities (1)
-
Column tree (Tcol) and row tree (Trow) representation
Cite this review
Pith. "Pith review of Orthogonal Hierarchical Decomposition for Structure-Aware Table Understanding with Large Language Models." pith.science (2026). https://pith.science/paper/ZIL3BVPW
@misc{pith2026260201969,
author = {Pith},
title = {Pith review of: Orthogonal Hierarchical Decomposition for Structure-Aware Table Understanding with Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZIL3BVPW}},
note = {Machine review of arXiv:2602.01969}
}
read the original abstract
Complex tables with multi-level headers, merged cells and heterogeneous layouts pose persistent challenges for LLMs in both understanding and reasoning. Existing approaches typically rely on table linearization or normalized grid modeling. However, these representations struggle to explicitly capture hierarchical structures and cross-dimensional dependencies, which can lead to misalignment between structural semantics and textual representations for non-standard tables. To address this issue, we propose an Orthogonal Hierarchical Decomposition (OHD) framework that constructs structure-preserving input representations of complex tables for LLMs. OHD introduces an Orthogonal Tree Induction (OTI) method based on spatial--semantic co-constraints, which decomposes irregular tables into a column tree and a row tree to capture vertical and horizontal hierarchical dependencies, respectively. Building on this representation, we design a dual-pathway association protocol to symmetrically reconstruct semantic lineage of each cell, and incorporate an LLM as a semantic arbitrator to align multi-level semantic information. We evaluate OHD framework on two complex table question answering benchmarks, AITQA and HiTab. Experimental results show that OHD consistently outperforms existing representation paradigms across multiple evaluation metrics.
Figures
Reference graph
Works this paper leans on
-
[1]
Baktash, J. A. and Dawodi, M. Gpt-4: A review on advance- ments and opportunities in natural language processing. arXiv preprint arXiv:2305.03195,
-
[6]
Crafting papers on machine learning
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.),Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207–1216, Stan- ford, CA,
2000
-
[7]
Deepseek- v3 technical report.arXiv preprint arXiv:2412.19437, 2024a
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek- v3 technical report.arXiv preprint arXiv:2412.19437, 2024a. Liu, T., Wang, F., and Chen, M. Rethinking tabular data un- derstanding with large language models. InProceedings of the 2024 Conference of the North American Chapter of the Association...
arXiv 2024
-
[8]
Team, Q. et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2(3),
-
[10]
Zhang, Z., Gao, Y ., and Lou, J.-G. e5: Zero-shot hierar- chical table analysis using augmented llms via explain, extract, execute, exhibit and extrapolate. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 1244–1258, 2024b. Zhao, B....
arXiv 2024
-
[11]
Zhao, Y ., Li, Y ., Li, C., and Zhang, R. Multihiertt: Numeri- cal reasoning over multi hierarchical tabular and textual data.arXiv preprint arXiv:2206.01347,
-
[12]
10 Orthogonal Hierarchical Decomposition for Structure-Aware Table Understanding with Large Language Models A. Extended Related Work The evolution of Table Question Answering (Table QA) has shifted from simple grid-based parsing toward the structural modeling ofcomplex heterogeneous tables(Zheng et al., 2023; Fang et al., 2024). Such tables, as exemplifie...
2023
-
[13]
However, these methods rely on anormalization bias, assuming that tables can be perfectly mapped to a flat relational header-row format
represents tables as structured objects, such as SQL tables or DataFrames, conforming to canonical relational schemas. However, these methods rely on anormalization bias, assuming that tables can be perfectly mapped to a flat relational header-row format. In complex tables in the real-world, unconventional layouts—such asirregularly merged headers, embedd...
2021
Show all 13 references
-
[2020]
X., and Wen, J.-R
Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W. X., and Wen, J.-R. Structgpt: A general framework for large language model to reason over structured data. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9237–9251,
2023
-
[2022]
A., Zhang, J., Hu, Z., Qi, Y ., Nick- leach, S., Socolinsky, D., Sengamedu, S., and Faloutsos, C
Fang, X., Xu, W., Tan, F. A., Zhang, J., Hu, Z., Qi, Y ., Nick- leach, S., Socolinsky, D., Sengamedu, S., and Faloutsos, C. Large language models (llms) on tabular data: Pre- diction, generation, and understanding–a survey.arXiv preprint arXiv:2402.17944,
-
[2023]
Ait-qa: Question answering dataset over complex tables in the airline industry
Katsis, Y ., Chemmengath, S., Kumar, V ., Bharadwaj, S., Canim, M., Glass, M., Gliozzo, A., Pan, F., Sen, J., Sankaranarayanan, K., et al. Ait-qa: Question answering dataset over complex tables in the airline industry. InPro- ceedings of the 2022 Conference of the North Americ...
2022
-
[2024]
Tablellama: Towards open large generalist models for tables
Zhang, T., Yue, X., Li, Y ., and Sun, H. Tablellama: Towards open large generalist models for tables. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 6...
2024
-
[2025]
Large language models are few (1)-shot table rea- soners
Chen, W. Large language models are few (1)-shot table rea- soners. InFindings of the association for computational linguistics: EACL 2023, pp. 1120–1130,
2023
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.