Pith. sign in

REVIEW 3 major objections 4 minor 34 references

Synthetic Data Augmentation for Table Detection: Re-evaluating TableNet's Performance with Automatically Generated Document Images

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A LaTeX pipeline that renders synthetic document pages can train TableNet to a 4.04% pixel-wise XOR error on synthetic test pages and a 9.18% error on the real Marmot benchmark.

desk verdict A useful synthetic page generator and a defensible resolution study, but the 'augmentation' claim in the title and abstract isn't tested. read the letter →

arxiv 2506.14583 v1 pith:TISDN3LE submitted 2025-06-17 cs.CV cs.AI

Krishna Sahukara , Zineddine Bettouche , Andreas Fischer This is my paper · ORCID
classification cs.CVcs.AI
keywords tabledetectionsyntheticdatagenerationLaTeXdocumentrenderingsemanticsegmentationNetMarmotbenchmarkXORerrorrateinputresolution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces an automated LaTeX-based pipeline that renders synthetic two-column document pages containing diverse tables, with ground-truth masks generated automatically at render time. The authors train TableNet on this corpus and report pixel-wise XOR errors of 4.04% on their synthetic test set and 9.18% on the real Marmot benchmark, both at 256x256 resolution. They present the corpus as a way to cut manual annotation effort and to enable a controlled comparison of input resolutions. The paper's findings are that higher resolution does not universally help and that synthetic-to-synthetic evaluation can expose architecture-dependent trends without domain noise.

What carries the argument

The load-bearing mechanism is the generation pipeline: a Python script drives LaTeX, taking four parameters (number of rows, number of columns, datatypes, and table style) and producing a rasterized page image plus an aligned binary ground-truth mask. Tables are placed either alone or surrounded by two-column text, forcing a detector to reason about the page layout rather than classify isolated table crops. The trained model is TableNet, an encoder-decoder semantic-segmentation network with a shared VGG-19 encoder and separate table and column decoder branches, and the evaluation metric is the pixel-wise XOR error rate, which counts every misclassified pixel.

What would settle it

Run the same TableNet training on the synthetic corpus and score predictions on Marmot with region-level metrics such as IoU or mAP; if the model shows low XOR error but high false-positive or false-negative rates at the bounding-box level, the central transfer claim fails under a different metric, whereas strong region-level scores would support it.

Watch

Extended reading notes

Core claim

The paper's central claim is that a fully automated LaTeX-based pipeline can generate page-level synthetic documents with diverse table styles and exact ground-truth masks, and that this corpus is sufficient to train TableNet to competitive table-detection performance. The headline numbers are a 4.04% pixel-wise XOR error on the synthetic test set at 256x256 input resolution, 4.33% at 1024x1024, and a best 9.18% on the real Marmot benchmark, also at 256x256. From these experiments the authors conclude that higher resolution does not universally improve performance and that synthetic-to-synthetic evaluation can reveal architectural behaviour while holding domain noise fixed.

Load-bearing premise

The load-bearing premise is that LaTeX-rendered synthetic pages are realistic and diverse enough that performance measured on synthetic test pages and pixel-wise XOR error transfers to real-world document images and meaningful detection quality.

Editorial extensions

If this is right

  • Extending training on synthetic data continues to reduce XOR error, reaching 4.04% at 256x256 after 1540 epochs, while the same model on Marmot overfits after 500 epochs.
  • At 1024x1024, the model reaches 4.33% on synthetic pages but only 13.83% on Marmot, so resolution is not a reliable lever for real-world transfer.
  • Because ground-truth masks are generated rather than hand-labeled, the pipeline can cheaply produce large training sets for layout-analysis models.
  • The synthetic-to-synthetic protocol offers a sandbox for isolating the effect of data distribution from architecture when tuning detectors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the paper's 'augmentation' framing would mix synthetic and Marmot images in one training set; the reported experiments train and evaluate separately, so the added value of combining the corpora remains untested.
  • If pixel-wise XOR is as sensitive to thin cell borders as the paper argues, then ablating generation parameters (for example, removing borders or color) could reveal which visual cues TableNet actually relies on.
  • Comparing models trained on this page-level corpus against models trained on isolated-crop corpora such as PubTables-1M under the same evaluation protocol could quantify how much page context helps detection.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces an automated LaTeX-based pipeline that renders two-column scientific-style pages containing diverse table layouts with pixel-level ground-truth masks. The authors train the TableNet segmentation model separately on the synthetic corpus and on the Marmot benchmark at two input resolutions (256x256 and 1024x1024), evaluate with a pixel-wise XOR error rate, and report that the synthetic-trained model reaches 4.04% error on the synthetic test set at 256x256, while the Marmot-trained model reaches 9.18% at 256x256. They conclude that the synthetic corpus augments the real-world Marmot benchmark and that higher input resolution does not universally improve table detection.

Significance. If the central claims were fully demonstrated, the paper would make a useful practical contribution: a fully automated, publicly released synthetic document generation pipeline that reduces manual annotation effort and provides a controlled environment for studying architecture and resolution effects in table detection. The authors provide code and data, which is a genuine strength for reproducibility, and the motivation of moving from isolated table crops to full-page layouts is well placed. The significance is currently limited, however, because the augmentation claim is not tested by the experiments as designed, and the reported error rates are selected on the test set, which makes the quantitative claims optimistic lower bounds rather than reliable performance estimates.

major comments (3)
  1. [Abstract, Section I, Section V.A] The abstract and introduction state that the generated corpus 'augments' the real-world Marmot benchmark, but Section V.A describes only separately trained models: 'For each corpus—both the synthetic images introduced in this work and the original Marmot pages—we randomly split the data, using 90% of the images for training and reserving the remaining 10% for testing.' No experiment combines synthetic and real training data, such as training on Marmot plus synthetic images or pre-training on synthetic data and fine-tuning on Marmot. Consequently, the reported Marmot results (e.g., 9.18% at 256x256) come from a model trained only on Marmot and neither support nor refute an augmentation benefit. The paper should either add a combined-training experiment or reframe the contribution as a dataset and resolution study rather than a demonstrated augmentation result.
  2. [Table I, Section VI] The reported error rates are selected as the minimum over a grid of training epochs using the test set. Section VI states, for example, 'The lowest XOR error rate on the Marmot dataset was 9.18% at 500 epochs' and Table I shows the lowest error for each resolution/dataset combination in bold. Selecting the best epoch from test-set performance is a form of test-set leakage, because the test set is used for model selection. The text mentions EarlyStopping based on validation loss in Section V.A, but the reported numbers are not the validation-selected models. This makes all headline numbers (4.04%, 4.33%, 9.18%, 13.83%) optimistic and invalidates the comparative claims about resolutions and epoch counts. The authors should use a held-out validation set for epoch selection and report the corresponding test errors, or report the final model after training for a fixed number of epochs.
  3. [Section VI, Table I] The resolution comparison rests on single point estimates with no uncertainty quantification and with different batch sizes and epoch ranges for the two resolutions (e.g., batch size 256 for 256x256 versus 64 for 1024x1024). A difference such as 4.04% versus 4.33% on the synthetic test set, or 9.18% versus 13.83% on Marmot, is not established as meaningful without multiple runs and a common model-selection rule. The conclusion that 'higher resolution does not universally improve performance' should be supported by confidence intervals or repeated runs, and by a protocol that treats both resolutions symmetrically.
minor comments (4)
  1. [Section IV.B] The paper does not report any corpus statistics, such as the total number of generated document pages, the number of pages per table style, the number of tables per page, or the sizes of the training and test splits for both the synthetic corpus and Marmot. These numbers are necessary for reproducibility and for interpreting the reported error rates.
  2. [Section IV.D.1] The rationale for using XOR error over IoU, F1, or mAP is plausible, but the paper reports no standard region-level metric. Adding at least one conventional metric (e.g., IoU or F1) would allow readers to compare the method with prior table-detection work and would show that the XOR-based conclusions are not metric-specific.
  3. [Section V.A] The sentence 'To ensure the uniform input size of 256x256 pixels for images and 1024x1024 pixels for masks' is confusing, because the resolution study consistently compares 256x256 and 1024x1024 for both images and masks. Please clarify the intended image and mask dimensions for each configuration.
  4. [Section VI] The epoch-interval selection procedure is described only as 'progressive validation'; please specify the exact epoch grid, the EarlyStopping patience and threshold, and whether any validation split was used before the reported test evaluation, so that the experiments can be reproduced exactly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical dataset and resolution study with no fitted input renamed as prediction and no load-bearing self-citation.

full rationale

This paper contains no mathematical derivation chain that reduces to its own inputs; it is an empirical study of a synthetic data generation pipeline and TableNet's performance. The synthetic corpus is generated by an explicit LaTeX-based procedure with randomized parameters (rows, columns, datatypes, style), and the ground-truth masks are produced by the same pipeline that renders the pages, so the synthetic-to-synthetic evaluation is an internally consistent domain-matched test rather than a circular prediction. The XOR error metric in Eq. (1) is defined directly from the prediction mask and ground-truth mask and is not fitted to the results; no parameter is estimated from the test set and then reported as a prediction. The citation of the authors' prior work on process mining appears only as general inspiration and is not load-bearing for the reported results. The claim that the generated corpus 'augments' the Marmot benchmark is not demonstrated because no experiment combines synthetic and real training data, and the selection of the lowest test-set error across epochs is a statistical leakage/optimism concern, but both are correctness or framing issues, not circularity. The central reported numbers (4.04%, 4.33%, 9.18%, 13.83%) are empirical measurements from a reproducible pipeline with publicly available code, so the derivation is self-contained and no circular step can be exhibited.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted to the reported error rates; the paper is an empirical study. The assumptions listed are domain assumptions about transferability and realism, which are not directly tested. No new entities are introduced.

assumptions (3)
  • domain assumption VGG-19 pretrained on ImageNet provides useful features for table segmentation
    The TableNet implementation uses the VGG-19 backbone with ImageNet weights (Section II-C and Section V-A); this transfer-learning assumption is standard but not tested here.
  • domain assumption LaTeX-rendered synthetic pages are visually realistic enough to serve as a proxy for scanned document images
    The entire synthetic-to-real transfer conclusion depends on the realism of the generated pages (Section IV-B). No quantitative realism check is provided.
  • domain assumption Poisson noise and random perspective transforms simulate scanning artifacts
    Section IV-C lists these augmentations without validation that they match real scanner noise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Synthetic Data Augmentation for Table Detection: Re-evaluating TableNet's Performance with Automatically Generated Document Images." pith.science (2026). https://pith.science/paper/TISDN3LE

@misc{pith2026250614583,
  author       = {Pith},
  title        = {Pith review of: Synthetic Data Augmentation for Table Detection: Re-evaluating TableNet's Performance with Automatically Generated Document Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TISDN3LE}},
  note         = {Machine review of arXiv:2506.14583}
}
read the original abstract

Document pages captured by smartphones or scanners often contain tables, yet manual extraction is slow and error-prone. We introduce an automated LaTeX-based pipeline that synthesizes realistic two-column pages with visually diverse table layouts and aligned ground-truth masks. The generated corpus augments the real-world Marmot benchmark and enables a systematic resolution study of TableNet. Training TableNet on our synthetic data achieves a pixel-wise XOR error of 4.04% on our synthetic test set with a 256x256 input resolution, and 4.33% with 1024x1024. The best performance on the Marmot benchmark is 9.18% (at 256x256), while cutting manual annotation effort through automation.

Figures

Figures reproduced from arXiv: 2506.14583 by the authors.

Figure 3
Figure 3. Example Result after Performing Bitwise XOR. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. Types of Generated Image: (a) with text; (b) without text. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Sample Image from Marmot Dataset and Predicted Mask with [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Sample Image from Marmot Dataset and Predicted Mask with [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 28 canonical work pages

  1. [1]

    A survey on image data augmen- tation for deep learning,

    C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmen- tation for deep learning,”Journal of Big Data, vol. 6, no. 1, p. 60, 2019

    work page 2019

  2. [2]

    A survey on data augmentation for text classification,

    M. Bayer, M. Kaufhold, and C. Reuter, “A survey on data augmentation for text classification,”ACM Computing Surveys, vol. 55, no. 1, pp. 1–35, 2022

    work page 2022

  3. [3]

    Synthesizing search engine queries from task-oriented user profiles,

    T. R. Anjali, Z. Bettouche, and A. Fischer, “Synthesizing search engine queries from task-oriented user profiles,”Expert Systems with Applica- tions, vol. 213, p. 118991, 2023

    work page 2023

  4. [4]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”arXiv preprint arXiv:1409.1556, 2014

    arXiv 2014

  5. [5]

    A review of semantic segmentation using deep neural networks,

    Y . Guo, Y . Liu, T. Georgiou, and M. S. Lew, “A review of semantic segmentation using deep neural networks,”International Journal of Multimedia Information Retrieval, vol. 7, pp. 87–93, 2018

    work page 2018

  6. [6]

    Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images,

    S. Paliwal, V . D, R. Rahul, M. Sharma, and L. Vig, “Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images,” 2019

    work page 2019

  7. [7]

    Recognition of tables and forms,

    B. Cou ¨asnon and A. Lemaitre, “Recognition of tables and forms,” in Handbook of Document Image Processing and Recognition, D. Doer- mann and K. Tombre, Eds. Springer London, 2014, pp. 647–677

    work page 2014

  8. [8]

    A survey of table recognition: Models, observations, transformations, and inferences,

    R. Zanibbi, D. Blostein, and J. R. Cordy, “A survey of table recognition: Models, observations, transformations, and inferences,”International Journal on Document Analysis and Recognition (IJDAR), vol. 7, no. 1, pp. 1–16, 2004

    work page 2004

Show all 34 references
  1. [9]

    Design of an end-to-end method to extract information from tables,

    A. C. e. Silva, A. M. Jorge, and L. Torgo, “Design of an end-to-end method to extract information from tables,”International Journal of Document Analysis and Recognition (IJDAR), vol. 8, no. 2-3, pp. 144– 171, 2006

    2006

  2. [10]

    On methods and tools of table detection, extraction and annotation in pdf documents,

    S. Khusro, A. Latif, and I. Ullah, “On methods and tools of table detection, extraction and annotation in pdf documents,”Journal of Information Science, vol. 41, no. 1, pp. 41–57, 2015

    2015

  3. [11]

    Table-processing paradigms: A research survey,

    D. W. Embley, M. Hurst, D. Lopresti, and G. Nagy, “Table-processing paradigms: A research survey,”International Journal of Document Analysis and Recognition (IJDAR), vol. 8, no. 2-3, pp. 66–86, 2006

    2006

  4. [12]

    Tintin: A system for retrieval in text tables,

    P. Pyreddy and W. B. Croft, “Tintin: A system for retrieval in text tables,” inProceedings of the second ACM international conference on Digital libraries, 1997, pp. 193–200

    1997

  5. [13]

    Trainable table location in document images,

    F. Cesarini, S. Marinai, L. Sarti, and G. Soda, “Trainable table location in document images,” in2002 International Conference on Pattern Recognition, vol. 3, 2002, pp. 236–240

    2002

  6. [14]

    Learning to detect tables in scanned document images using line information,

    T. Kasar, P. Barlas, S. Adam, C. Chatelain, and T. Paquet, “Learning to detect tables in scanned document images using line information,” in ICDAR, 2013, pp. 1185–1189

    2013

  7. [15]

    Learning rich hidden markov models in document analysis: Table location,

    A. C. e. Silva, “Learning rich hidden markov models in document analysis: Table location,” in2009 10th International Conference on Document Analysis and Recognition, 2009, pp. 843–847

    2009

  8. [16]

    The t-recs table recognition and analysis system,

    T. Kieninger and A. Dengel, “The t-recs table recognition and analysis system,” inDocument Analysis Systems, ser. Lecture Notes in Computer Science, S.-W. Lee and Y . Nakano, Eds. Springer, 1999, pp. 255–270

    1999

  9. [17]

    Table structure understand- ing and its performance evaluation,

    Y . Wang, I. T. Phillips, and R. M. Haralick, “Table structure understand- ing and its performance evaluation,”Pattern Recognition, vol. 37, no. 7, pp. 1479–1497, 2004

    2004

  10. [18]

    Configurable table structure recognition in untagged pdf documents,

    A. Shigarov, A. Mikhailov, and A. Altaev, “Configurable table structure recognition in untagged pdf documents,” inProceedings of the 2016 ACM Symposium on Document Engineering, DocEng 2016, R. Sablatnig and T. Hassan, Eds. ACM, 2016, pp. 119–122

    2016

  11. [19]

    A table detection method for pdf documents based on convolutional neural networks,

    L. Hao, L. Gao, X. Yi, and Z. Tang, “A table detection method for pdf documents based on convolutional neural networks,” in12th IAPR Workshop on Document Analysis Systems (DAS) 2016. IEEE Computer Society, 2016, pp. 287–292

    2016

  12. [20]

    Table detection from document image using vertical arrangement of text blocks,

    D. N. Tran, T. A. Tran, A.-R. Oh, S.-H. Kim, and I.-S. Na, “Table detection from document image using vertical arrangement of text blocks,”International Journal of Contents, vol. 11, no. 4, pp. 77–85, 2015

    2015

  13. [21]

    Deepdesrt: Deep learning for detection and structure recognition of tables in document images,

    S. Schreiber, S. Agne, I. Wolf, A. Dengel, and S. Ahmed, “Deepdesrt: Deep learning for detection and structure recognition of tables in document images,” in2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1. IEEE, 2017, pp. 1162–1167

    2017

  14. [22]

    A saliency-based convolutional neural network for table and chart detection in digitized documents,

    I. Kavasidis, S. Palazzo, C. Spampinato, C. Pino, D. Giordano, D. Giuf- frida, and P. Messina, “A saliency-based convolutional neural network for table and chart detection in digitized documents,”arXiv preprint arXiv:1804.06236, 2018

    2018 arXiv

  15. [23]

    Learning table extraction from examples,

    A. Tengli, Y . Yang, and N. L. Ma, “Learning table extraction from examples,” inProceedings of the 20th International Conference on Computational Linguistics. Association for Computational Linguistics, 2004, p. 987

    2004

  16. [24]

    Multido- main document layout understanding using few shot object detection,

    P. Singh, S. Varadarajan, A. N. Singh, and M. M. Srivastava, “Multido- main document layout understanding using few shot object detection,” arXiv preprint arXiv:1808.07330, 2018

    2018 arXiv

  17. [25]

    Synthtabnet: A synthetically generated dataset for document table recognition,

    M. Agarwal, A. Mondal, and C. Jawahar, “Synthtabnet: A synthetically generated dataset for document table recognition,” https://arxiv.org/abs/ 2105.01238, 2021

    2021 arXiv

  18. [26]

    Pubtables-1m: Towards comprehensive table extraction from unstructured documents,

    B. Smock, R. Pesala, and R. Abraham, “Pubtables-1m: Towards comprehensive table extraction from unstructured documents,”CoRR, vol. abs/2110.00061, 2021. [Online]. Available: https://arxiv.org/abs/ 2110.00061

    2021 arXiv

  19. [27]

    Docbank: A benchmark dataset for document layout analysis,

    M. Li, Y . Xu, L. Cui, S. Huang, F. Wei, Z. Li, and M. Zhou, “Docbank: A benchmark dataset for document layout analysis,” 2020. [Online]. Available: https://arxiv.org/abs/2006.01038

    2020 arXiv

  20. [28]

    Table-transformer: A transformer-based model for table detection and structure recognition,

    B. Smock, R. Pesala, and R. Abraham, “Table-transformer: A transformer-based model for table detection and structure recognition,” inDocument Analysis and Recognition–ICDAR 2021: 16th International Conference, Lausanne, Switzerland, September 5–10, 2021, Proceed- ings, Part I....

    2021

  21. [29]

    Tableformer: Robust transformer modeling for table-text encoding,

    J. Yang, A. Gupta, S. Upadhyay, L. He, R. Goel, and S. Paul, “Tableformer: Robust transformer modeling for table-text encoding,”

  22. [30]

    Mask r-cnn,

    K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick, “Mask r-cnn,” 2018. [Online]. Available: https://arxiv.org/abs/1703.06870

    2018 arXiv

  23. [31]

    Marmot dataset

    Institute of Computer Science and Technology, Peking University. Marmot dataset. Accessed on April 30, 2024. [Online]. Available: https://www.icst.pku.edu.cn/cpdp/sjzy/

    2024

  24. [32]

    Tensorflow,

    TensorFlow Authors, “Tensorflow,” https://www.tensorflow.org/, 2015, accessed: May 10. 2024

    2015

  25. [33]

    Tablenet enhancement,

    Z. Bettouche, “Tablenet enhancement,” https://github.com/ zineddine-bettouche/tablenet-enhancement, 2024

    2024

  26. [2022]

    Available: https://arxiv.org/abs/2203.00274

    [Online]. Available: https://arxiv.org/abs/2203.00274

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.