{"id":"bfeb7b80-fcb2-438c-9acf-cec6b104ab4c","arxiv_id":"2412.18111","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A prompt-enhanced language model with a column-partitioning algorithm generates synthetic tabular data that beats existing methods on 14 of 20 public datasets.","lead":"AIGT adds table descriptions and column names as prompts when training a language model to create synthetic spreadsheet rows, and splits extra-wide tables into overlapping chunks. The method is reported to beat several existing synthetic-data tools on 14 of 20 public datasets and two Alipay risk-control tables.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Re-labeling with a real-data-trained LightGBM confounds the ML-efficiency metric; without an ablation, AIGT's SOTA claim may reflect the surrogate labeler, not its generative quality.","rationale":"The central claim is that AIGT achieves state-of-the-art machine-learning efficiency on 14/20 public datasets, but the evaluation pipeline in Section 5.2 uses synthetic tables whose labels have been re-labeled by a LightGBM trained on the original data (Section 4.4). Because the downstream evaluator is also a tuned LightGBM, the reported scores can be dominated by the surrogate predictor's ability to imprint original-label information, rather than by AIGT's ability to model the joint distribution of features and labels. This is exactly the reader's weakest assumption, and it is load-bearing: if the re-labeling step is removed or applied uniformly to all baselines, the ranking may change substantially. The paper does not provide this ablation, and it does not state explicitly whether baselines other than TapTap also receive re-labeling. The concern is addressable, so conditional acceptance is appropriate, not rejection. The secondary issue that 'AIGT' includes both the distilGPT-2 and Llama-3.1 variants when counting 14/20 is worth noting but does not change the primary verdict.","tokens_in":16780,"tokens_out":7970,"duration_ms":69210,"concrete_test":"Reproduce Table 3 with the re-labeling step in Section 4.4 disabled for AIGT, leaving all baselines unchanged. If AIGT wins fewer than 14 out of 20 datasets, the SOTA claim depends on re-labeling and is confounded; if it still wins 14/20, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sections 4.4 and 5.2 together create a circularity risk. In 4.4, AIGT's generated labels are replaced by y'_i = P(x'_i), where P is a LightGBM fit to the original training data. In 5.2, the same synthetic table is used to train another tuned LightGBM, and its test AUC/R2 is the headline metric. Thus the evaluated label column is not a sample from AIGT's generative distribution; it is the output of a supervised model that already knows the real labels. The metric then upper-bounds at P's accuracy plus the fidelity of the marginal feature distribution, and any generator that produces plausible x' can inherit P's performance even if its joint distribution is wrong. The paper does not report the MLE result without re-labeling, and it does not specify whether the same re-labeling was applied to all baselines. If it was applied only to AIGT (and perhaps TapTap), then the comparison against CTGAN, TVAE, TabDDPM, and GReaT is not a fair test of generative quality. Table 3 even shows AIGT exceeding the 'Real' row on CG (78.1 vs 76.9), which is a red flag that the surrogate labeler, not the generator, is driving the score. This directly undermines the central 'SOTA on 14/20' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AIGT, a prompt-enhanced autoregressive language model for tabular data synthesis. It uses table metadata (captions and column meanings) as a fixed prompt, serializes each row into text, fine-tunes a language model on the downstream table, and generates synthetic rows. To handle wide tables, it introduces a long-token partitioning algorithm with overlapping columns. The experimental evaluation covers 20 public datasets and two Alipay risk-control datasets, using machine-learning efficiency, DCR, data augmentation, and ablations as metrics, with a claim of state-of-the-art performance on 14 of 20 public datasets and both industry datasets.","tokens_in":17057,"tokens_out":4972,"duration_ms":45769,"significance":"If the reported results are valid, the paper makes a useful contribution: it demonstrates that metadata prompts improve LLM-based tabular synthesis and provides a partitioning scheme that scales to hundreds of columns, along with an open-source pre-training corpus. The evaluation is extensive (10 seeds, tuned LightGBM, multiple metrics, and ablation studies). However, the validity of the headline ML-efficiency comparison is undermined by the re-labeling procedure described in Section 4.4, so the significance of the SOTA claim cannot be assessed without a corrected protocol.","major_comments":[{"comment":"The main ML-efficiency evidence in Table 3 is computed on synthetic tables whose label column has been replaced by y'_i = P(x'_i), where P is a LightGBM trained on the original training data. Because the downstream evaluator is also a tuned LightGBM, the reported AUC/R2 values can be inflated by the surrogate predictor's access to the original labels, and they do not measure AIGT's joint generative distribution. The manuscript does not state whether CTGAN, TVAE, TabDDPM, and GReaT were also re-labeled; if they were not, Table 3 is not a fair comparison of generative quality. Please report the ML-efficiency results without re-labeling for AIGT and all baselines, and specify the re-labeling status of each method.","section":"Section 4.4 and Section 5.2, Table 3"},{"comment":"AIGT (78.1±2.3) exceeds the Real row (76.9±1.8) on Credit-g. Under the standard ML-efficiency protocol, a model trained on synthetic data should not systematically outperform a model trained on real data unless the synthetic labels are produced by a supervised model that already incorporates the real labels. This observation is consistent with the re-labeling confound described above and should be explained or removed by an ablation.","section":"Table 3, Credit-g row"},{"comment":"In the partition-experiment comparison (Table 5), AIGT-part is compared with CTGAN, TVAE, and TabDDPM on the Alipay datasets SYH and NonBD, but the AIGT rows for those datasets are marked as not computable because the tables are too wide for the unpartitioned model. Since the re-labeling procedure is described only for AIGT in Section 4.4, it is unclear whether the baseline rows in Table 5 received any re-labeling. The reported gains (e.g., 57.2 vs 45.7 on SYH) could reflect the surrogate labeler rather than the partitioning algorithm; please clarify the protocol for each method in this table.","section":"Section 4.4 and Table 5"},{"comment":"The stated limitations (processing speed and numerical value encoding) do not mention the re-labeling step's effect on the evaluation. Given that the main SOTA claim rests on the re-labeled ML-efficiency metric, the paper should either justify that the re-labeling is applied identically to all methods and report an ablation, or reframe the claim as one about a specific label-refinement pipeline rather than about generative fidelity.","section":"Limitations section"}],"minor_comments":[{"comment":"The heading contains a typo: \"Ailpay Dataset\" should be \"Alipay Dataset\".","section":"Table 2 title"},{"comment":"The typo \"diff-sion method\" should be \"diffusion method\".","section":"Section 5.1"},{"comment":"The provided code uses the deprecated davinci-codex engine and an HTTP endpoint that is no longer the standard OpenAI API; the code should be updated to a working gpt-3.5-turbo example.","section":"Appendix B, Listing 1"},{"comment":"The discriminator table lists two rows labeled \"CR\": the first appears to refer to the Car dataset and the second to California Housing; rename the second row to \"CA\" for clarity.","section":"Table 7"},{"comment":"The abstract claims AIGT \"achieves state-of-the-art performance on 14 out of 20 public datasets,\" while Section 5.2 says AIGT \"match[es] or exceed[s] state-of-the-art on 14 out of 20 datasets.\" These are different claims and should be reconciled.","section":"Abstract and Section 5.2"},{"comment":"The y-axis label \"average metric values\" is vague; specify that it is the average AUC or R2 across the selected datasets.","section":"Figure 5"},{"comment":"The reference list contains duplicate entries for Xu et al. (2019a,b); both entries cite the same paper with the same title and should be merged.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the re-labeling protocol. If the authors can show that the same re-labeling was applied to all baselines and that the results are robust without re-labeling, the paper could be publishable. The industrial deployment claims are interesting but the evaluation must be made fair first."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a useful new mechanism for scaling LLM-based table synthesis, but its headline ML-efficiency numbers are confounded by a re-labeling step that substitutes LightGBM predictions trained on the real data for the synthesized labels. The 'SOTA on 14/20' claim does not hold up until that is addressed.\n\nWhat is actually new: AIGT adds metadata prompts, label prioritization, and a column-partitioning algorithm to the GReaT/TapTap line. The partition algorithm is the real contribution—overlapping column subsets let you train and generate on parts of a wide table and stitch them together, and the industrial results on 45- and 251-column tables show it works. The prompt-enhancement and label-prioritization ablations in Figure 5 support those design choices.\n\nWhat the paper does well: the experimental effort is substantial. Twenty public datasets, two industrial tables, ten seeds, error bars, and multiple metrics. The DCR and correlation-distance results (Figure 7) are particularly nice: AIGT's mean correlation distance is 1.33, below TapTap, TVAE, CTGAN, and TabDDPM. Those metrics are independent of the label confound and give genuine evidence of better joint distribution fidelity.\n\nThe soft spot: Section 4.4's re-labeling. AIGT trains a LightGBM on the real data and replaces every synthetic label with that model's prediction. Then the MLE section trains another tuned LightGBM on the re-labeled synthetic table. That measures the surrogate labeler's accuracy, not the generator's fidelity. It can even beat the Real row—CG 78.1 vs 76.9—because the surrogate labels are a smoothed target. The paper never shows an ablation without re-labeling, and it doesn't say whether baselines got the same treatment. If only AIGT (and maybe TapTap) had re-labeling, the comparison against CTGAN/TVAE/TabDDPM/GReaT is not fair. That is a load-bearing omission for the central quantitative claim.\n\nMinor issues: no code or data link (STABS is claimed open-source but no URL), the 'any scale' claim overstates what a 251-column test supports, and the partition overlap parameter is fixed without sensitivity analysis.\n\nWho it's for: anyone working on LLM-based tabular synthesis will find the partition algorithm and prompt design worth reading. The evaluation, however, needs a no-relabeling ablation and clear statements about baseline parity. The method is plausible and the flaws are addressable, so I'd send it to peer review. I'd also bring it to a reading group, mainly to discuss how easy it is to accidentally overstate synthetic-data quality when a learned labeler is inserted mid-pipeline.","headline":"AIGT's partition algorithm and prompt design are worth attention, but the re-labeling step confounds the headline ML-efficiency results, so the SOTA claim needs an ablation without re-labeling.","tokens_in":17591,"tokens_out":4826,"would_cite":true,"duration_ms":43783,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompt-enhanced LLMs can generate synthetic tables that beat GAN, VAE, and diffusion baselines on most benchmarks.","keywords":["synthetic tabular data","large language models","prompt enhancement","tabular data synthesis","long-token partitioning","data augmentation","pre-training corpus","machine learning efficiency"],"falsifier":"Generate AIGT synthetic data with the re-labeling step disabled and rerun Table 3's ML-efficiency protocol on the same 20 datasets plus the two Alipay tables; if AIGT no longer matches or exceeds state-of-the-art on a majority of datasets, the headline advantage rests on the surrogate label imputer rather than on prompt-enhanced generation.","tokens_in":16586,"feed_emoji":"📊","tokens_out":5066,"duration_ms":42527,"temperature":0.7,"pith_summary":"The paper introduces AIGT, a table generator built on an auto-regressive language model that uses a table's metadata—its caption, column names, and column meanings—as a fixed prompt during training and sampling. It claims that this prompt enhancement, combined with putting the label column first and pre-training on a large corpus of real tables (STABS, 978 datasets), lets a DistilGPT-2-sized model synthesize tables whose machine-learning utility matches or beats CTGAN, TVAE, TabDDPM, GReaT, and TapTap on 14 of 20 public benchmarks and on two Alipay risk-control tables. A long-token partitioning algorithm with overlapping 'cover' columns is proposed so the same method can handle tables with hundreds of columns, which prior LLM-based generators could not. If correct, the paper shows that cheap metadata, not larger models, is what language-model synthesizers were missing, and that wide industrial tables become tractable for generative LLMs.","feed_headline":"LLM table generator wins 14 of 20 synthetic-data benchmarks","feed_subtitle":"Metadata prompts and a column-partitioning trick let an LLM make wide, realistic tables for data augmentation.","key_machinery":"The load-bearing mechanism is the prompt-enhanced textual encoding: a fixed metadata prompt is prepended to each serialized row, with the label column fixed first and all other columns randomly shuffled, while the loss ignores prompt tokens. This teaches the language model the semantic role of each column instead of just the token statistics. The second mechanism is the long-token partitioning algorithm, which splits wide tables into overlapping column partitions, trains them jointly, and generates partitions from the last to the first, using each partition's overlap as the conditioning context; this is what removes the token-limit barrier for LLM-based synthesizers.","core_discovery":"AIGT's central claim is that synthetic tabular data of high machine-learning efficiency can be produced by prompting an auto-regressive language model with semantic metadata about the table. Each row is serialized as sentences of the form '[Feature] is [Value]', with the label fixed in first position and the other features randomly shuffled; the sequence is prefixed by a prompt describing the dataset and the meaning of each column, and the loss is computed only on the data tokens, not the prompt tokens. After pre-training on 978 cleaned OpenML tables and fine-tuning on a target table, rows are sampled autoregressively, and the generated label is replaced by the prediction of a LightGBM trained on the original data (re-labeling). For wide tables, columns are split into overlapping partitions that are trained jointly and generated back-to-front using the overlap as a starting column, enabling tables of arbitrary width. On ML efficiency measured with a tuned LightGBM, AIGT matches or exceeds the previous state of the art on 14 of 20 public datasets and on both Alipay datasets, and it also improves data augmentation and lowers correlation distance to the original data.","pith_inferences":["The re-labeling step (Section 4.4) may carry part of the ML-efficiency advantage: since the downstream evaluator is also LightGBM, replacing generated labels with LightGBM predictions imprints original-label information, and the paper reports no ablation without re-labeling; a fair test would generate labels directly from the LLM and compare.","The partition algorithm introduces a design choice—how many overlapping cover columns to keep—and the paper fixes it at 1 for industry tables without a sensitivity analysis on the overlap width; the loss of cross-partition correlations likely grows as partition count increases, unless the overlap is widened.","The prompt construction relies on a GPT-3.5 call to expand metadata; this means the method depends on an external model for prompt quality, and the same table could produce different prompts across calls, so prompt robustness is a testable extension.","A natural extension is to apply the same metadata-prompt recipe to other backbone LLMs (e.g., larger instruction-tuned models) and to other downstream metrics such as privacy and leakage, since the paper's DCR and discriminator results suggest low copying but not formal privacy guarantees."],"forward_implications":["LLM-based table generators can now handle arbitrarily wide tables, since the partitioning algorithm removes the token-limit constraint.","Metadata prompts improve generation enough to move a DistilGPT-2 backbone past GAN-, VAE-, and diffusion-based synthesizers on most of the tested benchmarks.","Pre-training on a large corpus of named, semantically meaningful tables (STABS) transfers to downstream tables, so a single pre-trained synthesizer can be fine-tuned per table.","Synthetic data from AIGT is useful both as a replacement for original training data and as an augmentation set, improving tuned LightGBM performance on most of the 20 datasets."],"supporting_citations":[{"why":"GReaT, the LLM-based tabular generator that AIGT extends with metadata prompts and whose serialization, sampling, and temperature settings are reused.","marker":"(Borisov et al., 2023)"},{"why":"TapTap, the pre-trained table generator baseline and source of the re-labeling idea.","marker":"(Zhang et al., 2023)"},{"why":"CTGAN and TVAE, the GAN and VAE baselines that AIGT must beat on the 20 public datasets.","marker":"(Xu et al., 2019b)"},{"why":"TabDDPM, the diffusion-model baseline for tabular synthesis.","marker":"(Kotelnikov et al., 2023)"},{"why":"GPT-2, the auto-regressive language model whose distilled version (DistilGPT-2) serves as the backbone for AIGT.","marker":"(Radford et al., 2019)"},{"why":"Prompt-tuning loss design, which grounds AIGT's choice to ignore prompt-token losses during training.","marker":"(Lester et al., 2021)"}],"fun_headline_variants":["Metadata prompts help LLM top 14 of 20 tabular benchmarks","AIGT: prompt-enhanced LLM wins 14 of 20 synthetic data tests","Partitioning trick lets LLM model any-width tables for synthetic data","LLM with metadata prompts beats SOTA on 14/20 tabular datasets","Prompted LLM generates high-quality synthetic tables, SOTA on 14/20"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The re-labeling step assumes that replacing model-generated labels with predictions from a LightGBM trained on the original table preserves or improves synthetic-data quality without biasing the reported machine-learning-efficiency gains, and the paper does not isolate this effect.","fun_headline_variants_meta":{"raw":{"variants":["Metadata prompts help LLM top 14 of 20 tabular benchmarks","AIGT: prompt-enhanced LLM wins 14 of 20 synthetic data tests","Partitioning trick lets LLM model any-width tables for synthetic data","LLM with metadata prompts beats SOTA on 14/20 tabular datasets","Prompted LLM generates high-quality synthetic tables, SOTA on 14/20"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1403,"prompt_tokens":951,"completion_tokens":452,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":567,"tokens_out":452,"duration_ms":5408,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:01:03.387238+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate AIGT synthetic data with the re-labeling step disabled and rerun Table 3's ML-efficiency protocol on the same 20 datasets plus the two Alipay tables; if AIGT no longer matches or exceeds state-of-the-art on a majority of datasets, the headline advantage rests on the surrogate label imputer rather than on prompt-enhanced generation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TabDDPM, the diffusion-model baseline for tabular synthesis."}],"review_version":1}