{"id":"85d9482c-ce10-4d2f-8756-7f3f3a05f76d","arxiv_id":"2508.20824","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"GPT-FT replaces the LSTM encoder-decoder of MOAT with a small decoder-only GPT that both reconstructs transformation sequences and predicts their performance, enabling faster gradient-based feature search.","lead":"The paper trains a small GPT-style network to automatically rewrite table features, then searches over the rewrite rules it learned. It reports beating nine existing feature-engineering methods on 15 datasets while using roughly half the model size and less inference time.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 contradicts the central claim: on Geographical Origin of Music, GPT-FT (0.508) is below DIFER (0.632), so 'outperforms all others across datasets' is false as printed.","rationale":"The reader's weakest_assumption focused on the smoothness/calibration of the learned embedding space for gradient ascent. While that is a legitimate methodological concern, it is secondary to a more immediate and decisive problem: the paper's own Table 1 contains a dataset where GPT-FT is not the best, directly contradicting the central performance claim. The abstract and Section 4.2 both claim GPT-FT matches or exceeds all baselines across all datasets; the Geographical Origin of Music row shows DIFER outperforming GPT-FT by a large margin. This is not a matter of statistical noise or hidden assumptions—it is a factual inconsistency in the reported numbers. Because the claim of universal superiority is the paper's main contribution, and because it is falsified by the paper's own data, the verdict should move from CONDITIONAL to REJECT. The efficiency claims (parameter size and inference time) may still hold, but the primary performance claim is unsupported as written. I agree with the reader that missing error bars and unclear selection protocols weaken the paper, but the Table 1 discrepancy is more load-bearing and must be addressed before any conditional acceptance. A simple rerun of that single dataset would settle whether this is a real counterexample or a transcription error. If it is a typo, the paper could be revised; as submitted, it is not a reliable scientific record.","tokens_in":12721,"tokens_out":5499,"duration_ms":48564,"concrete_test":"Re-run the Geographical Origin of Music experiment with GPT-FT and DIFER under identical train/test splits, downstream model, and evaluation metric (1-RAE). If GPT-FT's mean 1-RAE is below DIFER's (e.g., 0.508 vs 0.632), the headline claim is refuted. Also inspect the table row for a possible column transposition; report the corrected values.","verdict_should_be":"REJECT","load_bearing_attack":"Section 4.2 states: 'Table 1 compares GPT-FT with other models on F1-score and 1-RAE metrics, showing GPT-FT outperforms all others across datasets.' Yet Table 1, row 'Geographical OriginalofMusic' (regression, 1-RAE, higher is better), lists GPT-FT at 0.508 and DIFER at 0.632 — a 0.124 gap in DIFER's favor. This is a direct internal contradiction between the paper's headline claim and its own reported results. Unlike concerns about embedding smoothness or missing error bars, this is a concrete factual inconsistency that does not rely on statistical interpretation. If the table is correct, the claim 'outperforms all others across datasets' is false. If the table contains a typo, the paper as submitted is unreliable and needs correction. Either way, the central claim is not supported by the evidence as presented.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GPT-FT, a four-stage framework for automated feature transformation: (1) collecting transformation records from an RL-based feature generation framework (GRFG), (2) training a small decoder-only GPT with joint sequence-reconstruction (NLL) and performance-estimation (MSE) losses to build a continuous embedding space, (3) performing gradient ascent on the performance head starting from the top-k seed embeddings, and (4) autoregressively reconstructing candidate transformation sequences and selecting the feature space with the highest downstream performance. The authors claim that GPT-FT matches or exceeds state-of-the-art baselines while significantly reducing parameter size and inference time, based on experiments on 15 classification/regression datasets compared against nine baselines, plus ablations and robustness checks.","tokens_in":12806,"tokens_out":4092,"duration_ms":41046,"significance":"If the results are correct, the efficiency contribution is meaningful: Tables 4-5 show roughly a 50% parameter reduction and substantial inference speedups over the LSTM-based MOAT, and the idea of a compact decoder-only transformer with a dual reconstruction/performance head is a reasonable step for continuous AFT search. The paper also provides ablation and sensitivity analyses (Figures 4-6) that help localize the contribution of Steps 1 and 3. However, the central performance claim is undermined by an internal contradiction in the main results table and by the absence of statistical rigor. The paper's significance therefore depends on correcting the experimental reporting and re-framing the claims.","major_comments":[{"comment":"The text states that \"Table 1 compares GPT-FT with other models on F1-score and 1-RAE metrics, showing GPT-FT outperforms all others across datasets.\" This is directly contradicted by the Geographical Origin of Music row: DIFER scores 0.632 and GPT-FT scores 0.508, a 0.124 gap in DIFER's favor. Since Table 1 is the central evidence for the headline claim, either the table or the claim is wrong. This must be corrected and the affected conclusions re-evaluated.","section":"Section 4.2, Table 1"},{"comment":"The assertion \"The performance satisfies G(Ē) ≥ G(E)\" is a property of gradient ascent on the fitted surrogate G, not a guarantee about actual downstream performance. The real performance after decoding is only empirically verified. The paper itself notes that α∈[0.4,0.9] \"fails to generate valid records\" and that embedding size 32 yields \"occasional invalid records\" (Section 4.2), showing the surrogate-to-real transfer is fragile. The authors should present direct evidence (e.g., decoding results over many seeds) that gradient ascent on G reliably improves P, and discuss the invalid-record cases.","section":"Section 3.4"},{"comment":"All performance tables report a single run without standard deviations, confidence intervals, or significance tests. Many differences are small (e.g., Heart Disease: GPT-FT 0.867 vs. MOAT 0.866; Weather: 0.980 vs. 0.976). Without repeated trials, the claimed superiority is not statistically supported. In addition, Section 3.5 selects the final feature set as the one yielding the highest downstream performance; if this selection is not also applied to baselines, the comparison is unfair. Report means and variances over multiple runs, and specify the selection protocol for all methods.","section":"Section 4.2, Tables 1-3"},{"comment":"Step 1 collects transformation records using GRFG, which is itself one of the baselines. Training GPT-FT on GRFG's RL-collected records and then comparing against GRFG is not an independent evaluation. The authors should clarify what GPT-FT adds beyond GRFG's own search, for example by comparing against a variant trained on records from a different collector or from random transformation sequences, to show that the gains are not inherited from the data collector.","section":"Section 3.2 and Baseline List"}],"minor_comments":[{"comment":"The dataset name \"Geographical OriginalofMusic\" should be \"Geographical Origin of Music\" (space missing).","section":"Table 1"},{"comment":"The NLL loss expression is written as a sum over log probabilities, but the distribution p(γ̂_i | γ_i) is not parameterized. Specify the token-level distribution and how teacher forcing is used during training.","section":"Section 3.3"},{"comment":"The symbol G is used loosely: it is first called \"GPT-FT\" and then differentiated as ∂G/∂E. Clarify that the gradient is taken with respect to the embedding through the task classifier head δ, not the whole model.","section":"Section 3.4"},{"comment":"\"α ∈ [0.4, 0.9] fails to generate valid records\" is a strong statement but no quantitative criterion is given. Define \"valid records\" and report how many seeds/sequences failed in each setting.","section":"Section 4.2, Parameter Sensitivity α"},{"comment":"The caption \"NoDataPreprocess\" is ambiguous: the ablation removes Step 1 (transformation records collection), not generic data preprocessing. Rename for clarity.","section":"Figure 4 and Ablation"},{"comment":"The top-42 seed count and α=0.133 are reported after NNI optimization, but no validation/test split for this tuning is described. Clarify whether selection of these hyperparameters used the test set.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The efficiency claims (Tables 4-5) appear credible and are the strongest part of the paper. The performance claim is not acceptable as printed because of the Table 1 contradiction and the lack of repeated-run statistics. I would ask for a corrected table or claim, error bars/significance tests, and a clarification of the selection protocol before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this one. First, the efficiency story is probably real: swapping MOAT's LSTM encoder-evaluator-decoder for a single-layer decoder-only GPT halves the parameter count and cuts inference time by roughly 10-40% across all 15 reported datasets (Tables 4-5). That is concrete and worth taking seriously. Second, the performance claim does not survive contact with the paper's own Table 1. The text says 'GPT-FT outperforms all others across datasets,' but on the Geographical Origin of Music row, DIFER scores 0.632 and GPT-FT scores 0.508. That is not a small gap; it is a direct contradiction. Either the table is a typo or the sentence is wrong. As submitted, the central claim is false.\n\nThe methodological delta over the authors' own MOAT is a backbone swap and a joint NLL+MSE loss. That is not a new paradigm, but it is a sensible engineering step, and the paper is honest about its sensitivity failures: α ≥ 0.4 generates no valid records and embedding size 32 occasionally yields invalid sequences. Those admissions cut in the paper's favor.\n\nThe soft spots are mostly about support. There are no error bars or repeated runs anywhere, so 'outperforms all' is a claim about single draws. The selection protocol in Section 3.5 doesn't tie the final feature space to a held-out validation set, so we can't rule out selection-on-test. And Section 3.4's statement that 'G(Ẽ) ≥ G(E)' is a tautology about the fitted surrogate, not a guarantee about real performance; the paper should say so explicitly. The α restriction to [0.1, 0.3] after observing failures for α ≥ 0.4 is post hoc tuning, and the baseline numbers come without code or detailed protocols.\n\nOverall: the framework is coherent, the efficiency evidence is solid, and the writing is candid. But the performance claim is overblown and the paper cannot be accepted as is. A serious referee should see it, because the efficiency result is reproducible in principle and the contradiction is fixable. After the table/claim mismatch is corrected, error bars or repeated runs are added, and the selection procedure is clarified, this might be a useful contribution to the AFT niche. For now I'd view it as a promising draft in need of major revision.","headline":"Efficiency gains look real, but the paper's own Table 1 contradicts its central performance claim; needs major revision before it can be trusted.","tokens_in":13471,"tokens_out":2776,"would_cite":false,"duration_ms":24666,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A decoder-only GPT with a 64-dimensional embedding can be trained to reconstruct feature transformation sequences and estimate their performance, then refined by gradient ascent to generate transformations that beat nine prior automated fea","keywords":["Automated feature transformation","GPT","Gradient-ascent search","Embedding space","Sequence reconstruction","Performance estimation","Multi-objective optimization","Feature engineering"],"falsifier":"Take a dataset, train GPT-FT with the paper's settings, and compare the true held-out performance of sequences decoded after gradient ascent against the top-42 seed records without gradient ascent. If the best decoded sequence does not beat the best seed more often than random gradient directions do, the central claim fails; the paper's zero-gradient ablation is the natural control.","tokens_in":12461,"feed_emoji":"⚙️","tokens_out":8254,"duration_ms":76355,"temperature":0.7,"pith_summary":"The paper proposes GPT-FT, a four-stage pipeline that turns automated feature transformation into continuous embedding optimization instead of discrete search. It trains a small decoder-only GPT to both reconstruct transformation sequences and estimate their downstream performance, then runs gradient ascent on the learned embeddings and decodes improved sequences. On 15 benchmark datasets, it reports F1-score and 1-RAE results that match or beat nine earlier methods, including the larger encoder-decoder LSTM model MOAT, with roughly half the parameters and faster inference. The intended payoff is that transformer-based sequence models can make automated feature engineering scalable and cheap enough for practical, even real-time, use.","feed_headline":"A 64-dim GPT embedding beats nine baselines on all 15 datasets","feed_subtitle":"Automated feature search becomes gradient ascent in a compact learned embedding, cutting parameters in half and speeding up inference.","key_machinery":"The central object is the revised GPT: a single-layer decoder-only transformer with embedding size 64 that maps a postfix transformation sequence to an embedding, reconstructs the sequence through a text predictor head, and estimates its downstream performance through a task classifier head. The joint loss L = αL_pre + (1−α)L_cls trains all heads together, balancing sequence reconstruction against performance estimation. The same embedding space is then searched by gradient ascent (Ẽ = E + η ∂G/∂E) starting from the top-42 seed records, and the optimized embeddings are decoded autoregressively into candidate transformation sequences.","core_discovery":"The paper's central claim is that a decoder-only GPT-1-style model with embedding size 64, trained from scratch on reinforcement-learning-collected transformation records, can serve simultaneously as a sequence generator and a performance estimator, and that gradient ascent in this embedding space yields better transformation sequences than the discrete search used by earlier approaches. GPT-FT replaces MOAT's separate encoder, evaluator, and decoder with one compact model trained on a joint loss of reconstruction and performance estimation. The paper asserts that Table 1 shows GPT-FT outperforming all nine baselines across all 15 datasets, with parameter sizes about half of MOAT's and infer","pith_inferences":["A direct extension the authors do not explore: the same joint reconstruction-plus-performance embedding could be applied to other discrete search problems, such as neural architecture search, prompt optimization, or program synthesis, wherever sequence generation and a scalar reward can be paired.","Because the training records come from a specific RL collector, GPT-FT's ceiling is inherited from the diversity of that collector; if the collector never generates certain useful operation patterns, gradient ascent cannot recover them however smooth the embedding is.","The reported efficiency win is partly architectural, but the paper does not separate the cost of collecting records (Step 1) from the cost of the search itself; a full end-to-end cost comparison would need to count the RL data-collection time as well."],"forward_implications":["Feature transformation can be formulated as differentiable optimization in a compact learned embedding space, replacing combinatorial search with a few gradient steps.","A decoder-only GPT trained from scratch with embedding size 64 can carry both reconstruction and performance-estimation functions, making the larger encoder-decoder LSTM design unnecessary.","The generated feature spaces transfer across downstream model families, so one transformed dataset can serve linear models, trees, SVMs, and neural models without re-running the transformation search.","Because inference time is roughly halved on several datasets, the approach is more plausible for online or large-scale feature engineering workloads.","The ablations imply that both RL-collected records and gradient-ascent refinement contribute to the gains, so neither stage can be dropped without losing performance."],"supporting_citations":[{"why":"Supplies the reinforcement-learning feature-generation framework used to collect the transformation records that seed GPT-FT, and is one of the nine baselines.","marker":"[40]"},{"why":"Provides the original GPT architecture that GPT-FT revises by reducing embedding size to 64 and using a single-layer embedding generator.","marker":"[35]"},{"why":"Defines MOAT, the encoder-evaluator-decoder LSTM baseline that GPT-FT claims to beat on accuracy, parameter size, and inference time.","marker":"[41]"},{"why":"Introduces DIFER, the seq2seq embedding-plus-gradient-search baseline that GPT-FT extends with a decoder-only model.","marker":"[48]"},{"why":"NFS models feature transformation as a search problem and is one of the comparison baselines.","marker":"[3]"},{"why":"TTG treats transformation as a graph-based reinforcement learning search and is one of the comparison baselines.","marker":"[23]"},{"why":"AFAT iteratively generates and selects features and is one of the comparison baselines.","marker":"[12]"},{"why":"LDA derives a new feature space via matrix factorization and is one of the comparison baselines.","marker":"[1]"}],"fun_headline_variants":["GPT-FT: One GPT cuts feature search cost in half","Tiny GPT beats nine baselines on 15 datasets","Automated feature transformation with a 64-dim GPT","GPT-FT: faster feature search, half the parameters","From discrete search to gradient ascent for features"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the embedding space learned from the RL-collected records is smooth and calibrated enough that moving along the gradient of the learned performance head lands on embeddings that decode into valid transformation sequences that genuinely improve real downstream performance, not just the predicted score.","fun_headline_variants_meta":{"raw":{"variants":["GPT-FT: One GPT cuts feature search cost in half","Tiny GPT beats nine baselines on 15 datasets","Automated feature transformation with a 64-dim GPT","GPT-FT: faster feature search, half the parameters","From discrete search to gradient ascent for features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000134,"raw_usage":{"total_tokens":958,"prompt_tokens":706,"completion_tokens":252,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":450,"completion_tokens_details":{"reasoning_tokens":174}},"tokens_in":450,"tokens_out":252,"duration_ms":3068,"temperature":1.0,"reasoning_tokens":174,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:48:20.810435+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset, train GPT-FT with the paper's settings, and compare the true held-out performance of sequences decoded after gradient ascent against the top-42 seed records without gradient ascent. If the best decoded sequence does not beat the best seed more often than random gradient directions do, the central claim fails; the paper's zero-gradient ablation is the natural control.","supporting_citations":[{"cited_title":"In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","cited_arxiv_id":null,"evidence_quote":"Supplies the reinforcement-learning feature-generation framework used to collect the transformation records that seed GPT-FT, and is one of the nine baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the original GPT architecture that GPT-FT revises by reducing embedding size to 64 and using a single-layer embedding generator."},{"cited_title":"Advances in Neural Information Processing Systems36, 43563–43578 (2023)","cited_arxiv_id":null,"evidence_quote":"Defines MOAT, the encoder-evaluator-decoder LSTM baseline that GPT-FT claims to beat on accuracy, parameter size, and inference time."},{"cited_title":"In: International Conference on Automated Machine Learning","cited_arxiv_id":null,"evidence_quote":"Introduces DIFER, the seq2seq embedding-plus-gradient-search baseline that GPT-FT extends with a decoder-only model."},{"cited_title":"In: 2019 IEEE International Conference on Data Mining (ICDM)","cited_arxiv_id":null,"evidence_quote":"NFS models feature transformation as a search problem and is one of the comparison baselines."},{"cited_title":"In: Proceedings of the AAAI Conference on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"TTG treats transformation as a graph-based reinforcement learning search and is one of the comparison baselines."},{"cited_title":"In: Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I","cited_arxiv_id":null,"evidence_quote":"AFAT iteratively generates and selects features and is one of the comparison baselines."},{"cited_title":"Journal of machine Learning research 3(Jan), 993–1022 (2003)","cited_arxiv_id":null,"evidence_quote":"LDA derives a new feature space via matrix factorization and is one of the comparison baselines."}],"review_version":1}