{"id":"6ba9f697-4cfe-4d0d-8aec-0ee4a7d149b4","arxiv_id":"2607.05380","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"TabPack packs MLPs with diverse sampled hyperparameters into one vectorized model, selects ensemble members online during training, and matches tuned baselines at a fraction of the compute cost.","lead":"TabPack trains dozens of MLPs with randomly sampled hyperparameters in a single packed run, then greedily selects the best ensemble on the fly — achieving competitive tabular ML performance without hyperparameter tuning. A smart generalist reads this because it could make strong deep learning on tabular data accessible without expensive GPU tuning infrastructure.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Default hyperparameter ranges in Table 7 were designed by the same group that created the benchmark; no held-out validation confirms they generalize, though TabArena provides partial mitigation.","rationale":"The reader correctly identified the default range generalizability as the weakest assumption. I sharpen this to the more specific concern: the ranges were designed by the same group that created the benchmark, the selection process is undocumented, and the independent TabArena check uses a different ensemble score (training loss), which is itself a form of dataset-specific adjustment. However, this concern does not rise to the level of changing the verdict. The paper's claims are explicitly scoped to 'medium-to-large public datasets' and are well-supported on 17 datasets plus TabArena and large-scale experiments. The ranges are broad (not finely tuned), the method is clearly described, code is shipped, and the conservative evaluation protocol is reasonable. The concern is about generalization to unseen domains — a practical adoption limitation that the authors partially acknowledge in Appendix C — rather than an internal inconsistency or correctness error in the evaluated results. The paper makes a useful, well-engineered contribution and the ACCEPT verdict with MODERATE confidence is appropriate. The confidence should perhaps be slightly tempered given the undocumented range selection process, but this is already captured by the MODERATE confidence rating.","tokens_in":23567,"tokens_out":2859,"duration_ms":200794,"concrete_test":"Evaluate TabPack with the exact default ranges from Table 7 (unchanged) on a set of 5-10 tabular datasets from a completely independent source (e.g., OpenML AutoML benchmark, or Kaggle competition datasets not derived from TabReD or Gorishniy et al. 2025). Use the validation metric as ensemble score (not training loss). If TabPack's mean rank against tuned TabM/RealMLP/XGBoost shifts by more than 1.0 rank position relative to the main benchmark result, the default ranges do not generalize and the 'tuning-free' claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — that TabPack is 'tuning-free' and competitive with tuned baselines — depends on the default sampling ranges in Table 7 (depth UniformInt[1,3], width 384, dropout {0.0, Uniform[0.0,0.5]}, Muon LR LogUniform[1e-3,1e-1], etc.) being good across datasets. These ranges were designed by the authors, who also created the benchmark (derived from Gorishniy et al. 2025 and TabReD, both from the same group). The paper does not describe how these ranges were chosen, and no held-out datasets were used to validate them. If the ranges were refined based on performance on these same 17 datasets, the 'tuning-free' claim becomes circular: tuning happened at the range-design level rather than per-dataset. The paper does provide partial mitigation: Appendix A.2 evaluates on TabArena (an independent benchmark) and Appendix A.1 evaluates on large-scale datasets, and TabPack performs competitively in both. However, the TabArena evaluation switches the ensemble score from validation metric to training loss (Appendix A.2 implementation note), which is itself a dataset-specific adjustment that undermines the 'no tuning' framing. The concern is not that the ranges are narrow (they are reasonably broad), but that the process of selecting them is undocumented and potentially benchmark-overfit, and the independent checks are imperfect. This is a generalizability concern, not an internal inconsistency — the claims as stated for the evaluated benchmarks are well-supported.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper introduces TabPack, an efficient ensemble of MLPs with randomly sampled hyperparameters for tabular deep learning. The core idea is to pack many MLPs with diverse hyperparameters (depth, dropout, learning rate, etc.) into a single model, train them in parallel, and select ensemble members online during training. This eliminates the need for per-dataset hyperparameter tuning. Experiments on 17 medium-to-large datasets show that a single tuning-free run of TabPack achieves mean rank 2.6, competitive with extensively tuned TabM (rank 2.8) and RealMLP (rank 3.6), while requiring substantially less compute (2.4h total vs 47.2h and 250.9h respectively). Additional evaluations on TabArena and large-scale datasets (Appendix A.1, A.2) provide further evidence of generalizability.","tokens_in":23779,"tokens_out":1461,"duration_ms":270678,"significance":"The paper makes a practical and well-executed contribution to tabular deep learning. The core technical idea of packing heterogeneous-architecture MLPs into a single trainable module (Model Pack and Optimizer Pack, Section 3.3-3.4) is clean and the implementation is non-trivial. The experimental methodology is thorough: 17 datasets, multiple baselines with 100+ tuning iterations, a conservative multi-seed evaluation protocol, per-dataset results with standard deviations (Table 14), ablations (Table 1, Figure 8), and additional benchmarks (Appendix A.1, A.2). The falsifiable claim that TabPack on a MacBook outperforms tuned baselines on an A100 in terms of total compute is a striking and verifiable result. The analysis in Section 5.1 showing that hyperparameter diversity's main role is enabling the tuning-free workflow rather than improving raw performance is a valuable insight. Reproducible code is provided.","major_comments":[{"comment":"§3.7 and Table 7: The central 'tuning-free' claim depends on the default hyperparameter sampling ranges (e.g., depth UniformInt[1,3], width 384, Muon LR LogUniform[1e-3,1e-1]). The paper does not describe how these ranges were selected. Given that the benchmark (derived from Gorishniy et al. 2025) and the method share authors, there is a risk that the ranges were refined based on performance on these same 17 datasets, which would make the 'tuning-free' claim partially circular (tuning happened at the range-design level). The independent evaluations in Appendix A.1 and A.2 provide partial mitigation, but the TabArena evaluation (Appendix A.2) switches the ensemble score from validation metric to training loss, which is itself a dataset-specific adjustment. The paper should explicitly state how the ranges in Table 7 were chosen (e.g., based on prior work, held-out datasets, or intuition) —","section":null},{"comment":"§3.5 and Appendix A.2 implementation note: The switch from validation metric to training loss as the ensemble score for TabArena is presented as a practical adjustment, but it is a dataset-specific hyperparameter choice that sits in tension with the 'tuning-free' framing. The paper should either (a) run TabArena with the default validation-metric ensemble score and report the resulting performance drop, or (b) explicitly acknowledge this as a limitation and discuss when users should expect to make such adjustments. As stated, the reader cannot assess how much the 'tuning-free' claim is weakened by this adjustment.","section":null},{"comment":"§5.1, Table 1: TabPack_SameHP achieves rank 1.4±0.7 vs TabPack's 1.6±0.6, meaning traditional per-dataset tuning of TabPack's architecture actually outperforms the tuning-free default. The paper frames this as 'the tuning slightly improves the performance at the cost of noticeable increase of the total runtime.' However, the rank difference (1.4 vs 1.6) is within one standard deviation, and the paper does not test whether this difference is statistically significant. Given that this comparison directly bears on the claim that hyperparameter diversity is 'a reasonable default strategy' (§5.1), a statistical test or a more careful discussion of whether the difference is meaningful would strengthen the conclusion.","section":null}],"minor_comments":[{"comment":"Figure 6 (left): The triangle-square-triangle markers for dataset groups are explained in the legend but the specific group assignments per dataset are not immediately clear. Consider adding a table or annotation clarifying which datasets fall into 'worst 25%,' 'middle 50%,' and 'best 25%' groups.","section":null},{"comment":"§3.3: The statement 'one can usually use smaller maximum widths and depths than in traditional single models' is not directly supported by the experiments. Table 7 shows width is fixed at 384, which is within the tuning range of baselines (UniformInt[64,1024] for TabM in Table 10). Consider softening this claim or providing evidence.","section":null},{"comment":"Appendix A.3: The 'thought experiment' about why TabPack performs well on large regression tasks is speculative. While clearly labeled as such, it could benefit from being shortened or moved to a discussion of limitations, as it does not provide empirical evidence for the proposed mechanisms.","section":null},{"comment":"Table 7: The dropout distribution '{0.0, Uniform[0.0,0.5]}' is ambiguous — it is unclear whether this means 50% chance of dropout=0.0 and 50% chance of sampling from Uniform[0,0.5], or something else. Clarify the mixture probability.","section":null},{"comment":"§4.2: The conservative evaluation protocol reruns TabPack with only the M_ens selected base models. It would be useful to report what fraction of the full performance is retained in the conservative vs optimistic protocol, to quantify the cost of this conservatism.","section":null},{"comment":"Figure 8: The y-axis label 'Rel. improv. over MLP' and the specific values (2.0%-3.0%) are not immediately interpretable. Consider clarifying whether this is the same relative improvement metric used in Figure 6.","section":null},{"comment":"The paper introduces 'new periodic embeddings' in Appendix B but does not compare them against prior PLR or PBLD embeddings. A brief comparison or justification for why a new variant was needed would help readers.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The stress-test concern about circularity of the default ranges is valid and is the most important issue to address. However, I assess it as a generalizability concern rather than a load-bearing internal inconsistency — the claims as stated for the evaluated benchmarks are well-supported by the data, and the independent TabArena and large-scale evaluations provide genuine (if imperfect) mitigation. The TabArena ensemble-score switch is the more concrete concern and should be addressed directly. I recommend minor revision because the central claim is defensible and the issues are addressable through clarification and additional discussion rather than new experiments."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The three major comments all raise legitimate points about the strength of the 'tuning-free' claim and the statistical interpretation of our ablation. We address each below and will revise the manuscript accordingly.","responses":[{"response":"The referee raises a valid concern. We will add an explicit description of how the sampling ranges in Table 7 were chosen. To be transparent: the ranges were designed based on our prior experience with tabular MLPs (Gorishniy et al. 2025) and general intuition about reasonable hyperparameter values for this model family, not through systematic optimization on the 17 benchmark datasets. For example, the depth range UniformInt[1,3] and width 384 reflect typical values from prior work; the Muon learning rate range LogUniform[1e-3,1e-1] matches the tuning space used for TabM (Table 11). We agree that some informal iteration on range design is difficult to fully rule out, and we will acknowledge this explicitly in the revised manuscript. The independent evaluations in Appendix A.1 (large-scale datasets) and A.2 (TabArena) provide partial mitigation, as these datasets are not part of the main benchmark. We will state all of this clearly so readers can assess the claim for themselves.","revision_made":"yes","referee_comment":"§3.7 and Table 7: The central 'tuning-free' claim depends on the default hyperparameter sampling ranges. The paper does not describe how these ranges were selected. Given that the benchmark and the method share authors, there is a risk that the ranges were refined based on performance on these same 17 datasets, which would make the 'tuning-free' claim partially circular."},{"response":"This is a fair point. We will take option (b): explicitly acknowledge this adjustment as a limitation and discuss when users should expect to make it. Specifically, we will note that on datasets with small validation sets (as is common in TabArena), the validation metric becomes a noisier signal for greedy ensemble selection, and using training loss can be more stable. We will add a brief discussion of this trade-off in the revised manuscript. Running TabArena with the default validation-metric ensemble score and reporting the performance drop (option a) is a reasonable suggestion, and we will attempt it if time permits before the camera-ready deadline, but we commit to at least the explicit acknowledgment as a limitation.","revision_made":"partial","referee_comment":"§3.5 and Appendix A.2 implementation note: The switch from validation metric to training loss as the ensemble score for TabArena is presented as a practical adjustment, but it is a dataset-specific hyperparameter choice that sits in tension with the 'tuning-free' framing. The paper should either (a) run TabArena with the default validation-metric ensemble score and report the resulting performance drop, or (b) explicitly acknowledge this as a limitation and discuss when users should expect to make such adjustments."},{"response":"The referee is correct that the rank difference between TabPack_SameHP (1.4±0.7) and TabPack (1.6±0.6) is within one standard deviation and that we did not test for statistical significance. We will revise the discussion in §5.1 to be more careful: rather than implying that tuning 'slightly improves' performance, we will state that TabPack_SameHP performs comparably to TabPack, with no statistically significant difference detected, and that the main benefit of the tuning-free default is the dramatic reduction in runtime (2.4h vs 65.5h) rather than any performance sacrifice. We will also add a note that the comparison is underpowered for detecting small differences given the number of datasets. The core claim — that random hyperparameter sampling is a reasonable default strategy — is supported by the fact that it achieves statistically indistinguishable performance at a fraction of the cost, not by claiming it outperforms tuning.","revision_made":"yes","referee_comment":"§5.1, Table 1: TabPack_SameHP achieves rank 1.4±0.7 vs TabPack's 1.6±0.6, meaning traditional per-dataset tuning of TabPack's architecture actually outperforms the tuning-free default. The paper frames this as 'the tuning slightly improves the performance at the cost of noticeable increase of the total runtime.' However, the rank difference (1.4 vs 1.6) is within one standard deviation, and the paper does not test whether this difference is statistically significant."}],"tokens_in":23481,"tokens_out":953,"duration_ms":161785,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"TabPack's core idea is simple and new: instead of packing homogeneous MLPs (as TabM does), pack MLPs with different architectures and optimizer hyperparameters into one vectorized system, train them in parallel, and greedily select ensemble members online during training. This gives you an ensemble that doesn't need per-dataset tuning — you specify ranges, not exact values. That's a real contribution to the tabular DL toolkit, and the engineering is solid: the model pack and optimizer pack constructions are clean, the online ensemble selection is well-motivated, and the whole thing ships as code. The experimental work is thorough — 17 datasets, strong baselines with 100+ tuning iterations, a conservative multi-seed protocol that actually penalizes TabPack, per-dataset results with standard deviations, ablations, and additional benchmarks (TabArena, large-scale datasets). The MacBook experiment is a nice touch that grounds the compute claims. The central result holds: default TabPack† achieves mean rank 2.6, slightly better than tuned TabM† at 2.8, at a fraction of the runtime (2.4h vs 47h). Figure 7 showing TabPack dominating across time budgets is particularly convincing. The paper also honestly shows (Table 1) that tuning TabPack's ranges per-dataset gives a small additional gain — so the tuning-free story is about diminishing returns, not magic. The soft spot is real but bounded. The default sampling ranges in Table 7 were designed by the same group that built the benchmark, and the paper doesn't describe how they were chosen. If these ranges were refined on these 17 datasets, the 'tuning-free' claim has a circularity problem — tuning happened at the range-design level. The TabArena evaluation (Appendix A.2) provides partial independent validation, but there the authors switch the ensemble score from validation metric to training loss, which is itself a dataset-specific adjustment that undercuts the no-tuning framing. That said, the ranges are reasonably broad, and the TabArena results do show competitive performance on an independent benchmark. This is a generalizability concern, not an internal inconsistency — the claims as stated for the evaluated datasets are well-supported. Who is this for? Practitioners and researchers working on tabular DL who want a strong default without tuning cycles. It deserves a serious referee. The reviewer should push the authors to (a) document how the ranges were designed, and (b) address the TabArena ensemble-score switch more transparently. Neither issue is load-bearing, but both affect the framing.","headline":"TabPack packs MLPs with heterogeneous hyperparameters into one vectorized system with online greedy ensemble selection — a genuinely new pattern for tabular DL that works well in practice.","tokens_in":24368,"tokens_out":1165,"would_cite":true,"duration_ms":101197,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Tuning-free tabular model matches tuned baselines at 1/20th the compute","keywords":[],"falsifier":"Run TabPack on a new tabular domain where the optimal hyperparameters fall outside the default sampling ranges (e.g., where depth >3 or learning rates outside the specified log-uniform interval are needed) and check whether mean rank degrades relative to tuned baselines.","tokens_in":23561,"feed_emoji":"📊","tokens_out":1074,"duration_ms":82441,"temperature":0.7,"pith_summary":"TabPack replaces hyperparameter tuning with hyperparameter sampling. Instead of searching for the best single configuration of a multilayer perceptron (MLP) for a tabular dataset, TabPack packs dozens of MLPs with randomly sampled hyperparameters into one training run, trains them all in parallel through vectorized operations, and greedily selects the best ensemble members on the fly during training. The central claim is that this single tuning-free run achieves mean rank 2.6 across 17 datasets, matching or beating extensively tuned prior methods like TabM (rank 2.8) and RealMLP (rank 3.6), while requiring 2.4 hours of total compute versus 47.2 and 250.9 hours respectively. The key technical object is the packed hyperparameter ensemble: a single module containing m MLPs that differ in depth, dropout, learning rate, weight decay, and embedding parameters, all trained simultaneously and pruned selectively. The paper argues that the main value of hyperparameter diversity is not improving ensemble quality beyond what tuned same-hyperparameter ensembles achieve, but rather eliminating the need to tune at all.","feed_headline":"One-run tabular model matches tuned baselines at 1/20th the compute","feed_subtitle":"TabPack samples dozens of MLPs with different hyperparameters in parallel and greedily selects the best ensemble, eliminating the tuning","key_machinery":"Packed hyperparameter ensembling: stacking m MLPs with different hyperparameters into a single tensor module, training them in parallel via batched matrix multiplications, and selecting ensemble members greedily during training using validation-set performance as the ensemble score.","core_discovery":"The paper shows that for tabular deep learning, the traditional hyperparameter tuning loop can be replaced by a single run that samples many configurations in parallel and lets greedy ensemble selection find what works. The authors demonstrate this with a packed-ensemble architecture where heterogeneous MLPs are stacked along a new dimension and trained together via batched matrix operations, with ensemble members chosen online based on validation performance. The result is that default TabPack on a MacBook runs faster than tuning some baselines on an A100 GPU, while achieving comparable or better mean rank across 17 medium-to-large datasets. The paper also finds that online ensemble (re-e-e","pith_inferences":[],"forward_implications":["Practitioners can skip expensive hyperparameter search for tabular MLPs by running TabPack once with default sampling ranges, reducing experiment cycles from days to hours.","The packed-ensemble pattern could extend to other parts of the training pipeline — loss functions, data preprocessing, architectural choices — diversifying any vectorizable degree of freedom without separate training runs.","The finding that hyperparameter diversity primarily enables tuning-free workflows rather than improving raw ensemble quality suggests that future tabular architectures should be designed with built-in configuration robustness rather than relying on external search.","The ability to run competitively on consumer hardware (MacBook M4 Pro) shifts the accessibility boundary for tabular deep learning, potentially broadening adoption beyond GPU-equipped research labs."],"fun_headline_variants":["TabPack replaces tabular hyperparameter tuning with a single parallel run","Single-run MLP ensemble matches tuned tabular baselines without GPU tuning","TabPack skips tabular tuning via parallel MLP ensembles and greedy selection","Untuned TabPack ensemble matches GPU-tuned tabular baselines on a MacBook","Parallel hyperparameter ensembles remove the tuning loop for tabular models"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The claim that TabPack is tuning-free rests on the assumption that the authors' default hyperparameter sampling ranges (e.g., depth 1-3, width 384, learning rate from a fixed log-uniform interval) are good enough across all tabular datasets. If these ranges are poorly suited to a new domain, performance could degrade with no recourse beyond adjusting the ranges — which is itself a form of tuning. The paper evaluates on 17 datasets but does not test sensitivity to range miss","fun_headline_variants_meta":{"raw":{"variants":["TabPack replaces tabular hyperparameter tuning with a single parallel run","Single-run MLP ensemble matches tuned tabular baselines without GPU tuning","TabPack skips tabular tuning via parallel MLP ensembles and greedy selection","Untuned TabPack ensemble matches GPU-tuned tabular baselines on a MacBook","Parallel hyperparameter ensembles remove the tuning loop for tabular models"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1091,"prompt_tokens":503,"completion_tokens":588,"prompt_tokens_details":null},"tokens_in":503,"tokens_out":588,"duration_ms":11795,"temperature":1.0,"reasoning_tokens":574,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T13:39:27.036340+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Run TabPack on a new tabular domain where the optimal hyperparameters fall outside the default sampling ranges (e.g., where depth >3 or learning rates outside the specified log-uniform interval are needed) and check whether mean rank degrades relative to tuned baselines.","supporting_citations":[],"review_version":1}