Pith. sign in

REVIEW 4 major objections 4 minor 18 references

Encoding categorical variables as smoothed logit coordinates, rather than one-hot vectors, improves fidelity of continuous generative models on mixed tabular data, especially under rare-category imbalance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:40 UTC pith:PWHNURJX

load-bearing objection Honest and solid within-family empirical study of logit coordinates for mixed tabular generation, but the rate theory covers only the componentwise generator while the headline gains come from the flat joint model, so the experiments carry the main claim alone. the 4 major comments →

arxiv 2607.23348 v1 pith:PWHNURJX submitted 2026-07-25 stat.ML cs.LG

Logit-Coordinate Generative Models for Mixed Continuous-Categorical Tabular Data

Yuefei Shen , Xiaotong Shen This is my paper
classification stat.ML cs.LG MSC 62B1062G0562G2062H30
keywords mixed tabular datalogit coordinatesflow matchingdiffusion modelsrare-category imbalanceinformation geometrynonparametric ratesWasserstein discrepancy
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that how categorical variables are embedded into Euclidean space is a consequential modeling choice for continuous generative models trained on mixed tabular data. It proposes embedding categories as smoothed natural parameters (logit coordinates) instead of one-hot vectors, then running standard Flow Matching or Gaussian diffusion in the resulting space. If the paper is right, a simple coordinate swap—no change to architecture or training objective—improves fidelity on imbalanced mixed data, and the theoretical analysis explains why: logit coordinates give local quadratic control of KL divergence, while one-hot Euclidean error does not. The paper also introduces a mixed-distribution discrepancy that separates categorical mass error from conditional continuous transport error, and derives nonparametric rates that make rare-cell sample-size loss explicit.

Core claim

Embedding categorical variables via smoothed additive log-ratio coordinates eta_k = log(q_k / q_r) turns the categorical part of mixed data into an unconstrained Euclidean vector while preserving exponential-family structure. For this representation, the paper proves a stability bound of the form Dmix <= eps + C1 n^{-alpha_Z/(2alpha_Z+d_Z+1)} + sum_k pi*_k C2,k (n pi*_k)^{-alpha_c/(2alpha_c+p+1)}, making explicit that rare cells learn from effective sample size n pi*_k. Empirically, within-family comparisons across ten splits show Logit Flow Matching improves or matches One-Hot FM on the primary distributional metrics on three of four datasets; Block-Conditional Logit FM consistently improve

What carries the argument

The additive log-ratio (ALR) coordinate map eta_k = log(q_k / q_r) with softmax decoding, combined with the mixed discrepancy Dmix = TV(pi*, pi_hat) + sum_k pi*_k W2(P*_k, P_hat_k). The stability argument uses the Wasserstein transport bound W2(P*_1, P_hat_1) <= C * integral sqrt(E||v_hat - v*||^2) dt, plus the softmax Lipschitz bound ||S(eta) - S(eta')||_1 <= (sqrt(K)/2) ||eta - eta'||_2, which converts logit-space error into categorical probability error.

Load-bearing premise

The theoretical guarantees are proved for an idealized componentwise generator—a separate logit flow plus per-categorical-class continuous flows—not for the flat joint Logit FM that produces several headline empirical results; if flat joint training cannot satisfy the needed conditional-stability assumptions, the reported gains rest on the experiments alone.

What would settle it

Run the severe-imbalance simulation (rho = 0.01) with the ALR reference category cycled through all 24 joint cells and with the smoothing parameter eps varied over a grid; if there exists a reference choice or a eps value where One-Hot FM matches or beats Logit-FM on Dmix, the claimed representation-level advantage is not invariant to coordinate conventions.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Logit Flow Matching improves the primary distributional metrics over One-Hot Flow Matching on Adult, Cardio, and Buddy, and is comparable on Churn2.
  • Block-Conditional Logit Flow Matching consistently improves over flat Logit FM across all four datasets, showing that categorical representation and conditional factorization address complementary sources of error.
  • Logit Diffusion generally improves over or matches One-Hot Diffusion, with near ties on Buddy, suggesting the coordinate benefit transfers across generative dynamics.
  • Under severe rare-cell imbalance (rho = 0.01), logit coordinates reduce categorical total variation, Dmix, Dbal, conditional label error, and two-sample AUC relative to one-hot in controlled simulations.
  • The nonparametric rate shows rare-cell conditional components converge at rate (n pi*_k)^{-alpha_c/(2alpha_c+p+1)}, so preserving rare categorical mass is statistically harder than preserving common cells.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the representation effect is general, the same coordinate swap should help other Euclidean-space generative families on mixed tabular data—for example, score-based models with different noise schedules or variational autoencoders—though the paper does not test this.
  • Because ALR coordinates depend on a reference category, the reported logit gains could be partly reference-dependent; a permutation-invariant log-ratio formulation would test whether the improvement is intrinsic to natural-parameter structure or to the particular reference choice.
  • The rare-cell rate suggests a practical resource bound: with total sample n, cells with mass below roughly 1/n cannot support consistent conditional continuous estimation on their own, so generators should pool across rare cells or share structure—an extension the paper's block-conditional results hint at but do not develop.
  • A direct testable extension is to vary the smoothing parameter eps and the scaling constant a_K; if the logit advantage over one-hot disappears for large eps, the benefit is tied to boundary behavior rather than to natural-parameter geometry per se.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies a logit-coordinate (additive log-ratio) representation for categorical variables in mixed continuous–categorical tabular data, combined with Flow Matching or Gaussian diffusion. It introduces a mixed-distribution discrepancy Dmix that separates categorical marginal error from class-conditional continuous Wasserstein error, proves softmax Lipschitz and stability bounds, and derives nonparametric convergence rates for an idealized componentwise generator (a logit flow plus separate per-class continuous flows). Simulations compare logit vs one-hot embeddings under rare-cell imbalance, and real-data experiments on Adult, Churn2, Cardio, and Buddy compare flat and block-conditional logit FM, logit/one-hot diffusion, and TabDDPM baselines across ten splits. The paper claims that scaled-logit coordinates improve or match one-hot coordinates, especially under severe imbalance, and that block-conditional structure further improves fidelity.

Significance. If the results hold, the paper offers a useful representation-level insight: the coordinate system used to embed categorical variables materially affects the fidelity of continuous generative models on mixed tabular data. The Dmix criterion is a sensible decomposition, and the stability and rate arguments are standard and largely correct. The paper is transparent about several limitations—most importantly, that the main theory covers only a componentwise generator and not the flat joint Logit FM used for several headline experiments, and that the experiments use maximum-probability decoding while the theory controls randomized softmax decoding. The ten-split real-data benchmark is a practical strength, and the within-family comparisons (logit vs one-hot under fixed architectures) are carefully framed. However, the gap between theory and the headline experiments, together with the absence of a comparative theoretical guarantee, means the representation-level claim rests primarily on the empirical results.

major comments (4)
  1. [§5.4, §6, §7.2] The rate analysis and mixed-fidelity bounds (Corollary 5.7, Theorem 6.6) are proved for a componentwise generator—a separate logit flow plus per-class conditional continuous flows—whereas the headline Logit FM results come from flat joint models. This is stated explicitly in §5.4 ('For the flat joint model, Theorem 5.2 ... does not by itself give the class-conditional transport bounds assumed below') and in §6 ('This avoids treating the calculation as an implementation-level guarantee for the flat joint neural generator'). Since the real-data simulations and benchmarks use flat joint generators, the theory does not substantiate the main empirical comparisons; the logit-representation conclusion rests on the experiments alone. The paper should either reframe the theory as an idealized analysis that motivates, but does not explain, the flat-model results, or add experiments with the compon
  2. [§4.6, Theorem 6.6, Remark 6.7] The theoretical guarantees apply to randomized softmax decoding: Corollary 5.6 and Theorem 6.6 control Dmix for bX(d) ~ Categorical(p̂), while the experiments use maximum-probability decoding for both one-hot and scaled-logit variants (§4.6, §8.1). Remark 6.7 admits that maximum-probability decoding 'requires margin control' and is 'outside the scope of the theorem.' Thus the primary theoretical quantity Dmix is not the quantity evaluated in the experiments. The authors should either implement randomized softmax decoding in the experiments, or provide margin diagnostics showing that max-probability decoding is consistent with the theoretical decoder, or explicitly separate the theory from the experimental protocol.
  3. [§3.4, Theorem 6.6] The paper does not provide a formal argument that logit coordinates improve over one-hot coordinates. Proposition 3.1 and Proposition 3.4 compare KL control in natural-parameter space vs probability space, but the Flow Matching training objective is Euclidean in the embedding space and the decoding is argmax; no analogue of Theorem 6.6 is derived for one-hot embeddings, so no comparative rate is available. The abstract's claim 'scaled-logit coordinates improve or match one-hot coordinates, especially under severe rare-cell imbalance' is therefore an empirical assertion rather than a consequence of the theory. The authors should either derive a comparative bound (e.g., using margin or Lipschitz properties of one-hot decoding) or temper the theoretical framing so it does not imply a proof of the representation-level advantage.
  4. [Table 1, Table 3, §7.4] In the controlled simulations, the reported logit-vs-one-hot differences at severe imbalance are comparable to the standard deviations across the ten seeds. For example, at ρ=0.01, Logit-FM vs OneHot-FM gives Dmix 0.157±0.007 vs 0.165±0.008 and Dbal 0.195±0.007 vs 0.203±0.008; the diffusion comparisons show similar effect sizes. No significance tests or paired comparisons are reported. Given the abstract's emphasis on 'especially under severe rare-cell imbalance,' the authors should quantify the uncertainty (e.g., paired tests or confidence intervals) or restate the simulation claim as a directional improvement, rather than a definitive demonstration.
minor comments (4)
  1. [§4.2, §7, §8] The smoothing parameter ε and the logit scaling aK are central to the method, but the values used in the simulations and real-data experiments are not reported. This is needed for reproducibility and for interpreting the stability constants.
  2. [§5.3, Eq. (1)] The definition of Dmix assumes population distributions. For empirical evaluations, generated samples can have empty cells; the paper mentions a support convention but does not state which convention is used in the reported tables. Please specify.
  3. [§3.3, §7.2] The ALR reference category is chosen as a high-frequency category in the training data, and sensitivity to this choice is deferred. This should at least be documented per dataset, since the reference category can affect finite-sample training.
  4. [Throughout] Minor typographical issues: 'H¨ older' should be 'Hölder', 'ETrain' appears in §A.5, and some inline math is inconsistently formatted. These do not affect the substance.

Circularity Check

0 steps flagged

No significant circularity: the derivation chain is self-contained and the paper's own caveats concern coverage gaps, not circular reasoning.

full rationale

Walking the claimed derivation chain, I find no step in which a prediction reduces by construction to a fitted input, a self-defined criterion, or a self-citation. The mixed discrepancy Dmix is defined in Eq. (1) as TV(pi*, pi-hat) plus the pi*-weighted sum of class-conditional Wasserstein errors. This is an evaluation criterion; it is not used to fit any parameter, and the later bounds do not infer empirical success from the criterion's definition. Corollary 5.7 and Theorem 6.6 are honest mathematical consequences of the stability bound (Theorem 5.2), the softmax Lipschitz bound (Theorem 5.3), and standard nonparametric sieve regression rates (Lemma 6.3). The effective-sample-size terms (n pi*_k) appear because the class-conditional regressions are estimated from the samples in that cell; this is a derivation, not a relabeling. The scaling aK is set by the chosen smoothing epsilon, not fitted to experimental outcomes. There are no load-bearing self-citations: the references are to standard or external work (Amari, Lipman et al., Kotelnikov et al., Williams et al., etc.), and no uniqueness theorem from the present authors is invoked to force the representation choice. The paper is also transparent about the theory's scope: Section 5.4 states that the flat joint model is not covered by the class-conditional transport bounds, and Section 6 states that the sieve result is for the componentwise model, explicitly avoiding 'treating the calculation as an implementation-level guarantee for the flat joint neural generator.' These are acknowledged limitations of the theory-empirics link, not circular steps. No fitted input is renamed as a prediction, and no known result is merely renamed as organization. The empirical claims rest on controlled simulations and ten-split benchmarks; the theory provides supporting stability/rate statements for an idealized componentwise model. Thus the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claims rest on standard Gronwall, softmax-Lipschitz, and sieve-regression tools plus several unverified regularity assumptions and an idealized componentwise model. No new physical entities are introduced. The most load-bearing items are the componentwise-assumption mismatch and the unquantified density-ratio bound κ; there is little circularity from fitting.

free parameters (3)
  • smoothing parameter ε = not reported in experiments
    Required for logit embedding (Eq. 4.2); controls smoothing bias ε in Corollary 5.6 and Theorem 6.6 and the logit prototype magnitude aK; no numerical value is given for simulations or real data.
  • ALR reference category = high-frequency category chosen from training data
    ALR coordinates depend on the reference category (Section 3.3); paper notes finite-sample sensitivity left for future work; not a theoretical parameter but a modeling choice affecting optimization and decoded geometry.
  • logit scaling factor aK = derived from ε; exact value not reported
    Divides logits to produce scaled-logit coordinates z=η/aK (Section 7.2); changes the softmax Lipschitz constant Csm (Theorem 5.3) and hence the stability bound; user-controlled through ε.
axioms (5)
  • domain assumption True and estimated velocity/score fields are globally L-Lipschitz in state, uniformly in t (Assumption 5.4 / Prop 5.11).
    Required for Gronwall stability bounds Theorems 5.2/5.5 and Prop 5.11; not verified for neural networks or for trajectories leaving bounded domains.
  • domain assumption Componentwise flow structure: a separate logit flow plus per-class conditional continuous flows, with common base initialization (Cor 5.7, Thm 6.6).
    This is the model for which Dmix bounds and rates are proved; the flat joint Logit FM used in experiments is explicitly not covered (§5.4).
  • domain assumption Hölder smoothness αZ, αc of velocities on bounded domains, finite conditional variance, linear sieve classes (Assumption 6.2).
    Needed for the sieve regression rate Lemma 6.3; smoothness constants are not estimated or verified.
  • domain assumption Bounded density ratio dμ_path/dμ_train ≤ κ (Assumption 6.4).
    Bridges path error to training error in Lemma 6.5; κ is unquantified and can be arbitrarily large in principle.
  • ad hoc to paper Experiments use maximum-probability decoding while the theory controls randomized softmax decoding (Section 4.6, Remark 6.7).
    The paper states a margin condition would be needed for argmax perturbation guarantees; without it the theoretical bound does not directly apply to the decoding rule used in the experiments.

pith-pipeline@v1.3.0-alltime-deepseek · 29193 in / 16988 out tokens · 163806 ms · 2026-07-31T23:40:27.437947+00:00 · methodology

0 comments
read the original abstract

Mixed continuous--categorical data pose a representation problem for continuous generative models. Flow Matching and Gaussian diffusion operate in Euclidean spaces, whereas categorical laws lie on probability simplices and may be highly imbalanced. We study a logit-coordinate framework that encodes categorical variables as smoothed natural parameters and combines them with transformed numerical variables. This yields common formulations of Logit Flow Matching and Logit Diffusion. We introduce a mixed-distribution discrepancy separating categorical marginal error from conditional continuous Wasserstein error, and derive stability bounds and imbalance-aware nonparametric rates linking vector-field or drift error to decoded mixed-distribution error. Controlled simulations show that scaled-logit coordinates improve or match one-hot coordinates, especially under severe rare-cell imbalance. Across four real-data benchmarks and ten splits per dataset, Logit FM improves the primary distributional metrics on three datasets and is comparable on Churn2; Block-Conditional Logit FM consistently improves the flat model; and Logit Diffusion generally improves over or matches One-Hot Diffusion.

Figures

Figures reproduced from arXiv: 2607.23348 by Xiaotong Shen, Yuefei Shen.

Figure 1
Figure 1. Figure 1: Adult marginal numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p040_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Churn2 marginal numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p040_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cardio marginal numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p041_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Buddy marginal numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p041_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Adult marginal categorical distributions on the original benchmark split. Columns [PITH_FULL_IMAGE:figures/full_fig_p042_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Churn2 marginal categorical distributions on the original benchmark split. Columns [PITH_FULL_IMAGE:figures/full_fig_p042_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Cardio marginal categorical distributions on the original benchmark split. Columns [PITH_FULL_IMAGE:figures/full_fig_p043_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Buddy marginal categorical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p043_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Adult conditional numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p044_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Churn2 conditional numerical distributions on the original benchmark split. Columns [PITH_FULL_IMAGE:figures/full_fig_p044_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cardio conditional numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p045_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Buddy conditional numerical distributions on the original benchmark split. Columns show [PITH_FULL_IMAGE:figures/full_fig_p045_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 2 linked inside Pith

  1. [1]

    (2016).Information geometry and its applications

    Amari, S. (2016).Information geometry and its applications. Springer

    2016

  2. [2]

    (2000).Methods of information geometry

    Amari, S., and Nagaoka, H. (2000).Methods of information geometry. American Mathematical Society

    2000

  3. [3]

    Arjovsky, M., Chintala, S., and Bottou, L. (2017). Wasserstein GAN.Proceedings of the 34th International Conference on Machine Learning (ICML), 214–223

    2017

  4. [4]

    Cheng, C., Li, J., Peng, J., and Liu, G. (2024). Categorical flow matching on statistical manifolds. Advances in Neural Information Processing Systems

    2024

  5. [5]

    Fukumizu, K., Suzuki, T., Isobe, T., Oko, K., and Koyama, M. (2024). Flow matching achieves almost minimax optimal convergence.arXiv preprint arXiv:2405.20879. Guzm´ an-Cordero, A., Eijkelboom, F., and van de Meent, J.-W. (2025). Exponential family variational flow matching for tabular data generation.Proceedings of the 42nd International Conference on Ma...

    Pith/arXiv arXiv 2024

  6. [6]

    Hoogeboom, E., Nielsen, D., Jaini, P., Forr´ e, P., and Welling, M. (2021). Argmax flows and multino- mial diffusion: Learning categorical distributions.Advances in Neural Information Processing Systems

    2021

  7. [7]

    Jang, E., Gu, S., and Poole, B. (2017). Categorical reparameterization with Gumbel-Softmax. International Conference on Learning Representations (ICLR)

    2017

  8. [8]

    Kotelnikov, A., Baranchuk, D., Rubachev, I., and Babenko, A. (2023). TabDDPM: Modelling tabular data with diffusion models.Proceedings of the 40th International Conference on Machine Learning (ICML)

    2023

  9. [9]

    Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. (2022). Flow matching for generative modeling.Advances in Neural Information Processing Systems

    2022

  10. [10]

    Maddison, C., Mnih, A., and Teh, Y. W. (2017). The concrete distribution: A continuous relaxation of discrete random variables.International Conference on Learning Representations (ICLR)

    2017

  11. [11]

    Mueller, M., Gruber, K., and Fok, D. (2026). Cascaded flow matching for heterogeneous tabular data with mixed-type features.arXiv preprint arXiv:2601.22816. Peyr´ e, G., and Cuturi, M. (2019).Computational optimal transport.Foundations and Trends in Machine Learning, 11(5–6), 355–607

    Pith/arXiv arXiv 2026

  12. [12]

    I., Ambrogioni, L., and van de Meent, J.-W

    Roos, D., Davis, O., Eijkelboom, F., Bronstein, M., Welling, M., Ceylan, I. I., Ambrogioni, L., and van de Meent, J.-W. (2026). Categorical flow maps.arXiv preprint arXiv:2602.12233

    arXiv 2026

  13. [13]

    P., Kumar, A., and Ermon, S

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., and Ermon, S. (2021). Score-based genera- tive modeling through stochastic differential equations.International Conference on Learning Representations (ICLR)

    2021

  14. [14]

    Tsybakov, A. B. (2009).Introduction to nonparametric estimation. Springer. 46

    2009

  15. [15]

    (2009).Optimal transport: Old and new

    Villani, C. (2009).Optimal transport: Old and new. Springer

    2009

  16. [16]

    M., Hartmann, M., and Klami, A

    Williams, B., Yeom-Song, V. M., Hartmann, M., and Klami, A. (2025). Simplex-to-Euclidean bijections for categorical flow matching.arXiv preprint arXiv:2510.27480

    arXiv 2025

  17. [17]

    Xu, L., Skoularidou, M., Cuesta-Infante, A., and Veeramachaneni, K. (2019). Modeling tabular data using conditional GAN.Advances in Neural Information Processing Systems

    2019

  18. [18]

    Zhao, Z., Kunar, A., Birke, R., and Chen, L. Y. (2021). CTAB-GAN: Effective table data synthesizing. Proceedings of the 13th Asian Conference on Machine Learning, PMLR 157:97–112. 47

    2021