{"id":"64e67b75-651c-453a-b3bc-18f67b3af4c9","arxiv_id":"2501.18990","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A new permutation rank test, MPRT, controls Type I error for cross-covariance rank testing even in the presence of discretized variables.","lead":"This paper presents a permutation-based test for the rank of a cross-covariance matrix that works when some variables are only observed as ordered categories. It is aimed at causal discovery with mixed continuous and discretized data, where standard rank tests fail to control false positives.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 5's consistency claim for the permuted cross-covariance is unproven: the pseudo-likelihood estimator is applied to permuted pairs that are not i.i.d. from the Gaussian-copula model, so the null distribution estimated by MPRT may be incorrect.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The most load-bearing condition for the central Type I error claim is the consistency of the permuted statistic estimation in Theorem 5. The proof in Appendix A.3 assumes that the pseudo-likelihood estimator applied to \\tilde D_X and P^T \\tilde D_Y consistently estimates D_X^T P^T D_Y/(N-1). This is not automatically true: (i) under H_k^0 with k>0, the full variables X and Y are not independent, so the permuted pairs are not i.i.d. from a joint Gaussian distribution; (ii) even when X and Y are independent, the permutation without replacement makes the pairs dependent; (iii) the MLE estimates the population cross-covariance (zero under permutation), not the sample cross-covariance that determines the permutation null distribution. These points are not addressed in the proof. The synthetic experiments are encouraging and provide partial support, but they do not resolve the theoretical gap, especially because the experimental settings may not stress the misspecification (e.g., they use three categories and moderate sample sizes). We therefore agree with the reader's weakest_assumption. No independent computation or formal verification is reported. A targeted simulation comparing the MPRT permutation null to the oracle null would settle whether the concern lands. If the distributions match, the paper's conditional acceptance is justified; if not, the Type I error claim would need revision.","tokens_in":16879,"tokens_out":12784,"duration_ms":118206,"concrete_test":"Simulate N=1000 i.i.d. latent Gaussian pairs (X,Y) with rank(Σ_{X,Y})=1 (so H_0^1 holds but X,Y are dependent). Discretize Y into 3 bins using fixed thresholds. For B=2000 random permutations P, compute (a) the oracle singular values of the CCA matrix between permuted tail components using the latent continuous data, and (b) the MPRT permuted singular values from Eq. 8 using the discretized data and the pseudo-likelihood MLE. Compare the empirical distributions of the largest singular value (or the statistic λ^P_k) between (a) and (b). If the distributions differ by more than Monte Carlo error (e.g., a Kolmogorov-Smirnov test with α=0.01), Theorem 5's consistency claim fails and Type I error control is not guaranteed. Additionally, compute MPRT's empirical Type I error at α=0.05 under this setting; if it deviates from 0.05 by more than sampling error, the central claim is contradicted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that MPRT controls Type I error with discretized data rests on Theorem 5 (Appendix A.3), which asserts that the permuted sample cross-covariance D_X^T P^T D_Y/(N-1) can be consistently estimated by the Gaussian pseudo-likelihood estimator (Eq. 17) applied to \\tilde D_X and P^T \\tilde D_Y. The proof argues that \\tilde D_X and P^T \\tilde D_Y can be taken as samples from independent Gaussian vectors, and therefore are jointly Gaussian. This is problematic in two ways. First, under H_k^0 with k>0, X and Y are not independent; only the tail canonical components CXk: and CYk: are asymptotically independent. The full variables X and Y can have rank-k dependence, so the permuted pairs (X_i, Y_{π(i)}) are not i.i.d. from a joint Gaussian, and marginal Gaussianity does not imply joint Gaussianity without independence. Second, even under full independence, the permuted pairs are a without-replacement permutation of an i.i.d. sample and are not i.i.d. themselves. More fundamentally, the pseudo-likelihood MLE is consistent for the population cross-covariance—zero under permutation—not for the random sample quantity D_X^T P^T D_Y/(N-1). The permutation null distribution is determined by the sampling fluctuations of the latter; substituting a population-consistent estimator can change the null distribution and break Type I error control. The appendix provides no argument that the MLE has the same limiting distribution as the sample cross-covariance, nor does it quantify the misspecification error. Thus the theoretical guarantee for the mixed/discretized case is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MPRT (Mixed data Permutation-based Rank Test), a permutation test for the rank of a cross-covariance matrix that is designed to remain valid when some or all variables are observed only through order-preserving discretization. The method first estimates the correlation matrix by a pairwise Gaussian pseudo-likelihood, computes CCA projections, and then estimates the null distribution of the rank statistic by randomly permuting one block of the data. The paper claims that the method controls Type I error in the presence of discretization, which existing CCA-based rank tests do not, and demonstrates the method on synthetic data, on the PC algorithm, and on a Big Five personality dataset. The theoretical support consists of an exchangeability result for the trailing canonical variables (Theorem 4) and a consistency result for the permutation statistic under discretization (Theorem 5).","tokens_in":17220,"tokens_out":7199,"duration_ms":73388,"significance":"If the theoretical claims were fully established, the paper would make a useful contribution: rank tests have applications in causal discovery, and a permutation-based version that handles discretized variables would address a real gap in the literature. The experimental results are consistent with the paper's Type I error claim, and the inclusion of real-data causal discovery is a strength. However, the central proof of Theorem 5 is not valid as written: it conflates consistency to a population quantity with the distributional approximation required for a permutation null. Because the abstract's headline claim of Type I error control rests on that theorem, the significance is currently conditional on a substantial proof repair.","major_comments":[{"comment":"The proof of Theorem 5 does not establish the claimed consistency. The final step asserts that \\tilde D_X and P^T \\tilde D_Y can be treated as samples from independent Gaussian vectors, and therefore that D_X^T P^T D_Y/(N-1) can be consistently estimated by the pseudo-likelihood estimator. This conflates two different targets. A consistent estimator of the cross-covariance under the permutation null converges to the population cross-covariance, which is zero, whereas the permutation statistic \\lambda_k^P depends on the random sample cross-covariance whose fluctuations of order N^{-1/2} define the null distribution. Replacing the sample quantity by an estimate that converges to zero would make the permutation distribution degenerate and cannot yield a valid p-value. Moreover, the permuted pairs are not i.i.d. from a joint Gaussian: permutation without replacement introduces dependence across pairs, and when k>0 the full vectors X and Y are not independent even though their trailing canonical variables are. A valid proof would need to show that the estimated permutation statistic has the same limiting distribution as the statistic computed from the unobserved continuous permuted data, not merely that it converges to a population quantity.","section":"Appendix A.3, Theorem 5, Eq. (8)"},{"comment":"The proof of Theorem 4 invokes the continuous mapping theorem for SVD under the assumption of no repeated singular values. Under the null hypothesis H_k^0, however, the matrix \\Sigma_X^{-1/2} \\Sigma_{X,Y} \\Sigma_Y^{-1/2} has K-k zero singular values, which are repeated whenever K-k>1. The singular vectors spanning the zero-singular-value subspace are not unique, so the claimed convergence A \\to A_* and B \\to B_* for the trailing columns is not justified. Since the statistic \\lambda_k in Lemma 1 uses exactly those trailing canonical correlations, the paper needs to argue either that the test statistic is invariant to rotations within the zero-singular-value subspace or that the permutation null is consistently estimated despite the non-identifiability. This gap affects the validity of the permutation approach even in the all-continuous case.","section":"Appendix A.1, Theorem 4"},{"comment":"The estimator in Eq. (8) uses A and B that are themselves estimated from the original data, but the proof treats them as fixed matrices. In the actual procedure, A and B are random and depend on the same data that are subsequently permuted, so the permutation distribution must account for this dependence. As stated, the consistency claim is not a well-defined asymptotic statement about the permutation statistic, because the probability space over which the convergence holds is not specified and the randomness of A and B is not separated from the randomness of the permutation.","section":"Section 3.2, Theorem 5, Eq. (8)"}],"minor_comments":[{"comment":"The section heading contains a typo: \"Takses\" should be \"Takes\".","section":"Section 2.2"},{"comment":"The word \"likeilhood\" should be \"likelihood\".","section":"Appendix A.3"},{"comment":"Step 8 says to calculate \\lambda_k^P following Eq. (4), but the statistic used for the permutation is defined in Eq. (6); this is inconsistent with the text and should be corrected.","section":"Algorithm 1, line 8"},{"comment":"The phrase \"cannot be not compared\" is a double negative and should be rewritten, for example as \"cannot be compared\".","section":"Section 4.3"},{"comment":"The Type I and Type II error plots do not include confidence intervals or standard error bars; with 3000 trials the standard error is small, but reporting it would make the comparison more informative.","section":"Figures 2 and 3"},{"comment":"The description of threshold generation says \"uniformly generate two thresholds from [-1.5,1.5]\" but does not specify that the thresholds are sorted before discretization; please clarify.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The reader's main concern is valid: Theorem 5 proves at most consistency to a population cross-covariance, not the distributional approximation that a permutation null requires. The repeated-singular-value issue in Theorem 4 is also a genuine technical gap. I believe the method may be salvageable, but the revision will need a substantive proof overhaul rather than minor corrections. The experimental evidence alone is not sufficient to support the central theoretical claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes MPRT, a permutation-based rank test for cross-covariance matrices when some or all variables are discretized. That is a real gap: existing rank tests assume perfect measurement, and the common workaround of estimating correlations first and plugging them into a classical test does not control Type I error. The paper identifies this cleanly and the empirical section backs the claim: MPRT tracks the nominal level across sample sizes, in mixed and fully continuous settings, and the PC/real-data applications are sensible.\n\nThe continuous-case logic is standard and solid: under the null, the trailing canonical variables are asymptotically independent, so permutation gives a valid null distribution. That part is fine. The discretized case, however, rests on Theorem 5, and the proof has a load-bearing gap. The appendix asserts that permuted discretized data can be treated as samples from independent Gaussian vectors and therefore jointly Gaussian. That is not true in general, particularly under H_k^0 with k>0, where X and Y still have rank-k dependence; only the trailing canonical components are independent. And even under full independence, a without-replacement permutation of an i.i.d. sample is not itself an i.i.d. sample. More fundamentally, the pseudo-likelihood estimator is consistent for the population cross-covariance (which is zero after permutation), not for the random sample quantity D_X^T P^T D_Y/(N-1). The permutation null distribution is driven by those sampling fluctuations, and the proof does not show the MLE-based statistic has the same asymptotic distribution. Without that, the central Type I error guarantee for the discretized case is not established.\n\nI want to be clear about proportion: this is not a papers with a fabricated flaw. The experiments are consistent with the claim, and the method may well work in practice. But for a statistical test, asymptotic validity is the main selling point, and the proof is not there yet. The missing power analysis is minor by comparison and explicitly acknowledged.\n\nWho is this for? Anyone in causal discovery, psychometrics, or econometrics working with ordinal or discretized measurements. It is an important problem and the paper deserves a real referee. My recommendation: send it to peer review, but the reviewers should push for a corrected or replaced proof of Theorem 5, or at minimum a much more careful argument plus additional simulation diagnostics on the permutation null distribution.","headline":"A genuinely new permutation-based rank test for discretized mixed data with strong empirical support, but the core proof of the permuted-statistic consistency has a gap that leaves the theoretical Type I error claim unproven.","tokens_in":17778,"tokens_out":2202,"would_cite":false,"duration_ms":24464,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G10","62G09","62H20","62H17"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that permutation-based null-distribution estimation makes rank tests of cross-covariance matrices valid when some or all variables are discretized.","keywords":["permutation rank test","discretization","mixed data","causal discovery","cross-covariance rank","polychoric correlation","polyserial correlation","canonical correlation analysis"],"falsifier":"Generate independent standard Gaussian $X$ and $Y$, discretize both into three ordinal categories, fix a random permutation $P$, and compare the likelihood-based estimate of $(D_X^T P^T D_Y)/(N-1)$ with the actual sample value as $N$ grows; a persistent bias would falsify Theorem 5. A complementary check is that, under a true null, MPRT p-values must be uniform across repeated trials, so a Type I error rate systematically above the nominal level would refute the central claim.","tokens_in":16683,"feed_emoji":"📊","tokens_out":10429,"duration_ms":88137,"temperature":0.7,"pith_summary":"Statistical tests of the rank of a cross-covariance matrix are used in causal discovery, but they assume continuous measurements. This paper introduces MPRT, a permutation-based rank test for the realistic case where some or all variables arrive as ordinal, discretized values. The central claim is that the null distribution can be estimated by permuting the discretized data and re-estimating the canonical correlation scores, so the test controls the Type I error rate (false rejections) where plug-in CCA tests fail. That matters because psychometric, financial, and biomedical data are routinely discretized, and rank tests also subsume partial-correlation conditional independence tests.","feed_headline":"Permutation rank test controls Type I errors on discretized data","feed_subtitle":"MPRT permutes discretized data to build the null distribution, so rank tests stay valid with ordinal variables.","key_machinery":"The engine is the exchangeability of the trailing canonical variables under the null, combined with Equation (8), which expresses the permuted CCA scores as singular values of a matrix whose cross term $A^T (D_X^T P^T D_Y/(N-1)) B$ can be estimated from discretized data. The pairwise pseudo-likelihood (a tractable sum of pair-wise log-likelihoods, with polyserial terms for continuous-discrete pairs and polychoric terms for discrete-discrete pairs) supplies the cross-covariance estimate under a unit-variance Gaussian assumption, and a spherical parameterization of correlation matrices keeps the estimate in the pseudo-correlation space needed for SVD. This lets the permutation distribution be computed without ever applying a linear transformation to ordinal columns.","core_discovery":"Under the null hypothesis $H_0^k$ that $\\operatorname{rank}(\\Sigma_{X,Y}) \\le k$, the paper establishes asymptotic exchangeability of the trailing canonical variables $C_{X,k:}$ and $C_{Y,k:}$ even when $X$ or $Y$ contains discretized components. Since ordinal columns cannot be linearly transformed into canonical scores, it proves (Theorem 5) that the permuted CCA scores can instead be consistently estimated from permuted discretized data through likelihood-based cross-covariance estimation. The consequence is that the empirical permutation distribution of Anderson's likelihood-ratio statistic is an asymptotically valid null distribution, so MPRT controls Type I error in fully continuous, partially discretized, and fully discretized settings.","pith_inferences":["A natural extension the paper does not pursue is replacing the Gaussian pseudo-likelihood with a nonparametric copula estimator, which could retain Type I error control when the parametric form of the underlying distribution is unknown.","The load-bearing consistency step for permuted pairs could be tested directly: for a fixed permutation, compare the likelihood-based estimate of $D_X^T P^T D_Y/(N-1)$ to the actual sample quantity as $N$ grows; this isolates whether the discretized permutation trick is really delivering the claimed null distribution.","If the permutation trick works for likelihood-based covariance estimates, the same construction may transfer to other consistent estimators of mixed-data correlation, such as rank-based or moment-based estimators, broadening the method's applicability."],"forward_implications":["MPRT extends valid rank testing to all three data regimes: fully continuous, partially discretized, and fully discretized variables, with the continuous case as a special case.","Used inside constraint-based causal discovery such as the PC algorithm, MPRT recovers skeletons more accurately than Fisher-Z or plug-in CCA tests when some variables are discretized, according to the paper's experiments.","Fields that routinely collect ordinal data, such as psychometrics and biometrics, gain a rank test whose Type I error is controlled without needing continuous measurements.","The validity holds for any discretization with at least two categories, and more categories improve power because less information is lost."],"supporting_citations":[{"why":"It supplies the likelihood-ratio statistic and the CCA framework that MPRT permutes.","marker":"Anderson, 1984"},{"why":"It gives the polychoric maximum-likelihood estimator for correlations between discretized pairs.","marker":"Olsson, 1979"},{"why":"It gives the polyserial maximum-likelihood estimator for continuous-discrete pairs.","marker":"Olsson et al., 1982"},{"why":"It provides the pseudo-likelihood consistency rationale for the pairwise objective.","marker":"Besag, 1974"},{"why":"It supplies pseudo-maximum-likelihood consistency used to justify estimating cross-covariances from discretized data.","marker":"Gourieroux et al., 1984"},{"why":"It supplies a consistent mixed-data latent graphical model estimator that supports the pseudo-likelihood approach.","marker":"Fan et al., 2017"},{"why":"It provides the spherical parameterization of correlation matrices used to enforce the pseudo-correlation constraint.","marker":"Rousseeuw & Molenberghs, 1993"}],"fun_headline_variants":["Rank test stays valid with discretized variables","Permutation test for causal discovery with mixed data","MPRT: robust rank test for discretized variables","Permuting discretized data for correct rank tests","Causal discovery with discretized data made reliable"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes that fitting the Gaussian model to discretized $X$ and a permuted, discretized copy of $Y$ gives a consistent estimate of the true cross-covariance $D_X^T P^T D_Y/(N-1)$, even though the permuted pairs are not i.i.d. draws from any joint Gaussian.","fun_headline_variants_meta":{"raw":{"variants":["Rank test stays valid with discretized variables","Permutation test for causal discovery with mixed data","MPRT: robust rank test for discretized variables","Permuting discretized data for correct rank tests","Causal discovery with discretized data made reliable"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000466,"raw_usage":{"total_tokens":2301,"prompt_tokens":894,"completion_tokens":1407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":1336}},"tokens_in":510,"tokens_out":1407,"duration_ms":8420,"temperature":1.0,"reasoning_tokens":1336,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:42:55.770315+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate independent standard Gaussian $X$ and $Y$, discretize both into three ordinal categories, fix a random permutation $P$, and compare the likelihood-based estimate of $(D_X^T P^T D_Y)/(N-1)$ with the actual sample value as $N$ grows; a persistent bias would falsify Theorem 5. A complementary check is that, under a true null, MPRT p-values must be uniform across repeated trials, so a Type I error rate systematically above the nominal level would refute the central claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the likelihood-ratio statistic and the CCA framework that MPRT permutes."},{"cited_title":"Maximum likelihood estimation of the polychoric correlation coefficient","cited_arxiv_id":null,"evidence_quote":"It gives the polychoric maximum-likelihood estimator for correlations between discretized pairs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It gives the polyserial maximum-likelihood estimator for continuous-discrete pairs."},{"cited_title":"Spatial interaction and the statistical analysis of lattice systems","cited_arxiv_id":null,"evidence_quote":"It provides the pseudo-likelihood consistency rationale for the pairwise objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the spherical parameterization of correlation matrices used to enforce the pseudo-correlation constraint."}],"review_version":1}