{"id":"ac5020fe-8d83-4625-bdf9-0f21309680e2","arxiv_id":"2501.03381","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"THOI is a PyTorch-based library that computes higher-order interaction metrics orders of magnitude faster than existing tools by batch-processing Gaussian-copula covariance matrices.","lead":"THOI is a new Python library that speeds up higher-order interaction analysis by batch-computing Gaussian-copula entropy estimates in PyTorch. The authors report that it can analyze systems with up to 30 variables in hours rather than weeks, but the code is not yet available.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The accuracy validation is circular: Supplementary A.5 computes ground-truth Omega with THOI itself, so any systematic bias in THOI's entropy or Omega implementation is shared by both reference and measured values; an independent reference implementation is required.","rationale":"The reader's weakest-assumption analysis identifies exactly the same load-bearing concern: the ground-truth Omega values used for validation are computed with THOI itself, making the validation unable to detect systematic errors in THOI's entropy or Omega implementation. My stress-test pass confirms this by locating the specific statement in Supplementary A.5 and tracing how the same code path is used for both reference and measured values. The paper's strongest claim is primarily about efficiency and scalability, and the performance numbers are not directly undermined by this concern. However, the accuracy validation is part of the central contribution: the library is presented as validated and reliable for HOI computation in real-world applications. If a systematic bug exists, the validation would not reveal it, so the evidence as presented is insufficient to establish accuracy independently. The proposed concrete test is straightforward and should settle the issue: compute ground-truth values for the Gaussian PGM systems using an independent implementation of Eq. 8 and compare against THOI. This does not require new data or new theory, only a re-implementation of a few standard formulas. Since the reader's verdict is already CONDITIONAL and this concern supports that judgment, no change to the verdict is needed: the paper should not be accepted unconditionally until the independent validation is performed and the implementation is made available. I agree fully with the reader's identification of the weakest assumption.","tokens_in":20453,"tokens_out":2064,"duration_ms":22636,"concrete_test":"Implement an independent reference for the Gaussian PGM systems in Supplementary A.5: for each exact covariance matrix Sigma, compute joint entropies as H = 0.5 * log((2*pi*e)^k * |Sigma|) using a separate linear algebra stack (e.g., numpy.linalg.slogdet or sympy), then assemble TC, DTC, Omega, and S via the definitions in Eqs. 4-7 without ever calling THOI. Next, generate finite samples from the same PGMs, compute Omega with THOI for the same n-plets, and compare THOI's output against the independent exact values. Also add a small analytic test, such as a 3-variable Gaussian with a known covariance matrix and a hand-derived Omega. If any discrepancy exceeds a numerical tolerance of about 1e-6 for exact-covariance inputs, the circular validation in the paper has masked an implementation error and the accuracy claim would need to be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim includes accurate computation of TC, DTC, Omega, and S-information, but the validation of this accuracy is circular. Supplementary A.5 states: \"Using the aforementioned ground-truth covariance matrices, the entropy equation (eq. 8) and THOI, we computes all the ground truth metrics of each system.\" The PGM covariance matrices are mathematically specified, so in principle they define exact Gaussian entropies, but the actual ground-truth values are computed by THOI's own implementation of Eq. 8 and Eq. 6. In the validation experiments, both the reference values and the THOI-estimated values pass through the same determinant, entropy, and Omega code paths. Consequently, any systematic error in those formulas—for example, a wrong coefficient in the Gaussian entropy, an incorrect determinant routine, or a sign error in the composition of Omega—would appear in both the 'ground truth' and the measured values and would cancel in the comparison. What the reported agreement can validate is the finite-sample behavior of the covariance estimator and the behavior of the greedy/simulated-annealing heuristics, but not the correctness of the metric computation itself. This is load-bearing because the paper presents THOI as a trustworthy tool for HOI analysis, not merely as a fast implementation of an independently verified formula. The absence of a code repository link in the manuscript compounds the problem, since the reader cannot check the implementation directly. The concern is not that the implementation is wrong, but that the evidence provided cannot detect a systematic error if one exists.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces THOI, a PyTorch-based Python library for computing higher-order information-theoretic measures (TC, DTC, O-information, S-information) from continuous multivariate data using Gaussian-copula entropy estimation. The main claims are: (i) batched, parallelized computation of all subsets for systems up to about 30 variables, with reported timings (5.8 hours for all orders 3–30 on a 30-variable system) and memory footprint below 3 GB; (ii) heuristic search (greedy and simulated annealing) for larger systems, validated on synthetic probabilistic graphical models with known redundancy/synergy structure; (iii) empirical demonstrations on fMRI data in wakefulness vs. deep anesthesia and on a benchmark of 920 datasets. The paper's central efficiency claim is supported by benchmarks against HOI_toolbox, HOI, and JIDT, while the accuracy claim is validated only against ground-truth values that are themselves computed with THOI's own implementation of the Gaussian entropy and O-information formulas.","tokens_in":20660,"tokens_out":1917,"duration_ms":19308,"significance":"If the efficiency claims hold, THOI is a practically useful tool: exhaustive higher-order interaction analysis for N up to ~30 is currently infeasible in most open-source libraries, and the paper demonstrates a concrete speedup and low memory footprint, with the additional strength that the software is designed to run on standard laptops and supports batched multi-dataset and on-the-fly analyses. The heuristic validation on synthetic PGM systems is also valuable because it tests whether greedy and simulated annealing recover known redundancy/synergy subsystems. However, the paper's accuracy validation is weakened by the circularity identified in Supplementary A.5, and the benchmark section lacks error bars and does not include a code repository link, which limits the reproducibility of the headline numerical claims.","major_comments":[{"comment":"The validation of metric accuracy is circular: Supplementary A.5 states 'Using the aforementioned ground-truth covariance matrices, the entropy equation (eq. 8) and THOI, we computes all the ground truth metrics of each system.' The PGM covariance matrices are analytically specified, but the ground-truth O-information values are evaluated with THOI's own implementation of Eq. 8 and Eq. 6. Any systematic bias in the determinant routine, the entropy constant, or the composition of Eq. 6 is shared by the reference and the measured values, so the reported agreement cannot detect such a bias. What the experiments do validate is the finite-sample behavior of the covariance/copula estimator and the behavior of the greedy and simulated-annealing heuristics. This point is load-bearing because the paper presents THOI as a tool whose accuracy on TC, DTC, O, and S is established; an independent reference implementation (e.g., analytic Gaussian entropies computed with a different library, or a non-parametric estimator on the same covariance-specified models) should be added to break the circularity.","section":"Supplement A.5 and §2.3"},{"comment":"The headline speed comparisons are not accompanied by error bars or repeated runs, and the comparison may not be apples-to-apples: THOI uses batched PyTorch operations, while HOI_toolbox, HOI, and JIDT are reported 'without leveraging additional parallelization' and, for JIDT, run in their default mode. The extrapolations (JIDT would take ~17 years, HOI_toolbox ~221 days) are presented without uncertainty ranges. Since the central efficiency claim is quantitative, the paper should report at least three repeated timings per condition, state the number of CPU threads used for each library, and clarify whether the comparison libraries were given any parallelization or memory-usage settings that would be fair relative to THOI.","section":"§2.2 and Figure 1B,C"},{"comment":"The padding derivation states H(Σk + I^{n-k}) = H(Σk) + H(I^{n-k}) because the added component is independent. This is correct only if the block-diagonal structure of the padded covariance matrix is preserved; the text notes that the sampled sub-covariance matrix and the independent components 'are not sorted to be separated, but maintain the original positions of the variables in the full original covariance matrix.' Under that construction the padded matrix is not block-diagonal with an identity block in a fixed corner, so the stated identity H(Σk) = H(Σk + I^{n-k}) − (n−k)·1.4189 requires justification. The degree of freedom in the placement of the independent components changes the determinant of the padded matrix only through row/column permutations, so the identity may still hold, but the paper should state this explicitly and either prove it or reference the permutation-invariance of the determinant.","section":"§3.4.1, Eq. for padding"},{"comment":"The fMRI analysis claims large-effect-size differences with Cohen's |d| > 3 and Wilcoxon p < 0.001, but these are explicitly uncorrected for multiple comparisons ('Wilcoxon p >0.001' is the stated threshold in Figure 3C and the text says 'not corrected'). Given that the GA explores many candidate n-plets at many orders, the reported p-values are likely inflated by selection; the paper should report the actual number of hypotheses tested and apply an appropriate multiple-comparison correction or state clearly that the results are exploratory. This does not undermine the library's computational contribution, but it is load-bearing for the paper's empirical claim that anesthesia 'compresses' the range of O-information.","section":"§2.4 and §3.6"},{"comment":"The manuscript never provides a code repository URL or version number for THOI, and the benchmark environment (Intel Core i9, 64 GB RAM, Linux Mint 21) does not specify the number of CPU cores or the PyTorch version. Since the paper is about a software library and the main claims are computational, the absence of a repository link prevents the reader from verifying the implementation, re-running the benchmarks, or breaking the circular validation described in Supplement A.5. A public repository with the benchmark scripts should be added before publication.","section":"General reproducibility"}],"minor_comments":[{"comment":"The abstract says 'we analyzed over 900 real-world and synthetic datasets,' while §2.5 reports 920 datasets; the numbers should be consistent.","section":"Abstract and §1"},{"comment":"The sentence 'The anesthesia analysis described in Section 3.6 was conducted using a publicly available dataset of 17 healthy adults' is self-referential; it should refer to Section 2.4 and should not describe the analysis in a section that is about the methods.","section":"§3.6"},{"comment":"Equation (16) has a typo: 'Σ(rank(rank(X))/(T+1))' appears to use rank twice, and the intended rank transformation is not clearly defined; the paper should define rank(·) precisely and fix the duplicate application.","section":"Supplement A.3.2, Eq. (16)"},{"comment":"In the enumeration of required terms, 'H(Xj)' is defined but the displayed list subsequent text speaks of 'the entire system,' which is fine; however, the notation H(X^n_{-j}) is introduced only in the list and should be defined before use in Eq. (6) or in the list itself.","section":"§3.3.1"},{"comment":"Figure 3C uses 'Wilcoxon p >0.001' as a threshold, but the text says 'no significant difference was found (Wilcoxon p >0.001)'; this is either a typo (p < 0.001) or a mis-statement of the test result, and it should be corrected.","section":"Figure 3 and §2.4"},{"comment":"Reference [30] is cited for the Gaussian copula estimator but the supplementary text has placeholder '[cite ince, cite]' and '[cite]' in Supplement A.3; these should be replaced with actual citations, or removed if redundant.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's overall direction is sound: a batched Torch implementation of Gaussian-copula entropy for higher-order information metrics is a genuinely useful contribution to the field, and the efficiency gains are plausible. The main reservation is the circular accuracy validation (Supplement A.5), which is fixable by adding an independent reference computation of the Gaussian ground-truth values; I would also ask the authors to include a code repository URL and to add repeated-run statistics to the benchmarks before publication. The fMRI analysis is secondary and exploratory, but the multiple-comparison issue should be addressed. If the authors provide the independent validation and the benchmark error bars, I would be willing to recommend acceptance; as it stands, the central accuracy claim is not yet substantiated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: THOI is a genuinely useful engineering contribution. The identity-padding trick for batching different interaction orders in one tensor operation is a real idea, and the headline result—exhaustively computing all HOI orders for a 30-variable system in 5.8 hours on a laptop—is a practical step that matters for people doing high-order interaction analysis on continuous data. The 920-dataset analysis in under 20 minutes also demonstrates real accessibility. The underlying measures (O-information, TC, DTC, S) and the Gaussian copula estimator are prior work, but the library packaging them with parallel batch processing is new and useful.\n\nThe soft spot is load-bearing: the accuracy validation is circular. Supplementary A.5 says the ground-truth Omega values for the PGM systems were computed using THOI's own implementation of the Gaussian entropy formula. The PGM covariance matrices mathematically determine the exact Gaussian entropies, but by running those through the same determinant/entropy/Omega code path as the estimator, any systematic error in THOI's formulas would appear in both the reference and the measured values and cancel. The reported agreement can validate the finite-sample behavior of the covariance estimator and the heuristics, but not the correctness of the metric computation itself. This is fixable, but it needs to be fixed: compute ground-truth values with an independent implementation (plain Python, MATLAB, or analytic expressions for small systems) and show they match.\n\nThe benchmarks also have some roughness: no error bars, and THOI's batched parallelism versus comparison libraries in default mode isn't strictly apples-to-apples. The extrapolations for HOI_toolbox and JIDT are dramatic but should be labeled as extrapolations. And there is no code repository link in the manuscript, which is a problem for a software-library paper.\n\nNone of this contradicts the core performance claim, which is plausible and externally checkable once the code is released. The paper deserves a serious referee and likely acceptance after revision, with three conditions: code with a commit hash, repeated benchmarks with hardware/version details and error bars, and an independent ground-truth validation. I'd bring it to a reading group and would cite it if I worked on HOI methods.\n\nRecommendation: engage. The engineering contribution is real, and the circularity is an oversight, not a scam—it just needs to be closed.","headline":"THOI is a genuinely useful engineering contribution that makes exhaustive HOI analysis feasible for ~30 variables, but its accuracy validation is circular and needs an independent reference check before the numbers can be trusted.","tokens_in":21293,"tokens_out":1599,"would_cite":true,"duration_ms":17120,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"THOI computes all higher-order interactions for systems up to 30 variables in hours on a laptop, far outpacing existing open-source toolboxes.","keywords":["higher-order interactions","O-information","synergy","redundancy","Gaussian copula","batch processing","PyTorch","simulated annealing"],"falsifier":"Compute the reference $\\Omega$ for the synthetic Gaussian R- and S-systems of Supplementary A.5 with an independent closed-form calculation or with a different entropy estimator, and compare it against THOI's output; a systematic offset would show that the validation cannot detect such a bias.","tokens_in":20200,"feed_emoji":"⚡","tokens_out":7593,"duration_ms":64664,"temperature":0.7,"pith_summary":"THOI is a Python library that claims to make exhaustive higher-order interaction (HOI) analysis practical for systems with up to 30 continuous variables, and heuristic searches practical for larger ones. It computes the four standard multivariate information measures — total correlation, dual total correlation, O-information ($\\Omega$), and S-information — from Gaussian-copula entropy estimates, batched through PyTorch tensor operations. On a 30-variable benchmark with 1000 samples, the paper reports computing every interaction order from 3 to 30 in 5.8 hours using under 3 GB of memory, where compared open-source libraries would take an extrapolated 221 days, 2 days with a 240 GB memory overload, or roughly 17 years. The authors validate accuracy on synthetic probabilistic graphical models with known synergy and redundancy structure, apply the tool to fMRI data from wakeful and deeply anesthetized subjects, and exhaustively analyze 920 datasets in under 30 minutes on a laptop. The claim matters because pairwise analyses can miss collective dependencies, and the combinatorial cost has previously blocked routine HOI use.","feed_headline":"All interactions of a 30-variable system in 5.8 hours","feed_subtitle":"New open-source Python library makes exhaustive synergy-redundancy analysis feasible on laptops instead of months of compute.","key_machinery":"The load-bearing object is the batched sub-covariance matrix pipeline built on the Gaussian copula entropy estimator. For data with T samples, the copula transform turns each variable's marginal into a standard normal via ranks, so the whole system is summarized by one N by N covariance matrix; every n-plet's entropy then reduces to a determinant of a submatrix. THOI forms a batch of these submatrices by masking the full covariance matrix with binary index tensors, computes all determinants at once in PyTorch, and combines them with additions and subtractions into total correlation, dual total correlation, $\\Omega$, and S-information. A second mechanism, independent-variable padding, inserts an identity-matrix block so n-plets of different orders can share one batch, and the known entropy of a standard normal (1.4189 nats per padded variable) is subtracted afterward.","core_discovery":"The paper's central claim is that the combinatorial explosion of HOI metrics can be defeated in practice by restructuring the computation as batched linear algebra. Instead of estimating joint probability densities directly, THOI applies the Gaussian copula transform once to obtain a covariance matrix, extracts the sub-covariance matrix for every k-variable combination using binary masks, and computes determinants in parallel batches to obtain entropies and therefore the total correlation, dual total correlation, $\\Omega$, and S-information. The authors report that this makes exhaustive HOI computation over all combinations of a 30-variable system finish in 5.8 hours on a laptop, a task they extrapolate would take HOI_toolbox about 221 days and JIDT roughly 17 years. For systems beyond exhaustive reach, the paper provides greedy and simulated-annealing heuristics and shows on a 100-variable concatenated system that both recover the known synergistic and redundant subsystems. It also reports that deep anesthesia compresses the range of $\\Omega$ values in fMRI data and that four principal components explain about 95 percent of the variance across 920 datasets.","pith_inferences":["Because the Gaussian copula estimator only captures covariance-level dependence after rank normalization, the empirical $\\Omega$ values for fMRI and real data are copula-level summaries; coupling that lives in higher moments or non-Gaussian structure could go undetected.","The masked-submatrix batching idea is not specific to information metrics and could accelerate other subset-indexed statistics, such as partial correlations, in the same memory regime.","The order at which $\\Omega$ is maximized or minimized, which the heuristics can approximate cheaply, is treated in the paper as a proxy for the proportion of synergistic n-plets; testing that proxy on non-Gaussian synthetic families would clarify how general the link is."],"forward_implications":["Exhaustive HOI analysis becomes a routine laptop task for systems up to roughly 30 variables, turning a computation previously estimated at 221 days or 17 years into a same-day job.","All four measures (total correlation, dual total correlation, $\\Omega$, and S-information) are produced in one run, and population-level summaries can be computed on the fly without storing the exponentially many individual values.","The greedy and simulated-annealing heuristics identify known synergistic and redundant subsystems in a 100-variable system, making heuristic HOI searches usable beyond exhaustive scale.","In the anesthesia application, the paper reports a compression of the $\\Omega$ range, with an eight-region interaction shifting from synergy-dominated to redundancy-dominated and a 14-region interaction losing redundancy while remaining redundancy-dominated.","The 920-dataset benchmark supports a four-dimensional description of complex systems: overall interdependence, overall independence, proportion of synergistic n-plets, and O-information balance."],"supporting_citations":[{"why":"Supplies the Gaussian copula estimator of covariance and entropy that THOI's pipeline is built on.","marker":"[30]"},{"why":"HOI_toolbox is the main baseline whose extrapolated 221-day runtime THOI beats on the 30-variable benchmark.","marker":"[31]"},{"why":"HOI is the second performance baseline, extrapolated to 2 days with a 240 GB memory overload for index creation.","marker":"[32]"},{"why":"JIDT provides the KSG-based baseline whose extrapolated 17-year computation is compared against THOI.","marker":"[28]"},{"why":"PyTorch is the tensor engine enabling the batched determinant computations and CPU, GPU, and TPU execution.","marker":"[34]"},{"why":"Defines O-information as total correlation minus dual total correlation, the central metric being computed and optimized.","marker":"[8]"},{"why":"Provides the entropy conjugation framework and the additive property used to assign ground-truth $\\Omega$ values to the synthetic concatenated systems.","marker":"[19]"},{"why":"Supplies the fMRI dataset of wakeful and deeply anesthetized subjects used in the empirical demonstration.","marker":"[38]"}],"fun_headline_variants":["THOI library: exhaustive higher-order interactions in hours, not months","From 17 years to 5.8 hours: THOI computes all higher-order interactions","Batch-processing enables full synergy-redundancy analysis on a laptop","New Python library tames combinatorial explosion for higher-order interactions","THOI: higher-order interaction analysis without the wait"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy validation assumes that THOI's Gaussian entropy and $\\Omega$ formulas are themselves correct, because the reference ground-truth $\\Omega$ values in Supplementary A.5 are computed with THOI using the same formulas, so a systematic implementation bias would be invisible to the comparison.","fun_headline_variants_meta":{"raw":{"variants":["THOI library: exhaustive higher-order interactions in hours, not months","From 17 years to 5.8 hours: THOI computes all higher-order interactions","Batch-processing enables full synergy-redundancy analysis on a laptop","New Python library tames combinatorial explosion for higher-order interactions","THOI: higher-order interaction analysis without the wait"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000827,"raw_usage":{"total_tokens":3659,"prompt_tokens":1031,"completion_tokens":2628,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":2537}},"tokens_in":647,"tokens_out":2628,"duration_ms":18379,"temperature":1.0,"reasoning_tokens":2537,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:52:21.254772+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the reference $\\Omega$ for the synthetic Gaussian R- and S-systems of Supplementary A.5 with an independent closed-form calculation or with a different entropy estimator, and compare it against THOI's output; a systematic offset would show that the validation cannot detect such a bias.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian copula estimator of covariance and entropy that THOI's pipeline is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HOI_toolbox is the main baseline whose extrapolated 221-day runtime THOI beats on the 30-variable benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HOI is the second performance baseline, extrapolated to 2 days with a 240 GB memory overload for index creation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"JIDT provides the KSG-based baseline whose extrapolated 17-year computation is compared against THOI."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PyTorch is the tensor engine enabling the batched determinant computations and CPU, GPU, and TPU execution."},{"cited_title":"E., Mediano, P","cited_arxiv_id":null,"evidence_quote":"Defines O-information as total correlation minus dual total correlation, the central metric being computed and optimized."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the fMRI dataset of wakeful and deeply anesthetized subjects used in the empirical demonstration."}],"review_version":1}