{"id":"7ea92e8b-d439-442b-8075-5edc64f2a1f3","arxiv_id":"2506.17248","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A lightweight estimator quantifies sample-level multimodal interactions (redundancy, uniqueness, synergy) in continuous distributions and uses them for data partitioning, distillation, and ensembling.","lead":"This paper introduces LSMI, a lightweight estimator that measures per-sample redundancy, uniqueness, and synergy between two modalities using pointwise information and small neural entropy models. If it holds up, it gives multimodal learning a practical tool for data partitioning, distillation, and model ensembling based on how each sample uses its modalities.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 6 defines r as a difference of two component-wise minima, but the paper only proves each component is lattice-compatible; monotonicity of the difference is asserted, not shown, and Table 8 exposes negative r. If r is not a valid redundancy measure, the u1, u2, s derived from Eq.","rationale":"The paper's central claim is that LSMI quantifies sample-level redundancy, uniqueness, and synergy. For this to be true, the quantity r in Eq. 6 must be a meaningful pointwise redundancy measure, not merely an arbitrary number inserted into an underdetermined linear system. The weakest point in the derivation is the transition from positive, lattice-compatible components i+ and i- to their difference: monotonicity of the two min-terms does not imply monotonicity of the difference, and the paper supplies no theorem or proof covering the negative pointwise-MI regime. Table 8 directly shows the estimator producing large negative r and s values, which at minimum requires an explicit definition of what negative redundancy means and why it is compatible with the redundancy lattice. The synthetic validation is encouraging but aggregate: comparing average R, U, S to distribution-level ground truth can mask systematic pointwise violations, and the ground truth itself is defined through a different PID convention. The paper does have independent support: the code is released, the synthetic circuit experiments match ground truth closely for the tested aggregates, and the applications are concrete and reasonable. The concern is therefore not that the method is worthless, but that its theoretical foundation is incomplete in exactly the place the reader identified. Since the reader already rendered a CONDITIONAL verdict on this basis, my stress-test does not move the verdict; it sharpens the condition: add a proof or a restrictive condition under which Eq. 6 satisfies the lattice monotonicity, or verify the pointwise redundancy bounds empirically on the synthetic benchmarks.","tokens_in":18403,"tokens_out":10186,"duration_ms":108268,"concrete_test":"Run the released LSMI code on the synthetic XOR/OR/XOR+NOT and Gaussian-mixture datasets, and for every sample compute r from Eq. 6 together with i(x1;y) and i(x2;y). Tabulate the fraction of samples violating the redundancy inequalities Eq. 3 is intended to express, such as r <= i(x1;y), r <= i(x2;y), and, where both pointwise MIs are nonnegative, r >= 0. If a non-negligible fraction violates these bounds, the monotonicity-inheritance claim is empirically false. For a sharper analytic check, search the 16-vertex simplex of joint distributions over two binary sources and a binary target for any distribution in which Eq. 6 violates 0 <= r <= min(i1, i2), which would settle the question independently of the entropy estimators.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central step is Eq. 6: r = min(i+(x1;y), i+(x2;y)) - min(i-(x1;y), i-(x2;y)), with i+ = h(x) and i- = h(x|y). Section 3.2 argues that because i+ and i- are positive, each is compatible with the redundancy lattice, and 'Consequently' r inherits the required monotonicity. This does not follow: the set of monotone functions on a lattice is closed under min, but not under subtraction. A difference of two monotone functions can increase or decrease arbitrarily, and no step in the paper shows that the specific pair (min_i i+, min_i i-) preserves the decomposition inequality (Eq. 3) pointwise. The problem is not hypothetical: Table 8 reports per-sample r = -13.719 and s = -5.582 under label noise, i.e., the estimator can produce negative 'redundancy' on individual samples. If r is not a genuine redundancy component, then u1 = i(x1;y) - r, u2, and s from Eq. 2 are just algebraic residuals of an arbitrary r, and the claim that LSMI quantifies sample-level multimodal interactions is unsupported. The synthetic validation checks average R, U, S against distribution-level GT, which can average away pointwise violations; it never verifies the pointwise lattice property. This is the load-bearing gap in the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LSMI, a sample-level estimator of multimodal interaction components (redundancy r, uniqueness u1, u2, synergy s) for continuous distributions. The method defines redundancy as the difference of two component-wise minima of pointwise surprisals and conditional surprisals (Eqs. 5-6), derives the remaining components from the linear relations in Eq. (2), and estimates the required entropy terms with the KNIFE differential entropy estimator and learned discriminative models. The authors validate on synthetic circuit-logic, Gaussian-mixture, and preset-interaction datasets, then apply the estimator to several real-world multimodal datasets, including application studies on data partitioning, distillation, and ensembling.","tokens_in":18781,"tokens_out":11330,"duration_ms":114951,"significance":"If the sample-level decomposition were theoretically sound, this would be a useful practical contribution: it is, to my knowledge, the first estimator aimed specifically at sample-level PID-style interaction quantification in continuous settings, it is computationally lighter than distribution-level baselines, and the downstream experiments demonstrate plausible practical value. The paper also ships code and reports runtime comparisons, which are strengths. However, the central theoretical claim that r in Eq. (6) is a valid lattice-compatible redundancy measure is not established, and the current evidence does not directly validate the sample-level quantities that the method advertises.","major_comments":[{"comment":"The inference 'both measures, i+ and i-, are compatible with the lattice... Consequently ... r = r+ - r-' is not valid. The set of lattice-compatible functions is closed under minima, but it is not closed under subtraction, and no proof is given that the pointwise difference r inherits the required monotonicity or the inequality in Eq. (3). Table 8 in Appendix A.3.2 reports negative values of r (e.g., -13.719 under label noise), which is consistent with the claimed monotonicity failing at the sample level. Since u1, u2, and s are computed from Eq. (2) once r is fixed, the validity of the entire decomposition rests on this property. The synthetic validation in Section 4.1 compares only averaged R, U, S against distribution-level ground truth, so it cannot detect pointwise violations. Please provide a proof of the lattice properties of r, or a direct per-sample validation against ground-truth pointwise components, or revise the theoretical claims accordingly.","section":"Section 3.2, Eq. (4)"},{"comment":"The paper justifies the decomposition i(x;y) = i+(x;y) - i-(x;y) with i+(x;y) = -log p(x) and i-(x;y) = -log p(x|y) by stating that these components are positive. For continuous distributions, -log p(x) and -log p(x|y) are not guaranteed to be positive; they are negative wherever the density exceeds one. Since LSMI targets continuous data and uses KNIFE differential entropy estimates, the positivity premise is false exactly in the intended regime. The argument that the components are lattice-compatible therefore needs a different basis, or the method needs to be restricted or reinterpreted so that the relevant pointwise quantities are known to be nonnegative.","section":"Section 3.2, Eq. (4)"},{"comment":"The preset-interaction experiments, which are the primary evidence for sample-level accuracy, report only dataset-level averages (R, U, S) in Figure 4. They do not compare per-sample estimates of r, u1, u2, and s with the known per-sample interaction type. Given the paper's central claim is sample-level quantification, this is an important missing validation. Please add a pointwise comparison or report the distribution of sample-level estimates for the synthetic datasets.","section":"Section 4.1.4 and Appendix A.1"}],"minor_comments":[{"comment":"The displayed inequality in Eq. (3) appears to state that the shared component is larger than i(x1;y), which is inconsistent with the preceding sentence that redundancy should not exceed the information in any individual source. Please clarify the intended direction or define the notation i((x1;y),(x2;y)).","section":"Section 3.2, Eq. (3)"},{"comment":"The phrase 'monopoly condition' appears to be a typo for 'monotonicity condition'.","section":"Section 3.2"},{"comment":"Eq. (8) uses h(y), but the algorithm does not specify how h(y) is computed; please state the estimator used for the target entropy.","section":"Algorithm 1 and Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The main theoretical concern is significant. If the authors cannot provide a proof of lattice compatibility for r in Eq. (6), or a convincing direct pointwise validation, I would not be able to support acceptance. The average-level agreement with PID ground truth in Section 4.1 is encouraging, but it does not by itself establish the sample-level claims that distinguish this paper from prior work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this paper builds a working sample-level estimator of multimodal redundancy, uniqueness, and synergy for continuous data, and the synthetic validation is convincing. The theoretical foundation has a gap worth flagging, but it is not fatal to the practical claims.\n\nThe core definition is not new: r = min(h(x1), h(x2)) - min(h(x1|y), h(x2|y)) is the pointwise common change in surprisal from Ince (2017), and the specificity/ambiguity split is from Finn and Lizier (2018a). Both are cited. The novel piece is the engineering: pairing that decomposition with KNIFE neural entropy estimators and discriminative models to get fast, per-sample interaction values on real-world continuous data. The code is available, which helps.\n\nThe synthetic validation is the paper's strongest asset. On circuit logic, Gaussian mixtures with varying correlation, and preset interaction datasets, LSMI closely tracks ground truth and beats PID-CVX and PID-Batch. The real-world applications, such as redundancy-based data partitioning, interaction-guided distillation, and ensemble weighting, are reasonable and appear to work, though the gains are modest.\n\nNow the soft spots. The main one is the monotonicity claim in Section 3.2. The paper argues that since i+ and i- are positive, each is lattice-compatible, and \"consequently\" r inherits the required property. Subtraction does not preserve monotonicity, and no proof is given. Table 8 confirms the problem: r = -13.7 under label noise. So pointwise r is not a non-negative redundancy component; it is a signed quantity. The paper occasionally acknowledges this, but it does not engage with what that means for the \"decomposition\" claim. If r is just an arbitrary signed function, then u1, u2, s are algebraic residuals, not PID components. The synthetic validation only checks averages, which can hide pointwise violations.\n\nThat said, I do not think this sinks the paper. The estimator appears to work on average, and the applications use the values for ranking, not for rigorous PID interpretation. The authors need to either prove the lattice property, which I suspect fails, or reframe the contribution as a practical signed interaction measure and drop the PID-rigor rhetoric. I also noticed Eq. 3 seems to have the inequality direction backwards for a redundancy measure, which is worth checking. The \"first\" claim is strong given the cited pointwise work; softening it to \"first practical continuous estimator\" would be more accurate.\n\nMinor issues: no error bars on real-world numbers, and the human-judgment comparison, on a 0-5 scale, is not an information-theoretic ground truth.\n\nWho benefits: applied multimodal learning researchers who want a fast, per-sample interaction score for interpretability or data selection. It deserves a serious referee; the theoretical concerns are addressable and the empirical work is solid. My recommendation: major revision, with the monotonicity question and the \"first\" claim as the two items to resolve.","headline":"Practical sample-level interaction estimator with solid synthetic validation, but the redundancy measure's monotonicity is asserted rather than proven, and negative pointwise values complicate the PID framing.","tokens_in":19287,"tokens_out":8641,"would_cite":true,"duration_ms":87366,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LSMI gives a per-sample decomposition of multimodal information into redundancy, uniqueness, and synergy using component-wise minima of pointwise surprisals, and claims to be the first estimator that does this efficiently on continuous…","keywords":["multimodal interaction","partial information decomposition","pointwise mutual information","redundancy","uniqueness","synergy","sample-level estimation","entropy estimation"],"falsifier":"On the noisy-label setup shown in Table 8, check whether any sample satisfies $r+u_1>i(x_1;y)$ or $r+u_2>i(x_2;y)$; either inequality forces a negative uniqueness in Eq. 2 and falsifies the claim that $r$ is a monotone, bounded pointwise redundancy. More directly, any single sample with estimated $r<0$ while both components are nonnegative contradicts the intended redundancy bound.","tokens_in":18195,"feed_emoji":"🧩","tokens_out":12430,"duration_ms":111928,"temperature":0.7,"pith_summary":"Multimodal data carry three kinds of task-relevant information: redundant information shared by both modalities, unique information available from only one, and synergistic information that appears only when both are used together. This paper's central claim is that all three can be computed for each individual sample, not just averaged over a dataset, and that the computation can be done cheaply on continuous real-world data. The estimator, LSMI, sets sample-level redundancy to the difference between two component-wise minima of pointwise surprisals, and then derives uniqueness and synergy from a pointwise decomposition identity. The authors support the claim with exact matches to ground truth on logic-circuit and Gaussian-mixture benchmarks, with agreement with human judgments on real datasets, and with downstream gains in data partitioning, distillation, and ensembling. If the claim holds, multimodal interaction moves from a dataset-level abstraction to a per-example diagnostic that can guide model training and selection.","feed_headline":"Every sample gets its own redundancy, uniqueness, and synergy scores","feed_subtitle":"Estimates per-sample redundancy, uniqueness, and synergy to guide data selection, distillation, and model ensembling.","key_machinery":"The central object is the pointwise redundancy $r$, built from two component-wise minima of surprisals. Since $i(x;y)=h(x)-h(x|y)$ with $h(x)=-\\log p(x)$ and $h(x|y)=-\\log p(x|y)$, the paper defines $r^+ = \\min(h(x_1), h(x_2))$ and $r^- = \\min(h(x_1|y), h(x_2|y))$, then sets $r=r^+-r^-$. The nonnegativity of each surprisal component is what makes each minimum monotone over the event lattice, so the shared component can be quantified as redundancy; the three equations in Eq. 2 then determine $u_1,u_2,s$ from $r$ and the pointwise informations. For continuous data, the entropy values are supplied by a lightweight neural estimator that upper-bounds differential entropy by minimizing a KL divergence, which is why the method scales to real datasets without modeling the joint distribution.","core_discovery":"LSMI computes, for every sample $(x_1,x_2,y)$, four numbers $(r,u_1,u_2,s)$ satisfying $i(x_1;y)=r+u_1$, $i(x_2;y)=r+u_2$, and $i(x_1,x_2;y)=r+u_1+u_2+s$. The paper defines redundancy by splitting pointwise mutual information into a positive component $i^+(x;y)=-\\log p(x)$ and a negative component $i^-(x;y)=-\\log p(x|y)$, taking the minimum of each component across the two modalities, and subtracting: $r = \\min(i^+(x_1;y), i^+(x_2;y)) - \\min(i^-(x_1;y), i^-(x_2;y))$. Uniqueness and synergy then follow algebraically from the three pointwise mutual informations. The authors claim this is the first estimator that gives reliable sample-level interaction values on continuous real-world data, that it reproduces known ground truth on synthetic systems, and that it is fast because it needs only two per-modality entropy estimators rather than an optimized joint distribution.","pith_inferences":["Beyond the paper, the sign flip of redundancy and uniqueness under label noise shown in Table 8 suggests LSMI could serve as a label-quality probe: samples whose redundancy turns strongly negative are candidates for mislabeling.","Beyond the paper, the component-wise minimum construction extends naturally to more than two modalities by taking minima across all sources, so the same estimator could be stress-tested as a higher-order interaction measure, though the authors restrict evaluation to pairwise decompositions.","Beyond the paper, a practical robustness check would be to vary the fitted classifiers $p(y|x_1)$, $p(y|x_2)$, and $p(y|x_1,x_2)$ across random seeds and architectures, since the paper reports accuracy but not stability of the interaction values themselves."],"forward_implications":["On synthetic logic circuits, mixtures of Gaussians, and preset-interaction mixtures, LSMI tracks ground-truth redundancy, uniqueness, and synergy more closely than PID-CVX and PID-Batch.","Averaged sample-level interactions match human judgments on Food-101 (Pearson 0.98 for redundancy, 0.95 for text uniqueness) and reproduce expected interaction patterns on KS, CMU-MOSEI, and UR-Funny.","Fine-tuning an aligned multimodal embedding model on high-redundancy subsets strengthens alignment-driven modalities, while low-redundancy subsets help information-poor modalities, so interaction scores can steer data partitioning.","Interaction-weighted distillation and interaction-aware ensembling outperform plain feature distillation and weighted ensembling on KS, UCF, and CREMA-D.","LSMI's runtime stays roughly constant as class count grows and is far below PID-Batch (504 s vs 59,680 s on Food-101), because it fits per-modality entropy estimators rather than a joint distribution."],"supporting_citations":[{"why":"Introduces partial information decomposition, the redundancy/uniqueness/synergy framework that LSMI adapts to sample level.","marker":"(Williams & Beer, 2010)"},{"why":"Defines uniqueness-based PID and supplies the ground-truth interaction values used to validate LSMI on circuit logic and Gaussian mixtures.","marker":"(Bertschinger et al., 2014)"},{"why":"Proposes pointwise PID on specificity and ambiguity lattices, the prior sample-level decomposition LSMI extends to continuous data.","marker":"(Finn & Lizier, 2018a)"},{"why":"Motivates measuring redundancy on positive/negative surprisal components, the construction behind the component-wise minima.","marker":"(Ince, 2017)"},{"why":"Shows pointwise mutual information can be negative and break lattice monotonicity, the obstacle the two-component split addresses.","marker":"(Finn & Lizier, 2018b)"},{"why":"Provides the differential entropy estimator (KNIFE) that makes sample-wise entropy estimation efficient for continuous distributions.","marker":"(Pichler et al., 2022)"},{"why":"Contributes PID-CVX and PID-Batch, the distribution-level baselines LSMI is compared against on synthetic and real datasets.","marker":"(Liang et al., 2023b)"}],"fun_headline_variants":["Per-sample redundancy, uniqueness, synergy in one go","LSMI: fast, sample-level multimodal interaction scores","First sample-wise estimator for continuous multimodal data","Measure redundancy, uniqueness, synergy for each sample","Unlock per-sample interaction: LSMI estimator"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that subtracting the two component-wise minima preserves the monotonicity each minimum has on its own, so that the resulting $r$ is a valid pointwise redundancy; the paper offers no proof of this for samples where pointwise mutual information is negative, and negative $r$ values under label noise indicate the premise is not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["Per-sample redundancy, uniqueness, synergy in one go","LSMI: fast, sample-level multimodal interaction scores","First sample-wise estimator for continuous multimodal data","Measure redundancy, uniqueness, synergy for each sample","Unlock per-sample interaction: LSMI estimator"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000247,"raw_usage":{"total_tokens":1556,"prompt_tokens":972,"completion_tokens":584,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":512}},"tokens_in":588,"tokens_out":584,"duration_ms":5622,"temperature":1.0,"reasoning_tokens":512,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:44:35.609150+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the noisy-label setup shown in Table 8, check whether any sample satisfies $r+u_1>i(x_1;y)$ or $r+u_2>i(x_2;y)$; either inequality forces a negative uniqueness in Eq. 2 and falsifies the claim that $r$ is a monotone, bounded pointwise redundancy. More directly, any single sample with estimated $r<0$ while both components are nonnegative contradicts the intended redundancy bound.","supporting_citations":[],"review_version":1}