{"id":"5ea958c5-bbbd-48a5-9bb1-f2b7b1f27a2f","arxiv_id":"2412.14489","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A dynamic multimodal network whose per-sample depth and parameters are set by prototype-based quality estimates, reporting large gains over prior trustworthy classifiers.","lead":"This paper introduces Multi-QuAD, a multimodal classification network that estimates how noisy each input sample and each feature is, then adjusts the network's depth and parameters per sample. A generalist might care because the method reports large accuracy and robustness gains over prior trustworthy multimodal classifiers on four benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eqs. (6)–(7) define quality via T ('true class probability') with implementation deferred to the Supplement; both GCND depth and LGP parameters depend on T, so the reported test-set gains may rest on label leakage unless T is label-free.","rationale":"The central claim is empirical superiority from dynamic adaptation, and that adaptation is driven entirely by the NFCE quality estimates δ_v and δ_f. Those estimates are the only link between sample quality and network adjustments. If T reads true labels at inference, the method has test-set leakage and the reported margins of 4.8%, 5.9%, 4.4%, and 1.2% are not trustworthy; if T is label-free, the implementation is still undisclosed, so the preprint cannot be independently verified. I considered whether other issues are more load-bearing: the inconsistent numbers between Table I, Table II, and Table IV, and the missing variance estimates, weaken confidence but do not invalidate the mechanism; the T ambiguity directly threatens the mechanism itself. The manuscript explicitly flags its own omission by deferring T to a Supplement that is not present in the posted version. I am not claiming the authors used test labels; rather, the paper as written does not rule it out. A conditional verdict is the right target: ask for the T implementation and the label-permutation invariance check before accepting Table I and Fig. 4. This matches the reader's weakest_assumption, so no verdict adjustment is needed.","tokens_in":21740,"tokens_out":6356,"duration_ms":60955,"concrete_test":"Run a label-permutation invariance check on the Table I protocol: randomly permute the test labels, recompute δ_v and δ_f exactly as in the paper, and check whether GCND depths and LGP parameters change. A label-free T must leave all depth assignments and parameter predictions unchanged; if they change, Eqs. (6)–(8) leak test labels. As a complementary test, obtain the Supplemental Materials implementation of T and rerun Table I with T computed from true test labels versus a disclosed label-free proxy; if the label-free version loses more than the reported standard deviation on any dataset, the headline gains are contaminated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-B2 defines δ_v(x_i^m)=T(p_v(x_i^m)) and δ_f(x_i^m)=T(p_f(x_i^m)), where T is called the true class probability and cited to MD [4]. The manuscript says only that 'the detailed implementation of T is given in the Supplemental Materials'; no supplement or code accompanies arXiv v3. These δ values are load-bearing: Eq. (8) feeds δ_v into the global depth normalization GCND, and Eq. (10) plus Algorithm 1 line 17 use Δδ_f in the greedy parameter mechanism. Thus both sample-specific depth and sample-specific parameters are functions of T. If T requires the ground-truth label y_i, then at test time the network is told the answer before classification, and Table I / Fig. 4 report a leaked protocol. If T is instead a label-free proxy such as the maximum prototype probability, that substitution is never disclosed or validated, so the claimed 'noise-free prototype confidence' is not what is evaluated. This is the single most load-bearing premise for the central empirical claim: none of the dynamic adaptation results can be accepted until the inference-time computation of T is specified and shown not to use test labels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Multi-QuAD, a multimodal classification framework that estimates sample quality at modality and feature levels using noise-free prototypes (NFCE), then adapts network depth via Global Confidence Normalized Depth (GCND) and per-sample network parameters via a Layer-wise Greedy Parameter (LGP) mechanism implemented with an LSTM. Experiments on BRCA, ROSMAP, CUB, and FOOD101 report consistent accuracy gains of 4.4–5.9 points over the best static reliable-classification baselines and over the dynamic DynMM baseline, with smaller accuracy degradation under Gaussian noise. The central claim is that prototype-based quality estimates enable dynamic depth and parameter assignment that is both more accurate and more robust than static networks.","tokens_in":21970,"tokens_out":8288,"duration_ms":69987,"significance":"If the reported results are reproducible under a label-free inference protocol, the paper makes a strong empirical contribution to reliable multimodal classification: it combines quality estimation with dynamic architecture design, provides ablations for each component, and demonstrates consistent gains across four benchmarks, including both biomedical and vision-language data. The comparison against DynMM and the static state-of-the-art methods is useful, and the visualizations of learned biomarkers add interpretive value. The significance is therefore primarily empirical, and it hinges on the clarity and validity of the quality estimator at test time; the lack of code or the referenced supplement limits independent verification.","major_comments":[{"comment":"The paper defines δ_v and δ_f using T, the “true class probability” from [4], and defers the implementation to the Supplemental Materials, which is not available with the arXiv v3 submission. These quantities are load-bearing: Eq. (8) uses δ_v to set all sample depths, and Eq. (10) plus Algorithm 1 line 17 use δ_f to train and operate LGP. The manuscript never states whether T is computed with ground-truth labels or with a label-free proxy at inference. If labels are used, the test-set results in Table I and Fig. 4 are invalid because the network is given the answer during depth and parameter assignment; if a proxy such as maximum predicted probability is used, that substitution is not disclosed or validated. Please specify the exact computation of T, state the inference-time protocol, and confirm that no test labels are used in any step of dynamic depth or parameter prediction.","section":"Section III-B2, Eqs. (6)–(7)"},{"comment":"The prototypes ω used by NFCE are optimized on the training set, and the LGP loss maximizes Δ, which is measured as improvement in δ_f evaluated against those same prototypes. This makes the “quality gain” objective self-referential: it rewards moving features toward the training prototypes rather than toward an independent measure of held-out quality. The held-out classification results in Table II are not invalidated by this, but the interpretation that LGP “maximizes feature-level quality enhancement” would be much stronger with a validation of δ_f against held-out label correctness (e.g., correlation between δ_f and per-feature classification utility) and with reported test-set values of Δ. I frame this as a correctness-risk concern with a concrete test, not as a claim of leakage.","section":"Section III-B1 and III-E, Eqs. (3), (10), (13)"}],"minor_comments":[{"comment":"Please specify over which axis the softmax is applied for p_f and how T reduces the class dimension to produce a per-feature δ_f; as written, p_f ∈ R^{d_m×C} and δ_f ∈ R^{d_m} leave this implicit.","section":"Section III-B2, Eq. (5)"},{"comment":"The notation \\hat{I}^{m+1}_i is confusing because \\hat{I} indexes layer t, not modality; it should be something like \\hat{I}^{m,t+1}_i to avoid implying that the (m+1)-th modality is used.","section":"Algorithm 1, line 16"},{"comment":"The phrase “such that Δ^t_i is maximized” describes a constraint, but LGPt is a learned LSTM and does not solve an optimization at inference; recommend rewording to “trained to maximize” or “aims to maximize.”","section":"Section III-D, Eq. (10)"},{"comment":"State explicitly whether the confidence evaluation on the “same clean testing samples” uses ground-truth labels; this is the same ambiguity as the T definition and should be resolved in the main text rather than only in the supplement.","section":"Section IV-D1 and Fig. 6"},{"comment":"Clarify whether the depth normalization N_K is computed once on the training set or recomputed for each test batch; this matters for deploying the model on individual samples in a streaming setting.","section":"Section III-C, Eq. (8)"},{"comment":"Multi-QuAD's accuracy changes from 92.2 at K=3 to 91.6 at K=4, which looks larger than the “relatively insensitive” claim; please add a brief discussion of the variance across K.","section":"Figure 5, ROSMAP panel"},{"comment":"The dataset name “UMPC FOOD101” appears in the figure labels, while the text and Table I use “UPMC FOOD101”; please unify the naming.","section":"Figures 5 and 9"},{"comment":"The accuracy drop is reported in parentheses for the baselines but not for Multi-QuAD's ACC*; include the same notation for Multi-QuAD to make the comparison symmetric.","section":"Section IV-H, Table VII"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the definition of T. The cited TCP in [4] is a label-dependent quantity in its original formulation, so the authors must be explicit about how it is adapted for unlabeled test inputs. The missing supplement is a concrete reproducibility blocker for this specific point. I did not see evidence of intentional leakage; the gap appears to be an omitted specification. If T is label-free, the paper can be repaired by adding the definition and validating the proxy; if T is label-based, the central empirical claim would not be valid as reported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe short version: the idea is worth a look, and the reported numbers are big, but the paper as posted cannot support its central empirical claim because the quality estimator T is underspecified and its implementation is deferred to a supplement that does not exist on arXiv.\n\nWhat's new: combining classifier-free prototype confidence (NFCE) with sample-adaptive depth (GCND) and LSTM-generated layer parameters (LGP) is a genuinely new integration. The prototype pre-optimization with mutual information and inter-class orthogonality is a reasonable way to get noise-robust quality estimates. The GCND move – normalizing depth across all modalities and samples before quantizing to [1,K] – is a simple, sensible way to stop hard samples from getting absurdly deep networks. And the ablations are the right kind: fixed-depth and intermediate-classifier versions are proper controls, and the paper includes a useful sensitivity analysis of K.\n\nWhere it falls down: Eq. (6) defines delta_v = T(p_v), with T called the 'true class probability' and cited to MD. That name, and the original TCP definition, strongly suggest T requires ground-truth labels. The paper says the implementation is in the Supplemental Materials. There is no supplement on arXiv. Since both GCND depth and LGP parameters are computed from delta_v and delta_f, the entire dynamic mechanism depends on a quantity the paper never defines at inference time. If T uses labels, the test-set protocol is leaked and the accuracy gains are meaningless. If T is actually a label-free proxy like maximum predicted probability, that substitution is not disclosed or validated, so the reported 'noise-free' quality estimation is not what was evaluated. This is not a minor omission; it is the load-bearing premise of the empirical section.\n\nThere are also smaller problems. The same configuration reports different numbers in different tables (Multi-QuAD on BRCA, sigma=0: 93.4 in Tables I-II, 93.7 in Table IV). No code is available, so the missing T cannot be reconstructed. The claim that LGP is the first parameter prediction paradigm for variable-architecture multimodal networks is probably over-strong, but that is a minor point.\n\nBottom line: the problem is important, the architecture is a plausible integration, and the paper deserves a serious referee. But a referee should require the supplement with T's exact implementation, code, and a consistent set of results before the central claim can be accepted. My verdict: promising idea, unverified as posted.","headline":"Promising integration of prototype-based confidence with dynamic depth and parameters, but the test-time computation of T is unspecified and the missing supplement makes the large reported gains unverifiable.","tokens_in":22483,"tokens_out":3180,"would_cite":false,"duration_ms":25951,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multi-QuAD claims that a multimodal classifier that adapts its depth and parameters to each sample's estimated quality outperforms static reliable classifiers by up to 5.9 percentage points and degrades less under Gaussian noise.","keywords":["multimodal classification","reliable classification","dynamic neural networks","quality estimation","noise-free prototypes","sample-adaptive depth","sample-adaptive parameters","Gaussian noise robustness"],"falsifier":"Reproduce the NFCE and GCND depth assignments on a held-out test split of FOOD101 with labels withheld; if computing $\\delta_v$ and $\\delta_f$ via Eqs. (6)-(7) requires the ground-truth class, the reported 94.5% ACC and the Fig. 4 robustness curves are not attainable at test time.","tokens_in":21480,"feed_emoji":"📈","tokens_out":6958,"duration_ms":51104,"temperature":0.7,"pith_summary":"Multi-QuAD argues that reliable multimodal classification should not use a fixed network for every sample. It estimates the quality of each modality and each feature of every input, then adjusts the network's depth and parameters accordingly. On four benchmarks, the paper reports that this dynamic strategy beats state-of-the-art reliable multimodal classifiers by 1.2 to 5.9 percentage points in accuracy under clean conditions, and loses substantially less accuracy under Gaussian noise. The reason to care is that reliability in safety-critical multimodal settings is usually pursued with static architectures and confidence scores; this work makes depth and parameters themselves the adaptive resource.","feed_headline":"Sample-adaptive network beats multimodal SOTA by up to 5.9%","feed_subtitle":"Multi-QuAD tunes each sample's network depth and weights, gaining accuracy and noise robustness.","key_machinery":"Three components carry the argument. Noise-free Prototype Confidence Estimation (NFCE) constructs class prototypes optimized to remove noise by maximizing cross-modal mutual information and inter-class orthogonality, then produces modality-level quality $\\delta_v(x_i^m)$ and feature-level quality $\\delta_f(x_i^m)$ without a classifier. Global Confidence Normalized Depth (GCND) ranks $1-\\delta_v(x_i^m)$ for all modalities of all samples and scales them into integer depths $D_i^m \\in [1,K]$, so lower-quality modalities get deeper unimodal networks while extreme samples are reined in by the global normalization. Layer-wise Greedy Parameter (LGP) uses an LSTM unit to remember previously enhanced features and predicts the weights of all unimodal blocks at depth $t$ jointly, maximizing the feature-quality gain $\\Delta_i^t$ and a sparsity loss. The central object is the quality-to-architecture mapping: per-sample quality estimates drive both depth and parameters.","core_discovery":"The central discovery is that a multimodal classifier can be made more reliable by making its architecture depend on a classifier-free, prototype-based estimate of input quality. Multi-QuAD computes modality-level and feature-level qualities by comparing each input with pre-optimized 'noise-free prototypes' (class centers with noise removed), avoiding the overfitting to training noise that the paper attributes to classifier-based confidence measures. The Global Confidence Normalized Depth mechanism maps these quality estimates to per-sample network depths, normalized across modalities and samples so that extreme low-quality inputs do not force excessive depth. The Layer-wise Greedy Parameter mechanism then predicts per-sample weights at each depth, layer by layer, to maximize feature-level quality gain. On BRCA, ROSMAP, CUB, and FOOD101 the paper reports ACC gains of 4.8%, 5.9%, 4.4%, and 1.2% over the strongest baseline, with smaller accuracy drops under noise.","pith_inferences":["The same prototype-based quality estimate could be reused as a drop-in confidence module for other multimodal architectures, since NFCE is trained separately from the classifier.","A label-free proxy for the true-class-probability transform $T$, such as maximum predicted probability or prediction entropy, would make the method deployable at test time; whether the reported margins survive that substitution is a concrete follow-up test.","GCND's global normalization across the whole dataset implies that reliability depends on the population distribution of input quality, not only on each sample in isolation; that principle could transfer to any per-sample adaptive system.","The paper's own future-work list suggests extending the same quality-adaptive mechanism from classification to regression and segmentation, where quality estimation for continuous targets would need a new definition of prototype."],"forward_implications":["Each modality of each sample should get its own network depth, with lower-quality inputs processed by deeper unimodal networks.","Network parameters should be predicted per sample and per layer, not fixed after training, so that a single block can map different feature-quality patterns to cleaner representations.","Confidence estimates built from noise-free prototypes rather than from the classifier maintain stable behavior across training noise levels at both modality and feature levels.","Dynamic multimodal networks such as DynMM gain reliability when their depth assignment is driven by normalized quality estimates rather than by intermediate gating or classifier signals."],"supporting_citations":[{"why":"Supplies the true-class-probability transform T used in NFCE's quality equations and is the static trustworthy-fusion baseline.","marker":"[4]"},{"why":"GCFANet is the leading static baseline on BRCA and ROSMAP; its fixed-depth behavior motivates the need for dynamic depth.","marker":"[1]"},{"why":"PDF is the leading static baseline on CUB and FOOD101 in Table I, giving the reported 4.4% and 1.2% ACC gaps.","marker":"[2]"},{"why":"DynMM is the only dynamic multimodal baseline; comparing against it isolates the contribution of reliable quality estimation and normalized depth.","marker":"[56]"},{"why":"Used to implement the intermediate-classifier dynamic depth (ICD) baseline that GCND is compared against under noise.","marker":"[50]"}],"fun_headline_variants":["Per-sample depth and weights boost multimodal classification reliability","Quality-adaptive network tunes depth per sample, beats SOTA by 5.9%","Classifier-free quality estimate drives each sample's architecture","Multi-QuAD adapts depth and parameters per sample for reliable AI","Sample-adaptive network boosts accuracy and noise robustness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The test-time quality scores depend on a 'true class probability' transform that the paper does not specify for inference; if that transform requires the sample's ground-truth label, the reported accuracies would not transfer to unlabeled test data.","fun_headline_variants_meta":{"raw":{"variants":["Per-sample depth and weights boost multimodal classification reliability","Quality-adaptive network tunes depth per sample, beats SOTA by 5.9%","Classifier-free quality estimate drives each sample's architecture","Multi-QuAD adapts depth and parameters per sample for reliable AI","Sample-adaptive network boosts accuracy and noise robustness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000819,"raw_usage":{"total_tokens":3608,"prompt_tokens":992,"completion_tokens":2616,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":2532}},"tokens_in":608,"tokens_out":2616,"duration_ms":14611,"temperature":1.0,"reasoning_tokens":2532,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:11:18.055648+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the NFCE and GCND depth assignments on a held-out test split of FOOD101 with labels withheld; if computing $\\delta_v$ and $\\delta_f$ via Eqs. (6)-(7) requires the ground-truth class, the reported 94.5% ACC and the Fig. 4 robustness curves are not attainable at test time.","supporting_citations":[{"cited_title":"Multimodal dynamics: Dynamical fusion for trustworthy multimodal classification,","cited_arxiv_id":null,"evidence_quote":"Supplies the true-class-probability transform T used in NFCE's quality equations and is the static trustworthy-fusion baseline."},{"cited_title":"Zheng, M","cited_arxiv_id":null,"evidence_quote":"GCFANet is the leading static baseline on BRCA and ROSMAP; its fixed-depth behavior motivates the need for dynamic depth."},{"cited_title":"Predictive dynamic fusion,","cited_arxiv_id":null,"evidence_quote":"PDF is the leading static baseline on CUB and FOOD101 in Table I, giving the reported 4.4% and 1.2% ACC gaps."},{"cited_title":"Dynamic multimodal fusion,","cited_arxiv_id":null,"evidence_quote":"DynMM is the only dynamic multimodal baseline; comparing against it isolates the contribution of reliable quality estimation and normalized depth."}],"review_version":1}