{"id":"2c6251df-a55c-4cac-a918-e2aa0b6e6f5b","arxiv_id":"2507.00234","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Fusing Grad-CAM and attention rollout plus an NLP module is presented as an interpretable time-series framework, but the claims are undercut by unsupported theory and dataset errors.","lead":"A proposed hybrid AI system combines a ResNet and a transformer into fused heatmaps for time-series predictions, then turns them into plain-language reports. The paper reports strong accuracy gains on ECG and energy datasets, but the evidence is undermined by internal contradictions and missing artifacts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central fusion rule H(t)=αG(t)⊙A(t) is asserted to maximize I(H;Y) without a derivation; if this optimality is false, the claimed full spatial-temporal alignment and interpretability gains have no basis.","rationale":"The abstract promises a fused visualization with full spatial-temporal alignment and real-time performance, and the whole framework's novelty is the fusion rule. The reader's weakest_assumption points at the missing theoretical derivation of that rule; I agree. This is more load-bearing than the real-time contradiction because even if latency is fixed, an unjustified fusion mechanism would not support the interpretability claims. The empirical section also fails to isolate the fusion effect: the deletion test is hybrid vs ResNet, not vs attention rollout or vs other fusion operators. A synthetic experiment with known ground truth would directly test whether multiplication is actually better. Other issues (PhysioNet class/count mismatch, sensor-pruning misattribution, no code) are real but secondary to the unsupported mechanism. I therefore keep the reader's REJECT verdict unchanged.","tokens_in":14238,"tokens_out":3634,"duration_ms":41650,"concrete_test":"On the synthetic benchmark of §V.A.3, where injected patterns define ground-truth importance masks, train the same ResNet and Transformer, extract G and A, then build H_mul=αG⊙A, H_avg=αG+(1−α)A, and H_cat=[G;A] with a 1×1 projection. Compare top-20% deletion/insertion faithfulness and correlation with the known masks over at least 10 seeds using paired Wilcoxon tests. If H_mul does not significantly outperform both inputs and the two alternative fusion operators, Section III.3's optimality claim is falsified; an independent re-derivation of the stated inequality should also be run, since no Gaussian-noise model is specified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.3 makes the paper's central technical claim: element-wise multiplication of the Grad-CAM map G(t) and attention-rollout map A(t) \"maximizes the mutual information I(H;Y)\" and dominates concatenation under \"Gaussian noise assumptions.\" No statistical model, theorem, or proof is given. The causal equation ∂P(y|do(t))/∂t ∝ ∇_t f_ResNet ⊙ E[f_Transformer|do(t)] is also stated without defining the causal graph or deriving the do-calculus step, so the \"causal fidelity\" claim rests on an undefined premise. The asserted inequality is not generally true: for additive zero-mean noise, E[G⊙A] = H*² when both maps are unbiased estimates of H*, introducing bias, and the variance depends on the joint distribution of the noise terms. Therefore the mechanism that is supposed to deliver \"full spatial-temporal alignment\" is unsupported. The reported faithfulness gain (41.3% vs 28.5% deletion drop) compares the hybrid to ResNet only, never to the Transformer alone or to concatenation/weighted-average fusion, so it cannot demonstrate that multiplication is what helps. The dataset-size mismatch and real-time contradiction are independently serious, but the fusion rule is more load-bearing: if it is invalid, the central contribution collapses regardless of those issues.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an interpretability framework for multivariate time-series: a ResNet-18 branch with Grad-CAM, a 2D Transformer with attention rollout, a fusion step that combines the two heatmaps, and an NLP module that converts the fused heatmap into template-based or T5-generated narratives. The authors claim that the element-wise fusion H(t)=αG(t)⊙A(t) maximizes the mutual information I(H;Y), achieves 'full spatial-temporal alignment' while 'preserving real-time performance', and yields better predictive accuracy as well as better faithfulness than the individual branches. Experiments are reported on the PhysioNet ECG dataset and the UCI Appliances energy dataset, with BLEU/ROUGE scores and a small clinician study for the narratives. The paper also proposes a 'causal interpretability' equation involving do(t) interventions.","tokens_in":14477,"tokens_out":7590,"duration_ms":83107,"significance":"If the central claims were substantiated, the framework would address a real and well-motivated gap: combining the local precision of CNN-based attribution with the global coherence of Transformer attention, and then translating the resulting heatmap into stakeholder-readable text. The authors make a serious attempt at validation, including several baselines (ResNet, Transformer, LSTM, InceptionTime), statistical tests (Wilcoxon), and bootstrap confidence intervals, and they explicitly list limitations in Section VII. These are strengths. However, the theoretical justification for the fusion rule is not supplied, the experimental section does not actually isolate the proposed fusion mechanism, and the real-time claim in the abstract is directly contradicted by the paper's own Limitations section. As it stands, the contribution is a heuristic architecture with promising but unverified interpretability claims, and the paper does not provide the formal or empirical support needed for the claims made.","major_comments":[{"comment":"The central theoretical claim is unproven. Section III.3 states that element-wise multiplication H(t)=αG(t)⊙A(t) 'theoretically maximizes the mutual information I(H;Y)' and that multiplication dominates concatenation 'under Gaussian noise assumptions', but no statistical model, theorem, or proof is provided. The reference [13] is a general information theory textbook and does not contain a result of this kind. Moreover, the asserted inequality E[∥H_mul−H*∥2] ≤ E[∥H_cat−H*∥2] is not generally true: if G and A are unbiased estimators of the true importance H*, then under independence E[G⊙A]=H*², which is a biased estimator, and the variance of the product depends on the joint noise distribution and can exceed that of a weighted average. Since the claimed 'full spatial-temporal alignment' and the subsequent causal statements all rest on this fusion rule, the theoretical foundation of the paper is not established.","section":"III.3"},{"comment":"The paper's own description of the fusion mechanism is internally inconsistent. Section III.3 proposes element-wise multiplication as the fusion rule, but Section IV.C.2 defines fusion as static weighted averaging H_fused=α·H_ResNet+(1−α)·H_Transformer, with α tuned on validation data, or as a learned 1D-convolution fusion, and Section IV.C.3 reports an ablation in which adaptive fusion 'improved faithfulness metrics' and concatenation 'increased parameter count without significant performance gains' — with no results for multiplication. The experimental section never compares H=αG⊙A against the weighted-average, concatenation, or learned-fusion alternatives defined in Section IV.C. Therefore the reported improvements cannot be attributed to the specific multiplication rule that is the paper's headline contribution.","section":"III.3 and IV.C.2"},{"comment":"The real-time claim is contradicted by the paper's own Limitations. The abstract and Section I claim that the framework achieves 'full spatial-temporal alignment while preserving real-time performance', and Table 1 marks 'Ours' as real-time capable. Yet Section VII reports 30% additional inference latency from dual-model inference, states that no streaming or edge-device deployment tests were performed, and notes that transformer-based explanation generation introduces delays up to 500 ms. These are mutually inconsistent claims. The 'preserving real-time performance' statement should either be removed or be substantiated with end-to-end latency measurements against a stated deployment budget on the target platform.","section":"Abstract and VII"},{"comment":"The faithfulness evaluation is not sufficient to support the fusion claim. Section VI.D reports that masking top-20% heatmap regions reduces ECG accuracy by 41.3% versus 28.5% for ResNet, but this only compares the hybrid against one branch, not against Transformer-only heatmaps, weighted-average fusion, concatenation, or learned fusion. The sensor-pruning sentence then says that removing low-variance sensors improves UCI RMSE 'from 0.35 to 0.28 kWh'; those two numbers are, respectively, the InceptionTime baseline and the proposed hybrid model RMSE in Table 2, not a before/after pruning result on the same model. As written, these results do not demonstrate that the proposed fusion mechanism causes the reported interpretability or performance gains.","section":"VI.D"},{"comment":"The dataset description is inaccurate, which undermines the reproducibility of the headline accuracy numbers. Section V.A.1 says the experiments use the 'PhysioNet/Computing in Cardiology Challenge 2017 dataset' but cites reference [5], which is Rajpurkar et al.'s 2017 arrhythmia detection paper, not the Challenge dataset. The same paragraph states that there are five classes but then lists four ('normal sinus rhythm, atrial fibrillation, other arrhythmias, and noise'). The authors need to identify the exact dataset version, the class distribution, the recording lengths, and the preprocessing steps, and they must correct the reference.","section":"V.A.1"}],"minor_comments":[{"comment":"The text says the ECG dataset has five classes but lists four; correct the class count and terminology.","section":"V.A.1"},{"comment":"Table 3 is captioned 'Qwen-Generated Reports', but Section IV.D and the experimental text describe T5 and template-based generation; clarify which system actually produced the reported BLEU-4 and ROUGE-L scores.","section":"VI.C, Table 3"},{"comment":"Section IV.C.3 says adaptive fusion improved 'faithfulness metrics (e.g., AUC-ROC)', but faithfulness is defined elsewhere via deletion tests; the metric should be stated consistently.","section":"IV.C.3"},{"comment":"The Implementation Details list 'PyTorch 3.12.x', but PyTorch has no 3.12 release; this is likely intended to be a Python version, and the framework version should be corrected.","section":"V.C.1"},{"comment":"Heatmap notation is inconsistent: H_ResNet, H_Transformer, G(t), A(t), H_fused, and H_mul are all used without a single defining notation; define each term once and use it consistently.","section":"Throughout"},{"comment":"The user study with 10 clinicians lacks procedural details such as recruitment criteria, blinding, inter-rater agreement, and whether ethical approval was obtained; either report these details or characterize the results as informal feedback.","section":"IV.D.4"},{"comment":"Placeholder names such as '[Hospital Name]' and '[Industry Partner]' appear in the future-work discussion; these should be replaced with concrete descriptions or removed.","section":"VII and VIII"}],"recommendation":"reject","confidential_remarks":"I would not send this paper for major revision. The central theoretical assertion about the fusion rule is not derived, the experimental section does not test the proposed fusion rule against the alternatives defined in the architecture section, and the abstract's real-time claim is contradicted by the paper's own limitations. Correcting these issues would require re-deriving or repositioning the main contribution, running new ablations, and correcting the reported performance claims, which is beyond a normal revision. I would be willing to reconsider a substantially revised version that either supplies a valid derivation or honestly positions the fusion as a heuristic, and that evaluates all fusion variants in the same setting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. The paper's central fusion rule — multiply ResNet's Grad-CAM map by the Transformer's attention rollout — is presented as theoretically optimal, but the proof is not there. Section III.3 asserts that multiplication maximizes I(H;Y) and beats concatenation under Gaussian noise, with no statistical model or derivation. And the abstract promises \"preserving real-time performance\" while Section VII reports 30% extra inference latency and no real-time deployment tests. That is a direct contradiction.\n\nWhat is genuinely new: combining these two heatmaps into one visualization, with a template-based and T5-based NLP module that generates sentences like \"Elevated ST-segment between 2-4 seconds suggests myocardial ischemia.\" The empirical work includes ECGs and energy data, with some ablations on fusion strategies. That is a reasonable system configuration even if each piece is known.\n\nThe soft spots are load-bearing. The mutual-information claim and the inequality about multiplication vs. concatenation are stated without proof, so the mechanism that is supposed to deliver \"full spatial-temporal alignment\" is unsupported. The causal equation for P(y|do(t)) is also written down without defining the causal graph, so the causal fidelity language overreaches. The dataset facts are wrong: PhysioNet 2017 has four classes and ~8,528 recordings, not five and 10,000. The sensor-pruning sentence says removing low-variance sensors improves RMSE from 0.35 to 0.28, but those numbers are from the model comparison table (InceptionTime vs. the hybrid), so the improvement is not from pruning. No code is released despite the reproducibility section.\n\nThis paper is for readers who want a quick tour of how Grad-CAM, attention rollout, fusion, and NLP can be stitched together for time-series interpretability. As a published claim, it does not hold up. I would desk-reject it in its current form; the authors would need to provide a real derivation or at least a proper empirical comparison that isolates the fusion rule, correct the dataset description, and align the abstract with the limitations. If they do that, there may be a useful contribution underneath.","headline":"A reasonable system idea undercut by an unsupported fusion theory and a real-time claim that its own limitations section contradicts.","tokens_in":15027,"tokens_out":3778,"would_cite":false,"duration_ms":37317,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that multiplying a ResNet's Grad-CAM heatmap by a Transformer's attention-rollout map produces a single, temporally aligned explanation that is both locally precise and globally coherent, and that this fused explanation…","keywords":["Explainable AI","time-series interpretability","heatmap fusion","Grad-CAM","attention rollout","ECG arrhythmia detection","energy forecasting","NLP explanation generation"],"falsifier":"On a dataset with known ground-truth important timesteps, compute the fused, ResNet-only, and Transformer-only heatmaps and run deletion tests that mask the top-ranked regions; if the fused map's accuracy drop is not larger than the better single-branch map, the claimed fusion advantage is refuted.","tokens_in":13975,"feed_emoji":"📊","tokens_out":8452,"duration_ms":84240,"temperature":0.7,"pith_summary":"The paper claims that multiplying a ResNet's Grad-CAM heatmap with a Transformer's attention-rollout map, after time alignment and normalization, produces a single explanation heatmap that is both locally precise and globally coherent for time-series data. On two benchmarks—ECG arrhythmia classification and residential energy-use regression—the fused model reportedly reaches 94.1% accuracy (F1 0.93) and RMSE 0.28 kWh (R² 0.95), beating standalone ResNet, Transformer, LSTM, and InceptionTime baselines. An NLP stage converts the heatmaps into sentence explanations, scoring BLEU-4 0.586 and ROUGE-L 0.650. The authors also present interpretability as a causal-fidelity problem and claim that the fusion rule maximizes the mutual information between the explanation and the prediction, while acknowledging a roughly 30% inference-latency overhead from running both branches.","feed_headline":"Fused heatmaps align local and global time-series explanations","feed_subtitle":"Multiplying ResNet and Transformer attention maps gives one visualization, with 94.1% ECG accuracy.","key_machinery":"The load-bearing mechanism is the element-wise fusion rule H(t)=αG(t)⊙A(t), which combines a ResNet Grad-CAM map G(t) with a Transformer attention-rollout map A(t) after spatial alignment by bilinear interpolation and temporal alignment by dynamic time warping. The paper defends multiplication over concatenation by claiming that it acts as a consensus filter that maximizes mutual information I(H;Y) and reduces expected explanation error under Gaussian noise assumptions. Around this rule, the framework wraps a global attention aggregation (average of attention maps across heads and layers), min-max normalization and moving-average smoothing, and a two-path NLP generator (domain templates plus a fine-tuned T5 model) that renders fused heatmaps into readable sentences.","core_discovery":"The central claim is that the long-standing trade-off between local and global interpretability in time-series models can be removed by fusion rather than by choosing one architecture. A ResNet-18 branch produces a localized Grad-CAM heatmap; a modified 2D Transformer with a global attention module produces an attention-rollout heatmap that carries cross-channel, long-horizon context. After upsampling and dynamic time warping, the two maps are combined by element-wise multiplication, H(t)=αG(t)⊙A(t), which the paper argues suppresses uncorrelated activations and amplifies consensus regions, thereby maximizing the mutual information I(H;Y) between the explanation and the target. The output is temporally smoothed, normalized, and fed to a template or fine-tuned T5 generator that writes domain narratives such as \"Elevated ST-segment between 2 and 4 seconds suggests myocardial ischemia.\" The paper further casts the fusion as causal, stating that the derivative of the prediction probability under intervention at timestep t is proportional to the element-wise product of the ResNet gradient and the Transformer's counterfactual expectation. The empirical claim is that this fused explanation is more faithful than either branch alone: masking the top 20% of fused heatmap regions drops ECG accuracy by 41.3% versus 28.5% for ResNet alone, and low-variance sensors identified from the heatmaps can be pruned to improve energy regression RMSE.","pith_inferences":["A decisive test of the consensus principle would compare the product fusion against the average or concatenation of the same two heatmaps on a synthetic dataset with known ground-truth importance; if the product is not more faithful or sparser, the mutual-information rationale would not be what drives the reported gains.","If this local-global fusion recipe generalizes, it could be transferred to any paired architecture with complementary saliency maps, such as a CNN plus a vision Transformer on images, with spatial registration replacing dynamic time warping.","The NLP module is scored with translation-style metrics; an implication left implicit is that practical utility in clinical or industrial settings should ultimately be measured by decision outcomes, such as time-to-diagnosis or operator error rate, rather than by lexical overlap."],"forward_implications":["A single fused heatmap can carry both the localized precision of a CNN and the global context of a Transformer, which the paper identifies as the missing property in existing interpretability methods.","Masking the top 20% of the fused map degrades ECG accuracy by 41.3%, compared with 28.5% for the ResNet map alone, so the fused map is claimed to be more faithful to the model's decision.","Pruning sensors that the fused heatmap marks as low-variance improves energy regression RMSE from 0.35 to 0.28 kWh, so the explanations also serve as a feature-selection tool.","Domain narratives generated from the heatmap achieve BLEU-4 0.586 and ROUGE-L 0.650, providing a machine-checkable route from attention values to stakeholder-readable text."],"supporting_citations":[{"why":"Supplies the Grad-CAM method whose localized heatmap forms the ResNet branch of the fusion.","marker":"[6]"},{"why":"Supplies the attention-rollout procedure that produces the Transformer branch's global heatmap.","marker":"[8]"},{"why":"Provides the PhysioNet ECG dataset that grounds the classification experiments.","marker":"[5]"},{"why":"Provides the UCI energy appliance dataset used for the regression experiments.","marker":"[19]"},{"why":"Cited as the information-theoretic basis for the claim that the product fusion maximizes mutual information I(H;Y).","marker":"[13]"},{"why":"Supplies the T5 text-to-text model used for free-text explanation generation.","marker":"[17]"},{"why":"Supplies the BLEU-4 metric used to score generated explanations.","marker":"[22]"},{"why":"Supplies the ROUGE-L metric used to score generated explanations.","marker":"[23]"}],"fun_headline_variants":["Fusing heatmaps aligns local and global time-series explanations","ResNet + Transformer heatmap fusion hits 94.1% ECG accuracy","Multi-model heatmap fusion removes interpretability trade-off","Unified heatmap fusion yields local and global time-series insights"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the unproven claim that multiplying the two heatmaps produces an explanation that carries more information about the model's decision than either heatmap alone.","fun_headline_variants_meta":{"raw":{"variants":["Fusing heatmaps aligns local and global time-series explanations","ResNet + Transformer heatmap fusion hits 94.1% ECG accuracy","Multi-model heatmap fusion removes interpretability trade-off","Unified heatmap fusion yields local and global time-series insights"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1693,"prompt_tokens":1130,"completion_tokens":563,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":746,"completion_tokens_details":{"reasoning_tokens":492}},"tokens_in":746,"tokens_out":563,"duration_ms":6641,"temperature":1.0,"reasoning_tokens":492,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:20:53.198019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a dataset with known ground-truth important timesteps, compute the fused, ResNet-only, and Transformer-only heatmaps and run deletion tests that mask the top-ranked regions; if the fused map's accuracy drop is not larger than the better single-branch map, the claimed fusion advantage is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Grad-CAM method whose localized heatmap forms the ResNet branch of the fusion."},{"cited_title":"Abnar and W","cited_arxiv_id":null,"evidence_quote":"Supplies the attention-rollout procedure that produces the Transformer branch's global heatmap."},{"cited_title":"Candanedo, D","cited_arxiv_id":null,"evidence_quote":"Provides the UCI energy appliance dataset used for the regression experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cited as the information-theoretic basis for the claim that the product fusion maximizes mutual information I(H;Y)."},{"cited_title":"Raffel, N","cited_arxiv_id":null,"evidence_quote":"Supplies the T5 text-to-text model used for free-text explanation generation."},{"cited_title":"Papineni, S","cited_arxiv_id":null,"evidence_quote":"Supplies the BLEU-4 metric used to score generated explanations."},{"cited_title":"Lin, ‘‘Rouge: A package for automatic evaluation of summaries,’’ in ACL Workshop, 2004","cited_arxiv_id":null,"evidence_quote":"Supplies the ROUGE-L metric used to score generated explanations."}],"review_version":1}