{"id":"68b59641-0de8-4829-86cc-f4f35e1b666e","arxiv_id":"2505.11612","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A wearable-ECG system with a time-frequency transformer and a contestable chatbot reports 91.7% accuracy for separating schizophrenia/bipolar patients from controls on a 60-person dataset.","lead":"This paper builds a three-part system that reads heart-rhythm data from a wearable chest strap, classifies it as healthy or possibly needing psychiatric care, and lets a clinician question the AI's answer through a chatbot. The authors report 91.7% accuracy on a 60-person dataset and argue that spotting disagreements between two explanation methods can catch the model's mistakes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline accuracy is computed on heavily overlapping windows as independent observations; per-subject evaluation is the missing load-bearing check.","rationale":"The paper describes a coherent prototype with public code, a detailed architecture, and an honest limitations section; those are genuine strengths. The strongest claim, however, is the MSTFT's 91.7% accuracy, and the reader correctly identified that the evaluation treats heavily overlapping sliding windows as independent observations. Section 5.1's stride-1 windowing with T=300 and per-participant z-scoring means each participant contributes thousands of near-duplicate test windows, so the reported accuracy, precision, and recall are window-level statistics, not per-patient diagnostic accuracy. The paper even acknowledges in Section 6.2.1 that threshold calibration across diverse populations remains future work, and Section 6.2.2 states that full clinical validation requires more extensive real-world testing. These self-stated limitations reinforce, rather than resolve, the concern. The concrete test I propose would settle whether the headline number survives subject-level aggregation, which is the minimal condition for the abstract's claim to be clinically meaningful. I agree with the reader's weakest assumption, and the conditional verdict remains appropriate: the work is promising but the central accuracy claim needs per-subject validation before it can be taken as stated.","tokens_in":33879,"tokens_out":3309,"duration_ms":34849,"concrete_test":"Re-run the LOOCV experiment and aggregate predictions at subject level before computing accuracy: for each held-out participant, take the majority vote over their T=300 windows (and, separately, over stride-300 non-overlapping windows), then report accuracy, F1, and an exact 95% binomial CI over the 60 participants. Also rerun with z-scoring fit only on the 59 training participants before windowing. If the per-subject majority-vote accuracy is not close to 91.7% with a CI excluding chance, or if non-overlapping windows sharply reduce performance, the abstract's accuracy claim is not supported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the MSTFT's 91.7% LOOCV accuracy. That number supports a psychiatric screening claim only if it approximates per-patient diagnostic accuracy. Section 5.1 (Eq. 41) builds windows of T=300 RRI samples with stride 1 after per-participant z-scoring, so one participant yields thousands of windows that overlap in 299 of 300 samples. Section 5.2.1 says LOOCV tests a single held-out subject, but Table 4b reports metrics over these windows, not over the 60 participants. The effective sample size is 60, not the tens of thousands of windows; no confidence intervals are given. The per-window independence assumption is therefore the load-bearing statistical premise behind the headline number. Additionally, per-participant z-scoring before windowing uses the held-out subject's full signal statistics, so test-window normalization depends on test-subject data. The SAE discrepancy threshold is fitted on the same evaluation cases, and the LLM contestation result rests on only 6 error cases; these are secondary concerns. The primary gap is that the headline accuracy has not been shown to be per-patient accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Heart2Mind, a full-stack system for psychiatric disorder screening that combines a wearable ECG/RRI monitoring interface (CMI), a Multi-Scale Temporal-Frequency Transformer (MSTFT) classifier, and a Contestable Diagnosis Interface (CDI) built on Self-Adversarial Explanations (SAEs) and contestable LLMs. The central technical claims are that MSTFT achieves 91.7% accuracy on the HRV-ACC dataset under leave-one-out cross-validation, outperforming published baselines, and that SAEs detect unreliable model predictions by comparing attention-based and gradient-based explanations, which then enables LLMs to validate correct predictions and contest incorrect ones. The manuscript includes detailed architecture descriptions, hyperparameters, prompt templates, case studies from LLM outputs, and a public code repository.","tokens_in":34079,"tokens_out":2666,"duration_ms":29444,"significance":"If the headline accuracy were demonstrably per-patient diagnostic accuracy, the system would make a meaningful contribution: it combines a novel time-frequency transformer for RRI classification, a concrete implementation of contestable AI using explanation discrepancies, and an interactive LLM-based clinician interface, all in a reproducible open-source form. The strengths of the paper are its systems-level integration, the delivery of a working artifact, and the explicit engagement with emerging regulatory notions of contestability. However, the statistical evaluation does not currently support the per-patient screening claim: the reported metrics are computed over heavily overlapping windows rather than over patients, and the SAE discrepancy threshold is fitted on the same cases used to demonstrate its utility. These issues put the central quantitative claims on uncertain ground and need to be resolved before the results can be accepted.","major_comments":[{"comment":"The headline 91.7% LOOCV accuracy is computed over sliding windows of length T=300 with stride 1, so successive test inputs overlap in 299 of 300 samples and each participant contributes thousands of near-duplicate windows. LOOCV holds out one participant, but the metrics in Table 4b are aggregated over windows, not over the 60 participants. The effective sample size is therefore 60, not the tens of thousands of windows, and the reported accuracy, precision, recall, F1, and AUC do not establish per-patient diagnostic accuracy. A per-participant evaluation (e.g., majority vote over each subject's windows, or a mixed-effects model with participant as a random effect) with confidence intervals is the load-bearing check required to support the abstract's screening claim.","section":"Section 5.1, Eq. (41) and Table 4b"},{"comment":"The manuscript states that 'before windowing, we rescaled each participant's RRI signal to zero mean and unit variance.' In LOOCV, the held-out subject's full-signal mean and standard deviation are therefore used to normalize that subject's test windows, meaning test-window values depend on statistics of the test subject's entire recording. This is a form of information leakage and can inflate performance measures. The authors should either use normalization statistics derived only from training subjects or provide evidence that the reported performance is insensitive to this choice.","section":"Section 5.1, preprocessing before windowing"},{"comment":"The SAE discrepancy-detection claim is partially fitted rather than validated. The threshold rho = 0.5 in Eq. (32) is set empirically, and the paper's key operational criterion—that roughly 5-6 discrepancy regions indicate unreliable predictions—is inferred from the discrepancy counts of the same MSTFT checkpoint's correct and incorrect outputs (Figure 10). This is circular when the same cases are then used to claim that SAEs flag unreliable predictions. A held-out validation set, or a pre-specified threshold selection procedure, is needed before the discrepancy count can be claimed as an inference-time uncertainty quantification mechanism.","section":"Section 5.3.1, Eq. (32) and Figure 10"},{"comment":"Several baselines (Buza et al., Książek et al.) are imported directly from their original publications, while other baselines are re-implemented, without a common preprocessing and evaluation pipeline across all methods. No confidence intervals or significance tests accompany the comparisons in Table 4a or 4b. The claim of outperforming state-of-the-art methods is therefore not statistically substantiated; the authors should report paired per-participant comparisons, confidence intervals, or significance tests, ideally using a shared windowing and normalization protocol.","section":"Section 5.2.1 and Table 4b"}],"minor_comments":[{"comment":"The text states that precision of 0.963 means 'fewer than 4% of health controls were wrongly flagged as positive.' This is an incorrect interpretation: precision is the fraction of predicted positives that are true positives, not the false positive rate among controls. The sentence should be rephrased.","section":"Section 5.2.3"},{"comment":"The sentence 'These pooled representations are concatenated to form a comprehensive feature vector' appears twice in consecutive lines; one instance should be deleted.","section":"Section 4.2.6"},{"comment":"The use of Dynamic Time Warping to align attention-based and gradient-based explanations is described only in one line. It would clarify the method to state whether DTW is used for temporal alignment, how the aligned maps are returned to the original time grid, and whether the alignment affects the threshold comparison in Eq. (32).","section":"Section 4.3.1, Eq. (30)"},{"comment":"The contestable LLM evaluation rests on only 6 erroneous predictions (3 FN and 3 FP cases), and the conclusion that 'all three LLMs successfully contested at least one erroneous prediction' is drawn from this very small sample. The paper should explicitly caveat the statistical fragility of these numbers, including in the abstract's claim of 'successfully challenging 50% of erroneous ones.'","section":"Section 5.3.2 and Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a systems contribution with a reproducible implementation, and the per-participant evaluation issue is fixable within the manuscript's scope by re-analyzing the existing data. I would not reject on the current evidence, but the headline numbers should not be published without subject-level evaluation and significance testing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the headline 91.7% LOOCV accuracy is not yet a per-patient number, and the paper doesn't say how subject-level predictions are derived from the thousands of windows per person. Don't repeat the number; do look at the SAE mechanism and the contestable LLM loop, which are the actual contributions.\n\nWhat's new and good: SAEs compare DTW-aligned attention maps against gradient-based maps, find discrepancy regions, and compute HRV features within those regions. That is a concrete, plausible safeguard, and I haven't seen it done this way. The contestable LLM interface with a structured prompt, three open models, and an explicit retain/overturn protocol is a thoughtful step toward contestability in practice. The implementation is public, and the limitations section honestly flags threshold calibration and the need for clinical studies. The architecture itself is incremental—dilated convolutions, wavelet-like separable convs, cross-attention, self-attention—but sensible.\n\nWhere it's soft: (1) The main evaluation. Section 5.1 constructs stride-1 windows of T=300; one subject yields thousands of overlapping windows. LOOCV holds out one subject, but Table 4 reports no subject-level aggregation and no confidence intervals. Then Section 5.3 casually reports per-subject counts (27 TP, 27 TN, 3 FN, 3 FP), which implies majority voting or something similar—but that step is never described. That missing detail carries the headline claim. (2) Baselines for Buza et al. and Książek et al. are imported from the original papers, so the comparison isn't apples-to-apples. (3) The discrepancy threshold rho=0.5 and the '5-6 regions' flag are derived from the same evaluation data; the authors acknowledge this but it still limits the claim that SAEs detect errors prospectively. (4) The LLM contestation result is 3 of 6 errors overturned by the best model; that's a pilot, not a result.\n\nOverall: the direction is sound, and the problems are fixable. Per-subject majority-vote or mixed-effects accuracy with intervals, explicit aggregation, and a held-out threshold calibration would address the main objections. I'd send it to a serious venue—referees can push on this—but not accept the current empirical claims as stated. The SAE idea is worth citing once the evaluation is tightened.","headline":"A convincing contestable-AI prototype whose headline accuracy is not yet shown to be per-patient; the SAE mechanism and public code are worth engaging.","tokens_in":34651,"tokens_out":3535,"would_cite":false,"duration_ms":37559,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper aims to show that a chest-strap ECG, a time-frequency transformer, and a clinician chatbot can together screen for schizophrenia and bipolar disorder, with the model reaching 91.7% accuracy on a 60-person dataset and the chatbot…","keywords":["psychiatric disorder diagnosis","wearable ECG","heart rate variability","R-R intervals","transformer classifier","explainable AI","contestable AI","large language models"],"falsifier":"Evaluate MSTFT at the patient level: take all windows from each of the 60 participants, form one prediction per participant by majority vote, and build the 60-subject confusion matrix. If patient-level accuracy is close to chance while window-level accuracy stays high, the central screening claim is refuted.","tokens_in":33656,"feed_emoji":"🫀","tokens_out":7787,"duration_ms":74028,"temperature":0.7,"pith_summary":"The paper sets out to show that the full loop of psychiatric screening—heart-signal capture, model prediction, explanation, and human override—can be assembled around a consumer chest-strap ECG. Its core model, a Multi-Scale Temporal-Frequency Transformer, is reported to classify R-R interval windows as control or treatment (schizophrenia/bipolar disorder) with 91.7% accuracy on the 60-participant HRV-ACC dataset under leave-one-out cross-validation, above the published baselines it compares against. The system then uses self-adversarial explanations to flag predictions whose attention-based and gradient-based explanations disagree, and a clinician-facing chatbot that can confirm or overturn the model's decision. If these results hold, wearable heart-rate variability could become an objective, accessible screening signal for schizophrenia and bipolar disorder while keeping the clinician as the final decision-maker.","feed_headline":"Wearable ECG plus contestable AI hits 91.7% in psychiatric screening","feed_subtitle":"A transformer reading heart-rate intervals beats prior models, and a chatbot lets clinicians overturn its mistaken calls.","key_machinery":"The load-bearing object is the Multi-Scale Temporal-Frequency Transformer (MSTFT), a classifier that reads 300-beat sliding windows of R-R intervals and passes them through parallel branches: dilated causal convolutions with stochastic skips for temporal patterns, and separable convolutions acting as learnable wavelet transforms for frequency patterns. A cross-attention block fuses the two branches by treating temporal features as queries and frequency features as keys and values, followed by a gated multi-head self-attention block and a pooled classification head. Around this model, the paper builds two further mechanisms: Self-Adversarial Explanations (SAEs), which align averaged attention maps with gradient-based maps via dynamic time warping and threshold their absolute difference to find regions of disagreement; and a contestable LLM prompt that receives the baseline prediction, whole-signal HRV metrics, and discrepancy-region HRV metrics so the chatbot can justify, retain, or overturn the diagnosis. The classifier is the engine, the SAE discrepancy count is the built-in safeguard, and the LLM is the channel through which clinicians exercise contestation.","core_discovery":"The central claim is that R-R interval time series from a single-lead wearable ECG carry enough autonomic information to separate people with schizophrenia/bipolar disorder from healthy controls, and that this information can be presented to clinicians in a form they can examine and dispute. The paper reports that MSTFT reaches 91.7% accuracy, 0.963 precision, 0.867 recall, and 0.940 AUC under leave-one-out cross-validation on HRV-ACC, beating 1D-CNN, a plain Transformer, and the published results of prior methods. It further reports that correct predictions have few discrepancies between attention-based and gradient-based explanations (means of 0.72–0.78 regions) while incorrect predictions have many (means of 7.0–7.67 regions), so the discrepancy count can flag unreliable predictions during inference. Finally, all three tested large language models retained every one of the 54 correct baseline predictions, and they overturned one, one, and three of the six incorrect predictions respectively, with the strongest model correcting half of the baseline errors.","pith_inferences":["A stricter patient-level test would aggregate each participant's window predictions (for example by majority vote) and report the 60-subject confusion matrix; the current paper reports window-level metrics only, so the patient-level diagnostic rate remains an open question.","The SAE discrepancy mechanism could be reused as a label-free uncertainty estimator for other transformer classifiers on physiological time series, since it only needs attention weights and gradients.","Because the three LLMs overturn different subsets of errors, an ensemble that combines their votes would plausibly overturn more than any single model; this is a natural extension the paper does not test.","Adding other wearable streams, such as electrodermal activity or motion, could turn the binary screening into a finer distinction among schizophrenia, bipolar disorder, and healthy states; the paper identifies this as a direction but does not pursue it."],"forward_implications":["A consumer chest-strap ECG can support objective, continuous screening for schizophrenia and bipolar disorder outside the clinic, potentially shortening the path from symptoms to treatment.","MSTFT provides a new reference result on the HRV-ACC dataset; future HRV-based psychiatric classification work will need to report comparable leave-one-out metrics to be measured against it.","The discrepancy count between explanation types can act as an inference-time uncertainty flag, sending only suspicious predictions to human review instead of requiring every case to be checked.","Contestable LLMs give clinicians a natural-language channel to validate or override model decisions, matching the regulatory push toward contestability in AI-assisted healthcare.","Because the three LLMs reach at least one correct overturn without medical fine-tuning, domain-specific tuning or ensembling is a direct route to higher correction rates."],"supporting_citations":[{"why":"Supplies the HRV-ACC dataset of 60 participants' RRI time series from a Polar H10 chest strap on which all experiments are run.","marker":"[46]"},{"why":"Publishes the 80-83% accuracy baseline on the same dataset that MSTFT is claimed to outperform.","marker":"[44]"},{"why":"Provides the convolutional nearest-neighbor RRI classifier whose published leave-one-out results serve as a comparison point.","marker":"[17]"},{"why":"Provides the attention-guided deep learning baseline for psychiatric disorder detection that MSTFT and SAEs are compared against.","marker":"[61]"},{"why":"Supplies the finding that attention maps alone are not explanations, motivating the comparison of attention-based and gradient-based explanations in SAEs.","marker":"[37]"},{"why":"Supplies the faithfulness-violation test concept underlying the use of explanation disagreement to detect unreliable model behavior.","marker":"[52]"},{"why":"Establishes the physiological premise that heart rate variability tracks disease severity in psychosis spectrum disorders, grounding RRI as a biomarker.","marker":"[11]"},{"why":"Provides the contested-AI-by-design framework that the contestable LLM interface operationalizes.","marker":"[4]"}],"fun_headline_variants":["Contestable ECG AI reaches 91.7% for psychiatric screening","Wearable ECG AI lets clinicians contest its 91.7% diagnoses","ECG transformer with clinician override hits 91.7% accuracy","ECG data + contestable AI: 91.7% in psychiatric detection","Wearable ECG AI gives psychiatrists a say in its 91.7% diagnoses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the tens of thousands of heavily overlapping 300-beat windows cut from each person's 70–120 minute recording can be treated as roughly independent observations, so the 91.7% accuracy is a per-window number rather than a demonstrated per-patient diagnostic accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Contestable ECG AI reaches 91.7% for psychiatric screening","Wearable ECG AI lets clinicians contest its 91.7% diagnoses","ECG transformer with clinician override hits 91.7% accuracy","ECG data + contestable AI: 91.7% in psychiatric detection","Wearable ECG AI gives psychiatrists a say in its 91.7% diagnoses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001473,"raw_usage":{"total_tokens":5980,"prompt_tokens":1060,"completion_tokens":4920,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":676,"completion_tokens_details":{"reasoning_tokens":4818}},"tokens_in":676,"tokens_out":4920,"duration_ms":29638,"temperature":1.0,"reasoning_tokens":4818,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:51:22.821150+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate MSTFT at the patient level: take all windows from each of the 60 participants, form one prediction per participant by majority vote, and build the 60-subject confusion matrix. If patient-level accuracy is close to chance while window-level accuracy stays high, the central screening claim is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the HRV-ACC dataset of 60 participants' RRI time series from a Polar H10 chest strap on which all experiments are run."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Publishes the 80-83% accuracy baseline on the same dataset that MSTFT is claimed to outperform."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the attention-guided deep learning baseline for psychiatric disorder detection that MSTFT and SAEs are compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the finding that attention maps alone are not explanations, motivating the comparison of attention-based and gradient-based explanations in SAEs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the faithfulness-violation test concept underlying the use of explanation disagreement to detect unreliable model behavior."}],"review_version":1}