{"id":"aa32c0fd-38ca-4760-b67d-4b584015865d","arxiv_id":"2506.23648","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A regression-based deep learning model with feature selection, amplification, and Mixture-of-Experts pooling grades mitral regurgitation from 4-chamber color Doppler videos with 89.36% accuracy.","lead":"Researchers built MReg, a deep learning model that reads color Doppler ultrasound videos of the heart and grades mitral regurgitation as normal, mild, or moderate-severe. It combines automatic selection of the most informative frames, local feature amplification, and expert-style pooling, and reports higher accuracy than several video classification baselines on a private dataset of 1,868 cases.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported F1 scores are mathematically inconsistent with the paper's own Precision/Recall values, undermining the SOTA comparison.","rationale":"Good-faith reading: MReg is a plausible application of MIL-style feature selection, feature amplification, and MoE feature summarization to video-based MR grading; the ablations in Table 2 support the contributions of FS and Amp, and releasing code is a positive. The central claim, however, is the SOTA result in Table 1. My stress-test focused on the numerical consistency of that table. The F1 column cannot be reconciled with the reported Precision and Recall under any standard definition: for micro-averaged metrics, F1 <= min(P,R), and for macro-averaged metrics, average F1_i <= average min(P_i,R_i) <= min(average P_i, average R_i). The table reports F1=94.28 alongside Precision=86.83 and Recall=85.93, violating this bound. The same pattern repeats across every row, so this is not a one-off typo in the MReg row. Because F1 is one of the headline metrics highlighted in the abstract, the evidence for 'state-of-the-art' is internally inconsistent. The paper may still be correct, possibly because F1 is computed one-vs-rest for a single grade or with a nonstandard weighting, or because the Precision/Recall columns are not the averages used to compute F1. A confusion-matrix reproduction would settle this quickly. This concern is distinct from the reader's ground-truth-label concern, which is valid but concerns external validity; the metric issue is a direct check on the paper's own numbers. I therefore recommend keeping the CONDITIONAL verdict, with the added condition that the authors provide test confusion matrices and a precise definition of F1.","tokens_in":7130,"tokens_out":13881,"duration_ms":162626,"concrete_test":"Release or reconstruct the full 3x3 test confusion matrix for MReg and each baseline, then compute Accuracy, micro Precision/Recall/F1, macro Precision/Recall/F1, and class-wise F1 using standard formulas. Check whether any standard definition reproduces Table 1 exactly, especially F1=94.28 with Precision 86.83 and Recall 85.93. If no standard definition reproduces the table, request a corrected table or an explicit nonstandard metric definition before relying on the SOTA claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3 (Table 1), MReg is reported with Accuracy 89.36, Recall 85.93, Precision 86.83, Specificity 86.36, and F1 94.28. Under any standard multi-class averaging (micro or macro), F1 = 2*P*R/(P+R) <= min(P,R), so an F1 of 94.28 cannot coexist with Recall 85.93 and Precision 86.83. The same violation appears in every row of Tables 1 and 2: for example, Video-Swin has F1 93.18 with Recall 85.89, and the Baseline has F1 90.09 with Recall 75.57. Unless F1 is defined for a single class or with a nonstandard weighting, the headline F1 column is not computed from the reported Precision/Recall values. Since the central SOTA claim rests on these numbers, the evaluation is internally inconsistent and requires clarification.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MReg, a regression-based framework for automated mitral regurgitation (MR) diagnosis from four-chamber color Doppler echocardiography videos. The method combines MIL-based instance selection, feature amplification, and an MoE-based feature summary module on top of X-CLIP video features, and is trained with a combination of binary focal loss, MSE regression loss, expert focal loss, and smoothing/sparsity losses. Experiments are reported on an in-house dataset of 1,868 A4C-CDV cases with three grades (normal, mild, moderate-severe), comparing against supervised video classification and weakly supervised video anomaly detection baselines. The authors report state-of-the-art accuracy (89.36%), recall, precision, specificity, and F1 on a held-out test split, with code released.","tokens_in":7341,"tokens_out":4452,"duration_ms":48662,"significance":"If the reported results are reliable, MReg would be a practically useful automated MR grading tool with a sensible clinical motivation (regression over ordinal grades, selection of the most informative cardiac cycle, and amplification of regurgitation features). The paper provides a held-out test split, a clear architecture description, and comparisons to several baselines. However, the evaluation section has unresolved issues: the F1 values are not consistent with the reported precision/recall under common averaging schemes, the ground-truth labels come from a single sonographer with no variability analysis, and all results are single-run without confidence intervals. These issues directly affect the strength of the central SOTA claim.","major_comments":[{"comment":"The F1 column is not derivable from the reported Precision and Recall values under micro or weighted averaging. For example, MReg has Precision=86.83 and Recall=85.93, whose harmonic mean is approximately 86.38, yet the table reports F1=94.28. The same pattern appears for Video-Swin (84.39 harmonic vs. reported 93.18) and Baseline (78.35 harmonic vs. reported 90.09). If F1 is macro-averaged, this can be reconciled because macro F1 is not constrained by global precision/recall, but the paper must state the averaging scheme explicitly and report per-class precision, recall, and F1. As written, the headline F1 numbers are not reproducible from the reported metrics and undermine the SOTA comparison.","section":"Section 3, Tables 1 and 2"},{"comment":"The ground-truth labels for all 1,868 cases were provided by a single experienced sonographer, with no inter-observer or intra-observer variability reported, no second-reader adjudication, and no external reference standard (e.g., cardiac MRI or quantitative Doppler parameters). Since every reported accuracy, recall, precision, specificity, and F1 value is computed against these labels, any label subjectivity or noise directly limits the validity of the central clinical claim. The authors should report label variability on a subset or at minimum thoroughly discuss this limitation in the paper.","section":"Section 3, Dataset and Implementations"},{"comment":"All metrics are reported from a single training run with no confidence intervals, standard deviations, or repeated trials. The P-values in the tables are only comparisons of each method against MReg (Chi-square test), not pairwise significance tests among ablations or baselines. Given the small number of Grade 2 test cases (98) and the observed sensitivity of results to module changes (e.g., Accuracy 88.74 vs. 88.37 in Table 2), the claim that MReg 'significantly outperforms' all competing methods needs variance estimates and appropriate statistical testing to be convincing.","section":"Tables 1 and 2"},{"comment":"The contribution of the MoE-based feature summary module is not isolated in the ablation study. Adding MoE without the auxiliary expert loss (Baseline + FS + Amp + MoE) slightly decreases Accuracy and F1 compared to Baseline + FS + Amp (88.37 vs. 88.74 and 93.90 vs. 93.98, respectively), and only the full MReg with Lexpert improves accuracy to 89.36. The paper should provide an ablation that separates the effect of the MoE module from the effect of Lexpert, and explain why MoE alone degrades performance. This is important because the MoE module is presented as a key contribution.","section":"Section 2.2 and Table 2"}],"minor_comments":[{"comment":"The feature dimension notation for f3_img,text is confusing: it is written as R3×16×196×3×512, which likely contains a typo. Please clarify the intended tensor shape.","section":"Section 2, Methodology"},{"comment":"Please define how F1-score is computed (micro, macro, or weighted) and report per-class precision/recall/F1, especially for the Grade 2 category, which is critical for clinical use.","section":"Section 3, Metrics"},{"comment":"The phrase 'as largest gaps: ∼0.05 vs. ∼4 in C' is unclear. Please clarify what the numbers refer to and how the gaps were measured.","section":"Figure 4, caption"},{"comment":"The caption and text should state exactly what statistical test is used (e.g., Chi-square on which contingency table) and what hypothesis is being tested. Currently the P-value column is underspecified.","section":"Tables 1 and 2, P-Value column"},{"comment":"The thresholds (thre1=0.5, thre2=1.5), amplification coefficient β=2, and loss weights λ1=0.01, λ2=0.001 are set manually. A sensitivity analysis for these key hyperparameters would strengthen the paper.","section":"Section 2.1, Hyperparameters"},{"comment":"In the text describing Table 2, 'rows 3-4' should clearly map to the Baseline+FS and Baseline+FS+Amp rows; the current wording ('rows 3-4, Table 2') is ambiguous because the table rows include additional variants.","section":"Section 3, Ablation description"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a clinically motivated and potentially useful method, and the authors have released code, which is commendable. The main concern is the evaluation: the F1 inconsistency in Tables 1 and 2 must be resolved by specifying the averaging scheme and reporting per-class metrics, and the single-run, single-reader evaluation needs to be strengthened with variability analysis and confidence intervals. If these issues are addressed, the paper may be suitable for publication. I recommend major revision rather than rejection because the core architecture and experimental design are largely sound and the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: MReg is a sensible application of known video-understanding blocks to an important clinical task, and the authors deserve credit for releasing code and for a clear ablation structure. But the headline evaluation is internally inconsistent: every F1 in Tables 1 and 2 is larger than the harmonic mean of the reported Precision and Recall. For MReg, F1 = 2*86.83*85.93/(86.83+85.93) ≈ 86.4, not 94.28. Same for Video-Swin (93.18 vs ~84.4) and Baseline (90.09 vs ~78.3). No standard micro/macro F1 can exceed that bound, so the table as printed is wrong. That is a load-bearing problem because \"state-of-the-art\" is the paper's core claim.\n\nWhat is actually new: applying a regression objective to MR severity video grading, with MIL-based instance selection and feature amplification that mirror sonographer workflow, and a three-expert MoE summary to decouple category features. The idea is coherent and the ablations suggest FS and Amp help. The methods text is clear enough to reproduce, and the code being public is a real plus.\n\nOther soft spots are less severe but worth saying. The labels come from a single experienced sonographer with no inter-observer variability, and the ground truth is measured against no external reference. Moderate and severe MR are collapsed into one \"intervention\" grade, which hides ordinal information that the regression task is supposed to capture. No confidence intervals or repeated runs are reported. The thresholds (0.5, 1.5), loss weights, and beta are manually set. These are normal caveats for a medical imaging paper, but combined with the F1 issue they mean the quantitative claims cannot be trusted as printed.\n\nBottom line: this is not a desk-reject. The architecture and clinical motivation deserve a serious referee, and the paper could become solid after the tables are corrected and the metrics clarified. But I would not cite the reported numbers until the authors either recompute F1 from the confusion matrix or clearly state the nonstandard definition they used. The reader's conditional verdict is close; the stress-test finding is correct and should be the first thing the authors are asked to address.\n\nRecommendation: send to peer review; require the authors to fix the metric computation, add uncertainty quantification, and ideally get a second reader for at least a subset of labels.","headline":"Sensible architecture and clear clinical motivation, but the reported F1 scores are arithmetically impossible given the paper's own Precision/Recall values, so the headline SOTA claim is currently unsupported.","tokens_in":7871,"tokens_out":3843,"would_cite":false,"duration_ms":41488,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MReg reports 89.36 percent accuracy in grading mitral regurgitation from color Doppler video, using regression and MoE feature mining to outperform classification baselines.","keywords":["mitral regurgitation","color Doppler echocardiography","video regression","mixture-of-experts","feature selection","feature amplification","multiple instance learning","medical video analysis"],"falsifier":"Grade a held-out set of 200 videos by three independent cardiologists with consensus adjudication and compare MReg's predictions against the consensus grades, or against quantitative Doppler measurements such as effective regurgitant orifice area; if MReg's accuracy and F1 do not remain above the compared baselines on this reference labeling, the central claim is falsified.","tokens_in":6967,"feed_emoji":"💓","tokens_out":5304,"duration_ms":48760,"temperature":0.7,"pith_summary":"This paper introduces MReg, an automated model that grades mitral regurgitation (MR) from four-chamber color Doppler echocardiography videos into normal, mild, and moderate-to-severe categories. The authors argue that MR grading should be treated as a regression problem rather than classification, because severity is a continuous, ordinal spectrum. To capture the relevant visual evidence, MReg selects the cardiac-cycle instance most likely to contain regurgitation, amplifies its features by combining the original video features with a convolution applied to their L2 norm, and uses a Mixture-of-Experts module whose three experts each summarize features for one severity grade. On an in-house dataset of 1,868 videos, the model reports Accuracy 89.36%, Recall 85.93%, Precision 86.83%, Specificity 86.36%, and F1-score 94.28%, outperforming the compared supervised video classifiers and weakly supervised anomaly detectors.","feed_headline":"Video AI model grades mitral regurgitation at 89 percent accuracy","feed_subtitle":"MReg mines color Doppler echo videos with feature selection, amplification, and MoE grading to beat classification baselines.","key_machinery":"The central object is the two-stage X-CLIP-based pipeline. Stage I performs binary normal-versus-MR classification with focal loss and chooses the instance index $\\alpha = \\arg\\max(f_{\\text{2cls\\_out}}[:, 1])$ with the highest MR probability. Stage II computes an amplified video feature $f_{\\text{amp}} = N(f_{\\text{video}} + \\beta \\times \\text{Conv}(\\|f_{\\text{video}}\\|_2))$ and feeds it, together with three-category text features, into a MoE module with one expert per grade; each expert's weight is $w_{E_i} = \\text{SoftMax}(\\|W_i\\|_F)$, and the final regression output is mapped to three grades by thresholds (0.5 and 1.5). The total loss combines binary focal loss, MSE regression loss, an expert focal loss, and smoothing/sparsity regularizers.","core_discovery":"The central claim is that combining three feature mining strategies—instance selection via multiple instance learning, feature amplification, and MoE-based feature summary—within a regression formulation yields a clinically aligned and accurate MR diagnosis system. Specifically, the model first uses X-CLIP video and text features, then selects the instance with the highest MR probability to focus on the most severe regurgitation, amplifies regurgitation-related signals via a residual convolution on the L2-normalized feature map, and finally sums category-specific expert features to improve grading of complex cases. The regression loss with thresholds preserves ordinal continuity between grades. The paper's experiments show that each added module improves recall, especially for the underrepresented moderate-to-severe grade, and that the full model outperforms all compared SVC and WS-VAD methods on every reported metric.","pith_inferences":["The single-sonographer labels are treated as ground truth with no inter-observer variability reported; if label noise varies by grade, the reported confusion-matrix improvements for Grade 2 (56.12 to 76.53 recall) could partly reflect the labeler's bias rather than true disease severity.","The thresholds 0.5 and 1.5 on the regression output are fixed hyperparameters; a clinically deployed system would likely need them calibrated against quantitative reference standards such as effective regurgitant orifice area or regurgitant volume.","The model currently merges moderate and severe MR into one intervention class; extending to four grades would require retraining the MoE to four experts and would likely need additional data for the severe class.","Because X-CLIP is pretrained on Kinetics-400 action videos, the method should transfer to other echocardiographic views (e.g., parasternal long axis) with only fine-tuning, but this is untested in the present study."],"forward_implications":["If MReg's results hold, automatic MR screening can run on standard A4C color Doppler videos without manual keyframe selection, since the MIL-based instance picker replaces the sonographer's search for the worst jet.","The regression formulation with thresholds (0.5 and 1.5) implies that the same network can be recalibrated for different clinical cutoffs without retraining, because the output is a continuous severity score rather than a discrete class.","Feature amplification increases the gap between decision frames and other frames in an instance, making the model's decision frame dominant and improving interpretability of which part of the cardiac cycle drove the prediction.","The MoE feature summary assigns each expert to a severity grade; this category-level decoupling is what handles eccentric and multi-jet regurgitation, where jet area alone is ambiguous.","Equipping the model with $L_{\\text{expert}}$, a focal loss on the unmixed expert outputs, resolves the performance drop that occurs when adding MoE without supervision, bringing the full model to its best metrics."],"supporting_citations":[{"why":"Provides the X-CLIP pretrained video and text feature extractor used as the backbone of the whole framework.","marker":"[13]"},{"why":"Video-Swin is the strongest supervised video classification baseline that MReg must outperform to support its state-of-the-art claim.","marker":"[10]"},{"why":"AnomalyCLIP is the weakly supervised video anomaly detection baseline that shares a similar weak-label mechanism and is used for comparison.","marker":"[21]"},{"why":"ViVit is one of the supervised video transformer baselines in the comparison table.","marker":"[1]"},{"why":"ActionCLIP is another supervised video classification baseline used to benchmark MReg.","marker":"[19]"},{"why":"Focal loss supervises the binary instance-selection stage and the expert-specific loss, and is load-bearing for the MIL design.","marker":"[9]"},{"why":"YOLOv8 detects the ROI that is cropped from the videos, which is the first preprocessing step for all methods.","marker":"[16]"}],"fun_headline_variants":["MReg uses MoE feature mining to grade MR from echo videos","Regression formulation boosts mitral regurgitation grading accuracy","Video AI model combines MIL, amplification, MoE for MR diagnosis","MReg's category-level MoE summary improves MR severity grading","Feature selection and amplification applied to echo video MR grading"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the grade assigned by a single experienced sonographer is the correct diagnosis, with no second reader, no measurement of reader variability, and no external quantitative reference; if these labels are noisy or subjective, the reported accuracy and F1 are not a true measure of diagnostic performance.","fun_headline_variants_meta":{"raw":{"variants":["MReg uses MoE feature mining to grade MR from echo videos","Regression formulation boosts mitral regurgitation grading accuracy","Video AI model combines MIL, amplification, MoE for MR diagnosis","MReg's category-level MoE summary improves MR severity grading","Feature selection and amplification applied to echo video MR grading"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000174,"raw_usage":{"total_tokens":1281,"prompt_tokens":945,"completion_tokens":336,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":254}},"tokens_in":561,"tokens_out":336,"duration_ms":5003,"temperature":1.0,"reasoning_tokens":254,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:35:47.445420+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Grade a held-out set of 200 videos by three independent cardiologists with consensus adjudication and compare MReg's predictions against the consensus grades, or against quantitative Doppler measurements such as effective regurgitant orifice area; if MReg's accuracy and F1 do not remain above the compared baselines on this reference labeling, the central claim is falsified.","supporting_citations":[{"cited_title":"Computer Vision and Image Understanding 249, 104163 (2024)","cited_arxiv_id":null,"evidence_quote":"AnomalyCLIP is the weakly supervised video anomaly detection baseline that shares a similar weak-label mechanism and is used for comparison."},{"cited_title":"Machine Learning and Knowledge Extraction5(4), 1680–1716 (2023)","cited_arxiv_id":null,"evidence_quote":"YOLOv8 detects the ROI that is cropped from the videos, which is the first preprocessing step for all methods."}],"review_version":1}