{"id":"2b317d53-c3c7-4fcc-9339-682e18dbbae2","arxiv_id":"2507.04383","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"ViTaL provides 496 patients' ultrasound images, lab tables, and reports for six-class ovarian tumor recognition, with a cross-attention fusion model reaching 85.59% accuracy.","lead":"A new medical dataset, ViTaL, pairs 2,216 ultrasound images of ovarian tumors with blood-test results and written reports from 496 patients to support six-way tumor typing. The authors also propose an attention-based fusion network that reaches about 85% accuracy on this benchmark, though key evaluation details are missing.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No patient-level split protocol is reported, so the 85.59% accuracy may reflect slice leakage from the same tumor appearing in both training and test sets; this is the most load-bearing threat to the benchmark claim.","rationale":"The reader's weakest_assumption identifies the same missing split protocol as the primary threat, and I agree that it is the most load-bearing issue. The paper's dataset contribution is real: it provides a multimodal collection with visual, tabular, and linguistic data for 496 patients, and the authors have committed to releasing data and code. The ablation study in Table 5 provides internal support for the value of multimodal fusion, and the attention mechanism is at least plausible. However, the headline benchmark numbers — 85% overall accuracy and a 14.5% improvement over prior methods — are only meaningful if the test set contains no images from patients seen during training. The paper's own description of multiple scans per patient makes image-level splitting a genuine risk, and the absence of any split description in Sections 5.1 or 5.2 is not a stylistic gap; it is the exact condition needed to rule out leakage. I would not move the verdict to REJECT because the dataset and code release may allow the authors to supply the missing split information, and conditional acceptance is the appropriate stance. I also considered the internal inconsistency between Table 3 (83.08%) and Table 5 (85.59%) for the same full-modality model, and the post-hoc exclusion of the 'Other' category; both are legitimate concerns, but they are secondary to the split question because even a consistent 83.08% would still be suspect if leakage occurred. The concrete test I propose — checking the patient-level split and retraining with it — would settle the central question directly and could be done with the released assets and a small amount of additional metadata.","tokens_in":16575,"tokens_out":2590,"duration_ms":30270,"concrete_test":"Request the exact patient-level split used for the experiments (or the patient IDs in each partition). Then retrain ViTaL-Net using the released code and this split, and report accuracy on the held-out patients. If any patient contributes images to both training and test sets, or if the retrained accuracy drops substantially relative to 85.59%, the reported performance is inflated by slice leakage and the benchmark claim must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that ViTaL-Net achieves 85.59% accuracy and 0.95 AUC on six-way multi-pathology ovarian tumor classification (Table 5, Abstract). This claim depends entirely on a valid train/test split, yet Sections 5.1 and 5.2 describe data preprocessing, augmentation, optimizers, and learning-rate schedules but never specify how the 2,216 images from 496 patients were partitioned. The dataset is highly patient-correlated: multiple ultrasound slices are collected per patient, and Section 3.1 explicitly states that multiple scans with different viewing angles are selected per patient. If the split is done at the image level rather than the patient level, the same tumor can contribute slices to both training and testing. Because the model can memorize patient-specific or tumor-specific imaging artifacts, the reported accuracy and AUC would be inflated, and the 14.5% margin over the best competitor (Abstract) would not reflect genuine generalization. The concern is concrete and internal: the paper itself provides the 496-patient and 2,216-image counts but omits the one piece of information needed to rule out this leakage. Unlike the Other-category exclusion or the accuracy discrepancy between Table 3 (83.08%) and Table 5 (85.59%), which are also real issues, the split protocol directly determines whether the headline numbers are meaningful at all. The dataset release itself is a genuine contribution, but the benchmark numbers cannot be evaluated without this information.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ViTaL, an ovarian-tumor dataset of 496 patients containing 2,216 2D ultrasound images, tabular examination data, and ultrasound report text, labeled with six pathological categories plus an 'Other' category. It also proposes ViTaL-Net, a multimodal classifier using MobileNet, TabNet, and BERT backbones fused through a Triplet Hierarchical Offset Attention Mechanism (THOAM). The authors report 85.59% accuracy and 0.95 AUC on six-way classification, claim a 14.5% improvement over prior methods, and provide ablations across modality combinations and fusion strategies. The dataset and code are promised publicly.","tokens_in":16857,"tokens_out":4344,"duration_ms":48386,"significance":"If the evaluation is valid, ViTaL is a valuable resource: it is among very few ovarian ultrasound datasets that pair images with per-patient tabular and textual data, it releases code, and it is ethically approved. The modality ablation in Table 5 and the attention-vs-concatenation comparison in Table 6 are useful checks of the proposed fusion. However, the central benchmark claim is not yet supported because the train/test split protocol is unspecified, the six-class benchmark is defined by post-hoc exclusion of the 'Other' category, and several reported numbers are inconsistent across tables. The dataset contribution itself is plausible, but the performance claims need substantially stronger evaluation before the benchmark can be adopted.","major_comments":[{"comment":"The paper never specifies how the 2,216 ultrasound images from 496 patients are partitioned into training and test sets. This is load-bearing because Section 3.1 states that multiple scans with different viewing angles are selected per patient; if the split is at image level rather than patient level, slices from the same tumor can appear in both training and test, which can inflate the reported 85.59% accuracy and 0.95 AUC. Please state the split unit (patient or image), the number of patients/images in each split, the random seed or cross-validation scheme, and whether validation data were used for model selection.","section":"Sections 5.1, 5.2, 3.1"},{"comment":"The reported accuracy is internally inconsistent: Table 3 lists THOAM accuracy as 83.08%, while Tables 5 and 6 report 85.59% for the same attention-fusion configuration. Additionally, Section 5.3 describes Table 3 as image-unimodal, yet the table includes THOAM and other multimodal fusion methods. Please clarify which experiments are unimodal, which are multimodal, how the claimed 14.5% improvement over the best competitor is computed, and whether all comparison methods used the same modality set and the same data split.","section":"Tables 3, 5, 6 and Section 5.3"},{"comment":"The benchmark excludes the 'Other' category after observing that including it lowers accuracy to 70.39%, so the six-class task and the headline 85.59% figure are defined post hoc rather than prospectively. Because the dataset actually contains seven categories, the paper should either justify this exclusion with a pre-specified criterion, report results with and without 'Other' including per-class metrics, or redefine the task to cover all collected categories. The current presentation risks overstating the model's practical diagnostic performance.","section":"Section 6.2 and Table 6"}],"minor_comments":[{"comment":"The sentence 'we have collected 30 cases from patients aged 6 to 23' should clarify whether these are cases or patients, and how this subset is flagged in the released data.","section":"Section 3.1"},{"comment":"There are typos in the text surrounding these equations: 'Sof tmax' should be 'Softmax', 'Concact' should be 'Concatenate', and 's a scaling factor' should be 'a scaling factor'.","section":"Equations (4), (7)"},{"comment":"The SEN and SPE columns run together without clear spacing, making the table difficult to read; formatting should be cleaned.","section":"Table 4"},{"comment":"The software name 'MMpretrain' should be spelled consistently (e.g., MMPretrain) and the reference/version details should be provided.","section":"Section 5.2.1"},{"comment":"The captions refer to the 'horizontal axis of the table' where they mean the horizontal axis of the figure; please correct.","section":"Figures 2, 3, 4"},{"comment":"The paper should state the dataset license, the de-identification process, and any data-use restrictions to accompany the public release.","section":"Dataset release"}],"recommendation":"major_revision","confidential_remarks":"The split-protocol omission is the decisive issue: without a patient-level split, the benchmark numbers cannot be interpreted. The dataset itself may be a useful community resource, but the manuscript's evaluation section needs substantial revision before the headline claims are credible. I would also ask the editor to verify the 'first multimodal multipathology dataset' claim against the cited MMOTU and OVATUS-V1 entries, since the comparison in Table 1 is central to the novelty statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the ViTaL dataset is the real contribution, and the benchmark numbers are not yet trustworthy until the split protocol is disclosed. I'd engage with the dataset; I'd hold off on citing the 85% number.\n\nThe dataset is genuinely new — to my knowledge no prior public ovarian tumor set combines 2D ultrasound images, structured labs, and clinical text for six-way pathology classification. 496 patients, 2216 images, with age and category distributions, ethics approval, and a promised release. That fills a real gap. The ablation is also suggestive: adding tabular and textual features to images moves accuracy from 74.5% to 85.6%, and the per-modality contributions look plausible. If the data and code actually ship, this will be a useful benchmark for multimodal fusion work.\n\nThe soft spots are in the evaluation. The load-bearing one: the paper never says how the 2216 images from 496 patients were split into train and test. Since Section 3.1 says multiple slices per patient are collected, an image-level split can leak the same tumor into both sides. Without a patient-level split (or an explicit defended alternative), the 85.59% accuracy and 0.95 AUC cannot be interpreted. That is the first thing a referee should ask for.\n\nSecond, the 'Other' category is excluded after noting it drops accuracy to 70.39%. That is at least disclosed, but the headline number is then a six-class number on a selected subset. The seven-class result should be in the main table, and the framing should say 'six-class excluding rare/heterogeneous cases' rather than implying full-coverage six-way diagnosis.\n\nThird, the numbers inside the paper do not line up. Table 3 gives THOAM 83.08% accuracy; Table 5 gives 85.59% for the same all-modality setting. The text claims 14.5% over the best competitor, but the best competitor in Table 3 is TF A-LT at 76.72%, which is 6.36 points below 83.08 and 8.87 below 85.59. The 14.5% only matches a comparison to Densenet121, not to the stated best competitor. This needs a clean pass over tables and prose.\n\nThe method itself is a minor issue: Equations 3–7 are standard two-stage cross-attention, and no 'offset' operation is ever defined. It is fine as a simple fusion baseline, but the name oversells it. Also, no error bars, and several classes are small enough that per-class CIs are needed.\n\nVerdict: send to review. The dataset deserves referee time. The revision must specify the split, report seven-class performance, fix the inconsistencies, and temper the method claims. If the split turns out to be image-level and the accuracy collapses at patient level, the paper will still be a dataset paper — which is worth having, just not with the current benchmark claims.","headline":"Useful new multimodal ovarian ultrasound dataset, but the benchmark numbers are only as good as the undisclosed train/test split; needs major revision before the 85% claim can be taken at face value.","tokens_in":17393,"tokens_out":3721,"would_cite":true,"duration_ms":40313,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new dataset fuses ultrasound images, exam tables, and report text to classify six ovarian tumor types, reaching 85% accuracy.","keywords":["ovarian tumor classification","multimodal medical dataset","ultrasound image analysis","attention-based fusion","tabular clinical data","radiology text reports","computer-aided diagnosis"],"falsifier":"Train the same ViTaL-Net under a patient-disjoint split—grouping all slices of each patient together and assigning whole patients to train or test—and compare accuracy and AUC to the reported 85.59% and 0.95. A large drop would show that the reported multimodal gain depends on seeing slices of the same tumor in both training and testing.","tokens_in":16386,"feed_emoji":"🩺","tokens_out":4856,"duration_ms":50389,"temperature":0.7,"pith_summary":"The paper introduces ViTaL, a dataset of 496 patients' ultrasound images, medical-examination tables, and ultrasound reports, labeled into six ovarian tumor pathological types. Its aim is to show that combining visual, tabular, and linguistic data yields substantially more accurate tumor typing than images alone, which matters because clinical practice needs finer distinctions than a benign/malignant split. The proposed network, ViTaL-Net, uses a Triplet Hierarchical Offset Attention Mechanism (THOAM) to fuse features from the three modalities with less information loss than simple concatenation. On its own benchmark, ViTaL-Net reports 85.59% accuracy and an average AUC of 0.95, about 14.5 percentage points above image-only models. The paper offers ViTaL as a public benchmark for multi-pathology, multi-modality ovarian tumor recognition.","feed_headline":"Multimodal fusion hits 85% accuracy on six ovarian tumor types","feed_subtitle":"Ultrasound images alone reach 74.5%; adding exam tables and report text lifts accuracy to 85.6%.","key_machinery":"Triplet Hierarchical Offset Attention Mechanism (THOAM), a two-stage cross-attention fusion module. Image and table features are fused first through scaled dot-product attention, the result is attended once more with text features, and the final output is concatenated with the original visual feature map and passed to a linear classifier. This is the component the paper credits with reducing fusion loss and improving relevance and complementarity among the three modalities.","core_discovery":"The central claim is that a six-way ovarian tumor classification task can be performed from three complementary clinical modalities—2D ultrasound images, structured examination values, and free-text ultrasound reports—and that an attention-based hierarchical fusion of these modalities outperforms image-only and simple-concatenation baselines. The authors report that ViTaL-Net reaches 85.59% overall accuracy and a 0.95 mean AUC across the six pathology classes, with per-class accuracy above 90% for the two most common types. They also state that this is the first dataset of its kind to combine visual, tabular, and linguistic modalities specifically for multi-pathology ovarian tumor recognition.","pith_inferences":["The dataset split protocol is not described; if ultrasound slices from the same patient appear in both training and test sets, the reported 85.59% accuracy and 0.95 AUC are likely optimistic relative to a patient-disjoint evaluation.","Because all images come from one scanner at one hospital, a natural test is whether ViTaL-Net's gains persist on data from other institutions or devices; the tabular and text modalities may partly encode institution-specific reporting style.","The tabular modality's contribution could be isolated further by ablating individual markers such as CA-125, which would reveal whether the gain comes from broad tabular structure or from a few high-signal tumor markers."],"forward_implications":["Adding tabular and linguistic data to ultrasound images is the main driver of the reported gain: accuracy rises from 74.54% with visual data only to 85.59% with all three modalities.","The multi-pathology setting is feasible: the model distinguishes six tumor types, not just benign versus malignant, with class-specific AUC values mostly at or above 0.9.","The heterogeneous 'Other' category currently degrades performance, so the paper excludes it from the benchmark and identifies handling it as future work.","Attention-based fusion beats direct concatenation on the same three modalities (85.59% vs 75.88% accuracy), supporting the claim that THOAM reduces information loss during fusion."],"supporting_citations":[{"why":"Supplies the prior multi-modality ovarian ultrasound benchmark that ViTaL extends by adding table and text modalities.","marker":"[57]"},{"why":"Defines the earlier benign/borderline/malignant ultrasound classification setting that ViTaL replaces with six pathology classes.","marker":"[54]"},{"why":"MobileNet is the visual backbone whose image-only result of 74.54% serves as the baseline the multimodal model must beat.","marker":"[40]"},{"why":"CBAM provides an attention-based comparison method used to benchmark ViTaL-Net's classification performance.","marker":"[13]"},{"why":"ITCM supplies the image-plus-tabular fusion baseline that ViTaL-Net is compared against.","marker":"[12]"},{"why":"TF A-LT supplies the image-plus-text fusion baseline that ViTaL-Net is compared against.","marker":"[14]"}],"fun_headline_variants":["New multimodal benchmark fuses ultrasound, labs, and reports for ovarian tumor typing","ViTaL dataset brings visual, tabular, and linguistic data to ovarian tumor recognition","Ultrasound, labs, and reports combine for 85% accuracy on six ovarian tumor types","Triple-modality fusion for ovarian tumor typing reaches 85% overall accuracy","ViTaL-Net: fusing imaging, labs, and text to spot six ovarian tumor types at 85%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the train/test split keeps all ultrasound slices of a single patient on one side; the paper never states this split protocol, and if patients are split across train and test, the model could memorize patient-specific slices and the reported accuracy would be optimistic.","fun_headline_variants_meta":{"raw":{"variants":["New multimodal benchmark fuses ultrasound, labs, and reports for ovarian tumor typing","ViTaL dataset brings visual, tabular, and linguistic data to ovarian tumor recognition","Ultrasound, labs, and reports combine for 85% accuracy on six ovarian tumor types","Triple-modality fusion for ovarian tumor typing reaches 85% overall accuracy","ViTaL-Net: fusing imaging, labs, and text to spot six ovarian tumor types at 85%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000643,"raw_usage":{"total_tokens":2975,"prompt_tokens":983,"completion_tokens":1992,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":1876}},"tokens_in":599,"tokens_out":1992,"duration_ms":16071,"temperature":1.0,"reasoning_tokens":1876,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:48:19.622403+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same ViTaL-Net under a patient-disjoint split—grouping all slices of each patient together and assigning whole patients to train or test—and compare accuracy and AUC to the reported 85.59% and 0.95. A large drop would show that the reported multimodal gain depends on seeing slices of the same tumor in both training and testing.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the earlier benign/borderline/malignant ultrasound classification setting that ViTaL replaces with six pathology classes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ITCM supplies the image-plus-tabular fusion baseline that ViTaL-Net is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TF A-LT supplies the image-plus-text fusion baseline that ViTaL-Net is compared against."}],"review_version":1}