pith. sign in

arxiv: 2311.10798 · v1 · pith:5BUXAITNnew · submitted 2023-11-17 · 💻 cs.LG · cs.AI· cs.CV· eess.IV

INSPECT: A Multimodal Dataset for Pulmonary Embolism Diagnosis and Prognosis

Shih-Cheng Huang , Zepeng Huo , Ethan Steinberg , Chia-Chun Chiang , Matthew P. Lungren , Curtis P. Langlotz , Serena Yeung , Nigam H. Shah
show 1 more author
Jason A. Fries
This is my paper
classification 💻 cs.LG cs.AIcs.CVeess.IV
keywords datainspectmultimodaldatasetavailablecontainsde-identifiedembolism
0
0 comments X
read the original abstract

Synthesizing information from multiple data sources plays a crucial role in the practice of modern medicine. Current applications of artificial intelligence in medicine often focus on single-modality data due to a lack of publicly available, multimodal medical datasets. To address this limitation, we introduce INSPECT, which contains de-identified longitudinal records from a large cohort of patients at risk for pulmonary embolism (PE), along with ground truth labels for multiple outcomes. INSPECT contains data from 19,402 patients, including CT images, radiology report impression sections, and structured electronic health record (EHR) data (i.e. demographics, diagnoses, procedures, vitals, and medications). Using INSPECT, we develop and release a benchmark for evaluating several baseline modeling approaches on a variety of important PE related tasks. We evaluate image-only, EHR-only, and multimodal fusion models. Trained models and the de-identified dataset are made available for non-commercial use under a data use agreement. To the best of our knowledge, INSPECT is the largest multimodal dataset integrating 3D medical imaging and EHR for reproducible methods evaluation and research.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors

    cs.CL 2026-06 unverdicted novelty 6.0

    RAD3D-Prefix is a diagnostic-prior conditioning framework for 3D CT report generation that integrates image embeddings with multi-label classification logits, showing that freezing larger LLMs and training only projec...

  2. ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training

    cs.CV 2026-05 unverdicted novelty 6.0

    ASAP introduces an anatomy-aware semantically-adaptive pre-training method for medical volumetric vision-language models and reports state-of-the-art results on a new benchmark spanning 15 datasets and 22 tasks.

  3. Astra: a generalizable report generation foundation model for 3D computed tomography

    cs.CV 2026-05 unverdicted novelty 6.0

    Astra is a 3D CT vision-language foundation model trained on 90,678 thoracoabdominal scans that claims 44.1% better diagnostic metrics on internal and six external cohorts plus 29.6% faster chest reporting in real workflows.

  4. Learning Robust Visual Features in Computed Tomography Enables Efficient Transfer Learning for Clinical Tasks

    cs.CV 2026-04 conditional novelty 6.0

    VoxelFM learns robust 3D CT visual features via DINO self-distillation that transfer effectively to seven clinical task categories using frozen backbones and lightweight heads, outperforming prior CT foundation models...

  5. Medical Image Understanding Improves Survival Prediction via Visual Instruction Tuning

    cs.CV 2026-04 unverdicted novelty 5.0

    A vision-language model pre-trained via instruction tuning on CT-report pairs improves survival prediction accuracy over baselines, especially when clinical data alone is weak, while also producing text answers to cli...