Pith. sign in

REVIEW 9 cited by

Quilt-1M: One Million Image-Text Pairs for Histopathology

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.11207 v4 pith:IOS6RK5V submitted 2023-06-20 cs.CV cs.CLcs.LG

Wisdom Oluchi Ikezogwo , Mehmet Saygin Seyfioglu , Fatemeh Ghezloo , Dylan Stefan Chan Geva , Fatwir Sheikh Mohammed , Pavan Kumar Anand , Ranjay Krishna , Linda Shapiro This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords histopathologydatasetdatasetsmodelsquiltquilt-1mcurateddata
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Recent accelerations in multi-modal applications have been made possible with the plethora of image and text data available online. However, the scarcity of analogous data in the medical field, specifically in histopathology, has slowed comparable progress. To enable similar representation learning for histopathology, we turn to YouTube, an untapped resource of videos, offering $1,087$ hours of valuable educational histopathology videos from expert clinicians. From YouTube, we curate QUILT: a large-scale vision-language dataset consisting of $802, 144$ image and text pairs. QUILT was automatically curated using a mixture of models, including large language models, handcrafted algorithms, human knowledge databases, and automatic speech recognition. In comparison, the most comprehensive datasets curated for histopathology amass only around $200$K samples. We combine QUILT with datasets from other sources, including Twitter, research papers, and the internet in general, to create an even larger dataset: QUILT-1M, with $1$M paired image-text samples, marking it as the largest vision-language histopathology dataset to date. We demonstrate the value of QUILT-1M by fine-tuning a pre-trained CLIP model. Our model outperforms state-of-the-art models on both zero-shot and linear probing tasks for classifying new histopathology images across $13$ diverse patch-level datasets of $8$ different sub-pathologies and cross-modal retrieval tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics

    q-bio.GN 2025-08 unverdicted novelty 6.0 of 10

    HESCAPE shows that cross-modal contrastive pretraining helps mutation classification but hurts gene expression prediction in spatial transcriptomics, implicating batch effects.

  2. PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions

    cs.CV 2025-06 conditional novelty 6.0 of 10

    PathDiff jointly trains a latent diffusion model on unpaired text-image and mask-image histopathology datasets, enabling text-, mask-, or both-conditioned generation with reported gains in fidelity and downstream tasks.

  3. PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology

    cs.CV 2025-02 conditional novelty 6.0 of 10

    PathFinder, a multi-agent system that iteratively navigates and describes histopathology slides, reports 74% accuracy on a small balanced melanoma test set, topping a 65% average human benchmark.

  4. Learning to Rank Pre-trained Vision-Language Models for Downstream Tasks

    cs.CV 2024-12 conditional novelty 6.0 of 10

    VEGA ranks pre-trained vision-language models for unlabeled datasets by measuring how well a model's own pseudo-labeled visual clusters align with class-name text features.

  5. Aligning Knowledge Concepts to Whole Slide Images for Precise Histopathology Image Analysis

    cs.CV 2024-11 conditional novelty 6.0 of 10

    ConcepPath aligns WSI patches to GPT-4-induced expert concepts plus learned concepts and uses a two-stage aggregation to outperform prior MIL methods on five histopathology classification tasks.

  6. VELVET-Med: Vision and Efficient Language Pre-training for Volumetric Imaging Tasks in Medicine

    cs.CV 2025-08 conditional novelty 5.0 of 10

    VELVET-Med combines a sentence-aware TriBERT text encoder, hierarchical contrastive learning, and uni-modal self-supervision to pre-train 3D CT vision-language encoders on 38,875 scan-report pairs.

  7. Data Augmentation with Diffusion Models for Colon Polyp Localization on the Low Data Regime: How much real data is enough?

    cs.CV 2024-11 conditional novelty 5.0 of 10

    Synthetic images from diffusion models, used as pre-training data, substantially improve polyp localization when real data is scarce, with the benefit shrinking as more real images are added.

  8. SimMIL: A Universal Weakly Supervised Pre-Training Framework for Multi-Instance Learning in Whole Slide Pathology Images

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A weakly supervised pre-training scheme that propagates bag labels to patches improves downstream MIL classification and survival prediction on WSI datasets, but the comparison baselines are not trained on the same ta...

  9. Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation

    cs.CL 2025-02 conditional novelty 3.0 of 10

    A structured survey of multimodal RAG systems, covering datasets, benchmarks, methods, and open challenges, with a public resource repo.

Pith tools