Pith. sign in

REVIEW 4 cited by

Reliable Fidelity and Diversity Metrics for Generative Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2002.09797 v2 pith:LCZ2SVX7 submitted 2020-02-23 cs.CV cs.LGstat.ML

Muhammad Ferjad Naeem , Seong Joon Oh , Youngjung Uh , Yunjey Choi , Jaejun Yoo This is my paper · ORCID
classification cs.CVcs.LGstat.ML
keywords metricsreliablecoveragedensitydiversityevaluationfidelitygenerated
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Devising indicative evaluation metrics for the image generation task remains an open problem. The most widely used metric for measuring the similarity between real and generated images has been the Fr\'echet Inception Distance (FID) score. Because it does not differentiate the fidelity and diversity aspects of the generated images, recent papers have introduced variants of precision and recall metrics to diagnose those properties separately. In this paper, we show that even the latest version of the precision and recall metrics are not reliable yet. For example, they fail to detect the match between two identical distributions, they are not robust against outliers, and the evaluation hyperparameters are selected arbitrarily. We propose density and coverage metrics that solve the above issues. We analytically and experimentally show that density and coverage provide more interpretable and reliable signals for practitioners than the existing metrics. Code: https://github.com/clovaai/generative-evaluation-prdc.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A new evaluation framework, DIMCIM, measures default-mode diversity and prompted generalization in text-to-image models, finding a scale trade-off and a 0.85 correlation between default diversity and training data diversity.

  2. Improved Training Technique for Latent Consistency Models

    cs.CV 2025-02 conditional novelty 6.0 of 10

    Latent consistency models can be trained from scratch for one- or two-step generation when Huber loss is replaced by Cauchy loss and combined with early-timestep diffusion loss, OT coupling, an adaptive scaling schedu...

  3. DS@GT ARC at ImageCLEFmed GANs 2026: Geometric Filtering for Privacy-Preserving CT Slice Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Geometric filtering of synthetic CT slices reduces nearest-neighbor and membership-inference leakage, but patient re-identification leakage remains near 0.97–1.0 across all submissions.

  4. SynEHRgy: Synthesizing Mixed-Type Structured Electronic Health Records using Decoder-Only Transformers

    cs.LG 2024-11 conditional novelty 5.0 of 10

    SynEHRgy tokenizes mixed-type MIMIC-III records into one sequence and trains a small decoder-only transformer to generate new synthetic patient records.

Pith tools