Pith. sign in

REVIEW 3 cited by

Pros and Cons of GAN Evaluation Measures

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1802.03446 v5 pith:QWJNWSXX submitted 2018-02-09 cs.CV

Ali Borji This is my paper · ORCID
classification cs.CV
keywords measuresmodelsbeengenerativeevaluatingevaluationgansmeasure
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Generative models, in particular generative adversarial networks (GANs), have received significant attention recently. A number of GAN variants have been proposed and have been utilized in many applications. Despite large strides in terms of theoretical progress, evaluating and comparing GANs remains a daunting task. While several measures have been introduced, as of yet, there is no consensus as to which measure best captures strengths and limitations of models and should be used for fair model comparison. As in other areas of computer vision and machine learning, it is critical to settle on one or few good measures to steer the progress in this field. In this paper, I review and critically discuss more than 24 quantitative and 5 qualitative measures for evaluating generative models with a particular emphasis on GAN-derived models. I also provide a set of 7 desiderata followed by an evaluation of whether a given measure or a family of measures is compatible with them.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Synthetic Data for Portfolios: A Throw of the Dice Will Never Abolish Chance

    q-fin.PM 2025-01 conditional novelty 6.0 of 10

    Generating excessive synthetic returns from small samples biases statistics, and generic GANs learn high-variance components that matter least for long-short portfolios.

  2. A Unifying Information-theoretic Perspective on Evaluating Generative Models

    cs.LG 2024-12 conditional novelty 5.0 of 10

    A new information-theoretic evaluation metric (PCE, RCE, RE) is proposed to separately detect fidelity loss, mode dropping, and mode shrinkage in generative models, and existing kNN precision/recall metrics are unifie...

  3. Deep Learning Models for Physical Layer Communications

    cs.LG 2025-02 conditional novelty 3.0 of 10

    A compilation of deep learning methods for channel modeling, neural decoding, mutual information estimation, and capacity learning, applied to power line communications.

Pith tools