Pith. sign in

REVIEW 5 major objections 4 minor 54 references

Synonymous Variational Inference for Perceptual Image Compression

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Viewing each image as a member of a set of perceptually synonymous images, this paper proves that the minimum rate for perceptual compression is a triple tradeoff among rate, distortion, and expected KL divergence, and that this bound…

desk verdict Solid progressive codec engineering, but the SVI proof rests on a circular invariance assumption and Jensen bounds treated as equivalences, so the central theorem does not stand as written. read the letter →

arxiv 2505.22438 v1 pith:UYSVFQ3P submitted 2025-05-28 cs.IT cs.AIcs.CVcs.LGeess.IVmath.IT

Zijian Liang , Kai Niu , Changshuo Wang , Jin Xu , Ping Zhang This is my paper
classification cs.ITcs.AIcs.CVcs.LGeess.IVmath.IT MSC 94A3494A1594A17
keywords perceptualimagecompressionrate-distortion-perceptiontradeoffsynonymousvariationalinferencesemanticinformationtheorysynsetpartialKLdivergenceprogressivecodec
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper re-derives perceptual image compression from a synonymity-based semantic information viewpoint: the 'meaning' of an image is carried not by its pixels but by the set of images that look similar to it under a perceptual criterion, which the authors call the ideal Synset. Its central result, Theorem 3.3, states that the minimum achievable coding rate is set by a triple tradeoff — semantic mutual information between the source and the reconstructed synset, constrained by an expected distortion and an expected KL divergence between source and reconstruction distributions. Because the existing rate-distortion-perception tradeoff is exactly the single-reconstruction special case and classical rate-distortion is the single-image-synset special case, the theorem unifies the optimization goals of traditional and learned codecs. The divergence term that perceptual-compression losses always add is not an ad hoc choice: the analysis shows it arises because the reconstruction target is a set of synonymous images rather than a single image. The accompanying Synonymous Image Compression scheme, which codes only the shared latent part and samples the remaining details, delivers 16 rate points from one progressive model with rate-distortion-perception performance comparable to single-rate generative baselines.

What carries the argument

The load-bearing object is the ideal Synset $\mathcal{X}$, the set of images that share meaning with the source image under a perceptual criterion, together with the partial semantic KL divergence $D_{\mathrm{KL},s}[q\|p_s]$, which measures the distance between a syntactic parametric density and a semantic distribution. The latent representation is split into a synonymous part $\tilde{y}_s$, which must be coded and transmitted, and a detailed part $\tilde{y}_\epsilon$, which the decoder samples rather than receives, so only the synonymous part contributes to the rate. Lemma 3.2 converts the synonymous likelihood term into an expected-distortion plus expected-KL objective, and Theorem 3.3 packages that equivalence into the rate bound whose rate term is semantic mutual information $I(X;\hat{\mathring{X}}) = H_s(\hat{\mathring{X}}) - H_s(\hat{\mathring{X}}|X)$. The unification claim is carried by two degenerations: a single-member reconstructed synset reproduces the rate-distortion-perception tradeoff, and a single-member ideal synset reproduces classical rate-distortion.

What would settle it

Test equation (20) directly on a source whose synonym structure is known: train a SIC-style codec, generate several reconstructions from one code by sampling details, re-encode those reconstructions, and measure the dispersion of the recovered synonymous representations. If different synset members yield materially different reconstruction distributions, or if re-encoding different reconstructions gives significantly different $\hat{y}_s$, the equivalence in Lemma 3.2 fails and the rate bound of Theorem 3.3 is not established. A quantitative companion check on the same synthetic source is to compute the claimed minimum rate and verify that a codec that ignores the synset violates it.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that perceptual image compression should be optimized against a set of synonymous reconstructions rather than a single reconstruction, and that this shift turns the optimization objective into a synonymous rate-distortion-perception tradeoff. Theorem 3.3 states that the minimum achievable rate is $$R(X) = \min_{p(\hat{X}|x)} I(X;\hat{\mathring{X}}) \quad \text{s.t.} \quad \mathbb{E}[d(x,\hat{x}_i)] \le D,\; \mathbb{E}[D_{\mathrm{KL}}[p_x \| p_{\hat{x}_i}]] \le P,$$ where the rate is the semantic mutual information between the source and the reconstructed synset. The load-bearing step is Lemma 3.2: minimizing the expected negative log synonymous-likelihood term is equivalent to minimizing a weighted expected distortion plus a weighted expected KL divergence, because the likelihood of the whole synset integrates over its members. The paper claims this is the first theoretical explanation of why a divergence measure must appear in perceptual image compression: the E-KLD term measures the gap between the reconstructed distribution and the ideal synset, and it disappears only when the ideal synset collapses to the original image. Both existing bounds then fall out as degenerations.

Load-bearing premise

The derivation assumes, in equation (20), that every image in the ideal Synset already maps to the same reconstruction distribution under the current encoder — the very invariance the training is meant to create — so the proof leans on its target property before that property exists.

Editorial extensions

If this is right

  • If Theorem 3.3 is right, the classical rate-distortion bound and the existing rate-distortion-perception bound are the two degenerate edges of a single synonymous tradeoff, obtained by collapsing the ideal synset or the reconstructed synset to one sample.
  • Because only the synonymous representation is coded, the achievable rate satisfies $I(X;\hat{\mathring{X}}) \le I(X;\hat{X})$, so a synonymity-aware codec can beat a symbol-level codec at the same rate whenever the decoder is free to sample details rather than receive them.
  • A single progressive codec with $L$ split levels yields $L$ rate points from one generator; the implemented $L=16$ codec covers the full rate range on Kodak, CLIC2020, and DIV2K, with each rate sharing one analysis and synthesis transform.
  • The bound identifies the role of the perceptual loss term: it is the expected-KL constraint that drives the reconstructed synset toward the ideal synset, so divergence-based perceptual measures such as LPIPS, DISTS, and adversarial losses are all proxies for the same underlying term.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the framework predicts that the benefit of synonymity-aware coding grows as the perceptual criterion becomes more resampling-tolerant (DISTS over LPIPS over MSE), a ranking that could be tested by training the same codec under each criterion and comparing the measured rate gap against the bound.
  • Editorial inference: the limited FID gains the paper reports point to its own next bottleneck — the detail sampler draws from a scalar uniform distribution instead of the conditional vector prior assumed by the derivation, so replacing the sampler is a direct and testable upgrade path.
  • Editorial inference: the synonymous idempotence constraint, which re-encodes reconstructions and penalizes $\|\hat{y}'_s - \hat{y}_s\|^2$, gives a practical, human-free measure of how close a codec is to overlapping the ideal synset, and could serve as a convergence criterion in future training.
  • Editorial inference: the same set-level reasoning should apply to other image restoration tasks whose success is judged by perceptual similarity, predicting that their optimal objectives also contain a divergence term of exactly this form.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a semantic-information-theoretic analysis of perceptual image compression. It defines an 'ideal synset' of perceptually synonymous images, introduces synonymous variational inference (SVI), and states Lemma 3.2 and Theorem 3.3, claiming that the minimum achievable rate of perceptual compression follows a 'synonymous rate-distortion-perception tradeoff' that contains the existing rate-distortion-perception tradeoff as a special case. It then presents synonymous image compression (SIC), a progressive codec that encodes only the 'synonymous' latent part and samples the remaining detail latents, with experiments comparing DISTS, PSNR, LPIPS, and FID against HiFiC and MS-ILLM. The appendix contains the detailed proofs of the two central results.

Significance. The paper is unusually detailed in its implementation documentation, providing architecture choices, hyperparameter tables, training schedules, and additional GAN fine-tuning experiments, and it is transparent about the preliminary nature of the experimental results. If the theoretical claims were correct, the work would offer a unified semantic-information explanation for the appearance of distribution-divergence terms in perceptual compression and would introduce a practical single-model variable-rate codec. However, the central proof contains load-bearing gaps: the key invariance assumption is circular, Jensen's inequality steps are treated as equivalences, the E-KLD term is identified with a constant rather than derived, and the rate identity is not justified. The main theorem is therefore not established, and the claimed theoretical significance is not currently realized.

major comments (5)
  1. [A.1, Eq. (20)] The proof of Lemma 3.2 assumes p_{\tilde x_j|x_i}(\tilde x_j|x_i)=p_{\tilde x_j|x}(\tilde x_j|x) for every x_i in the ideal synset, justified by an ideal encoder \phi^*_g. This invariance is precisely what the training objective is supposed to learn (Figure 1); before convergence the equality is generically false. Using it as a premise to derive the training objective is therefore circular. Without Eq. (20), the subsequent Bayes manipulation leading to Eq. (21) cannot be performed, so Lemma 3.2 is unproved.
  2. [A.1, Eqs. (21)(d)-(e)] The derivation applies Jensen's inequality twice to obtain upper bounds on the negative log expected likelihood, then absorbs the slack into scalar weights \lambda_d and \lambda_p (Eqs. (23) and (29)). The slack is a function of the distributions p_{x|\tilde x_j}, p_{\tilde x_j}, and p_{x_i}, not merely of the synset sizes |X| and |\tilde X|, so it cannot be absorbed into a constant hyperparameter. Minimizing an upper bound is not equivalent to minimizing the original objective; consequently the claimed equivalence in Eq. (17) is not established.
  3. [A.1, Eqs. (25)-(28)] Eq. (25) shows that the sum of the second and third terms in Eq. (21) equals f(x,X), which the paper identifies as a constant for given x and X. The subsequent split introduces the E-KLD term and the \delta_p term, and Eq. (28) states E-KLD - \delta_p = f(x,X). This means the E-KLD is not derived as part of the objective; it is an additional quantity that the authors propose to minimize to 'approximate' a constant. This is not an equivalence and cannot support Lemma 3.2.
  4. [A.2, Eq. (35)] The identification of the rate term with I(X;\hat{\tilde X}) relies on H_s(\hat{\tilde X}|X)=0, justified by log q(\tilde y_s|x)=0 and a 'determined encoder.' However, reconstruction is not determined by x because the decoder samples \hat y_{\epsilon,j}; conditional semantic entropy is not generally zero. Moreover, Theorem 3.3 is stated as a 'minimum achievable rate' without an operational coding theorem (achievability, converse, block-length definitions); the proof only manipulates the variational objective. The central rate-distortion-perception theorem is therefore not proven.
  5. [3.1 and A.2] Because the ideal synset is defined by a perceptual-similarity criterion, the appearance of a distribution-divergence perception term in the objective is an input of the model rather than an independent prediction. The paper's repeated claim to be the first work to 'theoretically explain the fundamental reason for the divergence measure's existence' is stronger than what the derivation supports; at best it shows that a synset defined by perceptual similarity leads to a perceptual penalty.
minor comments (4)
  1. [2.1, Eq. (2); 3.2, Eq. (10)] The statement that the first term 'equals 0 under the assumption of a uniform density on the unit interval' is inaccurate; a uniform density has a constant log-density value, and the KL term is not itself zero. The text should say the term is constant and can be absorbed.
  2. [3.2 and A.2] The notation for the reconstructed synset is inconsistent: Lemma 3.2 uses \tilde X, Theorem 3.3 uses \hat X, and the mutual information I(X;\hat{\tilde X}) is not formally defined. Please unify the notation.
  3. [A.1, after Eq. (28)] The reference to 'Eq. (3.2)(d)' should be to Eq. (21)(d); there are also several typos, including 'appplied' in Section 3.1 and 'Kodat' in Figure 17.
  4. [C.2] The authors describe the reported experimental results as 'preliminary results' and note insufficient hyperparameter exploration; this caveat should be stated in the main text, as it tempers the claim that the experiments verify the analysis.

Circularity Check

3 steps flagged · score 6.0 of 10

The E-KLD 'prediction' is built in: Eq. (20) assumes the synset invariance the loss is meant to create, the Jensen-bound slack is hidden in fitted lambdas, and Theorem 3.3's rate/distance vocabulary is imported from the authors' own prior semantic-information paper.

  1. self definitional [Appendix A.1, Eq. (20) (used in Eq. (21)(a)-(b)); cf. Sec. 3.1 Fig. 1]
    "Furthermore, since any sample xi in the ideal synset (including the original image sample x) shares the same synonymous representation ~ys by the ideal parametric SIC encoder with phi*_g, the posterior term in the (19) satisfy the following equations: p~xj|xi (~xj|xi) = p~xj|~ys (~xj|~ys) = p~xj|x (~xj|x). (20)"

    Lemma 3.2's equivalence—the paper's central derivation that the perception loss equals weighted E-MSE plus E-KLD—substitutes p~xj|xi = p~xj|x for every synset member (Eq. 21(a)). The only justification is 'the ideal parametric SIC encoder', whose defining property is that all synset members share one synonymous representation. But that invariance is exactly the convergence state the derived objective is supposed to create (Fig. 1: the reconstructed synset approaches the ideal synset 'until complete overlap occurs'). Before convergence, p~xj|xi and p~xj|x are generically distinct, so the proof fails precisely where the training loss is applied; granting Eq. (20) makes the E-KLD prediction an assumption of its own conclusion.

  2. other [Appendix A.1, Eqs. (21)(d)-(e), (25)-(30)]
    "If these two factors at the source are determined, the result of the function will be constant. ... although f(x,X) is a constant when x and X are determined, we can approximate it by minimizing the E-KLD term ... the multiplier alpha_d(|~X|) is a scaling factor influenced by the size of the reconstructed synset ~X, and it can be implicitly incorporated into the value of the hyperparameter lambda_d thus it does not need to be explicitly assigned."

    Steps (d)-(e) of Eq. (21) are Jensen upper bounds, yet Eq. (17)/(30) assert an equivalence to the weighted E-MSE + E-KLD objective. The bound slack is absorbed into alpha_d and alpha_p, 'implicitly incorporated into the value of the hyperparameter', so the weights are fitted, not fixed. Moreover, Eq. (25) shows the second and third terms jointly equal a model-independent constant f(x,X) = (1/|X|) integral_DKL[px||pxi]; the paper then declines to keep it constant and re-introduces the E-KLD as a minimizable proxy to 'determine the meaning of the constant value'. The perception term is therefore an input decision, not a forced consequence, so the claimed theoretical explanation of the divergence measure's existence is built into the construction.

1 more flagged steps
  1. self citation load bearing [Sec. 2.3 Eq. (6), Sec. 3.2 Eq. (7), Appendix A.2 Eqs. (34)-(35)]
    "As the foundation of the synonymous variational inference proposed in this paper, a new form of KL divergence needs to be introduced from (Niu & Zhang, 2024), referred to as partial semantic KL divergence ... (a) is achieved based on the definition of semantic entropy in Niu and Zhang's paper (2024), with the help of the weak law of large numbers."

    Theorem 3.3's rate term I(X;~X) = H_s(~X) - H_s(~X|X) and the minimized divergence D_KL,s are imported from Niu & Zhang (2024), whose authors include the present authors (K. Niu and P. Zhang). The central novelty claim—'the first work that can theoretically explain the fundamental reason for the divergence measure's existence'—rests on this self-cited framework: the KL-type input D_KL,s already fixes the divergence-type output, and the semantic rate identity comes from the prior work's semantic entropy. Since the prior paper is peer-reviewed and its grouping argument is elementary, this is load-bearing self-citation rather than a black-box uniqueness import, but it means the semantic content of the tradeoff is not independently derived here.

full rationale

The experimental contribution is self-contained: a working progressive codec is compared against external benchmarks (HiFiC, MS-ILLM, BPG, VTM), and the authors are candid about limitations (Section 6 replaces the 'derived' E-KLD with LPIPS; Section C.1 admits the detail-sampling mechanism 'cannot fit the ideal conditional distribution' of Eq. (18), and that the rate-representation mapping is a compromise). Those parts are not circular. The circularity is concentrated in the theoretical derivation billed as the main result. (i) Eq. (20) assumes p~xj|xi = p~xj|x for all ideal-synset members, justified by 'the ideal parametric SIC encoder'—the invariance the training objective is supposed to create—so Lemma 3.2's E-KLD term is assumed rather than derived. (ii) The Jensen steps in Eq. (21)(d)-(e) produce upper bounds whose slack is hidden in fitted scale factors alpha_d/alpha_p, and the combined second+third terms are a model-independent constant f(x,X) that the paper re-splits into an E-KLD proxy; the perception term is thus reinstated by choice. (iii) The rate identity and the partial semantic KL divergence come from the authors' own Niu & Zhang (2024) framework, so the semantic tradeoff in Theorem 3.3 is substantially a re-expression in the authors' own vocabulary: the paper itself shows R(X) reduces to the Blau-Michaeli R(D,P) when the reconstructed synset has one sample (Eq. (37)). The claimed 'theoretical explanation of the fundamental reason for the divergence measure's existence' therefore reduces to the definition of the synset via perceptual similarity plus a proxy choice plus a self-cited divergence input. Score 6: the key perception prediction reduces by construction, while the codec engineering and external benchmarks retain independent content.

Assumptions & free parameters 6 free parameters · 6 assumptions · 2 invented entities

The central claim rests on the postulated existence and encoder-invariance of ideal synsets, on the nonstandard semantic information measures taken from the authors' own prior work, and on a large set of empirically chosen loss weights. None of these is independently established, and the derivation is not machine-checked.

free parameters (6)
  • lambda_r^(l) for l=1..16 = 128 to 2048 in steps
    Rate weights in the loss (13), set empirically in Table 1 to define each operating point.
  • lambda_d^(l) for l=1..16 = 2^(39/8) down to 2^(24/8)
    Distortion weights, empirically chosen per synonymous level in Table 1.
  • lambda_p^(l) for l=1..16 = 2^(45/8) down to 2^0
    Perception weights, empirically chosen per synonymous level in Table 1.
  • alpha = 0.5
    Borrowing factor between level-specific and full-level losses in Eq. (12), set empirically.
  • detail sampling range = U(-2,2)
    Empirical uniform sampling range for the detailed representation in Eq. (46), acknowledged by the authors as suboptimal.
  • warm-up weights a and b = a=4, b=64
    Empirical constants in the warm-up constraint Eq. (47) used to keep all channel groups active.
assumptions (6)
  • ad hoc to paper An ideal synset X exists for each source image, containing all images that share a given perceptual meaning.
    Introduced in Section 3.1; the entire derivation of the E-KLD term depends on the existence and structure of this set.
  • ad hoc to paper All samples x_i in the ideal synset satisfy p_{tilde x_j|x_i} = p_{tilde x_j|x} under the current encoder.
    Equation (20) in Appendix A.1; this invariance is the very property the training is supposed to learn, not a given.
  • domain assumption The partial semantic KL divergence and semantic entropy from Niu and Zhang (2024) are valid measures for variational inference.
    Used as the foundation of Definition 3.1 and Eq. (10); the paper does not derive or verify these measures independently.
  • ad hoc to paper The coding rate of the synonymous representation equals the semantic mutual information I(X; hat dot X).
    Equation (35) in Appendix A.2; this rate identity is asserted without a coding theorem and is central to Theorem 3.3.
  • domain assumption The size |X| of the ideal synset is finite and well-defined in the continuous image space.
    Used in Eqs. (21)(c)-(e) to introduce arithmetic means and treat -log|X| as a constant; no rigorous definition is given.
  • domain assumption LPIPS or adversarial loss is a valid surrogate for the E-KLD term in the practical loss.
    The training loss in Eq. (13) replaces the KL divergence with LPIPS, and Section 6 replaces it further with an adversarial loss; the equivalence is not shown.
invented entities (2)
  • Ideal Synset X and reconstructed Synset hat X
    purpose: A set of images considered synonymous with the source image; the decoder is trained to generate members of this set rather than a single reconstruction.
    No independent measurement or falsifiable prediction is provided for the synset; its existence and size are postulated.
  • Latent synonymous representation y_s and detailed representation y_epsilon
    purpose: Latent variables that separate shared semantic content from per-instance detail; only y_s is coded, while y_epsilon is sampled at the decoder.
    These are model constructs introduced for the SIC scheme; no independent evidence outside the paper supports their interpretation as semantic content.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Synonymous Variational Inference for Perceptual Image Compression." pith.science (2026). https://pith.science/paper/UYSVFQ3P

@misc{pith2026250522438,
  author       = {Pith},
  title        = {Pith review of: Synonymous Variational Inference for Perceptual Image Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UYSVFQ3P}},
  note         = {Machine review of arXiv:2505.22438}
}
read the original abstract

Recent contributions of semantic information theory reveal the set-element relationship between semantic and syntactic information, represented as synonymous relationships. In this paper, we propose a synonymous variational inference (SVI) method based on this synonymity viewpoint to re-analyze the perceptual image compression problem. It takes perceptual similarity as a typical synonymous criterion to build an ideal synonymous set (Synset), and approximate the posterior of its latent synonymous representation with a parametric density by minimizing a partial semantic KL divergence. This analysis theoretically proves that the optimization direction of perception image compression follows a triple tradeoff that can cover the existing rate-distortion-perception schemes. Additionally, we introduce synonymous image compression (SIC), a new image compression scheme that corresponds to the analytical process of SVI, and implement a progressive SIC codec to fully leverage the model's capabilities. Experimental results demonstrate comparable rate-distortion-perception performance using a single progressive SIC codec, thus verifying the effectiveness of our proposed analysis method.

Figures

Figures reproduced from arXiv: 2505.22438 by the authors.

Figure 1
Figure 1. An illustration of the optimization directions of synonymous image compression. By continuously minimizing the partial semantic KL divergence DKL,s q||py˜s|X [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Left: Representation of the proposed encoder as a synonymous variational inference model, and corresponding decoder as a generative Bayesian model. The latent representation y˜ is a merge of the synonymous representation y˜s and the detailed representation y˜ϵ , achieved through some form of merging or splicing. A fully factorized (Balle et al. ´ , 2017) or a hyperprior-like (Balle et al. ´ , 2018; Minnen et al., 20… view at source ↗
Figure 3
Figure 3. Processing frameworks of SIC. (a): The general framework. (b): The progressive framework. Due to space limitations, only a brief outline of the proof is provided above. Please refer to Section A.2 for detailed proof. Additionally, we state that the existing rate-distortion￾perception tradeoff (4) is a special case of (11) when there is only one sample xˆ in the reconstructed synset Xˆ . More comprehensive discussion… view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Comparisons of methods using DISTS on different datasets. Each point on the HiFiC and MS-ILLM performance curves is from a single model, while our entire performance curves are achieved by a single progressive SIC model. divergence term since accurately calculating the…
Figure 5
Figure 5. Figure 5: Comparisons of our progressive SIC schemes with different sampling numbers in reconstructed Xˆ on different datasets. age improving as the coding rate increases. For the concerned DISTS measure, our method surpasses the No-GAN MS-ILLM solution (also trained with LPIPS)…
Figure 6
Figure 6. Figure 6: Comparisons of methods using DISTS on different datasets (supplemented fine-tuned model performance). Each point on the HiFiC and MS-ILLM performance curves is from a single model, while our entire performance curves are achieved by a single progressive SIC model. dire…
Figure 7
Figure 7. Figure 7: The implementation details of the auto-encoder framework designed for the progressive SIC model. B.2. Extended Thinking: About Synonymous Idempotence Constraints As discussed earlier, since the ideal synset is not explicitly constructed, directly determining the overla…
Figure 8
Figure 8. Figure 8: The joint autoregressive and hierarchical prior architecture of the progressive SIC model. C. Experimental Illustrations: Implementations and Supplementary Results In this section, we provide implementation details of the progressive SIC model and supplementary results…
Figure 9
Figure 9. Figure 9: The implementation details of the modules in the joint autoregressive and hierarchical prior architecture. and “backward adaptation” are stated in (Balle et al. ´ , 2020). For the synonymous representations yˆs , an Entropy Parameters modules are employed to integrate …
Figure 10
Figure 10. Figure 10: The 3D mask for the masked convolutional layer in the context model [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: Comparisons of methods using PSNR and LPIPS on different datasets. Each point on the HiFiC and MS-ILLM performance curves is from a single model, while our entire performance curves are achieved by a single progressive SIC model. During training, we treat the first 12…
Figure 12
Figure 12. Figure 12: Visualization results of reconstructed images at different synonymous levels using progressive SIC (M = 1). Image from the CLIC2020 test set. Original 0.0532 bpp 0.0664 bpp 0.0766 bpp 0.0863 bpp 0.1243 bpp 0.1927 bpp 0.2450 bpp 0.3859 bpp [PITH_FULL_IMAGE:figures/ful…
Figure 13
Figure 13. Figure 13: Visualization results of reconstructed images at different synonymous levels using progressive SIC (M = 1). Image from the DIV2K validation set. Original 0.0661 bpp 0.0795 bpp 0.0936 bpp 0.1055 bpp 0.1502 bpp 0.2259 bpp 0.2868 bpp 0.4466 bpp [PITH_FULL_IMAGE:figures/…
Figure 14
Figure 14. Figure 14: Visualization results of reconstructed images at different synonymous levels using progressive SIC (M = 1). Image from the Kodak validation set. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]
Figure 15
Figure 15. Figure 15: Visualization comparison of reconstructed images at synonymous level l = 5 using progressive SIC with M = 1 and M = 5. Image from the CLIC2020 test set. Original Rate = 0.2158 bpp / DISTS = 0.0501 Rate = 0.2156 bpp / DISTS = 0.0492 [PITH_FULL_IMAGE:figures/full_fig_p…
Figure 16
Figure 16. Figure 16: Visualization comparison of reconstructed images at synonymous level l = 6 using progressive SIC with M = 1 and M = 5. Image from the DIV2K validation set. Original Rate = 0.3600 bpp / DISTS = 0.1021 Rate = 0.3281 bpp / DISTS = 0.0935 [PITH_FULL_IMAGE:figures/full_fi…
Figure 17
Figure 17. Figure 17: Visualization comparison of reconstructed images at synonymous level l = 7 using progressive SIC with M = 1 and M = 5. Image from the Kodat dataset. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_17.png]
Figure 18
Figure 18. Figure 18: Employed Discriminator Architecture in our finetuning attempts. semantic information and progressively enhances the details in the reconstructed image. These visualization results demonstrate that the progressive SIC model we implemented can effectively leverage the s…
Figure 19
Figure 19. Figure 19: Comparisons of methods using PSNR, LPIPS, DISTS, and FID on different datasets (Supplemented fine-tuned model performance). Each point on the HiFiC and MS-ILLM performance curves is from a single model, while our entire performance curves are achieved by a single prog…
Figure 20
Figure 20. Figure 20: Visualization results of reconstruction images: No-GAN vs. with GAN (Synonymous level l = 1). Images from the CLIC2020 test set, the DIV2K validation set, and the Kodak dataset. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_20.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 46 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Timofte, R

    Agustsson, E. and Timofte, R. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017

    work page 2017

  3. [3]

    Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., and Gool, L. V. Generative adversarial networks for extreme learned image compression. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    work page 2019

  4. [4]

    Multi-realism image compression with a conditional generator

    Agustsson, E., Minnen, D., Toderici, G., and Mentzer, F. Multi-realism image compression with a conditional generator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 22324--22333, June 2023

    work page 2023

  5. [5]

    Fuzzy entropy: a brief survey

    Al-Sharhan, S., Karray, F., Gueaieb, W., and Basir, O. Fuzzy entropy: a brief survey. In 10th IEEE international conference on fuzzy systems.(Cat. No. 01CH37297), volume 3, pp.\ 1135--1139. IEEE, 2001

    work page 2001

  6. [6]

    Ball \'e , J., Laparra, V., and Simoncelli, E. P. End-to-end optimized image compression. In International Conference on Learning Representations (ICLR), 2017

    work page 2017

  7. [7]

    J., and Johnston, N

    Ball \'e , J., Minnen, D., Singh, S., Hwang, S. J., and Johnston, N. Variational image compression with a scale hyperprior. In International Conference on Learning Representations (ICLR), 2018

    work page 2018

  8. [8]

    A., Minnen, D., Singh, S., Johnston, N., Agustsson, E., Hwang, S

    Ball \'e , J., Chou, P. A., Minnen, D., Singh, S., Johnston, N., Agustsson, E., Hwang, S. J., and Toderici, G. Nonlinear transform coding. IEEE Journal of Selected Topics in Signal Processing, 15 0 (2): 0 339--353, 2020

    work page 2020

Show all 54 references
  1. [9]

    Bao, J., Basu, P., Dean, M., Partridge, C., Swami, A., Leland, W., and Hendler, J. A. Towards a theory of semantic communication. In 2011 IEEE Network Science Workshop, pp.\ 110--117. IEEE, 2011

    2011

  2. [10]

    and Carnap, R

    Bar-Hillel, Y. and Carnap, R. Semantic information. The British Journal for the Philosophy of Science, 4 0 (14): 0 147--157, 1953

    1953

  3. [11]

    and Perry, J

    Barwise, J. and Perry, J. Situations and attitudes. The Journal of Philosophy, 78 0 (11): 0 668--691, 1981

    1981

  4. [12]

    BPG image format, 2015

    Bellard, F. BPG image format, 2015. URL https://bellard.org/bpg

    2015

  5. [13]

    and Michaeli, T

    Blau, Y. and Michaeli, T. The perception-distortion tradeoff. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    2018

  6. [14]

    and Michaeli, T

    Blau, Y. and Michaeli, T. Rethinking lossy compression: The rate-distortion-perception tradeoff. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning (ICML), pp.\ 675--685. PMLR, 2019

    2019

  7. [15]

    J., Verbeek, J., and Lathuili \`e re, S

    Careil, M., Muckley, M. J., Verbeek, J., and Lathuili \`e re, S. Towards image compression with perfect realism at ultra-low bitrates. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  8. [16]

    and Bar-Hillel, Y

    Carnap, R. and Bar-Hillel, Y. An outline of a theory of semantic information. 1952

    1952

  9. [17]

    Learned image compression with discretized gaussian mixture likelihoods and attention modules

    Cheng, Z., Sun, H., Takeuchi, M., and Katto, J. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  10. [18]

    and Termini, S

    De Luca, A. and Termini, S. A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory. Information and Control, 20 0 (4): 0 301--312, 1972

    1972

  11. [19]

    and Termini, S

    De Luca, A. and Termini, S. Entropy of L -fuzzy sets. Information and Control, 24 0 (1): 0 55--73, 1974

    1974

  12. [20]

    Ding, K., Ma, K., Wang, S., and Simoncelli, E. P. Image quality assessment: Unifying structure and texture similarity. IEEE transactions on pattern analysis and machine intelligence, 44 0 (5): 0 2567--2581, 2020

    2020

  13. [21]

    Outline of a theory of strongly semantic information

    Floridi, L. Outline of a theory of strongly semantic information. Minds and machines, 14: 0 197--221, 2004

    2004

  14. [22]

    J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y

    Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS). Curran Associates, Inc., 2014

    2014

  15. [23]

    B., and Gunduz, D

    Hamdi, Y., Wagner, A. B., and Gunduz, D. The rate-distortion-perception trade-off with algorithmic realism, 2025

    2025

  16. [24]

    Checkerboard context model for efficient learned image compression

    He, D., Zheng, Y., Sun, B., Wang, Y., and Qin, H. Checkerboard context model for efficient learned image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 14771--14780, June 2021

    2021

  17. [25]

    ELIC : Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding

    He, D., Yang, Z., Peng, W., Ma, R., Qin, H., and Wang, Y. ELIC : Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 5718--5727, Ju...

    2022

  18. [26]

    PO-ELIC : Perception-oriented efficient learned image coding

    He, D., Yang, Z., Yu, H., Xu, T., Luo, J., Chen, Y., Gao, C., Shi, X., Qin, H., and Wang, Y. PO-ELIC : Perception-oriented efficient learned image coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.\ 1764--1769, Ju...

    2022

  19. [27]

    GANs trained by a two time-scale update rule converge to a local nash equilibrium

    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances i...

    2017

  20. [28]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems (NeurIPS), pp.\ 6840--6851. Curran Associates, Inc., 2020

    2020

  21. [29]

    High-fidelity image compression with score-based generative models

    Hoogeboom, E., Agustsson, E., Mentzer, F., Versari, L., Toderici, G., and Theis, L. High-fidelity image compression with score-based generative models. arXiv preprint arXiv:2305.18231, 2023

    2023 arXiv

  22. [30]

    Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  23. [31]

    The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale

    Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International jour...

    1956

  24. [32]

    Spatial-channel context-based entropy modeling for end-to-end optimized image compression

    Li, C., Luo, J., Dai, W., Li, C., Zou, J., and Xiong, H. Spatial-channel context-based entropy modeling for end-to-end optimized image compression. In 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp.\ 222--225, 2020

    2020

  25. [33]

    Frequency-aware transformer for learned image compression

    Li, H., Li, S., Dai, W., Li, C., Zou, J., and Xiong, H. Frequency-aware transformer for learned image compression. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  26. [34]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 10012--10022, October 2021

    2021

  27. [35]

    D., Tschannen, M., and Agustsson, E

    Mentzer, F., Toderici, G. D., Tschannen, M., and Agustsson, E. High-fidelity generative image compression. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems (NeurIPS), pp.\ 11913--11924. Curran Associ...

    2020

  28. [36]

    Minnen, D., Ball\' e , J., and Toderici, G. D. Joint autoregressive and hierarchical priors for learned image compression. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems (Neur...

    2018

  29. [37]

    J., El-Nouby, A., Ullrich, K., Jegou, H., and Verbeek, J

    Muckley, M. J., El-Nouby, A., Ullrich, K., Jegou, H., and Verbeek, J. Improving statistical fidelity for neural image compression with implicit local likelihood models. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of t...

    2023

  30. [38]

    and Zhang, P

    Niu, K. and Zhang, P. A mathematical theory of semantic communication. Journal on Communications, 45 0 (6): 0 7--59, 2024. URL https://www.joconline.com.cn/en/article/doi/10.11959/j.issn.1000-436x.2024111/

    2024 doi

  31. [39]

    A rate-distortion-perception theory for binary sources

    Qian, J., Zhang, G., Chen, J., and Khisti, A. A rate-distortion-perception theory for binary sources. In International Zurich Seminar on Information and Communication (IZS 2022). Proceedings, pp.\ 34--38. ETH Zurich, 2022

    2022

  32. [40]

    Generative compression

    Santurkar, S., Budden, D., and Shavit, N. Generative compression. In 2018 Picture Coding Symposium (PCS), pp.\ 258--262, 2018

    2018

  33. [41]

    Shannon, C. E. A mathematical theory of communication. The Bell system technical journal, 27 0 (3): 0 379--423, 1948

    1948

  34. [42]

    Shannon, C. E. et al. Coding theorems for a discrete source with a fidelity criterion. IRE Nat. Conv. Rec, 4 0 (142-163): 0 1, 1959

    1959

  35. [43]

    Position: What makes an image realistic? In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024

    Theis, L. Position: What makes an image realistic? In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024

    2024

  36. [44]

    and Agustsson, E

    Theis, L. and Agustsson, E. On the advantages of stochastic encoders. In Neural Compression: From Information Theory to Applications -- Workshop @ ICLR 2021, 2021

    2021

  37. [45]

    and Wagner, A

    Theis, L. and Wagner, A. B. A coding theorem for the rate-distortion-perception function. In Neural Compression: From Information Theory to Applications -- Workshop @ ICLR 2021, 2021

    2021

  38. [46]

    D., and Mentzer, F

    Theis, L., Salimans, T., Hoffman, M. D., and Mentzer, F. Lossy compression with gaussian diffusion. arXiv preprint arXiv:2206.08889, 2022

    2022 arXiv

  39. [47]

    and Joy, A

    Thomas, M. and Joy, A. T. Elements of information theory. Wiley-Interscience, 2006

    2006

  40. [48]

    CLIC 2020: Challenge on learned image compression, 2020, 2020

    Toderici, G., Theis, L., Johnston, N., Agustsson, E., Mentzer, F., Ball \'e , J., Shi, W., and Timofte, R. CLIC 2020: Challenge on learned image compression, 2020, 2020

    2020

  41. [49]

    Wallace, G. K. The JPEG still picture compression standard. Communications of the ACM, 34 0 (4): 0 30--44, 1991

    1991

  42. [50]

    Recent contributions to the mathematical theory of communication

    Weaver, W. Recent contributions to the mathematical theory of communication. ETC: a review of general semantics, pp.\ 261--281, 1953

    1953

  43. [51]

    Conditional perceptual quality preserving image compression

    Xu, T., Zhang, Q., Li, Y., He, D., Wang, Z., Wang, Y., Qin, H., Wang, Y., Liu, J., and Zhang, Y.-Q. Conditional perceptual quality preserving image compression. arXiv preprint arXiv:2308.08154, 2023

    2023 arXiv

  44. [52]

    Idempotence and perceptual image compression

    Xu, T., Zhu, Z., He, D., Li, Y., Guo, L., Wang, Y., Wang, Z., Qin, H., Wang, Y., Liu, J., and Zhang, Y.-Q. Idempotence and perceptual image compression. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  45. [53]

    On perceptual lossy compression: The cost of perceptual reconstruction and an optimal training framework

    Yan, Z., Wen, F., Ying, R., Ma, C., and Liu, P. On perceptual lossy compression: The cost of perceptual reconstruction and an optimal training framework. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning (ICML), pp.\ 11682-...

    2021

  46. [54]

    A., Shechtman, E., and Wang, O

    Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.