Pith. sign in

REVIEW 4 major objections 6 minor 51 references

LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that LaMI-GO, which sends masked image tokens plus a text caption and regenerates the rest with a pre-trained latent diffusion model, improves quality while cutting bandwidth roughly tenfold versus GO-COM.

desk verdict A genuinely new latent mixture integration rule lets a frozen diffusion model fill masked codebook indices with large bandwidth savings, but the headline comparisons and calibration questions need work before the numbers can be fully trusted. read the letter →

arxiv 2412.17839 v1 pith:MBQH3F66 submitted 2024-12-18 cs.LG cs.AIeess.IV

Achintha Wijesinghe , Suchinthaka Wanninayaka , Weiwei Wang , Yu-Chieh Chao , Songyang Zhang , Zhi Ding This is my paper · ORCID
classification cs.LGcs.AIeess.IV
keywords goal-orientedcommunicationssemanticlatentdiffusionvectorquantizationVQGANimagereconstructionpacketlossbandwidthefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Goal-oriented communication (GO-COM) aims to deliver what a receiver needs for a task rather than reproducing every bit of a source. This paper proposes LaMI-GO, a system that quantizes an image into codebook indices, transmits a subset of those indices together with a compressed text caption, and reconstructs the image on the receiver side with a pre-trained latent diffusion model. The paper's central claim is that this design outperforms prior GO-COM systems on perceptual quality, downstream-task accuracy, bandwidth, and recovery time without retraining the generative backbone. In the Cityscapes comparisons, LaMI-GO reached FID 22.72 at 9.39 KB per image, while the diffusion-based competitor Diff-GO+ reached FID 58.20 at 87.04 KB, and LaMI-GO needed only 8 diffusion steps where earlier methods used 1000.

What carries the argument

The central mechanism is the latent mixture integration update, $\hat{I}_i = (\mathbf{1}-M) \odot \hat{I}_e + M \odot \hat{I}_{p,i}$, where $M$ marks the positions masked by the transmitter, $\hat{I}_e$ is the received index matrix, and $\hat{I}_{p,i}$ is a predicted index matrix for the masked positions. At the first step $\hat{I}_{p,1}$ is sampled randomly from the shared codebook $W$; in later steps it is drawn by multinomial sampling from the softmax output of the pre-trained latent diffusion model, possibly after a re-noising operation for early iterations. This mechanism keeps transmitted indices intact while letting the diffusion model fill in the masked ones, which is what allows the system to reuse a pre-trained generative backbone unchanged.

What would settle it

Measure the pre-trained diffusion model's per-token prediction accuracy on Cityscapes images under the three masking policies at masking rates 25% and 35%, especially for contiguous blocks as in entropy-based masking; if accuracy on masked positions falls to the rate of the most common codebook class or degrades sharply when received tokens are corrupted, the claimed bandwidth savings would carry an unmeasured reconstruction cost.

Watch

Extended reading notes

Core claim

The paper claims that a goal-oriented communication link can be built by encoding an image into a VQGAN latent space, transmitting only a masked subset of the resulting codebook indices plus a text description, and using the pre-trained Paella latent diffusion model to regenerate the omitted indices. The core discovery is the latent mixture integration update, which holds the received indices fixed while iteratively replacing masked positions with model predictions, so a diffusion model trained for random masking behaves as a calibrated predictor for the communication masks without retraining or fine-tuning. On Cityscapes, LaMI-GO reports FID 29.84 with 8 diffusion steps versus FID 55.85 to 96.41 for prior GO-COM methods with 1000 steps, and at 9.39 KB per image it reports FID 22.72 versus Diff-GO+ at 87.04 KB with FID 58.20.

Load-bearing premise

The load-bearing premise is that the pre-trained diffusion model, trained with its own random masking schedule, gives reliable predictions for LaMI-GO's arbitrary masking policies and for corrupted received positions; the paper itself notes in Section V-C that the random-mask model "has limited capabilities to combat higher distortion rates without fine-tuning the diffusion model."

Editorial extensions

If this is right

  • A GO-COM system can reuse a pre-trained latent diffusion model and VQGAN codebook as-is, so deployers do not need to train or fine-tune a diffusion backbone for each new dataset or task.
  • Bandwidth becomes continuously tunable through the masking probability and policy: on Cityscapes, LaMI-GO sends about 9.39 KB per image at 25% masking and about 8.50 KB at 35% masking, with measured FID cost from 22.72 to 27.23.
  • Receiver recovery time drops from the order of 1000 diffusion steps used by prior diffusion GO-COM systems to 8 to 200 steps, with a reported showtime of about 1.29 seconds at 4/8 steps.
  • Reconstruction starts from real image tokens rather than random noise, so the same recovered image supports several downstream tasks; depth-estimation RMSE improves from 6.58 (Diff-GO+) to 5.59 (LaMI-GO), and object-detection mAP reaches about 64% on tested datasets.
  • With block interleaving, the system maintains acceptable perceptual quality up to about 20% packet loss and remains robust at practical bit-error rates, since lost or corrupted tokens can be regenerated by the diffusion model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the latent mixture integration rule only requires a model that predicts codebook indices from partial masks, so the same recipe may transfer to other discrete latent diffusion or VQ models, making the contribution a general mechanism rather than a single-system result.
  • Inference: the three masking policies suggest an adaptive policy that chooses which tokens to drop based on content entropy and expected channel conditions; the paper does not test such an adaptive policy, and it could improve the bandwidth-quality tradeoff further.
  • Inference: the comparisons against classical JPEG at similar bandwidth are not computation-free, because LaMI-GO moves the cost to a receiver that must run a diffusion model; how that tradeoff is valued depends on the receiver's hardware budget.
  • Inference: the paper's reported savings assume the pre-trained model's predictions remain well-calibrated for contiguous entropy-based masks and corrupted received positions; a direct test of per-token prediction accuracy under those masks would quantify how much of the bandwidth gain is real.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LaMI-GO, a goal-oriented communication system that compresses an image into VQGAN codebook indices, masks a fraction of the indices under one of three policies (pseudo-random, pre-determined, and entropy-based), transmits the remaining indices along with an automatically generated caption, and reconstructs the image at the receiver by iterative latent diffusion using a frozen Paella model. The central claim is that the proposed latent mixture integration rule in Eqs. (12)-(13), which preserves unmasked indices at every diffusion step while multinomial-sampling the masked positions, enables a pre-trained text-to-image diffusion model to achieve strong perceptual quality, downstream-task accuracy, bandwidth savings, and reduced showtime without retraining the generative backbone. Supporting experiments on Cityscapes, Flickr, COCO-Stuff, and COCO compare LaMI-GO with prior GO-COM systems (Diff-GO, Diff-GO+, GESCO) and with generative text-to-image baselines.

Significance. If the central claim is established, the contribution is practically valuable: it decouples a GO-COM system from the need to train or fine-tune a generative backbone, reuses a public latent diffusion prior, and achieves large bandwidth reductions with few diffusion steps. The ablation in Fig. 13 is a clear strength: it demonstrates that without latent mixture integration the receiver drifts toward a generic text-conditional image, whereas with it the recovered image remains close to the transmitted source. The paper also provides a concrete, falsifiable recipe (masking policies plus an integration rule) and evaluates several datasets and downstream tasks. However, some load-bearing comparisons are not matched in information content, and the calibration of the frozen Paella model at the operating points that deliver the largest bandwidth savings is not quantified; these issues should be resolved before the headline claims are fully supported.

major comments (4)
  1. [Section V-C, Tables VII, Figs. 8-9] The paper's own text concedes that "the PRM model has limited capabilities to combat higher distortion rates without fine-tuning the diffusion model" (Section V-C), yet the lowest-bandwidth headline number in Table VII is LaMI-GO with PRM p=0.35 (8.50 KB, FID 27.23), and the p=0.25 row (9.39 KB, FID 22.72) is the second headline. The no-retraining claim is therefore demonstrated precisely in the regime the text flags as degraded, and no calibration check, error bar, or sensitivity analysis across p is reported. Please quantify the change in FID/LPIPS as p varies, with uncertainty estimates over images or splits, and either support the p=0.35 operating point with a calibration analysis or restrict the no-retraining claim to the p=0.25 configuration.
  2. [Section V-A, Table II] Table II compares LaMI-GO FID scores against generative models (DMGAN, XMCGAN, DFGAN, SSA-GAN, DSE-GAN, VQ-Diffusion, VQ-GAN, Stackformer) that are conditioned only on text. LaMI-GO additionally receives a large set of unmasked, position-aware codebook indices for every image, so the information available to the two kinds of systems is not matched, and the table cannot support the statement that LaMI-GO is "flexible, competitive, and better" as a generative model. Please remove this comparison from the central claims or convert it into a clearly labeled side-information upper bound, and base the headline comparison on GO-COM systems with comparable conditioning information.
  3. [Section IV-C, Eq. (13); Section IV-D.3, Table IV] The method rests on the assumption that the frozen Paella model f_phi remains a calibrated predictor for index distributions under LaMI-GO's input distribution (fixed unmasked positions, multinomial-sampled masked positions, and, in the channel experiments, corrupted received tokens). No calibration diagnostic is reported. The EBM results in Table IV themselves indicate distribution shift: at the same step budget, EBM degrades FID on Flickr from 35.62 (random) to 54.15 and on COCO-Stuff from 10.37 to 29.31. Please report a calibration measure (for example, expected calibration error of the predicted index probabilities against the transmitted indices, separately for PRM, PDM, and EBM) or explicitly restrict the no-retraining claim to the policies for which calibration is verified.
  4. [Section V-E, Table VII] The bandwidth comparison in Table VII does not provide a complete bit budget. The transmitted object is mc = compress(ˆIe, cT) (Algorithm 1), but the table does not state how many bits are spent on the caption, on the mask description (PRNG seed for PRM, mask index for PDM, or message-dependent masks for EBM), or on channel coding, nor whether the same entropy coding is applied to all compared systems. Because the central claim is spectrum efficiency, please give the exact per-component bit counts and state the assumptions about overhead for every method in the table.
minor comments (6)
  1. [Section IV-C, Eq. (13)] The piecewise definition of \hat I_{p,i} leaves i=tau and i=T undefined (the conditions are i<tau and tau<i<T); please clarify the endpoint handling.
  2. [Algorithm 2] In Algorithm 2 the condition "If t < tau" should be "If i < tau" (or otherwise t should be defined before use), and the equation for \hat I_1 uses (1a\times b) while Eq. (12) uses (1h'\times w'); please align the notation.
  3. [Section IV-D.3] The entropy-based masking policy depends on a threshold eta and on "selecting the best n candidate indices," but the paper never states how eta or n is chosen or how the mask overhead is transmitted; please specify the procedure and include these parameters in the bandwidth accounting.
  4. [Abstract, Section III-A] The abstract contains a grammatical error ("common-feature codebookthe receiver side") and, more importantly, says the system "trains" a codebook at the receiver, while Section III-A states that a pre-trained codebook is shared as a one-time configuration; please correct the wording.
  5. [References [7] and [22]] References [7] and [22] cite the same work (Esser et al., CVPR 2021, "Taming Transformers for High-Resolution Image Synthesis") as if they were different publications; please consolidate or disambiguate them.
  6. [Table V] The method name is spelled inconsistently ("Diff-Go+" in the table body and "Diff-GO+" in the caption and text); please standardize the spelling.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: LaMI-GO is an inference-time application of an external pre-trained Paella/VQGAN backbone; self-citations are only baseline comparisons.

full rationale

The central LaMI-GO derivation is an inference-time mixing rule, Eq. (12)-(13), that clamps received unmasked codebook indices and fills masked positions using multinomial samples from the externally pre-trained Paella model f_phi(., t, C). No parameter in the paper is fitted to the reported FID/LPIPS/bandwidth numbers, so the headline metrics are not reproduced by construction. The masking policies (PRM, PDM, EBM) select which indices are withheld; the receiver's predictions are generated by Paella rather than optimized against the test set, so these are not fitted inputs renamed as predictions. Comparisons to the authors' earlier Diff-GO/Diff-GO+ systems are empirical baselines, not load-bearing derivations: the LaMI-GO results do not reduce to those papers' outputs. The only potentially load-bearing premise is that Paella remains calibrated for the specific masks (especially EBM contiguous masks and corrupted received tokens); the paper's own Section V-C concedes limited capability at higher distortion without fine-tuning, but this is an honest limitation and a correctness/robustness concern rather than a circular step, since the reported metrics are measured, not entailed by the algorithm's definition. External benchmarks (VQGAN, Paella, JPEG/JPEG-2000, SPADE, etc.) ground the comparison, so the paper is self-contained against external reference points.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

LaMI-GO introduces no new physical entities; its central claim rests on the transferability of pre-trained generative components and on hand-chosen experimental parameters. The codebook size, the VQGAN downsample factor, and the Paella checkpoints are inherited from prior work and are treated as fixed inputs.

free parameters (4)
  • Masking probability p = 0.25 or 0.35
    Chosen by hand for PRM and EBM experiments; directly controls bandwidth and reconstruction quality, and all headline comparisons use these values.
  • Noising and denoising step pairs (tau/T) = 4/8, 28/32, 50/62, 94/100, 194/200
    Selected per experiment; more steps improve FID, LPIPS, and robustness but increase latency. The paper does not provide a selection rule.
  • EBM masking threshold eta = Not specified
    Eq. (17) masks entries whose distance to the codebook is below eta; the paper never states how eta is set or tuned, which is required to reproduce the EBM results.
  • PDM fixed mask pattern = 1 if a and b are both even
    A hand-designed one-time checkerboard pattern masking 1/4 of tokens; the paper acknowledges other patterns could be used, but results depend on this choice.
assumptions (4)
  • domain assumption The pre-trained VQGAN encoder, decoder, and codebook and the Paella discrete diffusion model share the same quantized latent space and codebook indexing.
    The transmitter quantizes with a codebook and the receiver's Paella predicts indices in the same codebook. If the latent grids or codebooks were misaligned, Eqs. (6) through (14) would be invalid. Invoked in Sections III-A and IV-A.
  • domain assumption Paella's trained denoising and re-noising behavior transfers to LaMI-GO's fixed unmasked positions and arbitrary mask patterns without fine-tuning.
    Eq. (13) uses f_phi(., t, C) as a calibrated predictor on masked positions while keeping unmasked positions frozen, even though Paella was trained under its own random masking schedule. Section V-C concedes the model has limited ability to combat higher distortion without fine-tuning.
  • domain assumption The text captions produced by the selected lightweight captioning model are sufficiently aligned with Paella's text-conditioning distribution to guide reconstruction.
    Section III-A sends the caption as a condition; no experiment quantifies caption quality or its effect on the final metrics.
  • domain assumption Packet erasures after interleaving are well modeled as a binary erasure channel, and fixed-length coding avoids Huffman sensitivity to bit errors.
    Section V-F adopts this model from reference [49] for the noise-setup experiments; the reported robustness results depend on this idealized channel model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency." pith.science (2026). https://pith.science/paper/MBQH3F66

@misc{pith2026241217839,
  author       = {Pith},
  title        = {Pith review of: LaMI-GO: Latent Mixture Integration for Goal-Oriented Communications Achieving High Spectrum Efficiency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MBQH3F66}},
  note         = {Machine review of arXiv:2412.17839}
}
read the original abstract

The recent rise of semantic-style communications includes the development of goal-oriented communications (GOCOMs) remarkably efficient multimedia information transmissions. The concept of GO-COMS leverages advanced artificial intelligence (AI) tools to address the rising demand for bandwidth efficiency in applications, such as edge computing and Internet-of-Things (IoT). Unlike traditional communication systems focusing on source data accuracy, GO-COMs provide intelligent message delivery catering to the special needs critical to accomplishing downstream tasks at the receiver. In this work, we present a novel GO-COM framework, namely LaMI-GO that utilizes emerging generative AI for better quality-of-service (QoS) with ultra-high communication efficiency. Specifically, we design our LaMI-GO system backbone based on a latent diffusion model followed by a vector-quantized generative adversarial network (VQGAN) for efficient latent embedding and information representation. The system trains a common feature codebook the receiver side. Our experimental results demonstrate substantial improvement in perceptual quality, accuracy of downstream tasks, and bandwidth consumption over the state-of-the-art GOCOM systems and establish the power of our proposed LaMI-GO communication framework.

Figures

Figures reproduced from arXiv: 2412.17839 by the authors.

Figure 1
Figure 1. Overall architecture of the proposed LaMI-GO . Transmitter: LaMI-GO has an encoder model to extract essential information of a given image. This [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Channel noise model. We consider digital communication with packet [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The iterative process of latent mixture integration. Here, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: The showtime on the receiver end. matrix for masking out 1/4 of the tokens using the following mask, Mf (a, b) = ( 1 if a and b both are even 0 otherwise, (15) This exemplary PDM repeats a 2 × 2 pattern. 3) Entropy Based Masking (EBM): Lastly, we present an entropy-bas…
Figure 8
Figure 8. Figure 8: Perceptual quality evaluation using FID for different semantic [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Perceptual quality evaluation using LPIPS for different seman [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Variation of the perceptual quality on the receiver side in terms of [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Variation of the perceptual quality on the receiver side in terms of [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 13
Figure 13. Figure 13: Effect of latent mixture integration process. The first column repre [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: Misclassified indices vs denoising iterations. [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 15
Figure 15. Figure 15: Variation of the perceptual quality in a noisy channel. The model is [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 16
Figure 16. Figure 16: Variation of the perceptual quality in a noisy channel according to [PITH_FULL_IMAGE:figures/full_fig_p013_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 45 canonical work pages

  1. [1]

    Implementation Challenges and Opportunities in Beyond-5G and 6G Communication,

    U. Gustavsson et al., “Implementation Challenges and Opportunities in Beyond-5G and 6G Communication,” in IEEE Journal of Microwaves , vol. 1, no. 1, pp. 86-100, Jan. 2021

    work page 2021

  2. [2]

    A mathematical theory of communication,

    C. E. Shannon, “A mathematical theory of communication,” in The Bell System Technical Journal , vol. 27, no. 3, pp. 379-423, Jul. 1948

    work page 1948

  3. [3]

    Diff-go: diffusion goal-oriented communications to achieve ultra-high spectrum efficiency,

    A. Wijesinghe, S. Zhang, S. Wanninayaka, W. Wang, and Z. Ding, “Diff-go: diffusion goal-oriented communications to achieve ultra-high spectrum efficiency,” arXiv:2312.02984, Nov. 2023

    arXiv 2023

  4. [4]

    GO+: An Efficient Diffusion Goal-Oriented Communication System with Local Feedback,

    A. Wijesinghe, S. Zhang, S. Wanninayaka, W. Wang, and Z. Ding, “GO+: An Efficient Diffusion Goal-Oriented Communication System with Local Feedback,” TechRxiv., June 24, 2024

    work page 2024

  5. [5]

    6G networks: Beyond Shannon towards semantic and goal-oriented communications,

    E.C. Strinati, and S. Barbarossa, “6G networks: Beyond Shannon towards semantic and goal-oriented communications,” in Computer Networks, vol. 190, p.107930, May, 2021

    work page 2021

  6. [6]

    Diff-GO n: Enhancing Diffusion Models for Goal-Oriented Communications,

    S. Wanninayake, A. Wijesinghe, W. Wang, Y .-C. Chao, S. Zhang, and Z. Ding, “Diff-GO n: Enhancing Diffusion Models for Goal-Oriented Communications,” arXiv preprint arXiv:2412.06980 , 2024

    work page arXiv 2024

  7. [7]

    Taming transformers for high- resolution image synthesis

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high- resolution image synthesis”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2021, pp. 12873–12883

    work page 2021

  8. [8]

    Generative semantic communication: diffusion models beyond bit recovery,

    E. Grassucci, S. Barbarossa, and D. Comminiello, “Generative semantic communication: diffusion models beyond bit recovery,” arXiv:2306.04321, Jun. 2023

    arXiv 2023

Show all 51 references
  1. [9]

    Semantic-preserving image coding based on conditional diffusion models

    F. Pezone, O. Musa, G. Caire, and S. Barbarossa, “Semantic-preserving image coding based on conditional diffusion models”, in ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 13501–13505

    2024

  2. [10]

    Robust semantic communications with masked vq-vae enabled codebook

    Q. Hu, G. Zhang, Z. Qin, Y . Cai, G. Yu, and G. Y . Li, “Robust semantic communications with masked vq-vae enabled codebook”, IEEE Transactions on Wireless Communications , pp. 1–1, 2023

    2023

  3. [11]

    Autoencoder-based semantic communication systems with relay channels

    X. Luo, B. Yin, Z. Chen, B. Xia, and J. Wang, “Autoencoder-based semantic communication systems with relay channels”, in 2022 IEEE ICC Workshops, 2022, pp. 711–716

    2022

  4. [12]

    Communication beyond transmitting bits: Semantics-guided source and channel coding

    J. Dai, P. Zhang, K. Niu, S. Wang, Z. Si, and X. Qin, “Communication beyond transmitting bits: Semantics-guided source and channel coding”, IEEE Wireless Communications , vol. 30, no. 4, pp. 170–177, 2023

    2023

  5. [13]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyals, and others, “Neural discrete representation learning,” Advances in neural information processing systems , vol. 30, 2017

    2017

  6. [14]

    A theory of goal-oriented communication,

    O. Goldreich, B. Juba, and M. Sudan, “A theory of goal-oriented communication,” in Journal of the ACM (JACM) , vol. 59, no. 2, pp. 1-65, May 2012. 14

    2012

  7. [15]

    Deep learning enabled semantic communications with speech recognition and synthesis,

    Z. Weng, Z. Qin, X. Tao, C. Pan, G. Liu and G. Y . Li, “Deep learning enabled semantic communications with speech recognition and synthesis,” in IEEE Transactions on Wireless Communications , vol. 22, no. 9, pp. 6227-6240, Sep. 2023

    2023

  8. [16]

    Rethinking modern communication from semantic coding to semantic communication,

    K. Lu, Q. Zhou, R. Li, Z. Zhao, X. Chen, J. Wu, and H. Zhang, “Rethinking modern communication from semantic coding to semantic communication,” in IEEE Wireless Communications , vol. 30, no. 1, pp. 158-164, Feb. 2023

    2023

  9. [17]

    A lite distributed semantic communication system for internet of things,

    H. Xie and Z. Qin, “A lite distributed semantic communication system for internet of things,” in IEEE Journal on Selected Areas in Commu- nications, vol. 39, no. 1, pp. 142-153, Jan. 2021

    2021

  10. [18]

    Multimodal semantic communication accelerated bidirectional caching for 6G mec

    C. Wang, X. Yu, L. Xu, Z. Wang, W. Wang, “Multimodal semantic communication accelerated bidirectional caching for 6G mec”, in Future Generation Computer Systems , vol. 140, pp. 225-237, Mar. 2023

    2023

  11. [19]

    A unified multi- task semantic communication system for multimodal data,

    G. Zhang, Q. Hu, Z. Qin, Y . Cai, G. Yu and X. Tao, “A unified multi- task semantic communication system for multimodal data,” in IEEE Transactions on Communications , Early Access

  12. [20]

    Large ai model empowered multimodal semantic communications,

    F. Jiang, Y . Peng, L. Dong, K. Wang, K.Yang, C. Pan, and X. You, “Large ai model empowered multimodal semantic communications,” arXiv:2309.01249, Sep. 2023

    2023 arXiv

  13. [21]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in NeurIPS, vol. 33, 2020, pp. 6840–6851

    2020

  14. [22]

    Taming transformers for high- resolution image synthesis,

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high- resolution image synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 12873–12883

    2021

  15. [23]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models”, in International conference on machine learning , 2023, pp. 19730–19742

    2023

  16. [24]

    A novel sampling scheme for text-and image-conditional image synthesis in quantized latent spaces

    D. Rampas, P. Pernias, and M. Aubreville, “A novel sampling scheme for text-and image-conditional image synthesis in quantized latent spaces”, arXiv:2211. 07292 , 2022

    2022

  17. [25]

    Xue et al., ‘Byt5: Towards a token-free future with pre-trained byte- to-byte models’, Transactions of the Association for Computational Linguistics, vol

    L. Xue et al., ‘Byt5: Towards a token-free future with pre-trained byte- to-byte models’, Transactions of the Association for Computational Linguistics, vol. 10, pp. 291–306, 2022

    2022

  18. [26]

    Radford et al., ‘Learning transferable visual models from natural language supervision’, in International conference on machine learning, 2021, pp

    A. Radford et al., ‘Learning transferable visual models from natural language supervision’, in International conference on machine learning, 2021, pp. 8748–8763

    2021

  19. [27]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele. “The cityscapes dataset for semantic urban scene understanding,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , Paradise, NV , USA, J...

    2016

  20. [28]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

    P. Young, A. Lai, M. Hodosh, and J. Hockenmaier, “From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions”, Transactions of the Association for Computational Linguistics , vol. 2, pp. 67–78, 2014

    2014

  21. [29]

    Coco-stuff: Thing and stuff classes in context

    H. Caesar, J. Uijlings, and V . Ferrari, “Coco-stuff: Thing and stuff classes in context”, in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 1209–1218

    2018

  22. [30]

    Microsoft coco: Common objects in context

    T.-Y . Lin et al., “Microsoft coco: Common objects in context”, in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, 2014, pp. 740–755

    2014

  23. [31]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochre- iter,“Gans trained by a two time-scale update rule converge to a local nash equilibrium,” NeurIPS, vol. 30, 2017

    2017

  24. [32]

    Learned perceptual image enhancement,

    H. Talebi and P. Milanfar, “Learned perceptual image enhancement,” in 2018 IEEE International Conference on Computational Photogra- phy(ICCP), 2018, pp. 1–13

    2018

  25. [33]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    C. Schuhmann et al., “Laion-5b: An open large-scale dataset for training next generation image-text models”, Advances in Neural Information Processing Systems, vol. 35, pp. 25278–25294, 2022

    2022

  26. [34]

    Retrieval-based spatially adaptive normalization for semantic image synthesis,

    Y . Shi, X. Liu, Y . Wei, Z. Wu, and W. Zuo, “Retrieval-based spatially adaptive normalization for semantic image synthesis,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Jun. 2022

    2022

  27. [35]

    Learning to predict layout-to- image conditional convolutions for semantic image synthesis,

    X. Liu, G. Yin, J. Shao, X. Wang et al., “Learning to predict layout-to- image conditional convolutions for semantic image synthesis,” NeurIPS, vol. 32, 2019

    2019

  28. [36]

    Semantically multi-modal image synthesis,

    Z. Zhu, Z. Xu, A. You, and X. Bai, “Semantically multi-modal image synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 5467–5476

    2020

  29. [37]

    You only need adversarial supervision for semantic image synthesis,

    V . Sushko, E. Sch ¨onfeld, D. Zhang, J. Gall, B. Schiele, and A. Khoreva,“You only need adversarial supervision for semantic image synthesis,” arXiv preprint arXiv:2012.04781 , 2020

    2012 arXiv

  30. [38]

    Semantic image synthesis via diffusion models,

    W. Wang, J. Bao, W. Zhou, D. Chen, D. Chen, L. Yuan, and H. Li, “Semantic image synthesis via diffusion models,” arXiv preprint arXiv:2207.00050, 2022

    2022

  31. [39]

    Ranftl, K

    R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V . Koltun, ”Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer”, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 3, pp. 1623-1637, Aug. 2022

    2022

  32. [40]

    Detrs with collaborative hybrid as- signments training

    Z. Zong, G. Song, and Y . Liu, “Detrs with collaborative hybrid as- signments training”, in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 6748–6758

    2023

  33. [41]

    Extreme image compression using fine-tuned vqgans

    Q. Mao et al., “Extreme image compression using fine-tuned vqgans”, in 2024 Data Compression Conference (DCC) , 2024, pp. 203–212

    2024

  34. [42]

    Dm-gan: Dynamic memory generative adversarial networks for textto-image synthesis,

    Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. “Dm-gan: Dynamic memory generative adversarial networks for textto-image synthesis,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp 5802–5810

    2019

  35. [43]

    Cross-modal contrastive learning for text-to-image generation,

    Han Zhang, Jing Yu Koh, Jason Baldridge, Honglak Lee, and Yinfei Yang. “Cross-modal contrastive learning for text-to-image generation,” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 833–842

    2021

  36. [44]

    Df-gan: Deep fusion generative adversarial networks for text-to-image synthesis,

    Ming Tao, Hao Tang, Songsong Wu, Nicu Sebe, Xiao-Yuan Jing, Fei Wu, and Bingkun Bao. “ Df-gan: Deep fusion generative adversarial networks for text-to-image synthesis,” arXiv preprint arXiv:2008. 05865, vol. 2, no. 6, 2020

    2008

  37. [45]

    Text to image generation with semantic-spatial aware gan

    W. Liao, K. Hu, M. Y . Yang, and B. Rosenhahn, “Text to image generation with semantic-spatial aware gan”, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 18187–18196

    2022

  38. [46]

    Dse-gan: Dynamic semantic evolution generative adversarial network for text- to-image generation

    M. Huang, Z. Mao, P. Wang, Q. Wang, and Y . Zhang, “Dse-gan: Dynamic semantic evolution generative adversarial network for text- to-image generation”, in Proceedings of the 30th ACM International Conference on Multimedia , 2022, pp. 4345–4354

    2022

  39. [47]

    Vector quantized diffusion model for text-to-image synthesis

    S. Gu et al., “Vector quantized diffusion model for text-to-image synthesis”, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 10696–10706

    2022

  40. [48]

    Not all image regions matter: Masked vector quantization for autoregressive image genera- tion

    M. Huang, Z. Mao, Q. Wang, and Y . Zhang, “Not all image regions matter: Masked vector quantization for autoregressive image genera- tion”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2002–2011

    2023

  41. [49]

    Lapidoth, ”The performance of convolutional codes on the block erasure channel using various finite interleaving techniques,” in IEEE Transactions on Information Theory, vol

    A. Lapidoth, ”The performance of convolutional codes on the block erasure channel using various finite interleaving techniques,” in IEEE Transactions on Information Theory, vol. 40, no. 5, pp. 1459-1473, Sept. 1994

    1994

  42. [50]

    J. G. Apostolopoulos, ‘Reliable video communication over lossy packet networks using multiple state encoding and path diversity’, in Visual Communications and Image Processing 2001, 2000, vol. 4310, pp. 392–409

    2001

  43. [51]

    S. S. Hemami and R. M. Gray, ”Subband-coded image reconstruction for lossy packet networks,” in IEEE Transactions on Image Processing, vol. 6, no. 4, pp. 523-539, April 1997

    1997

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.