{"id":"b73971ae-45c0-4fc2-90bf-18da1d2f8256","arxiv_id":"2412.17839","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LaMI-GO sends partially masked visual code indices plus a text caption, and a pre-trained latent diffusion model fills in the masked indices to reconstruct the image at the receiver.","lead":"This paper introduces a new image-transmission method that sends only a compressed set of visual codes plus a short text description, and uses an AI diffusion model at the receiver to fill in the missing parts. It reports better image quality, lower bandwidth, and faster reconstruction than earlier goal-oriented communication systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline bandwidth and FID comparisons rest on an untested assumption that the frozen Paella model stays calibrated when unmasked tokens are held fixed and at the p=0.35 operating point, which the paper's own Section V-C concedes is limited without fine-tuning.","rationale":"The reader's weakest assumption correctly identifies the frozen Paella backbone as the load-bearing element: LaMI-GO's efficiency claim depends on Paella's out-of-the-box predictive distribution being reliable when unmasked tokens are held fixed and when masking/corruption patterns deviate from Paella's training-time random masking. I agree with that core concern. I sharpen it in two ways. First, the paper's own Section V-C contains an explicit limitation statement: the PRM model has limited capabilities at higher distortion rates without fine-tuning the diffusion model. This is not an external criticism; it is an internal concession that directly threatens the lowest-bandwidth row of Table VII, which uses p=0.35. If the frozen model is already limited at that masking probability, then the reported 8.50 KB / FID 27.23 point is not a validated no-retraining result. Second, the EBM policy, which is used in the showtime comparison of Table X, produces contiguous image-dependent masks that are structurally different from the random masks seen in Paella training; Table IV shows EBM's FID degrades sharply on Flickr and COCO-Stuff, so the same assumption fails in a regime the paper itself highlights. The attack is not that the method cannot work; the empirical trends suggest it does work for random masks at low p. The attack is that the central claim is stated as a general no-retraining bandwidth-quality advantage, while the evidence only covers a narrow, incompletely specified operating envelope, with no calibration diagnostics, no error bars, and no code to separate implementation artifacts from genuine properties of the method. Because these are concrete but correctable omissions, the appropriate verdict remains CONDITIONAL rather than ACCEPT or REJECT, so I leave the reader's verdict unchanged. I mark agreement as partial because I place more weight on the p=0.35 internal concession and on bandwidth accounting than the reader's stated weakest assumption, which emphasizes arbitrary masks and packet-drop conditions.","tokens_in":19123,"tokens_out":7627,"duration_ms":76829,"concrete_test":"Pick a held-out set of 200 Cityscapes images. For each image, run LaMI-GO with the random policy at p=0.25 and p=0.35 and also with corrupted received tokens (PER 10% with interleaving). Compare Paella's predicted multinomial distribution at the masked positions against the ground-truth tokens using log-loss and top-k accuracy, and re-run full reconstruction 10 times with different random seeds to report FID/LPIPS mean with 95% confidence intervals. If at p=0.35 the predicted log-loss is not better than the codebook prior, or if the FID confidence interval overlaps Diff-GO+'s reported 58.20, the no-retraining/bandwidth claim is not supported at the headline operating point. Separately, recompute the Table VII bandwidth values with an explicit bit budget for codebook indices, mask overhead (including EBM's message-dependent Me), and compressed text.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; Tables I and VII) is that LaMI-GO with a frozen Paella backbone and frozen VQGAN codebook outperforms prior GO-COM systems in perceptual quality, downstream accuracy, bandwidth, and recovery time. The load-bearing condition is that Paella's softmax predictions in Eq. (13), trained on random masking schedules, remain calibrated for the exact inputs LaMI-GO feeds them: a fixed set of unmasked codebook indices preserved at every step via Eq. (12), masked positions filled by multinomial samples, and, in the noise experiments, positions whose 'known' values are actually corrupted by bit flips or packet erasure. The paper provides no calibration check, no confidence intervals, and no code. More importantly, Section V-C concedes that 'the PRM model has limited capabilities to combat higher distortion rates without fine-tuning the diffusion model,' but the lowest-bandwidth headline point in Table VII (8.50 KB, FID 27.23) uses p=0.35, exactly the higher-distortion regime. If Paella's predictions are not calibrated for p=0.35 random masks or for corrupted received tokens, the bandwidth saving is bought at an unquantified reconstruction cost, and the claim that no retraining is needed is unsupported at the operating point that makes the strongest efficiency case. The same concern is sharper for EBM (Section IV-D.3), whose contiguous, image-dependent masks (Eq. 17) are far from Paella's random-mask training distribution; Table IV shows EBM FID collapses on Flickr and COCO-Stuff, yet EBM is used in Table X without a calibration caveat.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LaMI-GO, a goal-oriented communication system that compresses an image into VQGAN codebook indices, masks a fraction of the indices under one of three policies (pseudo-random, pre-determined, and entropy-based), transmits the remaining indices along with an automatically generated caption, and reconstructs the image at the receiver by iterative latent diffusion using a frozen Paella model. The central claim is that the proposed latent mixture integration rule in Eqs. (12)-(13), which preserves unmasked indices at every diffusion step while multinomial-sampling the masked positions, enables a pre-trained text-to-image diffusion model to achieve strong perceptual quality, downstream-task accuracy, bandwidth savings, and reduced showtime without retraining the generative backbone. Supporting experiments on Cityscapes, Flickr, COCO-Stuff, and COCO compare LaMI-GO with prior GO-COM systems (Diff-GO, Diff-GO+, GESCO) and with generative text-to-image baselines.","tokens_in":19414,"tokens_out":6260,"duration_ms":56844,"significance":"If the central claim is established, the contribution is practically valuable: it decouples a GO-COM system from the need to train or fine-tune a generative backbone, reuses a public latent diffusion prior, and achieves large bandwidth reductions with few diffusion steps. The ablation in Fig. 13 is a clear strength: it demonstrates that without latent mixture integration the receiver drifts toward a generic text-conditional image, whereas with it the recovered image remains close to the transmitted source. The paper also provides a concrete, falsifiable recipe (masking policies plus an integration rule) and evaluates several datasets and downstream tasks. However, some load-bearing comparisons are not matched in information content, and the calibration of the frozen Paella model at the operating points that deliver the largest bandwidth savings is not quantified; these issues should be resolved before the headline claims are fully supported.","major_comments":[{"comment":"The paper's own text concedes that \"the PRM model has limited capabilities to combat higher distortion rates without fine-tuning the diffusion model\" (Section V-C), yet the lowest-bandwidth headline number in Table VII is LaMI-GO with PRM p=0.35 (8.50 KB, FID 27.23), and the p=0.25 row (9.39 KB, FID 22.72) is the second headline. The no-retraining claim is therefore demonstrated precisely in the regime the text flags as degraded, and no calibration check, error bar, or sensitivity analysis across p is reported. Please quantify the change in FID/LPIPS as p varies, with uncertainty estimates over images or splits, and either support the p=0.35 operating point with a calibration analysis or restrict the no-retraining claim to the p=0.25 configuration.","section":"Section V-C, Tables VII, Figs. 8-9"},{"comment":"Table II compares LaMI-GO FID scores against generative models (DMGAN, XMCGAN, DFGAN, SSA-GAN, DSE-GAN, VQ-Diffusion, VQ-GAN, Stackformer) that are conditioned only on text. LaMI-GO additionally receives a large set of unmasked, position-aware codebook indices for every image, so the information available to the two kinds of systems is not matched, and the table cannot support the statement that LaMI-GO is \"flexible, competitive, and better\" as a generative model. Please remove this comparison from the central claims or convert it into a clearly labeled side-information upper bound, and base the headline comparison on GO-COM systems with comparable conditioning information.","section":"Section V-A, Table II"},{"comment":"The method rests on the assumption that the frozen Paella model f_phi remains a calibrated predictor for index distributions under LaMI-GO's input distribution (fixed unmasked positions, multinomial-sampled masked positions, and, in the channel experiments, corrupted received tokens). No calibration diagnostic is reported. The EBM results in Table IV themselves indicate distribution shift: at the same step budget, EBM degrades FID on Flickr from 35.62 (random) to 54.15 and on COCO-Stuff from 10.37 to 29.31. Please report a calibration measure (for example, expected calibration error of the predicted index probabilities against the transmitted indices, separately for PRM, PDM, and EBM) or explicitly restrict the no-retraining claim to the policies for which calibration is verified.","section":"Section IV-C, Eq. (13); Section IV-D.3, Table IV"},{"comment":"The bandwidth comparison in Table VII does not provide a complete bit budget. The transmitted object is mc = compress(ˆIe, cT) (Algorithm 1), but the table does not state how many bits are spent on the caption, on the mask description (PRNG seed for PRM, mask index for PDM, or message-dependent masks for EBM), or on channel coding, nor whether the same entropy coding is applied to all compared systems. Because the central claim is spectrum efficiency, please give the exact per-component bit counts and state the assumptions about overhead for every method in the table.","section":"Section V-E, Table VII"}],"minor_comments":[{"comment":"The piecewise definition of \\hat I_{p,i} leaves i=tau and i=T undefined (the conditions are i<tau and tau<i<T); please clarify the endpoint handling.","section":"Section IV-C, Eq. (13)"},{"comment":"In Algorithm 2 the condition \"If t < tau\" should be \"If i < tau\" (or otherwise t should be defined before use), and the equation for \\hat I_1 uses (1a\\times b) while Eq. (12) uses (1h'\\times w'); please align the notation.","section":"Algorithm 2"},{"comment":"The entropy-based masking policy depends on a threshold eta and on \"selecting the best n candidate indices,\" but the paper never states how eta or n is chosen or how the mask overhead is transmitted; please specify the procedure and include these parameters in the bandwidth accounting.","section":"Section IV-D.3"},{"comment":"The abstract contains a grammatical error (\"common-feature codebookthe receiver side\") and, more importantly, says the system \"trains\" a codebook at the receiver, while Section III-A states that a pre-trained codebook is shared as a one-time configuration; please correct the wording.","section":"Abstract, Section III-A"},{"comment":"References [7] and [22] cite the same work (Esser et al., CVPR 2021, \"Taming Transformers for High-Resolution Image Synthesis\") as if they were different publications; please consolidate or disambiguate them.","section":"References [7] and [22]"},{"comment":"The method name is spelled inconsistently (\"Diff-Go+\" in the table body and \"Diff-GO+\" in the caption and text); please standardize the spelling.","section":"Table V"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is from the same group that proposed Diff-GO and Diff-GO+, and the authors compare against their own earlier systems extensively. The comparisons appear direct, but given the self-citation pattern, it would be advisable to verify that the baselines are run under identical conditions (same dataset split, same bandwidth overhead assumptions, same evaluation code). The paper fits the journal's scope well if the calibration analysis and bit-budget clarification are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the thing about LaMI-GO: the core trick is real. They take a pre-trained Paella discrete diffusion model and a VQGAN codebook, transmit a subset of codebook indices plus a text caption, and at the receiver they freeze the received indices while letting the diffusion model fill the masked positions. That latent mixture integration (Eq. 12-13) is not in the cited GO-COM literature, and the ablation in Fig. 13 shows it matters: without it you get generic text-to-image samples; with it you preserve the source content. That is a genuine, useful idea, and the bandwidth numbers are large: 9.39 KB per Cityscapes image at FID 22.72 versus 13.5-87 KB for prior GO-COM systems. If those numbers hold, it's a real step forward.\n\nThe paper also deserves credit for being honest about its own method. Tables III and IV show EBM masking collapses on Flickr and COCO-Stuff, and Section V-C concedes that PRM degrades at p=0.35 without fine-tuning. That kind of reporting is a good sign.\n\nNow the soft spots, in descending order. First, Table II compares LaMI-GO against text-only generative models like DMGAN and Stackformer. Those models receive no source image information at all; LaMI-GO receives masked source indices. That is not a matched comparison, and it flatters the method. Second, there are no error bars or confidence intervals anywhere, and no code, so the margin over Diff-GO+ could be smaller with variance. Third, the calibration worry in the stress-test note is real: Paella was trained on random masking schedules, and LaMI-GO feeds it fixed unmasked tokens at p=0.35, exactly the regime the paper admits is limited. The paper doesn't check whether Paella's predictions stay calibrated when unmasked indices are held fixed, nor for contiguous EBM masks. That doesn't sink the method, but it means the headline bandwidth savings come with an unquantified reconstruction cost at the lowest-bandwidth operating points. Fourth, implementation details like the EBM threshold and exactly how masked indices are handled before entropy coding are underspecified.\n\nOverall, this is a plausible engineering system with a new idea and a big claimed gain. The flaws are correctable: rerun Table II with matched baselines, add error bars, and test calibration on p=0.35 and EBM. It deserves a serious referee, not a desk reject.\n\nFor you: if you work on GO-COM or generative image coding, read it and consider citing the latent mixture idea. It's also a good reading-group paper for debating what counts as a fair baseline in this area.","headline":"A genuinely new latent mixture integration rule lets a frozen diffusion model fill masked codebook indices with large bandwidth savings, but the headline comparisons and calibration questions need work before the numbers can be fully trusted.","tokens_in":20019,"tokens_out":2520,"would_cite":true,"duration_ms":22530,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LaMI-GO, which sends masked image tokens plus a text caption and regenerates the rest with a pre-trained latent diffusion model, improves quality while cutting bandwidth roughly tenfold versus GO-COM.","keywords":["goal-oriented communications","semantic communications","latent diffusion","vector quantization","VQGAN","image reconstruction","packet loss","bandwidth efficiency"],"falsifier":"Measure the pre-trained diffusion model's per-token prediction accuracy on Cityscapes images under the three masking policies at masking rates 25% and 35%, especially for contiguous blocks as in entropy-based masking; if accuracy on masked positions falls to the rate of the most common codebook class or degrades sharply when received tokens are corrupted, the claimed bandwidth savings would carry an unmeasured reconstruction cost.","tokens_in":18908,"feed_emoji":"📡","tokens_out":15190,"duration_ms":120654,"temperature":0.7,"pith_summary":"Goal-oriented communication (GO-COM) aims to deliver what a receiver needs for a task rather than reproducing every bit of a source. This paper proposes LaMI-GO, a system that quantizes an image into codebook indices, transmits a subset of those indices together with a compressed text caption, and reconstructs the image on the receiver side with a pre-trained latent diffusion model. The paper's central claim is that this design outperforms prior GO-COM systems on perceptual quality, downstream-task accuracy, bandwidth, and recovery time without retraining the generative backbone. In the Cityscapes comparisons, LaMI-GO reached FID 22.72 at 9.39 KB per image, while the diffusion-based competitor Diff-GO+ reached FID 58.20 at 87.04 KB, and LaMI-GO needed only 8 diffusion steps where earlier methods used 1000.","feed_headline":"Sending 9KB beats sending 87KB on image quality","feed_subtitle":"A pre-trained latent diffusion model fills masked image tokens from text guidance, cutting bandwidth and recovery time.","key_machinery":"The central mechanism is the latent mixture integration update, $\\hat{I}_i = (\\mathbf{1}-M) \\odot \\hat{I}_e + M \\odot \\hat{I}_{p,i}$, where $M$ marks the positions masked by the transmitter, $\\hat{I}_e$ is the received index matrix, and $\\hat{I}_{p,i}$ is a predicted index matrix for the masked positions. At the first step $\\hat{I}_{p,1}$ is sampled randomly from the shared codebook $W$; in later steps it is drawn by multinomial sampling from the softmax output of the pre-trained latent diffusion model, possibly after a re-noising operation for early iterations. This mechanism keeps transmitted indices intact while letting the diffusion model fill in the masked ones, which is what allows the system to reuse a pre-trained generative backbone unchanged.","core_discovery":"The paper claims that a goal-oriented communication link can be built by encoding an image into a VQGAN latent space, transmitting only a masked subset of the resulting codebook indices plus a text description, and using the pre-trained Paella latent diffusion model to regenerate the omitted indices. The core discovery is the latent mixture integration update, which holds the received indices fixed while iteratively replacing masked positions with model predictions, so a diffusion model trained for random masking behaves as a calibrated predictor for the communication masks without retraining or fine-tuning. On Cityscapes, LaMI-GO reports FID 29.84 with 8 diffusion steps versus FID 55.85 to 96.41 for prior GO-COM methods with 1000 steps, and at 9.39 KB per image it reports FID 22.72 versus Diff-GO+ at 87.04 KB with FID 58.20.","pith_inferences":["Inference: the latent mixture integration rule only requires a model that predicts codebook indices from partial masks, so the same recipe may transfer to other discrete latent diffusion or VQ models, making the contribution a general mechanism rather than a single-system result.","Inference: the three masking policies suggest an adaptive policy that chooses which tokens to drop based on content entropy and expected channel conditions; the paper does not test such an adaptive policy, and it could improve the bandwidth-quality tradeoff further.","Inference: the comparisons against classical JPEG at similar bandwidth are not computation-free, because LaMI-GO moves the cost to a receiver that must run a diffusion model; how that tradeoff is valued depends on the receiver's hardware budget.","Inference: the paper's reported savings assume the pre-trained model's predictions remain well-calibrated for contiguous entropy-based masks and corrupted received positions; a direct test of per-token prediction accuracy under those masks would quantify how much of the bandwidth gain is real."],"forward_implications":["A GO-COM system can reuse a pre-trained latent diffusion model and VQGAN codebook as-is, so deployers do not need to train or fine-tune a diffusion backbone for each new dataset or task.","Bandwidth becomes continuously tunable through the masking probability and policy: on Cityscapes, LaMI-GO sends about 9.39 KB per image at 25% masking and about 8.50 KB at 35% masking, with measured FID cost from 22.72 to 27.23.","Receiver recovery time drops from the order of 1000 diffusion steps used by prior diffusion GO-COM systems to 8 to 200 steps, with a reported showtime of about 1.29 seconds at 4/8 steps.","Reconstruction starts from real image tokens rather than random noise, so the same recovered image supports several downstream tasks; depth-estimation RMSE improves from 6.58 (Diff-GO+) to 5.59 (LaMI-GO), and object-detection mAP reaches about 64% on tested datasets.","With block interleaving, the system maintains acceptable perceptual quality up to about 20% packet loss and remains robust at practical bit-error rates, since lost or corrupted tokens can be regenerated by the diffusion model."],"supporting_citations":[{"why":"Supplies the pre-trained latent diffusion model (Paella) that the receiver uses to regenerate masked codebook indices.","marker":"[24]"},{"why":"Supplies the VQGAN encoder, decoder, and shared codebook used to map images to and from quantized latent tokens.","marker":"[7]"},{"why":"Baseline diffusion GO-COM system that LaMI-GO compares against for perceptual quality, bandwidth, and recovery time.","marker":"[3]"},{"why":"Baseline Diff-GO+ system that provides the closest strong comparison on Cityscapes for FID, bandwidth, downstream depth estimation, and showtime.","marker":"[4]"},{"why":"Baseline GESCO system that LaMI-GO compares against and whose additive-noise channel model the paper argues is mismatched to packet networks.","marker":"[8]"},{"why":"Origin of vector-quantized codebook learning that justifies representing latents as shared codeword indices.","marker":"[13]"},{"why":"Supplies the lightweight image captioning that produces the text condition transmitted alongside the codebook indices.","marker":"[23]"},{"why":"Large-scale image-text training corpus used to pre-train the generative backbone, supporting the claim that no task-specific retraining is needed.","marker":"[33]"}],"fun_headline_variants":["Latent diffusion repairs masked image tokens for goal-oriented comms","9KB image link beats 87KB via latent diffusion inpainting","LaMI-GO: Pre-trained diffusion regenerates masked image tokens","Zero-tuning diffusion model fills masked image tokens for GO-COM","Shared codebook and latent diffusion cut bandwidth 9-fold for GO-COM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pre-trained diffusion model, trained with its own random masking schedule, gives reliable predictions for LaMI-GO's arbitrary masking policies and for corrupted received positions; the paper itself notes in Section V-C that the random-mask model \"has limited capabilities to combat higher distortion rates without fine-tuning the diffusion model.\"","fun_headline_variants_meta":{"raw":{"variants":["Latent diffusion repairs masked image tokens for goal-oriented comms","9KB image link beats 87KB via latent diffusion inpainting","LaMI-GO: Pre-trained diffusion regenerates masked image tokens","Zero-tuning diffusion model fills masked image tokens for GO-COM","Shared codebook and latent diffusion cut bandwidth 9-fold for GO-COM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000742,"raw_usage":{"total_tokens":3302,"prompt_tokens":927,"completion_tokens":2375,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":2283}},"tokens_in":543,"tokens_out":2375,"duration_ms":16668,"temperature":1.0,"reasoning_tokens":2283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:00:47.175426+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the pre-trained diffusion model's per-token prediction accuracy on Cityscapes images under the three masking policies at masking rates 25% and 35%, especially for contiguous blocks as in entropy-based masking; if accuracy on masked positions falls to the rate of the most common codebook class or degrades sharply when received tokens are corrupted, the claimed bandwidth savings would carry an unmeasured reconstruction cost.","supporting_citations":[{"cited_title":"A novel sampling scheme for text-and image-conditional image synthesis in quantized latent spaces","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained latent diffusion model (Paella) that the receiver uses to regenerate masked codebook indices."},{"cited_title":"Taming transformers for high- resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Supplies the VQGAN encoder, decoder, and shared codebook used to map images to and from quantized latent tokens."},{"cited_title":"GO+: An Efficient Diffusion Goal-Oriented Communication System with Local Feedback,","cited_arxiv_id":null,"evidence_quote":"Baseline Diff-GO+ system that provides the closest strong comparison on Cityscapes for FID, bandwidth, downstream depth estimation, and showtime."},{"cited_title":"Neural discrete representation learning,","cited_arxiv_id":null,"evidence_quote":"Origin of vector-quantized codebook learning that justifies representing latents as shared codeword indices."},{"cited_title":"Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models","cited_arxiv_id":null,"evidence_quote":"Supplies the lightweight image captioning that produces the text condition transmitted alongside the codebook indices."},{"cited_title":"Laion-5b: An open large-scale dataset for training next generation image-text models","cited_arxiv_id":null,"evidence_quote":"Large-scale image-text training corpus used to pre-train the generative backbone, supporting the claim that no task-specific retraining is needed."}],"review_version":1}