{"id":"2c5324c2-3145-4bb0-9673-869949e79d19","arxiv_id":"2504.21814","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A structured raster-scan prompt fed to GPT-4o can reconstruct images at ultra-low bitrates (around 0.001 bpp) with competitive perceptual quality versus trained generative codecs, though only demonstrated on 10 images.","lead":"This paper tests whether OpenAI's GPT-4o image generator, guided by a specially designed text prompt, can act as the decoder in a very low-bitrate image compression scheme, using either text alone or text plus a tiny low-resolution image. It reports that on 10 test images this beats several trained compression models on perceptual metrics, suggesting a future where compression is replaced by generation, though the evaluation is too small to be conclusive.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'competitive performance' claim rests on 3-run averages over 10 tuned DIV2K images with no error bars and an unpinned closed GPT-4o interface; the evaluation protocol is not yet sufficient to support the headline.","rationale":"Reading the paper in good faith, the method itself is a reasonable plug-and-play pipeline: an MLLM captions an image with a raster-scan prompt, the text is losslessly compressed, optionally an 8x-downsampled MS-ILLM image is transmitted, and GPT-4o regenerates the image. The contribution is prompt engineering, so the central quantitative claim must be that this untrained pipeline is competitive at ultra-low bitrates. That claim is exactly where the evidence is weakest. The evaluation uses 10 images, 3 runs, no error bars, and prompt hyperparameters selected on the same test set; moreover the decoder is a closed web service with no version pin. Each of these individually would be a limitation, and together they are load-bearing because the headline statement is a comparison of numbers. I found no internal contradiction in the method description, and the qualitative figures do illustrate the proposed effect (e.g., left-right orientation in Fig. 8), but qualitative illustrations cannot carry a competitive-performance claim against trained codecs. A held-out, larger, variance-reported, version-pinned rerun would settle whether the advantage is real or an artifact of selection and randomness. This matches the reader's weakest assumption, so no verdict change is needed: the paper should remain CONDITIONAL pending that check.","tokens_in":11253,"tokens_out":4766,"duration_ms":48202,"concrete_test":"Re-run the full pipeline on a held-out set of at least 100 images (e.g., Kodak plus DIV2K-val images 11-100), with prompt length fixed in advance from a separate validation split (e.g., 30 words for text-only, 15 words for text+image) and the GPT-4o model version pinned via the API rather than the web interface. Repeat each decoding at least 5 times and report means and 95% CIs for CLIP-IQA, MUSIQ, CLIPSIM and DISTS at matched bitrates, plus per-image deltas against MS-ILLM, Text+Sketch and PerCo. If the average gains and the Table 1 structure-prompt ablation fall inside the confidence intervals, the 'competitive' claim is not supported by the current evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Central claim: the proposed raster-scan prompt with GPT-4o generation 'achieved impressive performance' vs recent generative codecs at ultra-low bitrate, without training (Abstract; Sec. 5). This claim is carried by Fig. 3, Fig. 4, and Table 1. Those quantitative results use only the first 10 images of DIV2K-val, each averaged over 3 GPT-4o generations (Sec. 4.1), and the prompt-length variants (15/30/60/120 words) and text/image mix are selected after inspecting metrics on this same set (Sec. 4.3). No standard deviations, confidence intervals, or per-image breakdowns are reported. Because GPT-4o is accessed through a closed web interface (Sec. 4.1), the model version and sampling behavior are not pinned; the 3-run average is not reproducible as stated, and with 10 images the difference between, e.g., 'Ours' and 'Ours w/o structure raster-scan prompt' in Table 1 (CLIPSIM 0.901 vs 0.888; DISTS 0.289 vs 0.299) could easily lie within run-to-run and image-to-image variance. The smallest secure conclusion is therefore that the framework is promising; the load-bearing assumption that 10 tuned-image averages yield a fair, representative comparison against trained codecs at matched bitrates is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes an ultra-low-bitrate image compression framework based on GPT-4o image generation. It introduces a structural raster-scan prompt that describes image content in a top-to-bottom, left-to-right order with six consistency dimensions, and evaluates two paradigms: text-only and text-plus-low-resolution-image conditioning. The method is compared against MS-ILLM, Text+Sketch, and PerCo on the first ten DIV2K validation images using CLIP-IQA, MUSIQ, CLIPSIM, and DISTS. The authors report that their method achieves competitive performance at ultra-low bitrates without any additional training.","tokens_in":11536,"tokens_out":5213,"duration_ms":51215,"significance":"If the claims hold, the paper could open a new training-free direction for extreme image compression based on large multimodal generation models, and the raster-scan prompt mechanism is a simple, transferable design idea. The paper also provides ablation studies on prompt structure and length and qualitative results that illustrate consistency improvements. However, the evidence base is currently too small and uncontrolled to support the stated claims of 'impressive' and 'competitive' performance, and the use of a closed web interface raises reproducibility concerns.","major_comments":[{"comment":"The quantitative evaluation is based on only the first 10 images of the DIV2K validation set, with each result averaged over three GPT-4o generations. No standard deviations, confidence intervals, or per-image results are reported. Given that the central claim of 'competitive performance' is read directly from the curves in Fig. 3 and the numbers in Table 1, the reported differences, e.g., CLIPSIM 0.901 vs. 0.888 in the ablation, could easily lie within run-to-run and image-to-image variance. I request a larger evaluation set, more independent runs, and at least per-image breakdowns or error bars before the headline claim can be supported.","section":"§4.1 Datasets and Evaluation; Table 1"},{"comment":"The prompt length limits (30 words for text-only, 15 words for text+image in the final configuration) and the design of the raster-scan prompt's six dimensions appear to have been selected after inspecting metrics on the same 10 DIV2K images used for the final comparisons (Figs. 3, 4, and Table 1). This selection-on-the-test-set procedure risks overfitting the reported results to this particular set. The authors should either report results on a held-out set or explicitly describe the selection procedure and show that the chosen hyperparameters are not the result of cherry-picking.","section":"§4.3 Ablations; §4.1 Datasets and Evaluation"},{"comment":"GPT-4o is accessed through a closed web interface using automation scripts, so the model version, sampling parameters, and any server-side updates are not under the authors' control. Three-run averages over this stochastic, moving target are not reproducible, and the reported numbers may change if the service is updated. The manuscript should at minimum pin the exact access date, model version, and interface, and ideally evaluate with a locally runnable generative model (e.g., an open-weight model) to verify that the main claims do not depend on the specific proprietary service.","section":"§4.1 Implementation details"},{"comment":"The bitrate comparison is not tightly controlled: the points for 'Ours' and the baselines in Fig. 3 are at different rate points, and the ablation in Table 1 compares Ours (bpp 0.0008) with Ours w/o structure prompt (bpp 0.0007), which are not at identical rates. To fairly claim superior rate-distortion performance, the methods should be evaluated at matched anchor bitrates or compared via interpolation, and the rate variation across runs should be reported.","section":"§4.2 Main Results; Fig. 3"}],"minor_comments":[{"comment":"References [36] and [24] are cited as 'Mao' and 'Jia' without 'et al.'; please correct the citation format.","section":"§2.1 Related Works"},{"comment":"The caption uses 'PICS [29]' and 'PIC [29]' inconsistently; these should be 'Text+Sketch [29]' for clarity.","section":"Figure 5 caption"},{"comment":"Given the small sample size and stochastic generation, reporting bitrates to four decimal places (e.g., 0.0008 vs 0.0007) implies a precision the experiment does not support; report fewer decimal places or include standard deviations.","section":"Table 1"},{"comment":"The statement 'our method consistently achieves higher scores than existing approaches at similar or even lower bitrates' is too strong in the absence of error bars; it should be softened to 'tends to achieve' or accompanied by statistical significance testing.","section":"§4.2 Main Results"}],"recommendation":"major_revision","confidential_remarks":"The paper is borderline. The core idea is novel and timely, and the raster-scan prompt is a simple contribution that could be of interest to the community. However, the evaluation is too thin for the claimed performance: 10 images, no error bars, prompt selection on the test set, unmatched bitrate points, and reliance on a closed web interface undermine the quantitative conclusions. These issues are fixable with a more rigorous evaluation, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What's actually new here is the prompt design, not the paradigm. Text+Sketch and PerCo already showed that a frozen diffusion model plus text and a cheap visual cue can do ultra-low-bitrate compression. The paper's twist is a hand-built raster-scan prompt that forces an MLLM to describe a scene top-to-bottom, left-to-right across six consistency dimensions, then feeds that text (plus an optional heavily compressed image) to GPT-4o's image generator. That works on the qualitative examples, and the ablations show the structured prompt beats an unstructured one at roughly the same bitrate. The paper is also honest that it does not do any training, and it clearly states the closed web-interface setup. Credit where due: the idea is simple, the writing is readable, and the qualitative comparison in Fig. 5 is genuinely informative.\n\nNow the soft spots, and they are the ones the stress-test flags. The headline claim of \"impressive performance\" rests on metrics averaged over three generations on the first 10 DIV2K validation images. No standard deviations, no per-image breakdown, no significance test. The prompt lengths (15/30/60/120 words) and the text/image mix were chosen after looking at metrics on that same 10-image set, so the ablations are circular in the mild but real sense that the winners were selected on the test set. Bitrates for baselines are not matched rate points; Table 1 and Fig. 3 compare at whatever bpp each method happened to produce, which makes the rate-quality comparisons loose. And because GPT-4o is accessed through a web interface with an unpinned model version, the three-run average is not reproducible by anyone else. These are not fatal flaws in the concept, but they are load-bearing for the quantitative conclusion. The difference between the structured and unstructured prompt in Table 1 (CLIPSIM 0.901 vs 0.888) could easily sit inside run-to-run and image-to-image variance at n=10.\n\nThe related-work coverage is fair, and the authors do not oversell the novelty relative to Text+Sketch and PerCo. The citation pattern is normal for this area.\n\nWho is this for? Anyone working on generative compression or prompt engineering for closed foundation models will get a useful data point. It is a workshop-caliber result as it stands, but the concept deserves a real look.\n\nRecommendation: send to peer review. The evaluation is thin but the protocol is fixable: a proper held-out set (e.g., full DIV2K or Kodak), error bars, and ideally an open model or a pinned API snapshot. With those, the claim either holds or falls quickly. As is, I would not cite the quantitative numbers, but I would cite the prompt design if I worked on this problem.","headline":"A timely prompt-engineering demo for GPT-4o-based compression whose 'competitive' claim outruns the evidence: 10 tuned images, no error bars, closed web API.","tokens_in":12074,"tokens_out":955,"would_cite":false,"duration_ms":10931,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GPT-4o generation reconstructs images at 0.001 bpp without training","keywords":["image compression","GPT-4o image generation","multimodal compression","text-to-image generation","prompt engineering","ultra-low bitrate","raster-scan prompt","generative compression"],"falsifier":"Run the pipeline on the full DIV2K validation set or Kodak, with several repeated decodes and a fixed GPT-4o version, and compare at matched bitrates; if the raster-scan prompt no longer improves CLIPSIM/DISTS over a generic caption of equal length, or the metric advantages over Text+Sketch and PerCo disappear, the central claim is falsified.","tokens_in":11048,"feed_emoji":"🖼️","tokens_out":6778,"duration_ms":60100,"temperature":0.7,"pith_summary":"The paper asks why compress pixels that a powerful generative model could recreate from compact descriptors, and proposes an answer: use GPT-4o's image generation as a training-free decoder. It transmits either a text-only description or a text plus an extremely low-resolution image, and introduces a structural raster-scan prompt that forces the caption to list objects in top-to-bottom, left-to-right order with attention to six visual-consistency dimensions. The authors report that this prompt yields reconstructions that are competitive with or better than trained ultra-low-bitrate codecs on perceptual metrics such as CLIP-IQA and MUSIQ, while keeping semantic consistency measured by CLIPSIM, at bitrates near 0.001-0.006 bpp. The upshot is that the bottleneck of image compression may shift from pixel transform coding to prompt design and generative-model capability.","feed_headline":"GPT-4o generation reconstructs images at 0.001 bpp without training","feed_subtitle":"A raster-scan prompt lets GPT-4o rebuild 1024×1024 images with no training, rivaling tuned codecs.","key_machinery":"The central object is the structural raster-scan prompt, a captioning instruction that asks the multimodal language model to enumerate the main items of the image from top to bottom and left to right, then to refine the description along six named dimensions: feature correspondence, geometric consistency, photometric consistency, style consistency, semantic consistency, and structural integrity, while keeping the response within a fixed word limit. Its role is to convert spatial arrangement into textual form so that GPT-4o's decoder can reconstruct an image whose layout matches the original, countering the consistency failure that afflicts generic captions. The pipeline also leans on a visual-condition branch—a factor-8 downsampled image compressed by MS-ILLM—and Lempel-Ziv lossless coding of the text; together these carry the rate while GPT-4o supplies all pixel-level generation.","core_discovery":"In its own terms, the paper claims that a large multimodal model used for generation can replace most of the reconstruction machinery of a codec. The authors test two paradigms: textual coding, where only a losslessly compressed caption is sent, and multimodal coding, where the caption is joined by an image downsampled by a factor of eight and compressed with MS-ILLM. They claim the structural raster-scan prompt, which describes main items in spatial scan order and elaborates feature, geometric, photometric, style, semantic, and structural-integrity dimensions within a strict word budget, is what preserves layout and consistency. Their experiments on the first ten DIV2K validation images report higher CLIP-IQA and MUSIQ scores than MS-ILLM, Text+Sketch, and PerCo at comparable or lower bitrates, and best CLIPSIM among the compared methods, all with no training of the generative model. The paper's conclusion is that GPT-4o's generation capability is sufficient to bring competitive ultra-low-bitrate compression, so the field should consider generation-based coding as a viable paradigm.","pith_inferences":["A likely next test is whether the same prompting recipe transfers to open-weight models; if an open model matches GPT-4o, the paradigm becomes deployable without a closed API.","A fair rate-distortion comparison would count the compute or service cost of generation, not just transmitted bits; including that could change which operating points are actually advantageous.","The raster-scan ordering is a form of serialization, so the same idea may extend to diagrams, medical images, or video keyframes where spatial layout matters as much as semantics.","Re-running on full validation sets with model versions frozen is the natural stress test; the reported 10-image, 3-run averages are the paper's evidence but not a bound on the method."],"forward_implications":["If the claim holds, ultra-low-bitrate compression can be achieved without training or fine-tuning any pixel codec; the transmitted rate consists of a compressed caption plus an optional tiny image.","The structural raster-scan prompt is a transferable prompt-engineering mechanism: any text-to-image model that follows spatial ordering can use it to improve layout consistency.","Multimodal coding (short text plus low-resolution image) gives the best balance of perceptual quality and consistency, and increasing prompt length past a point degrades structural consistency.","Because GPT-4o decoding is stochastic, the same transmitted bitstream can produce several plausible reconstructions, and adding the visual condition reduces this randomness."],"supporting_citations":[{"why":"Supplies the GPT-4o image generation function that serves as the training-free decoder for both text-only and text+image paradigms.","marker":"[43]"},{"why":"Defines the Text+Sketch baseline and the usage of Lempel-Ziv coding and CLIPSIM that the paper adopts and compares against.","marker":"[29]"},{"why":"PerCo baseline: the trained diffusion-based ultra-low-bitrate codec whose bitrates and metrics the paper compares with.","marker":"[9]"},{"why":"MS-ILLM: provides the state-of-the-art perceptual codec used to compress the visual condition branch and serves as a baseline.","marker":"[42]"},{"why":"Supplies the DIV2K validation set, the evaluation data (first 10 images) for all experiments.","marker":"[2]"},{"why":"Provides the automation scripts through which the authors interact with the GPT-4o web interface for decoding.","marker":"[54]"},{"why":"Defines the CLIP embeddings used to compute CLIPSIM, the semantic-consistency metric in the comparisons.","marker":"[44]"},{"why":"Defines DISTS, the full-reference structural/texture similarity metric used to measure consistency.","marker":"[11]"},{"why":"Defines MUSIQ, one of the perceptual quality metrics where the method reports advantages.","marker":"[26]"},{"why":"Defines CLIP-IQA, the aesthetic quality metric where the method reports advantages.","marker":"[48]"}],"fun_headline_variants":["GPT-4o compresses images by generating them from text prompts","Raster-scan prompts let GPT-4o rebuild images at 0.001 bpp","Why encode when GPT-4o can generate? New compression paradigm","No-training ultra-low-bitrate image compression via GPT-4o","GPT-4o generates images from captions, rivaling trained codecs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's competitive-performance claim rests on three-run averages over the first ten DIV2K validation images, with the prompt variants tuned on that same set and GPT-4o accessed through a closed web interface whose version and behavior the authors do not control.","fun_headline_variants_meta":{"raw":{"variants":["GPT-4o compresses images by generating them from text prompts","Raster-scan prompts let GPT-4o rebuild images at 0.001 bpp","Why encode when GPT-4o can generate? New compression paradigm","No-training ultra-low-bitrate image compression via GPT-4o","GPT-4o generates images from captions, rivaling trained codecs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000269,"raw_usage":{"total_tokens":1668,"prompt_tokens":1035,"completion_tokens":633,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":534}},"tokens_in":651,"tokens_out":633,"duration_ms":6596,"temperature":1.0,"reasoning_tokens":534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:52:34.218664+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on the full DIV2K validation set or Kodak, with several repeated decodes and a fixed GPT-4o version, and compare at matched bitrates; if the raster-scan prompt no longer improves CLIPSIM/DISTS over a generic caption of equal length, or the metric advantages over Text+Sketch and PerCo disappear, the central claim is falsified.","supporting_citations":[{"cited_title":"Addendum to gpt-4o system card: 4o image gener- ation, 2025","cited_arxiv_id":null,"evidence_quote":"Supplies the GPT-4o image generation function that serves as the training-free decoder for both text-only and text+image paradigms."},{"cited_title":"Text + sketch: Image compression at ultra low rates","cited_arxiv_id":null,"evidence_quote":"Defines the Text+Sketch baseline and the usage of Lempel-Ziv coding and CLIPSIM that the paper adopts and compares against."},{"cited_title":"Towards image compression with per- fect realism at ultra-low bitrates","cited_arxiv_id":null,"evidence_quote":"PerCo baseline: the trained diffusion-based ultra-low-bitrate codec whose bitrates and metrics the paper compares with."},{"cited_title":"Improving statistical fi- delity for neural image compression with implicit local like- lihood models","cited_arxiv_id":null,"evidence_quote":"MS-ILLM: provides the state-of-the-art perceptual codec used to compress the visual condition branch and serves as a baseline."},{"cited_title":"Ntire 2017 challenge on single image super-resolution: Dataset and study","cited_arxiv_id":null,"evidence_quote":"Supplies the DIV2K validation set, the evaluation data (first 10 images) for all experiments."},{"cited_title":"Image quality assessment: Unifying structure and texture similarity","cited_arxiv_id":null,"evidence_quote":"Defines DISTS, the full-reference structural/texture similarity metric used to measure consistency."}],"review_version":1}