{"id":"a02228b7-1a45-4a3b-a443-d3b20a3376b1","arxiv_id":"2509.08913","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A user-query-guided image semantic coding system using frozen CLIP features and a LLaVA-based training loss improves answer match rate on held-out animal images by up to 34% over a random-initialized backbone.","lead":"This paper proposes UO-ISC, an image semantic coding framework that takes a user's text query as intent and transmits only CLIP-extracted features relevant to that query, reconstructing the image at the receiver. It reports zero-shot gains on animal images from the VQA dataset, using a LLaVA-based relevance loss and answer match rate.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation is circular: Luser (Eq. 11) trains LLaVA hidden-state similarity while the answer match rate is scored by LLaVA-generated answers, so reported gains may not reflect human intent.","rationale":"The reader's weakest assumption identifies the circular use of LLaVA in both the training loss and the evaluation metric, and I find that to be the most load-bearing concern. The paper's central claim explicitly depends on the answer match rate as a measure of user-intent preservation; if that measure is not anchored to an external notion of intent, the quantitative gains are uninterpretable. The ablation issue compounds this: the 34% improvement attributed to the large VLM backbone is not actually isolated, since the comparison model also lacks text alignment. These are internal consistency issues rather than mere disagreement with consensus, and they directly affect the validity of the strongest claim. The proposed test—independent judging—would settle whether the reported gains reflect genuine intent preservation or LLaVA-specific overfitting. I do not think the paper should be rejected outright; the architecture is coherent and the idea has merit, but the evaluation needs strengthening. Hence the conditional verdict remains appropriate, and my read does not change the reader's verdict.","tokens_in":9438,"tokens_out":3627,"duration_ms":40770,"concrete_test":"Re-run the zero-shot animal-subset evaluation using an independent judge—either human raters or a different, non-LLaVA VQA model (e.g., InstructBLIP)—to compute answer match rate between original and reconstructed images. If the relative ordering and effect sizes (5%, 34%, 4.8%) do not persist, the current metric has been gamed by the training loss and the claim of generalized user-intent preservation is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that UO-ISC preserves user intent is measured by an exact-match rate between LLaVA-generated answers on the original and reconstructed images, while the training objective Luser (Eq. 11) explicitly minimizes the cosine distance between LLaVA hidden states of those same two images. This creates a self-consistent loop: the model is optimized against the same VLM that later judges it. Because nanoLLaVA is not independently validated against human judgments of intent, the reported improvements (+5% text alignment, +34% VLM backbone, +4.8% over query-aware baseline) could reflect overfitting to LLaVA's internal geometry rather than genuine preservation of query-relevant semantics. A concrete example would be an image where LLaVA's hidden states are close but its answers are coincidentally matched, or where a human would disagree with LLaVA's notion of relevance. Additionally, the ablation for the pretrained VLM is confounded: the model 'without text alignment and pretrained VLM' differs from UO-ISC in both text alignment and pretrained initialization, so the 34% cannot be attributed to the large VLM backbone alone. The zero-shot split is also a single within-dataset category split with no error bars. Thus the evaluation is not sufficient to support the headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UO-ISC, a user-oriented image semantic coding framework. The transmitter extracts CLIP image features and aligns them with a user-provided text query via FiLM-conditioned projections, then quantizes the aligned features segment-wise and transmits discrete indices over a fading channel. The receiver reconstructs the image through a decoder with skip connections. Training uses an L1 loss, a quantization loss, a user-intent relevance loss (Eq. 11) defined as the cosine distance between LLaVA hidden states of the original and reconstructed images given the same query, and, in a second phase, an adversarial loss. The framework is evaluated on a zero-shot animal subset of VQA, reporting answer match rate and user-intent relevance loss versus SNR, plus ablations and a symbol-efficiency comparison with JPEG2000.","tokens_in":9746,"tokens_out":4794,"duration_ms":55607,"significance":"If the results hold, the framework would be a practical way to inject user intent into image semantic coding while leveraging pretrained vision-language models for zero-shot generalization. The architecture is plausible and the use of frozen CLIP with learnable projections and FiLM conditioning is well motivated. However, the evaluation has a load-bearing self-reference issue: the training objective (Eq. 11) and the main metric (LLaVA answer match rate) both rely on the same nanoLLaVA model. Without an independent human or cross-model validation of user-intent preservation, the reported gains may reflect optimization toward LLaVA's internal geometry rather than genuine semantic alignment. The single split and absence of error bars further weaken the empirical support. The contribution is potentially useful, but the evidence presented is not yet sufficient to support the central claim.","major_comments":[{"comment":"The evaluation metric is self-referential. The user-intent loss Luser in Eq. (11) minimizes the cosine distance between nanoLLaVA hidden states of the original and reconstructed images, while the answer match rate is computed by asking the same nanoLLaVA (or its LLaVA variant) to generate answers from both images. Since the generated answers depend directly on those hidden states, optimizing Eq. (11) can inflate the answer match rate without actually preserving human-judged intent. The paper does not validate nanoLLaVA against human annotations or a different VLM, so the central claim of 'user-intent preservation' is not established. Please add an independent evaluation, e.g., human raters or a different pretrained VLM, and report agreement between nanoLLaVA and human judgments.","section":"Section IV and Eq. (11)"},{"comment":"The experimental evidence rests on a single zero-shot split (train on non-animal VQA samples, test on animal samples) with no error bars, no multiple seeds, and no statistical significance tests. The curves in Fig. 4 show differences of a few percent, but without variance information we cannot assess whether the reported 5%, 34%, and 4.8% improvements are meaningful. Moreover, the text 'improves the answer match rate by 5% through text alignment and by 34% by using a large VLM backbone' is ambiguous. If the 34% comes from comparing 'UO-ISC without text alignment' with 'UO-ISC without text alignment and pretrained VLM', the isolation is valid but should be stated explicitly; if it instead compares the full UO-ISC with the random-initialized baseline, the contribution of the pretrained VLM is conflated with text alignment and Luser. Please clarify and provide multiple runs with confidence","section":"Section IV, ablation and statistics"},{"comment":"The paper claims to outperform the 'state-of-the-art query-aware image semantic coding' but compares against a single method, reference [7], which is an arXiv preprint not established as state of the art. The JPEG2000 baseline is not query-aware. To support the SOTA claim, please either expand the comparison to include other query-aware semantic coding methods or temper the claim to 'the compared baseline'.","section":"Section IV, baseline selection"}],"minor_comments":[{"comment":"The segment indexing uses '(l−1)L' in the subscript, but the segment length is N_L, not L. It should be '(l−1)N_L + 1' etc.","section":"Eq. (6)"},{"comment":"The argument order of Luser is inconsistent: Eq. (11) defines Luser(Y, X_I; x_T) while Eq. (13) calls Luser(X_I, Y; x_T). Please unify.","section":"Eq. (11) and Eq. (13)"},{"comment":"The footnote says 'nanoLLaV A' with an extra space due to a line break. Please use the correct model name consistently.","section":"Section IV"},{"comment":"The number of transmitted symbols per image is not fully defined. Please specify whether it includes LDPC coding overhead, and how the symbol count for JPEG2000 is computed.","section":"Section IV, Fig. 6"},{"comment":"The paper assumes LDPC coding and modulation but does not discuss residual channel errors after decoding. In a fading channel with noise, the received symbol vector may contain errors that affect the decoded indices. Please clarify whether the evaluation accounts for residual errors or assumes error-free decoding after LDPC.","section":"Section II, channel model"}],"recommendation":"major_revision","confidential_remarks":"The architecture and training formulation are reasonable, and the paper is within scope. The main weakness is the circular evaluation: the same LLaVA model is used in the training loss and as the answer-match judge, with no independent validation. This is fixable by adding human evaluation or a different VLM as the judge, and by reporting error bars over seeds. The ablation concern raised in the stress-test is less severe than it appears, since the two no-text-alignment baselines differ only in pretrained vs. random CLIP initialization, but the text is ambiguous and should be clarified. If the authors address the evaluation independence and statistical rigor, the paper may become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent engineering paper with a coherent architecture, but its headline results are not trustworthy because the evaluation metric is defined by the same model used in the training loss. You should read it for the system design, not for the numbers.\n\nWhat's new and good: the framework combines frozen CLIP features, FiLM-based text conditioning, segment-wise vector quantization, a U-Net-like decoder, and a LLaVA-based user-intent loss. It's a legitimate extension of query-aware semantic coding, and the zero-shot evaluation on unseen animal categories is a sensible step toward generalization. The two-phase training with adversarial loss is standard but properly implemented. The math is clean and internally consistent. The efficiency comparison (82% fewer symbols than JPEG2000) is useful, if not surprising from a semantic coder.\n\nThe soft spot is load-bearing. The user-intent loss in Eq. (11) minimizes the cosine distance between nanoLLaVA hidden states of the original and reconstructed images given the same query. The answer match rate in Section IV is then computed by asking the same nanoLLaVA to answer the query for both images and checking exact match. The model is literally trained to make LLaVA's internal representations closer, so a higher match rate on LLaVA-generated answers is expected even if no human would agree. Without an independent judge—human raters, a different VQA model, or at least a behavioral task—the +5% and +34% gains do not show that user intent is preserved. The paper does not acknowledge this circularity at all.\n\nThe ablation is also confounded: the \"without text alignment and pretrained VLM\" model differs from UO-ISC in both text alignment and CLIP initialization, so the 34% cannot be attributed to the VLM backbone alone. There are no error bars or multiple seeds, and the zero-shot test is a single within-dataset category split. These are fixable, but they need to be fixed before the claims can be accepted.\n\nAll that said, this is not a desk-reject. The idea of using frozen VLMs for generalized query-aware coding is worth exploring, and the system is a reasonable baseline for future work. The right audience is the semantic communication community, and the paper deserves a serious referee who can push for a proper evaluation.\n\nMy recommendation: send it to peer review, but expect a major revision. Insist on an independent metric, disentangled ablations, seeds, and code/data release. If those are addressed, the paper could become a solid contribution.","headline":"A plausible query-aware image semantic coding system whose evaluation is circular (LLaVA defines both the loss and the metric); the architecture is worth a look, but the numbers don't yet support the claims.","tokens_in":10236,"tokens_out":2411,"would_cite":false,"duration_ms":30046,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"User-intent image coding generalizes to unseen objects, lifting answer match by 34%","keywords":["semantic communication","user intent","image semantic coding","vision-language model","CLIP","LLaVA","zero-shot generalization","vector quantization"],"falsifier":"Run a human study in which raters are shown original and reconstructed images with the same query and asked whether the reconstruction preserves the queried content, then compare human agreement rates with the LLaVA-based answer match rates; if human gains are substantially smaller than the 34% and 5% improvements, the central claim of user-intent preservation is not supported.","tokens_in":9300,"feed_emoji":"🖼️","tokens_out":1701,"duration_ms":23273,"temperature":0.7,"pith_summary":"The paper proposes a user-oriented image semantic coding (UO-ISC) framework that transmits only the features relevant to a user's text query, rather than the whole image. By anchoring the encoder with a pretrained vision-language model (CLIP) and training with a query-relevance loss from another vision-language model (LLaVA), the system learns to prioritize intent-relevant content and generalize to object categories never seen in training. On zero-shot tests with animal images excluded during training, the framework improves the exact-answer match rate by 34% over a non-pretrained variant, adds 5% through text alignment, and beats a query-aware semantic-coding baseline by 4.8% at 20 dB SNR. The paper's central claim is that pretrained vision-language backbones make user-intent semantic coding both effective and generalizable.","feed_headline":"Query-guided image coding beats baselines on unseen objects","feed_subtitle":"A vision-language backbone lets transmitters send only intent-relevant features, lifting answer match by 34% at high SNR.","key_machinery":"The central mechanism is a segment-wise vector quantization pipeline applied to CLIP features that are text-conditioned through Feature-wise Linear Modulation (FiLM). The text query and image are projected into a shared embedding space; the text features generate scale and shift matrices that modulate the image features before quantization into discrete codewords. A user-intent relevance loss, computed as the cosine distance between LLaVA hidden states of the original and reconstructed images under the same query, is added to ℓ1, quantization, and adversarial losses in a two-phase training procedure. This machinery lets the transmitter send a compact sequence of integer indices that preserve","core_discovery":"The paper discovers that integrating a frozen, pretrained VLM (CLIP) as the image/text feature extractor makes user-intent-driven semantic image coding generalize to unseen object categories, while a separately defined user-intent relevance loss (computed by LLaVA) aligns the reconstruction with the user's query. In zero-shot evaluations on animal images absent from the VQA training set, the proposed UO-ISC framework achieves the lowest user-intent relevance loss and the highest answer match rate among four baselines, including a state-of-the-art LLM-based query-aware codec. The improvement is attributed to the pretrained backbone's transferable semantic representations and to the FiLM-based","pith_inferences":["Since the same LLaVA model is used for both the training loss and the evaluation metric, the reported answer-match gains may partly reflect the model learning to satisfy LLaVA's specific hidden-state geometry rather than genuine semantic preservation; a human-evaluation study or a different VLM as judge would clarify this.","The approach could be extended to multi-query or dynamically changing user intent, where the transmitter would need to prioritize content relevant to multiple simultaneous queries or update the transmitted features as intent shifts.","The segment-wise codebook design creates an explicit rate-quality trade-off (shorter segments improve relevance but increase symbol count), suggesting a rate-intent-relevance optimization problem that the paper leaves open.","If the user-intent relevance loss is robust across different VLM backbones, the methodology could also apply to video or 3D scene transmission, though the paper only demonstrates still images."],"forward_implications":["If the claim holds, wireless image transmission can become intent-adaptive: the same source image can be encoded differently for different user queries without retraining the backbone.","The zero-shot results suggest that semantic coding systems can be deployed on out-of-distribution imagery by leveraging pretrained VLMs, reducing the need for task-specific training data.","The framework transmits 82% fewer symbols per image than JPEG2000 at comparable segment settings, indicating potential bandwidth savings for semantic-aware communication.","The two-phase training recipe (semantic reconstruction first, adversarial refinement second) may be reusable for other discrete-feature generative transmission tasks."],"fun_headline_variants":["VLM lets user-intent image coding generalize to unseen objects","User-query image codec zero-shots better with pretrained VLM","Intent-driven image coding goes beyond training data via CLIP","Zero-shot user-oriented image coding improved by VLM features","Clip-backed user coding hits better answer match on unseen objects"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing assumption is that LLaVA's hidden-state cosine similarity and its exact-match answers are faithful, self-consistent proxies for what a human user actually intends; if LLaVA's judgments diverge from human intent, the reported gains overestimate true intent preservation.","fun_headline_variants_meta":{"raw":{"variants":["VLM lets user-intent image coding generalize to unseen objects","User-query image codec zero-shots better with pretrained VLM","Intent-driven image coding goes beyond training data via CLIP","Zero-shot user-oriented image coding improved by VLM features","Clip-backed user coding hits better answer match on unseen objects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001022,"raw_usage":{"total_tokens":4166,"prompt_tokens":778,"completion_tokens":3388,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":3302}},"tokens_in":522,"tokens_out":3388,"duration_ms":24887,"temperature":1.0,"reasoning_tokens":3302,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T19:58:45.609594+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a human study in which raters are shown original and reconstructed images with the same query and asked whether the reconstruction preserves the queried content, then compare human agreement rates with the LLaVA-based answer match rates; if human gains are substantially smaller than the 34% and 5% improvements, the central claim of user-intent preservation is not supported.","supporting_citations":[],"review_version":1}