{"id":"23b861db-5291-477c-a3a7-71e51f8287cd","arxiv_id":"2607.29637","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A three-part pipeline (blank-free rendering, instruction-aware visual-token pruning, and per-input compression settings) cuts visual tokens for code-image MLLM inputs by up to 71% without losing accuracy.","lead":"CodeShrink shrinks the cost of showing code to vision-language AI models by cutting blank space, pruning unneeded image patches, and letting a small learned agent pick how much to compress each example. If it holds up, it could cut the price of AI tools that read code from screenshots while keeping accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central result may rest on train/eval overlap: the 100 ACC/OCR training instances are never stated to be disjoint from the evaluation samples, and oracle-correctness labels make any overlap directly inflate the reported gains.","rationale":"The reader's weakest assumption — that Config Agent training and evaluation sets may overlap — is the same load-bearing concern I identify, so the verdict should remain CONDITIONAL rather than being relaxed. I mark agreement as partial because the leakage path is broader than the reader stated: the downstream MLLM is fine-tuned on 50 of the 100 training instances, and if those overlap with evaluation, the model may have memorized the exact code and answers, which would inflate accuracy independently of the Config Agent. I also note the concrete numeric discrepancy between the full CodeShrink row in Tables I/II and Table IV (Java clone R differs by ~18 points), which is not mentioned in the reader's rationale but strengthens the need for a clean, reproducible evaluation split. I do not treat the missing disjointness statement as evidence of misconduct; it is an underspecified experimental detail that must be clarified. If the authors can show zero overlap and reconcile the table discrepancy, the central claim may stand. If overlap exists, the reported gains are fitting artifacts and the paper would need substantial re-analysis. This reasoning supports keeping the reader's CONDITIONAL verdict: the paper is promising but not yet acceptable without this verification.","tokens_in":16928,"tokens_out":7012,"duration_ms":66722,"concrete_test":"Request the exact instance IDs or hashes of the 100 per-task training instances (split into OCR/SFT/RL subsets) and the 300/200/200 evaluation instances, then compute the set intersection. If the intersection is non-empty, rerun the full CodeShrink pipeline on a held-out eval set chosen before sampling the 100 training instances, and report Tables I and II on that disjoint set. In the same rerun, reconcile the full CodeShrink row between Tables I/II and Table IV (especially Java clone R: 71.2% vs 53.5%) to confirm which numbers are correct. If the corrected results no longer show CodeShrink meeting or exceeding text-only on most metrics, the RQ1 answer must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The RQ1 claim — CodeShrink matches or exceeds uncompressed text-only input while cutting visual tokens — depends on the evaluation instances being unseen by every trained component. Section V-A.3 says 100 instances per task are used for Config Agent training, split into 50% for downstream-MLLM OCR fine-tuning, 25% for SFT warm-up, and 25% for RL. Section V-B evaluates on separately 'randomly sampled' instances (300 QA, 200 clone, 200 completion), but the paper never states that these evaluation instances are disjoint from the 100 training instances. This matters because the precomputed environment D stores y(a), the oracle correctness label for every configuration a, and both SFT and GRPO train directly on those labels; the downstream MLLM is also fine-tuned on 50 of the same 100 instances. If any evaluation instance appears in D_sft, D_rl, or the OCR fine-tuning set, the reported 82.3% QA, 68.0% clone, and 60.3% completion numbers are at least partly memorization artifacts rather than evidence that the learned policy adapts to new code. The absence of an explicit disjointness statement is not proof of leakage, but it is the minimal condition for the empirical claim to be interpretable. A related internal inconsistency — the full CodeShrink row in Table IV reports Java clone R=53.5% while Table II reports R=71.2% for the same method — reinforces the need for a reproducible, clearly split evaluation protocol before the central claim can be accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CodeShrink, a three-stage adaptive visual compression framework for MLLM-based code understanding: Blank-Free Rendering (BFR) compacts code images by replacing whitespace with structural markers; Dominant Token Selection (DTS) prunes task-irrelevant visual tokens using instruction-conditioned attention; and an Adaptive Compression Configuration (ACC) agent trained with SFT and GRPO predicts per-sample scale and pruning ratios. Evaluation on code QA, clone detection, and code completion reports that CodeShrink reduces visual tokens by up to 71.2% while matching or exceeding uncompressed text-only input and outperforming text- and image-compression baselines. A cross-model study isolates the effect of BFR across three MLLMs.","tokens_in":17214,"tokens_out":4369,"duration_ms":41220,"significance":"If the reported results are reproducible, the paper makes a useful contribution to token-efficient code understanding with MLLMs: it addresses a real inefficiency in image-based code input, introduces a principled rendering/layout compaction step, and provides an adaptive per-sample configuration mechanism that is more flexible than fixed-ratio baselines. The cross-model BFR study (Table III) is a genuine strength, as it shows a model-agnostic gain without fine-tuning. The code release promise and the modest training cost are also positive. However, the central empirical claims currently rest on an inadequately specified evaluation protocol: the ACC agent is trained on oracle correctness labels, and the paper does not establish that the evaluation samples are disjoint from the training instances. The missing variance information and an internal inconsistency in the reported token-reduction rates further weaken the headline conclusions.","major_comments":[{"comment":"The paper trains the ACC agent on 100 sampled instances per task, split into 50% for downstream-MLLM OCR fine-tuning, 25% for SFT warm-up, and 25% for RL. Evaluation is on 'randomly sampled' instances (300 QA, 200 clone, 200 completion), but the paper never states that these evaluation sets are disjoint from the 100 training instances. Because the precomputed environment D stores y(a), the oracle correctness label for each configuration (Section IV-C), and SFT/GRPO train directly on those labels, any overlap would let the Config Agent memorize per-instance oracle configurations rather than learn an adaptive policy. The downstream MLLM is also fine-tuned on 50 of those same instances. Please state explicitly, and verify by releasing the exact split, that no evaluation instance appears in D_sft ∪ D_rl or the OCR fine-tuning set. Without this, the RQ1 numbers in Tables I and II are not inte","section":"§V-A.3 and §V-B"},{"comment":"The reported token reduction rate R for the full CodeShrink method is internally inconsistent. On Java clone detection, Table II reports R=71.2% while Table IV reports R=53.5% for the same method; smaller discrepancies also appear on Python (QA R 39.7 vs 39.5; completion R 21.1 vs 20.9). If the two tables use different sampled subsets or different variants, that needs to be stated with per-table experimental details. As written, a reader cannot tell which R is correct for the headline 'up to 71.2%' claim, and the inconsistency suggests unreported variability in the evaluation protocol.","section":"Table II vs Table IV"},{"comment":"The main comparison tables report single-run accuracy/ES/EM/R values without error bars, confidence intervals, or significance tests. Several headline margins are small (e.g., Python QA 82.3 vs 81.0; Java completion ES 63.8 vs 58.1; clone Acc 68.0 vs 64.5). Since Table III already reports results averaged over five independent runs, the experimental infrastructure exists. Please add repeated-run statistics or at least state the number of runs and per-run variance for the main tables before claiming that CodeShrink 'exceeds' uncompressed text input.","section":"Tables I–II"},{"comment":"The comparison is not apples-to-apples on training scope. CodeShrink uses a downstream MLLM that is LoRA fine-tuned on 50 OCR instances and an ACC agent trained with SFT+GRPO against oracle labels, while all baselines (LLMLingua, FastV, VisionZip, CodeOCR) are applied off-the-shelf without task-specific adaptation. The 'w/o SFT' ablation partially addresses this, but the main table still compares a tuned system against zero-shot baselines. Please provide as least one comparable setup, e.g., a CodeOCR baseline with the same OCR fine-tuning, or explicitly characterize the comparison as method-with-training vs off-the-shelf baselines and temper the claim accordingly.","section":"§V-A.2 and §V-B"}],"minor_comments":[{"comment":"The action space A = Rδ × Rρf × Rρb has 8 × 4 × 4 = 128 configurations per instance. The statement 'yielding 38,400 configuration entries across the three tasks' is misleading unless the reader infers 128 × 300 sampled instances. Please clarify the per-instance count and the total number.","section":"§IV-C"},{"comment":"Equation (4) defines compression as C(a) = n_s / N(a), while the reported metric is R = 1 - n_c_{img}/n_{img}. The relationship between these two quantities is not explained; please define both precisely and state how R is computed from the ACC output.","section":"§IV-C and §V-A.1"},{"comment":"The functions PAGES(T, W, H, f) and LAYOUT(T, W, f) are used but not formally defined. A short description or example would make the rendering algorithm reproducible.","section":"Algorithm 1"},{"comment":"The x-axis labels '1x 2x 4x 8x DS' are crowded and the curve legend is unclear. Please enlarge and separate the marks.","section":"Fig. 1(d)"},{"comment":"The threats-to-validity section is brief. In particular, it should discuss the use of oracle correctness labels in ACC training and the need for train/eval disjointness, as these are central to interpreting the results.","section":"§VII"}],"recommendation":"major_revision","confidential_remarks":"The paper has a plausible and interesting idea, and the cross-model BFR experiment is the strongest part of the empirical work. However, the main claims depend on a train/eval separation that is not documented, and the reported token-reduction rates are internally inconsistent. The authors should be asked to provide explicit split release and rerun the main tables with variance reporting. If the train/eval sets are in fact overlapping, the central results would be uninterpretable and the paper may need to be reconsidered after substantial revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CodeShrink is a genuine step beyond fixed-resolution code rendering: the BFR renderer, the foreground/background-split attention pruning, and the RL-trained per-sample config agent form a combination I haven't seen in CodeOCR, FastV, VisionZip, or LongCodeZip. The cross-model BFR results in Table III are convincing that the rendering idea transfers, and the ablation study (Table IV) shows each component pulls its weight. So the mechanism deserves attention.\n\nThe problem is the empirical headline. The Config Agent is trained on 100 instances per task, using oracle correctness labels y(a) for every configuration in the grid. The evaluation uses separately 'randomly sampled' instances, but the paper never states that the two sets are disjoint. If any of those 100 instances show up in the 300/200/200 evaluation samples, the reported 82.3% QA / 68.0% clone / 60.3% completion numbers would be partly memorization: the RL reward and the SFT warm-up both directly maximize correctness on those instances. That is the minimal condition for RQ1 to be interpretable, and it is missing.\n\nThere are also smaller but real issues. No error bars or significance tests on the main tables, even though sample sizes are 200-300 and fine-tuning has stochasticity. CodeShrink few-shot fine-tunes the downstream MLLM while the baselines do not, so the 'consistently outperforms' claim does not isolate compression. And there is an internal inconsistency: Table II reports Java clone R=71.2% for the full CodeShrink, Table IV reports R=53.5% for the same method (also Python clone acc 68.0 vs 66.5). That kind of discrepancy needs reconciliation before any single number is quoted.\n\nI want to be fair: none of this kills the core idea. The missing disjointness is not proof of leakage, and the ablations still directionally support BFR, DTS, and ACC. The cross-model BFR experiments don't involve the trained agent at all, so they stand as solid evidence. It's a plausible system that needs a cleaner evaluation, not a paper with a load-bearing contradiction.\n\nThis is for anyone working on token-efficient MLLMs or code-as-image understanding. It deserves a serious referee — the novelty is there — but the revision needs an explicit train/eval split statement, variance reporting, and matched fine-tuning for baselines. Send it to review.","headline":"Novel and plausible three-part compression pipeline for code-as-image MLLMs, but the headline numbers need a clean train/eval split and variance reporting before I'd trust them.","tokens_in":17789,"tokens_out":2767,"would_cite":true,"duration_ms":24740,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CodeShrink compresses code images by up to 71% while matching or beating text-only accuracy on code QA, clone detection, and completion.","keywords":["visual token compression","multimodal large language models","code understanding","blank-free rendering","dominant token selection","adaptive compression configuration","reinforcement learning","code-as-image"],"falsifier":"Inspect the released data splits and verify that the 100 instances per task used for SFT/RL are excluded from the 300-QA, 200-clone, and 200-completion evaluation sets. If any overlap exists, re-run the RQ1 comparison on a strictly disjoint sample and check whether the accuracy and token-reduction numbers persist.","tokens_in":16737,"feed_emoji":"📉","tokens_out":9200,"duration_ms":75848,"temperature":0.7,"pith_summary":"The paper sets out to make visual code understanding cheap enough to be practical: when source code is fed to a multimodal language model as an image, most of the input tokens go to blank pixels and irrelevant characters rather than the logic the model actually needs. CodeShrink attacks this on three fronts: a blank-free renderer that flattens whitespace into explicit structural markers, a token selector that prunes low-attention regions after reading the instruction, and a small reinforcement-learned agent that chooses a per-sample resolution and pruning budget. The result, if it holds, is that code images can be compressed by up to 71.2% with accuracy that matches or exceeds the uncompressed text input—on Python clone detection, for instance, accuracy rises from 64.5% to 68.0% while token count drops by over 40%. The broader point is that adaptive, instruction-aware compression beats any fixed-ratio scheme.","feed_headline":"CodeShrink cuts code image tokens up to 71% without accuracy loss","feed_subtitle":"A learned agent picks a per-snippet resolution and prunes uninformative pixels, so compressed images stay readable and can beat plain text.","key_machinery":"Three interacting mechanisms carry the argument. Blank-Free Rendering linearizes the code, replacing indentation and blank lines with <INDENT>/<DEDENT> and newline markers, then solves Eq. (1) by binary search to maximize font size within a fixed visual-token budget, removing the blank-patch overhead. Dominant Token Selection splits visual tokens into foreground characters and background blanks, fuses them with the instruction in shallow layers, and prunes the lowest-importance tokens separately in each set using the attention score from Eq. (2), conditioned on the final instruction token's query. Adaptive Compression Configuration is a tiny MLLM 'Config Agent' trained with SFT warm-up then","core_discovery":"On its own terms, the paper's central claim is that visual redundancy in code images is neither unavoidable nor best handled by a fixed resolution. CodeShrink's three modules—Blank-Free Rendering, Dominant Token Selection, and Adaptive Compression Configuration—each remove a different kind of waste, and together they let an MLLM read code from a much smaller number of visual tokens. The experimental answer to RQ1 is a direct statement: 'CodeShrink delivers substantial visual token compression across all three code understanding tasks while maintaining or even exceeding the uncompressed input, and it consistently outperforms both text-based and visual compression methods.' Concretely, on Pyth","pith_inferences":["The same three-stage recipe—dense rendering, instruction-aware pruning, learned configuration—could be applied to other visual document understanding tasks where layout is sparse, such as UI screenshots, diagrams, or scanned forms; the paper's task-dependence results suggest the gains would vary with how much of the image is whitespace.","A testable extension is to train the Config Agent on a language or task it has never seen and measure whether its per-sample choices transfer; the paper's cross-model BFR results hint at model-agnostic gains, but ACC transfer is unstated.","Because the Config Agent learns a mapping from rendered image to compression setting, its choices are a byproduct signal for which parts of a code snippet carry task-relevant information; this could be harvested as a weak form of code-salience annotation without extra labeling cost."],"forward_implications":["If the central claim holds, rendering code as images becomes a cost-competitive alternative to text for code understanding: roughly a third to two-thirds of visual tokens can be removed without hurting, and in several reported settings accuracy improves.","The blank-free rendering result implies that whitespace in code is pure overhead for MLLMs—structural markers can carry the same information in a smaller image, and this holds across multiple models and resolutions.","Instruction-aware pruning means the same code image can be compressed differently for different questions, which is a direct consequence of DTS conditioning on the query.","The adaptive configuration result implies that no single fixed scale ratio is optimal across tasks; tasks like clone detection prefer aggressive scaling while QA and completion prefer gentle scaling, so per-sample adaptation has a measurable ceiling above any fixed choice."],"fun_headline_variants":["Code images shrink 71% with adaptive token pruning","Adaptive visual compression cuts code tokens 71%","Smarter code images beat text for LLM efficiency","Per-input compression slashes visual tokens for code","CodeShrink: 71% fewer visual tokens, same or better accuracy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline numbers assume that the code samples used to train the Config Agent's reward are disjoint from the samples used in the reported evaluation; if they overlap, the per-sample compression choices could be memorizing answer labels rather than learning to generalize to new code.","fun_headline_variants_meta":{"raw":{"variants":["Code images shrink 71% with adaptive token pruning","Adaptive visual compression cuts code tokens 71%","Smarter code images beat text for LLM efficiency","Per-input compression slashes visual tokens for code","CodeShrink: 71% fewer visual tokens, same or better accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000114,"raw_usage":{"total_tokens":922,"prompt_tokens":777,"completion_tokens":145,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":75}},"tokens_in":521,"tokens_out":145,"duration_ms":2722,"temperature":1.0,"reasoning_tokens":75,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T02:58:22.357014+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the released data splits and verify that the 100 instances per task used for SFT/RL are excluded from the 300-QA, 200-clone, and 200-completion evaluation sets. If any overlap exists, re-run the RQ1 comparison on a strictly disjoint sample and check whether the accuracy and token-reduction numbers persist.","supporting_citations":[],"review_version":1}