Pith. sign in

REVIEW 4 major objections 5 minor 40 references

CodeShrink compresses code images by up to 71% while matching or beating text-only accuracy on code QA, clone detection, and completion.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:58 UTC pith:J7DBMQYT

load-bearing objection Novel and plausible three-part compression pipeline for code-as-image MLLMs, but the headline numbers need a clean train/eval split and variance reporting before I'd trust them. the 4 major comments →

arxiv 2607.29637 v1 pith:J7DBMQYT submitted 2026-07-31 cs.CV cs.SE

CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding

Wenxin Tang , Jingyu Xiao , Zhenyu Liu , Zipeng Xie , Junliang Liu , Wang Luo , Yuan Jiang , Yintong Huo
show 1 more author
Michael Lyu
This is my paper
classification cs.CV cs.SE
keywords visual token compressionmultimodal large language modelscode understandingblank-free renderingdominant token selectionadaptive compression configurationreinforcement learningcode-as-image
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to make visual code understanding cheap enough to be practical: when source code is fed to a multimodal language model as an image, most of the input tokens go to blank pixels and irrelevant characters rather than the logic the model actually needs. CodeShrink attacks this on three fronts: a blank-free renderer that flattens whitespace into explicit structural markers, a token selector that prunes low-attention regions after reading the instruction, and a small reinforcement-learned agent that chooses a per-sample resolution and pruning budget. The result, if it holds, is that code images can be compressed by up to 71.2% with accuracy that matches or exceeds the uncompressed text input—on Python clone detection, for instance, accuracy rises from 64.5% to 68.0% while token count drops by over 40%. The broader point is that adaptive, instruction-aware compression beats any fixed-ratio scheme.

Core claim

On its own terms, the paper's central claim is that visual redundancy in code images is neither unavoidable nor best handled by a fixed resolution. CodeShrink's three modules—Blank-Free Rendering, Dominant Token Selection, and Adaptive Compression Configuration—each remove a different kind of waste, and together they let an MLLM read code from a much smaller number of visual tokens. The experimental answer to RQ1 is a direct statement: 'CodeShrink delivers substantial visual token compression across all three code understanding tasks while maintaining or even exceeding the uncompressed input, and it consistently outperforms both text-based and visual compression methods.' Concretely, on Pyth

What carries the argument

Three interacting mechanisms carry the argument. Blank-Free Rendering linearizes the code, replacing indentation and blank lines with <INDENT>/<DEDENT> and newline markers, then solves Eq. (1) by binary search to maximize font size within a fixed visual-token budget, removing the blank-patch overhead. Dominant Token Selection splits visual tokens into foreground characters and background blanks, fuses them with the instruction in shallow layers, and prunes the lowest-importance tokens separately in each set using the attention score from Eq. (2), conditioned on the final instruction token's query. Adaptive Compression Configuration is a tiny MLLM 'Config Agent' trained with SFT warm-up then

Load-bearing premise

The headline numbers assume that the code samples used to train the Config Agent's reward are disjoint from the samples used in the reported evaluation; if they overlap, the per-sample compression choices could be memorizing answer labels rather than learning to generalize to new code.

What would settle it

Inspect the released data splits and verify that the 100 instances per task used for SFT/RL are excluded from the 300-QA, 200-clone, and 200-completion evaluation sets. If any overlap exists, re-run the RQ1 comparison on a strictly disjoint sample and check whether the accuracy and token-reduction numbers persist.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, rendering code as images becomes a cost-competitive alternative to text for code understanding: roughly a third to two-thirds of visual tokens can be removed without hurting, and in several reported settings accuracy improves.
  • The blank-free rendering result implies that whitespace in code is pure overhead for MLLMs—structural markers can carry the same information in a smaller image, and this holds across multiple models and resolutions.
  • Instruction-aware pruning means the same code image can be compressed differently for different questions, which is a direct consequence of DTS conditioning on the query.
  • The adaptive configuration result implies that no single fixed scale ratio is optimal across tasks; tasks like clone detection prefer aggressive scaling while QA and completion prefer gentle scaling, so per-sample adaptation has a measurable ceiling above any fixed choice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same three-stage recipe—dense rendering, instruction-aware pruning, learned configuration—could be applied to other visual document understanding tasks where layout is sparse, such as UI screenshots, diagrams, or scanned forms; the paper's task-dependence results suggest the gains would vary with how much of the image is whitespace.
  • A testable extension is to train the Config Agent on a language or task it has never seen and measure whether its per-sample choices transfer; the paper's cross-model BFR results hint at model-agnostic gains, but ACC transfer is unstated.
  • Because the Config Agent learns a mapping from rendered image to compression setting, its choices are a byproduct signal for which parts of a code snippet carry task-relevant information; this could be harvested as a weak form of code-salience annotation without extra labeling cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CodeShrink, a three-stage adaptive visual compression framework for MLLM-based code understanding: Blank-Free Rendering (BFR) compacts code images by replacing whitespace with structural markers; Dominant Token Selection (DTS) prunes task-irrelevant visual tokens using instruction-conditioned attention; and an Adaptive Compression Configuration (ACC) agent trained with SFT and GRPO predicts per-sample scale and pruning ratios. Evaluation on code QA, clone detection, and code completion reports that CodeShrink reduces visual tokens by up to 71.2% while matching or exceeding uncompressed text-only input and outperforming text- and image-compression baselines. A cross-model study isolates the effect of BFR across three MLLMs.

Significance. If the reported results are reproducible, the paper makes a useful contribution to token-efficient code understanding with MLLMs: it addresses a real inefficiency in image-based code input, introduces a principled rendering/layout compaction step, and provides an adaptive per-sample configuration mechanism that is more flexible than fixed-ratio baselines. The cross-model BFR study (Table III) is a genuine strength, as it shows a model-agnostic gain without fine-tuning. The code release promise and the modest training cost are also positive. However, the central empirical claims currently rest on an inadequately specified evaluation protocol: the ACC agent is trained on oracle correctness labels, and the paper does not establish that the evaluation samples are disjoint from the training instances. The missing variance information and an internal inconsistency in the reported token-reduction rates further weaken the headline conclusions.

major comments (4)
  1. [§V-A.3 and §V-B] The paper trains the ACC agent on 100 sampled instances per task, split into 50% for downstream-MLLM OCR fine-tuning, 25% for SFT warm-up, and 25% for RL. Evaluation is on 'randomly sampled' instances (300 QA, 200 clone, 200 completion), but the paper never states that these evaluation sets are disjoint from the 100 training instances. Because the precomputed environment D stores y(a), the oracle correctness label for each configuration (Section IV-C), and SFT/GRPO train directly on those labels, any overlap would let the Config Agent memorize per-instance oracle configurations rather than learn an adaptive policy. The downstream MLLM is also fine-tuned on 50 of those same instances. Please state explicitly, and verify by releasing the exact split, that no evaluation instance appears in D_sft ∪ D_rl or the OCR fine-tuning set. Without this, the RQ1 numbers in Tables I and II are not inte
  2. [Table II vs Table IV] The reported token reduction rate R for the full CodeShrink method is internally inconsistent. On Java clone detection, Table II reports R=71.2% while Table IV reports R=53.5% for the same method; smaller discrepancies also appear on Python (QA R 39.7 vs 39.5; completion R 21.1 vs 20.9). If the two tables use different sampled subsets or different variants, that needs to be stated with per-table experimental details. As written, a reader cannot tell which R is correct for the headline 'up to 71.2%' claim, and the inconsistency suggests unreported variability in the evaluation protocol.
  3. [Tables I–II] The main comparison tables report single-run accuracy/ES/EM/R values without error bars, confidence intervals, or significance tests. Several headline margins are small (e.g., Python QA 82.3 vs 81.0; Java completion ES 63.8 vs 58.1; clone Acc 68.0 vs 64.5). Since Table III already reports results averaged over five independent runs, the experimental infrastructure exists. Please add repeated-run statistics or at least state the number of runs and per-run variance for the main tables before claiming that CodeShrink 'exceeds' uncompressed text input.
  4. [§V-A.2 and §V-B] The comparison is not apples-to-apples on training scope. CodeShrink uses a downstream MLLM that is LoRA fine-tuned on 50 OCR instances and an ACC agent trained with SFT+GRPO against oracle labels, while all baselines (LLMLingua, FastV, VisionZip, CodeOCR) are applied off-the-shelf without task-specific adaptation. The 'w/o SFT' ablation partially addresses this, but the main table still compares a tuned system against zero-shot baselines. Please provide as least one comparable setup, e.g., a CodeOCR baseline with the same OCR fine-tuning, or explicitly characterize the comparison as method-with-training vs off-the-shelf baselines and temper the claim accordingly.
minor comments (5)
  1. [§IV-C] The action space A = Rδ × Rρf × Rρb has 8 × 4 × 4 = 128 configurations per instance. The statement 'yielding 38,400 configuration entries across the three tasks' is misleading unless the reader infers 128 × 300 sampled instances. Please clarify the per-instance count and the total number.
  2. [§IV-C and §V-A.1] Equation (4) defines compression as C(a) = n_s / N(a), while the reported metric is R = 1 - n_c_{img}/n_{img}. The relationship between these two quantities is not explained; please define both precisely and state how R is computed from the ACC output.
  3. [Algorithm 1] The functions PAGES(T, W, H, f) and LAYOUT(T, W, f) are used but not formally defined. A short description or example would make the rendering algorithm reproducible.
  4. [Fig. 1(d)] The x-axis labels '1x 2x 4x 8x DS' are crowded and the curve legend is unclear. Please enlarge and separate the marks.
  5. [§VII] The threats-to-validity section is brief. In particular, it should discuss the use of oracle correctness labels in ACC training and the need for train/eval disjointness, as these are central to interpreting the results.

Circularity Check

0 steps flagged

No demonstrated circularity: the claimed compression gains come from a rendering optimization, an attention-based pruner, and an RL-trained config agent; the main flagged issue is an unstated train/eval disjointness condition, which is a validity concern rather than a construct-level circular step.

full rationale

The derivation chain is self-contained and benchmarked externally. BFR (Algorithm 1) optimizes font size under a visual-token budget and does not encode task labels. DTS uses Eq. (2), an instruction-conditioned attention score, to prune the lowest-ranked foreground/background tokens; this is the method itself, not a fitted prediction of the reported accuracy. ACC trains a Config Agent using a precomputed environment D = D_sft ∪ D_rl containing y(a), the oracle correctness of each configuration, and the SFT warm-up labels are the most-compressed correct configurations; this is a standard reward/imitation signal. The agent outputs only (r_delta, rho), and the reported RQ1 numbers come from separately described random samples (300/200/200). No equation makes the evaluation metric identical to the training target by construction. The one real gap is that Section V-A.3 and V-B never state that the 100 per-task ACC/MLLM fine-tuning instances are disjoint from the evaluation samples; if they overlap, the gains would be fitting artifacts. I flag this as an omitted experimental guarantee, not as demonstrated circularity, since the paper does not exhibit an actual reduction of prediction to fit. Cross-model BFR results on Qwen3-VL, GPT-5-mini, and Gemini-3-Flash, plus ablations, provide independent evidence. There is no load-bearing self-citation or imported uniqueness theorem. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claim rests on three hand-set discrete grids (rδ, rρf, rρb), an unspecified DTS pruning layer, and an unspecified foreground/background segmentation threshold. The main domain assumptions are that MLLMs can read rendered code, that attention importance scores predict token dispensability, and that pruning visual tokens during prefill proportionally reduces cost. The only ad-hoc-to-paper choice is exposing just the first page of the code image to the Config Agent.

free parameters (3)
  • Candidate grids for scale and pruning ratios = Rδ={1,2,...,8}; Rρf={0,0.1,0.3,0.5}; Rρb={0.5,0.7,0.9,1}
    The action space A in Eq. (3) is hand-selected; the agent's optimal per-sample choice is constrained to these 128 discrete settings, so the benchmark results are bounded by this design choice.
  • DTS pruning layer depth
    Section IV-B prunes "in the shallow MLLM layers" without giving the layer index; the depth at which query-key attention is measured determines the importance scores and hence which tokens survive.
  • Foreground/background segmentation threshold
    Section IV-B separates code tokens from blank tokens using "edge and color-gradient cues" but gives no threshold, kernel, or algorithm; the split boundary is a hand-set component of the method.
axioms (5)
  • domain assumption MLLMs can read and reason over source code rendered as images at the tested resolutions
    The entire approach presupposes that a vision-language model can recover program text from pixels; the paper builds on CodeOCR [1] for this but does not independently verify OCR fidelity at every scale ratio rδ.
  • domain assumption Attention importance scores from Eq. (2) correlate with token dispensability
    DTS removes the lowest-scoring foreground/background tokens; the assumption that low attention implies safe removal is validated only by the ablation, not by a mechanistic analysis.
  • domain assumption Removing visual tokens during prefill reduces inference cost proportionally
    The claimed efficiency gain assumes KV-cache and FLOPs scale with the retained visual-token count n_img; this is standard in FastV/VisionZip but not measured end-to-end in wall-clock terms.
  • standard math GRPO objective (Eqs. 7-8) with the group-relative advantage is a valid optimization for the Config Agent
    The clipped surrogate and KL penalty are taken from DeepSeekMath [16] without re-derivation; standard RLHF machinery.
  • ad hoc to paper The first page of the code image contains enough information to select a good compression configuration
    Section IV-C limits the Agent's input to the first page "to keep training efficient"; no experiment shows that later pages do not change the optimal rδ or rρ.

pith-pipeline@v1.3.0-daily-deepseek · 16511 in / 17722 out tokens · 148452 ms · 2026-08-03T02:58:22.357014+00:00 · methodology

0 comments
read the original abstract

Rendering source code as images offers a promising way to reduce the input costs of Multimodal Large Language Models (MLLMs). Adjusting image resolution can trade visual token cost against content fidelity. However, resolution scaling alone overlooks two sources of inefficiency: blank regions created by line breaks and indentation, and code regions irrelevant to the current instruction. Moreover, the best compression setting varies across inputs, tasks, and models, limiting fixed-ratio strategies. We propose CodeShrink, an adaptive visual compression framework with three components. Blank-Free Rendering replaces whitespace-dependent layouts with compact layouts and explicit structural markers, removing layout-induced tokens. Adaptive Compression Configuration uses a lightweight agent trained with reinforcement learning to predict a per-input setting that balances token efficiency and readability. Dominant Token Selection jointly analyzes the instruction and code image to prune task-irrelevant visual tokens during inference. We evaluate CodeShrink on code question answering, clone detection, and code completion. CodeShrink reduces visual token use by up to 71.2\% while matching or exceeding uncompressed text-only inputs, and consistently outperforms text-based and visual compression baselines across all three tasks. These results show that combining layout compaction, adaptive configuration, and instruction-aware pruning can make multimodal code understanding more efficient. Our code is available at https://github.com/vinsontang1/CodeShrink.

Figures

Figures reproduced from arXiv: 2607.29637 by Jingyu Xiao, Junliang Liu, Michael Lyu, Wang Luo, Wenxin Tang, Yintong Huo, Yuan Jiang, Zhenyu Liu, Zipeng Xie.

Figure 1
Figure 1. Figure 1: Preliminary study of code image compression. (a) Original code rendering contains substantial blank regions, shown in gray. (b) Attention visualization shows that task-relevant code regions receive higher attention than other regions. (c) The preferred scale ratio varies across code completion samples. (d) Dynamic selection (DS) surpasses the best fixed scale ratio across all three tasks. Code QA and code … view at source ↗
Figure 2
Figure 2. Figure 2: The framework of CodeShrink. pages, initialized to 1 (line 3). The code image shape is deter￾mined from the current budget B/p and a target aspect ratio α, subject to the minimum and maximum pixel resolutions accepted by the model (line 5). When the smallest font cannot fit the code within the current page count, we raise the page count to the required value and retry. This fixed-point iteration converges … view at source ↗
Figure 3
Figure 3. Figure 3: Task-dependent compression configuration of CodeShrink on the three code understanding tasks. (a) shows the effect of the scale ratio [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Case study of the configuration selection result. The Config Agent selects 4 [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Code images rendered by different methods. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Case study of the DTS on the three tasks, where the red boxes mark the [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 2 canonical work pages

  1. [1]

    Codeocr: On the effectiveness of vision language models in code understanding,

    Y . Shi, C. Xie, Z. Sun, Y . Chen, C. Zhang, L. Yun, C. Wan, H. Zhang, D. Lo, and X. Gu, “Codeocr: On the effectiveness of vision language models in code understanding,”arXiv preprint arXiv:2602.01785, 2026

    Pith/arXiv arXiv 2026

  2. [2]

    Longcodebench: Evaluating coding llms at 1m context windows,

    S. Rando, L. Romani, A. Sampieri, L. Franco, J. Yang, Y . Kyuragi, F. Galasso, and T. Hashimoto, “Longcodebench: Evaluating coding llms at 1m context windows,”arXiv preprint arXiv:2505.07897, 2025

    arXiv 2025

  3. [3]

    Gptclonebench: A comprehensive benchmark of semantic clones and cross-language clones using gpt-3 model and semanticclonebench,

    A. I. Alam, P. R. Roy, F. Al-Omari, C. K. Roy, B. Roy, and K. A. Schnei- der, “Gptclonebench: A comprehensive benchmark of semantic clones and cross-language clones using gpt-3 model and semanticclonebench,” in2023 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2023, pp. 1–13

    2023

  4. [4]

    Longcoder: A long- range pre-trained language model for code completion,

    D. Guo, C. Xu, N. Duan, J. Yin, and J. McAuley, “Longcoder: A long- range pre-trained language model for code completion,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 12 098–12 107

    2023

  5. [5]

    Codereval: A benchmark of pragmatic code generation with generative pre-trained models,

    H. Yu, B. Shen, D. Ran, J. Zhang, Q. Zhang, Y . Ma, G. Liang, Y . Li, Q. Wang, and T. Xie, “Codereval: A benchmark of pragmatic code generation with generative pre-trained models,” inProceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY , USA: Association for Computing Machinery,

  6. [6]

    Repogenix: Dual context-aided repository-level code completion with language models,

    M. Liang, X. Xie, G. Zhang, X. Zheng, P. Di, W. Jiang, H. Chen, C. Wang, and G. Fan, “Repogenix: Dual context-aided repository-level code completion with language models,” inProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’24. New York, NY , USA: Association for Computing Machinery, 2024, p. 2466–2467....

    arXiv 2024

  7. [7]

    A 3a3-codgen: A repository-level code generation framework for code reuse with local-aware, global-aware, and third-party-library- aware,

    D. Liao, S. Pan, X. Sun, X. Ren, Q. Huang, Z. Xing, H. Jin, and Q. Li, “A 3a3-codgen: A repository-level code generation framework for code reuse with local-aware, global-aware, and third-party-library- aware,”IEEE Transactions on Software Engineering (TSE), vol. 50, no. 12, pp. 3369–3384, 2024

    2024

  8. [8]

    Visualrepair: Dynamic tool calling and region focusing for visual software issue repair,

    J. Xiao, Z. Zhang, H. Hou, Y . Wan, Y . Jiang, Y . Huo, and M. R. Lyu, “Visualrepair: Dynamic tool calling and region focusing for visual software issue repair,” 2026. [Online]. Available: https: //arxiv.org/abs/2607.14075

    Pith/arXiv arXiv 2026

  9. [9]

    Natural is the best: Model-agnostic code simplification for pre-trained large language models,

    Y . Wang, X. Li, T. N. Nguyen, S. Wang, C. Ni, and L. Ding, “Natural is the best: Model-agnostic code simplification for pre-trained large language models,”Proc. ACM Softw. Eng., vol. 1, no. FSE, Jul. 2024. [Online]. Available: https://doi.org/10.1145/3643753

    work page doi:10.1145/3643753 2024

  10. [10]

    Diet code is healthy: simplifying programs for pre-trained models of code,

    Z. Zhang, H. Zhang, B. Shen, and X. Gu, “Diet code is healthy: simplifying programs for pre-trained models of code,” inProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2022. New York, NY , USA: Association for Computing Machinery, 2022, p. 1073–1084. [Online]...

    arXiv 2022

  11. [11]

    Llmlingua: Com- pressing prompts for accelerated inference of large language models,

    H. Jiang, Q. Wu, C.-Y . Lin, Y . Yang, and L. Qiu, “Llmlingua: Com- pressing prompts for accelerated inference of large language models,” inProceedings of the 2023 conference on empirical methods in natural language processing, 2023, pp. 13 358–13 376

    2023

  12. [12]

    Longllmlingua: Accelerating and enhancing llms in long context sce- narios via prompt compression,

    H. Jiang, Q. Wu, X. Luo, D. Li, C.-Y . Lin, Y . Yang, and L. Qiu, “Longllmlingua: Accelerating and enhancing llms in long context sce- narios via prompt compression,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 1658–1677

    2024

  13. [13]

    Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression,

    Z. Pan, Q. Wu, H. Jiang, M. Xia, X. Luo, J. Zhang, Q. Lin, V . R ¨uhle, Y . Yang, C.-Y . Linet al., “Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 963– 981

    2024

  14. [15]

    Deepseek-ocr: Contexts optical compres- sion,

    H. Wei, Y . Sun, and Y . Li, “Deepseek-ocr: Contexts optical compres- sion,”arXiv preprint arXiv:2510.18234, 2025

    Pith/arXiv arXiv 2025

  15. [16]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wuet al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,”arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  16. [17]

    Visionzip: Longer is better but not necessary in vision language models,

    S. Yang, Y . Chen, Z. Tian, C. Wang, J. Li, B. Yu, and J. Jia, “Visionzip: Longer is better but not necessary in vision language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 19 792–19 802

    2025

  17. [18]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models,

    L. Chen, H. Zhao, T. Liu, S. Bai, J. Lin, C. Zhou, and B. Chang, “An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 19–35

    2024

  18. [19]

    Qwen3.5-omni technical report,

    Q. Team, “Qwen3.5-omni technical report,”arXiv preprint arXiv:2604.15804, 2026

    Pith/arXiv arXiv 2026

  19. [20]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” Iclr, vol. 1, no. 2, p. 3, 2022

    2022

  20. [21]

    Qwen3 technical report,

    Q. Team, “Qwen3 technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  21. [22]

    Gpt-5-mini model documentation,

    OpenAI, “Gpt-5-mini model documentation,” https://platform.openai. com/docs/models/gpt-5-mini, 2025, accessed: 2026-01-30

    2025

  22. [23]

    Gemini 3 flash model card,

    Google DeepMind, “Gemini 3 flash model card,” https://deepmind. google/models/model-cards/, 2025, accessed: 2026-01-30

    2025

  23. [24]

    SlideCoder: Layout- aware RAG-enhanced hierarchical slide generation from design,

    W. Tang, J. Xiao, W. Jiang, X. Xiao, Y . Wang, X. Tang, Q. Li, Y . Ma, J. Liu, S. Tang, and M. R. Lyu, “SlideCoder: Layout- aware RAG-enhanced hierarchical slide generation from design,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Peng, Eds. Suzho...

    2025

  24. [25]

    Mmcode: Bench- marking multimodal large language models for code generation with visually rich programming problems,

    K. Li, Y . Tian, Q. Hu, Z. Luo, Z. Huang, and J. Ma, “Mmcode: Bench- marking multimodal large language models for code generation with visually rich programming problems,” inFindings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 736–783

    2024

  25. [26]

    Humaneval-v: Systematic evaluation of visual reasoning in large multimodal models for code generation,

    F. Zhang, L. Wu, H. Bai, G. Lin, X. Li, X. Yu, Y . Wang, B. Chen, and J. Keung, “Humaneval-v: Systematic evaluation of visual reasoning in large multimodal models for code generation,”ACM Trans. Softw. Eng. Methodol (TOSEM)., May 2026, just Accepted. [Online]. Available: https://doi.org/10.1145/3813804

    work page doi:10.1145/3813804 2026

  26. [27]

    Efficientpostergen: Semantic-aware efficient poster generation via token compression and accurate violation detection,

    W. Tang, J. Xiao, Y . Gong, F. Ran, T. Xia, J. Liu, M. H. Lam, W. Wang, and M. R. Lyu, “Efficientpostergen: Semantic-aware efficient poster generation via token compression and accurate violation detection,” arXiv preprint arXiv:2603.00155, 2026

    arXiv 2026

  27. [28]

    Envisioning future interactive web development: Editing webpage with natural language,

    T. H. Dang, J. Xiao, and Y . Huo, “Envisioning future interactive web development: Editing webpage with natural language,”2025 2nd IEEE/ACM International Conference on AI-powered Software (AIware), pp. 61–66, 2025. [Online]. Available: https://api.semanticscholar.org/ CorpusID:282592059

    2025

  28. [29]

    Comuicoder: Component-based reusable ui code generation for complex websites via semantic segmentation and element-wise feedback,

    J. Xiao, J. Qin, S. Li, M. H. Lam, Y . Wan, J.-t. Huang, Y . Huo, and M. R. Lyu, “Comuicoder: Component-based reusable ui code generation for complex websites via semantic segmentation and element-wise feedback,”arXiv preprint arXiv:2602.19276, 2026

    arXiv 2026

  29. [30]

    Uibenchkit: A unified toolkit for design-to-code model evaluation,

    C. T. Le, T. O. Y . Siang, J. Xiao, Y . Wan, and Y . Huo, “Uibenchkit: A unified toolkit for design-to-code model evaluation,”arXiv preprint arXiv:2605.13141, 2026

    Pith/arXiv arXiv 2026

  30. [31]

    From runnable to shippable: Multi-agent test-driven development for gener- ating full-stack web applications from requirements,

    Y . Wan, T. Liang, J. Xu, J. Xiao, Y . Huo, and M. R. Lyu, “From runnable to shippable: Multi-agent test-driven development for gener- ating full-stack web applications from requirements,”arXiv preprint arXiv:2605.17242, 2026

    Pith/arXiv arXiv 2026

  31. [32]

    Mrweb: An exploration of generating multi-page resource-aware web code from ui designs,

    Y . Wan, Y . Dong, J. Xiao, Y . Huo, W. Wang, and M. R. Lyu, “Mrweb: An exploration of generating multi-page resource-aware web code from ui designs,”arXiv preprint arXiv:2412.15310, 2024

    Pith/arXiv arXiv 2024

  32. [33]

    Interaction2code: Benchmarking mllm-based interactive webpage code generation from interactive prototyping,

    J. Xiao, Y . Wan, Y . Huo, Z. Wang, X. Xu, W. Wang, Z. Xu, Y . Wang, and M. R. Lyu, “Interaction2code: Benchmarking mllm-based interactive webpage code generation from interactive prototyping,” in2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE Press, 2025, p. 241–253. [Online]. Available: https://doi.org/10.1109/A...

    arXiv 2025

  33. [34]

    Designrepair: Dual-stream design guideline-aware frontend repair with large language models,

    M. Yuan, J. Chen, Z. Xing, A. Quigley, Y . Luo, T. Luo, G. Mohammadi, Q. Lu, and L. Zhu, “Designrepair: Dual-stream design guideline-aware frontend repair with large language models,” inProceedings of the IEEE/ACM 47th International Conference on Software Engineering, ser. ICSE ’25. IEEE Press, 2025, p. 2483–2494. [Online]. Available: https://doi.org/10.1...

    arXiv 2025

  34. [35]

    Designbench: A comprehensive benchmark for mllm-based front-end code generation,

    J. Xiao, M. Wang, M. H. Lam, Y . Wan, J. Liu, Y . Huo, and M. R. Lyu, “Designbench: A comprehensive benchmark for mllm-based front-end code generation,”arXiv preprint arXiv:2506.06251, 2025

    arXiv 2025

  35. [36]

    Empowering agile-based generative software development through human-ai teamwork,

    S. Zhang, Z. Xing, R. Guo, F. Xu, L. Chen, Z. Zhang, X. Zhang, Z. Feng, and Z. Zhuang, “Empowering agile-based generative software development through human-ai teamwork,”ACM Transactions on Software Engineering and Methodology (TOSEM)., Jan. 2025. [Online]. Available: https://doi.org/10.1145/3702987

    doi:10.1145/3702987 2025

  36. [37]

    Token sugar: Making source code sweeter for llms through token-efficient shorthand,

    Z. Sun, C. Yang, X. Du, Z. Yang, L. Li, and D. Lo, “Token sugar: Making source code sweeter for llms through token-efficient shorthand,” in 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2025, pp. 2440–2451

    2025

  37. [38]

    Less is more: Docstring compression in code generation,

    G. Yang, Y . Zhou, W. Cheng, X. Zhang, X. Chen, T. Y . Zhuo, K. Liu, X. Zhou, D. Lo, and T. Chen, “Less is more: Docstring compression in code generation,”ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 35, no. 2, pp. 1–31, 2026

    2026

  38. [39]

    The hidden cost of readability: How code formatting silently consumes your llm budget,

    D. Pan, Z. Sun, C. Zhang, D. Lo, and X. Du, “The hidden cost of readability: How code formatting silently consumes your llm budget,” arXiv preprint arXiv:2508.13666, 2025

    Pith/arXiv arXiv 2025

  39. [40]

    Effi- cientuicoder: A bidirectional token compression framework for efficient mllm-based ui code generation,

    J. Xiao, Z. Zhang, Y . Wan, Y . Huo, Y . Liu, and M. R. Lyu, “Effi- cientuicoder: A bidirectional token compression framework for efficient mllm-based ui code generation,”Proceedings of the ACM on Software Engineering, vol. 3, no. FSE, pp. 2396–2418, 2026

    2026

  40. [2024]

    Available: https://doi.org/10.1145/3597503.3623316

    [Online]. Available: https://doi.org/10.1145/3597503.3623316

    arXiv