Pith. sign in

REVIEW 5 major objections 4 minor 55 references

Aggregated Structural Representation with Large Language Models for Human-Centric Layout Generation

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that swapping the visual encoder of a multimodal LLM for GNN graph features plus a human-editable relation matrix yields UI layouts with less overlap and more accurate structure than graph-based, transformer, and…

desk verdict A genuinely new GNN-for-ViT swap in an MLLM for layout generation, but the current evaluation does not yet support the headline claims. read the letter →

arxiv 2505.19554 v1 pith:OA6Q4OC5 submitted 2025-05-26 cs.CV

Jiongchao Jin , Shengchu Zhao , Dajun Chen , Wei Jiang , Yong Li This is my paper
classification cs.CV
keywords UIlayoutgenerationgraphneuralnetworksmultimodallargelanguagemodelsrelationmatrixcompletioneditinghuman-centricdesignRICOdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to settle a two-sided failure mode in automatic UI layout generation. Graph-based methods keep structure but generate poorly, while vision-based generative models produce richer layouts but forget the explicit relationships between components, so boxes overlap. The proposed answer is to feed the large language model a structural prior instead of an image: a graph neural network extracts a 1024-dimensional feature, and a decoder produces a relation matrix encoding containment, parallelism, and relative position that the LLM must respect. The paper reports that on the RICO dataset this combination beats four baselines on mean Intersection over Union, relation error, and overlap across direct generation, completion, and graph editing, and that human editing of the relation matrix gives users a handle on the output. A sympathetic reader would take this as evidence that explicit structure can be injected into an LLM without relying on pixel-level vision.

What carries the argument

The central object is the relation matrix $M = \{M_{sem}, M_{pos}\}$, a discrete encoding of semantic relations ('contain', 'parallel') and positional relations ('top', 'left') between UI components. It is produced by a decoder from the 1024-dimensional graph feature $h_a$ computed by a five-layer GNN over node features $N_f$ and edge matrix $E_M$, and it is then fed into the LLM together with $h_a$. The matrix does two jobs: it supplies the inter-node structure that the LLM is not naturally good at, and because it is a table of discrete relations, a designer can edit it or sample random perturbations of it to steer generation. The same matrix also supervises the generator, so the LLM's output is pinned to the stated structure while its grounding ability supplies coordinates and categories.

What would settle it

A controlled comparison would feed the same layout prompt to two otherwise identical MLLMs, one with the original ViT image stream and one with the ASR graph stream, and check whether the graph stream's mIoU, relation-error, and overlap advantages survive across the RICO, PubLayNet, and Magazine test splits. If the graph-stream model loses to the image-stream model on any dataset, the paper's attribution of the gain to structural representation would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that graph structure and LLM generation are complementary, not competing, and that a relation matrix is the right interface between them. Concretely, the LLM is fine-tuned with its Vision Transformer replaced by the graph representation module; during training the GNN sees a masked layout graph while the relation matrix comes from the ground truth, so the LLM learns to finish missing nodes and even add new ones without breaking existing relations. On all three tasks in the main RICO table, ASR reports the best mIoU and relation error and the lowest overlap; for example, on direct UI generation it reaches mIoU 0.63 with relation error 0.27, and the ablation shows that removing the relation matrix drops Max-IoU from 0.54 to 0.27 and raises relation error from 0.30 to 0.61. The paper also reports better crowd-sourced reasonableness scores on easy, medium, and hard layouts, with the largest gap on hard layouts.

Load-bearing premise

The load-bearing premise is that the language model still understands coordinates and component types when its image encoder is taken away and replaced by a graph feature plus relation matrix; the paper does not give details of how the graph feature is turned into tokens or which Intern-VL model is used.

Editorial extensions

If this is right

  • A designer can change one relation in the matrix and regenerate a full, structurally consistent layout without re-entering all coordinates.
  • The masked-graph training scheme lets the model complete partial layouts and add new components that do not violate the existing structure.
  • If the recipe transfers, any task expressible as a graph with typed nodes and relations—posters, documents, floorplans, scene graphs—can be generated by the same graph-to-LLM pipeline.
  • Because the image encoder is removed at inference, the required input is a compact graph feature rather than a rendered image, which may reduce inference cost and simplify conditioning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not in the paper: treating the relation matrix as a constraint language for accessibility rules—for example, requiring touch targets to be at least a given size or keeping high-contrast text separated from backgrounds—since the matrix already exposes exactly those relations.
  • Not in the paper: a user-in-the-loop study that records what happens when designers actually edit the relation matrix, rather than only scoring final generated layouts, would test the human-centric claim more directly.
  • Not in the paper: ablating perturbations of positional relations versus semantic relations separately would isolate which part of the editable matrix drives the reported diversity in generation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes Aggregated Structural Representation (ASR), a layout-generation pipeline in which a GNN encoder extracts graph features and an editable relation matrix from UI layout graphs, and these representations are fed to the Intern-VL MLLM after replacing its ViT visual stream. The LLM is fine-tuned to output JSON describing node categories, coordinates, and relations, and the relation matrix can be edited by users or randomly perturbed to inject diversity. The method is evaluated on RICO, PubLayNet, and Magazine for UI generation, completion, and graph editing, using mIoU, relation error, overlap, FID, and a crowdsourced perceptual study. The authors report consistent quantitative improvements and claim the relation matrix is essential for structural fidelity.

Significance. The core idea—replacing the visual token stream of an MLLM with explicit graph-derived structural features and an editable relation matrix—is timely and, if validated, would provide a useful recipe for injecting hard structural constraints into LLM-based layout generation. The paper's breadth is a strength: three datasets, three tasks, an ablation, and a user study, and the relation-matrix editability is a genuinely useful human-centric feature. The main weaknesses are in the evaluation protocol: the graph-editing comparison is asymmetric, the FID metric is nonstandard, the ablation numbers do not match the main table, and no uncertainty estimates are reported. These issues do not invalidate the idea, but they currently prevent the quantitative claims from being accepted at face value.

major comments (5)
  1. [Section 4.2, Graph Editing protocol] The protocol is asymmetric: the text states that ASR allows manual graph editing during generation whereas other models require all settings to be predefined, and for baselines the authors 'coarsely set some nodes and relations before generation.' This means the Graph Editing rows in Table 2 compare ASR with interactive refinement against baselines given coarse initial conditions, so the reported advantages (e.g., RE 0.09 vs 0.12) may come from additional information or iterative adjustment rather than from the graph-representation substitution. Please either give baselines an equivalent interactive protocol or restrict the comparison to a single-shot setting for all methods.
  2. [Section 4.2, FID definition] FID is computed with a model 'trained to classify whether an input layout is corrupted or not,' which is not the standard Inception-based FID used by the cited baselines. The resulting values are therefore not comparable across methods or to published FID numbers, and the FID column in Table 2 should either be replaced with the standard definition or removed. As written, the FID entries cannot support the claim that ASR 'produces more realistic and coherent layout components.'
  3. [Table 4 vs Table 2] The ablation reports 'ASR with RM' on RICO with mIoU 0.54 and RE 0.30, whereas Table 2's ASR UI-Gen row on RICO reports mIoU 0.63 and RE 0.27. If the ablation uses the same task and split, the numbers are inconsistent; if it uses a different setting, that setting is not described. This needs clarification before the ablation can be used to quantify the relation matrix's contribution.
  4. [Section 3.4, architecture details] The paper says Intern-VL is fine-tuned by replacing its ViT component with the graph representation module, but it does not specify which Intern-VL checkpoint is used, how the 1024-dimensional feature is projected into the LLM's token embedding space, or how the connector is initialized. Because the method's central premise is that Intern-VL can treat graph features as a substitute for visual tokens while retaining coordinate-grounding and language abilities, this missing detail is essential for reproducibility and for judging whether the approach is a genuine MLLM adaptation.
  5. [Section 4.3 and Table 2] The claim that ASR 'consistently outperforms' all baselines in mIoU, RE, and OL is not fully supported by the table: in Graph Editing, L-DM has higher mIoU (0.69 vs 0.68), and in Completion and Graph Editing ASR has worse FID than LDGM or L-DM. No error bars or significance tests are reported, so small differences such as the UI-Gen mIoU gap (0.63 vs 0.62) could be within run-to-run variation. Please provide uncertainty estimates and restrict the claim to metrics where the comparison is actually consistent.
minor comments (4)
  1. [Throughout] There are several typos and formatting inconsistencies: 'Defination' in Table 1, 'ultilized' in Section 4.1, 'Amanze Mechanical Turk' in Section 4.4, 'Rico Dataset' in the Table 2 caption, and 'langauge' in the PosterLLaMA reference.
  2. [Section 3.2, Eq. (3)] The denominator in Eq. (3) sums only over negative samples, whereas the standard SimSCE loss includes the positive pair in the denominator as well. If the implementation follows the equation as written, the loss is not the stated SimSCE loss; if it follows SimSCE, the equation should be corrected.
  3. [Section 4.4, user study] Only mean scores are reported; no standard deviations, number of ratings per layout, or inter-annotator agreement are given, so it is difficult to assess whether differences such as 4.30 vs 4.28 on easy layouts are meaningful.
  4. [Sections 3.2 and 3.3] The dimensions of the relation matrices Msem and Mpos are not defined, and the handling of graphs with variable node counts in the GNN/LLM interface is not described; a worked example showing how node sets map to matrix rows and to the JSON output would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the ASR pipeline is supervised learning from masked graphs to ground-truth layouts; the only self-citation is non-load-bearing.

full rationale

The paper's central claim is that replacing the ViT stream of an MLLM with graph features and an editable relation matrix improves layout generation. The derivation chain is self-contained in the supervised-learning sense: the graph representation module is trained with a triplet contrastive loss (Eq. 3) on masked ground-truth graphs, the relation-matrix decoder is supervised by ground-truth layouts (Sec. 3.3), and the LLM is fine-tuned with the masked graph feature and ground-truth relation matrix as input and ground-truth layout JSON as target (Sec. 3.4). No reported metric is constructed from the target by definition: mIoU compares predicted boxes to held-out ground truth, RE compares predicted relations to ground truth, and OL measures internal overlap. The only self-citation is Nong et al. 2024, co-authored by one of the present authors, and it appears in a related-work enumeration without bearing on the method or its evaluation. Concerns about the graph-editing protocol allowing ASR interactive refinement while baselines receive coarse fixed inputs, and about FID using a task-specific corruption classifier, are evaluation-comparability issues rather than circular reductions. The inconsistency between Table 4's ASR-with-RM row and Table 2's ASR UI-Gen row is an internal-consistency question, not a circular derivation. Accordingly, no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or ontological entities are introduced; the relation matrix is a learned representation rather than an invented entity. The central method relies on hand-set hyperparameters, a fixed relation vocabulary, and the architectural assumption that Intern-VL can ingest graph features in place of its ViT stream.

free parameters (4)
  • SimSCE temperature tau = not reported
    Used in the contrastive loss in Eq. 3; chosen by hand and not specified, affecting the learned graph representation.
  • Masking ratio for positive graphs = 5%-25%
    Section 3.2: positive graphs are created by randomly masking 5% to 25% of the ground truth graph; this range is a hand-set hyperparameter for training.
  • GNN output feature dimension = 1024
    Section 3.2: ha is a 1024-dimensional feature; this architectural choice is not derived from data.
  • GNN depth = 5 layers
    Section 3.2 states a five-layer GNN is used; this design choice is not motivated by experiments.
assumptions (4)
  • domain assumption Contrastive SimSCE pretraining makes masked positive graphs closer to ground truth than unrelated negatives, so the learned graph feature contains structure sufficient for the relation decoder.
    Section 3.2 Eq. 3; the method assumes the contrastive objective transfers to the downstream decoding task.
  • domain assumption The four relation types (Top, Left, Parallel, Contain) are sufficient to describe UI layout structure for the LLM to generate non-overlapping layouts.
    Section 3.1 defines relations; the paper does not test whether a richer relation vocabulary would change results.
  • ad hoc to paper Intern-VL can be fine-tuned with graph features replacing its ViT stream while retaining its language and grounding abilities.
    Section 3.4 states the replacement without reporting connector initialization or architectural adaptations; this is a core modeling assumption.
  • domain assumption The filtered RICO, PubLayNet, and Magazine datasets with a 7:2:1 split provide a fair and representative evaluation of layout generation.
    Section 4.1; no analysis of dataset bias or filtering criteria is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aggregated Structural Representation with Large Language Models for Human-Centric Layout Generation." pith.science (2026). https://pith.science/paper/OA6Q4OC5

@misc{pith2026250519554,
  author       = {Pith},
  title        = {Pith review of: Aggregated Structural Representation with Large Language Models for Human-Centric Layout Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OA6Q4OC5}},
  note         = {Machine review of arXiv:2505.19554}
}
read the original abstract

Time consumption and the complexity of manual layout design make automated layout generation a critical task, especially for multiple applications across different mobile devices. Existing graph-based layout generation approaches suffer from limited generative capability, often resulting in unreasonable and incompatible outputs. Meanwhile, vision based generative models tend to overlook the original structural information, leading to component intersections and overlaps. To address these challenges, we propose an Aggregation Structural Representation (ASR) module that integrates graph networks with large language models (LLMs) to preserve structural information while enhancing generative capability. This novel pipeline utilizes graph features as hierarchical prior knowledge, replacing the traditional Vision Transformer (ViT) module in multimodal large language models (MLLM) to predict full layout information for the first time. Moreover, the intermediate graph matrix used as input for the LLM is human editable, enabling progressive, human centric design generation. A comprehensive evaluation on the RICO dataset demonstrates the strong performance of ASR, both quantitatively using mean Intersection over Union (mIoU), and qualitatively through a crowdsourced user study. Additionally, sampling on relational features ensures diverse layout generation, further enhancing the adaptability and creativity of the proposed approach.

Figures

Figures reproduced from arXiv: 2505.19554 by the authors.

Figure 1
Figure 1. ASR extracts hierarchical representations from graphs with semantic and positional relationships, then [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview pipeline of ASR. The top subfigure shows the training phase of ASR while the bottom subfigure [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The explanation of semantic relations and [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The gallery of qualitative results in different tasks [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 27 canonical work pages

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    work page 2023

  2. [2]

    Michael Bauerly and Yili Liu. 2006. Computational modeling and experimental investigation of effects of compositional elements on interface and design aesthetics. International journal of human-computer studies, 64(8):670--682

    work page 2006

  3. [3]

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alex Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuh...

    arXiv 2023

  4. [4]

    Yutao Cheng, Zhao Zhang, Maoke Yang, Nie Hui, Chunyuan Li, Xinglong Wu, and Jie Shao. 2024. Graphic design with large multimodal model. arXiv preprint arXiv:2404.14368

    arXiv 2024

  5. [5]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. 2023. Palm-e: An emb...

    arXiv 2023

  6. [6]

    Shiyu Duan, Runsheng Zhang, Mengmeng Chen, Ziyi Wang, and Shixiao Wang. 2024. Efficient and aesthetic ui design with a deep learning-based interface generation tree algorithm. arXiv preprint arXiv:2410.17586

    work page Pith review arXiv 2024

  7. [7]

    Difei Gao, Ke Li, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2020. Multi-modal graph neural network for joint reasoning on vision and scene text. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12746--12756

    work page 2020

  8. [8]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821

    arXiv 2021

Show all 55 references
  1. [9]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024

  2. [10]

    Liu He, Yijuan Lu, John Corring, Dinei Florencio, and Cha Zhang. 2023. https://doi.org/10.1007/978-3-031-41676-7_21 Diffusion-based document layout generation . In Document Analysis and Recognition - ICDAR 2023, pages 361--378. Springer Nature Switzerland

    2023 doi

  3. [11]

    Elad Hoffer and Nir Ailon. 2015. Deep metric learning using triplet network. In Similarity-based pattern recognition: third international workshop, SIMBAD 2015, Copenhagen, Denmark, October 12-14, 2015. Proceedings 3, pages 84--92. Springer

    2015

  4. [12]

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, and Jie Tang. 2023. https://arxiv.org/abs/2312.08914 Cogagent: A visual language model for gui agents . Preprint, arXiv:2312.08914

    2023 arXiv

  5. [13]

    HsiaoYuan Hsu, Xiangteng He, and Yuxin Peng. 2023. Densitylayout: Density-conditioned layout gan for visual-textual presentation designs. In International Conference on Image and Graphics, pages 187--199. Springer

    2023

  6. [14]

    Ruizhen Hu, Zeyu Huang, Yuhan Tang, Oliver Van Kaick, Hao Zhang, and Hui Huang. 2020. Graph2plan: Learning floorplan generation from layout graphs. ACM Transactions on Graphics (TOG), 39(4):118--1

    2020

  7. [15]

    Mude Hui, Zhizheng Zhang, Xiaoyi Zhang, Wenxuan Xie, Yuwang Wang, and Yan Lu. 2023. Unifying layout generation with a decoupled diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1942--1951

    2023

  8. [16]

    Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. 2023. Layoutdm: Discrete diffusion model for controllable layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10167--10176

    2023

  9. [17]

    Zhaoyun Jiang, Huayu Deng, Zhongkai Wu, Jiaqi Guo, Shizhao Sun, Vuksan Mijovic, Zijiang Yang, Jian-Guang Lou, and Dongmei Zhang. 2022. Unilayout: Taming unified sequence-to-sequence transformers for graphic layout generation. arXiv preprint arXiv:2208.08037, 2(3):4

    2022 arXiv

  10. [18]

    Zhaoyun Jiang, Jiaqi Guo, Shizhao Sun, Huayu Deng, Zhongkai Wu, Vuksan Mijovic, Zijiang James Yang, Jian-Guang Lou, and Dongmei Zhang. 2023. Layoutformer++: Conditional graphic layout generation via constraint serialization and decoding space restriction. In Proceedings of the...

    2023

  11. [19]

    Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. 2021. Constrained graphic layout generation via latent optimization. In Proceedings of the 29th ACM International Conference on Multimedia, pages 88--96

    2021

  12. [20]

    Hsin-Ying Lee, Lu Jiang, Irfan Essa, Phuong B Le, Haifeng Gong, Ming-Hsuan Yang, and Weilong Yang. 2020. Neural design network: Graphic layout generation with constraints. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, P...

    2020

  13. [21]

    Fengheng Li, An Liu, Wei Feng, Honghe Zhu, Yaoyu Li, Zheng Zhang, Jingjing Lv, Xin Zhu, Junjie Shen, Zhangang Lin, et al. 2023 a . Relation-aware diffusion model for controllable poster layout generation. In Proceedings of the 32nd ACM International Conference on Information a...

    2023

  14. [22]

    Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, and Tingfa Xu. 2019. Layoutgan: Generating graphic layouts with wireframe discriminators. arXiv preprint arXiv:1901.06767

    2019 arXiv

  15. [23]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023 b . Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355

    2023 arXiv

  16. [24]

    Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang James Yang, Jian-Guang Lou, and Dongmei Zhang. 2023. Layoutprompter: Awaken the design ability of large language models. In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  17. [25]

    Dipu Manandhar, Dan Ruta, and John Collomosse. 2020. Learning structural similarity of user interface layouts using graph networks. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXII 16, pages 730--746. Springer

    2020

  18. [26]

    Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. 2023. Embodiedgpt: Vision-language pre-training via embodied chain of thought. https://arxiv.org/abs/2305.15021)

    2023 arXiv

  19. [27]

    Songqin Nong, Jiali Zhu, Rui Wu, Jiongchao Jin, Shuo Shan, Xiutian Huang, and Wenhao Xu. 2024. Mobileflow: A multimodal llm for mobile gui agent. arXiv preprint arXiv:2407.04346

    2024 arXiv

  20. [28]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2023

  21. [29]

    Akshay Gadi Patil, Manyi Li, Matthew Fisher, Manolis Savva, and Hao Zhang. 2021. Layoutgmn: Neural graph matching for structural layout similarity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11048--11057

    2021

  22. [30]

    Jie Qin, Jie Wu, Weifeng Chen, Yuxi Ren, Huixia Li, Hefeng Wu, Xuefeng Xiao, Rui Wang, and Shilei Wen. 2024. Diffusiongpt: Llm-driven text-to-image generation system. arXiv preprint arXiv:2401.10061

    2024

  23. [31]

    Soliha Rahman, Vinoth Pandian Sermuga Pandian, and Matthias Jarke. 2021. Ruite: Refining ui layout aesthetics using transformer encoder. In Companion Proceedings of the 26th International Conference on Intelligent User Interfaces, pages 81--83

    2021

  24. [32]

    Haoxing Ren, George F Kokai, Walker J Turner, and Ting-Sheng Ku. 2020. Paragraph: Layout parasitics and device parameter prediction using graph neural networks. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1--6. IEEE

    2020

  25. [33]

    Yuichi Sasazawa and Yasuhiro Sogawa. 2024. Layout generation agents with large language models. arXiv preprint arXiv:2405.08037

    2024 arXiv

  26. [34]

    Jaejung Seol, Seojun Kim, and Jaejun Yoo. 2024. Posterllama: Bridging design ability of langauge model to contents-aware layout generation. ECCV

    2024

  27. [35]

    Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, et al. 2023. Video understanding with large language models: A survey. arXiv preprint arXiv:2312.17432

    2023

  28. [36]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023 a . Llama: Open and efficient foundation...

    2023

  29. [37]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023

  30. [38]

    Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024 a . Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. arXiv preprint arXiv:2406.01014

    2024 arXiv

  31. [39]

    Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024 b . Mobile-agent: Autonomous multi-modal mobile device agent with visual perception. arXiv preprint arXiv:2401.16158

    2024 arXiv

  32. [40]

    Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, and Heng Ji. 2025. https://arxiv.org/abs/2501.11733 Mobile-agent-e: Self-evolving mobile assistant for complex tasks . Preprint, arXiv:2501.11733

    2025 arXiv

  33. [41]

    Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. 2024 c . Genartist: Multimodal llm as an agent for unified image generation and editing. arXiv preprint arXiv:2407.05600

    2024 arXiv

  34. [42]

    Ramon Elias Weber, Caitlin Mueller, and Christoph Reinhart. 2022. Automated floorplan generation in architectural design: A review of methods and applications. Automation in Construction, 140:104385

    2022

  35. [43]

    Yuxi Wei, Zi Wang, Yifan Lu, Chenxin Xu, Changxing Liu, Hao Zhao, Siheng Chen, and Yanfeng Wang. 2024. Editable scene simulation for autonomous driving via collaborative llm-agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  36. [44]

    Ying Cao Xinru Zheng, Xiaotian Qiao and Rynson W.H. Lau. 2019. Content-aware generative modeling of graphic design layouts. ACM Transactions on Graphics (Proc. of SIGGRAPH 2019), 38

    2019

  37. [45]

    Kota Yamaguchi. 2021. Canvasvae: Learning to generate vector graphic documents. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5481--5489

    2021

  38. [46]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024

  39. [47]

    Tao Yang, Yingmin Luo, Zhongang Qi, Yang Wu, Ying Shan, and Chang Wen Chen. 2024 b . https://arxiv.org/abs/2406.02884 Posterllava: Constructing a unified multi-modal layout generator with llm . Preprint, arXiv:2406.02884

    2024 arXiv

  40. [48]

    Xuyong Yang, Tao Mei, Ying-Qing Xu, Yong Rui, and Shipeng Li. 2016. Automatic generation of visual-textual presentation layout. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 12(2):1--22

    2016

  41. [49]

    Jianhao Yuan, Shuyang Sun, Daniel Omeiza, Bo Zhao, Paul Newman, Lars Kunze, and Matthew Gadd. 2024. Rag-driver: Generalisable driving explanations with retrieval-augmented in-context learning in multi-modal large language model. arXiv preprint arXiv:2402.10828

    2024

  42. [50]

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023. https://arxiv.org/abs/2312.13771 Appagent: Multimodal agents as smartphone users . Preprint, arXiv:2312.13771

    2023 arXiv

  43. [51]

    Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. 2019. Publaynet: largest dataset ever for document layout analysis. In 2019 International conference on document analysis and recognition (ICDAR), pages 1015--1022. IEEE

    2019

  44. [52]

    Wanrong Zhu, Jennifer Healey, Ruiyi Zhang, William Yang Wang, and Tong Sun. 2024. Automatic layout planning for visually-rich documents with instruction-following models. arXiv preprint arXiv:2404.15271

    2024 arXiv

  45. [53]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  46. [54]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  47. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.