Pith. sign in

REVIEW 5 major objections 6 minor 3 cited by

POML claims prompt formatting should be a separate, testable layer: on WikiTQ, style changes move GPT-3.5 Turbo accuracy from 6% to 62% and Phi-3 Medium by 4,450%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

POML is a markup language that structures LLM prompts, embeds multimodal data, and decouples formatting via stylesheets, with case studies showing strong prompt format sensitivity.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A well-built prompt engineering artifact, but the TableQA study overstates what it proves: it shows format sensitivity, not that POML improves accuracy. the 5 major comments →

arxiv 2508.13948 v1 pith:QA37IJOR submitted 2025-08-19 cs.HC cs.AIcs.CLcs.PL

Prompt Orchestration Markup Language

Yuge Zhang , Nan Chen , Jiahang Xu , Yuqing Yang This is my paper
classification cs.HC cs.AIcs.CLcs.PL
keywords prompt engineeringPOMLmarkup languageLLM format sensitivitystylesheet separationstructured promptingdata integrationdeveloper tooling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces POML, a markup language for writing LLM prompts as structured components (roles, tasks, examples) with dedicated tags for embedding documents, tables, and images. Its central proposal is to separate prompt content from prompt presentation through a CSS-like stylesheet system, so developers can change formatting without rewriting the prompt's logic. The paper argues this matters because LLMs are extremely sensitive to formatting: on a subset of WikiTableQuestions, choosing the best of 100 sampled styles instead of the worst raised GPT-3.5 Turbo's accuracy from 6% to 61.8% and Phi-3 Medium's from 0.7% to 32.2%. POML's value would be that it makes such variation systematically explorable — one 30-line base prompt plus generated stylesheets produced roughly 74,000 distinct prompt styles — and manageable in real applications. A two-day iOS agent prototype and a seven-participant user study support the broader claim that the language and its IDE tooling improve the prompt-engineering workflow.

Core claim

POML's core claim is that prompt presentation can be engineered as a first-class, content-independent layer. Using HTML-like components for intention (role, task, output-format) and data (document, table, image, folder, conversation), and a JSON stylesheet system controlling syntax (Markdown, JSON, XML, HTML), captions, list styles, chat-versus-block layout, and verbosity, a single POML source can be rendered into many presentation variants without touching the underlying content. The empirical heart is the TableQA case study: 100 randomly sampled styles applied to one base prompt on 283 WikiTQ samples produced accuracy swings of 16% to 4,450%, with optimal styles differing by model; some mo

What carries the argument

The load-bearing mechanism is POML's styling system: a CSS-like layer where a stylesheet (JSON or inline stylesheet tag) controls presentation attributes — overall syntax, table serialization format, caption style and visibility, example layout (chat vs introducer vs plain), list markers — independently of semantic markup. A three-pass rendering pipeline (parse into components, build an intermediate representation, serialize through a target writer) makes the decoupling operational, allowing one base prompt to be rendered into tens of thousands of styles and into different output formats (Markdown, JSON, plain text) from the same representation.

Load-bearing premise

The load-bearing premise is that changing a prompt's presentation never changes its meaning: POML assumes stylesheets can alter syntax, captions, and layout while the model still sees the same underlying content.

What would settle it

Render the same POML source under two stylesheets and record the exact final strings sent to the model. If two styles that produce identical strings nevertheless lead to different answers, the styling system is not the causal variable; if two styles that are meant to differ only in presentation change which content is visible (e.g., a hidden caption or a collapsed table drops a fact the model needs), the decoupling premise fails. A direct test would compare model accuracy on style pairs whose rendered strings differ only in whitespace or order versus pairs that delete or reorder content.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If styling is decoupled, a single prompt's content can be re-rendered for different target models, and the TableQA results indicate the optimal rendering is model-specific rather than universal.
  • Prompt teams can treat formatting as a tunable search dimension: the paper demonstrates 73,926 styles generated from one 30-line base POML file, so systematic experiments can replace hand-maintained parallel copies.
  • Format-sensitivity findings imply that published prompt-engineering results should specify or control presentation, since a style change can swing accuracy more than many content changes.
  • The PomLink 'Auto' table format shows the same data can be serialized differently per backend model, making style adaptation a runtime deployment decision rather than a manual rewrite.
  • Because POML files are text and modular, they fit standard version-control workflows, easing collaboration on prompt changes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that prompt-styling search can be automated: the stylesheet representation is machine-readable, so an optimizer could sample and score styles the way the paper did manually, and the model-specific rankings suggest the objective function differs per model.
  • The decoupling premise implies a testable boundary: if a style change ever alters token order, emphasis, or which content is visible, it is no longer pure presentation. That boundary is worth probing because hidden captions and collapsed tables can delete information.
  • A testable extension is to apply the same three-pass rendering idea beyond single-turn text prompts — for example, to agent system prompts or multimodal contexts, where the content tree stays stable but the target writer changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces POML (Prompt Orchestration Markup Language), an HTML-like markup language for authoring LLM prompts. POML provides structural components (<role>, <task>, <example>), data components for documents/tables/images, a CSS-inspired styling system that decouples content from presentation, and a templating engine. The authors also describe an IDE extension (VSCode IntelliSense, live preview, diagnostics) and Python/Node SDKs. Validation consists of two case studies: PomLink, an iOS agent prototype built in two days using POML, and TableQA, a study of 100 randomly sampled prompt styles applied to 283 WikiTQ samples across 8 LLMs. In addition, a user study with 7 participants assesses usability across five tasks. The paper claims that POML reduces formatting sensitivity and improves accuracy performance, and that its tooling improves developer workflow.

Significance. If the empirical claims were fully supported, POML would be a meaningful contribution to prompt-engineering tooling: the design is comprehensive, the implementation appears substantial (14.8k lines, 37 components, 115 test cases), and the detailed appendices (style search space, three-pass rendering architecture, IR specifications) are valuable. The PomLink case study and the qualitative user-study feedback suggest that POML's data components and live preview are genuinely useful in practice. However, the central quantitative claim—that POML's styling system improves accuracy or mitigates format sensitivity—is not established by the TableQA experiment as presented. The experiment shows, at best, that LLM performance is sensitive to prompt formatting, a known phenomenon, and that POML can generate many formatting variants. The paper overstates this as POML's impact on accuracy. The user study, while informative, is small and lacks a comparison baseline. With targeted revisions to the evaluation and a more measured interpretation, the paper could be a solid systems/HCI contribution.

major comments (5)
  1. [§7.2, Table 2] The TableQA experiment has no baseline. The reported 'accuracy improvements' of 929% and 4450% are relative differences between the worst and best of 100 randomly sampled styles, not improvements of POML over plain-text prompts or existing markup tools. The abstract and contribution (3) claim POML 'demonstrates its impact on accuracy performance,' but the design cannot support that. To support such a claim, the paper must include a baseline condition (e.g., the same TableQA task written as plain text or in an existing format such as ChatML/PromptML) and compare POML-rendered prompts against it. Without this, the paper should be reworded to claim only that styling variations affect LLM accuracy, not that POML improves it.
  2. [§7.2, Table 1 and §7.1 'Auto' option] The 'optimal' styles in Table 1 are selected post hoc on the same 283 samples used to compute all accuracy numbers. There is no held-out evaluation of the selected styles. This matters because the PomLink 'Auto' option (§7.1) consumes these fitted results to choose table formats for new prompts. The self-correlation metric (Table 2) is a split-half stability measure, not a validation of the top style. The authors should split the data, select styles on a training subset, and evaluate on a held-out subset; they should also report whether the 'Auto' choice outperforms a fixed default or random style on unseen data.
  3. [Table 2, Phi-3 Medium row] The extreme relative improvements are statistically fragile. Phi-3 Medium's minimum accuracy is 0.007, which on 283 samples corresponds to roughly 2 correct answers, so the denominator is indistinguishable from zero. Reporting a 4450% improvement from such a baseline is misleading. The paper should report absolute differences with confidence intervals (e.g., Wilson intervals) and should avoid presenting min/max extremes from 100 random draws as evidence of POML's benefit. Similarly, Table 6 performs many Mann-Whitney U tests without multiple-comparison correction; the p-values should be adjusted (e.g., Benjamini-Hochberg) and effect sizes reported.
  4. [§8 and §9.2] The user study's central claims are limited by the acknowledged small sample (N=7), the restricted 90-minute sessions, and the authors' own admission that most participants 'did not deeply engage with complex stylesheets' (§9.2). The paper nevertheless describes the study as a 'formal user study' and uses it to support POML's effectiveness. I would ask the authors to either temper the language (e.g., 'pilot study') or add complementary evidence, such as a comparison task with plain-text prompts or with another markup tool. The current design cannot distinguish POML's effect from the Hawthorne effect of a new tool in a lab setting.
  5. [§4.3, DG3] The design goal DG3 assumes that prompt presentation can be decoupled from content without changing the underlying meaning the model sees. The paper operationalizes this through stylesheets, but does not verify content invariance: changing syntax, caption styles, or list styles also changes tokenization, ordering, and emphasis, which could alter semantics. This is not necessarily a fatal flaw, but it is a load-bearing assumption for the claim that POML 'reduces formatting sensitivity' rather than simply shifting it. The TableQA study could partially address this by comparing styles that differ only in presentation while holding the exact words constant, but the current analysis does not isolate this. I recommend adding an explicit test or at least a careful discussion of this limitation.
minor comments (6)
  1. [Table 2 caption] The term 'Self-corr.' is used without a definition in the main text. It is defined only later in Appendix E. Please define it in the caption or in §7.2.
  2. [Figure 6] In Figure 6, the diagnostic message shows 'captionStyle should be one of header, bold, plain, hidden, not "head"' but the list of valid values is inconsistent with the text in §4.3, which mentions 'header, bold, plain, hidden'. This is a small documentation inconsistency that could confuse users.
  3. [Figure 10] The style search space diagram (Figure 10) is dense and hard to read, especially the labels for options (a) through (e). A table listing the discrete choices with their cardinalities would improve reproducibility and clarity.
  4. [§7.2] The paper says 'we randomly sampled 100 styles without replacement for evaluation' but does not report the random seed or whether the same styles were used across all models. Please clarify whether the 100 styles are identical across models and provide the seed for reproducibility.
  5. [Appendix E, Table 6] Table 6 uses arrows (D, ✗, –) with p-values in cells. The notation is not intuitive; a legend would help. Also, the interaction-effect rows are valuable but are not discussed in the main text.
  6. [Throughout] The paper uses the phrase 'rigorous empirical evaluation' (e.g., §7 and §10). Given the limitations of the TableQA and user studies, I recommend a more measured description such as 'initial empirical exploration' or 'case-study-based evaluation.'

Circularity Check

0 steps flagged

No significant circularity: POML's claims rest on new implementations, a user study, and fresh accuracy measurements; the TableQA 'optimal style' lookups are descriptive fits, not predictions.

full rationale

The paper's central derivation is self-contained. The TableQA study (Section 7.2) samples 100 styles, measures accuracy on 283 WikiTQ examples, and reports min/max values; the 'optimal style' entries in Table 1 are the best among those samples, i.e., descriptive maxima, not predicted values. PomLink's 'Auto' table-format option then looks up those empirically determined best formats (Section 7.1), which is a fitted input used as a feature, not a prediction of unseen style performance. The other load-bearing evidence for POML's utility is the PomLink implementation, the seven-participant user study (Section 8), and the IDE/SDK toolkit description, none of which reduce by construction to the claims being validated. Self-citations (e.g., [18], [48], [103]) appear as background or as design motivation; the format-sensitivity premise is also supported by independent external citations [73,76,89] and by the paper's own fresh measurements, so these self-citations are not load-bearing. No equation, definition, or fitted parameter is renamed as a prediction. Statistical and validity weaknesses in TableQA (no baseline, extreme min/max without confidence intervals) concern experimental rigor, not circularity, and are left out of this score.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on the assumption that formatting is separable from content, that the evaluated models and tasks are representative, and that the rendering pipeline is faithful. The only fitted quantities are the per-model optimal styles and the hand-constructed style search space. No new physical or conceptual entities are postulated beyond the POML language itself.

free parameters (2)
  • Model-specific optimal styling configurations (Table 1) = e.g., GPT-3.5 Turbo: Overall Syntax=Markdown, Table Syntax=TSV, Instruction Header=Plain-Upper-Colon, Example Caption=He
    These configurations are the best among 100 randomly sampled styles evaluated on 283 WikiTQ samples, selected post hoc for each model. They are descriptive fits, not validated predictions on held-out data.
  • Style search space dimensionality (Figure 10) = 74k combinations from hand-chosen style options
    The set of style dimensions and candidate values (e.g., caption styles, table syntaxes, example body formats) is chosen by the authors rather than derived from theory, and the 100 evaluated styles are a random subset of this space.
axioms (4)
  • domain assumption LLM prompt formatting sensitivity is a real, general phenomenon that can be measured on WikiTQ.
    Invoked in Section 2.1 and Section 7.2 via cited references and the TableQA experiment. If format sensitivity were negligible or task-specific, the central motivation for the styling system weakens.
  • domain assumption The 8 selected low-cost LLMs and the 283 WikiTQ samples are representative of LLM prompt behavior.
    Used in Section 7.2 to draw conclusions about model-specific optimal styles. The sample is small and the paper acknowledges the models are all low-cost as of February 2025.
  • domain assumption A single POML template combined with stylesheets can faithfully generate all 74k prompt variations without hidden interactions between content and styling.
    Assumed throughout Section 7.2, where the rendering pipeline is trusted to preserve content while changing presentation. Section 6 describes the architecture but provides no independent verification of rendering faithfulness.
  • domain assumption The user study's think-aloud sessions and self-reported metrics capture real workflow benefits.
    Section 8 relies on qualitative feedback from 7 participants, with no control condition and minimal intervention from facilitators.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompt Orchestration Markup Language." pith.science (2026). https://pith.science/paper/QA37IJOR

@misc{pith2026250813948,
  author       = {Pith},
  title        = {Pith review of: Prompt Orchestration Markup Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QA37IJOR}},
  note         = {Machine review of arXiv:2508.13948}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) require sophisticated prompting, yet current practices face challenges in structure, data integration, format sensitivity, and tooling. Existing methods lack comprehensive solutions for organizing complex prompts involving diverse data types (documents, tables, images) or managing presentation variations systematically. To address these gaps, we introduce POML (Prompt Orchestration Markup Language). POML employs component-based markup for logical structure (roles, tasks, examples), specialized tags for seamless data integration, and a CSS-like styling system to decouple content from presentation, reducing formatting sensitivity. It includes templating for dynamic prompts and a comprehensive developer toolkit (IDE support, SDKs) to improve version control and collaboration. We validate POML through two case studies demonstrating its impact on complex application integration (PomLink) and accuracy performance (TableQA), as well as a user study assessing its effectiveness in real-world development scenarios.

Figures

Figures reproduced from arXiv: 2508.13948 by Jiahang Xu, Nan Chen, Yuge Zhang, Yuqing Yang.

Figure 2
Figure 2. Figure 2: An example illustrating POML’s structured markup [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Examples of POML data components demonstrating integration of diverse data types (§ 4.2). (a) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Demonstrating POML styling capabilities (§ 4.3). (a) Default rendering of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Example of POML’s templating engine (§ 4.4): using [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: Using POML Software Development Kits (SDKs) to integrate into programming workflows. (a) JavaScript/TypeScript SDK example using JSX-like tagged template literals. (b) Python SDK example using a context manager approach. logs and streams the LLM’s response in real-time, enabling imme￾diate observation and early abortion of the generation in case it starts to deviate from the desired outcome ( [PITH_FULL_I… view at source ↗
Figure 9
Figure 9. Figure 9: PomLink iOS interface, powered by POML. (1) Main [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization of the prompt styling search space [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Frequency of POML component usage across user [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: The POML three-pass rendering architecture: (1)->(2) The Parser transforms POML markup into React JSX components, [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Correlation matrix showing relationships be [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗
Figure 13
Figure 13. Figure 13: Example of POML usage for the TableQA case [PITH_FULL_IMAGE:figures/full_fig_p024_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A Language for Describing Agentic LLM Contexts

    cs.AI 2026-05 accept novelty 7.0

    ACDL is a language for specifying the structure and dynamics of LLM input contexts in agent systems using constructs for roles, dynamic content, time references, and conditional structures.

  2. A Prompt-Aware Structuring Framework for Reliable Reuse of AI-Generated Content in the Agentic Web

    cs.AI 2026-05 unverdicted novelty 5.0

    A framework structures AI-generated content with prompt-aware metadata and verifiable credentials to support reliable assessment and reuse by agents.

  3. When the Code Autopilot Breaks: Why LLMs Falter in Embedded Machine Learning

    cs.SE 2025-09 conditional novelty 5.0

    LLM-based sketch generation for embedded ML is fragile, with success rates below 40%, and prompt structure alone can swing outcomes from 15% to 30%.

Reference graph

Works this paper leans on

107 extracted references · 45 canonical work pages · cited by 3 Pith papers

  1. [1]

    Azure AI. 2024. Evaluate your Generative AI application locally with the Azure AI Evaluation SDK. https://learn.microsoft.com/en-us/azure/ai-foundry/how- to/develop/evaluate-sdk

    2024

  2. [2]

    Cursor AI. 2025. Generate Cursor Project Rule (.mdc). https://cursor.directory/ generate

    2025

  3. [3]

    LangChain AI. 2022. LangChain. https://www.langchain.com/

    2022

  4. [4]

    LangChain AI. 2023. LangSmith. https://www.langchain.com/langsmith

    2023

  5. [5]

    Angular. 2010. Introduction to components and templates. https://v17.angular. io/guide/architecture-components

    2010

  6. [6]

    Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. https:// assets.anthropic.com/m/61e7d27f8c8f5919/original/Claude-3-Model-Card.pdf. Preprint, April 2025, arXiv POML Team

    2024

  7. [7]

    Anthropic. 2024. Use XML tags to structure your prompts. https://docs.anthropic. com/en/docs/build-with-claude/prompt-engineering/use-xml-tags

    2024

  8. [8]

    anysphere. 2024. priompt. https://github.com/anysphere/priompt

    2024

  9. [9]

    Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, and Elena L Glassman. 2024. ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–18

    2024

  10. [10]

    AutosseyAI. 2024. prxmpt. https://github.com/AutosseyAI/prxmpt

    2024

  11. [11]

    Stephen H Bach, Victor Sanh, Zheng-Xin Yong, Albert Webson, Colin Raffel, Nihal V Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, et al. 2022. Promptsource: An integrated development environment and reposi- tory for natural language prompts. arXiv preprint arXiv:2202.01279 (2022)

    Pith/arXiv arXiv 2022

  12. [12]

    Tim Berners-Lee. 1991. Re: status. Re: X11 BROWSER for WWW. https://lists. w3.org/Archives/Public/www-talk/1991SepOct/0003.html

    1991

  13. [13]

    Bradybry. 2023. ChatXML: A proposal for a structured LLM prompt method. https://github.com/Bradybry/chatXML

    2023

  14. [14]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    Pith/arXiv arXiv 2020

  15. [15]

    Fernando Miguel Carvalho, Luis Duarte, and Julien Gouesse. 2020. Text web templates considered harmful. In Web Information Systems and Technologies: 15th International Conference, WEBIST 2019, Vienna, Austria, September 18–20, 2019, Revised Selected Papers 15 . Springer, 69–95

    2020

  16. [16]

    ChatPDF. 2024. ChatPDF. https://www.chatpdf.com/

    2024

  17. [17]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    Pith/arXiv arXiv 2021

  18. [18]

    Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. 2024. VisEval: A Benchmark for Data Visualization in the Era of Large Language Models. IEEE Transactions on Visualization and Computer Graphics (2024)

    2024

  19. [19]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)

    Pith/arXiv arXiv 2021

  20. [20]

    Hai Dang, Lukas Mecke, Florian Lehmann, Sven Goller, and Daniel Buschek

  21. [21]

    Google Deepmind. 2024. Introducing Gemini 2.0: our new AI model for the agentic era. https://blog.google/technology/google-deepmind/google-gemini- ai-update-december-2024/

    2024

  22. [22]

    DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL]

    Pith/arXiv arXiv 2024

  23. [23]

    Ant Design. 2015. Ant Design. https://ant.design/

    2015

  24. [24]

    Michael Desmond and Michelle Brachman. 2024. Exploring Prompt Engineering Practices in the Enterprise. arXiv preprint arXiv:2403.08950 (2024)

    Pith/arXiv arXiv 2024

  25. [25]

    Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason E Weston. [n. d.]. Chain-of-Verification Reduces Hallucination in Large Language Models. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models

    2024

  26. [26]

    MDN Web Docs. 2025. CSS: Cascading Style Sheets. https://developer.mozilla. org/en-US/docs/Web/CSS

    2025

  27. [27]

    MDN Web Docs. 2025. HTML: HyperText Markup Language. https://developer. mozilla.org/en-US/docs/Web/HTML

    2025

  28. [28]

    MDN Web Docs. 2025. JavaScript. https://developer.mozilla.org/en-US/docs/ Web/JavaScript

    2025

  29. [29]

    Django documentation. 2024. Templates. https://docs.djangoproject.com/en/5. 1/topics/templates/

    2024

  30. [30]

    edspencer. 2024. mdx-prompt. https://github.com/edspencer/mdx-prompt

    2024

  31. [31]

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2024. Talk like a Graph: Encoding Graphs for Large Language Models. In The Twelfth International Conference on Learning Representations

    2024

  32. [32]

    fixie ai. 2024. AI.JSX — The AI Application Framework for Javascript. https: //github.com/fixie-ai/ai-jsx/

    2024

  33. [33]

    gensx inc. 2024. gensx. https://github.com/gensx-inc/gensx

    2024

  34. [34]

    Handlebars. 2011. Handlebars. https://handlebarsjs.com/

    work page 2011

  35. [35]

    Jia He, Mukund Rungta, David Koleczek, Arshdeep Sekhon, Franklin X Wang, and Sadid Hasan. 2024. Does Prompt Formatting Have Any Impact on LLM Performance? arXiv preprint arXiv:2411.10541 (2024)

    Pith/arXiv arXiv 2024

  36. [36]

    Xinyi He, Yihao Liu, Mengyu Zhou, Yeye He, Haoyu Dong, Shi Han, Zejian Yuan, and Dongmei Zhang. 2025. TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models. arXiv:2503.04396 [cs.CL] https://arxiv.org/abs/2503.04396

    Pith/arXiv arXiv 2025

  37. [37]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Understanding. In International Conference on Learning Representations

    2021

  38. [38]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guil- laume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Tev...

    Pith/arXiv arXiv 2024

  39. [39]

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=VTF8yNQM66

    work page 2024

  40. [40]

    Jinja. 2022. Jinja – Jinja documentation. https://jinja.palletsprojects.com/en/3.1. x/

    work page 2022

  41. [41]

    latitude dev. 2024. latitude-llm. https://github.com/latitude-dev/latitude-llm

    work page 2024

  42. [42]

    Avraham Leff and James T Rayfield. 2001. Web-application development using the model/view/controller design pattern. In Proceedings fifth ieee international enterprise distributed object computing conference . IEEE, 118–127

    work page 2001

  43. [43]

    leforestier. 2016. yattag. https://github.com/leforestier/yattag

    work page 2016

  44. [44]

    Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2024. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems 36 (2024)

    work page 2024

  45. [45]

    Demiao Lin. 2024. Revolutionizing Retrieval-Augmented Generation with En- hanced PDF Structure Recognition. arXiv:2401.12599 [cs.AI] https://arxiv.org/ abs/2401.12599

    Pith/arXiv arXiv 2024

  46. [46]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruction Tuning. arXiv:2304.08485 [cs.CV] https://arxiv.org/abs/2304.08485

    Pith/arXiv arXiv 2023

  47. [47]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2023. AgentBench: Evaluating LLMs as Agents. arXiv preprint arXiv: 2308.03688 (2023)

    Pith/arXiv arXiv 2023

  48. [48]

    Yuanye Liu, Jiahang Xu, Li Lyna Zhang, Qi Chen, Xuan Feng, Yang Chen, Zhongxin Guo, Yuqing Yang, and Peng Cheng. 2025. Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimiza- tion. arXiv:2502.04295 [cs.CL] https://arxiv.org/abs/2502.04295

    Pith/arXiv arXiv 2025

  49. [49]

    Arvid Lunnemark. 2023. Prompt Design. https://arvid.xyz/posts/prompt- design/

    work page 2023

  50. [50]

    mannaandpoem. 2025. Building OpenManus as a Service. https://openmanus. org/

    work page 2025

  51. [51]

    mattnigh. 2023. ChatGPT3-Free-Prompt-List: A free guide for learning to create ChatGPT3 Prompts. https://github.com/mattnigh/ChatGPT3-Free-Prompt-List

    work page 2023

  52. [52]

    Microsoft. 2016. Language Server Protocol. https://microsoft.github.io/language- server-protocol/

    work page 2016

  53. [53]

    Microsoft. 2023. Guidance. https://github.com/guidance-ai/guidance

    work page 2023

  54. [54]

    Microsoft. 2023. Semantic Kernel. https://github.com/microsoft/semantic- kernel

    work page 2023

  55. [55]

    Microsoft. 2024. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv:2404.14219 [cs.CL] https://arxiv.org/abs/2404. 14219

    Pith/arXiv arXiv 2024

  56. [56]

    Yasuhiko Minamide. 2005. Static approximation of dynamically generated web pages. In Proceedings of the 14th international conference on World Wide Web . 432–441

    work page 2005

  57. [57]

    Aditi Mishra, Utkarsh Soni, Anjana Arunkumar, Jinbin Huang, Bum Chul Kwon, and Chris Bryan. 2023. Promptaid: Prompt exploration, perturbation, testing and iteration using visual analytics for large language models. arXiv preprint arXiv:2304.01964 (2023)

    Pith/arXiv arXiv 2023

  58. [58]

    MUI. 2014. Material UI. https://mui.com/

    work page 2014

  59. [59]

    OpenAI. 2020. Playground. https://platform.openai.com/playground

    work page 2020

  60. [60]

    OpenAI. 2023. ChatGPT. https://chatgpt.com/

    2023

  61. [61]

    OpenAI. 2024. GPT-3.5 Turbo. https://platform.openai.com/docs/models/gpt- 3.5-turbo

    work page 2024

  62. [62]

    OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774

    Pith/arXiv arXiv 2024

  63. [63]

    OpenAI. 2024. GPT-4o System Card. https://cdn.openai.com/gpt-4o-system- card.pdf

    work page 2024

  64. [64]

    Panupong Pasupat and Percy Liang. 2015. Compositional Semantic Parsing on Semi-Structured Tables. In Proceedings of the 53rd Annual Meeting of the Associ- ation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . 1470–1480

    work page 2015

  65. [65]

    pezzolabs. 2024. pezzo. https://github.com/pezzolabs/pezzo

    work page 2024

  66. [66]

    {Structured} Prompt. 2023. {Structured} Prompt. https://structuredprompt.com/

    work page 2023

  67. [67]

    promptfoo. 2023. promptfoo. https://www.promptfoo.dev/

    work page 2023

  68. [68]

    PromptML. 2023. PromptML (Prompt Markup Language). https://www. promptml.org/

    work page 2023

  69. [69]

    React. 2013. Introducing JSX. https://legacy.reactjs.org/docs/introducing-jsx. html

    work page 2013

  70. [70]

    React. 2013. React. https://react.dev/

    work page 2013

  71. [71]

    React. 2024. Server Components. https://react.dev/reference/rsc/server- components. Prompt Orchestration Markup Language Preprint, April 2025, arXiv

    work page 2024

  72. [72]

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927 (2024)

    Pith/arXiv arXiv 2024

  73. [73]

    Abel Salinas and Fred Morstatter. 2024. The butterfly effect of altering prompts: How small changes and jailbreaks affect large language model performance. arXiv preprint arXiv:2401.03729 (2024)

    Pith/arXiv arXiv 2024

  74. [75]

    Tobias Schnabel and Jennifer Neville. 2024. Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimization. arXiv:2404.02319 [cs.CL] https://arxiv.org/abs/2404.02319

    Pith/arXiv arXiv 2024

  75. [76]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations

    work page 2024

  76. [77]

    Azure AI Services. 2024. Chat Markup Language ChatML (Preview). https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chat- markup-language

    work page 2024

  77. [78]

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. ALFWorld: Aligning Text and Em- bodied Environments for Interactive Learning. InProceedings of the International Conference on Learning Representations (ICLR) . https://arxiv.org/abs/2010.03768

    Pith/arXiv arXiv 2021

  78. [79]

    Hendrik Strobelt, Albert Webson, Victor Sanh, Benjamin Hoover, Johanna Beyer, Hanspeter Pfister, and Alexander M Rush. 2022. Interactive and visual prompt engineering for ad-hoc task adaptation with large language models. IEEE transactions on visualization and computer graphics 29, 1 (2022), 1146–1156

    work page 2022

  79. [80]

    Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. 2024. Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study. arXiv:2305.13062 [cs.CL] https: //arxiv.org/abs/2305.13062

    Pith/arXiv arXiv 2024

  80. [81]

    Michiaki Tatsubori and Toyotaro Suzumura. 2009. HTML templates that fly: a template engine approach to automated offloading from server to client. In Proceedings of the 18th international conference on World wide web . 951–960

    work page 2009

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.