Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

AccessGuru: Leveraging LLMs to Detect and Correct Web Accessibility Violations in HTML Code

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that AccessGuru, pairing Axe-Playwright with taxonomy-driven LLM prompting and corrective re-prompting, detects and corrects syntactic, semantic, and layout accessibility violations, cutting average violation scores by up…

desk verdict Genuinely useful taxonomy and benchmark, but the headline correction numbers are per-snippet and the 50% ceiling claim only holds for syntax/layout. read the letter →

arxiv 2507.19549 v1 pith:IM4LHI5D submitted 2025-07-24 cs.SE cs.AI

Nadeen Fathallah , Daniel Hernández , Steffen Staab This is my paper · ORCID
classification cs.SEcs.AI
keywords WebaccessibilityWCAGHTMLcorrectionLargeLanguageModelspromptengineeringviolationsAxe-Playwrightsemantic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AccessGuru is a two-stage pipeline that first detects Web accessibility violations using the Axe-Playwright engine for syntactic and layout issues and a multimodal LLM for semantic issues, then corrects each violation with a taxonomy-aware prompt. The central claim is that this combination makes automatic accessibility repair practical: on a benchmark of 3,500 real-world violations sampled to match WebAIM 2025 frequencies, the method reduces average Axe-computed violation scores by up to 84% with GPT-4, compared with at most 50% for three prompting baselines. For semantic violations such as meaningless alt text or labels, two human annotators judged 53 of 55 corrections as fully resolved under WCAG 2.1. If correct, this would lower the expertise and manual effort currently required to bring Web pages into compliance with accessibility guidelines.

What carries the argument

The load-bearing machinery is AccessGuru's three-part taxonomy (syntactic: missing or malformed accessibility attributes; semantic: attributes present but meaningless; layout: visual or structural barriers) plus a two-module pipeline. AccessGuruDetect runs Axe-Playwright for syntax and layout and a multimodal LLM, prompted with the HTML and a full-page screenshot, for semantic violations. AccessGuruCorrect builds each prompt from role-play (an accessibility-expert persona), contextual information (URL, domain, WCAG guidelines, category description), and metacognitive stages (self-understanding, preliminary judgment, critical evaluation, decision confirmation, confidence), then submits the output to Axe scoring and, if the score is nonzero, issues a corrective re-prompt; the version with the lowest violation score is kept.

What would settle it

Run AccessGuru on complete Web pages with external stylesheets and dynamic content, merge the per-violation corrections into full documents, and re-run Axe plus a manual screen-reader audit on the merged pages; if the full-page violation score does not drop by roughly the promised 84%, the snippet-level metric overstates real-world correction.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a large language model, guided by a structured taxonomy of violation types and a corrective re-prompting loop, can generate corrected HTML that satisfies automated accessibility checks and produces semantically meaningful attributes. AccessGuru consistently outperforms prior prompting-only baselines on both its own benchmark and the Huang et al. dataset, and the re-prompting step itself contributes a measurable gain (0.72 to 0.84 with GPT-4). The semantic detector, which reads the HTML alongside a screenshot, surfaces 104 semantic violations across 16 pages where the earlier dataset reported none, and AccessGuru's corrections align closely with human developer rewrites (average Sentence-BERT similarity of 0.77).

Load-bearing premise

The evaluation assumes that re-scanning individual corrected HTML snippets with Axe captures real accessibility, and that the benchmark's violation mix, sampled for syntax and layout from WebAIM frequencies but with semantic violations added manually, represents real-world conditions.

Editorial extensions

If this is right

  • With GPT-4, the average violation score on 250 syntax/layout samples drops to 0.16 of its initial value, and the corrective re-prompting step alone raises the decrease from 0.72 to 0.84.
  • On 16 pages from the Huang et al. dataset, AccessGuru detects 104 semantic violations where the original dataset reported none, because the LLM-based detector can interpret image content and page context that rule-based tools ignore.
  • Human annotators judge 53 of 55 semantic corrections as fully resolving the violation under WCAG 2.1, and AccessGuru's corrected attributes match human developer wording with an average Sentence-BERT similarity of 0.77.
  • The method transfers across LLMs: even the smallest model tested, Mistral-7B, reaches a 0.82 average violation score decrease with the full loop on the paper's dataset, above the 0.50 ceiling of the best prompting baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's per-snippet scoring leaves implicit that corrected code may not remain compliant once merged back into a full page, because several checks (color contrast, table headers, link targets) depend on external stylesheets or target-page content absent from the snippet.
  • Because the 55 semantic violations in the benchmark were manually added rather than sampled from observed Web frequencies, the paper does not establish how prevalent semantic violations are relative to syntactic and layout ones in the wild.
  • A direct next experiment, which the authors list as future work, is to reconstruct fully corrected pages from the per-violation outputs and re-run both Axe and manual assistive-technology audits to test whether the 84% reduction survives full-page integration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AccessGuru, a pipeline that combines Axe-Playwright with LLM prompting to detect and correct Web accessibility violations in HTML. It introduces a three-way taxonomy (syntactic, semantic, layout), an LLM-based semantic detector, a corrective re-prompting correction loop, a newly collected dataset of 3,500 real-world violations, and a 305-violation benchmark subset sampled to resemble WebAIM frequencies. The main reported result is an 84% average violation-score decrease on the syntactic/layout subset with GPT-4, with a 96% decrease on the 55 semantic cases, compared against contextual, ReAct, and zero-shot baselines. A human developer study reports an average 0.77 Sentence-BERT similarity between AccessGuru corrections and human corrections.

Significance. If the results are validated, the paper would be a useful practical contribution: it ships a public dataset, code, prompt templates, an ablation of the re-prompting component, and a human annotation study. The taxonomy itself is a reasonable organizing device, and the idea of separating syntactic, semantic, and layout violations is likely to be adopted by follow-up work. However, the current evaluation supports the claims only at the level of isolated corrected HTML snippets, not at the level of whole Web pages; the detection evaluation lacks precision and recall; and the headline comparison against baselines is contradicted by the paper's own semantic results. These points are load-bearing for the central claims, so the contribution needs a substantive revision rather than minor polishing.

major comments (4)
  1. [Section 4.2, Algorithm 1, Section 5.5.2, Section 7] The headline 'up to 84% average violation score decrease' is computed by re-running Axe-Playwright on each corrected HTML snippet in isolation, not on the corrected Web page. This is a direct consequence of Algorithm 1, where the scoring loop evaluates only the LLM's returned snippet, and of Section 7, which states that reconstruction of a fully corrected HTML document is not implemented. Axe rules such as color-contrast, link-name, heading-order, region, and landmark checks depend on document-level context, inherited CSS, or page structure. As written, the abstract's claim to 'detect and correct Web accessibility violations in HTML code' is only supported for per-violation fragments. The authors should either add a full-document re-scan evaluation, or clearly reframe the claims to fragment-level correction and adjust the abstract and Section 6 accordingly.
  2. [Abstract, Section 6.2, Table 6] The abstract states that prior methods achieve 'at most 50%' average violation-score decrease, but Table 6 reports ReAct prompting with GPT-4 achieving 0.87 on the semantic-violation subset, and AccessGuru itself is only 0.96 there. The 50% ceiling appears to hold only for the syntactic/layout evaluation in Table 5. Because the abstract does not qualify the claim by violation category, it is misleading as written. The authors should either report the overall average across all three categories, or explicitly separate the claims per category. In addition, Tables 5 and 6 report point estimates without confidence intervals or significance tests; given the small sample sizes (250 syntactic/layout and 55 semantic), the phrase 'significantly outperforming' is not statistically supported.
  3. [Section 5.5.1, Section 6.1, Section 7] The detection evaluation for RQ1 reports only detected-violation counts and never reports precision, recall, or F1. The authors themselves note in Section 7 that they did not establish ground-truth violations for each Web page and therefore do not report recall or precision. This is a serious limitation for the detection claim, especially because the LLM-based semantic detector is described in Section 7 as occasionally hallucinating violations or misidentifying affected elements. Without a false-positive rate, the '104 semantic violations' detected in Table 4 cannot be interpreted as valid detections. The authors should annotate a ground-truth subset of the pages and report precision, recall, and F1 for both the Axe-based and LLM-based detectors.
  4. [Section 5.2, Table 3, Section 6.3] The representativeness of the semantic-violation benchmark is not established. The 305-violation subset is sampled to match WebAIM 2025 frequencies, but WebAIM reports contain essentially no semantic-violation prevalence data, and the 55 semantic violations were manually added. Consequently, the 55-sample semantic evaluation is not grounded in a real-world distribution, and the resulting 0.96 decrease and 53/55 correction rate cannot be generalized without a stated prevalence source. The authors should either provide an explicit prevalence justification for the semantic category or present the semantic results as a proof-of-concept rather than a representative benchmark.
minor comments (5)
  1. [Section 4.2, Algorithm 1] Algorithm 1 has several textual slips: the 'output' label is misspelled as 'ouput', and Section 4.2 contains the phrase 'We accessibility violation' instead of 'Each accessibility violation'. These should be corrected.
  2. [Section 5.4, Table 12] The zero-shot baseline prompt appears multiple times in Table 12 with duplicated text ('Is the following HTML code accessible?' repeated). This is presumably an artifact of the table formatting, but it should be cleaned because it is shown as the actual prompt template.
  3. [Section 6.1, Table 4] The explanation for the drop in syntax violations from 118 to 82 relies on the observation that the Huang et al. data were collected in early 2024 and Web content may have changed. While plausible, this is speculative given that the same pages were not re-crawled at the same time; the authors should state this explicitly as a hypothesis and, if possible, provide a re-crawl date.
  4. [Section 5.2, Table 3] The table reports 112 violation types for the 305-sample subset, but the text says the full dataset has 112 types. It should be clarified whether all 112 types appear in the subset or only in the full 3,500-item dataset.
  5. [Section 6.3] The human annotation protocol for semantic corrections uses a binary resolved/not-resolved judgment that maps to the original violation score (0 or 5). This coarse scoring inflates the reported decrease when any adequate correction is accepted; reporting the distribution of annotator judgments and inter-rater agreement would make the 0.96 figure more interpretable.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the headline violation-score decrease is measured by an external tool (Axe-Playwright) and by human annotators, not by the model's own outputs.

full rationale

The paper's central quantitative claims are anchored to external oracles. Syntactic and layout correction quality is scored by re-running Axe-Playwright on the corrected HTML snippet (Algorithm 1, Section 4.2; Equation 1 and 2 in Section 5.5.2), and semantic correction quality is judged by two human annotators against WCAG 2.1 and by Sentence-BERT similarity to independently produced human developer corrections (Section 6.3). The taxonomy influences the prompts and the choice of metric, but no predicted quantity reduces to a fitted parameter or to the authors' own prior results; the reported 0.84 average violation-score decrease is an empirical outcome of LLM outputs, not a number forced by construction. The mild self-referential elements are (a) the syntactic/layout benchmark instances were themselves produced by the Axe-based AccessGuruDetect scan, and (b) Algorithm 1 selects the lowest-scoring candidate among the original and two LLM outputs, so some improvement over the input is partly a consequence of the selection rule rather than a purely independent prediction. These are evaluation-validity considerations, particularly because Section 7 admits that reconstruction of a fully corrected HTML document is not implemented and corrections are scored as isolated snippets. They do not amount to circular derivation. The only self-citation that appears in a supporting role, [17], is motivational for the human developer study and is not load-bearing.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The headline result rests on two hand-made measurement choices: the linear 1-5 impact-to-score mapping and the 305-sample benchmark composition. Neither is fitted to data, but both shape the reported percentages. The evaluation also assumes that Axe-Playwright, WCAG, and human annotations define accessibility. These are domain assumptions, not circular derivations, because the correction loop is scored with external tools and human judges rather than the system's own predictions.

free parameters (2)
  • impact-to-score linear mapping = cosmetic=1, minor=2, moderate=3, serious=4, critical=5
    The average violation score decrease, the headline metric, is computed from this hand-chosen linear scale (Section 4.1, Equation 1). A nonlinear mapping would change the reported percentages, so the 84% figure is not scale-invariant.
  • benchmark sample counts = 305 total: 195 syntax, 55 layout, 55 semantic
    Subset size was chosen for controlled comparison, not from a power calculation, and the semantic category count has no WebAIM prevalence grounding (Section 5.2, Table 3).
assumptions (4)
  • domain assumption Axe-Playwright rule results are treated as ground truth for syntactic and layout violation detection and correction success.
    The score decrease in Equation 2 is computed from Axe violation scores before and after correction; if Axe misses violations or reports false positives, the metric misstates accessibility (Section 4.2, Algorithm 1; Section 5.5).
  • domain assumption WCAG 2.1 success criteria define what counts as an accessibility violation.
    The taxonomy, prompts, and human annotation all use WCAG 2.1 as the normative standard for whether a violation exists and whether a correction resolves it (Sections 3 and 5).
  • domain assumption Human annotator judgments and Sentence-BERT similarity to human corrections are valid measures of semantic correctness.
    RQ3 relies on two annotators (no inter-annotator agreement reported) and S-BERT cosine similarity as proxies for semantic quality (Section 6.3, Table 7).
  • domain assumption WebAIM 2025 top-million violation frequencies describe the real-world distribution relevant for benchmarking.
    The representative subset is sampled to match WebAIM 2025 frequencies; WebAIM's automated methods do not cover semantic violations, so the semantic proportion is not empirically grounded (Section 5.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of AccessGuru: Leveraging LLMs to Detect and Correct Web Accessibility Violations in HTML Code." pith.science (2026). https://pith.science/paper/IM4LHI5D

@misc{pith2026250719549,
  author       = {Pith},
  title        = {Pith review of: AccessGuru: Leveraging LLMs to Detect and Correct Web Accessibility Violations in HTML Code},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IM4LHI5D}},
  note         = {Machine review of arXiv:2507.19549}
}
read the original abstract

The vast majority of Web pages fail to comply with established Web accessibility guidelines, excluding a range of users with diverse abilities from interacting with their content. Making Web pages accessible to all users requires dedicated expertise and additional manual efforts from Web page providers. To lower their efforts and promote inclusiveness, we aim to automatically detect and correct Web accessibility violations in HTML code. While previous work has made progress in detecting certain types of accessibility violations, the problem of automatically detecting and correcting accessibility violations remains an open challenge that we address. We introduce a novel taxonomy classifying Web accessibility violations into three key categories - Syntactic, Semantic, and Layout. This taxonomy provides a structured foundation for developing our detection and correction method and redefining evaluation metrics. We propose a novel method, AccessGuru, which combines existing accessibility testing tools and Large Language Models (LLMs) to detect violations and applies taxonomy-driven prompting strategies to correct all three categories. To evaluate these capabilities, we develop a benchmark of real-world Web accessibility violations. Our benchmark quantifies syntactic and layout compliance and judges semantic accuracy through comparative analysis with human expert corrections. Evaluation against our benchmark shows that AccessGuru achieves up to 84% average violation score decrease, significantly outperforming prior methods that achieve at most 50%.

Figures

Figures reproduced from arXiv: 2507.19549 by the authors.

Figure 1
Figure 1. Interface of the Web page before (a) and after (b) correction with [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Overview of the AccessGuruDetect. Given a raw HTML document (left), it applies two detectors: (1) a syntax and layout detector based on the Axe-Playwright accessibility testing engine and (2) an LLM-based semantic detector. The output set of detected accessibility violations (right). . . . . . . . . Input: HTML document AccessGuru Detection Module LLM-based Semantic Detector Accessibility testing engine-based Syntax… view at source ↗
Figure 3
Figure 3. Overview of AccessGuruCorrect in: For each Web accessibility violation detected by AccessGuruDetect (examples of the input accessibility violations are shown in Tables 1 and 2), the LLM is prompted to generate the corrected code. The generated code is assigned a violation score; if the violation score remains above zero, corrective re-prompting is applied to improve the response further. Output: Select lowest violat… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Breakdown of top 10 uncorrected Web accessibility violations by AccessGuru across three LLMs (GPT-4, [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Flow-A11y: Flow-Aware Accessibility Testing

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Runtime interaction traces plus criterion-specific evidence packets let an LLM judge dynamic WCAG criteria far more accurately than unconstrained browser-agent audits.

Reference graph

Works this paper leans on

66 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    AChecker: IDI Accessibility. [n. d.]. AChecker: IDI Accessibility. http: //www.atutor.ca/achecker/. Retrieved 25-06-2024

    work page 2024

  2. [2]

    Patricia Acosta-Vargas, Mario González, and Sergio Luján-Mora. 2020. Dataset for evaluating the accessibility of the websites of selected Latin American universities. Data in brief 28 (2020), 105013

    work page 2020

  3. [3]

    Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Unified Pre-training for Program Understanding and Gen- eration. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2021, Online, June 6-11, 2021 , Kristina Toutanova, Ann...

    2021

  4. [4]

    Wajdi Aljedaani, Abdulrahman Habib, Ahmed Aljohani, Marcelo Eler, and Yunhe Feng. 2024. Does ChatGPT Generate Accessible Code? Investigating Accessibility Challenges in LLM-Generated Source Code. In Proceedings of the 21st International Web for All Conference, W4A 2024, Singapore, May 13-14, 2024. ACM, 165–176. doi:10.1145/3677846.3677854

    arXiv 2024

  5. [5]

    Almasoud and Hassan I

    Suliman K. Almasoud and Hassan I. Mathkour. 2019. Instant Adaptation Enrichment Technique to Improve Web Accessibility for Blind Users. In Proceedings of the 3rd International Conference on Information System and Data Mining, ICISDM 2019, Houston, TX, USA, April 6-8, 2019 . ACM, 159–

    work page 2019

  6. [7]

    Nye, Maarten Bosma, Hen- ryk Michalewski, David Dohan, Ellen Jiang, Carrie J

    Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Hen- ryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. Program Synthesis with Large Language Models. CoRR abs/2108.07732 (2021). arXiv:2108.07732 https: //arxiv.org/abs/2108.07732

    arXiv 2021

  7. [8]

    Ana Baptista, José Martins, Ramiro Goncalves, Frederico Branco, and Tania Rocha. 2016. Web accessibility challenges and perspectives: A systematic literature review. In 2016 11th Iberian Conference on Information Systems and Technologies (CISTI). IEEE, 1–6

    work page 2016

  8. [9]

    Grundy, Thomas Zimmermann, and Denae Ford

    Tingting Bi, Xin Xia, David Lo, John C. Grundy, Thomas Zimmermann, and Denae Ford. 2022. Accessibility in Software Practice: A Practitioner’s Perspective. ACM Trans. Softw. Eng. Methodol. 31, 4 (2022), 66:1–66:26. doi:10.1145/3503508

    doi:10.1145/3503508 2022

Show all 66 references
  1. [10]

    Ben Caldwell, Michael Cooper, Loretta Guarino Reid, Gregg Vanderhei- den, Wendy Chisholm, John Slatin, and Jason White. 2008. Web content accessibility guidelines (WCAG) 2.0. WWW Consortium (W3C) 290, 1-34 (2008), 5–12. Nadeen Fathallah, Daniel Hernández, and Steffen Staab Fig...

    2008

  2. [11]

    Carmine Cesarano, Anna Rita Fasolino, and Porfirio Tramontana. 2007. Improving Usability of Web Pages for Blinds. InProceedings of the 9th IEEE International Symposium on Web Systems Evolution, WSE 2009, 5-6 October 2007, Paris, France, Shihong Huang and Massimiliano Di Penta ...

    2007

  3. [12]

    Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xiwei Xu, Liming Zhu, Guoqiang Li, and Jinshui Wang. 2020. Unblind your apps: predicting natural-language labels for mobile GUI components by deep learning. In ICSE ’20: 42nd International Conference on Software Engineering, Seoul, ...

    2020

  4. [13]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondá de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri swear here Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, ...

    2021 arXiv

  5. [14]

    Jenny Craven. 2006. Web accessibility: A review of research and initiatives. (2006)

    2006

  6. [15]

    Giovanni Delnevo, Manuel Andruccioli, and Silvia Mirri. 2024. On the In- teraction with Large Language Models for Web Accessibility: Implications and Challenges. In 21st IEEE Consumer Communications & Networking Conference, CCNC 2024, Las Vegas, NV, USA, January 6-9, 2024 . IE...

    2024

  7. [16]

    Iyad Abu Doush and Reem Qasem. 2024. Evaluating AI-Generated Web Code for Accessibility Compliance: A Metric-Driven Approach. InProceed- ings of Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI ’24) . ACM

    2024

  8. [17]

    Nadeen Fathallah, Monika Bhole, and Steffen Staab. 2024. Empowering the Deaf and Hard of Hearing Community: Enhancing Video Captions Using Large Language Models. In Proceedings of Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DS...

    2024 doi arXiv

  9. [18]

    Nadeen Fathallah, Arunav Das, Stefano De Giorgis, Andrea Poltronieri, Peter Haase, and Liubov Kovriguina. 2024. NeOn-GPT: A Large Language Model-Powered Pipeline for Ontology Learning. In The Extended Semantic Web Conference

    2024

  10. [19]

    Mexhid Ferati and Lirim Sulejmani. 2016. Automatic Adaptation Tech- niques to Increase the Web Accessibility for Blind Users. In HCI Interna- tional 2016 - Posters’ Extended Abstracts - 18th International Conference, HCI International 2016, Toronto, Canada, July 17-22, 2016, P...

    2016 doi

  11. [20]

    Stephen M Fleming and Hakwan C Lau. 2014. How to measure metacog- nition. Frontiers in human neuroscience 8 (2014), 443

    2014

  12. [21]

    Chris D Frith. 2012. The role of metacognition in human social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences 367, 1599 (2012), 2213–2223

    2012

  13. [22]

    Xiaodong Gu, Kang Min Yoo, and Sang-Woo Lee. 2021. Response Genera- tion with Context-Aware Prompt Learning. CoRR abs/2111.02643 (2021). arXiv:2111.02643 https://arxiv.org/abs/2111.02643

    2021 arXiv

  14. [23]

    Calista Huang, Alyssa Ma, Suchir Vyasamudri, Eugenie Puype, Sayem Kamal, Juan Belza Garcia, Salar Cheema, and Michael Lutz. 2024. AC- CESS: Prompt Engineering for Automated Web Accessibility Violation Corrections. CoRR abs/2401.16450 (2024). doi:10.48550/ARXIV.2401.16450 arXiv...

    2024 doi arXiv

  15. [24]

    Andrew Kirkpatrick, Joshue O’Connor, Alastair Campbell, and Michael Cooper. 2023. Web Content Accessibility Guidelines (WCAG) 2.1 . Technical report. World Wide Web Consortium (W3C). https://www.w3.org/TR/ 2023/REC-WCAG21-20230921/

    2023

  16. [25]

    Satwik Ram Kodandaram, Utku Uckun, Xiaojun Bi, IV Ramakrishnan, and Vikas Ashok. 2024. Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibili...

    2024

  17. [26]

    Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. 2024. Better Zero-Shot Reasoning with Role-Play Prompting. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Lingui...

    2024

  18. [27]

    Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, and Zhiwei Guan

  19. [28]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of sum- maries. In Text summarization branches out. 74–81

    2004

  20. [29]

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang

  21. [30]

    Juan-Miguel López-Gil and Juanan Pereira. 2024. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Uni- versal Access in the Information Society (2024), 1–16

    2024

  22. [31]

    Andrea Mangiatordi and Marco Lazzari. 2018. Combined use of artificial intelligence and crowdsourcing to provide alternative content for images on websites. In 15th IEEE Annual Consumer Communications & Networking Conference, CCNC 2018, Las Vegas, NV, USA, January 12-15, 2018 ...

    2018

  23. [32]

    Shridhar Mehendale and Ankit Walishetti. 2024. DexAssist: A Voice- Enabled Dual-LLM Framework for Accessible Web Navigation. arXiv preprint arXiv:2411.12214 (2024)

    2024 arXiv

  24. [33]

    Microsoft. 2024. Playwright API. https://playwright.dev. Accessed: 2024-06-30

    2024

  25. [34]

    Hellendoorn, Bogdan Vasilescu, and Brad A

    Daye Nam, Andrew Macvean, Vincent J. Hellendoorn, Bogdan Vasilescu, and Brad A. Myers. 2024. Using an LLM to Help With Code Understanding. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024. ACM,...

    2024

  26. [35]

    Ahmed Njifenjou, Virgile Sucal, Bassam Jabaian, and Fabrice Lefèvre. 2024. Role-Play Zero-Shot Prompting with Large Language Models for Open- Domain Human-Machine Conversation. CoRR abs/2406.18460 (2024). doi:10.48550/ARXIV.2406.18460 arXiv:2406.18460

    2024 doi arXiv

  27. [36]

    OpenAI. 2023. GPT-4 Technical Report. CoRR abs/2303.08774 (2023). doi:10.48550/ARXIV.2303.08774 arXiv:2303.08774

    2023 doi arXiv

  28. [37]

    Achraf Othman, Amira Dhouib, and Aljazi Nasser Al Jabor. 2023. Fostering websites accessibility: A case study on the use of the Large Language Models ChatGPT for automatic remediation. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Ass...

    2023

  29. [38]

    Letícia Seixas Pereira, João Guerreiro, André Rodrigues, Tiago João Guer- reiro, and Carlos Duarte. 2024. From Automation to User Empower- ment: Investigating the Role of a Semi-automatic Tool in Social Me- dia Accessibility. ACM Trans. Access. Comput. 17, 3 (2024), 13:1–13:25...

    2024 doi

  30. [39]

    Matt Post. 2018. A call for clarity in reporting BLEU scores. arXiv preprint arXiv:1804.08771 (2018)

    2018 arXiv

  31. [40]

    Shreyas Sundara Raman, Vanya Cohen, Eric Rosen, Ifrah Idrees, David Paulius, and Stefanie Tellex. 2022. Planning with Large Language Models via Corrective Re-prompting. CoRR abs/2211.09935 (2022). doi:10.48550/ ARXIV.2211.09935 arXiv:2211.09935

    2022 doi arXiv

  32. [41]

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. 2022. DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA,...

    2022

  33. [42]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence em- beddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  34. [43]

    Richard Rutter, Patrick H Lauke, Cynthia Waddell, Jim Thatcher, Shawn Lawton Henry, Bruce Lawson, Andrew Kirkpatrick, Christian Heilmann, Michael R Burks, Bob Regan, et al. 2007. Web accessibility: Web standards and regulatory compliance . Apress

    2007

  35. [44]

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A Systematic Survey of Prompt Engi- neering in Large Language Models: Techniques and Applications. CoRR abs/2402.07927 (2024). doi:10.48550/ARXIV.2402.07927 arXiv:2402.07927

    2024 doi arXiv

  36. [45]

    Elliot Salisbury, Ece Kamar, and Meredith Ringel Morris. 2017. Toward Scalable Social Alt Text: Conversational Crowdsourcing as a Tool for Refining Vision-to-Language Technology for the Blind. In Proceedings of the Fifth AAAI Conference on Human Computation and Crowdsourcing, ...

    2017

  37. [46]

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nat. 623, 7987 (2023), 493–498. doi:10.1038/ S41586-023-06647-8

    2023

  38. [47]

    Darja Šmite, Claes Wohlin, Zane Galvin, a, and Rafael Prikladnicki. 2014. An empirically based terminology and taxonomy for global software engi- neering. Empirical Software Engineering 19 (2014), 105–153

    2014

  39. [48]

    Dat Trinh Tuan, Van-Hung Phan, et al. 2012. Checking and correcting the source code of web pages for accessibility. In 2012 IEEE RIVF Interna- tional Conference on Computing & Communication Technologies, Research, Innovation, and Vision for the Future . IEEE, 1–4

    2012

  40. [49]

    Muhammad Usman, Ricardo Britto, Jürgen Börstler, and Emilia Mendes

  41. [50]

    Glassman

    Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Ex- pectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI ’22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 20...

    2022

  42. [51]

    Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015 . IEEE Computer Society, 3156–3164. doi:10.1109/CV...

    2015

  43. [52]

    Noah Wang, Z. y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. 2024. RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playi...

    2024

  44. [53]

    Yuqing Wang and Yun Zhao. 2023. Metacognitive Prompting Improves Understanding in Large Language Models. CoRR abs/2308.05342 (2023). doi:10.48550/ARXIV.2308.05342 arXiv:2308.05342

    2023 doi arXiv

  45. [54]

    WebAIM. 2025. WAVE. https://wave.webaim.org/api/. Retrieved 2025-01- 11

    2025

  46. [55]

    WebAIM. 2025. The WebAIM Million - An Annual Accessibility Analysis of the Top 1,000,000 Home Pages . Technical Report. WebAIM.org. https: //webaim.org/projects/million/

    2025

  47. [56]

    Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C. Schmidt

  48. [57]

    Shaomei Wu, Jeffrey Wieland, Omid Farivar, and Julie Schiller. 2017. Au- tomatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, ...

    2017

  49. [58]

    Yeliz Yesilada, Giorgio Brajnik, Markel Vigo, and Simon Harper. 2012. Understanding web accessibility and its drivers. In International Cross- Disciplinary Conference on Web Accessibility, W4A ’12, Lyon, France, April 16-17, 2012, Markel Vigo, Julio Abascal, Rui Lopes, and Pao...

    2012

  50. [59]

    2019.Web Accessibility - A Founda- tion for Research, Second Edition

    Yeliz Yesilada and Simon Harper (Eds.). 2019.Web Accessibility - A Founda- tion for Research, Second Edition . Springer. doi:10.1007/978-1-4471-7440-0

    2019 doi

  51. [60]

    Yujia Zhou, Zheng Liu, Jiajie Jin, Jian-Yun Nie, and Zhicheng Dou. 2024. Metacognitive Retrieval-Augmented Large Language Models. In Proceed- ings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, Tat-Seng Chua, Chong-Wah Ngo, Ravi Kumar, Hady W. Lauw, a...

    2024

  52. [61]

    CoRR abs/2302.11382 (2023)

    A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. CoRR abs/2302.11382 (2023). doi:10.48550/ARXIV.2302.11382 arXiv:2302.11382

    2023 doi arXiv

  53. [66]

    Enclose the exact HTML snippet using the markers[START]and[END]

    Identify the affected HTML element. Enclose the exact HTML snippet using the markers[START]and[END]

  54. [67]

    ###START###

    Specify the violation name. Table 9: Role-play persona used in the AccessGuru Cor- rect module to guide the LLM in generating WCAG- compliant HTML corrections. Persona You are a Web accessibility expert with strong HTML skills and a deep commitment to fixing accessibility viol...

  55. [164]

    doi:10.1145/3325917.3325931

  56. [2017]

    Information and Software Technology 85 (2017), 43–59

    Taxonomies in software engineering: A systematic mapping study and a revised taxonomy development method. Information and Software Technology 85 (2017), 43–59

    2017

  57. [2020]

    Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, Bonnie Webber, Trevor Cohn, Yulan He, and Yang Li...

    2020

  58. [2023]

    Is Your Code Generated by ChatGPT Really Correct? Rigor- ous Evaluation of Large Language Models for Code Generation. In Advances in Neural Information Processing Systems 36: Annual Con- ference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA,...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.