Pith Number

pith:OCST54C7

pith:2026:OCST54C7AW75KG5HL7ZXHVXVV6

not attested not anchored not stored refs resolved

Recommending Usability Improvements with Multimodal Large Language Models

Alexander Felfernig, Damian Garber, Manuel Henrich, Sebastian Lubos, Viet-Man Le

Multimodal large language models identify usability issues in screen recordings and suggest ranked fixes.

arxiv:2604.25420 v1 · 2026-04-28 · cs.SE · cs.HC

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{OCST54C7AW75KG5HL7ZXHVXVV6}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

The results demonstrate the potential of our approach to provide low-effort usability improvement recommendations. This makes it a promising complement to traditional evaluation methods, especially in settings with limited access to usability experts.

C2weakest assumption

That the MLLM can reliably identify real usability issues and produce actionable, correctly ranked recommendations from only limited context and screen recordings, and that feedback from a small group of software engineers in the user study is sufficient to establish practical usefulness.

C3one line summary

Multimodal LLMs can detect usability issues from screen recordings, explain them via Nielsen's heuristics, and rank improvement recommendations, with engineer feedback indicating practical usefulness for teams lacking experts.

References

33 extracted · 33 resolved · 3 Pith anchors

[1] Abdulaziz Alshayban and Sam Malek. 2022. AccessiText: automated detection of text accessibility issues in Android apps. InProceedings of the 30th ACM Joint European Software Engineering Conference and 2022 · doi:10.1145/3540250.3549118

[2] Moreno, María-Isabel Sánchez-Segura, and Ahmed Sef- fah 2013 · doi:10.1109/tse.2013.29

[3] Castro, Ignacio Garnica, and Luis A 2022

[4] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. MIND2WEB: towards a generalist agent for the web. InProceedings of the 37th International Conferenc 2023

[5] Peitong Duan, Jeremy Warner, Yang Li, and Bjoern Hartmann. 2024. Generating Automatic Feedback on UI Mockups with Large Language Models. InProceedings of the 2024 CHI Conference on Human Factors in Co 2024 · doi:10.1145/3613904.3642782

Receipt and verification

First computed	2026-06-12T01:09:28.100026Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

70a53ef05f05bfd51ba75ff373d6f5af9632c58ed749d16891bc0bac43fe0cca

Aliases

arxiv: 2604.25420 · arxiv_version: 2604.25420v1 · doi: 10.48550/arxiv.2604.25420 · pith_short_12: OCST54C7AW75 · pith_short_16: OCST54C7AW75KG5H · pith_short_8: OCST54C7

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/OCST54C7AW75KG5HL7ZXHVXVV6 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 70a53ef05f05bfd51ba75ff373d6f5af9632c58ed749d16891bc0bac43fe0cca

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "99863a0a8e037dac0600849cbe98cc001a2410d94651c0f469003aa5eda86e9f",
    "cross_cats_sorted": [
      "cs.HC"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.SE",
    "submitted_at": "2026-04-28T09:29:13Z",
    "title_canon_sha256": "b2711cfca3e279f68c6175bbd4b9e0cd989c304c647686ec1f6cda82335a7231"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2604.25420",
    "kind": "arxiv",
    "version": 1
  }
}