REVIEW 4 major objections 5 minor 1 references
The first broad survey of reviews on Meta and Steam VR stores finds accessibility is mentioned in only 0.078% of 1.37 million reviews, and that reported issues are mostly negative — dominated by motion sickness and hearing impairment.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
VR store reviews rarely discuss accessibility, and the few that do are mostly negative and dominated by motion sickness rather than clearly disability-specific issues.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection A useful first look at VR accessibility reviews, but the headline numbers rest on counting motion sickness as a disability without evidence, so treat the stats with care. the 4 major comments →
Investigating VR Accessibility Reviews for Users with Disabilities: A Qualitative Analysis
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper's central claim, on its own terms: measured through the reviews users actually post, VR accessibility is rarely discussed and predominantly reported as failure. Of 1,367,419 reviews of 100 Meta and Steam VR apps, only 1,076 (0.078%) qualified as accessibility reviews; 44.9% were negative, 37.9% positive, 17.1% neutral. Motion sickness dominates (818 reviews, 76%) and is the only category with positive feedback outweighing negative; hearing impairment (136 reviews, 90.4% negative) concentrates in social VR. Reported causes split into design barriers (no teleportation, blurry visuals, missing subtitles, flashing lights, forced two-handed controls), user adaptation (symptoms fading wi
What carries the argument
The load-bearing machinery is a three-stage corpus pipeline: purposive sampling of 100 VR apps (top, most popular, lowest-rated on Meta and Steam); string-matching of 1.37 million reviews against 294 keywords derived from the W3C XAUR guidelines and WHO/ICF disability glossaries; and manual inspection under explicit inclusion/exclusion criteria. The classification schema carrying the results is XAUR's six disability categories (auditory, cognitive, neurological, physical, speech, visual), extended inductively with an 'Other' theme, each review tagged positive/negative/neutral and mapped to one of six app genres. This turns free-text complaints into the countable claims — prevalence per disab
Load-bearing premise
The load-bearing premise is that a review mentioning motion sickness counts as an accessibility review written by a user with a disability — motion sickness is a transient symptom that also affects non-disabled players, and 818 of the 1,076 reviews (76%) rest on this without any check that the reviewer has a disability.
What would settle it
Re-run the filter with motion sickness separated out and require reviewers to self-identify a disability (as the paper effectively does for hearing and speech). If the corpus shrinks from 1,076 to roughly 250 reviews — as the hearing-, speech-, epilepsy-, and physical-disability counts suggest — then 'VR accessibility reviews are predominantly under-supported' would need restating as 'accessibility reviews are almost nonexistent except for comfort complaints about motion sickness,' and the 0.078% prevalence figure would fall by about three-quarters.
If this is right
- If VR practitioners rely on app-store reviews as their accessibility signal, the 0.078% rate will systematically under-invest accessibility work; the paper's cause catalogue — teleportation, comfort/vignette settings, subtitles, text chat, photosensitivity warnings, single-handed modes — is the to-do list users themselves supply.
- Communication-heavy social VR apps are where accessibility failure concentrates, and the paper shows that software updates (such as the Easy Anti-Cheat introduction) can silently regress accessibility by disabling third-party caption and text-to-speech mods — a regression risk for any platform that restricts modding.
- The low review counts cannot be read as satisfaction: the authors argue that disabled users may stay silent, fear discrimination, or report issues through community channels like Discord, so the scarcity finding establishes only that app stores are a thin feedback channel, not that VR is accessible.
- Because motion sickness is the largest category and its reported causes are mostly design choices (locomotion mode, turning style, frame rate, comfort settings), the largest share of VR accessibility complaints is addressable by configurable locomotion and comfort options rather than by specialized assistive technology.
Where Pith is reading between the lines
- Editorial inference: the counting frame may understate the true accessibility burden. The paper excludes non-disabled users who mention accessibility needs (e.g., seated play) and notes that disabled users may avoid reviewing, so the 0.078% figure is likely a strong lower bound; analyzing Discord communities or other support channels would probably surface many more issues than app stores do.
- Editorial inference: the motion-sickness-as-disability premise is the load-bearing pivot. If future work classifies motion sickness as a comfort complaint rather than a disability experience, the corpus drops to roughly a quarter of its size and concentrates in hearing and speech impairments — a materially different picture of which disability groups VR most fails.
- Editorial inference: testable extension — the paper's cause list predicts specific feature absences. A follow-up check of the 100 sampled apps for the top requested features (teleportation, savable comfort settings, subtitles in social apps, photosensitivity warnings) would validate whether 'predominantly under-supported' holds at the feature level, especially in the lowest-rated apps.
- Editorial inference: the large share of positive motion-sickness reviews — users praising an app for not making them sick — suggests comfort is a marketable differentiator; app stores could plausibly surface comfort and accessibility settings as searchable metadata, helping disabled users decide and giving developers an incentive to implement the missing features.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a large-scale analysis of user reviews from the Meta and Steam stores to characterize accessibility issues reported by users with disabilities in VR applications. The authors collected 1,367,419 reviews from 100 VR apps, applied a keyword filter based on XAUR and WHO/ICF glossaries, and manually inspected the matches to obtain 1,076 accessibility reviews. These are classified into disability categories, app categories, and positive/negative/neutral sentiment, and the causes of accessibility problems are analyzed qualitatively. The central claims are that accessibility reviews are very rare (0.078% of all reviews), predominantly negative, and dominated by motion sickness and hearing impairments, with Action and Social VR apps receiving the most accessibility feedback.
Significance. If the counting and classification decisions are sound, the paper provides a useful first empirical baseline for VR accessibility reviews, with practical design implications: teleport alternatives, subtitle support, anti-flashing safeguards, and single-handed interaction modes. The study's strengths include the large corpus, the transparent three-stage sampling and filtering procedure, the use of XAUR/WHO vocabularies to construct the keyword set, and the inclusion of numerous verbatim review excerpts that ground the qualitative themes. The paper also goes beyond simple counting by distinguishing positive, negative, and neutral feedback and by analyzing the reasons given by users. However, the headline quantitative claim rests on a construct-validity assumption about which reviews count as accessibility reviews by users with disabilities; this assumption is not adequately validated and affects the majority of the corpus.
major comments (4)
- [§3.2 and §4.2, Table 3] The central quantitative claim depends on treating motion-sickness mentions as accessibility reviews from users with disabilities. Table 3 counts 818 motion-sickness reviews (76.3% of the 1,076 total) under 'Neurological Disabilities', and §4.2 further counts 382 'positive' reviews where the reviewer did not experience motion sickness in the current app but had encountered it in other VR applications. The inclusion/exclusion criteria in §3.2 explicitly exclude reviews from users without disabilities (items 2 and 7), but no criterion verifies that a review mentioning motion sickness comes from a disabled user. Many non-disabled users experience transient VR sickness, and a symptom report is not self-identification of a disability. The quoted review in §4.4 ('the movement speed and style gave me a headache and motion sickness') illustrates the ambiguity. If these 818 reviews are not valid
- [§4.1, §4.2, Table 3] There are unresolved arithmetic and reporting inconsistencies that undermine confidence in the quantitative claims. The abstract and §4.1 report 1,076 reviews, but the rows of Table 3 sum to 1,073 (136+2+1+9+1+818+33+39+12+5+7+5+1+3+1 = 1,073). The percentage 0.078% in the abstract and 0.075% in §4.1 are both given; the actual value is 0.0787% of 1,367,419. Table 3 lists hearing-impairment negative reviews as 90.4%, while §4.2 says 127 negative reviews, i.e., 91.3% of 139? The row percentages themselves are internally inconsistent (e.g., hearing impairment row: 1.6+8.0+90.4 = 100.0, but the 'Percent' column in the SAMPLE column does not match a consistent denominator). Section 4.2 claims 17 disabilities, but Table 3 lists 15 named rows. These inconsistencies should be resolved with a full re-audit of the counts and percentages.
- [§3.3 and §4.2, Table 3] The disability classification is internally inconsistent. Section 4.2 states that cognitive disabilities include attention deficit hyperactivity disorder, autism spectrum disorder, and learning disability, and that physical disabilities include upper limb disability, lower limb disability, cerebral palsy, and asthma. However, Table 3 places autism spectrum disorder and cerebral palsy under 'Neurological Disabilities'. Additionally, Table 3 places social anxiety and asthma under 'Others', which conflicts with the XAUR-defined six categories described in §3.3. This matters because the paper's mapping to XAUR categories is a stated contribution; please align the table, the text, and the reasoning about category membership.
- [§3.2 and §6] The manual inspection step is the only filter between the 9,570 keyword matches and the final 1,076 reviews, but the paper provides no inter-rater reliability metric, no coding instrument, and no release of the final corpus or the annotation decisions. The inclusion criterion allowing third-person discussions 'if they provided sufficient information' is not operationalized. Because the central claims depend on subjective decisions about whether a review is from a user with disabilities, the absence of reliability evidence and the inability to audit the dataset are important threats to validity. Please provide a reliability analysis (e.g., Cohen's kappa on a sample) and make the anonymized dataset or detailed annotation guidelines publicly available, or explain why this is not feasible.
minor comments (5)
- [§4.1] The percentage is given as 0.075% but should be 0.078% (or 0.0787%); the abstract and Section 4.1 should agree.
- [§5.2] Typographical errors: 'participially' should be 'particularly'; 'XUAR' should be 'XAUR'; 'Stream' should be 'Steam' in several places.
- [§3.2 and Figure 1] The text describes a three-step sampling process, but Figure 1 is described as showing 'four main steps'. Please reconcile.
- [Table 2] Simulation row: '30.7.0%' is a typo for '30.7%'.
- [§5.2] The discussion states that sign language was a 'special accessibility feature that received positive reviews', but no counts or quotes supporting this are presented in the Results section. Please add evidence or soften the claim.
Circularity Check
No significant circularity: VR accessibility review counts are data-driven, and self-citations are not load-bearing.
full rationale
The paper's central quantitative claims (1,076 accessibility reviews; 0.078%; motion sickness and hearing impairment as the largest categories) are empirical results of keyword filtering plus manual inspection, not consequences of a fitted parameter or of a self-citation. The keyword list from XAUR/WHO and the disability classification are operational definitions; they do not by construction determine which reviews appear or what sentiment they carry. The motion-sickness inclusion rule is a construct-validity concern (whether transient sickness equals disability), not a circular reduction, because the prevalence of motion-sickness reviews is an outcome of coding, not an input. The self-citations (refs 60, 61, 66) support background claims about VR requirements engineering practice and cybersickness adaptation; even if removed, the study's data and RQ findings stand on the quoted reviews and counts. No equation is derived from itself, and no prediction is a renamed fit; the XAUR-keyword-to-XAUR-category flow is a standard deductive coding design rather than a derivation loop.
Axiom & Free-Parameter Ledger
axioms (5)
- domain assumption Motion sickness is a disability.
- domain assumption The XAUR/WHO keyword list of 294 terms is sufficient to capture all accessibility reviews.
- domain assumption Manual inspection by four researchers reliably identifies disability status and sentiment.
- domain assumption The 100 selected VR apps are representative of the broader VR application ecosystem.
- domain assumption Review text adequately signals the author's disability status.
Cite this review
Pith. "Pith review of Investigating VR Accessibility Reviews for Users with Disabilities: A Qualitative Analysis." pith.science (2026). https://pith.science/paper/SQMU6BOU
@misc{pith2026250813051,
author = {Pith},
title = {Pith review of: Investigating VR Accessibility Reviews for Users with Disabilities: A Qualitative Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/SQMU6BOU}},
note = {Machine review of arXiv:2508.13051}
}
read the original abstract
Accessibility reviews provide valuable insights into both the limitations and benefits experienced by users with disabilities when using virtual reality (VR) applications. However, a comprehensive investigation into VR accessibility for users with disabilities is still lacking. To fill this gap, this study analyzes user reviews from the Meta and Steam stores of VR apps, focusing on the reported issues affecting users with disabilities. We applied selection criteria to 1,367,419 reviews from the top 40, the 20 most popular, and the 40 lowest-rated VR applications on both platforms. In total, 1,076 (0.078%) VR accessibility reviews referenced various disabilities across 100 VR applications. These applications were categorized into Action, Sports, Social, Puzzle, Horror, and Simulation, with Action receiving the highest number of accessibility related-reviews. We identified 16 different types of disabilities across six categories. Furthermore, we examined the causes of accessibility issues as reported by users with disabilities. Overall, VR accessibility reviews were predominantly under-supported.
Reference graph
Works this paper leans on
-
[1]
Vero Vanden Abeele, Brenda Schraepen, Hanne Huygelier, Celine Gillebert, Kathrin Gerling, and Raymond Van Ee
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.