{"id":"66bab05e-d282-409e-b30e-c55456e0ecde","arxiv_id":"2608.11391","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Blind users of mental health tracking apps face structural access barriers, including paywall-gated accessibility evaluation and inaccessible community features, even when they possess the relevant digital literacy.","lead":"A mixed-methods study of 93 legally blind US adults and 10 follow-up interviews finds that mental health tracking apps often hide accessibility information behind paywalls and exclude blind users from community features. The authors argue that many blind users have the needed skills but are blocked by design choices, and they propose a literacy-versus-access-barrier distinction for eHealth research.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paywall-gated accessibility evaluation is asserted as a marketplace structure from a few participant quotes; no survey item or app audit verifies that accessibility information is actually behind paywalls.","rationale":"The reader's weakest_assumption was sample representativeness: advocacy-recruited, self-selected participants who are likely more digitally literate than the broader blind community. That is a legitimate concern, but my stress-test sharpens a different, more direct vulnerability in the central claim. The paper's most distinctive contribution is the concept of paywall-gated accessibility evaluation, which it explicitly labels a structural marketplace feature. The evidence for this concept consists of a small number of qualitative quotes, and no quantitative or market-level measurement. Even if the sample were perfectly representative, the inference from 'participants want to evaluate accessibility before paying' to 'the marketplace places accessibility information behind paywalls' is not warranted by the data as presented. A participant can want pre-purchase accessibility information even when some services already provide it; the structural claim requires knowing how prevalent the paywall pattern actually is. The community-exclusion finding is also thin (two interview participants), but the paywall claim is the one the paper brands as structural and uses to anchor its recommendations, so it is the most load-bearing. The paper is otherwise transparent, well-structured, and appropriately cautious in its quantitative claims; the Kruskal-Wallis null result is correctly interpreted as underpowered. The concern does not warrant rejection; it warrants retaining the CONDITIONAL verdict and requiring either a market audit or a direct survey item before the structural claim can be accepted at face value.","tokens_in":25377,"tokens_out":5153,"duration_ms":50468,"concrete_test":"Audit the pre-purchase evaluation path of the top 20 DMH tracking services named by participants (e.g., Calm, Headspace, Fitbit, Apple Health): record for each whether a new user can access an accessibility statement, screen-reader compatibility information, or a feature-preserving free trial before payment. If a majority of services expose accessibility information pre-purchase or offer a meaningful free tier, the 'paywall-gated accessibility evaluation' claim is overstated; if a majority require payment before such information is available, the claim is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical pillar for the paper's conceptual contribution is the claim that DMH services impose 'paywall-gated accessibility evaluation' (Sections 4.3.2, 4.4.3, contribution list). This is presented as 'a structural feature of the DMH marketplace' and drives the headline design/policy recommendations (surface accessibility before paywall, decouple evaluation from payment). Yet the supporting evidence is participant inference, not marketplace evidence. N3's quote expresses a wish to know whether a paid service will work with assistive technology and a worry that upgrades may lose functionality; P75 couples cost with 'accessibility challenges or glitches' generally. Neither establishes that accessibility-relevant information is actually placed behind a paywall, or that no free trial or accessibility statement exists. The survey contains no closed-ended item on pre-purchase accessibility disclosure: Appendix B items ask about accessibility, cost, privacy, and adoption prerequisites, but none asks whether or how accessibility was evaluated before payment. No audit of DMH services was conducted. The paper nonetheless elevates this interpretive pattern to a structural claim. If paywall-gating is not actually widespread, the paper's flagship concept and the derived recommendations lose their empirical footing. Sample representativeness is a related but secondary issue: the more immediate problem is that the structural generalization goes beyond what the instrument and transcripts directly capture.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an explanatory sequential mixed-methods study of 93 legally blind US adults who completed a survey about digital mental health (DMH) tracking services, followed by semi-structured interviews with 10 respondents. It documents usage concentrated in audio-first categories (mindfulness, sleep, and goal tracking), recurring accessibility barriers, privacy and data-agency preferences, and exclusion from community-based peer-support features. The authors interpret the findings through Norman and Skinner's eHealth Literacy framework and argue that its 'computer literacy' dimension is insufficient because many blind users possess the relevant literacies but are blocked by structural design and marketplace barriers. The paper contributes empirical findings, the concept of 'paywall-gated accessibility evaluation,' and design/policy recommendations for surfaceability of accessibility information before purchase, accessible community features, and user-controlled data agency.","tokens_in":25565,"tokens_out":4287,"duration_ms":39487,"significance":"If the central structural claims hold, this paper is a valuable addition to accessibility and digital mental health research. Its strengths include a transparent methods section, full survey instrument in the appendix, explicit positionality of blind researchers, and an accessible survey design that is itself a methodological contribution. The proposed distinction between literacy gaps and access barriers is a useful and well-motivated conceptual extension of the eHealth Literacy framework, consistent with the social model of disability. The empirical findings on usage patterns and community exclusion address a genuinely understudied population. However, the paper's headline contribution—paywall-gated accessibility evaluation as a 'structural feature of the DMH marketplace'—rests on limited participant inference rather than direct marketplace evidence, and the sample is narrow in ways that affect the scope of the generalizations. The core idea is promising and the evidence base is partially sufficient, but the manuscript overreaches in its framing and would require revision to support its strongest claims.","major_comments":[{"comment":"The concept of 'paywall-gated accessibility evaluation' is presented as a structural feature of the DMH marketplace and is used to derive the paper's first and fourth design/policy recommendations (Section 5.4). Yet the supporting evidence consists almost entirely of two participant quotes (P75 in Section 4.3.2 and N3 in Section 4.4.3). No closed-ended survey item in Appendix B asks whether or how participants evaluated accessibility before payment, and no audit of DMH services' pre-purchase accessibility disclosure was conducted. The quotes show that participants want to evaluate accessibility before paying, but they do not establish that accessibility-relevant information is actually placed behind paywalls or that free trials and accessibility statements are systematically absent. To sustain a structural claim, the paper needs either direct marketplace evidence (e.g., an audit of disclosure practices across a sample of DMH services) or a reframing of the finding as an emergent qualitative hypothesis that motivates future work instead of a documented marketplace feature.","section":"Sections 4.3.2, 4.4.3, and 5.2"},{"comment":"The sample is self-selected through four blind advocacy organizations (NFB, ACB, AFB, DO-IT) and therefore likely over-represents advocacy-connected and digitally literate blind adults. This is in fact an ideal population for demonstrating the paper's conceptual point—high literacy combined with persistent access barriers—but it limits the paper's broader generalizations to 'the blind community' that appear in the Abstract, the contribution list, and the Conclusion. The Limitations section (Section 6) acknowledges the lack of generalizability, yet the manuscript continues to use unqualified community-level language throughout the framing and recommendations. The authors should either scope all claims to the studied population (e.g., 'advocacy-connected blind adults in the United States') or provide additional evidence that the sample is representative enough for the stated generalizations.","section":"Sections 3.2 and 4.1"},{"comment":"The qualitative analysis was performed by the first author independently, with weekly discussion with the second author, and thematic saturation at 10 interviews is asserted without reporting code-saturation metrics, an audit trail, or independent coding by a second coder. Because the paper's central conceptual argument—the distinction between literacy gaps and access barriers—relies heavily on the interview and open-ended analysis, the absence of such evidence weakens the reproducibility of the qualitative claims. I recommend adding an inter-rater reliability check, a more detailed description of the coding process and theme formation, or an explicit acknowledgment that the themes are interpretive and would benefit from further confirmatory work.","section":"Sections 3.6 and 4.1"}],"minor_comments":[{"comment":"The text refers to 'Table 3 in Appendix 4.1,' but Table 3 appears in Appendix A.2 (Interview Demographics); the cross-reference should be corrected.","section":"Section 4.1"},{"comment":"The phrase 'at a large-scale' overstates the breadth of a 93-respondent convenience sample; consider replacing it with a more modest descriptor such as 'at a moderate scale' or 'across multiple recruitment channels.'","section":"Contribution list, Section 1"},{"comment":"The phrase 'the many shapes and forms of barriers to entry' is vague; consider removing it or replacing it with a concrete summary of the barrier types identified.","section":"Section 4.4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical work and conceptual contribution are worthwhile and within the scope of ASSETS, and the authors' positionality and accessible survey design are clear strengths. However, the 'paywall-gated accessibility evaluation' claim is the linchpin of the design and policy recommendations and is currently under-evidenced for the structural weight placed on it. This is not a fatal flaw because the underlying participant data and the proposed framework extension are valuable; the authors can address it through reframing or by adding direct marketplace evidence. I would support a major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a solid paper with a real contribution. The distinction between literacy gaps and access barriers is genuinely useful for accessibility and digital health research, and the interview data on community exclusion is compelling. The two blind authors bring a positionality that strengthens the work, the survey design is thoughtfully made screen-reader accessible, and the limitations section is honest. The survey instrument in the appendix is a plus.\n\nThe strongest empirical material is the qualitative core: participants who are clearly digitally literate, able to articulate their mental health goals, and still blocked by interfaces that assume sight. The community-exclusion finding—especially N6's account of being locked out of a women's wellness community that her sighted friends use—is well-evidenced and important. The framework extension follows from that material and is the paper's real intellectual contribution.\n\nNow the soft spots. The stress-test note is right. The paper names \"paywall-gated accessibility evaluation\" as a structural feature of the DMH marketplace, but the supporting evidence is a handful of participant quotes expressing reluctance to pay before knowing whether a product works. N3 does not say accessibility information is literally behind a paywall, and the survey contains no closed-ended item about pre-purchase accessibility disclosure. There is also no app audit. The authors may well be correct that the pattern exists, but the data as presented support a more modest claim: blind users are anxious that they might have to pay to find out. Calling it a structural feature of the marketplace goes beyond what the instrument and transcripts capture. That is a revision-level concern, not a rejection-level one.\n\nThe sample is also narrower than some of the claims. Advocacy-recruited respondents are likely more digitally literate and more aware of their rights than the general blind population. That actually fits the paper's thesis—literacy without access—but it means the prevalence of the paywall problem is not established. The single-coder thematic analysis and the absence of a published codebook are minor but worth noting. The Kruskal-Wallis null is appropriately caveated, so I would not hold that against the paper.\n\nThe citation pattern looks fine. The reuse of survey data from the earlier CHI EA paper is disclosed clearly, and the interview phase and conceptual extension are new.\n\nBottom line: worth serious peer review. I would recommend conditional accept, with the authors asked to either soften the structural claim or back it with marketplace-level evidence. I would cite this for the framework distinction and the community-exclusion finding.","headline":"Solid mixed-methods core with a genuinely useful conceptual distinction, but the paper overstates the evidence for its flagship 'paywall-gated accessibility evaluation' claim.","tokens_in":26110,"tokens_out":1871,"would_cite":true,"duration_ms":19894,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that blind users of digital mental health tracking apps are often blocked by paywalls that hide accessibility and by community features built for sighted users, not by missing digital skills.","keywords":["accessibility","digital mental health","blind and low vision","screen readers","eHealth literacy","mixed-methods study","paywall-gated accessibility evaluation","data agency"],"falsifier":"A decisive test would be to audit a random sample of DMH tracking apps by attempting, without paying, to obtain a structured accessibility statement and complete a core task with a screen reader, and then to check whether community features stay usable after purchase; a marketplace where most apps already disclose accessibility pre-purchase and keep community features accessible would undercut the paper's central barrier claim.","tokens_in":25140,"feed_emoji":"🧠","tokens_out":9225,"duration_ms":75881,"temperature":0.7,"pith_summary":"Digital mental health (DMH) tracking apps promise support for well-being, but this paper argues that for blind users the deciding obstacle is structural rather than personal. Based on a survey of 93 legally blind US adults and 10 follow-up interviews, it finds that many blind users have strong eHealth literacy—they are skilled screen-reader users who know what they want—yet cannot apply those skills because apps hide accessibility information behind paywalls, rely on visual-only charts, and make community features unusable. The paper names the central pattern \"paywall-gated accessibility evaluation\" and extends a standard eHealth literacy framework by separating literacy gaps (user lacks a skill) from access barriers (design blocks an existing skill). If the paper is right, making mental health apps work for blind users means changing disclosure, marketplace, and community-feature design, not retraining users.","feed_headline":"Blind users pay before they can test mental health apps","feed_subtitle":"A survey of 93 blind adults finds the real barrier is paywalls and design, not missing digital skills.","key_machinery":"The load-bearing mechanism is the distinction between a literacy gap and an access barrier, applied inside the eHealth Literacy framework's six-literacy decomposition. A literacy gap means the user lacks a competency the task requires; an access barrier means the user possesses the competency but the service's design prevents it from being exercised. This distinction does the paper's explanatory work: it turns the observation that skilled screen-reader users still fail at DMH apps into evidence that the failure belongs to the design, and it redirects intervention from user education to service change. The second named mechanism is paywall-gated accessibility evaluation, defined as a marketplace pattern in which accessibility-relevant information is withheld until after payment, so blind users cannot form the perceived usefulness that adoption models treat as the starting point of acceptance.","core_discovery":"The paper's central claim is that the failures blind users experience with DMH tracking services are overwhelmingly access barriers, not literacy gaps. Participants who could operate assistive technology, evaluate health information, and articulate their mental health goals were nevertheless stopped by interfaces designed for sighted users: unlabeled buttons, startup screens that could not be passed, progress data rendered only as charts, and community features that excluded them from the peer support that is part of the intervention. The paper documents a structural marketplace pattern it calls paywall-gated accessibility evaluation: blind users must pay before they can determine whether a service is usable, turning accessibility discovery into a financial risk. It extends the eHealth Literacy framework, whose six-literacy decomposition treats competencies as properties of individuals, by distinguishing a literacy gap from an access barrier; the latter is a design failure that no amount of user skill can overcome. The evidence is that usage concentrates in audio-first categories like mindfulness and sleep, where accessibility retrofits least damage the intervention, while adoption is gated by pre-purchase invisibility and community connection is unavailable.","pith_inferences":["My inference: because the mechanism is pre-purchase invisibility, paywall-gated accessibility evaluation likely also operates in other subscription digital health and wellness categories, not just DMH apps.","My inference: the literacy-gap versus access-barrier distinction can be used as a diagnostic in any eHealth intervention; an audit of existing digital health literacy programs that separates the two would reveal how many target user education where the actual failure is structural.","My inference: if usage concentrates in audio-first categories because those survive screen-reader use, then releasing an accessible mood-tracking or therapy-chat service should visibly shift blind users' category choices; that is a testable prediction.","My inference: a matched comparison of adoption rates for apps with and without pre-purchase accessibility trials would test the causal claim experimentally."],"forward_implications":["If the paywall-gated accessibility evaluation pattern holds, DMH services should publish structured accessibility statements before purchase and offer time-limited trials that include the features blind users most need to evaluate.","If community exclusion is as central as the paper argues, peer-support and community features should be treated as intervention accessibility, with the same standards as the core mental health content.","If adoption theory incorporates evaluability, then perceived usefulness should be preceded by a pre-purchase check that accessibility information is available to the user.","If regulators or app stores require accessibility disclosure as a listing condition, the cost of evaluating accessibility would shift from individual blind users to service providers.","If the personalization findings are implemented, users should be able to configure the balance of push-based and pull-based interactions and set screen-reader-friendly shortcuts to frequent content."],"supporting_citations":[{"why":"Supplies the six-dimension eHealth literacy model whose \"computer literacy\" framing the paper argues collapses literacy gaps into access barriers.","marker":"[52, 53]"},{"why":"Prior survey work with blind advocacy organizations that this study extends with interviews and the framework distinction.","marker":"[35]"},{"why":"Provides the explanatory sequential mixed-methods design that pairs the survey with follow-up interviews.","marker":"[23]"},{"why":"Supplies the Technology Acceptance Model that frames adoption; the paper argues pre-purchase evaluability is a missing antecedent.","marker":"[26]"},{"why":"Supplies the Client Satisfaction Questionnaire constructs that anchor the survey's satisfaction and non-adoption items.","marker":"[4]"},{"why":"Prior studies of blind users' personal health technologies establish the screen-reader and visualization barriers this paper builds on.","marker":"[21, 41, 42]"},{"why":"Documents elevated depression and anxiety among blind individuals, motivating why DMH inaccessibility is consequential.","marker":"[37, 64]"}],"fun_headline_variants":["Blind users pay to learn if apps are accessible","Mental health apps charge blind users to test accessibility","Blind users face paywalls, not skill gaps, in mental health apps","Accessibility discovery is a paywall for blind app users","Blind users are blocked by design, not digital literacy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 93 self-selected blind adults recruited through advocacy networks, and the 10 interviewed among them, represent the US blind community closely enough to support claims about marketplace-wide structural barriers; if the sample skews toward unusually literate and advocacy-connected users, the observed split between literacy and access could be overstated.","fun_headline_variants_meta":{"raw":{"variants":["Blind users pay to learn if apps are accessible","Mental health apps charge blind users to test accessibility","Blind users face paywalls, not skill gaps, in mental health apps","Accessibility discovery is a paywall for blind app users","Blind users are blocked by design, not digital literacy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000244,"raw_usage":{"total_tokens":1581,"prompt_tokens":1045,"completion_tokens":536,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":454}},"tokens_in":661,"tokens_out":536,"duration_ms":59170,"temperature":1.0,"reasoning_tokens":454,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:11:58.701989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be to audit a random sample of DMH tracking apps by attempting, without paying, to obtain a structured accessibility statement and complete a core task with a screen reader, and then to check whether community features stay usable after purchase; a marketplace where most apps already disclose accessibility pre-purchase and keep community features accessible would undercut the paper's central barrier claim.","supporting_citations":[{"cited_title":"Creswell and Vicki L","cited_arxiv_id":null,"evidence_quote":"Provides the explanatory sequential mixed-methods design that pairs the survey with follow-up interviews."}],"review_version":1}