{"id":"9a3b2fa3-d529-4405-89c2-c823eb4467d2","arxiv_id":"2605.02016","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Empirical study of 25 mental health apps identifies undisclosed trackers in all apps, permission-policy contradictions in 13 apps, and incomplete third-party AI disclosures in several.","lead":"This paper analyzes privacy policies versus actual data practices in 25 popular Android mental health and life-coaching apps using static analysis, network monitoring, and policy review. It finds consistent gaps such as undisclosed trackers and missing permission disclosures that undermine informed consent for highly sensitive user data.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Dynamic analysis and LLM policy extraction may miss undisclosed behaviors, risking incomplete evidence of transparency gaps.","rationale":"The reader's weakest assumption matches the methodological completeness issue identified here; it is the precise point where the empirical support for the central claim is least secure. Full-text methods details would be needed to assess validation rigor, but the concern itself is unchanged by abstract-only status.","tokens_in":1783,"tokens_out":313,"duration_ms":26769,"concrete_test":"Re-execute dynamic analysis on the same 25 APKs using expanded test suites that explicitly trigger every declared dangerous permission (camera, mic, storage) plus 50% more user-interaction sequences; re-compare detected trackers and contradictions against the original counts. A >15% increase in undisclosed items would indicate the original results understate the gaps.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that disclosure practices fall short of informed consent requires that the 25-app analysis (static APK inspection, dynamic network capture, LLM-assisted policy extraction with manual validation) has no material false negatives. Dynamic capture depends on test paths exercised; if not all permission-triggered flows or third-party SDK calls are triggered, undisclosed trackers (e.g., the reported 20 in Talkie) or AI recipients could be under-counted. LLM extraction, even validated, risks misinterpreting vague legal phrasing about data recipients. Either gap would mean the 68% non-disclosure rate and 16 contradictions do not fully demonstrate systemic shortfalls.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper presents a multi-method empirical analysis of 25 popular Android mental health and life-coaching apps. It combines static APK inspection for trackers and permissions, dynamic network traffic capture, and LLM-assisted extraction of privacy policies (validated against manual annotation). Key results include: every app embeds at least one tracker SDK not named in its policy, with 68% failing to disclose at least half of detected trackers (Talkie embeds 20, names none); 16 permission-policy contradictions across 13 apps, including 6 that request camera/microphone without disclosing collection; and 48% disclose third-party AI processing (e.g., OpenAI, Anthropic), while 7 use only generic language. The authors conclude that these disclosure gaps mean current practices fall short of the transparency needed for meaningful informed consent and argue for an updated regulatory framework modeled on standards for licensed therapists.","tokens_in":1926,"tokens_out":535,"duration_ms":30481,"significance":"If the detection methods prove comprehensive, the quantified findings provide concrete evidence of transparency shortfalls in a domain handling highly sensitive data, strengthening the case for policy intervention. The multi-method design and manual validation of LLM outputs are methodological strengths that enhance credibility over purely static or policy-only studies. The specific counts on undisclosed trackers, contradictions, and AI recipients offer falsifiable observations useful for future replication or regulatory reference.","major_comments":[{"comment":"§3.2 (Dynamic Analysis): The description of network traffic capture provides no details on exercised test paths, permission-triggered flows, or coverage metrics. This is load-bearing for the central claim because incomplete path coverage could produce false negatives in tracker and AI recipient detection, directly affecting the 'every app' undisclosed tracker result and the 68% non-disclosure statistic.","section":"§3.2"},{"comment":"§3.3 (LLM-assisted Policy Extraction): Although the method is stated to be validated against manual annotation, no quantitative validation metrics (precision, recall, disagreement rate, or example cases) are reported. This is load-bearing for the informed-consent conclusion because it affects confidence in the 48% AI-processing disclosure rate and the count of 7 apps using only generic language.","section":"§3.3"}],"minor_comments":[{"comment":"The abstract and results could more explicitly state the total number of trackers detected across the corpus to allow readers to assess the scale of the 68% figure.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback highlighting opportunities to strengthen the methodological transparency of our multi-method analysis. We address each major comment below and will revise the manuscript to incorporate the requested details.","responses":[{"response":"We agree that the current description of dynamic analysis in §3.2 is insufficiently detailed. In the revised manuscript we will expand this section to specify the exercised test paths (onboarding, account creation, journaling/mood logging, AI chat interactions, and explicit permission grants for camera/microphone), the automation approach used to trigger permission flows, session durations, and any coverage indicators obtained during network capture. These additions will directly support the reliability of the tracker-detection results.","revision_made":"yes","referee_comment":"[§3.2] §3.2 (Dynamic Analysis): The description of network traffic capture provides no details on exercised test paths, permission-triggered flows, or coverage metrics. This is load-bearing for the central claim because incomplete path coverage could produce false negatives in tracker and AI recipient detection, directly affecting the 'every app' undisclosed tracker result and the 68% non-disclosure statistic."},{"response":"We acknowledge that quantitative validation metrics for the LLM-assisted policy extraction were not reported, even though manual validation was performed. In the revision we will add a dedicated paragraph or table in §3.3 presenting precision, recall, disagreement rate, and representative examples of LLM vs. manual annotation outcomes. This will increase confidence in the 48% AI-disclosure figure and the count of generic-language apps.","revision_made":"yes","referee_comment":"[§3.3] §3.3 (LLM-assisted Policy Extraction): Although the method is stated to be validated against manual annotation, no quantitative validation metrics (precision, recall, disagreement rate, or example cases) are reported. This is load-bearing for the informed-consent conclusion because it affects confidence in the 48% AI-processing disclosure rate and the count of 7 apps using only generic language."}],"tokens_in":1556,"tokens_out":441,"duration_ms":25131,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"This study checks 25 Android therapy and coaching apps and reports that every single one ships at least one tracker SDK its privacy policy never names, 68 percent hide at least half the trackers found in the APK, and 13 apps show 16 cases where a dangerous permission appears in the manifest but not in the policy text. They also note that 48 percent mention third-party AI processing while seven use only vague language about recipients.\n\nThe work applies standard auditing tools to this specific category and produces domain-specific numbers that were not in the earlier literature. The combination of APK inspection, network capture during use, and LLM-assisted policy review with manual checks is a reasonable way to surface the gaps, and the validation step for the policy extraction strengthens the policy side of the results.\n\nThe main limitation is that dynamic network capture only records what the chosen test paths actually trigger. If some tracker or data-flow calls are not exercised, the reported non-disclosure rates could be under-counts, even though static analysis catches the embedded SDKs. The abstract gives counts but does not include coverage metrics or false-negative estimates for the dynamic runs, so the completeness claim rests partly on unshown test design. The final regulatory argument follows from the data but is not itself measured.\n\nThe paper is useful for anyone tracking privacy practices in health-related apps or working on consent standards. The empirical measurements are direct enough to be worth citing as a data point. It should go to peer review so the methods details and test coverage can be examined.","headline":"The paper delivers concrete counts of undisclosed trackers and permission contradictions across 25 mental health apps using a mix of static, dynamic, and policy analysis.","tokens_in":2390,"tokens_out":381,"would_cite":true,"duration_ms":17336,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Mental health apps embed trackers and request permissions their policies do not disclose.","keywords":["mental health apps","privacy policies","tracker SDKs","informed consent","Android permissions","third-party data sharing","AI processing","transparency gaps"],"falsifier":"A complete re-run of the same static, dynamic, and policy analysis on these 25 apps that finds every detected tracker named in the policies and zero permission contradictions would falsify the reported transparency gaps.","tokens_in":2696,"feed_emoji":"🔒","tokens_out":738,"duration_ms":29074,"temperature":0.7,"pith_summary":"The paper analyzes 25 popular Android mental health and life-coaching apps to test whether their privacy policies accurately reflect actual data practices. It combines static code inspection, live network monitoring, and automated policy text extraction to surface mismatches. Every app turns out to embed at least one tracker SDK that its policy never names, more than half the apps omit at least half their trackers, and 13 apps declare dangerous permissions such as camera or microphone access without ever mentioning the corresponding data collection. Because these apps routinely receive highly personal information about traumas, relationships, and mental states, the gaps prevent users from giving informed consent. The authors conclude that existing disclosure standards are inadequate and call for regulatory updates modeled on the ethical rules that govern licensed therapists.","feed_headline":"Mental health apps hide trackers their policies never name","feed_subtitle":"Analysis of 25 Android apps finds every one embeds at least one undisclosed tracker and 13 apps contradict their own permission declarations","key_machinery":"Side-by-side comparison of app manifest permissions, observed network destinations, and extracted privacy-policy statements across the 25 apps.","core_discovery":"Static analysis, dynamic network capture, and LLM-assisted policy extraction on the 25 apps show that every app contains at least one tracker SDK absent from its policy, 68 percent of apps fail to name at least half the trackers found in their APKs, 16 permission-policy contradictions appear across 13 apps (including six that request camera or microphone access without disclosure), 48 percent of apps acknowledge third-party AI processing while seven use only generic language that leaves the recipients unnamed, and one app forwards journal entries to three different AI providers at once. These concrete mismatches demonstrate that current disclosure practices fall short of the transparency req","pith_inferences":["App stores or regulators could require machine-readable lists of all embedded SDKs and AI recipients.","Similar side-by-side checks could be applied to other categories of apps that handle sensitive personal data.","Users might change behavior if they could see automated comparisons between an app's declared policy and its actual traffic.","Therapy-app developers would need to audit every third-party component before release to avoid the contradictions found here."],"forward_implications":["Every examined app embeds at least one undisclosed tracker SDK.","Thirteen apps declare dangerous permissions that their policies omit.","Six apps request camera or microphone access without any corresponding disclosure.","Nearly half the apps send data to third-party AI services, sometimes to multiple providers simultaneously.","Existing disclosure practices do not support meaningful informed consent, so a significantly updated regulatory framework is required."],"fun_headline_variants":["Every mental health app embeds a tracker absent from its policy","13 apps request camera access without policy disclosure","Mental health apps send data to undisclosed AI providers","68 percent of apps omit at least half their trackers from policy","Journal entries forwarded to multiple AI services by one app"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The 25 chosen apps together with the static, dynamic, and LLM-assisted methods capture the full set of real privacy practices without missing material trackers or misreading policies.","fun_headline_variants_meta":{"raw":{"variants":["Every mental health app embeds a tracker absent from its policy","13 apps request camera access without policy disclosure","Mental health apps send data to undisclosed AI providers","68 percent of apps omit at least half their trackers from policy","Journal entries forwarded to multiple AI services by one app"]},"model":"grok-4.3","cost_usd":0.003376,"raw_usage":{"total_tokens":1857,"prompt_tokens":798,"num_sources_used":0,"completion_tokens":74,"cost_in_usd_ticks":33762000,"prompt_tokens_details":{"text_tokens":798,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":985,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":798,"tokens_out":74,"duration_ms":9913,"temperature":1.0,"reasoning_tokens":985,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T00:07:44.656970+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A complete re-run of the same static, dynamic, and policy analysis on these 25 apps that finds every detected tracker named in the policies and zero permission contradictions would falsify the reported transparency gaps.","supporting_citations":[],"review_version":3}