{"id":"e65d4b4e-e433-4a77-9c79-bf820010fadf","arxiv_id":"2606.12418","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Users on Chinese social media use LLMs for divination on personal issues through prompt engineering, perceiving them as effective due to confirmation bias, while professionals reject them for lacking spiritual power.","lead":"This paper studies how Chinese social media users employ large language models for divination practices known as Xuanxue, based on analysis of over 23,000 posts and 32 interviews. A smart generalist might read it to understand how AI is being adopted in cultural and spiritual contexts beyond its technical capabilities.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"Representativeness of Xiaohongshu sample and unvalidated attribution of efficacy to confirmation bias weaken the preservation-of-core-functions claim","rationale":"The reader's weakest_assumption correctly isolates the two empirical pillars (sampling and causal attribution) on which the interpretive leap from observed posts to a general claim about preserved functions depends. No other internal inconsistency appears in the abstract-level argument; the concern is therefore load-bearing for the headline conclusion.","tokens_in":1768,"tokens_out":355,"duration_ms":13362,"concrete_test":"Re-run the thematic coding on a matched corpus of 5,000 posts collected via identical keyword and time filters from Weibo; if the share of posts citing pragmatic concerns or retrospective accuracy justifications shifts by >15 percentage points, the Xiaohongshu-derived description of user pathways and efficacy reasoning cannot be treated as representative.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim requires that observed practices on one platform plus 32 interviews suffice to establish both continuity with traditional Xuanxue functions and the reshaping of authority via prompt co-production. The 23k-post corpus is drawn exclusively from Xiaohongshu (a platform whose user base and posting norms favor trend-driven, visually performative content), creating selection bias toward users already inclined to share positive or novel experiences. Efficacy attributions are coded as biographical fit or retrospective confirmation without supplementary measures (e.g., longitudinal tracking of prediction accuracy or comparison against non-LLM controls), so the mapping onto Barnum/confirmation-bias mechanisms remains post-hoc interpretation rather than tested inference. If either the platform sample or the bias attribution fails, the argument that LLM divination “preserves core functions while introducing scalability” rests on an unrepresentative slice of practice.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper presents a mixed-methods study of LLM-mediated divination (Xuanxue) on Chinese social media. Drawing on 23,000+ Xiaohongshu posts/comments and 32 semi-structured interviews with users and professional diviners, it identifies pragmatic consultation pathways (trend-driven curiosity and event-driven anxiety), collaborative prompt refinement by users, positive efficacy perceptions justified via biographical fit and retrospective confirmation (aligned with Barnum and confirmation bias), verification practices like repeated trials, and professional diviners' rejection of LLMs for lacking spiritual power. The central argument is that LLM divination preserves core functions of traditional practice while introducing scalability, repeatability, and prompt-driven co-production that reshape divinatory authority.","tokens_in":1961,"tokens_out":577,"duration_ms":23340,"significance":"If the empirical claims hold after addressing sampling and validation issues, the work would be a valuable early systematic contribution to the anthropology of AI and divination practices. It explicitly credits the large post volume and interview data, situates findings in established cognitive-evolutionary and anthropological theories, and offers falsifiable observations about user behaviors and authority shifts that could be tested in follow-up studies.","major_comments":[{"comment":"Methods section (data collection description): The 23,000+ posts are drawn exclusively from Xiaohongshu, a platform whose norms favor trend-driven and visually performative content. This creates a selection bias toward users already inclined to share novel or positive experiences, which is load-bearing for the claim that observed practices establish both continuity with traditional Xuanxue functions and the reshaping of authority via prompt co-production.","section":"Methods"},{"comment":"Results section (efficacy attributions): Perceived efficacy is coded as biographical fit or retrospective confirmation without supplementary validation methods such as longitudinal tracking of prediction accuracy or comparison against non-LLM controls. This renders the mapping onto Barnum/confirmation-bias mechanisms post-hoc interpretation rather than tested inference, weakening the preservation-of-core-functions argument.","section":"Results"}],"minor_comments":[{"comment":"Methods section: Additional details on coding procedures for posts/comments, exclusion criteria, and inter-rater reliability would strengthen the mixed-methods design and allow assessment of interpretive consistency.","section":"Methods"},{"comment":"Abstract and introduction: The term 'Xuanxue' is used as an umbrella without a dedicated definition or historical grounding paragraph; a brief clarification would aid readers unfamiliar with the Chinese social media context.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":"The manuscript aligns with the journal's cultural-computing scope but would benefit from explicit discussion of how the Xiaohongshu sample limits generalizability to other platforms; this is a standard concern rather than a novelty issue."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive and detailed feedback. We address each major comment below and indicate planned revisions to strengthen the manuscript.","responses":[{"response":"We agree that sampling exclusively from Xiaohongshu introduces platform-specific selection bias, as its norms emphasize visual and trend-driven sharing that may overrepresent users inclined toward novel or positive experiences. Xiaohongshu was selected as the primary site of LLM-mediated Xuanxue proliferation on Chinese social media. We will add an expanded limitations subsection that explicitly discusses these biases and their implications for generalizability. We will also clarify how the 32 interviews provide triangulation by including users who may not post publicly. These changes qualify the scope of our claims without altering the core observations on prompt co-production and authority shifts, which remain grounded in the data from this key platform.","revision_made":"partial","referee_comment":"[Methods] Methods section (data collection description): The 23,000+ posts are drawn exclusively from Xiaohongshu, a platform whose norms favor trend-driven and visually performative content. This creates a selection bias toward users already inclined to share novel or positive experiences, which is load-bearing for the claim that observed practices establish both continuity with traditional Xuanxue functions and the reshaping of authority via prompt co-production."},{"response":"The efficacy analysis relies on thematic coding of posts and interviews, which is standard for qualitative exploration of cultural practices and does not claim experimental validation of mechanisms. We will revise the results and discussion sections to more explicitly frame the Barnum and confirmation-bias alignment as interpretive rather than tested inference, and we will add a limitations note acknowledging the absence of longitudinal tracking or non-LLM controls. The preservation-of-core-functions argument is supported by observed continuities in pragmatic consultation pathways and user behaviors, not by claims of predictive accuracy; the revisions will make this distinction clearer.","revision_made":"partial","referee_comment":"[Results] Results section (efficacy attributions): Perceived efficacy is coded as biographical fit or retrospective confirmation without supplementary validation methods such as longitudinal tracking of prediction accuracy or comparison against non-LLM controls. This renders the mapping onto Barnum/confirmation-bias mechanisms post-hoc interpretation rather than tested inference, weakening the preservation-of-core-functions argument."}],"tokens_in":1480,"tokens_out":484,"duration_ms":22908,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper maps out how users on Xiaohongshu are turning LLMs into divination tools for everyday worries like relationships, jobs, and game pulls. It pulls together over 23,000 posts plus 32 interviews and shows two entry points: viral curiosity and anxiety-driven searches. Users iterate on prompts together, and many rate the outputs as accurate because they match their life stories or past events.\n\nWhat stands out as new is the scale of the post collection and the split between regular users and professional diviners. The professionals reject the LLMs on grounds of missing spiritual power, which gives a clear contrast. The paper also notes verification habits like running the same query multiple times or across models. These details sit inside standard anthropological and cognitive accounts of divination, so the framing is not invented.\n\nThe soft spots are straightforward. The entire post sample comes from Xiaohongshu, a platform that rewards visible, trend-following content, so the positive tilt and the collaborative prompt story may over-represent certain users. The attribution of efficacy judgments to biographical fit and confirmation bias comes from coding comments rather than any check against actual prediction accuracy or a control group. That makes the stronger claim about preserving traditional functions while adding scalability and co-production more of an interpretation than a tested result.\n\nThis is useful for anyone working on technology and everyday belief practices or digital religion. A reader who wants concrete examples of how people adapt AI to old rituals will find material here. The data collection effort is real, so the paper deserves a serious referee even though the generalizability and causal links on bias need more work. I would send it out for review.","headline":"Descriptive map of LLM divination on one Chinese platform with solid data volume but interpretive claims on bias and authority that lack direct tests.","tokens_in":2424,"tokens_out":406,"would_cite":false,"duration_ms":22031,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"LLM divination on Chinese social media keeps traditional guidance functions but shifts authority to user prompt refinement and scalable verification.","keywords":["LLM divination","Xuanxue","prompt engineering","confirmation bias","divinatory authority","Xiaohongshu","mixed methods","anthropology of technology"],"falsifier":"A controlled study that tracks the same users across repeated LLM readings and traditional diviner sessions to test whether verification behaviors and authority judgments differ systematically from the patterns reported here.","tokens_in":2679,"feed_emoji":"🔮","tokens_out":575,"duration_ms":19308,"temperature":0.7,"pith_summary":"The paper studies how people on Xiaohongshu use large language models for divination on personal matters like relationships, jobs, and exams. It shows users reach these tools through viral trends or sudden uncertainty, then collaborate by refining prompts to shape the output. Perceived accuracy often rests on how well the reading matches the user's life story or past events, which aligns with known biases. Professional diviners reject the practice for missing spiritual elements, while users mix scientific and metaphysical explanations. The work claims this preserves divination's core role in handling uncertainty yet changes how authority forms through repeatability and co-production.","feed_headline":"LLM users co-produce divination readings via prompt edits","feed_subtitle":"Chinese social media practice keeps traditional guidance roles while moving authority to scalable, repeatable, user-driven verification.","key_machinery":"collaborative prompt refinement, in which users iteratively edit inputs to co-produce readings and thereby become active participants in constructing the divinatory output.","core_discovery":"LLM-mediated divination in Xuanxue preserves core functions of traditional practice while introducing scalability, repeatability, and prompt-driven co-production that reshape how divinatory authority is constructed and evaluated.","pith_inferences":["This pattern may extend to other cultural practices where AI becomes a co-author of interpretive knowledge.","Repeated use could alter how people experience uncertainty by making divination instantly available and revisable.","Economic effects on traditional diviners would depend on whether users see LLM versions as substitutes or supplements."],"forward_implications":["Users treat LLM outputs as repeatable and testable, leading to practices like multiple trials and cross-model checks.","Authority shifts toward prompt engineering skill rather than inherited or spiritual expertise.","Professional diviners respond by emphasizing ontological differences to maintain boundaries.","Participants reconcile AI results with both empirical testing and metaphysical beliefs."],"fun_headline_variants":["Users refine LLM prompts for Xuanxue divination","Prompt engineering reshapes Chinese AI divination","LLM co-production alters divinatory authority via edits","Scalable prompts enable user-driven Xuanxue practice","Prompt co-production preserves traditional divination roles"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The 23,000 posts and 32 interviews capture typical user practices and that stated reasons for believing the readings truly stem from biographical fit and confirmation bias.","fun_headline_variants_meta":{"raw":{"variants":["Users refine LLM prompts for Xuanxue divination","Prompt engineering reshapes Chinese AI divination","LLM co-production alters divinatory authority via edits","Scalable prompts enable user-driven Xuanxue practice","Prompt co-production preserves traditional divination roles","title"]},"model":"grok-4.3","cost_usd":0.007556,"raw_usage":{"total_tokens":3472,"prompt_tokens":684,"num_sources_used":0,"completion_tokens":60,"cost_in_usd_ticks":75562000,"prompt_tokens_details":{"text_tokens":684,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2728,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":684,"tokens_out":60,"duration_ms":24188,"temperature":1.0,"reasoning_tokens":2728,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T23:30:18.296834+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled study that tracks the same users across repeated LLM readings and traditional diviner sessions to test whether verification behaviors and authority judgments differ systematically from the patterns reported here.","supporting_citations":[],"review_version":1}