{"id":"9134e49a-4fd9-4096-b40c-53b100ff6dba","arxiv_id":"2411.10280","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A 32-interview study finds Chinese families use AI college-application tools mostly through parents, with tools optimizing admission odds while neglecting students' long-term career goals.","lead":"Researchers interviewed 32 students, parents, and experts in China about how families use Quark GaoKao, an AI tool that suggests colleges and admission odds for the national college entrance exam. They found parents do most of the work, students stay on the sidelines, and the AI focuses on scores rather than long-term careers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The generalization from 32 self-selected, retrospective interviews, concentrated in northern China and recruited through word-of-mouth, to a prevalence claim about 'Chinese family use' of AI tools is the load-bearing weak point; the findings are plausible but not yet representative.","rationale":"The reader's verdict identifies the same weakest assumption: sample representativeness. I agree. The paper's internal evidence is coherent: the code and quote patterns consistently describe parents triangulating AI outputs, live streams, books, and consultants, while children often narrow down lists. The authors are appropriately explicit in §7.3 about regional concentration. The problem is not internal inconsistency; it is the gap between the evidence and the inference. A 'predominantly used by parents' statement is a distributional claim, and distributional claims need either a representative sample or a clearly bounded scope. A 32-person convenience sample of existing Quark users, recruited partly through parent networks, cannot establish that Chinese families in general use these tools predominantly by parents. The paper's own counts in §5.4 (nine participants) further underline that the claim is a theme, not a measured prevalence. The inequity discussion is presented as an interpretation and is well grounded in expert and parent quotes, but it also depends on the same selective sample. I would not reject the paper; the qualitative findings and design implications are useful and honestly reported. The appropriate action is to keep the verdict at CONDITIONAL: the authors should either re-scope the title and abstract to 'a qualitative study of 32 families and experts in northern/central China' or provide additional evidence, such as a survey or telemetry, supporting the prevalence claim. The proposed stratified replication is a feasible check that would settle whether the pattern generalizes.","tokens_in":24666,"tokens_out":4984,"duration_ms":51706,"concrete_test":"Preregister and run a stratified replication: quota-sample at least 300 families across at least 8 PAs (e.g., Guangdong, Sichuan, Zhejiang, Jiangsu, Hubei, plus northern provinces), recruited independently of Quark user communities, balanced between current under-18 students and parents, and collect primary-operator (parent/student/shared) and career-goal-use data via structured survey plus a prospective 7-day usage diary during the application window. If parent-led use remains a clear majority across strata and regions, the concern is resolved; if usage roles vary by region or recruitment channel, the paper's title and abstract should be re-scoped to the sampled context.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—'AI tools are predominantly used by parents' and 'often focus on immediate exam results'—is stated at the population level (Abstract, §1, §8), but the dataset cannot carry that scope. Section 4.1 reports recruitment via word-of-mouth and social media, with family participants predominantly from northern/central PAs (Beijing, Hebei, Henan, Inner Mongolia, Ningxia, Shaanxi, Anhui; Table 1), and all participants were already Quark users. Section 7.3 explicitly concedes regional concentration. The evidence for limited student involvement in §5.4 is explicitly anchored on nine participants (PF1–4, PK6, F11, F14, E1, E7), fewer than a third of the sample, and the recruitment deliberately over-included parents (7 parent–child pairs plus 4 individual family members). Self-reported retrospective accounts, without transcripts, codebooks, or app telemetry, cannot verify who actually operated the tool or how often. The qualitative themes may be valid, but the prevalence and inequity conclusions are over-scoped.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a qualitative interview study (N=32) of how Chinese students, parents, and experts use Quark GaoKao, an AI tool for college application decisions, during the GaoKao admissions process. It describes the tool's main features, analyzes interview themes around parent-led use, limited student involvement, score-centric recommendations, and inequities, and offers design implications for family-centered educational technology. The central claims are that AI tools are predominantly used by parents, focus on immediate exam scores rather than long-term career goals, and can deepen inequities for families with fewer resources.","tokens_in":24819,"tokens_out":3905,"duration_ms":39410,"significance":"Read as an exploratory qualitative account, the paper is timely and useful for the HCI community. Its strengths include a detailed and well-contextualized description of the GaoKao application workflow and Quark's features, a multi-stakeholder interview design (7 parent-child pairs, 6 individual children, 4 individual family members, and 8 experts), and a limitations section that candidly acknowledges regional concentration and the exclusion of students under 18. The interview quotes provide rich illustrative material, and the design implications in Section 7.1 connect the findings to family-centered design in a productive way. The main weakness is that several findings are stated at the population level in the Abstract and Conclusions even though the evidence base is a small, self-selected, regionally concentrated sample. The paper does not contain mathematical derivations or fitted parameters, so the standard circularity concerns do not apply; the single self-citation (Wang 2021) is used appropriately in the Discussion to contextualize emotional asymmetry.","major_comments":[{"comment":"The Abstract and §8 state that AI tools are 'predominantly used by parents' and 'often focus on immediate exam results' as general findings, but the recruitment described in §4.1 drew a self-selected, word-of-mouth sample of Quark users concentrated in northern and central PAs (Beijing, Hebei, Henan, Inner Mongolia, Ningxia, Shaanxi, Anhui; Table 1), and §7.3 explicitly concedes the regional concentration. The sample also over-included parents by design (7 parent-child pairs plus 4 individual family members). Prevalence language such as 'predominantly' should be replaced with 'in the families we interviewed' or accompanied by a sampling justification; otherwise the population-level statements are not supported by the data.","section":"Abstract and Conclusions (§8)"},{"comment":"The 'limited involvement' finding is anchored on nine participants (PF1–4, PK6, F11, F14, E1, E7), fewer than a third of the sample, and several of these are parents or experts reporting on children rather than the students themselves. This is a legitimate emergent theme, but it should be presented as an emergent pattern rather than as a general property of students' engagement with the tool. The current wording in §5.4 and the Abstract overstates the strength of the evidence.","section":"§5.4"},{"comment":"The claim of 'irresponsible use' of AI by consultant agencies rests largely on expert assertions, especially E8, and on the observation that contracts limit accountability, while the six families who used agencies reported being satisfied at the time of the study. The paper should explicitly label this as an expert-reported concern and note that no agency-side data or longitudinal outcome verification was collected. As written, the Abstract's 'irresponsible use' overstates what the data can establish.","section":"§6.5"},{"comment":"The conclusion that AI tools 'can inadvertently deepen inequities' is framed as a direct finding, but the evidence is interpretative: participants varied in resources and data literacy, yet there is no outcome measure, no systematic comparison of low-resource and high-resource families beyond individual quotes, and no verification of actual admission results. This should be reframed as an interpretive hypothesis grounded in participant reports rather than an empirically established effect.","section":"§7.2 and Abstract"}],"minor_comments":[{"comment":"The text mentions 'P12 mentioned the reliability of information sourced from books', but participant codes in this study use PF, PK, F, K, and E; this appears to be a typo and should read F12 or a corresponding code.","section":"§5.2"},{"comment":"The text says 'We consider our table a starting point' but refers to Figure 4, which is a figure rather than a table; the wording should be adjusted.","section":"§7.1"},{"comment":"There are several typographical errors in the appendix, including 'mainlan China', 'hightest ordered', and 'althought it might differ'; these should be corrected.","section":"Appendix A.1"},{"comment":"The statement that 'All of our participants reported using the default “prioritize colleges” option' is a strong feature-level usage claim; it should be softened to refer to the participants who discussed the feature, or supported with additional interview evidence.","section":"§3.2"},{"comment":"The paper describes Quark as 'widely utilized' and 'one of the most popular apps' but does not cite independent usage statistics; adding a source or qualifying the claim would improve precision.","section":"§1 and §3.2"}],"recommendation":"major_revision","confidential_remarks":"This is a solid qualitative CHI submission with a clear contribution if the prevalence claims are aligned with the evidentiary scope. The main revision is to temper the Abstract and Conclusions so that 'predominantly', 'often', and 'irresponsible use' are presented as sample-grounded patterns or expert-reported concerns rather than population-level facts. I do not see circularity; the single self-citation is used appropriately in the Discussion. The regional concentration and self-selection are acknowledged in §7.3, which is a positive sign, but the framing in the Abstract and Contributions still overreaches."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper gives us the first real look at how Chinese families actually use Quark GaoKao, the free AI tool for college applications in the GaoKao process. The central finding—parents drive the process, students mostly cross out options—is supported by the interview counts and quotes. The tool walk-through is unusually clear, and the design/implication discussion around family-centered design is thoughtful and grounded in the data. For HCI and ed-tech, that is a genuine contribution.\n\nThe stress-test concern lands. The abstract and conclusion say 'AI tools are predominantly used by parents' as if that were a population fact. The evidence is 32 self-selected, retrospective interviews, recruited via word-of-mouth and social media, mostly from northern/central provinces, with all participants already Quark users. The 'limited student involvement' theme rests on nine participants. No transcripts, codebooks, or app telemetry are available to check against. The limitation section admits the regional concentration—that honesty is welcome—but the framing still reaches beyond what the sample can support.\n\nI want to say clearly that the findings are not implausible. They fit prior work on family decision-making and on parents' protective roles in high-stakes choices. The fix is mostly editorial: soften the population-level claims, present the study as an exploratory account of one tool used by self-selected families, share a codebook or more participant quotes. The consultant-agency material is thin (six hiring families plus expert opinion) and should be marked exploratory.\n\nWho it is for: HCI researchers in family-centered design, AI in education, and algorithmic decision support. It deserves referee time. I would accept it for peer review, expecting a revision that aligns claims with the sample.","headline":"A solid exploratory interview study of AI use in Chinese college applications that overclaims in the title and abstract; the findings are plausible but the 32-interview, northern-China sample cannot carry population-level statements.","tokens_in":25384,"tokens_out":2866,"would_cite":true,"duration_ms":28946,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that AI college-application tools like Quark GaoKao are used mainly by parents, not students, and that a score-first design that ignores career goals can deepen educational inequities.","keywords":["Gaokao","AI college applications","Quark GaoKao","family decision-making","education equity","parent-led AI use","admission probability","career development"],"falsifier":"One concrete check: if access to Quark GaoKao's own usage logs, or a nationwide survey of 2024 Gaokao families, showed that students, not parents, are the primary operators of the app, or that students' stated career goals change materially after using it, the paper's central pattern would fail. A smaller qualitative check would be whether southern-China families show the same parent-led pattern outside the paper's northern sample.","tokens_in":24431,"feed_emoji":"🎓","tokens_out":7247,"duration_ms":67698,"temperature":0.7,"pith_summary":"The paper studies how Chinese families use AI-powered college-application tools, with Quark GaoKao as the concrete case, during the high-stakes Gaokao process. Based on 32 interviews with students, parents, and experts, it claims that these tools are predominantly operated by parents, that students engage mostly by rejecting options rather than forming preferences, and that the tools optimize for admission probability using exam scores while leaving long-term career goals out of the model. It further argues that this pattern can deepen educational inequities, because families with more awareness, data literacy, time, and money are better able to validate and override the AI's suggestions. The paper's contribution is a grounded account of who actually uses these tools and a set of design directions for putting students' voices and career development at the center.","feed_headline":"Parents, not students, run AI college-pick tools in China","feed_subtitle":"Interviews with 32 families and experts show score-first AI misses careers and widens gaps.","key_machinery":"The central object is Quark GaoKao (夸克高考), a free AI feature inside the Quark browser that generates admission-probability estimates and a ranked college list from a student's Gaokao score, provincial rank, and stated preferences. Its color-coded reach/target/safety categories and probability figures are the machinery the paper studies: they are what parents trust, triangulate, and override, and they are what the study uses to expose the gap between score optimization and career development. The paper also treats live streaming and program-popularity rankings inside the app as part of the same mechanism, because they shape recommendations in practice.","core_discovery":"The central claim is that a new generation of AI tools built for China's Gaokao, exemplified by Quark GaoKao, changes who does the deciding but not what the decision optimizes. The tool turns a score, a provincial rank, and location preferences into a color-coded list of reach, target, and safety colleges with admission probabilities, and in the families interviewed it is mostly parents who operate it. Students, by contrast, enter the process late, often after exam exhaustion, and participate by crossing options off a list compiled by parents or consultants rather than by articulating what they want. The paper reports that all stakeholder groups agree the app meets the immediate goal of score optimization, but that it does not address long-term career goals, personal interests, or the localized nuances of admission policy. It also finds that these limitations are not neutral: families with stronger data literacy, social networks, and money compensate by triangulating with guidebooks, live streams, and paid consultants, while families without those resources take the AI's recommendations at face value, which can deepen existing inequities.","pith_inferences":["Editorial inference: the parent-led pattern is likely not specific to Quark GaoKao; competing Chinese apps share the same score-and-probability interface, so the same dynamics probably appear there.","Editorial inference: a quantitative study could compare application lists produced by students alone versus parents alone to measure how much the tool's ranking, rather than family preference, drives the final choice.","Editorial inference: as provinces raise the allowed number of choices to 96 or more, the information burden grows, which would likely make the resource advantages the paper describes more consequential, not less.","Editorial inference: the 'crossing out' behavior suggests a testable design feature—tools that ask students to build a 'want' list before seeing probabilities—could shift involvement from rejection to preference formation."],"forward_implications":["AI application tools will remain parent-operated unless student engagement is designed in, because the current workflow rewards data literacy and time that parents, not students, have.","Score-and-probability rankings will continue to steer families toward popular majors, since the interface displays popularity as a default signal.","Families with fewer resources will rely more on live streams and paid consultants to interpret the AI's output, making outcomes depend on social and financial capital.","Third-party agencies can exploit 'AI' claims without immediate consequence because the effects of a college choice take years to appear and contracts do not tie outcomes to accountability.","Redirecting the tools toward long-term career questions would require adding data the tools currently lack: program-level employment outcomes, major-switching costs, and interest-based exploration paths."],"supporting_citations":[{"why":"Supplies the inductive thematic analysis method used to code the 32 interviews.","marker":"[8]"},{"why":"Provides the scale and competitiveness figures that establish why Gaokao decisions are high-stakes and time-pressured.","marker":"[20]"},{"why":"Grounds the claim that Gaokao admissions are score-based and that college application has become a family-wide effort.","marker":"[42]"},{"why":"Supplies the family-centered design framing that organizes the paper's design implications.","marker":"[9]"},{"why":"Grounds the family decision-making theory used to interpret information asymmetries between parents and students.","marker":"[28]"},{"why":"Documents how parents exercise control over AI recommendations in family settings, the related-work backdrop for parent-led usage.","marker":"[7]"},{"why":"Grounds the emotional asymmetry concept used to explain why parents shield students from decision pressure.","marker":"[45]"}],"fun_headline_variants":["AI college picks: Chinese parents drive, students sit out","Score-first AI in China: parents pilot, kids copilot","Gaokao AI: Parents run the app, students lose out","Chinese AI college picks: parents decide, careers ignored","AI for Gaokao: Parents click, kids watch, careers miss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 32 retrospective, self-selected interviews, most from northern China, give an accurate picture of how Chinese families use AI college-application tools.","fun_headline_variants_meta":{"raw":{"variants":["AI college picks: Chinese parents drive, students sit out","Score-first AI in China: parents pilot, kids copilot","Gaokao AI: Parents run the app, students lose out","Chinese AI college picks: parents decide, careers ignored","AI for Gaokao: Parents click, kids watch, careers miss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000498,"raw_usage":{"total_tokens":2423,"prompt_tokens":914,"completion_tokens":1509,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":1423}},"tokens_in":530,"tokens_out":1509,"duration_ms":10205,"temperature":1.0,"reasoning_tokens":1423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:46:25.675972+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete check: if access to Quark GaoKao's own usage logs, or a nationwide survey of 2024 Gaokao families, showed that students, not parents, are the primary operators of the app, or that students' stated career goals change materially after using it, the paper's central pattern would fail. A smaller qualitative check would be whether southern-China families show the same parent-led pattern outside the paper's northern sample.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the scale and competitiveness figures that establish why Gaokao decisions are high-stakes and time-pressured."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the claim that Gaokao admissions are score-based and that college application has become a family-wide effort."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the family-centered design framing that organizes the paper's design implications."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the family decision-making theory used to interpret information asymmetries between parents and students."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents how parents exercise control over AI recommendations in family settings, the related-work backdrop for parent-led usage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the emotional asymmetry concept used to explain why parents shield students from decision pressure."}],"review_version":1}