{"id":"a941faff-14cf-4849-809c-b1c78ad2bbd3","arxiv_id":"2412.05004","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"PromptCD uses learnable soft prompts shared between source and target domains to improve few-shot cross-domain cognitive diagnosis for both student-aspect and exercise-aspect scenarios.","lead":"This paper proposes PromptCD, a prompt-transfer framework for cross-domain cognitive diagnosis that adapts student knowledge assessments across different schools or subjects. It reports large accuracy gains over existing methods on the SLP education dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline gains may partly reflect a fine-tuning effect: Section V-A4 specifies that 20% of target records are used for fine-tuning, but the manuscript never states whether Origin, Tech, Zero, and CCLMF are trained on those same target records; without that symmetry, the reported AUC…","rationale":"The paper proposes a simple, plausible prompt-transfer wrapper and supports it with broad experiments, but the strongest claim is an empirical superiority claim whose validity depends on comparison symmetry. The reader's identified weakest assumption, that the baseline fine-tuning protocol is unspecified, is also the most load-bearing issue I find. If the baselines do not receive the same 20% target-domain fine-tuning, the reported AUC gains are compatible with a fine-tuning effect, and the Nemenyi tests would be comparing unequal protocols. I agree with the reader that this is the central soft spot. I would not move the verdict beyond CONDITIONAL because the issue is empirical and addressable: code inspection plus one controlled rerun would settle it. Secondary inconsistencies do not change the central concern but reinforce the need for the released code and a careful re-check before accepting the empirical claims. In particular, Table VI states that both inter- and intra-cluster distances become smaller after prompting, while the table shows inter-cluster distance increasing from 0.2911 to 12.2945, and Table V contains identical IRT-Zero and IRT-CCLMF columns in two scenarios, suggesting copy-paste error. These issues are fixable, and the central PromptCD construction is not internally defective, so conditional acceptance remains the appropriate verdict.","tokens_in":21053,"tokens_out":5324,"duration_ms":51570,"concrete_test":"Inspect the released repository at https://github.com/Publisher-PromptCD/PromptCD and verify whether the scripts for Origin, Tech, Zero, and CCLMF fine-tune on the same 20% target split used by PromptCD. Then rerun the A-bin student-aspect and Chinese exercise-aspect experiments with an additional 'Origin-FT' baseline: same backbone and hyperparameters as IRT-Origin, but trained on exactly the same 20% target records and evaluated on the same test split, using the same target-side epoch budget as PromptCD. If IRT-Origin-FT approaches or exceeds IRT-Ours (0.871 AUC on A-bin; 0.864 AUC on Chinese), the reported gains are best explained by target fine-tuning rather than prompt transfer; if IRT-Ours still wins by a large margin, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, that PromptCD consistently outperforms state-of-the-art baselines (Section V-B), requires that every compared method receives the same information from the target domain. Section V-A4 fixes 20% of target records for fine-tuning, but the baseline descriptions in Section V-A2 only say that Origin is 'without cross-domain prompt and parameter transfer' and that Tech, Zero, and CCLMF are applied to the backbone; none of them is explicitly stated to be trained on those 20% target records. For PromptCD, Algorithm 1 fine-tunes all trainable parameters, including embeddings, prompts, and aligned layers, on the target few-shot data. If Origin is merely trained on source data and evaluated on target test data, and if Tech, Zero, and CCLMF do not use the same 20% target records for adaptation, then the reported AUC gaps (e.g., IRT-Origin 0.736 vs IRT-Ours 0.864 for Chinese; NCDM-Origin 0.687 vs NCDM-Ours 0.879 for A-bin) could be dominated by the presence or absence of target-domain fine-tuning rather than by the prompt-transfer design. The claim of roughly 20% improvement would then not support the stated conclusion. This is an experimental-protocol ambiguity rather than an error in the PromptCD construction itself; the released code could resolve it, but the manuscript as written does not.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PromptCD, a soft-prompt-transfer framework for cross-domain cognitive diagnosis (CDCD) that handles both student-aspect (PromptCD-S) and exercise-aspect (PromptCD-E) scenarios. The framework augments backbone cognitive diagnosis models (IRT, MIRT, NeuralCD, KSCD) with personalized prompts for overlapping entities and shared domain prompts for non-overlapping entities, using a two-stage pre-train-on-source/fine-tune-on-target training scheme. Experiments on the SLP dataset across multiple target domains and four metrics report that PromptCD consistently outperforms the origin backbones and the TechCD, ZeroCD, and CCLMF baselines, with claimed gains such as IRT AUC from 0.736 to 0.864 on Chinese and NeuralCD AUC from 0.687 to 0.879 on the A-bin student target. The manuscript also includes ablation studies on fine-tuning ratios, prompt dimensions, source-domain choice, and a feature-visualization analysis.","tokens_in":21369,"tokens_out":5887,"duration_ms":244420,"significance":"If the reported results hold, PromptCD is a simple, backbone-agnostic method for CDCD that covers both student- and exercise-aspect scenarios, which is a useful and timely contribution to educational data mining. The paper ships a public implementation, which is a concrete strength that aids reproducibility and follow-up work. The experimental breadth is substantial: four backbones, multiple target domains, four metrics, and several ablations. However, the central claim of consistent state-of-the-art performance is currently not fully supported by the manuscript as written: the baseline training protocol is ambiguous with respect to target-domain fine-tuning, and the statistical-significance evidence is under-reported. The feature-visualization section also contains a clear inconsistency between the text and the reported table.","major_comments":[{"comment":"The manuscript specifies in Section V-A4 that 20% of the target-domain interaction records are randomly selected for fine-tuning, but it never states whether the baselines (Origin, Tech, Zero, CCLMF) are also fine-tuned on those same target records. For PromptCD, Algorithm 1 explicitly fine-tunes all trainable parameters, including prompts and aligned layers, on the target few-shot data. If the baselines are trained only on source data and evaluated on target test data, then the reported gains (e.g., IRT-Origin 0.736 vs. IRT-Ours 0.864 for Chinese in Table II; NeuralCD-Origin 0.687 vs. NeuralCD-Ours 0.879 for A-bin in Table III) may reflect the presence or absence of target-domain fine-tuning rather than the benefit of prompt transfer. This is load-bearing for the Section V-B claim that PromptCD consistently outperforms state-of-the-art baselines. Please state explicitly whether every baseline receives the same 20% target fine-tuning, and if it does not, rerun the comparison with identical target-domain fine-tuning for all methods.","section":"Section V-A2/V-A4, Tables II and III, Algorithm 1"},{"comment":"The sentence 'the inter-cluster and intra-cluster distances are significantly smaller after introducing prompts compared to before' contradicts the numbers in Table VI. The table shows that intra-cluster distances decrease (e.g., exercise embedding 6.6352 → 2.8690; student embedding 7.8856 → 2.9120), but inter-cluster distances increase markedly (exercise 0.2911 → 12.2945; student 0.6041 → 13.0087). The correct reading is that prompts increase inter-cluster separation while reducing intra-cluster variance; the text as written misreports the evidence. Please correct the wording to match the table.","section":"Section V-D, Table VI"},{"comment":"The main results tables report only single numbers with no variance across random seeds or data splits, and the manuscript states that Nemenyi tests were conducted but provides no quantitative details (number of runs, critical difference values, p-values, or exact test procedure). The statement that PromptCD 'significantly outperforms' other models is therefore not verifiable from the information given. Please report mean and standard deviation over at least five seeds and provide the full Nemenyi test results, including the underlying per-run metrics.","section":"Section V-B, 'Significance Analysis', Tables II–III"}],"minor_comments":[{"comment":"In Section V-A4 the paper says '20% of the interaction records in the target domain are randomly selected for fine-tuning' but does not specify the random seed or whether the same split is used across all methods; please state this. In Section V-C, 'Various Cross-Domain Types' says 'in both cases' while Table V contains four scenarios; please make the description consistent.","section":"Section V-A4 / Section V-C"},{"comment":"Equation (7), o_orig_t = Linear_init(p_o), uses assignment-like notation for an initialization; it would be clearer to state that the initial values of the target original embeddings are computed as Linear_init(p_o), to avoid confusion with an equality constraint.","section":"Equation (7), Section III-C"},{"comment":"The inter-cluster and intra-cluster distances are not defined in the text; please specify how these quantities are computed (e.g., mean pairwise distances within and between clusters).","section":"Section V-D, Table VI"},{"comment":"There is a typo in Algorithm 1 line 2 and Algorithms 2 and 3 line 1: 'doamin' should be 'domain'. Also, the algorithms refer to 'epoch' limits but the exact epoch numbers and learning-rate settings are not given in the experimental section; please include these details for reproducibility.","section":"Algorithms 1, 2, 3"},{"comment":"Figure 6, referenced in the significance analysis, is not included in the text of the manuscript; ensure that the figure is present in the final version and that its contents (Nemenyi diagrams) are legible and self-explanatory.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real contribution, not a revolution. The paper takes soft prompt transfer, established in NLP and cross-domain recommendation, and adapts it to cognitive diagnosis with a clean split: personalized prompts for overlapping entities, shared domain prompts for non-overlapping ones, plus a linear mapping to initialize target representations. That split is the actual new bit. The two instantiations (student-aspect and exercise-aspect) are straightforward and the framework is model-agnostic across IRT, MIRT, NeuralCD, KSCD. Code is public. I believe the central idea is sound.\n\nWhat it does well: broad evaluation. Four backbones, four metrics, multiple target domains, Nemenyi significance tests, fine-tuning ratio and source domain ablations, and a small recommendation demo. The gains over Origin are large (e.g., IRT AUC 0.667 to 0.798 for Biology). The Ours+ variant with the prompt-to-representation mapping consistently helps. The visualization and cluster distance table support the claim that prompts tighten inter/intra-cluster structure.\n\nSoft spots: first, the comparison protocol. Section V-A4 says 20% of target records are used for fine-tuning, but the text never explicitly says TechCD, ZeroCD, CCLMF, or Origin are also fine-tuned on those same 20% records. If Origin is only trained on source and evaluated on target, and the cross-domain baselines do not get target adaptation, then a large part of the reported AUC gap is just the fine-tuning effect. This is the main thing I'd want fixed before trusting the headline numbers. The released code may resolve it, but the manuscript does not. Second, one dataset only (SLP), with no seed-level variance in the main tables. For a paper making 'consistently outperforms' claims, three seeds and std bars would be much more convincing. Third, in Table V the IRT-CCLMF row for Humanities-Sciences and Sciences-Sciences exactly duplicates the IRT-Zero row across all four metrics. That looks like a copy-paste error and needs correcting. Fourth, the claim in the text that 'in both cases, PromptCD outperforms' is confusing given the table has four scenarios; the writing slipped there.\n\nNone of these is fatal. The method is simple, the experiments are substantial, and the baseline-symmetry issue is an experimental-protocol ambiguity, not a flaw in the method's construction. This paper is for people working in educational data mining and cross-domain recommendation. A serious referee should get it, mainly to push the authors to disclose the exact training/fine-tuning protocol for every baseline and to add variance estimates. I'd bring it to reading group as a good case study in how much of a reported gain is due to the method versus due to giving one side extra target data.","headline":"PromptCD is a sensible, well-tested application of soft-prompt transfer to cross-domain cognitive diagnosis, but the main comparison table has an unresolved fine-tuning symmetry question and a couple of data-reporting slips.","tokens_in":21847,"tokens_out":2523,"would_cite":true,"duration_ms":21825,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that attaching learnable soft prompts to student and exercise representations and transferring them between domains substantially improves cross-domain cognitive diagnosis in both student-aspect and exercise-aspect…","keywords":["cognitive diagnosis","cross-domain cognitive diagnosis","soft prompt transfer","student-aspect CDCD","exercise-aspect CDCD","educational data mining","few-shot fine-tuning"],"falsifier":"Re-run the student-aspect A-bin experiment giving each baseline (Origin, Tech, Zero, CCLMF) the identical 20% of target records for fine-tuning that PromptCD receives; if the baselines' AUCs rise toward the reported 0.87-0.88, the gap is a fine-tuning effect rather than prompt transfer. A complementary check is to ablate the prompts inside PromptCD, keeping only the aligned linear layers and the same fine-tuning budget, and compare the resulting AUC against the full prompt version.","tokens_in":20896,"feed_emoji":"🎓","tokens_out":13940,"duration_ms":116998,"temperature":0.7,"pith_summary":"Cognitive diagnosis models predict whether a student will answer an exercise correctly, and they typically suffer accuracy drops when applied to a new school's students or a new subject's exercises, the cross-domain setting this paper targets. The paper claims that a simple, model-agnostic recipe called PromptCD fixes the drop: attach small learnable vectors (soft prompts) to student and exercise representations, train them on source domains, transfer them to the target domain, and fine-tune on only 20% of target records. Across four standard diagnosis backbones and two scenario families (student-aspect and exercise-aspect), the paper reports ranking-accuracy (AUC) gains of up to roughly 0.19, with the largest student-side jump from 0.687 to 0.879. If true, it offers a unified way to keep diagnosis accurate in a new domain without retraining from scratch, the condition under which personalized exercise recommendation and learning guidance become deployable in sparse-data settings.","feed_headline":"28 percent better AUC for cross-domain diagnosis via soft prompts","feed_subtitle":"Prompt framework keeps diagnosis accurate on new student groups and subjects with only 20% of target records.","key_machinery":"The load-bearing object is the pair of soft prompts: a personalized prompt $p^o_i$ for each entity in the overlapping set $O$ and a shared, domain-level prompt $p^d_k$ for all non-overlapping entities of a source domain. Entities enter the diagnosis model as concatenations $o^{cat}_{k,i} = [p^o_i, o^{orig}_{k,i}]$ and $d^{cat}_{k,j} = [p^d_k, d^{orig}_{k,j}]$, projected by a linear aligned layer, so the prompt becomes part of the representation that the interaction function scores. Transfer is the second mechanism: personalized prompts pass through one-to-one, $\\hat{p}^o_i = p^o_i$, while shared prompts from all source domains are fused by concatenation and a linear source-to-target map, $\\hat{p}^d_t = \\mathrm{Linear}_{s2t}(p^d_1 \\oplus p^d_2 \\oplus \\cdots \\oplus p^d_{|S|})$; an optional prompt-to-representation mapping, $o^{orig}_t = \\mathrm{Linear}_{\\mathrm{init}}(\\hat{p}^o)$, initializes overlapping target entities from their prompts instead of random vectors. The two-stage training (source pre-training, target fine-tuning) is what makes the transfer efficient.","core_discovery":"On the paper's own terms, the discovery is that cross-domain transfer in cognitive diagnosis can be carried entirely by learnable prompts, leaving the diagnosis model itself unchanged. Entities present in both source and target domains (the overlapping set $O$) each receive a personalized prompt $p^o_i$, while groups of entities unique to a domain (the non-overlapping set $D$) share one domain prompt $p^d_k$; the enhanced representations $o^{cat} = [p^o_i, o^{orig}]$ and $d^{cat} = [p^d_k, d^{orig}]$ are aligned by a linear layer and fed to the diagnosis model. Pre-training learns these prompts on source interaction records, and fine-tuning on a 20% slice of target records adapts them: personalized prompts transfer one-to-one, shared prompts from all sources are concatenated and mapped through a linear layer, and an optional prompt-to-representation mapping initializes overlapping target entities from their source prompts. The paper reports that PromptCD and its Ours+ variant beat the untransferred backbones and the prior cross-domain methods TechCD, ZeroCD, and CCLMF across exercise-aspect targets (humanities and sciences) and student-aspect targets (four school-ability bins), for IRT, MIRT, NeuralCD, and KSCD alike, with Nemenyi tests claimed to confirm the significance.","pith_inferences":["My inference: the mechanism implies that what transfers across domains is a compact per-entity identity code, not the diagnostic interaction function; if so, the same prompts learned for one backbone could seed a different backbone without re-running the source pre-training.","My inference: the shared-prompt fusion step (concatenation plus a linear map) is one of many possible fusion rules; attention-weighted or adversarial fusion of source prompts is a testable variant that might scale better beyond two source domains.","My inference: the personalized/shared prompt split should generalize beyond education to any two-sided prediction problem with partially overlapping entities, such as recommendation across user cohorts or item catalogs, where the overlapping side can carry personalized prompts."],"forward_implications":["A school or platform adopting PromptCD gains a working cross-domain diagnosis model with only about 20% of the new domain's interaction records, because the reported gains hold even at a 10% fine-tuning ratio.","Because the framework is model-agnostic, the improvements carry over to classic latent-trait models (IRT, MIRT) and neural models (NeuralCD, KSCD) alike, so institutions need not switch diagnosis backbones to benefit.","Multi-source settings help: using two source domains outperforms one in the paper's ablation, so pooling several schools or subjects should further improve target-domain diagnosis.","The prompt-to-representation mapping variant (Ours+) consistently edges out the base version, indicating that personalized prompts trained on overlapping entities carry reusable identity information for initializing new-domain representations.","Cross-domain drops between dissimilar subjects (such as sciences to humanities) shrink but remain slightly larger than within-discipline transfers, giving a practical ordering of which transfers are worth attempting."],"supporting_citations":[{"why":"Supplies the real-world SLP dataset used in every experiment, including the cross-school and cross-subject scenarios.","marker":"[20]"},{"why":"TechCD is the exercise-aspect cross-domain baseline whose performance PromptCD must beat.","marker":"[21]"},{"why":"ZeroCD is the few-shot cross-domain baseline PromptCD compares against.","marker":"[22]"},{"why":"CCLMF is the meta-learning cross-course baseline used in the comparison.","marker":"[30]"},{"why":"MIRT is one of the four backbones and the model used in the motivating in-domain versus cross-domain accuracy experiment.","marker":"[19]"},{"why":"NeuralCD provides the neural backbone and interpretable interaction framework that PromptCD wraps in the exercise-recommendation demonstration.","marker":"[15]"},{"why":"IRT is one of the four backbones whose cross-domain AUC gains PromptCD reports.","marker":"[29]"},{"why":"KSCD supplies the fourth backbone, exercising the framework on a knowledge-concept-aware diagnosis model.","marker":"[3]"}],"fun_headline_variants":["Prompt transfer improves cross-domain cognitive diagnosis","Soft prompts bridge domain gaps in cognitive diagnosis","Dual-aspect prompt learning for portable cognitive diagnosis","Learnable prompts adapt cognitive diagnosis across domains","Prompt framework excels in cross-domain student and subject shift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes every baseline received the same target-domain fine-tuning as PromptCD, but the paper does not state whether the baseline models were tuned on the same 20% of target records or only trained on source data.","fun_headline_variants_meta":{"raw":{"variants":["Prompt transfer improves cross-domain cognitive diagnosis","Soft prompts bridge domain gaps in cognitive diagnosis","Dual-aspect prompt learning for portable cognitive diagnosis","Learnable prompts adapt cognitive diagnosis across domains","Prompt framework excels in cross-domain student and subject shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000484,"raw_usage":{"total_tokens":2432,"prompt_tokens":1031,"completion_tokens":1401,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":1332}},"tokens_in":647,"tokens_out":1401,"duration_ms":15495,"temperature":1.0,"reasoning_tokens":1332,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:59:09.855895+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the student-aspect A-bin experiment giving each baseline (Origin, Tech, Zero, CCLMF) the identical 20% of target records for fine-tuning that PromptCD receives; if the baselines' AUCs rise toward the reported 0.87-0.88, the gap is a fine-tuning effect rather than prompt transfer. A complementary check is to ablate the prompts inside PromptCD, keeping only the aligned linear layers and the same fine-tuning budget, and compare the resulting AUC against the full prompt version.","supporting_citations":[{"cited_title":"Slp: A multi-dimensional and consecutive dataset from k-12 education","cited_arxiv_id":null,"evidence_quote":"Supplies the real-world SLP dataset used in every experiment, including the cross-school and cross-subject scenarios."},{"cited_title":"Leveraging transferable knowledge concept graph embedding for cold-start cognitive diagnosis","cited_arxiv_id":null,"evidence_quote":"TechCD is the exercise-aspect cross-domain baseline whose performance PromptCD must beat."},{"cited_title":"Zero-1-to-3: JOURNAL OF LATEX CLASS FILES, VOL","cited_arxiv_id":null,"evidence_quote":"ZeroCD is the few-shot cross-domain baseline PromptCD compares against."},{"cited_title":"Ptadisc: a cross-course dataset supporting personalized learning in cold-start scenarios","cited_arxiv_id":null,"evidence_quote":"CCLMF is the meta-learning cross-course baseline used in the comparison."},{"cited_title":"Multidimensional item response theory models","cited_arxiv_id":null,"evidence_quote":"MIRT is one of the four backbones and the model used in the motivating in-domain versus cross-domain accuracy experiment."},{"cited_title":"Neuralcd: A general framework for cognitive diagnosis","cited_arxiv_id":null,"evidence_quote":"NeuralCD provides the neural backbone and interpretable interaction framework that PromptCD wraps in the exercise-recommendation demonstration."},{"cited_title":"Item response theory","cited_arxiv_id":null,"evidence_quote":"IRT is one of the four backbones whose cross-domain AUC gains PromptCD reports."},{"cited_title":"Knowledge-sensed cognitive diagnosis for intelligent education platforms","cited_arxiv_id":null,"evidence_quote":"KSCD supplies the fourth backbone, exercising the framework on a knowledge-concept-aware diagnosis model."}],"review_version":1}