{"id":"4593b4f5-580e-4b4a-bb4a-bacc9f82b847","arxiv_id":"2505.22541","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"This thesis shows that post-hoc explainers disagree systematically in educational models, and that intrinsically interpretable architectures (MultiModN, InterpretCC) with an LLM explanation layer (iLLuMinaTE) keep accuracy while producing explanations users trust and act on.","lead":"An EPFL doctoral thesis argues that post-hoc explanation tools like LIME and SHAP are too inconsistent for trustworthy AI in education, and proposes interpretable-by-design neural architectures plus an LLM-based explanation layer instead. It supports this shift with four user studies involving professors, teachers, learning scientists, and students.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The thesis's human-centric claim rests on a self-defined five-target yardstick from the author's own under-review paper (§1.3); if stakeholders weight or extend those criteria differently, the comparative verdicts for post-hoc vs. intrinsic methods do not follow.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: the five design targets are self-authored, not elicited from stakeholders, yet every comparative claim is scored against them. I agree that this is the central point on which the thesis's human-centric framing depends. I also considered the CI-overlap parity claims in Chapters 6 and 7, the GPT-4o self-annotation in Chapter 9, and the small convenience samples; these are real secondary concerns, but they weaken specific chapters rather than the overall comparative framework. The thesis is a compilation of peer-reviewed work, discloses several limitations, and clearly attributes the framework to [SFK23], so the appropriate verdict remains CONDITIONAL rather than REJECT. No adjustment to the reader's verdict is needed; the burden is to validate the yardstick, and until that is done the central claim is conditionally supported at best.","tokens_in":49777,"tokens_out":11046,"duration_ms":147530,"concrete_test":"Pre-register a criterion-elicitation study with the same stakeholder groups (teachers, students, learning scientists) using open-ended questions plus Best-Worst Scaling over a candidate set containing the five targets and plausible omitted dimensions (fairness, causal validity, action cost, privacy, behavioral learning outcomes). Use the resulting criteria and weights to re-score the methods compared in Chapters 4, 7, and 9 (e.g., LIME/SHAP/CEM vs. InterpretCC and iLLuMinaTE). If the stakeholder-derived set or weights differ materially from the five self-defined targets, or if the method ordering changes under the new weights, the human-centric claim and the 'post-hoc unsuitable' verdict do not follow from the current evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 1.3 defines the five design targets (accurate-with-certainty, real-time, consistent, actionable, human-understandable) and states they are 'motivated originally in [SFK23]' — the author's own under-review viewpoint paper. Every contribution chapter then shows that its method satisfies these targets, so the thesis's central comparative claims — post-hoc explainers are unsuitable as a trust layer, intrinsic models plus iLLuMinaTE are the human-centric solution, and classroom adoption is feasible — all presuppose that this yardstick is correct, complete, and correctly weighted. The within-study user weights (e.g., InterpretCC Table 7.4) do not fix this, because the dimensions made available for ranking were themselves chosen by the authors. If real deployment requires additional or differently weighted criteria — fairness, causal validity, cost of acting on an explanation, privacy, or downstream learning outcomes — the evidence as presented cannot rank the proposed methods against those criteria. The thesis is transparent about the source of the framework, but transparency does not supply the missing stakeholder validation; this is the load-bearing premise least secured by the presented evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This PhD-style manuscript argues that post-hoc explainability methods (LIME, KernelSHAP, PermSHAP, CEM, DiCE) are systematically inconsistent when applied to student success prediction models and are therefore unsuitable as a trust layer for personalized education. It then proposes a shift toward intrinsically interpretable architectures: MultiModN (sequential multimodal fusion), InterpretCC (sparse mixture-of-experts with feature or group routing), adversarial training for explainer stability, and iLLuMinaTE (an LLM-based theory-driven communication layer that turns XAI outputs into feedback for students). The thesis evaluates these contributions across large MOOC datasets, five explainers, and four human studies with professors, teachers, learning scientists, and students, claiming that explainability can be made accurate, real-time, consistent, actionable, and human-understandable.","tokens_in":49869,"tokens_out":5876,"duration_ms":66502,"significance":"The constituent chapters of this thesis have already passed peer review at strong venues (NeurIPS, ICLR, AAAI, LAK, Learning@Scale, EDM), and the visible methodology is careful: balanced accuracy on imbalanced MOOCs, stratified sampling of representative students, five random seeds with overlapping confidence intervals in Chapter 8, and multiple human studies with domain experts. The authors also release code and models for the main technical contributions. If the framework-level claims are taken at face value, the work would be an important step toward human-centric XAI in education, particularly the move from post-hoc approximation to interpretable-by-design architectures combined with an LLM communication layer. The main risk is not the individual experiments but the framework that ties them together: the five design targets are sourced from the author's own under-review viewpoint paper and are then used as the yardstick for all comparative verdicts. This circularity is load-bearing and needs to be addressed head-on.","major_comments":[{"comment":"The five design targets (accurate with certainty, real-time, consistent, actionable, human-understandable) are introduced in Section 1.3 as 'motivated originally in [SFK23]', the author's own under-review viewpoint paper, rather than as a standard elicited from stakeholders or derived from an established framework. Every contribution chapter then demonstrates that its method satisfies these targets, and Chapter 10 organizes the conclusions around them. This creates a circularity at the framework level: the comparative verdicts (post-hoc explainers fail; intrinsic models plus iLLuMinaTE succeed) presuppose that this self-authored yardstick is correct, complete, and correctly weighted. The user studies in Chapter 7 (Table 7.4) and Chapter 9 (Figure 9.5) let participants rank or choose within the dimensions the authors made available, but they do not elicit the dimensions themselves or their relative weights. If real deployment requires additional or differently weighted criteria, such as fairness, causal validity, the cost of acting on an explanation, privacy, or downstream learning outcomes, the evidence as presented cannot rank the proposed methods against those criteria. The thesis is transparent about the source of the framework, but transparency does not supply the missing stakeholder validation. I would ask for either a stakeholder elicitation study that derives and weights the design targets independently, or a substantial reframing that treats the five targets as working hypotheses rather than as the established basis for comparative conclusions.","section":"Section 1.3 and Chapter 10"},{"comment":"The early-dropout filtering step removes students whose predicted failure probability exceeds 0.99 according to a logistic regression trained on assignment grades from the first two weeks. This is an ad hoc preprocessing choice: the threshold is tuned by grid search on the training courses, and the transfer results reported in Tables 3.2 and 3.3 and in Figures 3.3 are computed on the filtered population. Figure 3.4 shows that the broad patterns are similar on the full population, and the text states that the predictors 'have high accuracy on these students as well', but no quantitative comparison of transfer performance on filtered versus unfiltered populations is provided, nor is there a sensitivity analysis of the threshold. Because the filtering disproportionately affects courses with different assignment schedules, lengths, and dropout rates, this could bias the conclusion that behavior-plus-meta models transfer better than same-course baselines. The claim of generalizable student modeling needs a robustness check showing that the transfer rankings are stable across reasonable choices of the filtering threshold and across courses with very different early-assessment structures.","section":"Section 3.2.2 and Figure 3.4"},{"comment":"The central 'systematic disagreement' result is based on explanations for 100 uniformly sampled students per course and a single trained BiLSTM per course in Chapter 4, with a related design in Chapter 5. The sampling strategy balances on predicted probability intervals, but the resulting set is not shown to be representative of the full student distribution, and the PCA, Spearman correlation, and Jensen-Shannon distance analyses do not report bootstrap confidence intervals or variation across random seeds. Chapter 8 later studies seed variation and finds substantial explainer variance, but the earlier chapters' strong claim that the choice of explainer dominates the choice of course or model would be considerably more robust if the original comparison included multiple model initializations and repeated sampling with uncertainty estimates. As it stands, the claim could be sensitive to the particular 100-student samples and the single trained model per course.","section":"Section 4.3.2 and Figure 4.3"},{"comment":"The 'actionable' design target is validated through student self-reported preference and Likert ratings (e.g., 89.2% preference, usefulness and actionability scores in Figures 9.5 and 9.6), not by measuring whether students or teachers can actually implement the suggestions or whether doing so improves learning outcomes. The abstract's claim that the work 'lays a foundation' for human-centric AI systems is more modest than the introduction's suggestion that classroom adoption becomes feasible, but the gap remains: perceived actionability is demonstrated, realized actionability is not. A small deployment study, a follow-up behavior measurement, or an explicit limitation statement that realized actionability is outside the scope would help calibrate the conclusions.","section":"Chapter 9 and overall framing"}],"minor_comments":[{"comment":"The text says 'As illustrated in Figure 8.1' when describing the taxonomy of explainability stages; the correct cross-reference should be Figure 2.1.","section":"Section 2.2"},{"comment":"There are typos: 'innaccuracies' in Section 1.2 and 'vase majority' in Section 2.2. Also, the sentence 'most of these studies have performed a-posteriori or post-hoc analyses, and not models for early success prediction' in Section 2.1 is awkward and should be rephrased.","section":"Sections 1.2 and 2.2"},{"comment":"Both tables refer to 'Section 7.5' for the features identified as important by at least one explainability method, but the relevant analyses appear in Sections 4.3.3 and 5.3; these cross-references should be corrected.","section":"Tables 4.1 and 5.3"},{"comment":"Table 9.1 reports that explanations are 'Annotated by experts and GPT-4o', but the number of expert annotators and their inter-rater agreement are not stated in the main text; the reader should be pointed to Appendix E.2 with a concrete agreement figure.","section":"Table 9.1"}],"recommendation":"major_revision","confidential_remarks":"This is a strong compilation of already peer-reviewed work, and I see no reason to doubt the internal soundness of the individual chapters. My concern is the framing: the unified 'human-centric' claim rests on a five-target yardstick that originates from the author's own under-review viewpoint paper, and the chapters are then organized as demonstrations of that yardstick. That circularity is real and load-bearing, but it is fixable within the scope of a revision: either add an independent stakeholder elicitation or explicitly demote the five targets to working hypotheses and soften the comparative conclusions. I would also ask the editor to ensure that the thesis-level framing does not overstate the evidence for classroom adoption, since the actionability results are self-reported rather than behavioral."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take.\n\nThis is a cumulative PhD thesis, not a single-study paper. The core chapters already went through strong venues (L@S, EDM, LAK, NeurIPS, ICLR, AAAI), and the reprints are methodologically careful: balanced accuracy on imbalanced MOOCs, stratified sampling of students, multiple random seeds with overlapping confidence intervals, and honest disclosure of limitations. The genuinely new material is Chapter 8 on adversarial training for explainer consistency, plus the synthesis around five design targets.\n\nWhat the thesis does well: it makes a concrete empirical case that post-hoc explainers (LIME, SHAP, CEM, DiCE) systematically disagree when applied to the same educational models, and it backs the argument for interpretable-by-design architectures (MultiModN, InterpretCC) with human studies, including a teacher evaluation and a student preference study for the LLM layer. The iLLuMinaTE result that 89.2% of students prefer the LLM-generated explanations is worth taking seriously.\n\nThe soft spot is the one the stress-test flagged. Section 1.3 defines the five design targets (accurate-with-certainty, real-time, consistent, actionable, human-understandable) as \"motivated originally in [SFK23]\" — the author's own under-review viewpoint paper. Every contribution chapter then shows its method satisfies those targets. So the comparative verdicts — post-hoc explainers fail, intrinsic models plus iLLuMinaTE succeed — presuppose that this yardstick is correct, complete, and correctly weighted. The thesis is transparent about this, but transparency doesn't supply the missing stakeholder validation. If real deployments require additional criteria, say fairness, causal validity, cost of acting on an explanation, privacy, or downstream learning outcomes, the evidence as presented cannot rank the methods against those criteria. This is a real concern, but addressable. It doesn't undermine the constituent published results; it limits the strength of the summary claim that this is the \"human-centric\" solution.\n\nOther, smaller caveats: the central MOOC dataset is private, from one university, 2013–2015, so the quantitative claims can't be independently rerun. The human studies are small convenience samples measuring self-reported preference rather than behavioral outcomes. Chapter 9's alignment scores are annotated by GPT-4o, the same model family that generated the text — a point the thesis itself discloses.\n\nWho gets value from this: anyone working in AI for education or XAI who wants a consolidated view of interpretable-by-design architectures and their evaluation, and PhD students looking for a model of how to structure a cumulative thesis with aligned human studies.\n\nMy recommendation: send it to serious peer review. The reviewer should focus on framework validation and dataset accessibility. This is a solid body of work with a clearly stated limitation in its self-defined evaluative framework.","headline":"A cumulative PhD thesis with a strong peer-reviewed core and a self-defined evaluation framework that needs scrutiny on criterion completeness.","tokens_in":50567,"tokens_out":2537,"would_cite":true,"duration_ms":26490,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This thesis argues that post-hoc explainers are too inconsistent to serve as a trust layer in education, and that interpretable-by-design architectures plus an LLM explanation layer can deliver explanations that are accurate, real-time…","keywords":["Explainable AI","Human-Centric AI","Personalized education","Post-hoc explainers","Interpretable-by-design architectures","Student success prediction","Counterfactual explanations","LLM-XAI"],"falsifier":"Give teachers and students the same predictions with explanations from both a post-hoc pipeline and the proposed intrinsic-plus-LLM pipeline, ask them to make real advising decisions under blind conditions, and include fairness, causal validity, and cost of acting among the evaluation criteria; if post-hoc explanations prove equally actionable or more trusted, the thesis's central ranking would not hold.","tokens_in":49388,"feed_emoji":"🎓","tokens_out":4360,"duration_ms":45129,"temperature":0.7,"pith_summary":"The paper claims that popular post-hoc explainers—LIME, KernelSHAP, PermSHAP, CEM, and DiCE—systematically disagree when applied to the same student success models, making them unsuitable as a basis for trust in educational AI. It argues that the field should shift to inherently interpretable model architectures, which can match non-interpretable baselines in predictive performance while offering faithful, real-time explanations. The thesis then adds an LLM-based communication layer that turns those explanations into natural-language feedback, which students preferred 89.2% of the time in a user study. A sympathetic reader would take the central claim to be that a human-centric XAI stack—intrinsic interpretability plus an LLM explanation layer—can make AI-driven personalized education both accurate and trustworthy enough for real classrooms.","feed_headline":"Post-hoc AI explainers fail the trust test; new models pass","feed_subtitle":"Explanation method, not model, drives what AI highlights; interpretable-by-design replacements earn 89% student preference.","key_machinery":"The load-bearing mechanism is a set of five design targets for explanations—accurate with certainty, real-time, consistent, actionable, and human-understandable—used as the evaluation yardstick throughout the thesis. The technical machinery consists of three complementary components: MultiModN, which sequentially fuses latent representations from multiple modalities and therefore makes each modality's contribution naturally inspectable; InterpretCC, which uses adaptive sparsity to activate only the features or feature groups needed for each prediction, so the explanation is the actual decision path; and iLLuMinaTE, an LLM-XAI pipeline whose multi-stage prompts align generated feedback with cognitive theories of explanation. Together these components are intended to satisfy the five design targets simultaneously, without sacrificing accuracy.","core_discovery":"The core discovery is that explainability method choice, not the underlying model or the student data, dominates what an explanation highlights: across five courses and five explainers, the feature importance distributions clustered by explainer rather than by course, with LIME consistently farthest from the others. This systematic disagreement, together with expert educators disagreeing about which explanations to trust, motivates the thesis's move to architectures that are interpretable by design. The proposed models—MultiModN, a sequential multimodal modular network, and InterpretCC, a sparse mixture-of-experts model—match non-interpretable baselines on balanced accuracy while providing explanations directly tied to the model's own decision process. Adversarial training improves the consistency of counterfactual explanations but not of LIME and SHAP, showing that consistency must be addressed holistically. Finally, the iLLuMinaTE pipeline uses large language models to convert feature-based explanations into concise, actionable feedback aligned with social-science explanation theories, and students prefer these explanations for clarity and actionability.","pith_inferences":["If the evaluation criteria were broadened to include fairness, causal validity, or the cost of acting on an explanation, the relative ranking of post-hoc versus intrinsic approaches could change; the thesis does not test those criteria.","The same evaluation template—five design targets plus user studies—could be exported to other human-centric domains such as healthcare or hiring, although the thesis only demonstrates it in education.","A testable extension is whether the LLM communication layer would make post-hoc explanations appear equally actionable, which would separate the communication win from the architecture win."],"forward_implications":["Classroom adoption of AI becomes more feasible because students receive instant, understandable feedback that they can act on, rather than raw feature scores.","Post-hoc explainers should not be treated as a trust layer in education, since their explanations vary more with method choice than with the actual model or data.","Interpretable-by-design architectures can match black-box accuracy while providing faithful, real-time explanations, removing the presumed tradeoff in this setting.","Adversarial training can stabilize counterfactual explanations, but LIME and SHAP remain sensitive to random perturbations, so explainer consistency cannot be fixed by model robustness alone.","Course design decisions can be grounded in explanations when educators are shown an ensemble of explainers, with a majority finding at least one actionable interpretation."],"supporting_citations":[{"why":"Supplies the five design targets for human-centric explanations that structure the thesis's evaluation.","marker":"[SFK23]"},{"why":"Reports the systematic disagreement among post-hoc explainers across educational course models.","marker":"[Swa+22a]"},{"why":"Provides the educator validation study showing explanations are seen as actionable but experts disagree on trustworthiness.","marker":"[Swa+23]"},{"why":"Underpins the critique that post-hoc explanations can be unfaithful to the true model decision process.","marker":"[Rud19]"},{"why":"Documents inconsistency across explainers, supporting the thesis's motivation for stable explanations.","marker":"[Sla+20]"},{"why":"Contributes the faithfulness metrics used to evaluate explanation quality across models.","marker":"[Aga+22]"},{"why":"Introduces MultiModN, the interpretable multimodal modular architecture that matches baselines with a fraction of parameters.","marker":"[Swa+24c]"},{"why":"Introduces InterpretCC, the sparse mixture-of-experts model that produces concise explanations without sacrificing performance.","marker":"[Swa+24b]"},{"why":"Introduces iLLuMinaTE, the LLM-XAI pipeline evaluated with 114 students and 89.2% preference for clarity and actionability.","marker":"[Swa+24a]"}],"fun_headline_variants":["AI explainability: method choice trumps model and data","Interpretable AI models win over post-hoc explainers in class","Students trust AI that explains itself, not black-box guesswork","Why LIME and SHAP differ: method, not model, drives highlights","Explainable AI for education: design it in, don't add it on"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The five design targets are the thesis's own yardstick, drawn from a viewpoint paper rather than from stakeholders, so the verdict that post-hoc methods fail and intrinsic models succeed presupposes that these criteria are the right ones.","fun_headline_variants_meta":{"raw":{"variants":["AI explainability: method choice trumps model and data","Interpretable AI models win over post-hoc explainers in class","Students trust AI that explains itself, not black-box guesswork","Why LIME and SHAP differ: method, not model, drives highlights","Explainable AI for education: design it in, don't add it on"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1542,"prompt_tokens":1043,"completion_tokens":499,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":408}},"tokens_in":659,"tokens_out":499,"duration_ms":5705,"temperature":1.0,"reasoning_tokens":408,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:06:48.492781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give teachers and students the same predictions with explanations from both a post-hoc pipeline and the proposed intrinsic-plus-LLM pipeline, ask them to make real advising decisions under blind conditions, and include fairness, causal validity, and cost of acting among the evaluation criteria; if post-hoc explanations prove equally actionable or more trusted, the thesis's central ranking would not hold.","supporting_citations":[],"review_version":1}