Fine-tuned 7B and 13B LLaMA-2 models answered course-specific programming language MCQs about as accurately as the much larger 70B model, at a lower hardware cost.
Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Large Language Models (LLMs) have not only exhibited exceptional performance across various tasks, but also demonstrated sparks of intelligence. Recent studies have focused on assessing their capabilities on human exams and revealed their impressive competence in different domains. However, cognitive research on the overall knowledge structure of LLMs is still lacking. In this paper, based on educational diagnostic assessment method, we conduct an evaluation using MoocRadar, a meticulously annotated human test dataset based on Bloom Taxonomy. We aim to reveal the knowledge structures of LLMs and gain insights of their cognitive capabilities. This research emphasizes the significance of investigating LLMs' knowledge and understanding the disparate cognitive patterns of LLMs. By shedding light on models' knowledge, researchers can advance development and utilization of LLMs in a more informed and effective manner.
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Affordably Fine-tuned LLMs Provide Better Answers to Course-specific MCQs
Fine-tuned 7B and 13B LLaMA-2 models answered course-specific programming language MCQs about as accurately as the much larger 70B model, at a lower hardware cost.