Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Construction and Preliminary Validation of a Dynamic Programming Concept Inventory

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports the construction and preliminary validation of the first Dynamic Programming Concept Inventory, built around 15 student misconceptions and tested on 172 undergraduates, and argues that instructors can use it to assess…

desk verdict First DP concept inventory with public items and real psychometric data, but the 'validated' claim outruns the evidence. read the letter →

arxiv 2411.14655 v1 pith:W3AXAXCC submitted 2024-11-22 cs.DS cs.CY

Matthew Ferland , Varun Nagaraj Rao , Arushi Arora , Drew van der Poel , Michael Luu , Randy Huynh , Freddy Reiber , Sandra Ossman
show 2 more authors
Seth Poulsen Michael Shindler
This is my paper · ORCID
classification cs.DScs.CY
keywords dynamicprogrammingconceptinventorystudentmisconceptionsclassicaltesttheorypsychometricvalidationalgorithmseducationcomputersciencemultiple-choiceassessment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that a Dynamic Programming Concept Inventory (DPCI) is a valid, usable assessment of undergraduate mastery of dynamic programming. If the claim holds, instructors can use the DPCI to measure conceptual understanding, compare teaching approaches, and identify which DP misconceptions are widespread. The inventory was built by reanalyzing 64 interview transcripts to produce a list of 15 misconceptions, then writing multiple-choice questions whose wrong answers correspond to specific misconceptions. A first validation round with 93 students and a revised second round with 63 students gave difficulty and discrimination values mostly inside conventional ranges, with internal consistency reliability at $\alpha = 0.76$.

What carries the argument

The central object is the Dynamic Programming Concept Inventory (DPCI): a timed multiple-choice assessment whose wrong-answer choices are mapped one-to-one to 15 catalogued misconceptions about DP, such as 'DP always involves minimization or maximization' and 'conflating recursion with DP.' The construction used the standard concept-inventory pipeline—identify topics, establish misconceptions, write questions, expert review, validate, revise. The validation machinery is classical test theory: item difficulty (fraction answering correctly, target 0.2–0.8), item discrimination (point-biserial correlation of item score with total score, target above 0.2), and internal consistency reliability ($\alpha$, with 0.7 satisfactory and 0.8 good). These metrics were used to cut two non-discriminating items and to split one over-ambitious select-all question into three.

What would settle it

Administer the DPCI to a large, diverse sample while also grading students' handwritten DP solutions to a new problem; if DPCI scores correlate near zero with independently graded DP problem-solving performance, the inventory measures something other than conceptual mastery despite acceptable internal statistics.

Watch

Extended reading notes

Core claim

The central claim is that the DPCI is the first validated concept inventory for dynamic programming. The paper argues that its 15 misconception-targeted multiple-choice questions measure DP conceptual understanding and distinguish stronger from weaker students, based on classical test theory: most items fall in the preferred difficulty band of 0.2–0.8 and have point-biserial discrimination above 0.2, while the internal consistency coefficient is $\alpha = 0.76$, close to the recommended 0.8. Two items that repeatedly failed to discriminate (DV13 and DV14.2) were removed. The paper concludes that the resulting instrument lets instructors accurately assess DP mastery and offers a template for concept inventories in other advanced theoretical CS topics.

Load-bearing premise

The inventory's validity rests on the assumption that the 15 misconceptions found by re-reading 64 old interview transcripts, marking a misconception as real if it appeared even once, are the right and complete set of DP misconceptions for the general undergraduate population.

Editorial extensions

If this is right

  • Instructors can deploy the DPCI before and after teaching DP to measure conceptual gains and compare the effectiveness of different teaching methods.
  • The published question bank lets other institutions administer the same instrument, enabling cross-institution comparisons of DP instruction.
  • The 15-item misconception list gives researchers a taxonomy for studying DP learning, not just an assessment tool.
  • The successful split and removal decisions show that classical test theory metrics can guide iterative inventory revision, providing a template for other advanced CS topics.
  • If the validation holds, the DPCI fills a documented gap in concept inventory coverage for theoretical computer science topics such as greedy algorithms and divide-and-conquer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because misconception prevalence was coded as present or absent rather than counted, the inventory cannot yet rank misconceptions by frequency; a larger interview study with prevalence counts could weight items and guide shortening the test.
  • The internal consistency of 0.76 is below the paper's own 0.8 target, so a confirmatory factor analysis on a further sample would clarify whether the DPCI is unidimensional or measures several distinct DP skills.
  • Both validation samples came from large public U.S. universities with similar algorithms course contexts; testing at smaller or more varied institutions is needed to know whether the difficulty and discrimination values travel.
  • Comparing DPCI scores against a written DP problem-solving task would test whether the inventory predicts the procedural skill it is meant to complement, since the paper deliberately excluded recurrence construction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper reports the construction and preliminary psychometric validation of a Dynamic Programming Concept Inventory (DPCI), a multiple-choice instrument intended to reveal undergraduate student misconceptions about dynamic programming. The authors reanalyzed 64 interview transcripts from Shindler et al. (2022) to produce a list of 15 misconceptions, wrote and expert-reviewed questions targeting those misconceptions, administered the instrument to 172 students at two universities, and report classical test theory statistics: item difficulty, point-biserial discrimination, and Cronbach's alpha. They use these statistics to argue that the DPCI is a validated inventory that will allow instructors to accurately assess DP mastery.

Significance. The DPCI fills a genuine gap; no validated concept inventory exists for dynamic programming, and the paper provides a useful model for constructing such instruments. Strengths include the public PrairieLearn artifact, the iterative development process incorporating outside expert review, and the reporting of per-item difficulty and discrimination data across two institutions. The contribution is real, but the 'validated' claim exceeds the evidence. The reported statistics establish internal consistency and item-level performance; they do not establish construct validity, and the final alpha of 0.76 is below the threshold the authors themselves cite. The paper is best read as a preliminary validation, with the central claim needing substantial tempering or additional validation evidence.

major comments (4)
  1. [Section 6.2] The claim that 'misconceptions chosen by 15% or more of the students provide strong evidence that the misconception both targets and accurately measures the intended misunderstanding' is not supported by the data presented. A distractor may be selected frequently because it is a plausible surface-level answer, because of ambiguous wording, or because of a different misconception not on the authors' list. The paper reports no think-aloud interviews, no written explanations from respondents, and no external criterion (such as correlation with performance on open-ended DP problems) linking distractor selection to the hypothesized belief. Absent such evidence, the selection rates only show that the distractors are attractive wrong answers, not that the DPCI measures DP mastery.
  2. [Sections 4.1 and 6.2] The validation procedure is circular in an important respect. The misconception list was produced by the authors' reanalysis of interview transcripts from prior work, coding a misconception as present if seen at least once and explicitly not quantifying prevalence (Section 4.1). The 'construct validity' evidence in Section 6.2 then consists of selection rates for distractors built from that same list. This procedure cannot independently confirm the misconceptions, and the 15% prevalence threshold is introduced without justification. To make the validation non-circular, the authors need a coding protocol with inter-rater reliability, prevalence estimates, and at least one external source of evidence that the distractors elicit the intended beliefs.
  3. [Sections 5.3.3 and 6.1] Post-hoc removal of poorly performing items weakens the validation claim. After Round 2, DV13 and DV14.2 are removed because of low discrimination, yet Section 5.3.3 reports alpha after removing DV13 and DV14.1 and Section 6.1 says 'DV13 and DV14.2' were removed. The final item set and recomputed statistics after the actual removals are not clearly reported. Because the same data were used both to decide which items to delete and to estimate the quality of the remaining items, the reported difficulty and discrimination values are optimistic and not cross-validated. The paper should state the final item list and provide statistics for the final instrument as a whole.
  4. [Sections 5.1 and 5.3.1] Calling the reliability 'strong' is overstated. Cronbach's alpha is 0.76 in both rounds, below the 0.8 value the authors cite from Jorion et al. as 'good'; the text acknowledges it is 'close' but the abstract and conclusions still describe the inventory as 'validated.' The samples are also small and convenience-based (93 and 63 students, with optional participation and grade incentives), so the precision of the item statistics is limited. At minimum, the conclusions should be reframed as preliminary psychometric evidence.
minor comments (5)
  1. [Abstract and Section 7] The phrase 'validated DPCI will enable instructors to accurately assess student mastery of DP' should be softened to 'preliminary' and 'may support identification of misconceptions,' because the current wording is not supported by the evidence reported in the paper.
  2. [Section 4.1] The sentence 'the numbers found were not quantified' is ambiguous as written; specify whether prevalence frequencies were not computed or merely not reported, and add details about the coding procedure and any inter-rater reliability checks.
  3. [Table 3] The question ID VR2 appears in two rows with different statistics; relabel one of the items so that each row corresponds to a unique question.
  4. [Section 4.5] The text says that Misconception 6 is no longer being measured, but Table 1 still lists it; update the table or the text to be consistent.
  5. [General] There are several typographical errors that should be corrected: 'seperated' (Section 4.2), 'prevelant' (Section 6.2), 'were were' (Table 2 caption), and 'hypothesis about its prevalence' (Section 6.2).

Circularity Check

2 steps flagged · score 4.0 of 10

The DPCI's construct-validity claim reduces distractor selection to misconception presence, and the same response data used to prune items is then reported as validation; expert review and a second institution provide partial independence.

  1. self definitional [Section 6.2, 'Evidence that the Misconceptions Are Measured by the Questions']
    "Misconceptions chosen by 15% or more of the students provide strong evidence that the misconception both targets and accurately measures the intended misunderstanding, thus validating our hypothesis about its prevalence."

    The distractors were authored by the same group to embody the Table 1 misconceptions (Section 4.2: 'each author was given full freedom to create as many questions as they wished, targeting at least one misconception identified'). The validation then uses the rate at which students select those author-written distractors as evidence that the item 'accurately measures the intended misunderstanding.' No external criterion—think-aloud protocols, written justifications, or expert judgment of students' reasoning—is used to confirm that a choice was caused by the intended belief.

  2. fitted input called prediction [Sections 5.3.2 and 6.1 (item revisions and removal)]
    "Consequently, we removed these questions from the concept inventory, as they proved to be outliers."

    The validation statistics (difficulty, discrimination, Cronbach's alpha) are reported for the final question set after items with poor discrimination were revised or deleted based on their performance in the same validation rounds: DV14 was split, DV13 was rewritten, and later DV13 and DV14.2 were removed because they 'continued to perform poorly.' The same response data that determined which items survived is then cited as evidence that the surviving items are of 'appropriate difficulty and effectively discriminating.' The reported metrics are therefore conditional on a selection made from the very data used to compute them; nothing is predicted out-of-sample.

full rationale

The DPCI is not a formal derivation but an instrument-development study, so circularity must be assessed on how the validation argument is built. The paper's central claim is that the inventory is validated and will enable instructors to accurately assess DP mastery. The construct-validity evidence in Section 6.2 equates high distractor selection with evidence that the item measures the intended misconception; this is self-definitional because the distractor was authored to encode that misconception and no independent criterion links choice to belief. Additionally, the item set was revised and pruned using the same response data that is then reported as validation (Sections 5.3.2 and 6.1), so the final psychometric statistics are partly the product of selection on the same sample. These two issues make the 'validated DPCI ... accurately assess student mastery' claim overreach. However, the work has substantial non-circular components: misconceptions originated partly in external prior work ([40]), outside experts reviewed the questions (Section 4.5), and Round 2 was conducted at a different institution, providing some out-of-sample evidence. Thus the circularity is moderate, not total.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The central validation rests on the misconception list from prior interview data, CTT statistics, expert review, and a hand-chosen prevalence threshold. These are standard tools in concept inventory development, but the first two are load-bearing and only partially independent of the authors' own prior work.

free parameters (1)
  • Misconception prevalence threshold = 15%
    In Section 6.2, the authors designate misconceptions selected by 15% or more of students as 'highly prevalent' and those below as 'low prevalence.' This cutoff is chosen by hand, not derived from theory, and drives which misconceptions are considered validated.
assumptions (4)
  • domain assumption Classical test theory statistics (Cronbach's alpha, point-biserial discrimination) are valid evidence of construct validity for a concept inventory.
    The paper's validation rests on CTT metrics (Section 5) without discussing unidimensionality or other IRT assumptions; this is standard but an assumption.
  • domain assumption Misconception labels from reanalysis of 64 transcripts from Shindler et al. (2022) generalize to the undergraduate CS population at large.
    Section 4.1: the authors relabel the transcripts and treat any misconception seen once as a target. Representatives of this small interview corpus is assumed.
  • domain assumption Student selection of incorrect multiple-choice options reflects the intended misconception rather than guessing or test-taking artifacts.
    Used in Section 6.2 to claim construct validity; no distractor analysis or control for guessing is provided.
  • domain assumption The four outside expert reviews (Section 4.5) establish content validity.
    Expert feedback is used to revise questions, but no structured content validity index is reported; the assumption is that expert judgment suffices.
invented entities (1)
  • Dynamic Programming Concept Inventory (DPCI) independent evidence
    purpose: A multiple-choice assessment intended to measure student understanding of DP concepts and misconceptions.
    The DPCI is publicly accessible via PrairieLearn and can be independently administered, so it has a falsifiable handle outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Construction and Preliminary Validation of a Dynamic Programming Concept Inventory." pith.science (2026). https://pith.science/paper/W3AXAXCC

@misc{pith2026241114655,
  author       = {Pith},
  title        = {Pith review of: Construction and Preliminary Validation of a Dynamic Programming Concept Inventory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W3AXAXCC}},
  note         = {Machine review of arXiv:2411.14655}
}
read the original abstract

Concept inventories are standardized assessments that evaluate student understanding of key concepts within academic disciplines. While prevalent across STEM fields, their development lags for advanced computer science topics like dynamic programming (DP) -- an algorithmic technique that poses significant conceptual challenges for undergraduates. To fill this gap, we developed and validated a Dynamic Programming Concept Inventory (DPCI). We detail the iterative process used to formulate multiple-choice questions targeting known student misconceptions about DP concepts identified through prior research studies. We discuss key decisions, tradeoffs, and challenges faced in crafting probing questions to subtly reveal these conceptual misunderstandings. We conducted a preliminary psychometric validation by administering the DPCI to 172 undergraduate CS students finding our questions to be of appropriate difficulty and effectively discriminating between differing levels of student understanding. Taken together, our validated DPCI will enable instructors to accurately assess student mastery of DP. Moreover, our approach for devising a concept inventory for an advanced theoretical computer science concept can guide future efforts to create assessments for other under-evaluated areas currently lacking coverage.

Figures

Figures reproduced from arXiv: 2411.14655 by the authors.

Figure 1
Figure 1. Difficulty and discrimination for both rounds of [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 33 canonical work pages

  1. [1]

    Wendy K Adams and Carl E Wieman. 2011. Development and validation of instruments to measure learning of expert-like thinking. International journal of science education 33, 9 (2011), 1289–1312

    work page 2011

  2. [2]

    Murtaza Ali, Sourojit Ghosh, Prerna Rao, Raveena Dhegaskar, Sophia Jawort, Alix Medler, Mengqi Shi, and Sayamindu Dasgupta. 2023. Taking Stock of Concept Inventories in Computing Education: A Systematic Literature Review. InProceed- ings of the 2023 ACM Conference on International Computing Education Research - Volume 1 (Chicago, IL, USA) (ICER ’23). Asso...

    arXiv 2023

  3. [3]

    Dianne L Anderson, Kathleen M Fisher, and Gregory J Norman. 2002. Develop- ment and evaluation of the conceptual inventory of natural selection. Journal of research in science teaching 39, 10 (2002), 952–978

    work page 2002

  4. [4]

    Erin M Bardar, Edward E Prather, Kenneth Brecher, and Timothy F Slater. 2007. Development and validation of the light and spectroscopy concept inventory. Astronomy Education Review 5, 2 (2007), 103–113

    work page 2007

  5. [5]

    Yifat Ben-David Kolikant and Sara Genut. 2017. The effect of prior education on students’ competency in digital logic: the case of ultraorthodox Jewish students. Computer Science Education 27, 3-4 (2017), 149–174

    work page 2017

  6. [7]

    Ryan Bockmon, Stephen Cooper, William Koperski, Jonathan Gratch, Sheryl Sorby, and Mohsen Dorodchi. 2020. A CS1 spatial skills intervention and the impact on introductory programming abilities. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (Portland, OR, USA) (SIGCSE ’20). Association for Computing Machinery, New York, N...

    arXiv 2020

  7. [8]

    Will Crichton, Gavin Gray, and Shriram Krishnamurthi. 2023. A Grounded Conceptual Model for Ownership Types in Rust. Proceedings of the ACM on Programming Languages 7, OOPSLA2 (2023), 1224–1252

    work page 2023

  8. [9]

    Holger Danielsiek, Wolfgang Paul, and Jan Vahrenhold. 2012. Detecting and understanding students’ misconceptions related to algorithms and data struc- tures. In Proceedings of the 43rd ACM technical symposium on Computer Science Education. 21–26. Construction and Preliminary Validation of a Dynamic Programming Concept Inventory SIGCSE TS 2025, February 26...

    work page 2012

Show all 42 references
  1. [10]

    Adrienne Decker and Monica M. McGill. 2019. A topical review of evaluation instruments for computing education. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE ’19). Association for Computing Machinery, New York, ...

    2019

  2. [11]

    most difficult

    Emma Enström and Viggo Kann. 2017. Iteratively intervening with the “most difficult” topics of an algorithms and complexity course. ACM Transactions on Computing Education (TOCE) 17, 1 (2017), 1–38

    2017

  3. [12]

    Jerome Epstein. 2007. Development and validation of the Calculus Concept Inventory. In Proceedings of the ninth international conference on mathematics education in a global community , Vol. 9. Citeseer, 165–170

    2007

  4. [13]

    Mohammed F Farghally, Kyu Han Koh, Jeremy V Ernst, and Clifford A Shaffer

  5. [14]

    Ken Goldman, Paul Gross, Cinda Heeren, Geoffrey L Herman, Lisa Kaczmarczyk, Michael C Loui, and Craig Zilles. 2010. Setting the scope of concept inventories for introductory computing subjects. ACM Transactions on Computing Education (TOCE) 10, 2 (2010), 1–29

    2010

  6. [15]

    Richard R. Hake. 1998. Interactive-engagement versus traditional methods: A six- thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics 66, 1 (1998), 64–74. https://doi.org/10.1119/1.18809

    1998 doi

  7. [16]

    Geoffrey L Herman and Joseph Handzik. 2010. A preliminary pedagogical com- parison study using the digital logic concept inventory. In 2010 IEEE Frontiers in Education Conference (FIE). IEEE, F1G–1

    2010

  8. [17]

    Geoffrey L Herman, Shan Huang, Peter A Peterson, Linda Oliva, Enis Golaszewski, and Alan T Sherman. 2023. Psychometric Evaluation of the Cybersecurity Cur- riculum Assessment. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. 228–234

    2023

  9. [18]

    Geoffrey L Herman, Craig Zilles, and Michael C Loui. 2014. A psychometric evaluation of the digital logic concept inventory. Computer Science Education 24, 4 (2014), 277–303

    2014

  10. [19]

    David Hestenes, Malcolm Wells, and Gregg Swackhamer. 1992. Force concept inventory. The Physics Teacher 30, 3 (1992), 141–158. https://doi.org/10.1119/1. 2343497

    1992 doi

  11. [20]

    Association for Computing Machinery (ACM) Joint Task Force on Comput- ing Curricula and IEEE Computer Society. 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science . Association for Computing Machinery, New York, NY, USA

    2013

  12. [21]

    Natalie Jorion, Brian D Gane, Katie James, Lianne Schroeder, Louis V DiBello, and James W Pellegrino. 2015. An analytic framework for evaluating the validity of concept inventory claims. Journal of Engineering Education 104, 4 (2015), 454–496

    2015

  13. [22]

    Julie C Libarkin and Steven W Anderson. 2005. Assessment of learning in entry- level geoscience courses: Results from the Geoscience Concept Inventory.Journal of Geoscience Education 53, 4 (2005), 394–401

    2005

  14. [23]

    Jonathan Liu, Erica Goodwin, and Diana Franklin. 2025. Student Utilization of Metacognitive Strategies in Solving Dynamic Programming Problems. In Pro- ceedings of the 56th ACM Technical Symposium on Computer Science Education (Pittsburgh, Pennsylvania, USA) (SIGCSE ’25). Asso...

    2025

  15. [24]

    Jonathan Liu, Seth Poulsen, Erica Goodwin, Hongxuan Chen, Grace Williams, Yael Gertner, and Diana Franklin. 2024. Teaching Algorithm Design: A Literature Review. arXiv:2405.00832 [cs.DS]

    2024 arXiv

  16. [25]

    Michael Luu, Matthew Ferland, Varun Nagaraj Rao, Arushi Arora, Randy Huynh, Frederick Reiber, Jennifer Wong-Ma, and Michael Shindler. 2023. What is an Algorithms Course? Survey Results of Introductory Undergraduate Algorithms Courses in the U.S.. In Proceedings of the 54th ACM...

    2023

  17. [26]

    Lauren Margulieux, Tuba Ayer Ketenci, and Adrienne Decker. 2019. Review of measurements used in computing education research and suggestions for increasing standardization. Computer Science Education 29, 1 (Jan. 2019), 49–

    2019

  18. [27]

    Douglas R Mulford and William R Robinson. 2002. An inventory for alternate conceptions among first-semester general chemistry students.Journal of chemical education 79, 6 (2002), 739

    2002

  19. [28]

    Greg L Nelson, Benjamin Xie, and Amy J Ko. 2017. Comprehension first: eval- uating a novel pedagogy and tutoring system for program tracing in CS1. In Proceedings of the 2017 ACM Conference on International Computing Education Research. 2–11

    2017

  20. [29]

    Spencer Offenberger, Geoffrey L Herman, Peter Peterson, Alan T Sherman, Enis Golaszewski, Travis Scheponik, and Linda Oliva. 2019. Initial validation of the cybersecurity concept inventory: pilot testing and expert review. In 2019 IEEE Frontiers in Education Conference (FIE) ....

    2019

  21. [30]

    Miranda C Parker, Mark Guzdial, and Shelly Engleman. 2016. Replication, valida- tion, and use of a language independent CS1 knowledge assessment. In Proceed- ings of the 2016 ACM conference on international computing education research . 93–101

    2016

  22. [31]

    Wolfgang Paul and Jan Vahrenhold. 2013. Hunting high and low: Instruments to detect misconceptions related to algorithms and data structures. In Proceeding of the 44th ACM technical symposium on Computer science education . 29–34

    2013

  23. [32]

    Leo Porter, Saturnino Garcia, Hung-Wei Tseng, and Daniel Zingaro. 2013. Eval- uating student understanding of core concepts in computer architecture. In Proceedings of the 18th ACM conference on Innovation and technology in computer science education. 279–284

    2013

  24. [33]

    Leo Porter, Daniel Zingaro, Soohyun Nam Liao, Cynthia Taylor, Kevin C Webb, Cynthia Lee, and Michael Clancy. 2019. BDSI: A validated concept inventory for basic data structures. In Proceedings of the 2019 ACM Conference on International Computing Education Research. 111–119

    2019

  25. [34]

    Seth Poulsen, Geoffrey L Herman, Peter AH Peterson, Enis Golaszewski, Akshita Gorti, Linda Oliva, Travis Scheponik, and Alan T Sherman. 2021. Psychome- tric evaluation of the cybersecurity concept inventory. ACM Transactions on Computing Education (TOCE) 22, 1 (2021), 1–18

    2021

  26. [35]

    Michael Shindler, Natalia Pinpin, Mia Markovic, Frederick Reiber, Jee Hoon Kim, Giles Pierre Nunez Carlos, Mine Dogucu, Mark Hong, Michael Luu, Brian Anderson, et al . 2022. Student misconceptions of dynamic programming: a replication study. Computer Science Education 32, 3 (2...

    2022

  27. [36]

    Andrea Stone, Kirk Allen, Teri Reed Rhoads, Teri J Murphy, Randa L Shehab, and Chaitanya Saha. 2003. The statistics concept inventory: A pilot study. In 33rd Annual Frontiers in Education, 2003. FIE 2003. , Vol. 1. IEEE, T3D–1

    2003

  28. [37]

    Webb, Daniel Zingaro, Cynthia Lee, and Leo Porter

    Cynthia Taylor, Michael Clancy, Kevin C. Webb, Daniel Zingaro, Cynthia Lee, and Leo Porter. 2020. The Practical Details of Building a CS Concept Inventory. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (Portland, OR, USA) (SIGCSE ’20). Associ...

    2020

  29. [38]

    Kevin C Webb and Cynthia Taylor. 2014. Developing a pre-and post-course concept inventory to gauge operating systems learning. InProceedings of the 45th ACM technical symposium on Computer science education . 103–108

    2014

  30. [39]

    Benjamin Xie, Greg L Nelson, and Amy J Ko. 2018. An explicit strategy to scaffold novice program tracing. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. 344–349

    2018

  31. [40]

    Shamama Zehra, Aishwarya Ramanathan, Larry Yueli Zhang, and Daniel Zingaro

  32. [78]

    https://doi.org/10.1080/08993408.2018.1562145 Publisher: Routledge _eprint: https://doi.org/10.1080/08993408.2018.1562145

    2018

  33. [2017]

    In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education

    Towards a concept inventory for algorithm analysis topics. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education . 207–212

    2017

  34. [2018]

    In Proceedings of the 49th ACM technical symposium on Computer Science Education

    Student misconceptions of dynamic programming. In Proceedings of the 49th ACM technical symposium on Computer Science Education . 556–561

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.