Pith. sign in

REVIEW 3 major objections 4 minor 68 references

The Failure of Plagiarism Detection in Competitive Programming

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Similarity checkers miss AI-generated code and lightly rewritten copies, so competitive programming courses must add human interviews to the detection mix.

desk verdict A useful, honest practitioner report whose core claim is already established; the interview 'success rate' is circular and the empirical evidence is thinner than the narrative suggests. read the letter →

arxiv 2505.08244 v1 pith:EEXVYSZA submitted 2025-05-13 cs.CY cs.SE

Ethan Dickey This is my paper · ORCID
classification cs.CYcs.SE
keywords codeplagiarismcompetitiveprogrammingsimilaritydetectionMossKattisgenerativeAIoralinterviewsacademicintegrity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that in competitive programming courses, routine automated plagiarism detection fails against two common tactics: lightly transformed copies of known solutions and freshly generated AI code, both of which evade fingerprint-matching. Drawing on seven semesters of running a competitive programming course, the author claims that a multi-layered strategy—automated screening, manual review, and one-on-one code interviews—catches or deters the majority of plagiarism cases, with interviews doing the decisive work. The paper's central recommendation is that institutions combine improved detectors with authentic, human verification rather than relying on any single tool.

What carries the argument

The analysis is carried by comparing four detection mechanisms: token-fingerprinting similarity tools (Moss), n-gram and hash comparisons (Kattis), manual code review, and oral code-authorship interviews, with code stylometry suggested as a future addition. The load-bearing object is the interview protocol: a short oral exam in which a student explains their accepted solution, reproduces key reasoning, and describes a bug they fixed, without access to the code. The paper uses this interview as the ground truth against which the other methods are measured, and it introduces the notion of 'whiteboard similarity'—similarity that arises from two students solving a problem together without seeing each other's code—to explain why independent students rarely produce matches that similarity tools should flag.

What would settle it

A controlled study using the paper's Protocol 1: recruit students who genuinely wrote their own solutions and another group who copied or used AI but then studied the code until they could explain it; run blinded interviews and measure how many of the second group pass. If a substantial fraction passes, the near-100% success claim fails, while high failure rates in the first group would indicate a false-accusation problem.

Watch

Extended reading notes

Core claim

The paper's central discovery is that automated code-similarity systems such as Moss and Kattis's built-in checker are reliable only against direct copy-paste and lightly disguised copying; they are defeated by systematic rewrite tools and by generative-AI output that is original by construction. In the author's competitive programming course, a weekly routine of running similarity reports, manually inspecting flagged pairs, and interviewing a random subset of students uncovered plagiarism that no automated tool flagged—including fifteen students who submitted an advanced technique they had not been taught and could not explain. The paper claims that the interview protocol, in which a student must explain their solution, its runtime, and a struggle they overcame, approaches 100% success in confirming which students do not understand their own code, and that the mere prospect of interviews deters most repeat cheating.

Load-bearing premise

The reported success rate rests on treating the interview as a reliable lie detector: a student who cannot explain their submitted code is assumed to have plagiarized, and the paper assumes this failure cannot be convincingly faked.

Editorial extensions

If this is right

  • Courses that rely only on similarity scores will miss AI-generated submissions, since novel AI output has no matching source to flag.
  • A weekly rotation of short interviews, covering every student at least once, can serve as both a deterrent and a confirmation step at modest staff cost.
  • Detectors improve when instructors add known online solution repositories and a few generated AI solutions as base files in each similarity run.
  • Process anomalies, such as one-shot perfect solutions from students who usually struggle, are a useful complement to text similarity.
  • Cross-language plagiarism—translating a solution from one programming language to another—is currently invisible to all commercial tools and requires manual or stylistic review.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If interview-based verification is genuinely near-100% accurate, the most scalable institutional response is to shift more assessment weight to live, proctored demonstrations rather than trying to perfect text-similarity algorithms.
  • The paper's 'whiteboard similarity' observation implies that pairwise code similarity is a noisy proxy for collusion; a cleaner target for automated tools may be authorship consistency over time within one student, that is, stylometric anomaly detection.
  • Because the paper reports that most confirmed cheaters were caught early in the semester and few repeated after a warning, a testable extension is whether early-term interviews alone can reduce end-of-term plagiarism rates.
  • The reported rise in denials and the edge case of a student who learns AI-generated code well enough to pass an interview suggest that as interviews become expected, cheaters will invest in understanding copied code, eroding the ground truth the method relies on.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper is an experience-based analysis of plagiarism detection in competitive programming courses, drawing on the author's seven semesters of teaching CP1 at Purdue and on a redesign of the CP1/2/3 sequence. It reviews the literature on code plagiarism, automated similarity tools (Moss, Kattis, JPlag), manual review, oral interviews, and generative AI, and combines this with a set of incident counts (Table 1) and case studies. The central claims are that automated similarity checkers can be evaded by simple transformations and by novel AI-generated code, that human-centric interviews are the most effective verification method but are labor-intensive, and that a multi-faceted strategy combining tools, manual review, and interviews catches or deters most plagiarism. The paper also introduces the term 'whiteboard similarity,' provides an interview protocol (Protocol 1), a strengths/weaknesses comparison (Table 2), and appendices on tracked solution repositories, a plagiarism ring, and example MOSS outputs.

Significance. If read as a practitioner's experience report, the paper is useful: it gives a concrete interview protocol, documents real failure modes of Moss/Kattis, and offers a sensible set of recommendations for multi-pronged detection and authentic assessment. The detailed appendices and the explicit acknowledgment of the genAI edge case in Section 4.4 are strengths. However, the paper's quantitative empirical claims—especially the 'near 100% success rate' of interviews and the interpretation of Table 1—are not supported by the evidence as presented, and they are load-bearing for the recommendation to scale interviews. The central conceptual argument is defensible, but the empirical framing needs to be corrected before the claims can be accepted.

major comments (3)
  1. [Sections 3 and 4.3.2] The claim that suspect interviews have a 'near 100% success rate' in confirming plagiarism is circular as stated: Protocol 1 is used both to decide which students are suspected and to 'confirm' that the suspicion is correct, with no independent ground truth for false negatives. Section 4.4 explicitly acknowledges the edge case of a student who uses AI, thoroughly learns the generated code, and passes an interview, which means the interview is not a validated gold standard. The reported rate therefore measures agreement among detection steps rather than detection accuracy. Please reframe the claim as 'near 100% agreement among screening signals' or provide external validation (for example, confessions, follow-up assessment outcomes, or blinded re-interviews by a second interviewer).
  2. [Table 1 and Section 3] The incident counts in Table 1 are not comparable across semesters and do not by themselves support the conclusion that the multi-faceted strategy 'caught or deterred the majority' of plagiarism cases. The table spans periods with different detection effort and policy: MOSS runs began in S24, F24 included a genAI-allowed intervention for part of the course, S25 introduced weekly interviews and a required final, and the 15 BSTA cases in S25 were found by manual search rather than by the automated tools. The counts are therefore better described as detection-effort counts, not as plagiarism incidence. Please provide the relevant denominators, time at risk, and any changes in detection effort alongside the table, and avoid statements of success rates that go beyond what the counts can show.
  3. [Section 3, BSTA example] The BSTA case is presented as a 'failure of automated detection,' but the same paragraph indicates that a few students were initially flagged and that the remaining cases were found only after a manual search through all accepted submissions. The interpretation of this example depends on which of the 15 students were flagged by which tool and on the untested assumption that the interview outcome is ground truth. Please specify the detection pathway for each of the 15 students (Moss, Kattis, TA report, or manual search) and state how the interview protocol was validated in this particular episode, so readers can assess what exactly the case demonstrates.
minor comments (4)
  1. [References] Reference [54] is cited as 'arXiv preprint arXiv:TBA'; please provide a complete citation or mark it as forthcoming.
  2. [Section 4.3.2] The phrase 'our success rate approaches 100%' is stated without the underlying denominator; please report the number of suspect interviews conducted and the number of confirmed outcomes so the statement is meaningful.
  3. [Table 1] The definition of 'serious incidents' appears only in the table note and is easy to miss; please define it in the main text near the table.
  4. [Section 2.1] The new term 'whiteboard similarity' is defined only by contrast with 'code collaboration'; a short sentence relating it to existing notions of legitimate collaboration would help readers who are unfamiliar with the term.

Circularity Check

1 steps flagged · score 2.0 of 10

Only mild self-referentiality: the interview 'near 100% success' figure is measured by the same interview protocol used to define a positive case, but the paper's central claims about tool limitations and the value of a multi-faceted approach rest on independent evidence and are not circular.

  1. self definitional [Section 4.3.2 (Strengths of Interviews) and Section 3, Concluding Observations; Protocol 1]
    "In practice, our success rate approaches 100% in detecting plagiarism this way. ... In nearly 100% of the cases, the suspects faltered in explanation of their code or directly confessed, confirming the plagiarism, indicating we were erring on the side of caution."

    The claimed success rate is defined by the interview outcome itself: a student is counted as a confirmed plagiarism case when they 'faltered in explanation' or 'directly confessed' under Protocol 1. There is no external ground truth independent of the interview, so 'near 100% success' measures agreement between the prior suspicion and the interview verdict, not detection accuracy against an objective standard. The paper itself concedes the missing ground truth in Section 4.4, where a student who uses genAI and then thoroughly learns the generated code can pass an interview, meaning false negatives are unmeasured. The statistic therefore reduces to the definition of a positive case rather than an independent validation of the method.

full rationale

The paper is an experience-based analysis rather than a formal derivation, and its main claims about the limitations of Moss/Kattis, the ease of evading similarity checks, and the labor-intensity of interviews are supported by direct observations, cited external studies, and concrete course incidents. No fitted parameter, normalization, or hidden equation makes the central argument reduce to its inputs. The author's self-citations ([8], [21], [54], [56]) provide supporting context but are not load-bearing for the central thesis. The only genuine circularity is in the evaluation of interviews: the 'near 100% success rate' is measured using the same interview protocol that operationally defines a positive case, and Section 4.4 explicitly acknowledges an edge case that would break this ground truth. This makes the flagship empirical figure self-referential rather than externally validated, but it does not invalidate the paper's other observations. Given the paper's transparency about its limitations and the absence of any derived prediction that is equivalent by construction to an input, a score of 2 is appropriate.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces the concept 'whiteboard similarity' as a new label for legitimate collaboration, but this is a definitional category, not a new physical or formal entity. No new particles, forces, or mathematical objects are postulated. The main assumptions are the representativeness of a single course and the trustworthiness of oral interviews as a ground truth for authorship.

assumptions (5)
  • domain assumption Plagiarism is widespread in programming courses and undermines learning outcomes.
    The paper's motivation relies on this premise, supported by cited surveys in Section 2.2 but not re-established by the paper's own data.
  • domain assumption Moss and Kattis are representative of the automated plagiarism detection tools used in competitive programming courses.
    The analysis treats these two tools as the canonical examples of automated similarity checkers in Sections 2.3, 2.4, and 4.1.
  • domain assumption Interview performance is a valid gold standard for code authorship.
    The 'near 100% success rate' of interviews in Section 4.3.2 assumes that inability to explain one's code reliably indicates plagiarism, an assumption the paper itself partially undermines with the learned-AI edge case in Section 4.4.
  • domain assumption The Purdue CP1 experience is representative of competitive programming courses more broadly.
    The paper generalizes recommendations from a single course at one university to the wider CP community without a comparative multi-institution dataset.
  • ad hoc to paper Whiteboard similarity is a real and distinguishable category that does not produce code flagged as plagiarized.
    The term is introduced in Section 2.1 and used to argue that interviews, not similarity scores, are needed to distinguish legitimate collaboration from copying. This is a definitional claim, not empirically established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Failure of Plagiarism Detection in Competitive Programming." pith.science (2026). https://pith.science/paper/EEXVYSZA

@misc{pith2026250508244,
  author       = {Pith},
  title        = {Pith review of: The Failure of Plagiarism Detection in Competitive Programming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EEXVYSZA}},
  note         = {Machine review of arXiv:2505.08244}
}
abstract

Plagiarism in programming courses remains a persistent challenge, especially in competitive programming contexts where assignments often have unique, known solutions. This paper examines why traditional code plagiarism detection methods frequently fail in these environments and explores the implications of emerging factors such as generative AI (genAI). Drawing on the author's experience teaching a Competitive Programming 1 (CP1) course over seven semesters at Purdue University (with $\approx 100$ students each term) and completely redesigning the CP1/2/3 course sequence, we provide an academically grounded analysis. We review literature on code plagiarism in computer science education, survey current detection tools (Moss, Kattis, etc.) and methods (manual review, code-authorship interviews), and analyze their strengths and limitations. Experience-based observations are presented to illustrate real-world detection failures and successes. We find that widely-used automated similarity checkers can be thwarted by simple code transformations or novel AI-generated code, while human-centric approaches like oral interviews, though effective, are labor-intensive. The paper concludes with opinions and preliminary recommendations for improving academic integrity in programming courses, advocating for a multi-faceted approach that combines improved detection algorithms, mastery-based learning techniques, and authentic assessment practices to better ensure code originality.

Figures

Figures reproduced from arXiv: 2505.08244 by the authors.

Figure 1
Figure 1. Example MOSS similarity score and output for Dominos, parts 1 and 2. High similarity score due to common Tarjan’s algorithm source. [PITH_FULL_IMAGE:figures/full_fig_p018_1.png] view at source ↗
Figure 2
Figure 2. Example MOSS similarity score and output for Dominos, parts 1 and 2. Great example of a high similarity score due to common Tarjan’s [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Anonymized MOSS Summary report, truncated. [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 52 canonical work pages

  1. [54]

    Removing the competition from competitive program- ming: A novel curriculum design,

    E. Dickey, “Removing the competition from competitive program- ming: A novel curriculum design,” arXiv preprint arXiv:TBA, 2025

    work page 2025

  2. [1]

    Competitive programming: A teaching methodology analysis ap- plied to first-year programming classes,

    I. N. Bandeira, T. V . Machado, V . F. Dullens, and E. D. Canedo, “Competitive programming: A teaching methodology analysis ap- plied to first-year programming classes,” in 2019 IEEE Frontiers in Education Conference (FIE), 2019, pp. 1–8

    work page 2019

  3. [2]

    Didactic framework for teaching c programming lan- guage: A proposal based on cooperative and competitive learning techniques,

    W. S. de Picanc ¸o, J. M. M. de Lucena, A. F. de Lira, and V . F. de Lucena, “Didactic framework for teaching c programming lan- guage: A proposal based on cooperative and competitive learning techniques,” in 2018 IEEE Frontiers in Education Conference (FIE) , 2018, pp. 1–9. 18 Fig. 1. Example MOSS similarity score and output for Dominos, parts 1 and 2. H...

    work page 2018

  4. [3]

    Students motivation for adopting programming contests: Innovation-diffusion perspec- tive,

    R. Raman, H. Vachharajani, and K. Achuthan, “Students motivation for adopting programming contests: Innovation-diffusion perspec- tive,” Education and Information Technologies, vol. 23, no. 5, pp. 1919– 1932, 2018

    work page 1919

  5. [4]

    The learning curves of competitive programming,

    J. R. Garcia and V . E. Aguirre, “The learning curves of competitive programming,” in AIP Conference Proceedings, vol. 1618. American Institute of Physics, 2014, pp. 934–937

    work page 2014

  6. [5]

    Competitive programming in computational thinking and problem solving education,

    K. K. F. Yuen, D. Y. W. Liu, and H. V . Leong, “Competitive programming in computational thinking and problem solving education,” Computer Applications in Engineering Education , vol. 31, no. 4, pp. 850–866, 2023. [Online]. Available: https: //onlinelibrary.wiley.com/doi/abs/10.1002/cae.22610

    work page doi:10.1002/cae.22610 2023

  7. [6]

    Plagiarism and programming: A survey of student attitudes,

    C. L. Aasheim, P . S. Rutner, L. Li, and S. R. Williams, “Plagiarism and programming: A survey of student attitudes,” Journal of infor- mation systems education, vol. 23, no. 3, pp. 297–314, 2012

    work page 2012

  8. [7]

    Why we still can’t stop plagiarism in undergraduate computer science,

    K. Chen, “Why we still can’t stop plagiarism in undergraduate computer science,” Mar 2018. [Online]. Available: https:// kevinchen.co/blog/cant-stop-plagiarism-in-computer-science/

    work page 2018

Show all 68 references
  1. [8]

    Evaluating the ai-lab intervention: Impact on student perception and use of generative ai in early undergraduate computer science courses,

    E. Dickey, A. Bejarano, R. Kuperus, and B. Fagundes, “Evaluating the ai-lab intervention: Impact on student perception and use of generative ai in early undergraduate computer science courses,”

  2. [9]

    Why computer science students cheat,

    C. D. Marsan, “Why computer science students cheat,” Apr 2010. [Online]. Avail- able: https://www.networkworld.com/article/728346/ infrastructure-management-why-computer-science-students-cheat. html

    2010

  3. [10]

    Collaboration, collusion and plagiarism in computer sci- ence coursework,

    R. Fraser, “Collaboration, collusion and plagiarism in computer sci- ence coursework,” Informatics in Education-An International Journal , vol. 13, no. 2, pp. 179–195, 2014

    2014

  4. [11]

    A taxonomy of plagiarism in computer science,

    M. Joy, G. Cosma, J. Sinclair, and J. Yau, “A taxonomy of plagiarism in computer science,” in EDULEARN09 Proceedings. IATED, 2009, pp. 3372–3379

    2009

  5. [12]

    Winnowing: lo- cal algorithms for document fingerprinting,

    S. Schleimer, D. S. Wilkerson, and A. Aiken, “Winnowing: lo- cal algorithms for document fingerprinting,” in Proceedings of the 2003 ACM SIGMOD international conference on Management of data , 2003, pp. 76–85, mOSS plagiarism-detection tool guidelines and overview available at...

    2003

  6. [13]

    Tmoss: Using in- termediate assignment work to understand excessive collaboration in large classes,

    L. Yan, N. McKeown, M. Sahami, and C. Piech, “Tmoss: Using in- termediate assignment work to understand excessive collaboration in large classes,” in Proceedings of the 49th ACM technical symposium on computer science education, 2018, pp. 110–115

    2018

  7. [14]

    Discovering plagiarism in introductory programming courses through the application of multiple methods,

    G. Olsson and F. P . Norlin, “Discovering plagiarism in introductory programming courses through the application of multiple methods,” Master’s thesis, KTH Royal Institute of Technology,

  8. [15]

    Authentic assessment,

    K. H. Koh, “Authentic assessment,” 02 2017. [On- line]. Available: https://oxfordre.com/education/view/10.1093/ acrefore/9780190264093.001.0001/acrefore-9780190264093-e-22

    2017

  9. [16]

    How to avoid plagiarism: Is viva voce the answer?

    P . Dawson and M. Rae, “How to avoid plagiarism: Is viva voce the answer?” Feb 2025. [Online]. Available: https: //this.deakin.edu.au/society/how-to-avoid-plagiarism

    2025

  10. [17]

    From ”ban it till we understand it

    S. Lau and P . J. Guo, “From ”ban it till we understand it” to ”resistance is futile”: How university programming instructors plan to adapt as more students use ai code generation and explanation tools such as chatgpt and github copilot,” in Proceedings of the 2023 ACM Confere...

    2023

  11. [18]

    Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation,

    B. A. Becker, P . Denny, J. Finnie-Ansley, A. Luxton-Reilly, J. Prather, and E. A. Santos, “Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation,” in Proceedings of the 54th ACM Technical Symposium on Computer Science ...

    2023

  12. [19]

    The robots are coming: Exploring the implications of openai codex on introductory programming,

    J. Finnie-Ansley, P . Denny, B. A. Becker, A. Luxton-Reilly, and J. Prather, “The robots are coming: Exploring the implications of openai codex on introductory programming,” in Proceedings of the 24th Australasian Computing Education 20 Conference, ser. ACE ’22. New York, NY, ...

    2022

  13. [20]

    Github copilot ai pair programmer: Asset or liability?

    A. Moradi Dakhel, V . Majdinasab, A. Nikanjam, F. Khomh, M. C. Desmarais, and Z. M. J. Jiang, “Github copilot ai pair programmer: Asset or liability?” Journal of Systems and Software, vol. 203, p. 111734, 2023. [Online]. Available: https: //www.sciencedirect.com/science/articl...

    2023

  14. [21]

    Ai-lab: A framework for introducing generative artificial intelligence tools in computer pro- gramming courses,

    E. Dickey, A. Bejarano, and C. Garg, “Ai-lab: A framework for introducing generative artificial intelligence tools in computer pro- gramming courses,” SN Computer Science, vol. 5, no. 6, p. 720, 2024

    2024

  15. [22]

    Ai-ta: Towards an intelligent question-answer teaching assistant using open-source llms,

    Y. Hicke, A. Agarwal, Q. Ma, and P . Denny, “Ai-ta: Towards an intelligent question-answer teaching assistant using open-source llms,” arXiv preprint arXiv:2311.02775, 2023

    2023 arXiv

  16. [23]

    Plagiarism in the age of generative ai: cheating method change and learning loss in an intro to cs course,

    B. Chen, C. M. Lewis, M. West, and C. Zilles, “Plagiarism in the age of generative ai: cheating method change and learning loss in an intro to cs course,” in Proceedings of the Eleventh ACM Conference on Learning@ Scale, 2024, pp. 75–85

    2024

  17. [24]

    Large language models in computer science education: A systematic literature review,

    N. Raihan, M. L. Siddiq, J. C. Santos, and M. Zampieri, “Large language models in computer science education: A systematic literature review,” in Proceedings of the 56th ACM Technical Symposium on Computer Science Education V . 1, ser. SIGCSETS 2025. New York, NY, USA: Associa...

    2025 doi

  18. [25]

    An exploratory study on upper-level computing students’ use of large language models as tools in a semester-long project,

    B. A. Tanay, L. Arinze, S. S. Joshi, K. A. Davis, and J. C. Davis, “An exploratory study on upper-level computing students’ use of large language models as tools in a semester-long project,” arXiv preprint arXiv:2403.18679, 2024

    2024 arXiv

  19. [26]

    Chatgpt and large language models in academia: opportunities and challenges,

    J. G. Meyer, R. J. Urbanowicz, P . C. Martin, K. O’Connor, R. Li, P .-C. Peng, T. J. Bright, N. Tatonetti, K. J. Won, G. Gonzalez- Hernandez et al., “Chatgpt and large language models in academia: opportunities and challenges,” BioData mining, vol. 16, no. 1, p. 20, 2023

    2023

  20. [27]

    Plagiarism in entry- level computer science courses using chatgpt,

    Z. Taylor, C. Blair, E. Glenn, and T. R. Devine, “Plagiarism in entry- level computer science courses using chatgpt,” in 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE). IEEE, 2023, pp. 1135–1139

    2023

  21. [28]

    Fooling moss detection with pretrained language models,

    S. Biderman and E. Raff, “Fooling moss detection with pretrained language models,” in Proceedings of the 31st ACM international conference on information & knowledge management , 2022, pp. 2933– 2943

    2022

  22. [29]

    A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level,

    I. Drori, S. Zhang, R. Shuttleworth, L. Tang, A. Lu, E. Ke, K. Liu, L. Chen, S. Tran, N. Cheng, R. Wang, N. Singh, T. L. Patti, J. Lynch, A. Shporer, N. Verma, E. Wu, and G. Strang, “A neural network solves, explains, and generates university math problems by program synthesis...

    2022 doi

  23. [30]

    Solving linear algebra by program synthesis,

    I. Drori and N. Verma, “Solving linear algebra by program synthesis,” 2021. [Online]. Available: https://arxiv.org/abs/2111. 08171

    2021

  24. [31]

    Learning methods for solving astronomy course problems,

    A. Shporer, S. Tran, N. Singh, B. Kates, J. Lynch, and I. Drori, “Learning methods for solving astronomy course problems,” 2023

    2023

  25. [32]

    Solving machine learning problems,

    S. Tran, P . Krishna, I. Pakuwal, P . Kafle, N. Singh, J. Lynch, and I. Drori, “Solving machine learning problems,” in Proceedings of The 13th Asian Conference on Machine Learning , ser. Proceedings of Machine Learning Research, V . N. Balasubramanian and I. Tsang, Eds., vol. ...

    2021

  26. [33]

    GPT-J-6B: A 6 Billion Parameter Au- toregressive Language Model,

    B. Wang and A. Komatsuzaki, “GPT-J-6B: A 6 Billion Parameter Au- toregressive Language Model,” https://github.com/kingoflolz/ mesh-transformer-jax, May 2021

    2021

  27. [34]

    The Pile: An 800gb dataset of diverse text for language modeling,

    L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, S. Presser, and C. Leahy, “The Pile: An 800gb dataset of diverse text for language modeling,” arXiv preprint arXiv:2101.00027, 2020

    2020 arXiv

  28. [35]

    Computer algorithms for plagiarism detection,

    A. Parker and J. Hamblen, “Computer algorithms for plagiarism detection,” IEEE Transactions on Education, vol. 32, no. 2, pp. 94–99, 1989

    1989

  29. [36]

    An empirical approach for detecting program similarity and plagiarism within a university programming environment,

    J. Faidhi and S. Robinson, “An empirical approach for detecting program similarity and plagiarism within a university programming environment,” Computers & Education, vol. 11, no. 1, pp. 11–19, 1987. [Online]. Available: https://www.sciencedirect. com/science/article/pii/03601...

    1987

  30. [37]

    Style analysis for source code plagiarism detection,

    O. Mirza and M. Joy, “Style analysis for source code plagiarism detection,” in Plagiarism Across Europe and Beyond (ECEIA15) , 2015, pp. 53–61

    2015

  31. [38]

    Style analysis for source code plagiarism detection,

    O. M. Mirza, “Style analysis for source code plagiarism detection,” Ph.D. dissertation, University of Warwick, 2018

    2018

  32. [39]

    Addressing student cheating: definitions and solutions,

    M. Dick, J. Sheard, C. Bareiss, J. Carter, D. Joyce, T. Harding, and C. Laxer, “Addressing student cheating: definitions and solutions,” ACM SigCSE Bulletin, vol. 35, no. 2, pp. 172–184, 2002

    2002

  33. [40]

    Plagiarism in programming assessments: a systematic review,

    I. Albluwi, “Plagiarism in programming assessments: a systematic review,” ACM Transactions on Computing Education (TOCE), vol. 20, no. 1, pp. 1–28, 2019

    2019

  34. [41]

    Style analysis for source code plagiarism detection—an analysis of a dataset of student coursework,

    O. M. Mirza, M. Joy, and G. Cosma, “Style analysis for source code plagiarism detection—an analysis of a dataset of student coursework,” in 2017 IEEE 17th international conference on advanced learning technologies (ICALT). IEEE, 2017, pp. 296–297

    2017

  35. [42]

    On-line plagiarism detector helps computer science professors bust cheating programmers,

    R. Sanders, “On-line plagiarism detector helps computer science professors bust cheating programmers,” Nov 1997. [Online]. Available: https://newsarchive.berkeley.edu/news/ media/releases/97legacy/11 19 97b.html

    1997

  36. [43]

    Obfuscation- resilient software plagiarism detection with jplag,

    T. Sa ˘glam, S. Hahner, L. Schmid, and E. Burger, “Obfuscation- resilient software plagiarism detection with jplag,” in Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, 2024, pp. 264–265

    2024

  37. [44]

    Software plagiarism detection techniques: A comparative study,

    D. Luke, S. DP , S. L. Johnson, S. Sreeprabha, and E. B. Varghese, “Software plagiarism detection techniques: A comparative study,” International Journal of Computer Science and Information Technologies (IJCSIT) , vol. 5, no. 4, pp. 5020–5024, 2014. [Online]. Available: https:...

    2014

  38. [45]

    Taps: A moss extension for detecting software plagiarism at scale,

    D. Sheahen and D. Joyner, “Taps: A moss extension for detecting software plagiarism at scale,” in Proceedings of the Third (2016) ACM Conference on Learning @ Scale , ser. L@S ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 285–288. [Online]. Available: h...

    2016

  39. [46]

    An approach to detect abnormal submissions for codeworkout dataset,

    A. Hicks, Y. Shi, A.-B. Lekshmi-Narayanan, W. Yan, and S. Marwan, “An approach to detect abnormal submissions for codeworkout dataset,” arXiv preprint arXiv:2407.17475, 2024

    2024 arXiv

  40. [47]

    Mossad: defeating software plagiarism detection,

    B. Devore-McDonald and E. D. Berger, “Mossad: defeating software plagiarism detection,” Proc. ACM Program. Lang. , vol. 4, no. OOPSLA, Nov. 2020. [Online]. Available: https: //doi.org/10.1145/3428206

    2020 doi

  41. [48]

    How do code plagiarism detection systems work?: R/compsci,

    anonymousprofessor99, “How do code plagiarism detection systems work?: R/compsci,” Jun 2020. [Online]. Available: https://www.reddit.com/r/compsci/comments/hfb3qy/how do code plagiarism detection systems work/?rdt=35083#:∼: text=Sadly%20it%20is%20possible%20the,you%20that%20th...

    2020

  42. [49]

    Competition-level code generation with alphacode,

    Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. D. Lago, T. Hubert, P . Choy, C. de Masson d’Autume, I. Babuschkin, X. Chen, P .-S. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. S. Robson, P . ...

    2022 doi

  43. [50]

    An empirical evaluation of competitive programming ai: A case study of alphacode,

    S. Lertbanjongngam, B. Chinthanet, T. Ishio, R. G. Kula, P . Lee- laprute, B. Manaskasemsak, A. Rungsawang, and K. Matsumoto, “An empirical evaluation of competitive programming ai: A case study of alphacode,” in 2022 IEEE 16th International Workshop on Software Clones (IWSC),...

    2022

  44. [51]

    De-anonymizing programmers via code stylometry,

    A. Caliskan-Islam, R. Harang, A. Liu, A. Narayanan, C. Voss, F. Yamaguchi, and R. Greenstadt, “De-anonymizing programmers via code stylometry,” in 24th USENIX security symposium (USENIX Security 15), 2015, pp. 255–270

    2015

  45. [52]

    Are ai detectors good enough? a survey on quality of datasets with machine-generated texts,

    G. Gritsai, A. Voznyuk, A. Grabovoy, and Y. Chekhovich, “Are ai detectors good enough? a survey on quality of datasets with machine-generated texts,” 2025. [Online]. Available: https://arxiv.org/abs/2410.14677

    2025 arXiv

  46. [53]

    Detecting multimedia generated by large ai models: A survey,

    L. Lin, N. Gupta, Y. Zhang, H. Ren, C.-H. Liu, F. Ding, X. Wang, X. Li, L. Verdoliva, and S. Hu, “Detecting multimedia generated by large ai models: A survey,” 2024. [Online]. Available: https://arxiv.org/abs/2402.00045

    2024 arXiv

  47. [55]

    Curriculum design of competitive programming: a contest-based approach,

    Z. Luo, “Curriculum design of competitive programming: a contest-based approach,” arXiv preprint arXiv:2504.00533, 2025

    2025 arXiv

  48. [56]

    Gaide: A framework for using gen- erative ai to assist in course content development,

    E. Dickey and A. Bejarano, “Gaide: A framework for using gen- erative ai to assist in course content development,” in 2024 IEEE Frontiers in Education Conference (FIE), 2024, pp. 1–9

    2024

  49. [57]

    Automated assessment in computer science education: A state-of-the-art review,

    J. C. Paiva, J. P . Leal, and A. Figueira, “Automated assessment in computer science education: A state-of-the-art review,” ACM Trans. Comput. Educ. , vol. 22, no. 3, Jun. 2022. [Online]. Available: https://doi.org/10.1145/3513140

    2022 doi

  50. [58]

    Detection of source code similitude in academic environments,

    A. M. Bejarano, L. E. Garc ´ıa, and E. E. Zurek, “Detection of source code similitude in academic environments,” Computer Applications in Engineering Education, vol. 23, no. 1, pp. 13–22, 2015

    2015

  51. [59]

    Validity matters more than cheating,

    P . Dawson, M. Bearman, M. Dollinger, and D. B. and, “Validity matters more than cheating,” Assessment & Evaluation in Higher Education, vol. 49, no. 7, pp. 1005–1016, 2024. [Online]. Available: https://doi.org/10.1080/02602938.2024.2386662

    2024

  52. [60]

    A revision of bloom’s taxonomy: An overview,

    D. R. Krathwohl, “A revision of bloom’s taxonomy: An overview,” Theory into practice, vol. 41, no. 4, pp. 212–218, 2002. 21

    2002

  53. [61]

    An approach to source-code plagiarism detection and investigation using latent semantic analysis,

    G. Cosma and M. Joy, “An approach to source-code plagiarism detection and investigation using latent semantic analysis,” IEEE transactions on computers, vol. 61, no. 3, pp. 379–394, 2011

    2011

  54. [62]

    Program char- acterization using runtime values and its application to software plagiarism detection,

    Y.-C. Jhi, X. Jia, X. Wang, S. Zhu, P . Liu, and D. Wu, “Program char- acterization using runtime values and its application to software plagiarism detection,” IEEE Transactions on Software Engineering , vol. 41, no. 9, pp. 925–943, 2015

    2015

  55. [63]

    Program dependence graph generation and analysis for source code plagiarism detection,

    N. Holma, “Program dependence graph generation and analysis for source code plagiarism detection,” Ph.D. dissertation, Link ¨opings Universitet, 2012

    2012

  56. [64]

    Academic source code plagia- rism detection by measuring program behavioral similarity,

    H. Cheers, Y. Lin, and S. P . Smith, “Academic source code plagia- rism detection by measuring program behavioral similarity,” IEEE Access, vol. 9, pp. 50 391–50 412, 2021

    2021

  57. [65]

    Gplag: detection of software plagiarism by program dependence graph analysis,

    C. Liu, C. Chen, J. Han, and P . S. Yu, “Gplag: detection of software plagiarism by program dependence graph analysis,” in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’06. New York, NY, USA: Association for Comp...

    2006

  58. [66]

    Principles of algorithmic problem solving,

    J. Sannemo, “Principles of algorithmic problem solving,” Kth. se , 2018, problems avaible at https://open.kattis.com/ problem-sources/Principles%20of%20Algorithmic%20Problem% 20Solving

    2018

  59. [2020]

    Available: https://www.diva-portal.org/smash/ record.jsf?pid=diva2%3A1463303&dswid=1403

    [Online]. Available: https://www.diva-portal.org/smash/ record.jsf?pid=diva2%3A1463303&dswid=1403

  60. [2025]

    Available: https://arxiv.org/abs/2505.00100

    [Online]. Available: https://arxiv.org/abs/2505.00100

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.