Pith. sign in

REVIEW 2 major objections 4 minor 79 references

Evaluating Software Plagiarism Detection in the Age of AI: Automated Obfuscation and Lessons for Academic Integrity

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Layered defenses beat AI code obfuscation in plagiarism checks

desk verdict Solid broad evaluation of two existing defenses, but the external validity caveat about synthetic obfuscations is real and matters most for the AI-based results. read the letter →

arxiv 2505.20158 v1 pith:BA26XV4T submitted 2025-05-26 cs.SE

Timur Sağlam , Larissa Schmid This is my paper · ORCID
classification cs.SE
keywords sourcecodeplagiarismdetectionobfuscationattackstokensequencenormalizationsubsequencematchmergingJPlagGPT-4AI-generated
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that two existing defenses against code-plagiarism obfuscation, Token Sequence Normalization (TSN) and Subsequence Match Merging (SMM), remain effective when combined and when tested against a broader threat model than before, including LLM-based attacks. Across more than four million pairwise comparisons on six real course datasets, the authors find that the combined defenses widen the similarity gap between plagiarized pairs and unrelated pairs for insertion-based, refactoring-based, and GPT-4-based obfuscation. The gains matter because automated obfuscation has made token-based detectors such as JPlag much easier to evade; if the results hold in practice, educators can substantially harden existing tools with two relatively cheap, language-independent post-processing steps. A secondary result is that the same defenses improve the separation between AI-generated and human-written programs, even though they were not designed for that purpose.

What carries the argument

The load-bearing machinery is a two-stage pipeline wrapped around a token-based detector such as JPlag. TSN first builds a Token Normalization Graph, a graph-based abstraction of token interdependencies, to identify and remove dead-code nodes and to topologically sort reordered tokens, effectively de-obfuscating the token sequence before comparison. SMM then operates on the matching results, heuristically merging neighboring matched subsequences and iteratively filling the gaps that obfuscation has introduced, until no more neighboring matches can be merged. TSN is attack-specific, targeting structural insertions and reordering, while SMM is attack-independent, and their combination is what produces the measured separation.

What would settle it

One concrete test: collect confirmed, naturally obfuscated student plagiarism pairs from academic-integrity case files and measure the median similarity separation under JPlag with TSN+SMM; if the separation is close to the 10-20 point range typical of unrelated pairs, the defense gains do not transfer to real student behavior. Another test: run the same pipeline against obfuscation from a newer LLM that rewrites code idiomatically rather than inserting dead statements; if few fragmented matches remain, SMM will have little to repair and the measured resilience will drop.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the two defenses are complementary rather than redundant: TSN virtually eliminates insertion-based obfuscation (median similarity separation up to 99.65 percentage points on Java datasets), SMM carries most of the gains against refactoring-based attacks (up to 22 percentage points), and the combined pipeline still yields substantial improvements against GPT-4-based obfuscation (up to 19 percentage points) where either mechanism alone struggles. The same combined pipeline improves the median similarity separation between AI-generated and human programs by up to 8.92 percentage points, and it does so while leaving the similarity of unrelated, independently written programs practically unchanged, with measured effect sizes on unrelated pairs ranging from negligible to small. The authors therefore claim that token-based plagiarism detectors can be made substantially more resistant to automated and AI-assisted plagiarism without a meaningful false-positive cost, and that threshold-based obfuscation, which keeps inserting dead code until a similarity target is reached, becomes too slow and too conspicuous to be practical.

Load-bearing premise

The evaluation's external validity rests on the assumption that the synthetic plagiarism instances generated by MOSSad, PlagGen, Spoon-based refactorings, and GPT-4 prompts are representative of the ways real students actually obfuscate copied assignments.

Editorial extensions

If this is right

  • Insertion-based obfuscation, currently one of the cheapest ways to defeat JPlag, is rendered near-ineffective: the combined defenses achieve up to 99.65 percentage points of median similarity separation and eliminate measurable overlap between plagiarized and unrelated pairs.
  • Refactoring-based obfuscation is substantially mitigated through SMM, with median separation gains up to 22 percentage points, even though TSN alone adds almost nothing for this attack.
  • LLM-based obfuscation remains the hardest regime: gains reach only about 19 percentage points and vary by dataset and prompt, so AI-rewritten code still requires more human review than algorithmically obfuscated code.
  • The defenses also separate AI-generated from human submissions by up to 8.92 percentage points of median similarity, meaning the same tooling can flag suspicious clusters of LLM-written programs even without dedicated AI detectors.
  • Threshold-based obfuscation becomes slow and visible: MOSSad required up to roughly six hours per small C++ program and inflated code by up to about 1300 percent, making the obfuscated output conspicuous by size alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If real student obfuscation resembles the insertion and refactoring patterns tested here, then adopting the TSN+SMM pipeline in existing JPlag-like systems is a low-risk hardening step; but the paper's own threat-model discussion implies that a shift toward full reimplementation or idiomatic semantic rewrites would bypass both defenses.
  • The improved separation of AI-generated programs suggests a practical side channel for detecting LLM-assisted submissions: when many students prompt the same model for the same assignment, the resulting programs are more similar to each other than human work, and SMM amplifies that signal.
  • A natural next test is to re-run the same pipeline against newer LLMs; as model outputs become more varied and less insertion-laden, SMM's gap-merging heuristic may find fewer fragmented matches to repair, so the measured resilience should be expected to decay.
  • The layering logic generalizes beyond plagiarism detection: attack-specific countermeasures should be paired with attack-independent heuristics so that unknown or emerging transformations still leave some detectable trace.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper evaluates two defense mechanisms for token-based plagiarism detectors, Token Sequence Normalization (TSN) and Subsequence Match Merging (SMM), against automated obfuscation attacks. The evaluation uses JPlag as the baseline detector and six real-world student datasets in Java and C++, covering dead-code insertion (via PlagGen and MOSSad), Spoon-based semantic-preserving refactoring, GPT-4-based obfuscation with 15 prompts, and GPT-4-based whole-program generation. The primary outcome metric is the similarity separation between plagiarism-to-source pairs and unrelated original pairs, reported via median/mean differences, Wilcoxon signed-rank tests, and Cliff's delta with confidence intervals across more than 4.1 million pairwise comparisons. The central claim is that the combined TSN+SMM defense substantially increases this separation for insertion-based attacks (up to 99.65 percentage points), moderately for refactoring attacks (up to 22 points), and to a smaller but still significant degree for GPT-4-based obfuscation (up to 19 points) and AI-generated programs (up to 8.92 points), while leaving unrelated-pair similarity practically unchanged. The paper also discusses implications for academic integrity and provides a replication package.

Significance. The evaluation is carefully designed: it avoids arbitrary similarity thresholds, uses multiple real-world datasets, reports both statistical and practical significance with confidence intervals, and packages the artifacts for replication. The separation-metric framing is a methodological improvement over threshold-based evaluations that are sensitive to chosen cutoffs. If the results hold, the defenses offer a low-cost upgrade for JPlag and, plausibly, other token-based detectors, with the strongest benefit against insertion-based obfuscation and a measurable but weaker benefit against AI-based obfuscation and generation. The main limitations are that only JPlag is used as the detector, all plagiarism instances are synthetically generated rather than taken from real student plagiarism, and the AI-based attack results for the C++ Homework datasets are weak or non-significant. These limitations constrain the strength of the paper's broader practical conclusions but do not invalidate the core empirical comparison.

major comments (2)
  1. [Sections 4.3, 6; Tables 8-9] The external validity of the obfuscation results is a load-bearing concern. As stated in Section 6, 'all plagiarism instances are generated through controlled automated obfuscation', and Section 4.3 explains that labeled human plagiarism was deliberately removed from the Ljubovic-Pajic datasets. The measured gains vary by roughly an order of magnitude across attack types, from 99.65 percentage points for insertion-based attacks (Table 4) to about 19 points for GPT-4 obfuscation (Table 8). For the C++ Homework datasets under GPT-4 obfuscation, the combined defense shows negligible or non-significant effects (Table 9, Homework-1: delta=0.024; Homework-5: p=0.11). Since the paper's Section 5 conclusion that the defenses are 'highly effective across a range of automated obfuscation attacks' supports practical academic-integrity recommendations, the lack of any evaluation on real student obfuscations leaves the practical magnitude of the benefit uncertain. A concrete test using the labeled human plagiarism instances that were filtered during preprocessing, or a sensitivity analysis with alternative obfuscation distributions, would directly address this threat to validity.
  2. [Sections 3, 4.2; Section 5] The paper claims in Section 3 that TSN and SMM are 'agnostic to the underlying detection system' and suitable for integration into any state-of-the-art token-based detector such as MOSS, JPlag, or Dolos, but all empirical results in Section 5 are obtained with JPlag only. Section 4.2 justifies excluding MOSS and Dolos for practical reasons, and Figure 4 demonstrates that all three tools are vulnerable to insertion-based obfuscation at baseline, yet no experiment reports the defense mechanisms applied to a second detector. If the contribution is intended to be detector-agnostic, the generalizability claim is under-supported. At minimum, the evaluation should include a second open-source token-based detector, for example Dolos on the single-file datasets, or the claims should be narrowed to JPlag.
minor comments (4)
  1. [Table 2] The row labeled 'Alteration-based Obf.' is not described in Section 4.4 and its counts duplicate the 'Insertion-based Obf.' row. The caption states 851 programs in total, which matches the sum of the other four rows, not five rows; the row should be removed or explained, and all related totals should be reconciled.
  2. [Section 4.4] The text says the four techniques create 787 plagiarized programs, while Table 2's caption says 851 in total; the discrepancy should be resolved by clarifying which instances were excluded, for example due to invalid GPT-4 outputs.
  3. [Section 5.6] The answer block at the end of Section 5.6 is labeled 'Answer to Q5' but should be 'Answer to Q6'.
  4. [Section 5.3] In the paragraph on Token Sequence Normalization, the phrase 'has little to no effect effect' contains a duplicated word and should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper is an empirical benchmark whose conclusions are measured, not derived from its inputs.

full rationale

Score 0. This paper is a controlled empirical evaluation, not a derivation. The central result — that TSN, SMM, and their combination increase the similarity separation between obfuscated plagiarism pairs and unrelated pairs — is measured on independently constructed obfuscated instances (MOSSad/PlagGen insertions, Spoon refactorings, GPT-4 prompts) rather than derived from the defense mechanisms' definitions. TSN's strong performance against insertion-based attacks is consistent with its design goal, but that is a design-evaluation match, not circularity: the similarity scores are computed by JPlag and are not fitted to produce the claimed margins. The self-citations [60,63] identify prior implementations used as treatments under evaluation; no load-bearing argument rests on those citations as proof of the present conclusions. Section 6 explicitly acknowledges the external-validity limitation that synthetic obfuscations may not match real student behavior; that is a threat to generalization, not circularity. No equation-level reduction, renamed output, or fitted-parameter-as-prediction pattern is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted and no new entities are postulated; the study is an empirical benchmark. The load-bearing assumptions are domain assumptions about the realism of synthetic attacks and the validity of similarity-score separation as a detection proxy, plus standard statistical tools. The authors explicitly flag most of these in Section 6.

assumptions (4)
  • domain assumption Synthetic obfuscation instances produced by MOSSad, PlagGen, Spoon-based refactorings, and GPT-4 are representative of real student plagiarism.
    Section 4.4 defines the threat model; Section 6 acknowledges limited generalizability of attacks.
  • domain assumption JPlag similarity scores and their separation between plagiarism and original pairs are a valid proxy for detection quality.
    Section 4.1 justifies avoiding fixed thresholds and using score differences instead of precision and recall.
  • standard math One-sided Wilcoxon signed-rank tests and Cliff's delta are appropriate statistics for these comparison distributions.
    Section 4.1 uses them to claim statistical and practical significance; the paired-data assumption is not empirically verified.
  • domain assumption The insertion and refactoring tools are semantic-preserving as stated.
    Section 4.4 asserts behavior preservation; if violated, the attacks differ from the intended threat model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Software Plagiarism Detection in the Age of AI: Automated Obfuscation and Lessons for Academic Integrity." pith.science (2026). https://pith.science/paper/BA26XV4T

@misc{pith2026250520158,
  author       = {Pith},
  title        = {Pith review of: Evaluating Software Plagiarism Detection in the Age of AI: Automated Obfuscation and Lessons for Academic Integrity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BA26XV4T}},
  note         = {Machine review of arXiv:2505.20158}
}
read the original abstract

Plagiarism in programming assignments is a persistent issue in computer science education, increasingly complicated by the emergence of automated obfuscation attacks. While software plagiarism detectors are widely used to identify suspicious similarities at scale and are resilient to simple obfuscation techniques, they are vulnerable to advanced obfuscation based on structural modification of program code that preserves the original program behavior. While different defense mechanisms have been proposed to increase resilience against these attacks, their current evaluation is limited to the scope of attacks used and lacks a comprehensive investigation regarding AI-based obfuscation. In this paper, we investigate the resilience of these defense mechanisms against a broad range of automated obfuscation attacks, including both algorithmic and AI-generated methods, and for a wide variety of real-world datasets. We evaluate the improvements of two defense mechanisms over the plagiarism detector JPlag across over four million pairwise program comparisons. Our results show significant improvements in detecting obfuscated plagiarism instances, and we observe an improved detection of AI-generated programs, even though the defense mechanisms are not designed for this use case. Based on our findings, we provide an in-depth discussion of their broader implications for academic integrity and the role of AI in education.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 46 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al. Gpt-4 technical report. Technical report, OpenAI, 2024

    work page 2024

  2. [2]

    Moss software plagiarism detector website, 7 2022

    Alex Aiken. Moss software plagiarism detector website, 7 2022. URL http://theory. stanford.edu/~aiken/moss/. 42 Timur Sa˘ glam, Larissa Schmid

    work page 2022

  3. [3]

    246 reasons to cheat: An analysis of students’ reasons for seeking to outsource academic work

    Alexander Amigud and Thomas Lancaster. 246 reasons to cheat: An analysis of students’ reasons for seeking to outsource academic work. Computers & Education , 134:98–107, 6 2019. ISSN 0360-1315. doi:https://doi.org/10.1016/j.compedu.2019.01.017

    work page doi:10.1016/j.compedu.2019.01.017 2019

  4. [4]

    Security Engineering: A Guide to Building Dependable Distributed Systems, Third Edition

    Ross Anderson. Security Engineering: A Guide to Building Dependable Distributed Systems, Third Edition . John Wiley & Sons Inc., United States, 3 edition, 12 2020. ISBN 9781119642787. doi:10.1002/9781119644682

    doi:10.1002/9781119644682 2020

  5. [5]

    Source code plagiarism detection in an educational context: A literature mapping

    Rodrigo C Aniceto, Maristela Holanda, Carla Castanho, and Dilma Da Silva. Source code plagiarism detection in an educational context: A literature mapping. In 2021 IEEE Frontiers in Education Conference (FIE) , pages 1–9. IEEE, 10 2021. doi:10.1109/FIE49875.2021.9637155

    arXiv 2021

  6. [6]

    Fooling moss detection with pretrained lan- guage models

    Stella Biderman and Edward Raff. Fooling moss detection with pretrained lan- guage models. In Mohammad Al Hasan and Li Xiong 0001, editors, Proceedings of the 31st ACM International Conference on Information & Knowledge Manage- ment, CIKM ’22, page 2933–2943, NY, USA, 10 2022. ACM. ISBN 9781450392365. doi:10.1145/3511808.3557079

    arXiv 2022

  7. [7]

    Braumoeller and Brian J

    Bear F. Braumoeller and Brian J. Gaines. Actions do speak louder than words: Deterring plagiarism with the use of plagiarism-detection software. PS: Political Science and Politics, 34(4):835–839, 10 2001. ISSN 1049-0965. doi:10.1017/s1049096501000786

    work page doi:10.1017/s1049096501000786 2001

  8. [8]

    Preventing Automatic Code Plagiarism Generation Through Token String Normalization

    Moritz Br¨ odel. Preventing Automatic Code Plagiarism Generation Through Token String Normalization. bachelor’s thesis, Karlsruhe Institute of Technology (KIT), 2023

    work page 2023

Show all 79 references
  1. [9]

    Richards Adrion, Betsy Bizot, Susan Davidson, Mary Hall, Susanne Hambrusch, Ellen Walker, and Stuart Zweben

    Tracy Camp, W. Richards Adrion, Betsy Bizot, Susan Davidson, Mary Hall, Susanne Hambrusch, Ellen Walker, and Stuart Zweben. Generation cs: The growth of computer science. ACM Inroads, 8(2):44–50, 5 2017. ISSN 2153-2184. doi:10.1145/3084362

    2017 doi

  2. [10]

    Lewis, Matthew West, and Craig Zilles

    Binglin Chen, Colleen M. Lewis, Matthew West, and Craig Zilles. Plagiarism in the age of generative ai: Cheating method change and learning loss in an intro to cs course. In Proceedings of the Eleventh ACM Conference on Learning @ Scale , L@S ’24, page 75–85, NY, USA, 2024. AC...

    2024

  3. [11]

    How far are we? the triumphs and trials of generative ai in learning software engi- neering

    Rudrajit Choudhuri, Dylan Liu, Igor Steinmacher, Marco Gerosa, and Anita Sarma. How far are we? the triumphs and trials of generative ai in learning software engi- neering. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE ’24, page 1–1...

    2024

  4. [13]

    Dominance statistics: Ordinal analyses to answer ordinal questions

    Norman Cliff. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin , 114(3):494–509, 11 1993. ISSN 0033-2909. doi:10.1037/0033- 2909.114.3.494

    1993 doi

  5. [14]

    Statistical Power Analysis for the Behavioral Sciences

    Jacob Cohen. Statistical Power Analysis for the Behavioral Sciences . Routledge, 5

  6. [15]

    Towards a definition of source-code plagia- rism

    Georgina Cosma and Mike Joy. Towards a definition of source-code plagia- rism. IEEE Transactions on Education , 51(2):195–200, 5 2008. ISSN 0018-9359. doi:10.1109/te.2007.906776

    2008

  7. [16]

    Plagiarism issues for higher education

    Fintan Culwin and Thomas Lancaster. Plagiarism issues for higher education. VINE, 31(2):36–41, 1 2001. ISSN 0305-5728. doi:10.1108/03055720010804005

    2001 doi

  8. [17]

    How chatgpt will change software engineering ed- ucation

    Marian Daun and Jennifer Brings. How chatgpt will change software engineering ed- ucation. In Mikko-Jussi Laakso, Mattia Monga, Simon, and Judithe Sheard, editors, Proceedings of the 2023 Conference on Innovation and Technology in Computer Sci- ence Education V. 1 , ITiCSE 202...

    2023

  9. [18]

    Breanna Devore-McDonald and Emery D. Berger. Mossad: Defeating software plagia- rism detection. Proceedings of the ACM on Programming Languages , 4(OOPSLA): 1–28, 11 2020. ISSN 2475-1421. doi:10.1145/3428206

    2020 doi

  10. [19]

    Semantic similarity search for source code plagiarism detection: An exploratory study

    Fahad Ebrahim and Mike Joy. Semantic similarity search for source code plagiarism detection: An exploratory study. In Mattia Monga, Violetta Lonati, Erik Barendsen, Judithe Sheard, and James Paterson, editors, Proceedings of the 2024 on Innovation and Technology in Computer Sc...

    2024

  11. [20]

    Faidhi and S.K

    J.A.W. Faidhi and S.K. Robinson. An empirical approach for detecting program sim- ilarity and plagiarism within a university programming environment. Computers & Education, 11(1):11–19, 1 1987. ISSN 0360-1315. doi:10.1016/0360-1315(87)90042-x

    1987 doi

  12. [21]

    Detecting machine-obfuscated plagiarism

    Tom´ aˇ s Folt´ ynek, Terry Ruas, Philipp Scharpf, Norman Meuschke, Moritz Schubotz, William Grosky, and Bela Gipp. Detecting machine-obfuscated plagiarism. In Anneli Sundqvist, Gerd Berget, Jan Nolin, and Kjell Ivar Skjerdingstad, editors, Sustainable Digital Communities , vo...

    2020 doi

  13. [22]

    Sim: a utility for detecting similarity in com- puter programs

    David Gitchell and Nicholas Tran. Sim: a utility for detecting similarity in com- puter programs. In Jane Prey and Robert E. Noonan, editors, The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education , vol- ume 6 of SIGCSE ’99 , page 266–270, NY...

    1999

  14. [23]

    Grissom and John J

    Robert J. Grissom and John J. Kim. Effect Sizes for Research . Routledge, 4 2012. ISBN 9781136632358. doi:10.4324/9780203803233

    2012 doi

  15. [24]

    Evading watermark based detection of ai-generated content

    Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai-generated content. In Weizhi Meng, Christian Damsgaard Jensen, Cas Cremers, and Engin Kirda, editors, Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications ...

    2023

  16. [25]

    Plagiarism in programming assignments

    Mike Joy and Micheal Luck. Plagiarism in programming assignments. IEEE Transac- tions on Education , 42(2):129–133, 5 1999. ISSN 0018-9359. doi:10.1109/13.762946

    1999 doi

  17. [26]

    Do code clones matter? In Proceedings of the 31st International Conference on Software Engineering, ICSE ’09, pages 485–495, USA, 1 2009

    Elmar Juergens, Florian Deissenboeck, Benjamin Hummel, and Stefan Wagner. Do code clones matter? In Proceedings of the 31st International Conference on Software Engineering, ICSE ’09, pages 485–495, USA, 1 2009. IEEE Computer Society. ISBN 9781424434534. doi:10.1109/ICSE.2009.5070547

    2009

  18. [27]

    Detecting source code plagiarism on introductory programming course assignments using a bytecode approach

    Oscar Karnalim. Detecting source code plagiarism on introductory programming course assignments using a bytecode approach. In 2016 International Conference on Informa- tion & Communication Technology and Systems (ICTS) , pages 63–68,, 10 2016. IEEE. doi:10.1109/icts.2016.7910274

    2016

  19. [28]

    Similarities of human and ai assistance in programming plagiarism: Student perspective

    Oscar Karnalim. Similarities of human and ai assistance in programming plagiarism: Student perspective. In Michael E. Auer and Tiia R¨ u¨ utmann, editors,Futureproofing Engineering Education for Global Responsibility , pages 149–156, Cham, 2025. Springer Nature. ISBN 978-3-031...

    2025 doi

  20. [29]

    Preprocessing for source code similarity detection in introductory programming

    Oscar Karnalim, Simon, and William Chivers. Preprocessing for source code similarity detection in introductory programming. In Nick Falkner and Otto Sepp¨ al¨ a, editors, Proceedings of the 20th Koli Calling International Conference on Computing Education Research, volume 36 o...

    2020

  21. [30]

    Educating students about programming plagiarism and collusion via formative feedback.ACM Trans

    Oscar Karnalim, Simon, William Chivers, and Billy Susanto Panca. Educating students about programming plagiarism and collusion via formative feedback.ACM Trans. Com- put. Educ. , 22(3):1–31, 6 2022. ISSN 1946-6226. doi:10.1145/3506717

    2022 doi

  22. [31]

    Detecting ai assisted submissions in introductory programming via code anomaly.Education and Information Technologies, 29(13):16841–16866, 9 2024

    Oscar Karnalim, Hapnes Toba, and Meliana Christianti Johan. Detecting ai assisted submissions in introductory programming via code anomaly.Education and Information Technologies, 29(13):16841–16866, 9 2024. ISSN 1573-7608. doi:10.1007/s10639-024- 12520-6

    2024 doi

  23. [32]

    Will chatgpt get you caught? rethinking of plagiarism detection

    Mohammad Khalil and Erkan Er. Will chatgpt get you caught? rethinking of plagiarism detection. Interacci´ on, 14040:475–487, 2 2023. ISSN 0302-9743. doi:10.48550/arXiv.2302.04335. 10.48550/arXiv.2302.04335

    2023 doi arXiv

  24. [33]

    Automatic source code plagiarism detection

    Cynthia Kustanto and Inggriani Liem. Automatic source code plagiarism detection. In Haeng-Kon Kim and Roger Y. Lee, editors, 2009 10th ACIS International Conference on Software Engineering, Artificial Intelligences, Networking and Parallel/Distributed Computing, pages 481–486,...

    2009 doi

  25. [34]

    Artificial intelligence, text generation tools and chatgpt – does digital watermarking offer a solution? International Journal for Educational Integrity , 19(1), 7 2023

    Thomas Lancaster. Artificial intelligence, text generation tools and chatgpt – does digital watermarking offer a solution? International Journal for Educational Integrity , 19(1), 7 2023. ISSN 1833-2595. doi:10.1007/s40979-023-00131-6. 44 Timur Sa˘ glam, Larissa Schmid

    2023 doi

  26. [35]

    Educating computer programming students about plagiarism through use of a code similarity detection tool

    Tri Le, Angela Carbone, Judy Sheard, Margot Schuhmacher, Michael de Raath, and Chris Johnson. Educating computer programming students about plagiarism through use of a code similarity detection tool. In 2013 Learning and Teaching in Computing and Engineering, pages 98–105. IEE...

    2013 doi

  27. [36]

    Validating and restoring defense in depth using attack graphs

    Richard Lippmann, Kyle Ingols, Chris Scott, Keith Piwowarski, Kendra Kratkiewicz, Mike Artz, and Robert Cunningham. Validating and restoring defense in depth using attack graphs. In MILCOM 2006 - 2006 IEEE Military Communications conference , pages 1–10. IEEE, 10 2006. doi:10....

    2006

  28. [37]

    Plagiarism detection in computer programming using feature extraction from ultra-fine-grained repositories

    Vedran Ljubovic and Enil Pajic. Plagiarism detection in computer programming using feature extraction from ultra-fine-grained repositories. IEEE Access, 8:96505–96514,

  29. [38]

    Semantics-based obfuscation-resilient binary code similarity comparison with applications to software and algorithm plagiarism detection

    Lannan Luo, Jiang Ming, Dinghao Wu, Peng Liu, and Sencun Zhu. Semantics-based obfuscation-resilient binary code similarity comparison with applications to software and algorithm plagiarism detection. IEEE Transactions on Software Engineering , 43 (12):1157–1177, 12 2017. ISSN ...

    2017

  30. [39]

    Dolos: Language-agnostic plagiarism detection in source code

    Rien Maertens, Charlotte Van Petegem, Niko Strijbol, Toon Baeyens, Arne Carla Ja- cobs, Peter Dawyndt, and Bart Mesuere. Dolos: Language-agnostic plagiarism detection in source code. Journal of Computer Assisted Learning , 38(4):1046–1061, 8 2022. ISSN 0266-4909. doi:https://d...

    2022 doi

  31. [40]

    Preventing Refactoring Attacks on Software Plagiarism Detec- tion through Graph-Based Structural Normalization

    Robin Manuel Maisch. Preventing Refactoring Attacks on Software Plagiarism Detec- tion through Graph-Based Structural Normalization. master’s thesis, Karlsruhe Institute of Technology (KIT), 2024

    2024

  32. [41]

    Audentes: Automatic detection of tentative plagiarism according to a reference solution

    Leonardo Mariani and Daniela Micucci. Audentes: Automatic detection of tentative plagiarism according to a reference solution. ACM Trans. Comput. Educ. , 12(1):1–26, 3 2012. ISSN 1946-6226. doi:10.1145/2133797.2133799

    2012

  33. [42]

    Cheating in computer science

    William Murray. Cheating in computer science. Ubiquity, 2010(October):2, 06 2010. ISSN 1530-2180. doi:10.1145/1865907.1865908

    2010

  34. [43]

    Syntax- based improvements to plagiarism detectors and their evaluations

    Lawton Nichols, Kyle Dewey, Mehmet Emre, Sitao Chen, and Ben Hardekopf. Syntax- based improvements to plagiarism detectors and their evaluations. In Bruce Schar- lau, Roger McDermott, Arnold Pears, and Mihaela Sabin, editors, Proceedings of the 2019 ACM Conference on Innovatio...

    2019

  35. [44]

    Effect of source-code preprocessing techniques on plagiarism detection accuracy in student programming assignments

    Matija Novak. Effect of source-code preprocessing techniques on plagiarism detection accuracy in student programming assignments . PhD thesis, University of Zagreb. Fac- ulty of Organization and Informatics, 2 2020

    2020

  36. [45]

    Source-code similarity detection and detection tools used in academia: A systematic review

    Matija Novak, Mike Joy, and Dragutin Kermek. Source-code similarity detection and detection tools used in academia: A systematic review. ACM Transactions on Comput- ing Education, 19(3):1–37, 9 2019. ISSN 1946-6226. doi:10.1145/3313290

    2019 doi

  37. [46]

    Detecting llm-generated text in computing education: Compar- ative study for chatgpt cases

    Michael Sheinman Orenstrakh, Oscar Karnalim, Carlos An ´ ıbal Su´ arez, and Michael Liut. Detecting llm-generated text in computing education: Compar- ative study for chatgpt cases. In 2024 IEEE 48th Annual Computers, Soft- ware, and Applications Conference (COMPSAC) , pages 1...

    2024 arXiv

  38. [47]

    K. J. Ottenstein. An algorithmic approach to the detection and prevention of plagiarism. ACM SIGCSE Bulletin , 8(4):30–41, 12 1976. ISSN 0097-8418. doi:10.1145/382222.382462

    1976

  39. [48]

    Progpedia: Collection of source-code submitted to introductory programming assignments

    Jos´ e Carlos Paiva, Jos´ e Paulo Leal, and ´Alvaro Figueira. Progpedia: Collection of source-code submitted to introductory programming assignments. Data in Brief , 46: 108887, 2 2023. ISSN 2352-3409. doi:https://doi.org/10.1016/j.dib.2023.108887

    2023

  40. [50]

    In other (people’s) words: Plagiarism by university students–literature and lessons

    Chris Park. In other (people’s) words: Plagiarism by university students–literature and lessons. Assessment & Evaluation in Higher Education , 28(5):471–488, 10 2003. ISSN 0260-2938. doi:10.1080/02602930301677

    2003 doi

  41. [51]

    Benefits and drawbacks of source code plagiarism detection in engineering education

    Dieter Pawelczak. Benefits and drawbacks of source code plagiarism detection in engineering education. In 2018 IEEE Global Engineering Education Conference (EDUCON), pages 1048–1056. IEEE, 4 2018. doi:10.1109/EDUCON.2018.8363346

    2018

  42. [52]

    Spoon: Program Analysis and Transformation in Java

    Renaud Pawlak, Carlos Noguera, and Nicolas Petitprez. Spoon: Program Analysis and Transformation in Java. Research Report RR-5901, Inria, 2006

    2006

  43. [53]

    JPlag: Finding pla- giarisms among a set of programs

    Lutz Prechelt, Guido Malpohl, and Michael Philippsen. JPlag: Finding pla- giarisms among a set of programs . Karlsruhe Institute of Technology, 2000. doi:10.5445/ir/542000. Technical Report

    doi

  44. [54]

    Finding plagiarisms among a set of programs with jplag

    Lutz Prechelt, Guido Malpohl, and Michael Philippsen. Finding plagiarisms among a set of programs with jplag. Journal of Universal Computer Science , 8(11):1016, 11

  45. [55]

    J. Reason. The contribution of latent human failures to the breakdown of complex systems. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 327(1241):475–484, 7 1990. ISSN 00804622. doi:10.4324/9781315092898-2

    1990 doi

  46. [56]

    Romano, J.D

    J. Romano, J.D. Kromrey, J. Coraggio, and J. Skowronek. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’sd for evaluating group differences on the NSSE and other surveys? Inannual meeting of the Florida Association of Institutional Res...

    2006

  47. [57]

    Roy, James R

    Chanchal K. Roy, James R. Cordy, and Rainer Koschke. Comparison and evaluation of code clone detection techniques and tools: A qualitative approach. Science of Computer Programming, 74(7):470–495, 5 2009. ISSN 0167-6423. doi:10.1016/j.scico.2009.02.007

    2009 doi

  48. [58]

    Guidelines for conducting and reporting case study research in software engineering

    Per Runeson and Martin H¨ ost. Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering , 14(2):131–164, 12

  49. [59]

    Replication Package for ”Evaluating Software Plagiarism Detection in the Age of AI”, 5 2025

    Timur Sa˘ glam. Replication Package for ”Evaluating Software Plagiarism Detection in the Age of AI”, 5 2025. URL https://github.com/tsaglam/EMSE25-SM. Final replica- tion package will be published on Zenodo

    2025

  50. [60]

    Detect- ing automatic software plagiarism via token sequence normalization

    Timur Sa˘ glam, Moritz Br¨ odel, Larissa Schmid, and Sebastian Hahner. Detect- ing automatic software plagiarism via token sequence normalization. In Proceed- ings of the IEEE/ACM 46th International Conference on Software Engineering , ICSE ’24, pages 113:1–113:13, NY, USA, 4 ...

    2024

  51. [62]

    Obfuscation- resilient software plagiarism detection with jplag

    Timur Sa˘ glam, Sebastian Hahner, Larissa Schmid, and Erik Burger. Obfuscation- resilient software plagiarism detection with jplag. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, volume 8 of ICSE-Companion, pag...

    2024

  52. [63]

    Mitigating ob- fuscation attacks on software plagiarism detectors via subsequence merging

    Timur Sa˘ glam, Nils Niehues, Sebastian Hahner, and Larissa Schmid. Mitigating ob- fuscation attacks on software plagiarism detectors via subsequence merging. In 46th IEEE/ACM International Conference on Software Engineering: Companion Proceed- ings, CSEE&T 2025, 2025. doi:10....

    2025

  53. [64]

    Detecting modelling plagiarism: Navigating automated obfuscation and generative ai

    Timur Sa˘ glam. Detecting modelling plagiarism: Navigating automated obfuscation and generative ai. In Educators Symposium, MODELS 2024 , MODELS, 9 2024. Invited Keynote

    2024

  54. [65]

    Negotiating the maze of academic in- tegrity in computing education

    Simon, Judy Sheard, Michael Morgan, Andrew Petersen, Amber Settle, Jane Sin- clair, Gerry Cross, and Charles Riedesel. Negotiating the maze of academic in- tegrity in computing education. In Proceedings of the 2016 ITiCSE Working Group Reports, ITiCSE ’16, page 57–80, NY, USA,...

    2016

  55. [66]

    M.R. Stytz. Considering defense in depth for software applications. IEEE Security & Privacy, 2(1):72–75, 1 2004. ISSN 1540-7993. doi:10.1109/MSECP.2004.1264860

    2004 arXiv

  56. [67]

    Hyunjae Suh, Mahan Tafreshipour, Jiawei Li, Adithya Bhattiprolu, and Iftekhar Ahmed. An empirical study on automatically detecting ai-generated source code: How far are we? In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering, ICSE ’25, NY, USA,...

    2025 doi arXiv

  57. [68]

    A model for exploring student un- derstandings of plagiarism

    Anna Sutton, David Taylor, and Carol Johnston. A model for exploring student un- derstandings of plagiarism. Journal of Further and Higher Education , 38(1):129–146, 1

  58. [69]

    Pengcheng Wang, Jeffrey Svajlenko, Yanzhao Wu, Yun Xu, and Chanchal K. Roy. Ccaligner: A token based large-gap clone detector. In Michel Chaudron, Ivica Crnkovic, Marsha Chechik, and Mark Harman, editors, 2018 IEEE/ACM 40th Inter- national Conference on Software Engineering (I...

    2018

  59. [70]

    Plagiarism detectors are a crutch, and a problem

    Debora Weber-Wulff. Plagiarism detectors are a crutch, and a problem. Nature, 567 (7749):435–435, 3 2019. ISSN 1476-4687. doi:10.1038/d41586-019-00893-5

    2019 doi

  60. [71]

    Testing of detec- tion tools for ai-generated text

    Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tom´ aˇ s Folt´ ynek, Jean Guerrero-Dib, Olumide Popoola, Petr ˇSigut, and Lorna Waddington. Testing of detec- tion tools for ai-generated text. International Journal for Educational Integrity , 19(1): 1–39, 12 2023. I...

    2023 doi

  61. [72]

    Ohlsson, Bj¨ orn Regnell, and An- ders Wessl´ en.Experimentation in Software Engineering

    Claes Wohlin, Per Runeson, Martin H¨ ost, Magnus C. Ohlsson, Bj¨ orn Regnell, and An- ders Wessl´ en.Experimentation in Software Engineering . Springer, Berlin, Heidelberg, 6 2012. ISBN 978-3-642-29043-5. doi:10.1007/978-3-642-29044-2

    2012 doi

  62. [73]

    Bai, Robert Tairas, and Yu Huang

    Yuankai Xue, Hanlin Chen, Gina R. Bai, Robert Tairas, and Yu Huang. Does chatgpt help with introductory programming? an experiment of students using chatgpt in cs1. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and ...

    2024

  63. [74]

    Tmoss: Using inter- mediate assignment work to understand excessive collaboration in large classes

    Lisa Yan, Nick McKeown, Mehran Sahami, and Chris Piech. Tmoss: Using inter- mediate assignment work to understand excessive collaboration in large classes. In Proceedings of the 49th ACM Technical Symposium on Computer Science Educa- tion, SIGCSE ’18, page 110–115, NY, USA, 2 ...

    2018

  64. [75]

    Toward Obfuscation-resilient Plagiarism Detection

    Fangfang Zhang. Toward Obfuscation-resilient Plagiarism Detection. PhD dissertation, The Pennsylvania State University, University Park, PA, 03 2014. Graduate Program: Computer Science and Engineering

    2014

  65. [76]

    Provable robust watermarking for AI-generated text

    Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for AI-generated text. In The Twelfth International Conference on Learning Representations. OpenReview.net, 2024. doi:10.48550/arXiv.2306.17439

    2024 doi arXiv

  66. [1988]

    doi:10.4324/9780203771587

    ISBN 9781134742707. doi:10.4324/9780203771587. First published: 1988

    1988 doi

  67. [2002]

    doi:10.3217/jucs-008-11-1016

    doi

  68. [2008]

    doi:10.1007/s10664-008-9102-8

    ISSN 1382-3256. doi:10.1007/s10664-008-9102-8

    doi

  69. [2014]

    doi:10.1080/0309877X.2012.706807

    ISSN 0309-877X. doi:10.1080/0309877X.2012.706807

    2012

  70. [2020]

    doi:10.1109/ACCESS.2020.2996146

    ISSN 2169-3536. doi:10.1109/ACCESS.2020.2996146

    2020

  71. [2023]

    ISBN 9798400700507

    ACM. ISBN 9798400700507. doi:10.1145/3576915.3623189

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.