Pith. sign in

REVIEW 3 major objections 4 minor 36 references

Students say they distrust AI-generated code, yet in a hands-on classroom exercise over 95% accepted insecure code from a deliberately poisoned assistant; the paper argues that targeted feedback can begin to close that gap.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 20:07 UTC pith:756JUIC6

load-bearing objection The RQ1 finding — security-educated students who distrust LLM code still submit insecure code ~95% of the time — is a solid, useful empirical baseline; the RQ2 effectiveness claim is not supported by the design, and the reported Wilcoxon W looks internally inconsistent. the 3 major comments →

arxiv 2511.20878 v2 pith:756JUIC6 submitted 2025-11-25 cs.CR

Supporting Students in Navigating LLM-Generated Insecure Code

Jaehwan Park , Seonhye Park , Kyungchan Lim , Hyoungshick Kim , Doowon Kim This is my paper
classification cs.CR
keywords LLM-generated code securityAI code assistantssecurity educationpoisoning attacksstudent perceptionscritical thinkingVS Code extensionBifröst
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that undergraduate students in security courses have a perception–behavior gap when using AI code assistants: they report skepticism about the security of LLM output, but in a realistic task they overwhelmingly accept code with known vulnerabilities (ECB-mode AES and shell=True command execution). The authors present Bifröst, a classroom framework that pairs a poisoned code-generation model with a VS Code extension and automated vulnerability reports, and find preliminary evidence that receiving concrete feedback shifts students' stated trust toward greater distrust. If correct, the study shows that security education cannot rely on critical attitudes alone and that experiential feedback about specific flaws is a promising way to foster a security mindset.

Core claim

In the paper's own terms, the central finding is twofold. First, students' stated skepticism about LLM-generated code does not predict their behavior: 58 of 61 students (95%) submitted the intentionally insecure code for an AES task and 60 of 61 (98%) did so for a command-execution task, despite most having completed security coursework and expressed distrust in a pre-survey. Second, after receiving Bifröst's automated vulnerability reports, a self-selected subset of 21 students showed a statistically significant shift toward distrust of AI-generated code security (moderate effect size), with responses moving from 'somewhat trust' and neutral toward 'somewhat distrust' and 'highly distrust'.

What carries the argument

The carrying mechanism is Bifröst, a classroom measurement-and-feedback framework: a VS Code extension that lets students request code from a deliberately poisoned code-generation model (fine-tuned to suggest insecure patterns such as ECB mode and shell=True), a submission pipeline that runs static analysis to flag vulnerabilities, and an automated PDF report emailed to each student that names the vulnerable lines and explains the risk. The poisoned model makes insecure code appear functionally correct, so the only reliable signal students have is their own security judgment; the framework turns that invisible failure into visible feedback.

Load-bearing premise

The claim that Bifröst increases skepticism rests on the assumption that the 21 students who voluntarily completed the post-survey represent all 61 participants, and that the observed shift in distrust is caused by the feedback rather than by the act of being told their code was insecure or by wanting to appear more critical.

What would settle it

A randomized controlled experiment in which one group receives Bifröst's vulnerability report and a control group receives a generic 'your code was reviewed' message without specifics; if the control group shows the same shift in self-reported distrust, the feedback mechanism is not the active ingredient. Alternatively, if students who receive the report go on to accept insecure code at the same rate in a follow-up task, the framework's effect on actual security behavior is unsupported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Security educators should treat students' stated distrust of AI code as a starting point, not an outcome; attitudes do not automatically translate into secure behavior.
  • Feedback that points to specific vulnerabilities in code the student actually wrote can increase skepticism toward AI-generated code, suggesting a concrete intervention for classrooms.
  • The framework gives instructors a measure of student preparedness for LLM-assisted development, independent of self-report.
  • Because the vulnerable code runs without errors, exercises like these teach students that functional correctness is not evidence of security.
  • Future iterations of such frameworks may need instructor-led follow-up, since a few students in the study did not internalize the feedback.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The perception–behavior gap documented here likely extends beyond students to professional developers, who face similar pressures to accept working AI code; a replication with experienced practitioners would test that.
  • A stronger test of the framework's effectiveness would measure whether the observed shift in self-reported trust changes actual acceptance behavior in a follow-up task, not just survey responses.
  • The framework could be adapted to other vulnerability classes and languages, and to compare feedback formats (e.g., inline IDE warnings vs. emailed reports) for their effect on skepticism.
  • Since the post-survey was voluntary and only a third responded, the reported effect may overstate the intervention's reach; a mandatory post-survey or control group would clarify the true impact.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Bifröst, a classroom framework that combines a VS Code extension connected to a deliberately poisoned code-generation LLM with static-analysis-based vulnerability feedback, deployed in an undergraduate security class (n=61). The study reports two findings: (RQ1) despite students' stated skepticism toward LLM-generated code, over 95% of them submitted insecure code for both AES-encryption and command-execution tasks; and (RQ2) based on a voluntary post-survey (n=21), self-reported distrust increased after receiving Bifröst's feedback, supported by a one-sided Wilcoxon signed-rank test (W=80.5, p=0.033). The paper claims this is the first empirical investigation of student preparedness for insecure LLM code and that Bifröst fosters a security mindset.

Significance. If the RQ1 result holds, it provides a valuable and timely demonstration of a perception-behavior gap: students who say they distrust LLM code still submit vulnerable code in a realistic IDE setting. This is a useful contribution to security education and human-AI interaction. The Bifröst framework itself is a reasonable and reproducible intervention, and the deployment with a real course is a strength. However, the RQ2 effectiveness claim is currently not reliable: it rests on a self-selected 34% post-survey without a control condition, and the reported test statistic is numerically inconsistent with the data. The central RQ1 measurement appears directionally robust, but the paper's contribution to demonstrating that feedback increases skepticism is not yet established.

major comments (3)
  1. [§5.2, Statistical Validation] The reported Wilcoxon statistic W=80.5 is impossible with the data as described. The post-survey distribution implies 11 tied zero differences among the 21 pairs (2 initially "neither" who stayed, 9 initially "distrust" who stayed, and 0 from the "somewhat trust" group). With ties removed, the effective N for the signed-rank test is 10, so the maximum possible sum of positive ranks is 55, not 80.5. The p-value of 0.033 corresponds to a much smaller W (around 8-10). This indicates a computational or reporting error. Please re-run the exact test with zeros excluded and report the correct W, p, and effect size, or explain how W=80.5 was computed.
  2. [§5.2, Effectiveness of Bifröst] The causal claim that "Bifröst increases students' skepticism toward LLM-generated code" is not supported by the study design. The post-survey was completed by only 21 of 61 students (34%), a self-selected subset. No comparison of responders vs. non-responders is provided, so selection bias is possible. The absence of a control group means the observed shift could be due to test-retest effects, demand characteristics, or simply the act of receiving a report that labels one's submitted code as insecure. Even with a corrected Wilcoxon test, this design cannot support the strong conclusion in the RQ2 answer. The abstract's phrase "preliminary evidence" is appropriate; the body's "statistically significant evidence" and "can effectively develop" are overstatements. Please add explicit limitations and temper the claims.
  3. [§5.1, Student Ability to Identify Insecure AI Code] The paper claims students "accepted" insecure LLM-generated code, but the analysis only examines the submitted code, not whether the student clicked the plugin's "Use code" button. The plugin logs these decisions (as stated in §3.2), but the results do not report these logs. A student who manually typed the insecure code after seeing it, or who modified it, did not necessarily "accept" the LLM's output. This distinction matters for RQ1's framing about reliance on LLM-generated code. Please report the logged acceptance decisions and, if unavailable, soften the interpretation to "submitted vulnerable code" rather than "accepted the LLM's suggestion."
minor comments (4)
  1. [§5.2, Initially Neither paragraph] The text says that among the 7 initially "neither" students, 2 maintained "neither", 4 shifted to "somewhat distrust", and 1 to "highly distrust" — which sums to 7 — but then immediately mentions "one student (4.8%) changed their response to 'somewhat trust'". This appears to be an inconsistency or the sentence is misplaced. Please clarify the counts.
  2. [Figure 7] The percentage labels in Figure 7 are hard to map to the text (e.g., preliminary shows 48%, 19%, 33%, but the text reports 10/47.6%, 7/33.3%, 4/19.0%). Please label each bar with the exact n and percentage, and ensure the figures are consistent with the text.
  3. [General] The manuscript contains several formatting placeholders (e.g., "Conference'17", "July 2017", the ACM DOI template). These need to be updated before any venue submission.
  4. [§5.2, Effect size] The matched rank-biserial effect size of 0.53 is reported without a formula or confidence interval. Given the ties and the small effective N (10), this effect size should be recomputed and accompanied by a confidence interval or at least the underlying nonzero-difference count.

Circularity Check

0 steps flagged

No circular derivation; RQ1/RQ2 are direct empirical measurements, and the sole self-citation is background and non-load-bearing.

full rationale

The paper's claims are empirical, not derivational. Bifröst is an intervention (poisoned CodeGen model + VS Code plugin + Bandit/CodeQL feedback), and RQ1 is answered by logged accept/reject decisions against static-analysis flags, while RQ2 is answered by a paired pre/post self-report trust question. No fitted parameter is relabeled as a prediction, no quantity is defined in terms of the quantity it is claimed to explain, and no uniqueness theorem or prior ansatz from the authors is imported to force the result. The 'over 95% vulnerable' outcome is a direct observation that students did not modify suggested insecure code, not a consequence of the measurement definition. The RQ2 comparison is vulnerable to self-selection (21/61) and demand characteristics, but those are internal-validity threats, not circularity. The only self-citation is [33] (co-author Doowon Kim) in the background paragraph on poisoning attacks; it is background support, and the paper independently constructs its own poisoned model citing Trojanpuzzle [1] and evaluates submissions with external static analyzers, so the self-citation is not load-bearing. Hence no circular step is present; the low score reflects only the minor non-load-bearing self-citation.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The paper introduces no fitted free parameters and no new physical/conceptual entities. Its validity rests on measurement assumptions: static-analysis fidelity, model plausibility, the validity of self-reported trust as a proxy, the representativeness of the voluntary post-survey, and the causal interpretation of a pre-post design without a control group.

axioms (5)
  • domain assumption Bandit and CodeQL correctly classify student submissions as vulnerable or not, including ECB mode and shell=True, without false negatives that would misattribute insecure code.
    Section 3.2 stage 4 and Section 5.1 rely on these tools to identify the injected vulnerabilities; if a tool missed the vulnerability for some submissions, the reported 95% could be wrong.
  • domain assumption The fine-tuned poisoned CodeGen 6.1B model produces code that is functionally plausible enough that a student accepting it is making a security error, not a quality error.
    Section 3.2 'Generating Poisoned Model' asserts the model 'suggests' insecure code; the study's inference that students are 'vulnerable' assumes the code looks acceptable.
  • domain assumption Self-reported Likert trust in 'security of code snippets from AI tools' is a valid and stable proxy for security awareness.
    Used as the pre/post outcome in Sections 4.3 and 5.2; if self-reports are susceptible to social desirability or wording effects, the measured shift may not reflect a genuine mindset change.
  • domain assumption The 21 students who voluntarily completed the post-survey are representative of the 61 participants.
    Section 5.2 states '21 students (34%) who completed the optional post-survey'; if respondents were more engaged or more changed, the Wilcoxon result does not generalize.
  • domain assumption The observed trust shift is caused by the Bifröst feedback, not by test-retest effects or demand characteristics (no control group).
    Section 5.2 uses a one-group pre-post design with a one-sided Wilcoxon test; without a control condition, the causal attribution is an assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 10742 in / 13634 out tokens · 129328 ms · 2026-08-03T20:07:12.652677+00:00 · methodology

0 comments
read the original abstract

The advent of Artificial Intelligence (AI), particularly large language models (LLMs), has revolutionized software development by enabling developers to specify tasks in natural language and receive corresponding code, boosting productivity. However, this shift also introduces security risks, as LLMs may generate insecure code that can be exploited by adversaries. Conventional educational approaches emphasize efficiency while overlooking these risks, leaving students unprepared to identify and mitigate security issues in AI-assisted workflows. To surface this gap, we present \texttt{Bifr\"ost}, a classroom measurement and feedback framework that pairs an adversarially configured code-generation model with a VS Code extension and automated vulnerability reports that instructors can use to guide follow-up discussions. Through classroom deployments with undergraduate students ($n=61$), we observe that students frequently accepted insecure LLM-generated code despite prior security coursework and stated skepticism. A post-feedback survey ($n=21$) provides preliminary evidence that students' stated trust shifted toward greater skepticism after receiving Bifr\"ost feedback, and that some students articulated more security-specific concerns about AI-generated code.

Figures

Figures reproduced from arXiv: 2511.20878 by Doowon Kim, Hyoungshick Kim, Jaehwan Park, Kyungchan Lim, Seonhye Park.

Figure 1
Figure 1. Figure 1: Usage Example of Code-generation LLMs. Attacker Poisoned Model Malicious Open-source repository Insecure Code Compromised Systems Injection Training Generation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of Our Framework. LLM, with all activity logged for analysis. 2 Furthermore, the in￾structor distributes programming tasks and the VS Code plugin to students without disclosing that the LLM has been poisoned. 3 The students solve the tasks using the code suggested by the LLM. 4 The submitted code is analyzed for security vulnerabilities using static analysis tools (Bandit [24] and CodeQL [16]). Fr… view at source ↗
Figure 4
Figure 4. Figure 4: Example of VS Code Plugin. Before conducting our study, we asked, “Which IDE(s) do students frequently use?” and found that 64 out of 68 students reported using VS Code. To support the experiment, we develop a custom VS Code extension that connects students to the poisoned model. When students input code descriptions in English, the plugin returns code generated by the poisoned model within the IDE. In VS … view at source ↗
Figure 6
Figure 6. Figure 6: Student Responses to Insecure Code Generation: [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Preliminary Survey Results for Trust Levels. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Post Survey Results. Trust Level in Security of Code [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 3 linked inside Pith

  1. [1]

    Hojjat Aghakhani, Wei Dai, Andre Manoel, Xavier Fernandes, Anant Kharkar, Christopher Kruegel, Giovanni Vigna, David Evans, Ben Zorn, and Robert Sim

  2. [2]

    Aoife Ahern, Caroline Dominguez, Ciaran McNally, John J O’Sullivan, and Daniela Pedrosa. 2019. A Literature Review of Critical Thinking in Engineering Education.Studies in Higher Education44, 5 (2019), 816–828

    2019

  3. [3]

    Anthropic. 2025. Anthropic Claude. https://www.anthropic.com/ Accessed: 2025-05-25

    2025

  4. [4]

    Sam Attwood and Ashley Williams. 2023. Exploring the UK Cyber Skills Gap through a mapping of active job listings to the Cyber Security Body of Knowl- edge (CyBOK). InProceedings of the International Conference on Evaluation and Assessment in Software Engineering (EASE)

    2023

  5. [5]

    Louise Axon, Katherine Fletcher, Arianna Schuler Scott, Marcel Stolz, Robert Hannigan, Ali El Kaafarani, Michael Goldsmith, and Sadie Creese. 2022. Emerging Cybersecurity Capability Gaps in the Industrial Internet of Things: Overview and Research Agenda.Digital Threats: Research and Practice3, 4 (2022), 1–27

    2022

  6. [6]

    Shraddha Barke, Michael B James, and Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code-generating Models.Proceedings of the ACM on Programming Languages7, OOPSLA1 (2023), 85–111

    2023

  7. [7]

    Chris Bopp, Anne Foerst, and Brian Kellogg. 2024. The Case for LLM Workshops. InProceedings of the ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE TS)

    2024

  8. [8]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al . 2024. A Survey on Evaluation of Large Language Models.ACM Transactions on Intelligent Systems and Technology15, 3 (2024), 1–45

    2024

  9. [9]

    Peter Chapman, Jonathan Burket, and David Brumley. 2014. PicoCTF: A Game- Based Computer Security Competition for High School Students. InProceedings of the USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE)

    2014

  10. [10]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374(2021)

    Pith/arXiv arXiv 2021

  11. [11]

    DistantJob. 2024. Programming Languages Ranking: Top 9 in 2024. https: //distantjob.com/blog/programming-languages-rank/ Accessed: 2025-05-25

    2024

  12. [12]

    Manuel Egele, David Brumley, Yanick Fratantonio, and Christopher Kruegel

  13. [13]

    Laura Farinetti and Luca Cagliero. 2025. A Critical Approach to ChatGPT: An Experience in SQL Learning. InProceedings of the ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE TS)

    2025

  14. [14]

    Laura Farinetti and Lorenzo Canale. 2024. Chatbot Development Using LangChain: A Case Study to Foster Critical Thinking and Creativity. InProceed- ings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE)

    2024

  15. [15]

    Efstratios Gavas, Nasir Memon, and Douglas Britton. 2012. Winning Cybersecu- rity One Challenge at a Time.IEEE Security & Privacy10, 4 (2012), 75–79

    2012

  16. [16]

    GitHub Inc. 2025. CodeQL. https://codeql.github.com/ Accessed: 2025-05-25

    2025

  17. [17]

    Google DeepMind. 2025. Google DeepMind Gemini. https://deepmind.google/ models/gemini/ Accessed: 2025-05-25

    2025

  18. [18]

    Ying Guo and Daniel Lee. 2023. Leveraging ChatGPT for Enhancing Critical Thinking Skills.Journal of Chemical Education100, 12 (2023), 4876–4883

    2023

  19. [19]

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. StarCoder: May the Source Be with You!arXiv preprint arXiv:2305.06161(2023)

    Pith/arXiv arXiv 2023

  20. [20]

    Connor Nelson, Adam Doupé, and Yan Shoshitaishvili. 2025. SENSAI: Large Language Models as Applied Cybersecurity Tutors. InProceedings of the ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE TS)

    2025

  21. [21]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. InProceedings of the Inter- national Conference on Learning Representations (ICLR)

    2023

  22. [22]

    OpenAI. 2025. OpenAI ChatGPT. https://openai.com/ Accessed: 2025-05-25

    2025

  23. [23]

    Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2022. Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions. InProceedings of the IEEE Symposium on Security and Privacy (SP)

    2022

  24. [24]

    Python Software Foundation. 2025. Bandit. https://bandit.readthedocs.io/en/ latest/ Accessed: 2025-05-25

    2025

  25. [25]

    Stack Overflow. 2024. Stack Overflow Dev Survey. https://visualstudiomagazine. com/articles/2024/07/26/so-dev-survey.aspx Accessed: 2025-05-25

    2024

  26. [26]

    Zhendong Su and Gary Wassermann. 2006. The Essence of Command Injection Attacks in Web Applications.Acm Sigplan Notices41, 1 (2006), 372–382

    2006

  27. [27]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and Efficient Foundation Language Models.arXiv preprint arXiv:2302.13971(2023)

    Pith/arXiv arXiv 2023

  28. [28]

    Giovanni Vigna, Kevin Borgolte, Jacopo Corbetta, Adam Doupe, Yanick Fratanto- nio, Luca Invernizzi, Dhilung Kirat, and Yan Shoshitaishvili. 2014. Ten Years of iCTF: The Good, The Bad, and The Ugly. InProceedings of the USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE)

    2014

  29. [29]

    2019.Fostering Students’ Creativity and Critical Thinking: What It Means in School

    Stéphan Vincent-Lancrin, Carlos González-Sancho, Mathias Bouckaert, Federico De Luca, Meritxell Fernández-Barrerra, Gwénaël Jacotin, Joaquin Urgel, and Quentin Vidal. 2019.Fostering Students’ Creativity and Critical Thinking: What It Means in School. Educational Research and Innovation. ERIC

    2019

  30. [30]

    Jan Vykopal, Valdemar Švábensk`y, Michael Tuscano Lopez, and Pavel Čeleda

  31. [31]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. CodeT5: Identifier- aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. InProceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2021

  32. [32]

    Frank Wilcoxon. 1945. Individual comparisons by ranking methods.Biometrics bulletin1, 6 (1945), 80–83

    1945

  33. [33]

    Shenao Yan, Shen Wang, Yue Duan, Hanbin Hong, Kiho Lee, Doowon Kim, and Yuan Hong. 2024. An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection. InProceedings of the USENIX Security Symposium (USENIX Security)

    2024

  34. [2013]

    InProceedings of the ACM SIGSAC conference on Computer & communications security (CCS)

    An Empirical Study of Cryptographic Misuse in Android Applications. InProceedings of the ACM SIGSAC conference on Computer & communications security (CCS)

  35. [2024]

    InProceedings of the IEEE Symposium on Security and Privacy (SP)

    Trojanpuzzle: Covertly Poisoning Code-suggestion Models. InProceedings of the IEEE Symposium on Security and Privacy (SP)

  36. [2025]

    InProceedings of the ACM Technical Symposium on Computer Science Education V

    Cybersecurity Study Programs: What’s in a Name?. InProceedings of the ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE TS)