Pith. sign in

REVIEW 2 major objections 6 minor 44 references

Impossible to hide secret ...: Uncovering Security and Privacy Issues in LLM-native IDEs

T0 review · 2 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Most security and privacy problems in AI-native IDEs come from the tools' system design, not the language models themselves.

desk verdict A careful taxonomy of developer-reported security/privacy complaints in AI IDEs, but the headline '7 in 10 issues are system-level' is a coding judgment the paper never documents. read the letter →

arxiv 2607.26390 v3 pith:2YUXM7OY submitted 2026-07-29 cs.SE

Mostafijur Rahman Akhond , Md Afif Al Mamun , Gias Uddin , Song Wang This is my paper
classification cs.SE
keywords LLM-nativeIDEAIcodingassistantssecurityandprivacydeveloper-reportedissuestaxonomyRedditanalysisautonomousagentssecretsleakage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the security and privacy problems developers encounter in LLM-native IDEs (LIDEs) — AI assistants embedded in coding tools like Cursor, Copilot, and Codex — mostly originate in the integration layer rather than in the language models themselves. Analyzing 446 Reddit posts and 6,000 comments from 29 subreddits, the authors build a taxonomy of 32 reported issues grouped into 10 categories, from unauthorized file operations and destructive autonomous actions to opaque data flows and cross-session context leakage. They report that 7 out of 10 issues are system-level, arising from how LIDEs access files, enforce permissions, and execute autonomous actions, while only 2 out of 10 trace to the LLM's own reasoning. Developers respond with external safeguards — sandboxing, manual code review, version control, and sensitive-file protection — a pattern the paper reads as widespread mistrust of built-in LIDE controls. The stakes: if the paper is right, making AI coding tools safer means redesigning the IDE as a security boundary, not just improving the model.

What carries the argument

The central instrument is the taxonomy itself: 32 low-level issue codes grouped into 10 high-level categories, built through open and axial coding grounded in ISO/IEC 27001 (the CIA triad) and ISO/IEC 29100 (privacy principles). This taxonomy converts unstructured Reddit anecdotes into structured claims about where LIDEs fail. The accompanying system-level versus LLM-level attribution is the lens that carries the argument: for each issue, the authors judged whether the root cause lay in integration logic (file access, permission enforcement, autonomous action, data flow) or in model reasoning (unsafe generation, hallucination). A recall-oriented LLM filter plus full manual verification of al

What would settle it

A controlled experiment that configures several LIDEs with identical projects containing secrets in .env files, explicit ignore rules, and approval gates, then instruments the tool to log every file read, command executed, and network transmission. If, across thousands of runs, the majority of security violations trace to the model's output rather than to the integration layer's enforcement logic, the paper's 7/10 system-level attribution would be overturned.

Watch

Extended reading notes

Core claim

The central discovery, on the paper's own terms, is that LIDE security and privacy failures are primarily a system-engineering problem. The authors analyzed 1.1 million Reddit posts, manually verified 446 incident reports, and coded them into a taxonomy of 32 issues under 10 categories (five security, five privacy). The load-bearing result: roughly 7 of 10 issues are attributed to system-level design choices — an IDE reading a .env file despite ignore rules, bypassing approval gates, or deleting files autonomously — while about 2 in 10 trace to the LLM itself. The paper also documents 13 mitigation strategies developers use because they do not trust built-in safeguards, and concludes that LI

Load-bearing premise

The central claim rests on treating informal Reddit posts as reliable enough to attribute each issue to either system-level design or LLM-level behavior, even though the posts often lack configuration details and confirmed root-cause evidence.

Editorial extensions

If this is right

  • LIDE vendors should prioritize permission enforcement, file-access boundaries, and approval gates over model-level safety tuning, because most reported failures live in the integration layer.
  • Designing LIDEs as software systems first implies native verification layers that validate generated code and tool calls before they act, rather than relying on developers to catch mistakes after the fact.
  • The documented reliance on external mitigations (sandboxing, manual review, version control, sensitive-file protection) indicates that built-in safeguards are not currently meeting user expectations; closing that gap is a concrete design target.
  • The finding that external tool execution (e.g., MCP servers and plugins) is implicated in a disproportionate share of issues points to the need for centralized vetting and compliance processes for coding agents and extensions, analogous to app-store review.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the 7/10 split is inferred from Reddit anecdotes rather than instrumented reproductions, the ratio most plausibly describes the distribution of what developers notice and report, not the true prevalence of vulnerabilities; a controlled benchmark could test whether the same split holds under systematic probing.
  • If system-level issues dominate, then security research on AI coding tools should shift emphasis from prompt-injection and model-alignment failures toward the permission model of the IDE's agent harness — the layer that decides which files can be read and which commands can run.
  • A natural testable extension is a security benchmark for LIDEs: run a standard set of adversarial development tasks (secrets in .env files, explicit 'do not modify' instructions, malicious MCP servers) across tools and measure how often the integration layer violates the stated constraints; the paper's taxonomy provides the category labels for such a benchmark.
  • The paper's reliance on public Reddit reports means enterprise and closed-source deployments are underrepresented; if LIDEs are adopted in regulated settings, the same taxonomy could be validated against incident reports from those environments to see whether system-level issues still dominate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper investigates security and privacy issues in LLM-native IDEs (LIDEs) by mining Reddit. The authors collect a large corpus (reported inconsistently as 1.1M posts from 29 subreddits or 15.4M posts/comments from 46 subreddits), apply an LLM-based recall-oriented filter, manually validate all positive candidates, and identify 446 posts and 6,280 comments about LIDE security/privacy issues. They qualitatively code the posts into a 10-category taxonomy containing 32 low-level issues, extract 13 mitigation strategies from the comments, and derive six design recommendations. The headline finding is that '7 out of 10' reported issues are system-level (integration logic, permissions, transparency) rather than LLM-level, with about 1 in 10 attributed to both.

Significance. If the 70/20/10 system-vs-LLM split were supported, it would shift the security research agenda for AI coding tools from model alignment toward the IDE integration layer: file access, permission enforcement, autonomous action, and transparency. The paper has notable strengths: all LLM-positive posts were manually verified with near-perfect agreement (Cohen's kappa = 0.967), the recall-oriented filter was audited at 1.00 recall on 200 negative samples, and the authors share a replication package with data, prompts, codebooks, and scripts. The taxonomy of reported issues and the catalog of mitigation strategies are useful community resources independent of the split. However, the central quantitative claim is not backed by a described, audited attribution method, which currently limits the paper's contribution to the taxonomy and mitigation findings.

major comments (2)
  1. [§5, Fig. 11] The central claim '7 out of 10 security and privacy issues were system-level issues' is not reproducible from the manuscript. Section 5 refers only to 'the replication package'; no codebook, decision rule, coder agreement, or worked examples for the ternary system/LLM/both attribution are given in the paper, and raw counts are not reported. The taxonomy itself is defined in system-centric terms (e.g., C1 'Unauthorized File Operations', C6 'Lack of Transparency'), so if the attribution was made by mapping taxonomy categories to system/LLM, the 70/20 split is partly a restatement of category construction rather than an empirical finding about root cause. The paper should describe the attribution protocol, report inter-rater reliability for this label, give per-category agreement, and address multi-label posts (Table 3) explicitly.
  2. [§7] The construct-validity paragraph concedes that Reddit posts 'lack configuration detail or root-cause evidence,' yet the 7/10 split requires exactly that root-cause inference (e.g., deciding whether unauthorized file access is due to the LLM's plan or the system's failure to enforce constraints). The abstract's 'most issues in LIDEs stem from system-level design choices' therefore overstates what the data support. At minimum, the claim should be restricted to 'developer-reported issues' or to 'issue discussions on Reddit,' and the split should be presented as coder judgment. The authors' reproduction attempts (Fig. 1) are a promising triangulation but are not described in enough detail to validate the attribution.
minor comments (6)
  1. [§5, 'Verification-First Code Recommendations'] The text says '17.8% of security vulnerabilities to unsafe generation (O3)', but Section 3.2 and Figure 6 report C3 as 18.2%. Please reconcile or explain the different denominator.
  2. [Abstract vs §2.1/§2.2] The abstract states 1.1M posts from 29 subreddits; Section 2.1 says 15.4M posts/comments from 46 subreddits, and Section 2.2 says the final dataset came from 29 subreddits. Clarify what the 1.1M counts and why the corpus size differs.
  3. [§3.4] The text says Cursor accounts for '130 of the 376 IDE-referenced posts', but Section 3.1 says 383 posts explicitly named a LIDE, and the Figure 4 counts sum to 383. Correct the 376 figure.
  4. [§5, 'Promises and Perils of Coding Agents'] The assertion that EXTOOL is 'responsible for 5 out of 10' identified issues is causal and is not supported by the described feature-support matrix (Table 5) or Figure 12. Reframe as a hypothesis or provide an explicit association/attribution analysis.
  5. [§2.2] The audit of 200 LLM-negative posts found 0 missed relevant posts; report a confidence interval (e.g., binomial 95% CI) so the reader can calibrate the residual false-negative risk.
  6. [§3.2/§3.3] The 'General Security Concerns' (15.5%) and 'General Privacy Concerns' (26.8%) categories are large residual buckets. Consider breaking them down or discussing what they contain, as they limit the precision of the taxonomy.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the taxonomy is induced from external Reddit data; the system-vs-LLM split is an interpretive coding judgment, not a derivation from its own inputs.

full rationale

The paper's derivation chain is a qualitative empirical pipeline: collect 1.1M Reddit posts, apply a recall-oriented LLM filter, manually verify all positive candidates to obtain 446 posts, perform open and axial coding to build a taxonomy, and extract mitigation strategies from comments. There are no fitted parameters, equations, or target-defined constants that the conclusions reduce to by construction. The headline '7 out of 10 security and privacy issues were system-level issues' is an attribution made by the authors about qualitative posts, not a quantity computed from a definition that already asserts the conclusion. The main transparency concern, raised by the skeptics, is that the system-vs-LLM coding procedure is not documented in the main text and is delegated to the replication package; the paper itself acknowledges that Reddit posts 'lack configuration detail or root-cause evidence' (Section 7, Threats to Validity). However, an unvalidated or underexplained coding scheme is a validity/reproducibility weakness, not circularity: the coding does not use the paper's own results as inputs. The taxonomy categories are partially framed in system-centric terms (e.g., C1 Unauthorized File Operations, C6 Lack of Transparency), so the 7/10 split is sensitive to category construction, but the paper does not present a formal derivation that equates the finding with the category definitions. No load-bearing self-citation, imported uniqueness theorem, or ansatz smuggled in through prior work appears. Consequently, the central claim has independent empirical content and receives a circularity score of 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities; the study is an inductive qualitative analysis. Its load-bearing assumptions are about the evidential value of Reddit data and the reliability of manual coding, especially the system-vs-LLM attribution.

assumptions (3)
  • domain assumption Reddit posts and comments are a valid corpus for identifying real, current security and privacy issues experienced by LIDE users.
    The entire pipeline (Section 2.1) relies on Reddit discussions as the source of ground truth for developer-reported issues.
  • domain assumption Manual open coding of 446 posts into the security/privacy taxonomy and the 'system vs LLM' attribution is reliable enough to support aggregate percentages.
    Post relevance coding has high inter-rater agreement (kappa = 0.967), but the system-vs-LLM attribution behind Figure 11 does not report a separate inter-rater validation; see Section 5.
  • domain assumption The recall-oriented LLM filter (GPT-OSS:20B) does not systematically omit relevant discussions.
    An audit of 200 LLM-negative posts found 0 missed relevant posts (Section 2.2), but the sample is small relative to the 15.4M corpus.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Impossible to hide secret ...: Uncovering Security and Privacy Issues in LLM-native IDEs." pith.science (2026). https://pith.science/paper/2YUXM7OY

@misc{pith2026260726390,
  author       = {Pith},
  title        = {Pith review of: Impossible to hide secret ...: Uncovering Security and Privacy Issues in LLM-native IDEs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2YUXM7OY}},
  note         = {Machine review of arXiv:2607.26390}
}
read the original abstract

LLM-native IDEs (Integrated Development Environments), aka LIDEs, are designed from the ground up to work with Large Language Models (LLMs). LIDEs have found remarkable success in Software Engineering (SE) tasks such as coding, debugging, and program comprehension. LIDEs are software systems, and, like any system, they can exhibit vulnerabilities. In this paper, we study the security and privacy issues that developers reported while using popular LIDEs in their development tasks. We collected 1.1M posts from 29 popular subreddits related to LIDEs. We identified 446 posts and analyzed over 6K comments to the posts that discussed security and privacy issues in almost all popular LIDEs, such as Cursor, Copilot, Codex, etc. Using a mix of qualitative and quantitative methods, we constructed a taxonomy of the reported security and privacy issues. Our results show that most issues in LIDEs stem from system-level design choices, rather than the underlying LLMs, such as user data access, unchecked autonomous actions, etc. To overcome these issues, developers frequently relied on external safeguards like code sandboxing and manual reviewing, highlighting prevalent mistrust among developers about LIDEs. We share lessons from our study to support future design of secure and privacy-aware LIDEs.

Figures

Figures reproduced from arXiv: 2607.26390 by the authors.

Figure 1
Figure 1. A Reddit post reported unauthorized access to a secret (.env) file (left). We confirm that it still persists (right). [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Workflow of the Study attempted to modify the .env file, ignoring the configured access restrictions. This behavior raises serious security and privacy is￾sues, as the file contained private secrets and API keys, exposing developers to significant confidentiality risks. These findings motivated us to study (1) the security and privacy issues in popular LIDEs reported by developers in social forums like Reddit, (2) h… view at source ↗
Figure 3
Figure 3. Monthly counts of the security & privacy posts [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (10 more)
Figure 6
Figure 6. Figure 6: Taxonomy of the reported security issues [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: A post on Codex Unusual deletion of contents [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 5
Figure 5. Figure 5: Monthly pattern of Reddit posts on LIDEs [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 8
Figure 8. Figure 8: A Reddit post on generating spam with Cursor [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Taxonomy of Detected Privacy Issues 3.3 Reported Privacy Issues in the IDEs In total, 194 posts corresponded to privacy issues, grouped into five high-level and 15 low-level categories (see [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Percentage distribution of security and privacy [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 12
Figure 12. Figure 12: Mapping of AI-assisted IDE features to observed [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 11
Figure 11. Figure 11: LIDEs mentioned in System and LLM Level Issues [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 14
Figure 14. Figure 14: Comparison of our work with related works on [PITH_FULL_IMAGE:figures/full_fig_p011_14.png]
Figure 13
Figure 13. Figure 13: Distribution of security and privacy issues based [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 3 linked inside Pith

  1. [1]

    Naqvi Abiha, Jain Apeksha, Goyal Avisha, and Verma Ankita. 2025. Understand- ing Code Quality: A Qualitative Evaluation of LLM-Generated vs. Human-Written Code.Int. J. Perform. Eng.21 (2025), 559

    2025

  2. [2]

    Anisha Agarwal, Aaron Chan, Shubham Chandel, Jinu Jang, Shaun Miller, Roshanak Zilouchian Moghaddam, Yevhen Mohylevskyy, Neel Sundaresan, and Michele Tufano. 2024. Copilot evaluation harness: Evaluating llm-guided soft- ware programming.arXiv preprint arXiv:2402.14261(2024)

    arXiv 2024

  3. [3]

    2025.AI IDE List

    AI IDE List. 2025.AI IDE List. https://aiidelist.com/ide Accessed: 2025-11-02

    2025

  4. [4]

    Resul Sezgin Albayrak, Meltem Kurt Pehlivanoğlu, and Nur Banu Albayrak. 2025. Evaluating Security and Quality of Banking Software Generated by Large Lan- guage Models.2025 Innovations in Intelligent Systems and Applications Conference (ASYU)(2025), 1–7

    2025

  5. [5]

    Enna Basic and Alberto Giaretta. 2024. Large language models and code security: A systematic literature review.arXiv preprint arXiv:2412.15004(2024)

    arXiv 2024

  6. [6]

    Nguyen, Hridesh Rajan, Nikolaos Tsantalis, and Danny Dig

    Abhiram Bellur, Fraol Batole, Mohammed Raihan Ullah, Malinda Dilhara, Yaroslav Zharov, Timofey Bryksin, Kai Ishikawa, Haifeng Chen, Masaharu Morimoto, Takeo Hosomi, Tien N. Nguyen, Hridesh Rajan, Nikolaos Tsantalis, and Danny Dig. 2025. Together We are Better: LLM, IDE and Semantic Embedding to As- sist Move Method Refactoring.2025 IEEE International Conf...

    2025

  7. [7]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In30th USENIX security symposium (USENIX Security 21). 2633–2650

    2021

  8. [8]

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey.Comput. Surveys57, 6 (2025), 1–39

    2025

Show all 44 references
  1. [9]

    Databricks. 2024. Passing the Security Vibe Check: The Dangers of Vibe Cod- ing. https://www.databricks.com/blog/passing-security-vibe-check-dangers- vibe-coding. Accessed 2025

    2024

  2. [10]

    Yujia Fu, Peng Liang, Amjed Tahir, Zengyang Li, Mojtaba Shahin, Jiaxin Yu, and Jinfu Chen. 2023. Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study.ACM Transactions on Software Engineering and Methodology34 (2023), 1 – 34. https://api.semantic...

    2023

  3. [11]

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review.ACM Transactions on Software Engineering and Methodology33, 8 (2024), 1–79

    2024

  4. [12]

    Azizkhon YUnushon Ishankhonov, Dmitrii Viktorovich Pshychenko, Ev- genii Aleksandrovich Mozharovskii, and Andrei Sergeevich Aluev. 2024. The Role of LLM in Next-Generation Integrated Development Environments.Software systems and computational methods4 (2024), 140–150

    2024

  5. [13]

    Sajed Jalil. 2025. The transformative influence of llms on software development & developer productivity. In2025 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA). IEEE, 1–10

    2025

  6. [14]

    Impossible to hide secret

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation.ACM Transactions on Software Engineering and Methodology(2024). “Impossible to hide secret ... ”: Uncovering Security and Privacy Issues in LLM-native I...

    2024

  7. [15]

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues?arXiv preprint arXiv:2310.06770(2023)

    2023 arXiv

  8. [16]

    Ravie Lakshmanan. 2025. Researcher Uncovers 30+ Flaws in AI Coding Tools Enabling Data Theft and RCE Attacks.The Hacker News(Dec 2025). https://thehackernews.com/2025/12/researchers-uncover-30-flaws-in-ai.html

    2025

  9. [17]

    Shuang Li, Yuntao Cheng, Jinfu Chen, Jifeng Xuan, Sen He, and Weiyi Shang

  10. [18]

    Yichen Li, Yun Peng, Yintong Huo, and Michael R. Lyu. 2024. Enhancing LLM- Based Coding Tools through Native Integration of IDE-Derived Static Con- text.2024 IEEE/ACM International Workshop on Large Language Models for Code (LLM4Code)(2024), 70–74

    2024

  11. [19]

    Elizabeth Lin, Igibek Koishybayev, Trevor Dunlap, William Enck, and Alexandros Kapravelos. 2024. UntrustIDE: Exploiting Weaknesses in VS Code Extensions. Proceedings 2024 Network and Distributed System Security Symposium(2024)

    2024

  12. [20]

    Xinpeng Liu et al. 2025. Cuckoo Attack: Stealthy and Persistent Attacks Against AI-Integrated Development Environments.arXiv preprint arXiv:2509.15572 (2025)

    2025

  13. [21]

    Daniel Loureiro, Francesco Barbieri, Leonardo Neves, Luis Espinosa Anke, and Jose Camacho-Collados. 2022. TimeLMs: Diachronic language models from Twit- ter. InProceedings of the 60th annual meeting of the association for computational linguistics: System demonstrations. 251–260

    2022

  14. [22]

    My productivity is boosted, but

    Yunbo Lyu, Zhou Yang, Jieke Shi, Jianming Chang, Yue Liu, and David Lo. 2025. "My productivity is boosted, but... " Demystifying Users’ Perception on AI Cod- ing Assistants. InProceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE 2025...

    2025

  15. [23]

    Ari Marzouk. 2025. IDEsaster: A Novel Vulnerability Class in AI IDEs. MaccariTA blog post. https://maccarita.com/posts/idesaster/

    2025

  16. [24]

    2024.Code Generation and Testing in the Era of AI-Native Software Engineering

    Noble Saji Mathews. 2024.Code Generation and Testing in the Era of AI-Native Software Engineering. Ph. D. Dissertation. University of Waterloo. https:// uwspace.uwaterloo.ca/items/96c98e86-e652-43d4-be27-0b21351c777d

    2024

  17. [25]

    Yutao Mou, Xiao Deng, Yuxiao Luo, Shikun Zhang, and Wei Ye. 2025. Can You Really Trust Code Copilot? Evaluating Large Language Models from a Code Security Perspective. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2025

  18. [26]

    Hyrum Pearce, Baleegh Ahmad, Benjamin Tan, Sergei Dolgikh, Zachary Johnson, Matthew Miller, Prateek Tomar, Hassan Elahi, Adnan Habib, and Thomas Reps

  19. [27]

    Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot.arXiv preprint arXiv:2302.06590(2023)

    2023 arXiv

  20. [28]

    Dorin Pomian, Abhiram Bellur, Malinda Dilhara, Zarina Kurbatova, Egor Bogo- molov, Timofey Bryksin, and Danny Dig. 2024. Next-Generation Refactoring: Combining LLM Insights and IDE Capabilities for Extract Method.2024 IEEE International Conference on Software Maintenance and E...

    2024

  21. [29]

    2025.Best AI Coding Assistant Tools

    Qodo.ai. 2025.Best AI Coding Assistant Tools. https://www.qodo.ai/blog/best-ai- coding-assistant-tools Accessed 2025-11

    2025

  22. [30]

    Bozidar Radenkovic, Sergei Prokhorov, Milos Radenkovic, and Aleksandra Labus

  23. [31]

    2025.A list of a few AI IDEs - would love additions to try!Red- dit

    Daniel Rosehill. 2025.A list of a few AI IDEs - would love additions to try!Red- dit. https://www.reddit.com/r/ChatGPTCoding/comments/1imbxvq/a_list_of_ a_few_ai_ides_would_love_additions_to/ Accessed: 2025-11-02

    2025

  24. [32]

    2025.I’ve tried all (46) AI Coding Agents & IDEs

    John Rush. 2025.I’ve tried all (46) AI Coding Agents & IDEs. DEV Community. https://dev.to/johnrushx/ive-tried-all-46-ai-coding-agents-ides-c2g Accessed: 2025-11-02

    2025

  25. [33]

    2021.The Coding Manual for Qualitative Researchers(4th ed.)

    Johnny Saldaña. 2021.The Coding Manual for Qualitative Researchers(4th ed.). SAGE Publications Ltd, London

    2021

  26. [34]

    Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, and Michael Young

    D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, and Michael Young. 2014. Machine Learning: The High Interest Credit Card of Technical Debt. InSE4ML: Software Engineering for Machine Learning (NIPS 2014 Workshop)

    2014

  27. [35]

    Agnia Sergeyuk, Sergey Titov, and Maliheh Izadi. 2024. In-IDE Human-AI Experi- ence in the Era of Large Language Models; A Literature Review.2024 IEEE/ACM First IDE Workshop (IDE)(2024), 95–100

    2024

  28. [36]

    Faten Slama and Daniel Lemire. 2025. Enhancing Developer Productivity: Bench- marking LLM-Powered Tools like GitHub Copilot and TabNine in Real-Time Coding Environments. In2025 IEEE 11th International Conference on Intelligent Data and Security (IDS). IEEE Computer Society, 39–45

    2025

  29. [37]

    Maryam Taeb, Hongmei Chi, and Shonda Bernadin. 2024. Assessing the Effective- ness and Security Implications of AI Code Generators.Journal of The Colloquium for Information Systems Security Education(2024)

    2024

  30. [38]

    Ningzhi Tang, Meng Chen, Zheng Ning, Aakash Bansal, Yu Huang, Collin McMil- lan, and Toby Jia-Jun Li. 2024. Developer Behaviors in Validating and Repairing LLM-Generated Code Using IDE and Eye Tracking.2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC...

    2024

  31. [39]

    Weixi Tong and Tianyi Zhang. 2024. Codejudge: Evaluating code generation with large language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 20032–20051

    2024

  32. [40]

    Rebeka Tóth, Tamas Bisztray, and László Erdődi. 2024. Llms in web development: Evaluating llm-generated php code unveiling vulnerabilities and limitations. In International Conference on Computer Safety, Reliability, and Security. Springer, 425–437

    2024

  33. [41]

    2025.The Impact of LLM-Based Coding Assistants on Developer Productivity

    Annemarie Wittig. 2025.The Impact of LLM-Based Coding Assistants on Developer Productivity. Master’s thesis. Leipzig University, Faculty of Math- ematics and Computer Science. https://sws.informatik.uni-leipzig.de/wp- content/uploads/2025/09/master_thesis_the_impact_of_llm_bas...

    2025

  34. [2022]

    InProceedings of the 2022 IEEE Symposium on Security and Privacy (S&P)

    Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions. InProceedings of the 2022 IEEE Symposium on Security and Privacy (S&P). IEEE, 754–768

    2022

  35. [2025]

    In2025 6th International Workshop on Engineering Technologies and Computer Science (EnT)

    Application of Large Language Models in Software Development: Review of the Current State and Development Perspectives. In2025 6th International Workshop on Engineering Technologies and Computer Science (EnT). IEEE, 1–7

  36. [2026]

    Performance analysis of AI-generated code: A case study of Copilot, Copilot Chat, CodeLlaMa, and DeepSeek-Coder models.Empirical Software Engineering 31 (2026)

    2026

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.