Pith. sign in

REVIEW 4 major objections 6 minor 28 references

Using Contrastive Learning to Improve Two-Way Reasoning in Large Language Models: The Obfuscation Task as a Case Study

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that Contrastive Fine-Tuning, trained only on forward obfuscation and equivalence classification, lets code models reverse the transformation without reverse examples—deobfuscation rises from 0% to 39–52% on variable…

desk verdict A promising idea with a measurement flaw: the 52% reverse 'success' may measure valid renaming, not recovered inverse mapping. read the letter →

arxiv 2509.05553 v1 pith:EGWDYNH3 submitted 2025-09-06 cs.CL cs.AI

Serge Lionel Nikiema , Jordan Samhi , Micheline Bénédicte Moumoula , Albérick Euraste Djiré , Abdoul Kader Kaboré , Jacques Klein , Tegawendé F. Bissyandé This is my paper · ORCID
classification cs.CLcs.AI
keywords bidirectionalreasoningcontrastivefine-tuningcodeobfuscationdeobfuscationsemanticunderstandinglargelanguagemodelscognitivespecializationtransformation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that genuine understanding of a code transformation should be reversible: a model that learns to obfuscate should be able to deobfuscate without ever seeing reverse examples. It documents a failure mode it calls cognitive specialization, in which standard fine-tuning on forward obfuscation improves forward performance but leaves reverse deobfuscation at roughly 0%, across all tested models and prompting strategies. The proposed remedy, Contrastive Fine-Tuning (CFT), trains on three objectives at once: recognizing when original and obfuscated code are semantically equivalent, recognizing when code has different meaning, and generating obfuscated code in the forward direction. On variable renaming, CFT raises reverse success from 0% to 39–52% on GPT-3.5-Turbo, GPT-4.1-Mini, and QwenCoder while preserving forward quality; dead code insertion and string encryption do not show the same emergence. If the claim is right, bidirectional reasoning is both a practical training design and a behavioral test that separates semantic understanding from surface pattern matching.

What carries the argument

The load-bearing mechanism is Contrastive Fine-Tuning (CFT), a combined loss $\mathcal{L}_{\mathrm{CFT}} = \mathcal{L}_{\mathrm{pos}} + \mathcal{L}_{\mathrm{neg}} + \mathcal{L}_{\mathrm{gen}}$ over balanced triplets: positive pairs (an original program and its obfuscated version, labeled semantically equivalent), negative pairs (programs with different functionality), and forward obfuscation generation examples. The classification tasks force the model to represent what a transformation preserves and what it changes, while the generation task preserves forward skill; the paper's central claim is that this joint pressure lets the inverse operation $T^{-1}$ emerge without any reverse examples. The Bidirectional Reasoning Hypothesis supplies the evaluation criterion: for a semantics-preserving transformation $T$, genuine understanding requires success on both $T$ and $T^{-1}$, and the paper operationalizes that symmetry as the test that separates understanding from pattern matching.

What would settle it

Train a CFT variant with the forward generation and negative-pair losses unchanged but replace the positive-equivalence task with pairs of unrelated but semantically equivalent programs, so no original–obfuscated mapping is shown; if reverse deobfuscation on the held-out evaluation set stays near 39–52%, pairing was not the mechanism, and if it falls toward 0%, the paper's 'without explicit reverse training' claim is refuted.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the directional bias created by fine-tuning is not fixed. Standard fine-tuning on obfuscation produces a sharply asymmetric model: forward transformation success exceeds 85% while reverse deobfuscation success is effectively zero, and advanced prompting—few-shot, chain-of-thought, or both—does not recover the inverse operation. Training the same models with Contrastive Fine-Tuning (CFT) changes this: with only forward obfuscation examples plus positive and negative semantic-equivalence classification, reverse deobfuscation emerges at 39–52% for variable renaming in the GPT models and QwenCoder, with manual checks showing meaningful identifiers rather than copied obfuscated tokens. The paper reads this as evidence that contrastive supervision builds a semantic representation from which the inverse transformation can be derived, and that the standard fine-tuning objective instead collapses the model onto a unidirectional mapping.

Load-bearing premise

The load-bearing premise is that the positive-equivalence task, which Section 5.0.2 constructs from paired originals and their obfuscated versions, is not a form of reverse training; if that pairing counts as reverse exposure, the claimed 'emergence without explicit reverse training' collapses into a representation-alignment effect.

Editorial extensions

If this is right

  • Forward-only fine-tuning on code transformations should be re-evaluated for reverse performance, since high forward scores can hide the loss of reversible understanding.
  • CFT's jump from 0% to 39–52% reverse success on variable renaming provides a concrete baseline for any future training method that claims bidirectional reasoning.
  • The absence of the effect on dead code insertion and string encryption implies that transformation complexity, not just exposure to pairs, gates whether reverse ability emerges.
  • The capacity-dependent pattern—GPT models and QwenCoder succeed while other 7B–15B open models do not—implies that the contrastive objective alone is not sufficient and that model scale or pretraining matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same triplet recipe should be tested on other invertible code transformations, such as formatting changes, comment removal, or refactoring renames, which have cleaner ground truth and would show whether the effect generalizes beyond obfuscation.
  • Editorial inference: the decisive control for the 'without explicit reverse training' claim is to train on the forward generation loss without the positive-equivalence head; if reverse success then collapses toward 0%, the original–obfuscated pairs in the positive task were themselves the reverse training signal.
  • Editorial inference: the paper's negative results predict that harder transformations will require stronger contrastive signals, such as hard negatives or an explicit cycle-consistency loss, before reverse ability appears; this is directly testable with the same 300-sample evaluation set.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces the Bidirectional Reasoning Hypothesis (BRH), which defines genuine understanding of a code transformation as the ability to invert it without explicit reverse training. The authors evaluate seven LLMs on three obfuscation transformations (variable renaming, dead code insertion, string encryption), report that standard fine-tuning (SFT) yields near-zero reverse deobfuscation, and propose Contrastive Fine-Tuning (CFT), which combines positive equivalence classification, negative classification, and forward generation (Eq. 5). They report 39–52% reverse "semantic success" for variable renaming under CFT versus 0% for SFT, and claim this demonstrates emergent bidirectional reasoning. Dead code insertion and string encryption do not succeed in the CFT experiments.

Significance. If validated, the reversibility test would be a useful operational criterion for code understanding, and CFT would be a practical training method with broad relevance. The study has notable strengths: it covers multiple models and transformation types, uses real Java programs with test-suite verification, and reports consistent SFT reverse failure across models. However, the headline result rests on an undefined success metric and lacks a base-model baseline, so the significance is conditional on a careful re-analysis of the reverse evaluation and a more precise statement of what CFT actually learns.

major comments (4)
  1. [§4.3.2, §5.0.3] The reverse "semantic success" metric is never defined for CFT. §4.3.2 defines success via dissimilarity S(C_deobf, C_obf)→0 and readability R(C_deobf)→R(C_orig), but for variable renaming—a semantics-preserving transform—these criteria are satisfied by any readable, compiling, test-passing rename, regardless of whether the original identifiers or their semantic roles are recovered. §5.0.3 reports 39–52% "semantic success" without specifying the threshold or recovery criterion used. The finding therefore does not establish deobfuscation; it establishes only that CFT can produce semantically valid renamed code. Please report exact-match identifier recovery or a semantic-role recovery rate on a labeled subset as the primary reverse metric.
  2. [§1, §4.3.3] The claim that fine-tuning "actively degrades pre-existing bidirectional reasoning present in base models" is unsupported because no non-fine-tuned base model is evaluated on reverse deobfuscation. Experiment 3 tests only fine-tuned models; the observed P(T^{-1})≈0 could be the pre-existing ability of the base models rather than a degradation caused by SFT. Add a base-model reverse baseline for each transformation and prompting strategy to support the cognitive-specialization characterization.
  3. [§5.0.2] The claim that CFT enables reverse capability "without explicit reverse training" is undermined by the training data construction. Positive classification pairs are (original, obfuscated) code pairs for the equivalence task L_pos, so the model is exposed to both directions of the mapping during training. This paired exposure is a form of supervision on the reverse direction, even if it is not a generation loss. The manuscript should either justify why classification exposure does not constitute reverse training or provide an ablation that removes positive pairs or uses only forward generation plus negative classification.
  4. [Abstract, §5.0.3, §7] The conclusions generalize from a single successful transformation. CFT achieves 39–52% reverse performance only for variable renaming; §5.0.3 states that dead code insertion and literals encryption fail. The abstract and conclusion nevertheless claim that CFT "successfully achieved bidirectional reasoning" and enables "strong reverse performance" without restricting the claim to variable renaming. Restrict the headline claims to the successful setting, or provide a substantive account of why the other transformations fail and what that implies for the general framework.
minor comments (6)
  1. [§1, §3.1] The abstract and §1 say six models, but §3.1 lists seven models (five open-source plus two API models); clarify the discrepancy, and state which models are included in which experiments.
  2. [§4.2.2] The experimental setup refers to "GPT-3.5-Mini", but elsewhere the model is called GPT-3.5-Turbo; please confirm the intended model name and use it consistently.
  3. [§5.0.3] The text says the CFT evaluation uses "identical Experiment 4 protocols", but the previous experiment is labeled Experiment 3; renumber the experiments consistently.
  4. [§3.2.1, References] Reference [13] is cited for OpenAI's 50-to-1000 sample quality improvement, but [13] is the LoRA paper (Hu et al.); the reference list and in-text citations need to be corrected to match the claims about fine-tuning data sizes.
  5. [§4.1.3, Figure 3] The baseline CodeBLEU values between original and tool-obfuscated code are mentioned in the text (e.g., S(C_obf, C_orig)≈0.22, 0.33) but are not given consistently in a table or figure legend; please add these baselines so the dissimilarity thresholds are interpretable.
  6. [General] The manuscript provides no statement about code or data availability; please add a reproducibility section detailing prompts, hyperparameters, and evaluation scripts.

Circularity Check

2 steps flagged · score 6.0 of 10

The claimed 'reverse without reverse training' reduces by construction: CFT's positive pairs include the original code for each obfuscated input, and the reverse success metric counts any readable semantics-preserving rename.

  1. self definitional [Section 1 (CFT training triplets) and Section 5.0.2, Eq. (5), Training Data Construction]
    "we fine-tuned models with triplets consisting of: ① non-obfuscated–obfuscated code pairs with the same semantics, ② non-obfuscated–non-obfuscated code pairs with different semantics, and ③ obfuscated code examples for forward transformation learning. This contrastive approach forces the development of principled understanding rather than superficial pattern matching, enabling reverse capabilities to emerge naturally without explicit reverse training."

    In the reverse test the model is asked to map obfuscated code to a semantically equivalent original-like program. The positive classification pairs in CFT are precisely (original, obfuscated) pairs of the same program, so the original side of every reverse mapping is present in the training set. The only missing piece is a deobfuscation generation loss; calling the result 'without explicit reverse training' or 'without needing reverse examples' (Section 6.2) is a definitional dodge. The observed 39-52% reverse performance is therefore not an emergent property of forward obfuscation training alone; it is made possible by the reverse-direction pairs included by construction in L_pos.

  2. self definitional [Section 4.3.2 Success Criterion; Section 2.2 Definition 1]
    "Effective deobfuscation requires reducing syntactic similarity of the obfuscated code (S(C deobf, C obf) -> 0) while restoring readability to get close to the original code (R(Cdeobf) -> R(C orig))."

    For variable renaming, the forward transformation already preserves semantics and the test suite passes for every consistent renaming. The success criterion demands only (i) dissimilarity from the obfuscated input and (ii) readability near the original; it never requires recovering the original identifiers or their semantic roles, and Definition 1 explicitly weakens the reverse target to any semantically equivalent version. Hence the 52.03% 'semantic success' is satisfied by any readable, semantics-preserving rename, so the reported reverse performance is by construction a measure of generic renaming/readability rather than of inferring that 'i' represents a user index. The headline claim of bidirectional understanding is built into the evaluation criterion rather than demonstrated by it.

full rationale

The central derivation chain is: forward-only fine-tuning fails reverse; CFT adds positive/negative/forward losses; reverse emerges. The first circularity is at the data-construction level: the positive loss L_pos is trained on (original, obfuscated) pairs, so the original side of the reverse mapping is in the training data. The paper's 'without explicit reverse training' is therefore only true if 'training' is restricted to a deobfuscation generation objective; it is false for reverse examples in general. This makes the 39-52% reverse numbers a partial product of the input construction, not an emergent surprise. The second, related issue is that the reverse success criterion is defined as low syntactic similarity to the obfuscated code plus readability approaching the original, which for semantics-preserving variable renaming is met by any consistent readable renaming; it does not measure recovery of the original identifiers or semantic roles promised in the abstract. That is a definitional/tautological element: the positive-pair exposure and the lax criterion together nearly dictate the reported gap between SFT (0%, which just copies input) and CFT (52%, which has learned from the original side that renaming is allowed). I do not see load-bearing self-citation circularity: reference [19] is a same-author preprint but only motivates the problem; no uniqueness theorem or ansatz is imported from the authors' prior work. The result is not fully circular because the contrastive classification objective is not the same loss as deobfuscation generation, and a model could in principle satisfy the equivalence task without producing readable renames; but the reverse direction is present in the input and the metric is too weak. Score 6.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central result rests on domain assumptions about obfuscation fidelity, reversibility as an understanding criterion, and the validity of CodeBLEU/readability as proxies. The only hand-chosen numeric criterion is the CodeBLEU success threshold; no sensitivity analysis is given. No new entities are introduced.

free parameters (1)
  • Deobfuscation success threshold = CodeBLEU S < 0.4
    Hand-chosen criterion in Section 4.3.2; reported reverse success rates (0% vs 39-52%) depend on this threshold and no sensitivity analysis is given.
assumptions (4)
  • domain assumption Obfuscation transformations preserve program semantics.
    Section 3.2.2 assumes the Obfuscator tool maintains runtime behavior; no per-sample functional verification is reported.
  • domain assumption True understanding implies reversibility of a transformation.
    Section 2.1-2.3 postulates the Bidirectional Reasoning Hypothesis as a criterion for understanding; it is not proven or independently motivated.
  • domain assumption CodeBLEU and readability scores are valid proxies for semantic equivalence and understanding.
    Section 3.3 adopts these metrics without task-specific validation; success and failure classifications rely on them.
  • domain assumption The ConDefects evaluation set of 300 samples is representative and leak-free relative to CodeNet training data.
    Section 3.2.3 selects the eval set from a different dataset; no distributional analysis or leakage check is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Using Contrastive Learning to Improve Two-Way Reasoning in Large Language Models: The Obfuscation Task as a Case Study." pith.science (2026). https://pith.science/paper/EGWDYNH3

@misc{pith2026250905553,
  author       = {Pith},
  title        = {Pith review of: Using Contrastive Learning to Improve Two-Way Reasoning in Large Language Models: The Obfuscation Task as a Case Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGWDYNH3}},
  note         = {Machine review of arXiv:2509.05553}
}
read the original abstract

This research addresses a fundamental question in AI: whether large language models truly understand concepts or simply recognize patterns. The authors propose bidirectional reasoning,the ability to apply transformations in both directions without being explicitly trained on the reverse direction, as a test for genuine understanding. They argue that true comprehension should naturally allow reversibility. For example, a model that can change a variable name like userIndex to i should also be able to infer that i represents a user index without reverse training. The researchers tested current language models and discovered what they term cognitive specialization: when models are fine-tuned on forward tasks, their performance on those tasks improves, but their ability to reason bidirectionally becomes significantly worse. To address this issue, they developed Contrastive Fine-Tuning (CFT), which trains models using three types of examples: positive examples that maintain semantic meaning, negative examples with different semantics, and forward-direction obfuscation examples. This approach aims to develop deeper understanding rather than surface-level pattern recognition and allows reverse capabilities to develop naturally without explicit reverse training. Their experiments demonstrated that CFT successfully achieved bidirectional reasoning, enabling strong reverse performance while maintaining forward task capabilities. The authors conclude that bidirectional reasoning serves both as a theoretical framework for assessing genuine understanding and as a practical training approach for developing more capable AI systems.

Figures

Figures reproduced from arXiv: 2509.05553 by the authors.

Figure 1
Figure 1. Fine-tuned model performance across code obfus [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Reversibility Evaluation. Baseline represent the Score between Original code and Tool obfuscated code [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Comparison of single-task fine-tuning and Con [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 15 canonical work pages

  1. [1]

    Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Ilya Sutskever. Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    arXiv 2021

  2. [2]

    Repairagent: An autonomous, llm-based agent for program repair.arXiv preprint arXiv:2403.17134, 2024

    Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. Repairagent: An autonomous, llm-based agent for program repair.arXiv preprint arXiv:2403.17134, 2024

    arXiv 2024

  3. [3]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brock- man, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    arXiv 2021

  4. [4]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. InInternational Conference on Machine Learning, pages 1597–1607. PMLR, 2020

    2020

  5. [5]

    Addison-Wesley Profes- sional, Boston, MA, 2009

    Christian Collberg and Jasvir Nagra.Surreptitious Software: Obfuscation, Water- marking, and Tamperproofing for Software Protection. Addison-Wesley Profes- sional, Boston, MA, 2009

    work page 2009

  6. [6]

    Qlora: Ef- ficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Ef- ficient finetuning of quantized llms. InAdvances in Neural Information Processing Systems, volume 36, 2023

    work page 2023

  7. [7]

    Towards translat- ing real-world code with llms: A study of translating to rust.arXiv preprint arXiv:2405.11514v3, 2025

    Hasan Ferit Eniser, Haonan Zhang, Cristian David, Meng Wang, Maria Chris- takis, Brandon Paulsen, Josiah Dodds, and Daniel Kroening. Towards translat- ing real-world code with llms: A study of translating to rust.arXiv preprint arXiv:2405.11514v3, 2025

    arXiv 2025

  8. [8]

    Catastrophic forgetting in connectionist networks.Trends in Cognitive Sciences, 3(4):128–135, 1999

    Robert M French. Catastrophic forgetting in connectionist networks.Trends in Cognitive Sciences, 3(4):128–135, 1999

    1999

Show all 28 references
  1. [9]

    An empirical investigation of catastrophic forgetting in gradient-based neural networks

    Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshiv Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. InarXiv preprint arXiv:1312.6211, 2013

    2013 arXiv

  2. [10]

    Dimensionality reduction by learning an invariant mapping

    Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), volume 2, pages 1735–1742. IEEE, 2006

    2006

  3. [11]

    Momentum contrast for unsupervised visual representation learning.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729– 9738, 2020

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729– 9738, 2020

    2020

  4. [12]

    Unpaired image-to- image translation using cycle-consistent adversarial networks.Proceedings of the IEEE International Conference on Computer Vision, pages 2223–2232, 2017

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to- image translation using cycle-consistent adversarial networks.Proceedings of the IEEE International Conference on Computer Vision, pages 2223–2232, 2017

    2017

  5. [13]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  6. [14]

    Husein, Hasan Aburajouh, and Cagatay Catal

    Raed A. Husein, Hasan Aburajouh, and Cagatay Catal. Large language models for code completion: A systematic literature review.Computer Standards & Interfaces, 92:103917, 2024

    2024

  7. [15]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Scie...

    2017

  8. [16]

    Condefects: A large-scale dataset of real-world concurrency bugs and fixes

    Zhen Li, Yuxin Su, Zibin Zheng, Yifan Wu, Yao Wan, and Michael Lyu. Condefects: A large-scale dataset of real-world concurrency bugs and fixes. InProceedings of the 2023 ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering...

    2023

  9. [17]

    Catastrophic interference in connectionist networks: The sequential learning problem.Psychology of learning and motivation, 24:109–165, 1989

    Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem.Psychology of learning and motivation, 24:109–165, 1989

    1989

  10. [18]

    Codegen: An open large language model for code with multi-turn program synthesis

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. InProceedings of the Eleventh International Conference on Learning Representations (IC...

    2023

  11. [19]

    The code barrier: What llms actually understand? arXiv preprint arXiv:2504.10557, 2025

    Serge Lionel Nikiema, Jordan Samhi, Abdoul Kader Kabore, Jacques Klein, and Tegawendé François Bissyandé. The code barrier: What llms actually understand? arXiv preprint arXiv:2504.10557, 2025

    2025 arXiv

  12. [20]

    Wassi, Michele Merler, Boris Sobolev, Ruchir Pavuluri, Saurabh Sinha, and Reyhaneh Jabbarvand

    Rui Pan, Ali Reza Ibrahimzada, Rahul Krishna, Divya Sankar, Laith P. Wassi, Michele Merler, Boris Sobolev, Ruchir Pavuluri, Saurabh Sinha, and Reyhaneh Jabbarvand. Lost in translation: A study of bugs introduced by large language models while translating code. InProceedings of...

    2024

  13. [21]

    Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks.arXiv preprint arXiv:2105.12655, 2021

    Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Moninder Choudhury, Lindsey Decker, et al. Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks.arXiv preprint arXiv:2105.12655, 2021

    2021 arXiv

  14. [22]

    Codebleu: a method for automatic evaluation of code synthesis.arXiv preprint arXiv:2009.10297, 2020

    Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sun- daresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. Codebleu: a method for automatic evaluation of code synthesis.arXiv preprint arXiv:2009.10297, 2020

    2009 arXiv

  15. [23]

    A comprehensive model for code readability

    Simone Scalabrino, Mario Linares-Vásquez, Denys Poshyvanyk, and Rocco Oliveto. A comprehensive model for code readability. https://dibt.unimol. it/report/readability/, 2018. Accessed: August 1, 2025

    2018

  16. [24]

    Protecting software through obfuscation: Can it keep pace with progress in code analysis?ACM Computing Surveys, 49(1):1–37, 2016

    Sebastian Schrittwieser, Stefan Katzenbeisser, Johannes Kinder, Georg Merz- dovnik, and Edgar Weippl. Protecting software through obfuscation: Can it keep pace with progress in code analysis?ACM Computing Surveys, 49(1):1–37, 2016

    2016

  17. [25]

    Java obfuscator (gui)

    superblaubeere. Java obfuscator (gui). https://github.com/superblaubeere27/ob fuscator, 2018. Accessed: August 1, 2025

    2018

  18. [26]

    Generating adversarial examples for holding robustness of source code processing models

    Huangzhao Zhang, Zhuo Li, Ge Li, Zhi Fei, Yang Liu, Qiang Luo, and ChengXiang Zhai. Generating adversarial examples for holding robustness of source code processing models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1169–1176, 2020

    2020

  19. [27]

    Devign: Effective vulnerability identification by learning comprehensive program seman- tics via graph neural networks

    Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. Devign: Effective vulnerability identification by learning comprehensive program seman- tics via graph neural networks. InAdvances in Neural Information Processing Systems, pages 10197–10207, 2019

    2019

  20. [28]

    The use of large lan- guage models for program repair.Computer Standards & Interfaces, 93:103951, 2024

    Faizan Zubair, Mohammed Al-Hitmi, and Cagatay Catal. The use of large lan- guage models for program repair.Computer Standards & Interfaces, 93:103951, 2024. 8

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.