Pith. sign in

REVIEW 4 major objections 5 minor 73 references

Detecting Behavioral Changes in Python Refactoring Implementations with Foundation Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A foundation-model oracle that reads only the diff of a successfully applied Python refactoring detects behavioral changes with 0.91 recall, and the process uncovered 13 Rope bugs, 12 accepted by maintainers.

desk verdict Useful bug-finding study with real Rope bugs and honest limitations, but the oracle metrics hinge on an unstated prompt-overlap assumption that needs resolution before trusting the 0.84 accuracy figure. read the letter →

arxiv 2608.09919 v1 pith:XY7NME5J submitted 2026-08-10 cs.SE

Jonhnanthan Oliveira , Rohit Gheyi , Márcio Ribeiro , Alessandro Garcia This is my paper · ORCID
classification cs.SE
keywords refactoringbehavioralchangefoundationmodelsPythonlargelanguagediff-basedoracleRopebugdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Python refactorings can silently change a program's observable behavior even when the tool reports success. The paper proposes a validation layer that hands the git-style diff of a completed refactoring to a foundation-model oracle, which judges whether the edit preserves behavior without seeing the rest of the project. On 217 transformations produced by the Rope tool on the TextBlob codebase, the oracle achieved recall 0.91, precision 0.67, and accuracy 0.84 against a human expert's diff-scoped labels. The analysis surfaced 13 distinct Rope bugs across seven refactoring types, and 12 were accepted by the tool's maintainers. The claim is that a lightweight, tool-agnostic diff oracle can serve as a practical bug-finding aid for refactoring validation.

What carries the argument

The carrying mechanism is a diff-scoped foundation-model oracle: a zero-shot prompt that receives only a git-style unified diff of a refactoring and must answer YES or NO on whether observable behavior (outputs, exceptions, visible state, timing) is preserved. The prompt is refined through meta-prompting with a critic model, and answers are aggregated across repeated runs, using the modal response as the final verdict. The pipeline, built on SafeRefactorPy, applies the refactoring, computes the diff, submits it to the model, and a categorizer groups recurring failures into bug reports with reduced reproducers; the diff input is what keeps the approach lightweight and portable across tools.

What would settle it

Re-run the labeling of the 217 diffs with three independent Python experts under the same diff-scoped protocol and compute inter-rater agreement. If the labels disagree substantially (e.g., Cohen's kappa below about 0.5) or the majority labels flip enough instances to move the oracle's recall below about 0.8 or the confirmed bug count below 13, the central claim lacks a reliable baseline.

Watch

Extended reading notes

Core claim

The central discovery is that a foundation model prompted to reason about observable behavior from a git-style diff alone can detect behavioral changes introduced by apparently successful Python refactorings with high recall (0.91) and good accuracy (0.84), at the cost of false positives (precision 0.67). The evidence is the 13 distinct bugs uncovered in Rope 1.3.0, spanning five refactoring types, of which 12 were accepted by developers according to issue-tracker evidence. The bugs cluster into Python-specific failure categories: incorrect name and import handling, expression and call-site rewriting errors, violations of Python data-model constraints such as dunder methods, inheritance and contract violations, and circular dependencies.

Load-bearing premise

The ground truth is one human expert's judgment of behavioral change from the diff alone; if that judgment is wrong or unrepresentative, every reported metric and the 13-bug count shift.

Editorial extensions

If this is right

  • A validation layer for Python refactoring tools is feasible without executing the program or seeing the whole project.
  • Tool builders should add preconditions for dunder methods and abstract methods, validate name and import resolution after transformations, and check circular dependencies in Move refactorings.
  • Repeated queries (k=3) raise the chance of a correct decision (pass@3 = 0.903) with stable accuracy, supporting majority-vote use in practice.
  • The false-positive rate implies that model judgments should trigger human review rather than automatic rejection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A project-aware baseline could reclassify many of the 133 GPT-5.2 'risk' disagreements, potentially raising the confirmed bug count beyond 13; the paper leaves that as future work.
  • The same diff-only oracle protocol could be applied to other dynamically typed languages or to edits produced by AI coding agents, since the pipeline only needs a diff generator.
  • The high-recall, moderate-precision trade-off suggests the oracle is best used as a triage step: minimize missed bugs first, then spend human effort filtering false alarms.
  • Because the evaluation uses a single project and Inline Method dominates the behavioral-change instances, aggregate metrics may not transfer to other refactoring-type mixes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SafeRefactorPy, a foundation-model-based oracle that takes a git-style diff of a successful Python refactoring and classifies whether the transformation preserves observable behavior (outputs, exceptions, visible state, timing/synchronization). The oracle is evaluated on 217 transformation pairs generated by Rope 1.3.0 applied to the TextBlob project, with a single human expert providing ground-truth labels for the same diffs. The main evaluation uses gpt-oss-20b with k=3 repeated runs and reports precision 0.67, recall 0.91, accuracy 0.84, and F1 0.77. From the instances judged behavior-changing, the authors consolidate 13 distinct Rope bugs, submit them as GitHub issues, and count 12 as accepted by maintainers. A complementary, exploratory analysis with GPT-5.2 Thinking is also reported separately.

Significance. If the results hold, the paper demonstrates a lightweight, tool-agnostic validation layer for Python refactoring: the oracle consumes only the diff, not the full project, and can be retargeted to a new refactoring engine by changing only the instance-application step. The bug-finding outcome is externally validated by maintainer acceptance of 12 of 13 reported issues, which is a strong practical signal. The paper is also commendably explicit about several threats: the single-expert baseline, the diff-scoped nature of the oracle, the single-project evaluation, and the exploratory status of the GPT-5.2 comparison. The artifact is promised online, which would support reproducibility of the reported pipeline.

major comments (4)
  1. [§3.3 and §4.2.1] The prompt-refinement procedure and the evaluation data may overlap. Section 3.3 states that the prompt was iteratively refined on 'a set of representative diffs and their expected ground-truth labels' until correct on all representative cases, but Sections 4.2 and 4.3 never state that this representative set is disjoint from the 217 evaluation instances, nor that the expected labels were produced independently of the single expert of §4.2.1. If any representative diff or label was drawn from the evaluation set, the RQ1 metrics in §4.3.1 are not zero-shot generalization figures but test-set fitting. This concern is not addressed in Section 5. The authors should report the size and provenance of the representative set, explicitly confirm disjointness from the 217 pairs, and, if overlap exists, re-evaluate on a held-out subset or use a nested/leave-one-out protocol.
  2. [§4.2.1] The ground truth for all classification metrics rests on the judgment of a single human expert. Section 4.2.1 says 'A Python developer with ten years of professional experience acted as the sole judge,' and Section 5 acknowledges that future replications should use multiple evaluators and inter-rater agreement. This is load-bearing because the reported precision, recall, accuracy, and F1 are all computed against these labels, and the authors also consolidate the 13 bugs from the same expert's positive labels. Since the paper's central claim is an accuracy/recall level for the oracle, a single rater is insufficient to establish that level. The authors should at least provide the expert's written justifications and invite independent relabeling of a sample, or report agreement between multiple raters on a subset, before the metric values can be taken at face value.
  3. [§4.3.1, Table 3, and §4.4.5] The reporting of accuracy is internally inconsistent. RQ1 reports a modal-response accuracy of 0.84, while Table 3 lists Accuracy values of 0.793, 0.795, and 0.810 for k=1,2,3, and §4.4.5 then says cons@3 = 0.839 'corresponds to the modal-response accuracy reported in RQ1.' These three sets of numbers are not reconciled: the RQ1 value differs from the k=3 accuracy row, and the text does not define whether the RQ1 precision/recall/accuracy were computed on the modal response, the mean response, or some other aggregation. The authors should replace Table 3 with a clear definition of how each row is computed, and state which aggregation underlies each reported metric in §4.3.1.
  4. [§4.2.2 and §4.3.2] The aggregate results are dominated by a single refactoring type. Among 64 expert-labeled behavioral changes, 52 are Inline Method instances, and the paper provides no per-refactoring precision/recall/accuracy breakdown. Because the oracle's usefulness is claimed for 'Python refactoring implementations' generally, the authors should report metrics separately for each refactoring type, or at least give the confusion matrix per type. This is particularly important because the 13 bugs span only five of the seven studied types and the per-type sample sizes are small; the current aggregate numbers may not represent the harder or rarer types.
minor comments (5)
  1. [§7] The sentence 'Twelve of the 13 reported bugs reported bugs were accepted by developers' contains a duplicated phrase 'reported bugs reported bugs' and should be corrected.
  2. [§4.2.2 and Abstract] The paper refers to 'seven refactoring types studied' but Section 4.2.2 lists six items, one of which is 'Move Field/Method.' Please clarify whether Move Field and Move Method are counted separately and align the count in the abstract and Section 7.
  3. [§4.4.6] The GPT-5.2 Thinking analysis is explicitly exploratory and not a controlled comparison, which is appropriate; however, the phrase 'every instance labeled as a behavioral change by the expert was also flagged by the model' should be accompanied by the actual count (64) in the same paragraph to avoid an apparent inconsistency with the 20 YES/197 NO breakdown.
  4. [§3.3] The paper calls the approach 'zero-shot prompting' while also describing an iterative meta-prompting refinement on representative diffs. These statements can be reconciled, but the term should be qualified, e.g., by saying the final prompt is used in zero-shot inference, so that readers do not infer that no prompt engineering occurred.
  5. [Table 3] The caption lists 'Accuracy' as a row, but the definitions in Table 1 define 'Mean Accuracy' as a metric; the row should state whether the reported values are the mean accuracy over k attempts or something else. The same applies to the mention of 'mean accuracy of 0.799' in §4.4.5.

Circularity Check

0 steps flagged · score 0.0 of 10

No demonstrated circularity: the oracle is an empirical classifier, its bug findings are independently validated by maintainers, and no claim reduces by construction to a fitted input or self-citation.

full rationale

This paper contains no mathematical derivation chain that could collapse into its inputs; the central claims are empirical measurements of a foundation-model oracle against a human-expert baseline. Using the same behavioral-preservation definition in both the prompt and the expert labeling instructions (Sections 3.3 and 4.2.1) is task specification, not circularity: the model can still be wrong relative to that rubric, and the paper reports false positives and false negatives. The paper explicitly acknowledges the single-expert baseline and diff-scoped protocol as threats to validity in Section 5, and the bug-finding claim is independently supported by maintainer acceptance of 12 of 13 reported issues (Section 4.3.3), which breaks any self-validating loop. The reuse of 217 program pairs from the authors' prior study [33] is a dataset-source choice, not an appeal to an unverified theorem, and no uniqueness result is imported. The only residual concern is that Section 3.3 describes prompt refinement on 'representative diffs and their expected ground-truth labels' without stating that those diffs are disjoint from the 217 evaluation pairs; however, the paper also does not state that they overlap, so no specific reduction can be exhibited under the hard rule requiring quoted evidence of overlap. That is a potential external-validity threat, not a demonstrated circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numerical free parameters are fitted to data; the modal-response aggregation (k=3) and the manual bug-grouping rule are methodological choices, not fitted values. The paper introduces no new physical or theoretical entities; the oracle is a tool composition, not a postulated object.

assumptions (4)
  • domain assumption Behavioral preservation is defined by observable behavior: outputs, exceptions, externally visible state, timing or synchronization behavior.
    Used in Section 3.3 to instruct both the oracle and the human expert baseline.
  • domain assumption Code not shown in the diff is unchanged.
    Prompt assumption in Section 3.3; limits the oracle to diff-scoped reasoning and may miss project-wide effects.
  • domain assumption A single human expert's labels are the correct ground truth for behavioral equivalence.
    Section 4.2.1; all classification metrics are computed against these labels.
  • domain assumption TextBlob 0.17.1 is a representative Python project for this oracle evaluation.
    Section 4.2.1; the study uses one project, limiting generalizability, as acknowledged in Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Detecting Behavioral Changes in Python Refactoring Implementations with Foundation Models." pith.science (2026). https://pith.science/paper/XY7NME5J

@misc{pith2026260809919,
  author       = {Pith},
  title        = {Pith review of: Detecting Behavioral Changes in Python Refactoring Implementations with Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XY7NME5J}},
  note         = {Machine review of arXiv:2608.09919}
}
read the original abstract

Python is a widely adopted programming language, valued for its simplicity and flexibility. However, automated refactoring for Python remains challenging, even though refactoring is an essential practice in software evolution aimed at improving internal code structure without changing external behavior. Understanding how behavioral changes are introduced during refactoring is crucial, as such issues can compromise software reliability and reduce developer productivity. We propose an approach based on a foundation model oracle that analyzes git-style diffs to identify behavioral changes introduced by Python refactorings. We evaluated our technique on Rope refactoring implementations, reusing 1,152 refactoring attempts from a prior study and analyzing 217 resulting transformation pairs with the oracle. Our model-based analysis uncovered 13 distinct bugs among the seven refactoring types studied. All reported bugs were submitted to the respective developers, and 12 of the 13 resulting issue reports were accepted according to issue-tracker evidence. These results highlight the need to improve the robustness of current Python refactoring tools to ensure the correctness of automated code transformations and support reliable software maintenance.

Figures

Figures reproduced from arXiv: 2608.09919 by the authors.

Figure 1
Figure 1. SafeRefactorPy: a model-based pipeline for detecting behavioral changes introduced by Python refactorings. required parameters (Step 1). If the transformation succeeds, it com￾putes a git-style diff between the original and refactored programs (Step 2) and submits that diff to a foundation-model oracle that de￾cides whether the instance preserves observable behavior (Step 3). After all instances have been analyzed, … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 62 canonical work pages

  1. [1]

    LangChain AI. 2025. OllamaLLM. (2025). Available at: https://api.python. langchain.com/en/latest/ollama_llms/langchain_ollama.llms.OllamaLLM.html

    work page 2025

  2. [2]

    Eman Abdullah AlOmar, Mohamed Wiem Mkaouer, Christian Newman, and Ali Ouni. 2021. On preserving the behavior in software refactoring: A systematic mapping study.Information and Software Technology140 (2021), 106675

    work page 2021

  3. [3]

    Deterministic

    Berk Atil, Sarp Aykent, Alexa Chittams, Lisheng Fu, Rebecca J. Passonneau, Evan Radcliffe, Guru Rajan Rajagopal, Adam Sloan, Tomasz Tudrej, Ferhan Ture, Zhe Wu, Lixinyu Xu, and Breck Baldwin. 2025. Non-Determinism of "Deterministic" LLM Settings. arXiv:2408.04667 [cs.CL] https://arxiv.org/abs/2408.04667

    arXiv 2025

  4. [4]

    Basili, Gianluigi Caldiera, and H

    Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. 1994.The Goal Question Metric Approach. Wiley, New York, NY, USA. 528–532 pages

    work page 1994

  5. [5]

    Diego Cedrim, Alessandro Garcia, Melina Mongiovi, Rohit Gheyi, Leonardo Sousa, Rafael de Mello, Baldoino Fonseca, Márcio Ribeiro, and Alexander Chávez

    work page

  6. [6]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, Henrique Ponde de Oliveira Pinto Ponde, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, D. Such, D. Cummings, M. Plappert, F. Chantzis, E. Bar...

    arXiv 2021

  7. [7]

    Gonzalez, and Ion Stoica

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference. arXiv:2403.04132 [cs.AI]

    arXiv 2024

  8. [8]

    Rope contributors. 2023. Rope - open source Python refactoring library. https: //github.com/python-rope/rope

    work page 2023

Show all 73 references
  1. [9]

    DAIR.AI. 2024. Prompt Engineering Guide. https://www.promptingguide.ai/ techniques

    2024

  2. [10]

    Brett Daniel, Danny Dig, Kely Garcia, and Darko Marinov. 2007. Automated Testing of Refactoring Engines. InProceedings of the Foundations of Software Engineering (FSE’07). Association for Computing Machinery, New York, NY, USA, 185–194

    2007

  3. [11]

    Dilhara, A

    M. Dilhara, A. Bellur, T. Bryksin, and D. Dig. 2024. Unprecedented code change automation: The fusion of LLMs and transformation by example. InProceedings of the ACM on Software Engineering (FSE’24). Association for Computing Machinery, New York, NY, USA, 631–653

    2024

  4. [12]

    Malinda Dilhara, Ameya Ketkar, Nikhith Sannidhi, and Danny Dig. 2022. Dis- covering Repetitive Code Changes in Python ML Systems. InProceedings of the 44th International Conference on Software Engineering (ICSE’22). Association for Computing Machinery, New York, NY, USA, 736–748

    2022

  5. [13]

    C. Dong, Y. Jiang, Y. Zhang, Y. Zhang, and H. Liu. 2025. ChatGPT-based test generation for refactoring engines enhanced by feature analysis on examples. InInternational Conference on Software Engineering (ICSE’25). IEEE Computer Society/ACM, Washington, DC, USA, 2714–2725

    2025

  6. [14]

    1999.Refactoring: improving the design of existing code

    Martin Fowler. 1999.Refactoring: improving the design of existing code. Addison- Wesley, Boston, MA, USA

    1999

  7. [15]

    Rohit Gheyi, Rian Melo, Jonhnanthan Oliveira, Marcio Ribeiro, and Baldoino Fon- seca. 2026. Foundation Models as Oracles for Refactoring Correctness Detection. arXiv:2605.02096 [cs.SE] https://arxiv.org/abs/2605.02096

    2026 arXiv

  8. [16]

    Rohit Gheyi, Marcio Ribeiro, and Jonhnanthan Oliveira. 2025. Evaluating the Effectiveness of Small Language Models in Detecting Refactoring Bugs. arXiv:2502.18454 [cs.SE] https://arxiv.org/abs/2502.18454

    2025 arXiv

  9. [17]

    Milos Gligoric, Farnaz Behrang, Yilong Li, Jeffrey Overbey, Munawar Hafiz, and Darko Marinov. 2013. Systematic Testing of Refactoring Engines on Real Software Projects. InProceedings of the European Conference on Object-Oriented Programming (ECOOP’13). Springer Berlin Heidelbe...

    2013

  10. [18]

    Milos Gligoric, Tihomir Gvero, Vilas Jagannath, Sarfraz Khurshid, Viktor Kuncak, and Darko Marinov. 2010. Test generation through programming in UDITA. In Proceedings of International Conference on Software Engineering (ICSE). Associa- tion for Computing Machinery, New York, N...

    2010

  11. [19]

    Yaroslav Golubev, Zarina Kurbatova, Eman Abdullah AlOmar, Timofey Bryksin, and Mohamed Wiem Mkaouer. 2021. One thousand and one stories: a large- scale survey of software refactoring. InFoundations of Software Engineering. Association for Computing Machinery, New York, NY, USA...

    2021

  12. [20]

    Koki Horikawa, Yoshiki Higo, and Shinji Kusumoto. 2025. Agentic Refactoring: An Empirical Study of AI Coding Agents. arXiv:2511.04824 [cs.SE] https://arxiv. org/abs/2511.04824

    2025

  13. [21]

    Yutai Hou, Hongyuan Dong, Xinghao Wang, Bohan Li, and Wanxiang Che. 2023. MetaPrompting: Learning to Learn Better Prompts. arXiv:2209.11486 [cs.CL] https://arxiv.org/abs/2209.11486

    2023 arXiv

  14. [22]

    Miryung Kim, Thomas Zimmermann, and Nachiappan Nagappan. 2014. An Empirical Study of Refactoring Challenges and Benefits at Microsoft.IEEE Transactions on Software Engineering40, 7 (2014), 633–649

    2014

  15. [23]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2025. Prompt Repetition Improves Non-Reasoning LLMs. arXiv:2512.14982 [cs.LG] https://arxiv.org/abs/ 2512.14982

    2025

  16. [24]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.ACM Computing Surveys (CSUR)55, 9 (2023), 1–35

    2023

  17. [25]

    Tom Mens and Tom Tourwé. 2004. A Survey of Software Refactoring.IEEE Transactions on Software Engineering30, 2 (2004), 126–139

    2004

  18. [26]

    Alessandro Midolo, Emiliano Tramontana, and Massimiliano Di Penta. 2026. From Human to Machine Refactoring: Assessing GPT-4’s Impact on Python Class Quality and Readability. arXiv:2601.13139 [cs.SE] https://arxiv.org/abs/2601. 13139

    2026

  19. [27]

    Melina Mongiovi, Rohit Gheyi, Gustavo Soares, Márcio Ribeiro, Paulo Borba, and Leopoldo Teixeira. 2018. Detecting Overly Strong Preconditions in Refactoring Engines.IEEE Transactions on Software Engineering44, 5 (2018), 429–452

    2018

  20. [28]

    Melina Mongiovi, Rohit Gheyi, Gustavo Soares, Leopoldo Teixeira, and Paulo Borba. 2014. Making refactoring safer through impact analysis.Science of Computer Programming93 (2014), 39–64

    2014

  21. [29]

    Melina Mongiovi, Gustavo Mendes, Rohit Gheyi, Gustavo Soares, and Márcio Ribeiro. 2014. Scaling Testing of Refactoring Engines. InProceedings of the International Conference on Software Maintenance and Evolution (ICSME’14). IEEE Computer Society, Washington, DC, USA, 371–380

    2014

  22. [30]

    Emerson Murphy-Hill, Chris Parnin, and Andrew P. Black. 2012. How We Refactor, and How We Know It.IEEE Transactions on Software Engineering38, 1 (2012), 5–18

    2012

  23. [31]

    Daniel Oliveira, Wesley K. G. Assunção, Alessandro Garcia, Ana Carla Bibiano, Márcio Ribeiro, Rohit Gheyi, and Baldoino Fonseca. 2023. The untold story of code refactoring customizations in practice. InProceedings of the International Conference on Software Engineering. Associ...

    2023

  24. [32]

    Jonhnanthan Oliveira, Rohit Gheyi, Melina Mongiovi, Gustavo Soares, Márcio Ribeiro, and Alessandro Garcia. 2019. Revisiting the refactoring mechanics. Information and Software Technology110 (2019), 136–138

    2019

  25. [33]

    Jonhnanthan Oliveira, Rohit Gheyi, Márcio Ribeiro, and Alessandro Garcia. 2025. Bugs in the Shadows: Static Detection of Faulty Python Refactorings. InAnais do XXXIX Simpósio Brasileiro de Engenharia de Software (SBES’25). SBC, Porto Alegre, RS, Brasil, 182–192

    2025

  26. [34]

    Jonhnanthan Oliveira, Rohit Gheyi, Márcio Ribeiro, and Alessandro Garcia. 2026. Detecting Behavioral Changes in Python Refactoring Implementations with Foundation Models. https://github.com/jonh-copin/technique

    2026

  27. [35]

    Jonhnanthan Oliveira, Rohit Gheyi, Leopoldo Teixeira, Márcio Ribeiro, Osmar Leandro, and Baldoino Fonseca. 2023. Towards a better understanding of the mechanics of refactoring detection tools.Information and Software Technology 162, C (2023), 107273

    2023

  28. [36]

    1992.Refactoring Object-oriented Frameworks

    William Opdyke. 1992.Refactoring Object-oriented Frameworks. Ph. D. Disserta- tion. UIUC

    1992

  29. [37]

    William Opdyke and Ralph Johnson. 1990. Refactoring: An Aid in Designing Application Frameworks and Evolving Object-Oriented Systems. InProceedings of the Symposium Object-Oriented Programming Emphasizing Practical Applications (SOOPPA) (SOOPPA). Marist College, Poughkeepsie, ...

    1990

  30. [38]

    OpenAI. 2025. OpenAI gpt-oss-20b. (2025). Available at: https://platform.openai. com/docs/models/gpt-oss-20b

    2025

  31. [39]

    Yonnel Chen Kuang Piao, Jean Carlors Paul, Leuson Da Silva, Arghavan Moradi Dakhel, Mohammad Hamdaqa, and Foutse Khomh. 2025. Refactoring with LLMs: Bridging Human Expertise and Machine Understanding. arXiv:2510.03914 [cs.SE] https://arxiv.org/abs/2510.03914

    2025

  32. [40]

    Gustavo Pinto and Fernando Kamei. 2013. What programmers say about refac- toring tools? an empirical investigation of stack overflow. InProceedings of the 2013 ACM Workshop on Workshop on Refactoring Tools (WRT’13). Association for Computing Machinery, New York, NY, USA, 33–36

    2013

  33. [41]

    Dorin Pomian, Abhiram Bellur, Malinda Dilhara, Zarina Kurbatova, Egor Bogo- molov, Andrey Sokolov, Timofey Bryksin, and Danny Dig. 2024. EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs. InCompanion Proceedings of the 32nd ACM International Conference on the Found...

    2024

  34. [42]

    Felipe Pontes, Rohit Gheyi, Sabrina Souto, Alessandro Garcia, and Márcio Ribeiro

  35. [43]

    2019.Language Models are Unsupervised Multitask Learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.Language Models are Unsupervised Multitask Learners. Technical Report. OpenAI

    2019

  36. [44]

    Laria Reynolds and Kyle McDonell. 2021. Prompt Programming for Large Lan- guage Models: Beyond the Few-Shot Paradigm. InExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA ’21). ACM, New York, NY, USA, 1–7. doi:10.1145/3411763.3451760

    2021

  37. [45]

    1999.Practical Analysis for Refactoring

    Donald Roberts. 1999.Practical Analysis for Refactoring. Ph. D. Dissertation. University of Illinois at Urbana-Champaign

    1999

  38. [46]

    Van Rossum and F

    G. Van Rossum and F. L. Drake. 2011.An Introduction to Python. Network Theory Ltd, Bristol, UK

    2011

  39. [47]

    Max Schäfer. 2012. Refactoring Tools for Dynamic Languages. InProceedings of the Workshop on Refactoring Tools (WRT’12). Association for Computing Machinery, New York, NY, USA, 59–62

    2012

  40. [48]

    Max Schäfer and Oege de Moor. 2010. Specifying and implementing refac- torings. InProceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA’10). Association for Computing Machinery, New York, NY, USA, 286–301

    2010

  41. [49]

    Max Schäfer, Torbjörn Ekman, and Oege de Moor. 2008. Challenge Proposal: Verification of Refactorings. InProceedings of the 2008 ACM Workshop on Program- ming Languages Meets Program Verification (PLPV’08). Association for Computing Machinery, New York, NY, USA, 67–72

    2008

  42. [50]

    Max Schäfer, Torbjörn Ekman, and Oege de Moor. 2008. Sound and Extensible Renaming for Java. InProceedings of the 23rd ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA ’08). Association for Computing Machinery, New York, NY, US...

    2008

  43. [51]

    Max Schäfer, Mathieu Verbaere, Torbjörn Ekman, and Oege de Moor. 2009. Step- ping Stones over the Refactoring Rubicon. InProceedings of the 23rd European Conference on Object-Oriented Programming (ECOOP 2009) (Lecture Notes in Com- puter Science, Vol. 5653). Springer-Verlag, B...

    2009

  44. [52]

    Atsushi Shirafuji, Yusuke Oda, Jun Suzuki, Makoto Morishita, and Yutaka Watanobe. 2023. Refactoring Programs Using Large Language Models with Few-Shot Examples. InProceedings of the 2023 30th Asia-Pacific Software Engi- neering Conference (APSEC). IEEE Computer Society, Los Al...

    2023

  45. [53]

    Gustavo Soares, Rohit Gheyi, and Tiago Massoni. 2013. Automated Behavioral Testing of Refactoring Engines.IEEE Transactions on Software Engineering39, 2 (2013), 147–162

    2013

  46. [54]

    Gustavo Soares, Rohit Gheyi, Emerson Murphy-Hill, and Brittany Johnson. 2013. Comparing Approaches to Analyze Refactoring Activity on Software Repositories. Journal of Systems and Software86, 4 (2013), 1006–1022

    2013

  47. [55]

    Gustavo Soares, Rohit Gheyi, Dalton Serey, and Tiago Massoni. 2010. Making Program Refactoring Safer.IEEE Software27, 4 (2010), 52–57

    2010

  48. [56]

    Gustavo Soares, Melina Mongiovi, and Rohit Gheyi. 2011. Identifying overly strong conditions in refactoring implementations. InProceedings of the 2011 27th IEEE International Conference on Software Maintenance (ICSM’11). IEEE Computer Society, Los Alamitos, CA, USA, 173–182

    2011

  49. [57]

    Friedrich Steimann and Andreas Thies. 2009. From Public to Private to Absent: Refactoring Java Programs under Constrained Accessibility. InProceedings of the 23rd European Conference on Object-Oriented Programming (ECOOP 2009) (Lecture Notes in Computer Science, Vol. 5653). Sp...

    2009

  50. [58]

    Ke Sun, Yifan Zhao, Dan Hao, and Lu Zhang. 2022. Static Type Recommendation for Python. InProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE ’22). Association for Computing Machinery, New York, NY, USA, 1–13

    2022

  51. [59]

    Ewan Tempero, Tony Gorschek, and Lefteris Angelis. 2017. Barriers to Refactor- ing.Communications of the ACM60, 10 (2017), 54–61

    2017

  52. [60]

    Fuhrer, Adam Kieżun, Michael D

    Frank Tip, Robert M. Fuhrer, Adam Kieżun, Michael D. Ernst, Ittai Balaban, and Bjorn De Sutter. 2011. Refactoring Using Type Constraints.ACM Transactions on Programming Languages and Systems33, 3 (2011), 9:1–9:47

    2011

  53. [61]

    Frank Tip, Adam Kiezun, and Dirk Bäumer. 2003. Refactoring for Generalization Using Type Constraints. InProceedings of the 18th Annual ACM SIGPLAN Con- ference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA ’03). Association for Computing Machiner...

    2003

  54. [62]

    Lance Tokuda and Don Batory. 2001. Evolving Object-Oriented Designs with Refactorings.Automated Software Engineering8, 1 (2001), 89–120

    2001

  55. [63]

    Nikolaos Tsantalis and Alexander Chatzigeorgiou. 2009. Identification of Move Method Refactoring Opportunities.IEEE Transactions on Software Engineering 35, 3 (2009), 347–367

    2009

  56. [64]

    Nikolaos Tsantalis, Matin Mansouri, Laleh Mousavi Eshkevari, Davood Mazi- nanian, and Danny Dig. 2018. Accurate and efficient refactoring detection in commit history. InProceedings of the 40th International Conference on Software Engineering (ICSE ’18). Association for Computi...

    2018

  57. [66]

    Haibo Wang, Zhuolin Xu, Huaien Zhang, Nikolaos Tsantalis, and Shin Hwei Tan

  58. [67]

    Siqi Wang, Xing Hu, Bei Wang, WenXin Yao, Xin Xia, and Xinyu Wang. 2025. Refactoring Deep Learning Code: a Study of Practices and Unsatisfied Tool Needs. InProceedings of the 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE Computer Societ...

    2025

  59. [68]

    2009.Why Programs Fail: A Guide to Systematic Debugging(2nd ed.)

    Andreas Zeller. 2009.Why Programs Fail: A Guide to Systematic Debugging(2nd ed.). Morgan Kaufmann / Elsevier, Burlington, MA, USA

    2009

  60. [69]

    Yang Zhang, Yanlei Li, Grant Meredith, Kun Zheng, and Xiaobin Li. 2025. Move method refactoring recommendation based on deep learning and LLM-generated information.Inf. Sci.697, C (April 2025), 17 pages. doi:10.1016/j.ins.2024.121753

    2025

  61. [70]

    Yifan Zhang, Yang Yuan, and Andrew Chi-Chih Yao. 2025. Meta Prompting for AI Systems. arXiv:2311.11482 [cs.AI] https://arxiv.org/abs/2311.11482

    2025 arXiv

  62. [2017]

    InProceedings of the Foundations of Software Engineering (FSE’17)

    Understanding the Impact of Refactoring on Smells: A Longitudinal Study of 23 Software Projects. InProceedings of the Foundations of Software Engineering (FSE’17). Association for Computing Machinery, New York, NY, USA, 465–475

  63. [2019]

    InProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference SBES 2026, September 8–11, 2026, São Paulo, SP, Brazil Oliveira et al

    Java reflection API: revealing the dark side of the mirror. InProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference SBES 2026, September 8–11, 2026, São Paulo, SP, Brazil Oliveira et al. and Symposium on the Foundations of Software Engineer...

    2019

  64. [2024]

    arXiv:2409.14610 [cs.SE] https://arxiv.org/abs/2409.14610

    An Empirical Study of Refactoring Engine Bugs. arXiv:2409.14610 [cs.SE] https://arxiv.org/abs/2409.14610

    arXiv

  65. [2026]

    Towards Understanding Refactoring Engine Bugs.ACM Transactions on Software Engineering and Methodology35, 5, Article 138 (2026), 55 pages

    2026

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.