Pith. sign in

REVIEW 3 major objections 6 minor 65 references

Information-Theoretic Detection of Unusual Source Code Changes

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Shannon entropy of source code flags unusual commits with over 60% precision.

desk verdict Large-scale entropy-evolution study with a candid but author-labelled precision claim; worth peer review, but the >60% anomaly-detection number needs maintainer labels and a baseline before it becomes credible. read the letter →

arxiv 2506.06508 v1 pith:4VS2Y5UC submitted 2025-06-06 cs.SE

Adriano Torres , Sebastian Baltes , Christoph Treude , Markus Wagner This is my paper · ORCID
classification cs.SE
keywords informationtheoryentropysourcecodeanalysissoftwareevolutionanomalydetectionabstractsyntaxtreecomplexityLehman'slaws
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the entropy of source code, measured over token frequencies and over abstract-syntax-tree edge types, provides a practical signal for how surprising a commit is. Tracking every commit of 95 actively maintained Java projects, the authors find that commits whose entropy delta is an outlier against recent history are judged relevant by human review with over 60% precision, and up to 83% under a permissive definition. They also report that these entropy metrics correlate only weakly with McCabe's cyclomatic complexity and line/token counts, which they interpret as evidence that entropy captures a different dimension of code complexity. If the paper is right, a cheap per-commit entropy calculation could help maintainers decide where to focus their attention without running heavyweight analyses.

What carries the argument

The mechanism is the per-commit entropy delta. Textual entropy is the Shannon entropy of the token histogram, $H_{\text{TOKEN}} = -\sum_{w\in W} p(w)\log_2 p(w)$, and structural entropy is the same calculation over the frequencies of abstract-syntax-tree edge types. After each commit, the total entropy of the changed files is recomputed and compared with the previous state, producing a delta that is added to a running distribution of past deltas; a commit is flagged as unusual when its delta has a z-score of at least 3. A configurable memory window determines how many past deltas remain in the distribution, and the study varies this window at 50%, 75%, and 100% of the commit history.

What would settle it

Ask the maintainers of the sampled projects to label the detected outlier commits as surprising or routine without seeing the researchers' labels, then recompute precision against those maintainer labels; if agreement is no better than chance or precision falls below 60%, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the Shannon entropy of a project's source code, measured over token frequencies and over abstract-syntax-tree edge types, captures how much information each commit inserts or removes, and that large entropy deltas mark changes a maintainer would want to examine. Textual entropy is $H_{\text{TOKEN}} = -\sum_{w \in W} p(w)\log_2 p(w)$, and structural entropy applies the same formula to the histogram of AST edge types. Over 1,827,204 change events from 95 Java projects, commits whose delta against the recent distribution of deltas exceeded a z-score threshold were judged by the authors' manual review to be relevant in 58–83% of cases, with 72.5% under their main conservative definition. The same data yield weak correlations with McCabe's cyclomatic complexity and with line/token counts, leading the paper to claim that entropy measures a different dimension of complexity, and support Lehman's laws of continuing change and increasing complexity while contradicting the smooth-growth and organizational-stability laws.

Load-bearing premise

The load-bearing premise is that the research team's judgement of which outlier commits are important matches what the projects' own maintainers would consider important.

Editorial extensions

If this is right

  • On the full-history setting, flagged commits were judged relevant in 58% of cases under the most conservative definition and 83% under the least conservative, with 72.5% at the paper's main conservative level.
  • Entropy metrics correlate weakly with McCabe cyclomatic complexity (range -0.05 to 0.32) and with line/token counts, indicating that information-theoretic measures capture a different dimension of code complexity.
  • About two-thirds of detected unusual events involve internal, programmer-facing concerns, while user-facing changes are rare (about 10%) but tend to touch many files at once.
  • Limiting the memory window from full history to 50% of commits raises the number of detected events by roughly 20% but lowers precision, meaning detection thresholds must track the project's current pace of entropy change.
  • Lehman's laws of continuing change and increasing complexity are supported by the entropy data, while the smooth-growth and organizational-stability laws are not in this open-source sample.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test it, but the memory-size results suggest the z-score threshold should be adapted to the local slope of entropy growth, so that early steep phases and later stable phases receive different outlier criteria.
  • The paper argues for language agnosticism from representation histograms, yet only studies Java; a direct check is to run the same detector on Python or JavaScript repositories and see whether maintainer-validated precision holds above 60%.
  • Because entropy correlates weakly with cyclomatic complexity, a fused surprisal index may catch changes neither metric flags alone; the correlation tables in the paper make that a concrete next experiment rather than a mere suggestion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper defines Shannon-entropy metrics for two source-code representations (token streams and abstract syntax tree edges), mines the complete commit histories of 95 actively maintained Java GitHub projects, and investigates four research questions: trends of entropy evolution (RQ1), correlations with classic complexity metrics (RQ2), agreement with Lehman's laws of software evolution (RQ3), and entropy-based anomaly detection of unusual source-code changes (RQ4). The central quantitative claim is that the proposed detection approach can recognize unusual change events with over 60% precision, based on manually labeled samples of outlier commits.

Significance. If validated, the paper would provide a lightweight, deterministic, and language-agnostic signal for identifying unusual source-code changes, complementing existing complexity metrics and offering a reproducible basis for future information-theoretic work on software evolution. The study's strengths include its scale (95 projects, 1,827,204 change events), commit-by-commit granularity, promised availability of code and data, explicit reporting of inter-rater agreement, and unusually candid treatment of threats to validity. However, the headline precision claim is currently supported only by labels assigned by the authors themselves, and the detection threshold appears to be selected on the same labeled data, so the practical significance of the >60% figure remains conditional on external validation.

major comments (3)
  1. [Abstract; §4.4.1; Table 7] The central claim that the approach 'may effectively recognise unusual source code change events with over 60% precision' is measured against labels assigned by the authors, not by the maintainers of the 95 projects. Section 7 explicitly concedes that 'the precision of our detection may be vastly inferior to that obtained if events were labelled by the actual maintainers.' Moreover, Table 7 shows that the most conservative strictness level yields 58.12% precision for the 100% memory configuration, so the abstract's 'over 60%' is not supported by the most conservative row. The paper should either obtain maintainer ground-truth labels for a validation sample or reframe the claim as a proof-of-concept precision estimate under author-defined relevance, with the associated uncertainty made explicit.
  2. [§4.4.1; §7] The z-score threshold used to define outliers appears to be tuned on the same manually labeled data used for evaluation. Section 4.4.1 states that any change with z-score less than or equal to 3 was considered a false positive, while Section 7 reveals that the authors selected 'the definition of statistical outlier (i.e., z-score) for which we observed the best signal-to-noise ratio.' This in-sample threshold selection, combined with the additional free parameters of maximum memory size and the 50% feature-code contribution filter, means the precision estimates in Table 7 are likely optimistic. The authors should report how the threshold was chosen, justify it independently of the evaluation labels, or provide a sensitivity analysis that separates threshold selection from precision measurement.
  3. [§4.4.1; Table 7] The precision results are highly sensitive to the maximum memory configuration, yet the manuscript presents only the 100% memory setting as supporting the abstract's claim. Table 7 reports precision drops to 40.12% at 50% memory even in the least conservative strictness level, and the RQ4 summary notes that reducing memory increases detected events by about 20% but decreases precision. Since the choice of maximum memory is a free parameter with no clear a-priori justification, the paper should either justify the full-memory configuration as the intended deployment setting or present the precision claim across configurations with a clear recommendation.
minor comments (6)
  1. [§1.2, Eq. (2)] Equation (2) is missing the minus sign that appears in Equation (1); as written, H(X) would be negative for a non-degenerate distribution.
  2. [§1.1, §5, §10] There are several typos: 'cyclomacic complexity' should be 'cyclomatic complexity', 'Halmstead' should be 'Halstead', and 'README.me' should be 'README.md'.
  3. [Table 3] The table caption reads 'increasing percentual changes of information content' but the table reports counts of spikes and drops; please clarify whether these are absolute percentage changes and how the spike/drop rows are defined.
  4. [§4.4.2, Figure 10, Table 6] The text and Figure 10 refer to a sample of 350 points for the full context window, while Table 6 lists a 95% confidence-interval sample of 320 points and the accompanying text says n = 321; these numbers should be reconciled.
  5. [§6] The comparison with ChatGPT-4 is largely speculative and not grounded in the empirical data collected for the other research questions; consider moving it to a clearly marked discussion subsection or shortening it so it does not distract from the paper's contributions.
  6. [§3.2, Eq. (4)] The paper refers to 'structural entropy obtained from the edges of the respective AST' but does not explain whether edge types, node types, or both are counted; please clarify the exact histogram used for Equation (4).

Circularity Check

1 steps flagged · score 6.0 of 10

Precision claim in RQ4 is fitted to the same labels used for evaluation, not predicted independently.

  1. fitted input called prediction [Section 7 (Threats to validity), construct validity paragraph, re Section 4.4.1 and Table 7]
    "We attempted to mitigate both effects by being conservative when labelling points, i.e., by being willing to accept higher false positive rates, and by selecting the definition of statistical outlier (i.e., z-score) for which we observed the best signal-to-noise ratio."

    The detector's z-score cutoff is described as selected for the best signal-to-noise ratio on the observed data, and the same manually labelled events are then used to compute the precision in Table 7 that supports the abstract's 'over 60% precision' claim. This is a fitted parameter being presented as if it were an independent evaluation: the precision is not a neutral prediction of detector quality but partly an artifact of choosing the cutoff that makes the labelled data look best. Since the label set was created by the authors (Section 4.4.1) and not by project maintainers, there is no external ground truth to break the loop. The central RQ4 result is therefore statistically forced by the threshold selection rather than derived from the entropy measure alone.

full rationale

Most of the paper is self-contained: the entropy metrics in Equations 3 and 4 are standard Shannon entropy applied to token and AST-edge histograms, and the RQ1-RQ3 analyses (trends, correlations, Lehman's laws) do not reduce to their inputs. No load-bearing self-citation chain or imported uniqueness theorem appears; the cited prior work (Torres et al., 2023) is an earlier version rather than a forbidden premise. The circularity concern is concentrated in RQ4. The paper's own Section 7 admits that the z-score definition of an outlier was selected for the best observed signal-to-noise ratio, and the precision table used to support the 'over 60% precision' claim is computed on the same manually labelled events. That is a fitted parameter being evaluated on the fitting label set, so the headline precision is optimistically biased by construction. The authors also candidly state that maintainer-validated precision 'may be vastly inferior' to their own labels, reinforcing that the quantitative claim is not independently grounded. A maintainer relabeling study with a pre-registered threshold would be needed to turn the precision estimate into a genuine prediction. Because the entropy definitions and evolution analyses are independent, the circularity is partial rather than total.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central quantitative claims depend on three hand-set thresholds (z-score, memory size, and the 50% feature-code rule), on the now-standard assumption that token/AST histograms reflect information content, and on the representativeness of the project sample. The Surprisal Index is an explicit invented but not operationalized entity. No derived constants or new physical entities are introduced.

free parameters (3)
  • z-score outlier threshold = 3
    Events with z-score <= 3 are treated as false positives; Section 7 indicates this threshold was chosen after observing the best signal-to-noise ratio.
  • maximum memory size = 50%, 75%, 100% of commits
    Configurable context window; the precision results in Table 7 depend on this parameter.
  • false-positive feature-code threshold = 50% of entropy contribution
    Section 4.4.2 states that changes whose feature code contributes less than 50% of the total entropy change may be rejected, but the exact computation is not fully specified.
assumptions (4)
  • standard math The Shannon entropy formula (Eq. 1) is a valid measure of information content for a discrete source.
    The paper relies on the standard definition of entropy, Eq. 1, without proving or questioning it.
  • domain assumption Frequencies of tokens and AST edges in each file adequately represent the information content of source code.
    This is the core modeling choice underlying all entropy metrics; the paper argues for it in Sections 1 and 3 but does not validate it against an external criterion.
  • domain assumption The 95 selected GitHub projects are representative of actively maintained open-source software.
    The selection criteria in Section 3.1 (>1000 stars, >1000 commits, active last month) are convenience criteria that may not generalize to other projects.
  • domain assumption Software 'naturalness' (Hindle et al.) implies that language choices do not materially affect entropy evolution patterns.
    Section 3.1 argues the approach is language agnostic based on the naturalness hypothesis, but only Java is analyzed.
invented entities (1)
  • Surprisal Index (SI)
    purpose: Proposed composite metric combining entropy, cross-entropy, KL divergence, and mutual information to quantify surprisal of code changes.
    Introduced only as a speculative concept in Section 5 with mock UI examples; no implementation, formula, or validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information-Theoretic Detection of Unusual Source Code Changes." pith.science (2026). https://pith.science/paper/4VS2Y5UC

@misc{pith2026250606508,
  author       = {Pith},
  title        = {Pith review of: Information-Theoretic Detection of Unusual Source Code Changes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4VS2Y5UC}},
  note         = {Machine review of arXiv:2506.06508}
}
read the original abstract

The code base of software projects evolves essentially through inserting and removing information to and from the source code. We can measure this evolution via the elements of information - tokens, words, nodes - of the respective representation of the code. In this work, we approach the measurement of the information content of the source code of open-source projects from an information-theoretic standpoint. Our focus is on the entropy of two fundamental representations of code: tokens and abstract syntax tree nodes, from which we derive definitions of textual and structural entropy. We proceed with an empirical assessment where we evaluate the evolution patterns of the entropy of 95 actively maintained open source projects. We calculate the statistical relationships between our derived entropy metrics and classic methods of measuring code complexity and learn that entropy may capture different dimensions of complexity than classic metrics. Finally, we conduct entropy-based anomaly detection of unusual changes to demonstrate that our approach may effectively recognise unusual source code change events with over 60% precision, and lay the groundwork for improvements to information-theoretic measurement of source code evolution, thus paving the way for a new approach to statically gauging program complexity throughout its development.

Figures

Figures reproduced from arXiv: 2506.06508 by the authors.

Figure 1
Figure 1. Introduction of a new operation on a data access class [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Introduction of a new concern Let us consider a case when the proposed change is the one in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Typical time series of entropy evolution. Commits are presented according to [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Evolution plot with big spikes and drops. Textual entropies are omitted for the [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Project that keeps information spread stable throughout its lifetime. A single [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Initially increasing density of information per file, followed by stabilisation [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Ever-increasing information density per file (22% of projects). [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: An increasing entropy spread over time, albeit certain long ranges commits [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Categories for a sample of 50 random commits [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: Sample of 320 points detected by full context window [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: False positives for full context window the distribution in figure 9, which contains 50 usual events. There is no addition of new features in this case; only updates to existing ones. In fact, none of the 50 points assessed in figure 9 had more than 200 lines changed.…
Figure 12
Figure 12. Figure 12: False positives exclusive to 50% memory size [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 13
Figure 13. Figure 13: False positives exclusive to 75% memory size [PITH_FULL_IMAGE:figures/full_fig_p028_13.png]
Figure 14
Figure 14. Figure 14: IDE-generated message when surprisal Index is beyond threshold [PITH_FULL_IMAGE:figures/full_fig_p034_14.png]
Figure 15
Figure 15. Figure 15: IDE-generated message when surprisal Index is beyond threshold [PITH_FULL_IMAGE:figures/full_fig_p034_15.png]
Figure 16
Figure 16. Figure 16: Attaching surprisal Index Information to Merge Requests [PITH_FULL_IMAGE:figures/full_fig_p035_16.png]
Figure 17
Figure 17. Figure 17: Expanded View of Suprising Merge Request [PITH_FULL_IMAGE:figures/full_fig_p035_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 52 canonical work pages

  1. [1]

    Addison-Wesley

    Aho AV, Sethi R, Ullman JD (1986) Compilers, Principles, Techniques, and Tools . Addison-Wesley

    work page 1986

  2. [2]

    International Journal of System Assurance Engineering and Management 9, doi:10.1007/s13198-018-0740-y

    Akundi A, Smith E, Tseng TL (2018) Information entropy applied to software based control flow graphs. International Journal of System Assurance Engineering and Management 9, doi:10.1007/s13198-018-0740-y

    work page doi:10.1007/s13198-018-0740-y 2018

  3. [3]

    In: Proceedings Seventh International Software Metrics Symposium, pp 124--134, doi:10.1109/METRIC.2001.915521

    Allen E, Khoshgoftaar T, Chen Y (2001) Measuring coupling and cohesion of software modules: an information-theory approach. In: Proceedings Seventh International Software Metrics Symposium, pp 124--134, doi:10.1109/METRIC.2001.915521

    work page arXiv 2001

  4. [4]

    In: Submitted to The Thirteenth International Conference on Learning Representations, ://openreview.net/forum?id=AjXkRZIvjB, under review

    Anonymous (2024) GSM -symbolic: Understanding the limitations of mathematical reasoning in large language models. In: Submitted to The Thirteenth International Conference on Learning Representations, ://openreview.net/forum?id=AjXkRZIvjB, under review

    work page 2024

  5. [5]

    Empirical Software Engineering 27(4):94

    Baltes S, Ralph P (2022) Sampling in software engineering research: A critical review and guidelines. Empirical Software Engineering 27(4):94

    work page 2022

  6. [6]

    Berlinger E (1980) An information theory based complexity measure. In: Proceedings of the May 19-22, 1980, National Computer Conference, Association for Computing Machinery, New York, NY, USA, AFIPS '80, p 773–779, doi:10.1145/1500518.1500651, ://doi.org/10.1145/1500518.1500651

    work page arXiv 1980

  7. [7]

    Software Engineering, IEEE Transactions on 22:875 -- 885, doi:10.1109/32.553636

    Blackburn J, Scudder G, Van Wassenhove L (1997) Improving speed and productivity of software development: A global survey of software developers. Software Engineering, IEEE Transactions on 22:875 -- 885, doi:10.1109/32.553636

    work page doi:10.1109/32.553636 1997

  8. [8]

    Annals of software engineering 1:57--94

    Boehm B, Clark B, Horowitz E, Westland C, Madachy R, Selby R (1995) Cost models for future software life cycle processes: Cocomo 2.0. Annals of software engineering 1:57--94

    work page 1995

Show all 65 references
  1. [9]

    Prentice Hall Press

    Boehm BW, Abts C, Brown AW, Chulani S, Clark BK, Horowitz E, Madachy R, Reifer DJ, Steece B (2009) Software cost estimation with COCOMO II. Prentice Hall Press

    2009

  2. [10]

    Handbook of research methods in social and personality psychology pp 3--16

    Brewer MB, Crano WD (2000) Research design and issues of validity. Handbook of research methods in social and personality psychology pp 3--16

    2000

  3. [11]

    CoRR abs/2005.14165, ://arxiv.org/abs/2005.14165, 2005.14165

    Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert - Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler DM, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, M...

    2020 arXiv

  4. [12]

    Empirical Software Engineering 19(1):1--38

    Canfora G, Cerulo L, Cimitile M, Di Penta M (2014) How changes affect software entropy: an empirical study. Empirical Software Engineering 19(1):1--38

    2014

  5. [13]

    Int J Syst Assur Eng Manag 5(2):155--164, doi:10.1007/s13198-014-0226-5, ://doi.org/10.1007/s13198-014-0226-5

    Chaturvedi KK, Kapur PK, Anand S, Singh VB (2014) Predicting the complexity of code changes using entropy based measures. Int J Syst Assur Eng Manag 5(2):155--164, doi:10.1007/s13198-014-0226-5, ://doi.org/10.1007/s13198-014-0226-5

    2014 doi

  6. [14]

    IEEE Transactions on Software Engineering SE-4(3):187--194, doi:10.1109/TSE.1978.231497

    Chen E (1978) Program complexity and programmer productivity. IEEE Transactions on Software Engineering SE-4(3):187--194, doi:10.1109/TSE.1978.231497

    1978

  7. [15]

    In: 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE), pp 332--343

    Christakis M, Bird C (2016) What developers want and need from program analysis: An empirical study. In: 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE), pp 332--343

    2016

  8. [16]

    Computer 27(8):44--49, doi:10.1109/2.303623

    Coleman D, Ash D, Lowther B, Oman P (1994) Using metrics to evaluate software system maintainability. Computer 27(8):44--49, doi:10.1109/2.303623

    1994 doi

  9. [17]

    Software Engineering Strategies pp 52--60

    Cook C (1993) Information theory metric for assembly language. Software Engineering Strategies pp 52--60

    1993

  10. [18]

    The Annals of Probability 3(1):146 -- 158, doi:10.1214/aop/1176996454, ://doi.org/10.1214/aop/1176996454

    Csiszar I (1975) I -Divergence Geometry of Probability Distributions and Minimization Problems . The Annals of Probability 3(1):146 -- 158, doi:10.1214/aop/1176996454, ://doi.org/10.1214/aop/1176996454

    1975

  11. [19]

    In: 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021, IEEE , pp 560--564

    Dabic O, Aghajani E, Bavota G (2021) Sampling projects in github for MSR studies. In: 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021, IEEE , pp 560--564

    2021

  12. [20]

    IEEE Transactions on Software Engineering 14(9):1366--1372, doi:10.1109/32.6179

    Davis J, LeBlanc R (1988) A study of the applicability of complexity measures. IEEE Transactions on Software Engineering 14(9):1366--1372, doi:10.1109/32.6179

    1988 doi

  13. [21]

    In: Workshop on the Application of Engineering Principles to System Security Design, Citeseer, pp 1--10

    De Win B, Piessens F, Joosen W, Verhanneman T (2002) On the importance of the separation-of-concerns principle in secure software engineering. In: Workshop on the Application of Engineering Principles to System Security Design, Citeseer, pp 1--10

    2002

  14. [22]

    Dourish P, Bellotti V (1992) Awareness and coordination in shared workspaces. In: Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work, Association for Computing Machinery, New York, NY, USA, CSCW '92, p 107–114, doi:10.1145/143457.143468, ://doi.org/1...

    1992

  15. [23]

    Addison-Wesley Professional

    Fowler M (2018) Refactoring. Addison-Wesley Professional

    2018

  16. [24]

    Paul Graham

    Graham P (2012) Startup= growth. Paul Graham

    2012

  17. [25]

    Elsevier Science Inc., USA

    Halstead MH (1977) Elements of Software Science (Operating and Programming Systems Series). Elsevier Science Inc., USA

    1977

  18. [26]

    In: 2009 IEEE 31st international conference on software engineering, IEEE, pp 78--88

    Hassan AE (2009) Predicting faults using the complexity of code changes. In: 2009 IEEE 31st international conference on software engineering, IEEE, pp 78--88

    2009

  19. [27]

    Springer Series in Statistics, Springer New York Inc., New York, NY, USA

    Hastie T, Tibshirani R, Friedman J (2001) The Elements of Statistical Learning. Springer Series in Statistics, Springer New York Inc., New York, NY, USA

    2001

  20. [28]

    IEEE Transactions on Computers C-21(5):439--446, doi:10.1109/T-C.1972.223539

    Hellerman L (1972) A measure of computational work. IEEE Transactions on Computers C-21(5):439--446, doi:10.1109/T-C.1972.223539

    1972

  21. [29]

    Communications of the ACM 59(5):122--131

    Hindle A, Barr ET, Gabel M, Su Z, Devanbu P (2016) On the naturalness of software. Communications of the ACM 59(5):122--131

    2016

  22. [30]

    Journal of Open Source Software 3(24):653, doi:10.21105/joss.00653, ://doi.org/10.21105/joss.00653

    Hucka M (2018) Spiral: splitters for identifiers in source code files. Journal of Open Source Software 3(24):653, doi:10.21105/joss.00653, ://doi.org/10.21105/joss.00653

    2018 doi

  23. [31]

    Kalyuga S (2011) Cognitive load theory: How many types of load does it really need? Educational Psychology Review 23(1):1--19

    2011

  24. [32]

    u chemann S, Bannert M, Dementieva D, Fischer F, Gasser U, Groh G, G \

    Kasneci E, Se ler K, K \"u chemann S, Bannert M, Dementieva D, Fischer F, Gasser U, Groh G, G \"u nnemann S, H \"u llermeier E, et al. (2023) Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences 103:102274

    2023

  25. [33]

    Int J Syst Assur Eng Manag 8(2s):599--616, doi:10.1007/s13198-016-0479-2, ://doi.org/10.1007/s13198-016-0479-2

    Kaur A, Kaur K, Chopra D (2017) An empirical study of software entropy based bug prediction using machine learning. Int J Syst Assur Eng Manag 8(2s):599--616, doi:10.1007/s13198-016-0479-2, ://doi.org/10.1007/s13198-016-0479-2

    2017 doi

  26. [34]

    In: International Conference on Product-Focused Software Process Improvement, Springer, pp 282--297

    Keenan D, Greer D, Cutting D (2022) An investigation of entropy and refactoring in software evolution. In: International Conference on Product-Focused Software Process Improvement, Springer, pp 282--297

    2022

  27. [35]

    Software Quality Journal 3:79--103

    Khoshgoftaar TM, Allen EB (1994) Applications of information theory to software engineering measurement. Software Quality Journal 3:79--103

    1994

  28. [36]

    International Journal of Reliability, Quality and Safety Engineering 5(03):227--248

    Khoshgoftaar TM, Allen EB (1998) An information theoretic approach to predicting software faults. International Journal of Reliability, Quality and Safety Engineering 5(03):227--248

    1998

  29. [37]

    Kitchenham B, Pearl Brereton O, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering – a systematic literature review. Information and Software Technology 51(1):7--15, doi:https://doi.org/10.1016/j.infsof.2008.09.009, ://www.scie...

    2009 doi

  30. [38]

    Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 69(6):066138

    Kraskov A, St \"o gbauer H, Grassberger P (2004) Estimating mutual information. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 69(6):066138

    2004

  31. [39]

    biometrics pp 159--174

    Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. biometrics pp 159--174

    1977

  32. [41]

    Proceedings of the IEEE 68(9):1060--1076, doi:10.1109/PROC.1980.11805

    Lehman M (1980) Programs, life cycles, and laws of software evolution. Proceedings of the IEEE 68(9):1060--1076, doi:10.1109/PROC.1980.11805

    1980

  33. [42]

    Lehman M (1984) Program evolution. Information Processing & Management 20(1):19--36, doi:https://doi.org/10.1016/0306-4573(84)90037-2, ://www.sciencedirect.com/science/article/pii/0306457384900372, special Issue Empirical Foundations of Information and Software Science

    1984

  34. [43]

    In: Oh AH, Agarwal A, Belgrave D, Cho K (eds) Advances in Neural Information Processing Systems, ://openreview.net/forum?id=IFXTZERXdM7

    Lewkowycz A, Andreassen AJ, Dohan D, Dyer E, Michalewski H, Ramasesh VV, Slone A, Anil C, Schlag I, Gutman-Solo T, Wu Y, Neyshabur B, Gur-Ari G, Misra V (2022) Solving quantitative reasoning problems with language models. In: Oh AH, Agarwal A, Belgrave D, Cho K (eds) Advances ...

    2022

  35. [44]

    IEEE Transactions on Software Engineering SE-2(4):308--320, doi:10.1109/TSE.1976.233837

    McCabe T (1976) A complexity measure. IEEE Transactions on Software Engineering SE-2(4):308--320, doi:10.1109/TSE.1976.233837

    1976

  36. [45]

    Microsoft Press, Redmond, WA, USA

    McConnell S (2004) Code Complete, Second Edition. Microsoft Press, Redmond, WA, USA

    2004

  37. [46]

    Available at http://www.construx.com/uploadedfiles/resources/whitepapers/Managing\

    McConnell S (2008) Managing technical debt. Available at http://www.construx.com/uploadedfiles/resources/whitepapers/Managing\

    2008

  38. [47]

    Journal of Systems and Software 2(1):39--46, doi:https://doi.org/10.1016/0164-1212(81)90045-5, ://www.sciencedirect.com/science/article/pii/0164121281900455

    Mohanty SN (1981) Entropy metrics for software design evaluation. Journal of Systems and Software 2(1):39--46, doi:https://doi.org/10.1016/0164-1212(81)90045-5, ://www.sciencedirect.com/science/article/pii/0164121281900455

    1981

  39. [48]

    Nicholas Smith FT Danny van Bruggen (2023) Javaparser

    2023

  40. [49]

    ://arxiv.org/abs/2303.08774, 2303.08774

    OpenAI, Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, Almeida D, Altenschmidt J, Altman S, Anadkat S, Avila R, Babuschkin I, Balaji S, Balcom V, Baltescu P, Bao H, Bavarian M, Belgum J, Bello I, Berdine J, Bernadett-Shapiro G, Berner C, Bogdonoff L, Boiko O, Boyd...

    2024 arXiv

  41. [50]

    Communications of the ACM 44(10):43--50

    Ossher H, Tarr P (2001) Using multidimensional separation of concerns to (re) shape evolving software. Communications of the ACM 44(10):43--50

    2001

  42. [51]

    The Pragmatic Bookshelf

    Parr T (2013) The definitive ANTLR 4 reference. The Pragmatic Bookshelf

    2013

  43. [52]

    Available at https://github.com/antlr/grammars-v4

    Parr T (2025) Antlr grammars v4. Available at https://github.com/antlr/grammars-v4

    2025

  44. [53]

    IEEE Transactions on Software Engineering 18(3):174--189, doi:10.1109/32.126767

    Paulson D, Wand Y (1992) An automated approach to information systems decomposition. IEEE Transactions on Software Engineering 18(3):174--189, doi:10.1109/32.126767

    1992 doi

  45. [54]

    Ralph P, Tempero E (2018) Construct validity in software engineering research and software metrics. In: Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018, Association for Computing Machinery, New York, NY, USA, EASE '18,...

    2018

  46. [55]

    The Bell system technical journal 27(3):379--423

    Shannon CE (1948) A mathematical theory of communication. The Bell system technical journal 27(3):379--423

    1948

  47. [56]

    In: The 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), doi:10.1145/3236024.3264598

    Spadini D, Aniche M, Bacchelli A (2018) Pydriller: Python framework for mining software repositories. In: The 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), doi:10.1145/3236024.3264598

    2018

  48. [57]

    Available at https://survey.stackoverflow.co/2024/technology

    StackOverflow (2024) 2024 survey. Available at https://survey.stackoverflow.co/2024/technology

    2024

  49. [58]

    In: Proceedings of the 21st international conference on Software engineering, pp 107--119

    Tarr P, Ossher H, Harrison W, Sutton Jr SM (1999) N degrees of separation: Multi-dimensional separation of concerns. In: Proceedings of the 21st international conference on Software engineering, pp 107--119

    1999

  50. [59]

    Journal of Clinical Epidemiology 41(10):949--958, doi:https://doi.org/10.1016/0895-4356(88)90031-5, ://www.sciencedirect.com/science/article/pii/0895435688900315

    Thompson W, Walter SD (1988) A reappraisal of the kappa coefficient. Journal of Clinical Epidemiology 41(10):949--958, doi:https://doi.org/10.1016/0895-4356(88)90031-5, ://www.sciencedirect.com/science/article/pii/0895435688900315

    1988

  51. [60]

    Morgan Kaufmann Publishers Inc., San Francisco, CA, USA

    Torczon L, Cooper K (2007) Engineering A Compiler, 2nd edn. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA

    2007

  52. [61]

    Available at https://zenodo.org/records/11180885

    Torres A (2024) Supplementary material. Available at https://zenodo.org/records/11180885

    2024

  53. [62]

    In: 2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE), pp 48--55, doi:10.1109/NLBSE59153.2023.00017

    Torres A, Baltes S, Treude C, Wagner M (2023) Applying information theory to software evolution. In: 2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE), pp 48--55, doi:10.1109/NLBSE59153.2023.00017

    2023

  54. [63]

    Treude C, Figueira Filho F, Kulesza U (2015) Summarizing and measuring development activity. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, Association for Computing Machinery, New York, NY, USA, ESEC/FSE 2015, p 625–636, doi:10.1145/278...

    2015

  55. [64]

    Journal of Systems and Software 142:237--247, doi:https://doi.org/10.1016/j.jss.2018.04.063, ://www.sciencedirect.com/science/article/pii/S0164121218300876

    Treude C, Leite L, Aniche M (2018) Unusual events in github repositories. Journal of Systems and Software 142:237--247, doi:https://doi.org/10.1016/j.jss.2018.04.063, ://www.sciencedirect.com/science/article/pii/S0164121218300876

    2018 doi

  56. [65]

    science 185(4157):1124--1131

    Tversky A, Kahneman D (1974) Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science 185(4157):1124--1131

    1974

  57. [66]

    Zapf A, Castell S, Morawietz L, Karch A (2016) Measuring inter-rater reliability for nominal data – which coefficients and confidence intervals are appropriate? BMC Medical Research Methodology 16, ://api.semanticscholar.org/CorpusID:16038581

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.