Pith. sign in

REVIEW 5 major objections 5 minor 38 references

Identifier Name Similarities: An Exploratory Study

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read A seven-category taxonomy sorts the ways identifier names resemble each other inside a codebase, with frequency counts from five open-source Java projects.

desk verdict The taxonomy is the real contribution and is genuinely useful, but the RQ2 prevalence numbers are undermined by an invalid sampling-inference argument. read the letter →

arxiv 2507.18081 v1 pith:7A6VEG2Q submitted 2025-07-24 cs.SE

classification cs.SE
keywords identifiernamesnamesimilarityprogramcomprehensionnamingtaxonomycodequalitymanualclassificationopen-sourceJavaprojects
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a first taxonomy of the ways identifier names (the names of variables, methods, classes, and similar program elements) resemble one another within a codebase. It sorts the similarities into seven categories, from deliberate reuse of the same standardized name to abbreviated or single-character variants, and it applies the taxonomy by manually classifying identifiers in five open-source Java projects. The aim is to give researchers and tool builders a shared vocabulary so that confusingly similar names can be studied for their effect on code comprehension, maintainability, and collaboration. A sympathetic reader would take the contribution as the taxonomy itself, together with initial frequency evidence, not as a finished measurement.

What carries the argument

The carrying mechanism is the taxonomy itself, a seven-category classification of the relation between an identifier's name and its semantic role in the surrounding code. For each candidate pair of similar identifiers, a reviewer reads the surrounding code, judges whether the names are semantically equivalent, related, or unrelated, and assigns the pair to one of the seven categories, with a few subcategories such as Polymorphic Names and Cardinality Names. The taxonomy carries the argument by converting an unstructured inventory of extracted identifier names into frequency counts that can be compared across projects; without that classification step, raw string similarity would not distinguish harmless deliberate reuse from confusing collisions.

What would settle it

Run the taxonomy on the same five projects with a pre-specified random sample of identifier pairs, two independent annotators who classify blind, and a published agreement statistic; if agreement is near chance, or if the category frequencies shift enough that Standardized Repetitive Names is no longer among the top three in every project, the taxonomy's reliability and the reported frequencies would be undermined.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that identifier-name similarity is not one phenomenon but several distinct ones: same names reused intentionally, different names for the same concept, lexically identical names with unrelated meanings, type-based and derivation-based variants, numerically distinguished variants, and concise or abbreviated forms. The paper reports that the categories occur with widely different frequencies across the five projects, from a high of 90.5% of reviewed identifiers classified as variants to a low of 12.85%, and that Standardized Repetitive Names is among the three most common categories in every project. This supports the paper's claim that most codebases deliberately reuse names for consistency, but that the mix of similarity patterns varies strongly with project size and structure.

Load-bearing premise

The load-bearing assumption is that reviewers classifying the same identifier pairs by reading surrounding code would agree on the category, and that the inspected identifier sets stand in for each project's full set of identifiers.

Editorial extensions

If this is right

  • If the taxonomy holds, development teams can audit a codebase for the specific patterns that pose the greatest risk: Colliding Names, Inconsistent Semantic Names, and Concise Variants.
  • Automated detectors can be built from the taxonomy, but they will need to read context and data types, not just compare name strings, to assign pairs to the right category.
  • The frequency evidence suggests that a high variant percentage is not itself a quality score; boilerplate-heavy codebases will naturally show more Standardized Repetitive Names.
  • The categories give researchers a dependent variable for studies linking naming similarity to comprehension time, bug rates, or onboarding difficulty.
  • The taxonomy points to concrete refactoring moves: rename colliding or abbreviated identifiers, align inconsistent names for the same concept, and replace numeric variants with collections or arrays.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report an inter-rater reliability statistic, so a reader should treat the category frequencies as descriptive of the reviewed sets rather than as a stable measurement of the full identifier populations.
  • One test the paper does not run is whether the high-risk categories actually slow comprehension; a controlled experiment comparing code with Colliding or Concise names against renamed versions would give the taxonomy predictive teeth.
  • The categories may overlap in practice, and the paper does not fully specify an operational rule for assigning a pair to exactly one category; future refinements will need such a rule before automated tools can use the taxonomy directly.
  • If the taxonomy generalizes to dynamically typed languages, Type-Based Variants would likely shrink in importance, which would be a quick way to test whether the categories reflect universal naming behavior or Java-specific conventions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents an exploratory taxonomy of identifier name similarities in Java, derived from manual inspection of five open-source projects. Seven categories with subcategories are defined and illustrated with code examples. RQ2 reports per-project counts and percentages of identifiers classified as variants (Table I), and the paper discusses implications for researchers, developers, tool vendors, and educators.

Significance. The qualitative taxonomy is a plausible and potentially useful starting point, and the paper gives concrete code examples for every category, which aids comprehension and reuse. The artifact package linked in the paper is a positive step for reproducibility. If the prevalence results were properly supported, the frequency distribution across projects would be a useful empirical contribution. As it stands, the quantitative claims are not reliable because of an unspecified sampling mechanism, an internally inconsistent ranking, and the absence of reliability measures for the manual classification.

major comments (5)
  1. [Section II.C and Table I] The analyzed counts (295, 837, 1,167, 1,206, 724 identifiers) are presented as sufficient for statistical significance at a 95% confidence level and a 5% margin of error, but Section II.C states that reviewers analyzed 'sets of potentially similar identifiers.' The extracted inventories in Section II.B are much larger (494, 15,688, 1,697, 21,876, 7,590), and no random or representative sampling procedure is described. Without a probability sampling mechanism, the sample-size argument cannot establish representativeness, and the variant percentages in Table I are likely inflated if the subsets were enriched for similarity before classification. The authors should either specify and justify the selection mechanism or explicitly reframe Table I as describing the reviewed candidate subsets rather than the full identifier populations.
  2. [Section III, Table I] The text states that Apache Nutch had the lowest percentage of identifier variants, with 93 (12.85%) instances, but Dromara Sureness has 129/1,167 = 11.05%, which is lower. This internal contradiction invalidates the comparative ranking and the adjacent discussion of which projects have low similarity rates. Please correct the statement and re-check all percentages and rankings in Table I.
  3. [Section II.C] The taxonomy was iteratively refined and finalized on the same five projects whose identifiers are then counted using that taxonomy in Section III. This makes the frequency results descriptive of the taxonomy-building sample rather than an independent test of the categories. For an exploratory study this circularity is acceptable if clearly acknowledged, but RQ2's phrasing, which asks how frequently categories occur in real-world projects, should be softened to describe the projects used to develop the taxonomy, and the Threats to Validity section should state explicitly that the counts cannot validate the taxonomy categories.
  4. [Table I] The unit of counting is ambiguous: the text says the team calculated 'the number of occurrences for each category,' while Table I reports a total 'identifier similarities count' and per-category percentages. Because an identifier or an identifier pair can plausibly belong to more than one category (for example, an abbreviated name that also differs in cardinality), the denominator and the mutual exclusivity of categories need to be defined. Without this, the category percentages cannot be interpreted or reproduced from the reported totals.
  5. [Section II.C] Section II.C describes independent review and discussion to resolve disagreements, but no inter-rater reliability statistic (e.g., Cohen's kappa) is reported for the manual classification. Given that the taxonomy and all frequency counts depend on subjective semantic interpretation, the absence of a reliability measure substantially weakens confidence in both the categories and the counts. Reporting an agreement statistic on at least a subsample of identifiers would materially strengthen the claims.
minor comments (5)
  1. [References] Reference [32] for Spring Petclinic points to the Dromara Sureness repository; it should point to the Spring Petclinic repository.
  2. [Section III, Listing 11] The text refers to identifiers 'COUNT 2' and 'COUNT 3', but the code shows COUNT_2 and COUNT_3; the text should match the code.
  3. [Table I] Table I uses the label 'Name Collisions' while Section III uses 'Colliding Names' for the same category; the terminology should be consistent throughout.
  4. [Section III, Listing 9] Listing 9 contains 'rolesTmp = null ? null : ...', which is not valid Java; it should be 'rolesTmp == null'.
  5. [Section II.C] The project name is written 'ThymeLeaf' in one place and 'Thymeleaf' elsewhere; please standardize the spelling.

Circularity Check

2 steps flagged · score 6.0 of 10

RQ2 prevalence percentages are computed over a similarity-enriched subset, not a random sample; the reported 'variant' rates are forced by the selection rule.

  1. fitted input called prediction [Section II.C (Analysis Process) and Section III RQ2 / Table I]
    "The reviewers analyzed sets of potentially similar identifiers by inspecting the surrounding code, verifying whether pairs of identifiers had consistent semantically similar meanings. ... In each project, the number of identifiers analyzed exceeded the sample size necessary for statistical significance at a 95% confidence level with a 5% margin of error. This ensures that our findings are representative of the overall identifier population in each project."

    The identifiers counted in Table I (e.g., 295 for Spring Petclinic) are not a random or stratified sample of the extracted inventories (494 for Petclinic); they are the 'sets of potentially similar identifiers' that reviewers were asked to analyze. Because the selection step already enriches for name similarity, the reported percentages—Spring Petclinic 267/295 = 90.5% 'variants'—are largely determined by the candidate-generation rule, not by the project. The sample-size statement only applies to random samples; it cannot turn a similarity-enriched convenience set into a representative population estimate. RQ2's prevalence distribution is therefore the input selection re-labeled as a project-level finding.

  2. other [Section II.C (Analysis Process) and Section III RQ2]
    "Through iterative discussions, the reviewers resolved any discrepancies until they reached review saturation. At review saturation, additional reviews no longer provided new insights, and no significant changes were made to the taxonomy. A final taxonomy consisting of seven distinct categories was established. Using the finalized taxonomy, the team calculated the number of occurrences for each category for all reviewed projects."

    The taxonomy was iteratively refined on the same five projects until it stabilized, and then those same projects were used to compute the occurrence counts that form RQ2's quantitative contribution. Since the categories were adjusted to accommodate the observed examples from these projects, the counts are an in-sample description rather than an independent test of the taxonomy. This is not definitional circularity—the categories have external grounding in naming research—but it means the prevalence numbers partly restate the taxonomy-construction process instead of providing independent empirical confirmation.

full rationale

The paper is not definitionally circular in its central taxonomy: the seven categories are substantive, grounded in prior naming research, and no load-bearing self-citation or uniqueness theorem is invoked. The authors also honestly list threats to validity, including manual subjectivity and lack of practitioner validation. However, RQ2's central quantitative claim—category prevalence—reduces to the paper's own candidate-selection step. Section II.C says reviewers analyzed 'sets of potentially similar identifiers,' and Section II.B inventories show far more identifiers per project (e.g., 494 vs 295 for Petclinic; 15,688 vs 837 for Metacat), with no probability sampling described. Counting 'variants' within a set that was formed because it already looked similar makes percentages like 267/295 (90.5%) an artifact of set construction. The 95%-confidence/margin-of-error statement only applies to random samples and cannot rescue the inference. Additionally, the taxonomy was iteratively refined on the same five projects and then counted on those projects, so the RQ2 distribution is partly in-sample. These issues make the prevalence results partially circular, while the taxonomy itself retains independent descriptive content.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No numerical parameters are fitted. The study relies on two domain assumptions about manual annotation and one ad hoc representativeness assumption that is not justified by a sampling procedure. No new entities are postulated.

assumptions (3)
  • domain assumption Manual semantic interpretation of identifier intent from surrounding code is a reliable basis for classification.
    Section II.C states this necessity, but no inter-rater reliability or validation against practitioner judgment is provided.
  • ad hoc to paper The analyzed subset of identifiers is representative of each project's full identifier population.
    Section II.C claims sample sizes exceed 95% confidence and 5% margin requirements, but no random sampling procedure is described, so representativeness is an unverified assumption.
  • domain assumption Identifiers with similar names can increase cognitive load and cause confusion.
    This motivation is cited from prior work on naming and comprehension, and is not tested in this study.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identifier Name Similarities: An Exploratory Study." pith.science (2026). https://pith.science/paper/7A6VEG2Q

@misc{pith2026250718081,
  author       = {Pith},
  title        = {Pith review of: Identifier Name Similarities: An Exploratory Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7A6VEG2Q}},
  note         = {Machine review of arXiv:2507.18081}
}
read the original abstract

Identifier names, which comprise a significant portion of the codebase, are the cornerstone of effective program comprehension. However, research has shown that poorly chosen names can significantly increase cognitive load and hinder collaboration. Even names that appear readable in isolation may lead to misunderstandings in contexts when they closely resemble other names in either structure or functionality. In this exploratory study, we present our preliminary findings on the occurrence of identifier name similarity in software projects through the development of a taxonomy that categorizes different forms of identifier name similarity. We envision our initial taxonomy providing researchers with a platform to analyze and evaluate the impact of identifier name similarity on code comprehension, maintainability, and collaboration among developers, while also allowing for further refinement and expansion of the taxonomy.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 38 canonical work pages

  1. [1]

    Concise and consistent naming,

    F. Deissenboeck and M. Pizka, “Concise and consistent naming,” Soft- ware Quality Journal , vol. 14, pp. 261–282, 2006. 1 PREPRINT

  2. [2]

    Measuring program comprehension: A large-scale field study with professionals,

    X. Xia, L. Bao, D. Lo, Z. Xing, A. E. Hassan, and S. Li, “Measuring program comprehension: A large-scale field study with professionals,” IEEE Transactions on Software Engineering , vol. 44, no. 10, pp. 951– 976, 2017. 1

  3. [3]

    Shorter identifier names take longer to comprehend,

    J. C. Hofmeister, J. Siegmund, and D. V . Holt, “Shorter identifier names take longer to comprehend,” Empirical Software Engineering , vol. 24, pp. 417–443, 2019. 1, 5

  4. [4]

    Relating identifier naming flaws and code quality: An empirical study,

    S. Butler, M. Wermelinger, Y . Yu, and H. Sharp, “Relating identifier naming flaws and code quality: An empirical study,” in 2009 16th Working Conference on Reverse Engineering, pp. 31–35, IEEE, 2009. 1

  5. [5]

    The effect of poor source code lexicon and readability on developers’ cognitive load,

    S. Fakhoury, Y . Ma, V . Arnaoudova, and O. Adesope, “The effect of poor source code lexicon and readability on developers’ cognitive load,” in Proceedings of the 26th Conference on Program Comprehension , ICSE ’18, ACM, May 2018. 1

  6. [6]

    What’s in a name? a study of identifiers,

    D. Lawrie, C. Morrell, H. Feild, and D. Binkley, “What’s in a name? a study of identifiers,” in 14th IEEE International Conference on Program Comprehension (ICPC’06), pp. 3–12, 2006. 1

  7. [7]

    Descriptive compound identifier names improve source code comprehension,

    A. Schankin, A. Berger, D. V . Holt, J. C. Hofmeister, T. Riedel, and M. Beigl, “Descriptive compound identifier names improve source code comprehension,” in Proceedings of the 26th Conference on Program Comprehension, ICPC ’18, (New York, NY , USA), p. 31–40, Association for Computing Machinery, 2018. 1

  8. [8]

    A survey of the forms of java reference names,

    S. Butler, M. Wermelinger, and Y . Yu, “A survey of the forms of java reference names,” in 2015 IEEE 23rd International Conference on Program Comprehension, pp. 196–206, 2015. 1

Show all 38 references
  1. [9]

    Understanding digits in identifier names: An exploratory study,

    A. Peruma and C. D. Newman, “Understanding digits in identifier names: An exploratory study,” in Proceedings of the 1st International Workshop on Natural Language-Based Software Engineering , pp. 9–16,

  2. [10]

    The impact of identifier style on effort and comprehension,

    D. W. Binkley, M. Davis, D. J. Lawrie, J. I. Maletic, C. Morrell, and B. Sharif, “The impact of identifier style on effort and comprehension,” Empir. Softw. Eng., vol. 18, no. 2, pp. 219–276, 2013. 1

  3. [11]

    On the generation, structure, and semantics of grammar patterns in source code identifiers,

    C. D. Newman, R. S. AlSuhaibani, M. J. Decker, A. Peruma, D. Kaushik, M. W. Mkaouer, and E. Hill, “On the generation, structure, and semantics of grammar patterns in source code identifiers,” Journal of Systems and Software, vol. 170, p. 110740, 2020. 1

  4. [12]

    An empirical study of abbreviations and expansions in software artifacts,

    C. D. Newman, M. J. Decker, R. S. Alsuhaibani, A. Peruma, D. Kaushik, and E. Hill, “An empirical study of abbreviations and expansions in software artifacts,” in 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME) , pp. 269–279, 2019. 1

  5. [13]

    Contextualizing rename decisions using refactorings, commit messages, and data types,

    A. Peruma, M. W. Mkaouer, M. J. Decker, and C. D. Newman, “Contextualizing rename decisions using refactorings, commit messages, and data types,” Journal of Systems and Software , vol. 169, p. 110704,

  6. [14]

    An ensemble approach for annotating source code identifiers with part-of-speech tags,

    C. D. Newman, M. J. Decker, R. S. Alsuhaibani, A. Peruma, M. W. Mkaouer, S. Mohapatra, T. Vishnoi, M. Zampieri, T. J. Sheldon, and E. Hill, “An ensemble approach for annotating source code identifiers with part-of-speech tags,” IEEE Transactions on Software Engineering , vol. ...

  7. [15]

    A survey on renamings of software entities,

    G. Li, H. Liu, and A. S. Nyamawe, “A survey on renamings of software entities,” ACM Comput. Surv., vol. 53, Apr. 2020. 1

  8. [16]

    An approach to automatically assess method names,

    R. S. Alsuhaibani, C. D. Newman, M. J. Decker, M. L. Collard, and J. I. Maletic, “An approach to automatically assess method names,” in Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension, ICPC ’22, p. 202–213, ACM, May 2022. 1

  9. [17]

    Ideal: An open-source identifier name appraisal tool,

    A. Peruma, V . Arnaoudova, and C. D. Newman, “Ideal: An open-source identifier name appraisal tool,” in 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME) , p. 599–603, IEEE, Sept. 2021. 1

  10. [18]

    Scalar: A part- of-speech tagger for identifiers,

    C. D. Newman, B. Scholten, S. Testa, J. A. C. Behler, S. Banabilah, M. L. Collard, M. J. Decker, M. W. Mkaouer, M. Zampieri, E. A. Alomar, R. S. AlSuhaibani, A. Peruma, and J. I. Maletic, “Scalar: A part- of-speech tagger for identifiers,” in 2025 IEEE/ACM 33rd International C...

  11. [19]

    Identifier names in computer programs: Literature review.,

    I. Herka, “Identifier names in computer programs: Literature review.,” Advances in Cognitive Psychology , vol. 19, no. 3, 2023. 1

  12. [20]

    Would a rose by any other name smell as sweet? examining the cost of similarity in identifier naming.,

    N. Al Madi and M. Zang, “Would a rose by any other name smell as sweet? examining the cost of similarity in identifier naming.,” in PPIG, pp. 91–106, 2022. 1

  13. [21]

    Lexical categories for source code identifiers,

    C. D. Newman, R. S. Alsuhaibani, M. L. Collard, and J. I. Maletic, “Lexical categories for source code identifiers,” in 2017 IEEE 24th Inter- national Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 228–239, IEEE, 2017. 1

  14. [22]

    Contextual similarity among identifier names: An empirical study,

    R. Gresta and E. Cirilo, “Contextual similarity among identifier names: An empirical study,” in Workshop de Visualizac ¸˜ao, Evoluc ¸˜ao e Manutenc ¸˜ao de Software (VEM) , pp. 49–56, SBC, 2020. 1

  15. [23]

    Linguistic antipatterns: What they are and how developers perceive them,

    V . Arnaoudova, M. Di Penta, and G. Antoniol, “Linguistic antipatterns: What they are and how developers perceive them,” Empirical Software Engineering, vol. 21, no. 1, pp. 104–158, 2016. 1

  16. [24]

    Clone detection using abstract syntax trees,

    I. Baxter, A. Yahin, L. Moura, M. Sant’Anna, and L. Bier, “Clone detection using abstract syntax trees,” in Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272) , ICSM-98, p. 368–377, IEEE Comput. Soc. 1

  17. [25]

    Shobha, A

    G. Shobha, A. Rana, V . Kansal, and S. Tanwar, Code Clone Detec- tion—A Systematic Review , p. 645–655. Springer Nature Singapore,

  18. [26]

    A systematic literature review on the use of machine learning in code clone research,

    M. Kaur and D. Rattan, “A systematic literature review on the use of machine learning in code clone research,” Computer Science Review , vol. 47, p. 100528, Feb. 2023. 1

  19. [27]

    A systematic literature review on source code similarity measurement and clone detection: Techniques, applications, and challenges,

    M. Zakeri-Nasrabadi, S. Parsa, M. Ramezani, C. Roy, and M. Ekhtiarzadeh, “A systematic literature review on source code similarity measurement and clone detection: Techniques, applications, and challenges,” Journal of Systems and Software , vol. 204, p. 111796, Oct. 2023. 1

  20. [28]

    Sampling projects in github for MSR studies,

    O. Dabic, E. Aghajani, and G. Bavota, “Sampling projects in github for MSR studies,” in 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021 , pp. 560–564, IEEE, 2021. 2

  21. [29]

    From commit message generation to history-aware commit message completion,

    A. Eliseeva, Y . Sokolov, E. Bogomolov, Y . Golubev, D. Dig, and T. Bryksin, “From commit message generation to history-aware commit message completion,” in Proceedings of the 38th IEEE/ACM Inter- national Conference on Automated Software Engineering , ASE ’23, p. 723–735, IEE...

  22. [30]

    Code saturation ver- sus meaning saturation: how many interviews are enough?,

    M. M. Hennink, B. N. Kaiser, and V . C. Marconi, “Code saturation ver- sus meaning saturation: how many interviews are enough?,” Qualitative health research, vol. 27, no. 4, pp. 591–608, 2017. 2

  23. [31]

    Dromara Sureness

    “Dromara Sureness.” url = https://github.com/dromara/sureness. 2

  24. [32]

    Spring PetClinic

    “Spring PetClinic.” url = https://github.com/dromara/sureness. 2

  25. [33]

    Netflix Metacat

    “Netflix Metacat.” url = https://github.com/Netflix/metacat. 2

  26. [34]

    Thymeleaf

    “Thymeleaf.” url = https://github.com/thymeleaf/thymeleaf. 2

  27. [35]

    Apache Nutch

    “Apache Nutch.” url = https://github.com/apache/nutch. 2

  28. [36]

    Idbench: Evaluating semantic rep- resentations of identifier names in source code,

    Y . Wainakh, M. Rauf, and M. Pradel, “Idbench: Evaluating semantic rep- resentations of identifier names in source code,” in2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 562–573, IEEE, 2021. 2

  29. [37]

    Namesake: A checker of lexical similarity in identifier names,

    N. Al Madi, “Namesake: A checker of lexical similarity in identifier names,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering , pp. 1–5, 2022. 2

  30. [38]

    Artifact package

    “Artifact package.” url = https://doi.org/10.6084/m9.figshare.29127113. 6

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.