Pith. sign in

REVIEW 4 major objections 5 minor 75 references

On the Structure and Semantics of Identifier Names Containing Closed Syntactic Category Words

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Closed-category words in code names carry deliberate behavioral meaning, a new study finds.

desk verdict A solid, first-of-its-kind empirical study of closed-category words in identifiers, with a reusable dataset and a credible descriptive taxonomy, but the strongest behavioral claims outrun the evidence. read the letter →

arxiv 2505.18444 v4 pith:WWLXTGMT submitted 2025-05-24 cs.SE

Christian D. Newman , Anthony Peruma , Eman Abdullah AlOmar , Mahie Crabbe , Syreen Banabilah , Reem S. AlSuhaibani , Michael J. Decker , Farhad Akhbardeh
show 3 more authors
Marcos Zampieri Mohamed Wiem Mkaouer Jonathan I. Maletic
This is my paper · ORCID
classification cs.SE
keywords identifiernamingpart-of-speechtaggingclosed-categorytermsgrammarpatternsprogramcomprehensiongroundedtheorysoftwarelinguisticsconventions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the small, fixed set of grammatical words programmers sometimes include in identifier names—prepositions, determiners, conjunctions, and numerals—are not accidental filler but deliberate tools for encoding program behavior. Using a new manually annotated dataset of 1,275 identifiers drawn from 30 open-source systems, it shows that these terms systematically express ideas such as control flow, data transformation, timing, and entity selection. The study matters because naming research has focused almost entirely on nouns and verbs; this work gives the first empirical account of how the rare closed-category words behave and what role they play in program comprehension.

What carries the argument

The central object is the grammar pattern: a sequence of part-of-speech (PoS) tags abstracting an identifier's phrasal structure, e.g., P N for "to index" or N P N for "action to index". The paper constructs the Closed Category Identifier Dataset (CCID), a manually annotated corpus that deliberately oversamples identifiers containing preposition, determiner, conjunction, and numeral tags, and uses grounded-theory-style open, axial, and selective coding to link these patterns to behavioral roles. The grammar pattern is what allows the analysis to generalize across surface names and reveals whether a closed-category term forms a unary relation (modifying a single noun) or a binary relation (connecting two operands).

What would settle it

A comprehension experiment where developers are shown alternative names for the same variable (e.g., textures vs find all textures) and asked to describe the program's behavior would settle the core claim: if closed-category terms do not measurably improve accuracy or speed of behavioral inference, their deliberate status is weakened. A simpler check is whether the axial-code framework itself is unstable: asking a second independent team to re-code the CCID identifiers should reproduce high agreement (near the reported Fleiss' kappa of .916–1.0) on the behavioral categories; if the categories diverge substantially, the grounded theory is not robust.

Watch

Extended reading notes

Core claim

The central claim is that closed-category terms (prepositions, determiners, conjunctions, numerals) in identifier names are deliberate and behaviorally meaningful, serving functions that map onto program semantics: prepositions express transformation, direction, conditionality, and event triggers; determiners encode position in a sequence, population membership, and exclusion; numerals act as compact indices, version markers, or domain-specific codes; conjunctions encode guarded actions, data pairs, and logical alternatives. The paper demonstrates this through grounded-theory-inspired coding of 1,001 verified identifiers, revealing recurring axial codes such as Type Casting/Interpretation, Boolean Flow, and Temporal/Upcoming Element, and statistically significant correlations with code context: prepositions and conjunctions appear disproportionately in function names, numerals in parameters and class names, and determiners rarely in class names.

Load-bearing premise

The behavioral axial codes—categories like Type Casting and Boolean Flow—are treated as facts about developer intent even though they were inferred by four annotators reading identifier names and surrounding code, validated only by inter-rater agreement rather than by any direct measure of what programmers actually meant.

Editorial extensions

If this is right

  • Naming tools and linters could use grammar patterns to flag unusual or context-mismatched closed-category usage, prompting developers to reflect on their naming choices.
  • Automated code generation and large language models could be scaffolded to emit identifiers that instantiate common closed-category patterns (e.g., DT NM N, P N), better aligning generated names with human conventions.
  • Educators could teach closed-category terms as part of a naming lexicon, giving students behavior-specific guidance beyond generic advice like 'use descriptive names'.
  • The axial-code framework provides a schema for studying how naming choices correlate with code quality, comprehension, and maintainability in follow-up studies.
  • The finding that determiners like next and last serve a determinative rather than adjectival function suggests that standard PoS tagging of identifiers should treat sequence-positions as determiners for better semantic analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If closed-category terms are as meaningful as this study argues, then renaming analyses could treat a change from e.g. lastNode to previousNode as semantically near-synonymous, while a change from lastNode to someNode shifts the identifier's behavioral role—an insight that could sharpen automated refactoring recommendations.
  • The dual-axis framework for numerals (Role x Source of Meaning) hints that numeral-heavy identifiers such as m33 are only comprehensible to developers who share a local convention; tools that detect undocumented numerals and prompt for a comment or a rename could reduce comprehension barriers in domains with heavy matrix or coordinate usage.
  • The paper's own domain-selection logic suggests a testable extension: comprehension experiments comparing identifiers with and without closed-category terms (e.g., findTextures vs find all textures) could measure whether the behavioral specificity actually improves recall and understanding, a claim the paper asserts but does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies closed syntactic category words (prepositions, determiners, conjunctions, numerals) in identifier names. It constructs the Closed Category Identifier Dataset (CCID) by sampling 1,275 candidate identifiers from 30 open-source C/C++/Java systems and manually verifying and annotating 1,001 of them with part-of-speech grammar patterns. RQ1 applies a grounded-theory-inspired coding procedure (open, axial, and selective coding) to a subset of 618 identifiers following the top-10 grammar patterns, yielding behavioral taxonomies for each category (e.g., Type Casting / Interpretation, Boolean Flow / Control Flag, Temporal / Most Recent Element) and a cross-category synthesis. RQ2 tests correlations between closed-category tag usage and programming language, source-code context, and system domain, using Pearson chi-square tests with standardized residuals and Mann-Whitney U tests over a support-threshold sweep. The paper concludes that closed-category terms are deliberate, behaviorally meaningful components of identifier names and derives implications for naming tools, education, and future research.

Significance. The contribution is valuable: the CCID is a new manually annotated dataset, and the paper provides one of the first detailed qualitative taxonomies of closed-category terms in identifiers. The high inter-rater agreement (Fleiss' kappa 0.916-1.0) supports the reliability of the annotation scheme as applied, and the authors ship scripts and data for reproducibility. The paper is also well situated in prior grammar-pattern and part-of-speech-tagging work, and the RQ2 context correlations, if reproducible, offer concrete hypotheses about naming conventions. However, the significance is conditional: the dataset counts are internally inconsistent, and the central claim that closed-category terms are 'deliberate, behaviorally meaningful' rests on interpretive coding without external validation of developer intent or reader comprehension.

major comments (4)
  1. [§5.1, §10] The paper's headline conclusion—that closed-category terms are 'deliberate, behaviorally meaningful tools'—rests entirely on behavioral axial codes assigned by four co-trained annotators who examined identifiers in their source context. Fleiss' kappa shows the coding scheme was applied consistently, but it does not show that the codes capture developers' actual intent or that readers reliably recover the same meanings; the coding step is an interpretation, not a measurement of intent. The paper itself concedes in §10 that 'further validation of the behavioral categories ... especially through comprehension studies' is needed. Since RQ2's domain analysis (§6.1.3, Table 13) then uses these axial codes to select systems, the interpretive superstructure inherits this validity gap. Please either add an external validation (e.g., a developer survey, commit-intent analysis, or a comprehension experiment) or rewrite the conclusions and RQ2 framing to treat the behavioral codes as hypotheses rather than established developer intent.
  2. [§4.1, Tables 5 and 6] The size and composition of the CCID are presented inconsistently. The abstract and §4.1 describe the dataset as 1,275 identifiers; the text later states that the final CCID consists of 1,001 verified identifiers; and Table 6 reports a 'Balanced population' of 1,275 while being introduced as 'the CCID, but broken down by program context.' Table 5 also shows verified P totals (382) exceeding candidate P totals (364), which is impossible if verified identifiers are a subset of the candidate pool, and the note about tag-level versus identifier-level counting does not resolve the discrepancy. Because §6.1.2 says the RQ2 chi-square analyses use all 1,001 identifiers, the context totals in Tables 5 and 6 must be reconciled and clearly labeled as pre-verification candidates, verified identifier-level counts, or tag-level counts. Without this, the counts underlying Table 11 cannot be reproduced or checked.
  3. [§5.3–§5.5] Several axial codes are inferred from very few examples: Mathematical / Constraint Context has 2 items (§5.3); Quantity Threshold / Optional Extensibility has 4 items and Default / Fallback Value Representation has 2 items (§5.4); and Shared Interface for Alternatives, Combined Configuration / UI Concept, Boolean Concept Name, and Boolean Multi-Condition Test each have 1 item (§5.5). The selective-coding synthesis in §5.6 nevertheless presents these as recurring behavioral roles and uses them to support the general conclusion. For categories with single-digit counts, the paper should present them as preliminary observations requiring larger samples, or visibly down-weight them in the cross-category conclusions; the current wording overstates the support these sparse codes provide.
  4. [§6.1.3, Figures 2–4] The domain comparison is not as strong as the text suggests. The domain-specific systems in Table 13 were selected using the paper's own axial codes, so the comparison partly re-tests the taxonomy's internal assumptions. In addition, the Mann-Whitney threshold sweep is performed at many thresholds without multiple-comparison correction, which inflates the chance of finding 'significant' peaks; the tests also use predefined closed-term lexicons without verifying the grammatical role of each term in context, as the authors acknowledge in §6.1.3. The claim that 'domain-specific systems use closed-category terms more frequently than general-purpose ones' should be labeled exploratory, or the analysis should be augmented with a correction (e.g., Bonferroni or permutation-based) and with a domain selection that is not derived from the same coding that motivates the studied categories.
minor comments (5)
  1. [§4.1] The term 'CCID corpus' is used for the 279,000-identifier source set and later 'CCID' is used for the 1,001-identifier annotated dataset; rename the source corpus (e.g., 'source corpus') to avoid conflating the two artifacts.
  2. [§5.2] The m34 example first says 'matrix row 3 col 3' but then explains that 3 refers to the row index and 4 refers to the column; align the example with the intended interpretation.
  3. [§8, §10, §5.2] Typos should be corrected: §8 'simialar' → 'similar'; §10 'beahvoral' → 'behavioral'; §5.2 'collissions' → 'collisions'; several headers and table entries contain stray spaces (e.g., 'T ags', 'T echnology', 'V ersion').
  4. [§6.1.2, Tables 9–12] The raw counts used in the chi-square tests should be reported or linked unambiguously to a single reconciled dataset table, since Table 11's df=12 implies a 4×5 table and the residual values depend on which context totals (Table 5 or Table 6) were used.
  5. [§6.1.3, Figures 2–4] The threshold-sweep figures would be easier to interpret with the number of retained systems at each threshold and ideally pointwise confidence intervals; currently the reader cannot tell how quickly the system sample shrinks at high support thresholds.

Circularity Check

2 steps flagged · score 3.0 of 10

Partial interpretive circularity: RQ2's domain validation selects systems using the paper's own RQ1 axial codes and reads the result back through them, while RQ1 selective codes 'explain' correlations in the same dataset; measured frequencies are external and falsifiable, so the claim is not forced, but the behavioral categories lack independent validation.

  1. self definitional [Section 6.1.3, methodology paragraph and Table 13 (system selection); interpretation paragraph]
    "In RQ1, we developed a set of Axial Codes to describe the behavioral roles of closed-category terms in identifiers. To explore their importance at the level of system domain, we selected the two most common Axial Codes from each closed-category group (e.g., Prepositions, Determiners). For each code, we identified two software domains that we hypothesized would frequently use identifiers expressing that behavior."

    RQ1's axial codes were induced by the paper's own annotators from identifiers in the 30 general-purpose systems (Table 3) that also serve as the RQ2 baseline. The domain-specific systems in Table 13 are selected precisely because they were expected to instantiate those codes, and the significant Mann-Whitney result is interpreted through the same codes ('to express structural or behavioral distinctions central to their design'). Selection and interpretation are thus defined in terms of the very constructs whose external validity is open, validated only by Fleiss' kappa among co-trained annotators. The measured term-frequency difference itself is code-independent and uses new repositories, so this is a partial self-grounding, not a by-construction identity.

  2. other [Section 6.1.2, Conjunctions paragraph (same move repeated for Determiners, Numerals, Prepositions)]
    "The selective coding data from RQ1 explains this pattern. Conjunction-based grammar patterns tend to express compound logic, dual-purpose behavior, or guarded activation, which are most relevant when naming behaviors or actions rather than static values."

    The chi-square test finds conjunctions correlate with function names in the CCID; the 'explanation' comes from RQ1's selective codes, produced by four annotators from the 618 top-10-pattern identifiers — a subset of the same CCID whose correlations are explained — with 'Source Code Context' visible during coding. The same move recurs for determiners, numerals, and prepositions. The codes therefore encode the same context regularities the statistics later measure; citing them as the explanation is an endogenous reading of one dataset, not independent evidence that the correlation reflects developer intent. This is qualitative self-consistency rather than a by-construction reduction, so it is a weak circularity, but the RQ2 interpretive layer cannot validate RQ1's behavioral claims.

full rationale

This is an observational, mixed-methods study rather than a predictive derivation, so the classic circularity patterns — fitted parameters renamed as predictions, uniqueness theorems imported from the authors' prior work — do not apply. There is no fitted model: the chi-square and Mann-Whitney statistics measure observable quantities (closed-category tag frequencies in the CCID and per-LOC term counts in external repositories), and the language and context correlations would stand regardless of how the behavioral codes are labeled. Citations of prior work by the same authors are descriptive and non-load-bearing: the grammar-pattern framework builds on Newman et al. [55], and the numeral framework is explicitly 'inspired by a single-axis framework we created in prior work' [65], with the dual-axis scheme attributed to the present study's own observation. The genuine circularity pressure is interpretive and sits in two steps. First, the RQ2 domain comparison (Section 6.1.3): the domain-specific systems in Table 13 are selected on the hypothesis that they instantiate the paper's own RQ1 axial codes, which were induced from identifiers drawn from the very general-purpose baseline systems used in the comparison; the significant difference is then read back through those same codes ('to express structural or behavioral distinctions central to their design'). The measured term frequencies are code-independent and the repositories are new, so this is a partial self-grounding rather than a by-construction identity. Second, RQ2's significant context correlations are 'explained' using RQ1 selective codes generated from a subset of the same CCID identifiers, with the source context visible to annotators — an endogenous interpretation that cannot independently confirm that the correlations reflect developer intent rather than shared annotator projection. The paper itself concedes the load-bearing gap in Section 10, deferring 'further validation of the behavioral categories we propose, especially through comprehension studies' to future work, and Section 6.1.3 admits the term lists were used 'without verifying each term's function in context.' Internal inconsistencies — the CCID is stated as 1,001 verified identifiers while the abstract and Table 6 report 1,275, and the support-threshold sweep in Figures 2-4 is reported without multiple-comparison correction — are validity and transparency risks, not circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 1 invented entities

No parameters are fitted. The study's claims rest on transfer of English closed-class categories to code, a custom reclassification of next/last as determiners, and interpretive coding validated only by annotator consensus. The empirical statistics use standard significance thresholds; the domain comparison selects systems based on the paper's own axial codes, which is a potential source of confirmation bias.

assumptions (5)
  • domain assumption English closed-class lexicons (for determiners, conjunctions, prepositions) from external lists are transferable to source-code identifiers.
    Used in Phase 1 filtering (Section 4.1) to create candidate sets; if list coverage is poor, candidate sets and prevalence estimates change.
  • ad hoc to paper The reclassification of 'next', 'last', etc. as determiners rather than adjectives in source-code context.
    Section 5.4 states this explicitly; the choice increases determiner counts and shapes the determiner axial codes. Standard English PoS would tag these as adjectives.
  • domain assumption Annotators can infer developer intent and behavior from identifier plus source context, and consensus (Fleiss' kappa) is a valid ground truth.
    Section 5.1; the entire RQ1 axial code scheme rests on this interpretive assumption.
  • domain assumption srcML context classification and Spiral splitting are sufficiently accurate, with errors manually corrected during annotation.
    Sections 4.1 and 4.2; used to build the identifier corpus and context labels.
  • domain assumption The 30 open-source systems curated by Reaper (with CI and tests) are representative of production C, C++, and Java code.
    Section 4; external validity depends on this, and the paper acknowledges it as a threat.
invented entities (1)
  • Behavioral axial code categories (e.g., Distinguisher x Human-Named Convention, Boolean Flow / Control Flag, Temporal / Most Recent Element)
    purpose: Conceptual framework for labeling the behavioral role of closed-category terms in identifiers
    These categories are derived in this paper via grounded-theory coding and are validated only by inter-rater agreement; no external falsifiable handle is provided outside the dataset. They are the paper's main qualitative contribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Structure and Semantics of Identifier Names Containing Closed Syntactic Category Words." pith.science (2026). https://pith.science/paper/WWLXTGMT

@misc{pith2026250518444,
  author       = {Pith},
  title        = {Pith review of: On the Structure and Semantics of Identifier Names Containing Closed Syntactic Category Words},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WWLXTGMT}},
  note         = {Machine review of arXiv:2505.18444}
}
read the original abstract

Identifier names are crucial components of code, serving as primary clues for developers to understand program behavior. This paper investigates the linguistic structure of identifier names by extending the concept of grammar patterns, which represent the part-of-speech (PoS) sequences underlying identifier phrases. The specific focus is on closed syntactic categories (e.g., prepositions, conjunctions, determiners), which are rarely studied in software engineering despite their central role in general natural language. To study these categories, the Closed Category Identifier Dataset (CCID), a new manually annotated dataset of 1,275 identifiers drawn from 30 open-source systems, is constructed and presented. The relationship between closed-category grammar patterns and program behavior is then analyzed using grounded-theory-inspired coding, statistical, and pattern analysis. The results reveal recurring structures that developers use to express concepts such as control flow, data transformation, temporal reasoning, and other behavioral roles through naming. This work contributes an empirical foundation for understanding how linguistic resources encode behavior in identifier names and supports new directions for research in naming, program comprehension, and education.

Figures

Figures reproduced from arXiv: 2505.18444 by the authors.

Figure 1
Figure 1. Examples of noun, verb, and prepositional phrases NM N, represents the identifier’s grammar pattern. Crucially, this pattern general￾izes across many identifiers: RunUserQuery and WriteAccessToken share the same structure, despite using different terms. Grammar patterns thus allow us to relate identifiers by their syntactic form. We focus specifically on closed-category grammar patterns, which are patterns that cont… view at source ↗
Figure 2
Figure 2. Global Mann-Whitney U test significance across thresholds, showing divergence be￾tween domain-specific and general systems. Peaks at 0.6 and 0.8 suggest the importance of both ubiquitous and moderately specific closed-category terms [PITH_FULL_IMAGE:figures/full_fig_p030_2.png] view at source ↗
Figure 3
Figure 3. Per-category Mann-Whitney U test significance across thresholds. Prepositions domi￾nate across thresholds, while conjunctions and numerals contribute more variably. 6.1.3 Closed-Category Term Usage Across System Domains Having established correlations between closed-category terms, source code con￾text, and programming language, we now turn to a broader question: do these terms also vary with the domain of the softw… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Cliff’s Delta for closed-category terms across system support thresholds [PITH_FULL_IMAGE:figures/full_fig_p031_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 51 canonical work pages

  1. [1]

    In: Proceedings of the 2011 18th Working Conference on Reverse Engineering, WCRE ’11, p

    Abebe, S.L., Tonella, P.: Towards the extraction of domain concepts from the identifiers. In: Proceedings of the 2011 18th Working Conference on Reverse Engineering, WCRE ’11, p. 77–86. IEEE Computer Society, USA (2011). DOI 10.1109/WCRE.2011.19. URL https://doi.org/10.1109/WCRE.2011.19

    work page doi:10.1109/wcre.2011.19 2011

  2. [2]

    In: 2013 17th European Conference on Software Maintenance and Reengineering, pp

    Abebe, S.L., Tonella, P.: Automated identifier completion and replacement. In: 2013 17th European Conference on Software Maintenance and Reengineering, pp. 263–272 (2013). DOI 10.1109/CSMR.2013.35

    work page doi:10.1109/csmr.2013.35 2013

  3. [3]

    Empirical Software Engineering 16, 487–513 (2011)

    Adolph, S., Hall, W., Kruchten, P.: Using grounded theory to study the experience of software development. Empirical Software Engineering 16, 487–513 (2011). DOI 10.1007/ s10664-010-9152-6

    work page 2011

  4. [4]

    In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, ASE ’22

    Al Madi, N.: Namesake: A checker of lexical similarity in identifier names. In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, ASE ’22. Association for Computing Machinery, New York, NY, USA (2023). DOI 10. 1145/3551349.3560441. URL https://doi.org/10.1145/3551349.3560441

    arXiv 2023

  5. [5]

    In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pp

    Allamanis, M., Barr, E.T., Bird, C., Sutton, C.: Suggesting accurate method and class names. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pp. 38–49. ACM, New York, NY, USA (2015). DOI 10.1145/2786805.2786849. URL http://doi.acm.org/10.1145/2786805.2786849

    arXiv 2015

  6. [6]

    In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp

    Alsuhaibani, R., Newman, C., Decker, M., Collard, M., Maletic, J.: On the naming of methods: A survey of professional developers. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 587–599 (2021). DOI 10.1109/ICSE43902. 2021.00061

    work page arXiv 2021

  7. [7]

    In: 2015 IEEE 5th Workshop on Mining Unstructured Data (MUD), pp

    Alsuhaibani, R.S., Newman, C.D., Collard, M.L., Maletic, J.I.: Heuristic-based part-of- speech tagging of source code identifiers and comments. In: 2015 IEEE 5th Workshop on Mining Unstructured Data (MUD), pp. 1–6 (2015). DOI 10.1109/MUD.2015.7327960

    work page arXiv 2015

  8. [8]

    Aman, H., Amasaki, S., Yokogawa, T., Kawahara, M.: A quantitative investigation of trends in confusing variable pairs through commits: Do confusing variable pairs survive? In: Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, EASE ’24, p. 90–99. Association for Computing Machinery, New York, NY, USA (20...

    arXiv 2024

Show all 75 references
  1. [9]

    In: 2013 17th European Conference on Software Maintenance and Reengineering, pp

    Arnaoudova, V., Di Penta, M., Antoniol, G., Gu´ eh´ eneuc, Y.: A new family of software anti-patterns: Linguistic anti-patterns. In: 2013 17th European Conference on Software Maintenance and Reengineering, pp. 187–196 (2013). DOI 10.1109/CSMR.2013.28

    2013 doi

  2. [10]

    IEEE Trans

    Arnaoudova, V., Eshkevari, L.M., Penta, M.D., Oliveto, R., Antoniol, G., Gueheneuc, Y.G.: Repent: Analyzing the nature of identifier renamings. IEEE Trans. Softw. Eng. 40(5), 502–532 (2014). DOI 10.1109/TSE.2014.2312942. URL https://doi.org/10.1109/ TSE.2014.2312942

    2014

  3. [11]

    In: 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC), pp

    Avidan, E., Feitelson, D.G.: Effects of variable names on comprehension: An empirical study. In: 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC), pp. 55–65 (2017). DOI 10.1109/ICPC.2017.27

    2017 doi

  4. [12]

    In: Proceedings of the 8th Working Conference on Mining Software Repositories, MSR ’11, pp

    Binkley, D., Hearn, M., Lawrie, D.: Improving identifier informativeness using part of speech information. In: Proceedings of the 8th Working Conference on Mining Software Repositories, MSR ’11, pp. 203–206. ACM, New York, NY, USA (2011). DOI 10.1145/ 1985441.1985471. URL http...

    2011

  5. [13]

    Empirical Softw

    Binkley, D., Lawrie, D., Morrell, C.: The need for software specific natural language tech- niques. Empirical Softw. Engg. 23(4), 2398–2425 (2018). DOI 10.1007/s10664-017-9566-5. URL https://doi.org/10.1007/s10664-017-9566-5

    2018 doi

  6. [14]

    In: 2015 IEEE 23rd International Conference on Program Comprehension, pp

    Butler, S., Wermelinger, M., Yu, Y.: A survey of the forms of java reference names. In: 2015 IEEE 23rd International Conference on Program Comprehension, pp. 196–206 (2015). DOI 10.1109/ICPC.2015.30

    2015 doi

  7. [15]

    In: 2009 16th Working Conference on Reverse Engineering, pp

    Butler, S., Wermelinger, M., Yu, Y., Sharp, H.: Relating identifier naming flaws and code quality: An empirical study. In: 2009 16th Working Conference on Reverse Engineering, pp. 31–35 (2009). DOI 10.1109/WCRE.2009.50

    2009 doi

  8. [16]

    In: Software Maintenance and Reengineering (CSMR), 2010 14th European Conference on, pp

    Butler, S., Wermelinger, M., Yu, Y., Sharp, H.: Exploring the influence of identifier names on code quality: An empirical study. In: Software Maintenance and Reengineering (CSMR), 2010 14th European Conference on, pp. 156–165. IEEE (2010)

    2010

  9. [17]

    In: 2011 27th IEEE International Conference on Software Maintenance (ICSM), pp

    Butler, S., Wermelinger, M., Yu, Y., Sharp, H.: Mining java class naming conventions. In: 2011 27th IEEE International Conference on Software Maintenance (ICSM), pp. 93–102 (2011). DOI 10.1109/ICSM.2011.6080776 46 Christian D. Newman et al

    2011

  10. [18]

    In: Proceedings 2000 Interna- tional Conference on Software Maintenance, pp

    Caprile, Tonella: Restructuring program identifier names. In: Proceedings 2000 Interna- tional Conference on Software Maintenance, pp. 97–107 (2000). DOI 10.1109/ICSM.2000. 883022

    2000 doi

  11. [19]

    In: Sixth Working Conference on Reverse Engineering (Cat

    Caprile, C., Tonella, P.: Nomen est omen: analyzing the language of function identifiers. In: Sixth Working Conference on Reverse Engineering (Cat. No.PR00303), pp. 112–122 (1999). DOI 10.1109/WCRE.1999.806952

    1999

  12. [20]

    In: 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp

    Collard, M.L., Maletic, J.I.: srcml 1.0: Explore, analyze, and manipulate source code. In: 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 649–649 (2016). DOI 10.1109/ICSME.2016.36

    2016 doi

  13. [21]

    IBM Systems Journal 28(2), 294–306 (1989)

    Corbi, T.A.: Program understanding: Challenge for the 1990s. IBM Systems Journal 28(2), 294–306 (1989). DOI 10.1147/sj.282.0294

    1989 doi

  14. [22]

    Qualitative Sociology 19(6), 3–21 (1990)

    Corbin, J., Strauss, A.: Grounded theory research: Procedures, canons and evaluative crite- ria. Qualitative Sociology 19(6), 3–21 (1990). DOI https://doi.org/10.1007/BF00988593. URL https://doi.org/10.1515/zfsoz-1990-0602

    1990 doi

  15. [23]

    Software Quality Journal 14(3), 261–282 (2006)

    Deissenboeck, F., Pizka, M.: Concise and consistent naming. Software Quality Journal 14(3), 261–282 (2006). DOI 10.1007/s11219-006-9219-1. URL https://doi.org/10.1007/ s11219-006-9219-1

    2006 doi

  16. [24]

    In: In Proceedings of the 3rd International Workshop on Metamodels, Schemas, Grammars and Ontologies (ATEM’06 (2006)

    Deissenboeck, F., Ratiu, D.: A unified meta-model for concept-based reverse engineering. In: In Proceedings of the 3rd International Workshop on Metamodels, Schemas, Grammars and Ontologies (ATEM’06 (2006)

    2006

  17. [25]

    In: Proceedings of the 22Nd IEEE International Conference on Software Maintenance, ICSM ’06, pp

    Dragan, N., Collard, M.L., Maletic, J.I.: Reverse engineering method stereotypes. In: Proceedings of the 22Nd IEEE International Conference on Software Maintenance, ICSM ’06, pp. 24–34. IEEE Computer Society, Washington, DC, USA (2006). DOI 10.1109/ ICSM.2006.54. URL http://dx...

    2006 doi

  18. [26]

    In: Proceedings of the 30th IEEE/ACM Inter- national Conference on Program Comprehension, ICPC ’22, p

    Etgar, A., Friedman, R., Haiman, S., Perez, D., Feitelson, D.G.: The effect of information content and length on name recollection. In: Proceedings of the 30th IEEE/ACM Inter- national Conference on Program Comprehension, ICPC ’22, p. 141–151. Association for Computing Machine...

    2022

  19. [27]

    Empirical Softw

    Fakhoury, S., Roy, D., Ma, Y., Arnaoudova, V., Adesope, O.: Measuring the impact of lexical and structural inconsistencies on developers’ cognitive load during bug localization. Empirical Softw. Engg. 25(3), 2140–2178 (2020). DOI 10.1007/s10664-019-09751-4. URL https://doi.org...

    2020 doi

  20. [28]

    In: Proceedings of the 2010 IEEE 18th International Conference on Program Comprehension, ICPC ’10, p

    Falleri, J.R., Huchard, M., Lafourcade, M., Nebut, C., Prince, V., Dao, M.: Automatic extraction of a wordnet-like identifier network from software. In: Proceedings of the 2010 IEEE 18th International Conference on Program Comprehension, ICPC ’10, p. 4–13. IEEE Computer Societ...

    2010 doi

  21. [29]

    IET Software 2(1), 27–36 (2008)

    Fry, Z.P., Shepherd, D., Hill, E., Pollock, L., Vijay-Shanker, K.: Analysing source code: looking for useful verb-direct object pairs in all the right places. IET Software 2(1), 27–36 (2008). DOI 10.1049/iet-sen:20070112

    2008 doi

  22. [30]

    Proceedings of the 28th Annual ACM Symposium on User Inter- face Software & Technology (2015)

    Glassman, E.L., Fischer, L., Scott, J., Miller, R.: Foobaz: Variable name feedback for student code at scale. Proceedings of the 28th Annual ACM Symposium on User Inter- face Software & Technology (2015). URL https://api.semanticscholar.org/CorpusID: 15810023

    2015

  23. [31]

    In: 2013 21st International Conference on Program Comprehension (ICPC), pp

    Gupta, S., Malik, S., Pollock, L., Vijay-Shanker, K.: Part-of-speech tagging of program identifiers for improved text-based software engineering tools. In: 2013 21st International Conference on Program Comprehension (ICPC), pp. 3–12 (2013). DOI 10.1109/ICPC. 2013.6613828

    2013

  24. [32]

    In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018, p

    Hellendoorn, V.J., Bird, C., Barr, E.T., Allamanis, M.: Deep learning type inference. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2018, p. 152–162. Association...

    2018

  25. [33]

    Hill, E.: Integrating natural language and program structure information to improve soft- ware search and exploration. Ph.D. thesis, Newark, DE, USA (2010). AAI3423409

    2010

  26. [34]

    In: Proceedings of the 2008 International Working Conference on Mining Software Repositories, MSR ’08, p

    Hill, E., Fry, Z.P., Boyd, H., Sridhara, G., Novikova, Y., Pollock, L., Vijay-Shanker, K.: Amap: Automatically mining abbreviation expansions in programs to enhance software maintenance tools. In: Proceedings of the 2008 International Working Conference on Mining Software Repo...

    2008

  27. [35]

    IEEE Transactions on Software Engineering 48(10), 3808–3832 (2022)

    Hoda, R.: Socio-technical grounded theory for software engineering. IEEE Transactions on Software Engineering 48(10), 3808–3832 (2022). DOI 10.1109/TSE.2021.3106280

    2022

  28. [36]

    In: 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp

    Hofmeister, J., Siegmund, J., Holt, D.V.: Shorter identifier names take longer to compre- hend. In: 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 217–227 (2017). DOI 10.1109/SANER.2017.7884623

    2017

  29. [37]

    Host, E., Ostvold, B.: The programmer’s lexicon, volume i: The verbs. pp. 193 – 202 (2007). DOI 10.1109/SCAM.2007.18

    2007 doi

  30. [38]

    Høst, E.W.: Meaningful method names (2011)

    2011

  31. [39]

    In: Proceedings of the 23rd Euro- pean Conference on ECOOP 2009 — Object-Oriented Programming, Genoa, pp

    Høst, E.W., Østvold, B.M.: Debugging method names. In: Proceedings of the 23rd Euro- pean Conference on ECOOP 2009 — Object-Oriented Programming, Genoa, pp. 294–317. Springer-Verlag, Berlin, Heidelberg (2009). DOI 10.1007/978-3-642-03013-0 14. URL http://dx.doi.org/10.1007/978...

    2009 doi

  32. [40]

    Høst, E.W., Østvold, B.M.: The java programmer’s phrase book. In: D. Gaˇ sevi´ c, R. L¨ ammel, E. Van Wyk (eds.) Software Language Engineering, pp. 322–341. Springer Berlin Heidelberg, Berlin, Heidelberg (2009)

    2009

  33. [41]

    Høst, E.W., Østvold, B.M.: Canonical method names for java. In: B. Malloy, S. Staab, M. van den Brand (eds.) Software Language Engineering, pp. 226–245. Springer Berlin Heidelberg, Berlin, Heidelberg (2011)

    2011

  34. [42]

    Journal of Open Source Software 3, 653 (2018)

    Hucka, M.: Spiral: splitters for identifiers in source code files. Journal of Open Source Software 3, 653 (2018). DOI 10.21105/joss.00653

    2018 doi

  35. [43]

    In: 2014 Software Evolu- tion Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engi- neering (CSMR-WCRE), pp

    Kashiwabara, Y., Onizuka, Y., Ishio, T., Hayase, Y., Yamamoto, T., Inoue, K.: Recom- mending verbs for rename method using association rule mining. In: 2014 Software Evolu- tion Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engi- neering (CSMR-WCRE...

    2014

  36. [44]

    In: 14th IEEE International Conference on Program Comprehension (ICPC’06), pp

    Lawrie, D., Morrell, C., Feild, H., Binkley, D.: What’s in a name? a study of identifiers. In: 14th IEEE International Conference on Program Comprehension (ICPC’06), pp. 3–12 (2006). DOI 10.1109/ICPC.2006.51

    2006 doi

  37. [45]

    In: In Proc

    Liblit, B., Begel, A., Sweetser, E.: Cognitive perspectives on the role of naming in computer programs. In: In Proc. of the 18th Annual Psychology of Programming Workshop (2006)

    2006

  38. [46]

    IEEE Transactions on Software Engineering 41(9), 887– 900 (2015)

    Liu, H., Liu, Q., Liu, Y., Wang, Z.: Identifying renaming opportunities by expanding conducted rename refactorings. IEEE Transactions on Software Engineering 41(9), 887– 900 (2015)

    2015

  39. [47]

    In: Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on, pp

    Liu, H., Liu, Q., Staicu, C.A., Pradel, M., Luo, Y.: Nomen est omen: Exploring and exploiting similarities between argument and parameter names. In: Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on, pp. 1063–1073. IEEE (2016)

    2016

  40. [48]

    Bissyand´ e, T., Kim, T., Kim, K., Koyuncu, A., Kim, S., Le Traon, Y.: Learning to spot and refactor inconsistent method names

    Liu, K., Kim, D., F. Bissyand´ e, T., Kim, T., Kim, K., Koyuncu, A., Kim, S., Le Traon, Y.: Learning to spot and refactor inconsistent method names. In: Proceedings of the 40th International Conference on Software Engineering, ICSE 2019. ACM, New York, NY, USA (2019)

    2019

  41. [49]

    In: Proceedings of the 41st International Conference on Software Engineering, ICSE ’19, p

    Malik, R.S., Patra, J., Pradel, M.: Nl2type: Inferring javascript function types from natural language information. In: Proceedings of the 41st International Conference on Software Engineering, ICSE ’19, p. 304–315. IEEE Press (2019). DOI 10.1109/ICSE.2019.00045. URL https://d...

    2019

  42. [50]

    Prentice Hall PTR, Upper Saddle River, NJ, USA (2008)

    Martin, R.C.: Clean Code: A Handbook of Agile Software Craftsmanship, 1 edn. Prentice Hall PTR, Upper Saddle River, NJ, USA (2008)

    2008

  43. [51]

    Communications of the ACM 38(11), 39–41 (1995)

    Miller, G.A.: Wordnet: a lexical database for english. Communications of the ACM 38(11), 39–41 (1995)

    1995

  44. [52]

    Empirical Software Engineering 22(6), 3219–3253 (2017)

    Munaiah, N., Kroh, S., Cabrey, C., Nagappan, M.: Curating github for engineered software projects. Empirical Software Engineering 22(6), 3219–3253 (2017). DOI 10.1007/s10664-017-9512-6. URL https://doi.org/10.1007/s10664-017-9512-6

    2017 doi

  45. [53]

    [Online]

    Newman, C., Decker, M., Alsuhaibani, R.: Identifier name structure catalogue URL https://github.com/SCANL/identifier_name_structure_catalogue. [Online]. Avail- able: https://github.com/SCANL/identifier name structure catalogue

  46. [54]

    In: 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp

    Newman, C.D., AlSuhaibani, R.S., Collard, M.L., Maletic, J.I.: Lexical categories for source code identifiers. In: 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 228–239 (2017). DOI 10.1109/SANER.2017. 7884624

    2017 doi

  47. [55]

    Journal of Systems and Software 170, 110740 (2020)

    Newman, C.D., AlSuhaibani, R.S., Decker, M.J., Peruma, A., Kaushik, D., Mkaouer, M.W., Hill, E.: On the generation, structure, and semantics of grammar patterns in source code identifiers. Journal of Systems and Software 170, 110740 (2020). DOI https://doi. org/10.1016/j.jss.2...

    2020

  48. [56]

    In: Proceedings of the 35th IEEE International Conference on Software Maintenance

    Newman, C.D., Decker, M.J., AlSuhaibani, R.S., Peruma, A., Kaushik, D., Hill, E.: An empirical study of abbreviations and expansions in software artifacts. In: Proceedings of the 35th IEEE International Conference on Software Maintenance. IEEE (2019)

    2019

  49. [57]

    IEEE Transactions on Software Engineer- ing 48(9), 3506–3522 (2022)

    Newman, C.D., Decker, M.J., Alsuhaibani, R.S., Peruma, A., Mkaouer, M.W., Mohapatra, S., Vishnoi, T., Zampieri, M., Sheldon, T.J., Hill, E.: An ensemble approach for annotating source code identifiers with part-of-speech tags. IEEE Transactions on Software Engineer- ing 48(9),...

    2022

  50. [58]

    In: 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp

    Olney, W., Hill, E., Thurber, C., Lemma, B.: Part of speech tagging java method names. In: 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 483–487 (2016). DOI 10.1109/ICSME.2016.80

    2016 doi

  51. [59]

    In: 2022 29th Asia-Pacific Software Engineering Conference (APSEC), pp

    Osumi, Y., Umekawa, N., Komata, H., Hayashi, S.: Empirical study of co-renamed iden- tifiers. In: 2022 29th Asia-Pacific Software Engineering Conference (APSEC), pp. 71–80 (2022). DOI 10.1109/APSEC57359.2022.00019

    2022

  52. [60]

    Journal of Computer Languages 74, 101177 (2023)

    Parsa, S., Zakeri-Nasrabadi, M., Ekhtiarzadeh, M., Ramezani, M.: Method name rec- ommendation based on source code metrics. Journal of Computer Languages 74, 101177 (2023). DOI https://doi.org/10.1016/j.cola.2022.101177. URL https://www. sciencedirect.com/science/article/pii/S...

    2023

  53. [61]

    In: 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC), pp

    Peruma, A., Hu, E., Chen, J., AlOmar, E.A., Mkaouer, M.W., Newman, C.D.: Using grammar patterns to interpret test method name evolution. In: 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC), pp. 335–346 (2021). DOI 10.1109/ICPC52881.2021.00039

    2021

  54. [62]

    In: International Workshop on Refac- toring 2018 (2018)

    Peruma, A., Mkaouer, M.W., Decker, M.J., Newman, C.D.: An empirical investigation of how and why developers rename identifiers. In: International Workshop on Refac- toring 2018 (2018). DOI 10.1145/3242163.3242169. URL http://doi.acm.org/10.1145/ 3242163.3242169

    2018

  55. [63]

    In: Proceedings of the 19th IEEE Interna- tional Working Conference on Source Code Analysis and Manipulation

    Peruma, A., Mkaouer, M.W., Decker, M.J., Newman, C.D.: Contextualizing rename deci- sions using refactorings and commit messages. In: Proceedings of the 19th IEEE Interna- tional Working Conference on Source Code Analysis and Manipulation. IEEE (2019)

    2019

  56. [64]

    Journal of Systems and Software 169, 110704 (2020)

    Peruma, A., Mkaouer, M.W., Decker, M.J., Newman, C.D.: Contextualizing rename de- cisions using refactorings, commit messages, and data types. Journal of Systems and Software 169, 110704 (2020). DOI https://doi.org/10.1016/j.jss.2020.110704. URL http://www.sciencedirect.com/sc...

    2020

  57. [65]

    In: Proceedings of the 1st International Workshop on Natural Language-Based Software Engineering, NLBSE ’22, p

    Peruma, A., Newman, C.D.: Understanding digits in identifier names: An exploratory study. In: Proceedings of the 1st International Workshop on Natural Language-Based Software Engineering, NLBSE ’22, p. 9–16. Association for Computing Machinery, New York, NY, USA (2023). DOI 10...

    2023

  58. [66]

    Ratiu, D., Deissenboeck, F.: Programs are knowledge bases. pp. 79 – 83 (2006). DOI 10.1109/ICPC.2006.41

    2006 doi

  59. [67]

    In: Proceedings of the 15th IEEE International Conference on Program Comprehension, ICPC ’07, p

    Ratiu, D., Deissenboeck, F.: From reality to programs and (not quite) back again. In: Proceedings of the 15th IEEE International Conference on Program Comprehension, ICPC ’07, p. 91–102. IEEE Computer Society, USA (2007). DOI 10.1109/ICPC.2007.22. URL https://doi.org/10.1109/I...

    2007 doi

  60. [68]

    In: Proceedings of the 26th Conference on Program Comprehension, ICPC ’18, pp

    Schankin, A., Berger, A., Holt, D.V., Hofmeister, J.C., Riedel, T., Beigl, M.: Descriptive compound identifier names improve source code comprehension. In: Proceedings of the 26th Conference on Program Comprehension, ICPC ’18, pp. 31–40. ACM, New York, NY, USA (2018). DOI 10.1...

    2018

  61. [69]

    In: Proceedings of the 6th International Conference on Aspect-oriented Software Development, AOSD ’07, pp

    Shepherd, D., Fry, Z.P., Hill, E., Pollock, L., Vijay-Shanker, K.: Using natural language program analysis to locate and understand action-oriented concerns. In: Proceedings of the 6th International Conference on Aspect-oriented Software Development, AOSD ’07, pp. 212–224. ACM...

    2007

  62. [70]

    In: 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation, pp

    Singer, J., Kirkham, C.: Exploiting the correspondence between micro patterns and class names. In: 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation, pp. 67–76 (2008). DOI 10.1109/SCAM.2008.23

    2008 doi

  63. [71]

    DOI 10.1145/2884781.2884833

    Stol, K.J., Ralph, P., Fitzgerald, B.: Grounded theory in software engineering research: A critical review and guidelines (2016). DOI 10.1145/2884781.2884833

    2016

  64. [72]

    Takang, A.A., Grubb, P.A., Macredie, R.D.: The effects of comments and identifier names on program comprehensibility: an experimental investigation. J. Prog. Lang. 4, 143–167 (1996) Title Suppressed Due to Excessive Length 49

    1996

  65. [73]

    Toutanova, K., Manning, C.D.: Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proceedings of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora: Held in Conjunction with the 38th Annu...

    2000

  66. [74]

    In: Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, ICSE-SEET ’24, p

    van der Werf, V., Swidan, A., Hermans, F., Specht, M., Aivaloglou, E.: Teachers’ beliefs and practices on the naming of variables in introductory python programming courses. In: Proceedings of the 46th International Conference on Software Engineering: Software Engineering Educ...

    2024

  67. [75]

    IEEE Transactions on Software Engineering 49(4), 2597–2620 (2023)

    Zhang, J., Liu, S., Gong, L., Zhang, H., Huang, Z., Jiang, H.: Beqain: An effective and efficient identifier normalization approach with bert and the question answering system. IEEE Transactions on Software Engineering 49(4), 2597–2620 (2023). DOI 10.1109/TSE. 2022.3227559

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.