{"id":"b0dd0f1b-48b1-4baf-8623-21b5fc28897a","arxiv_id":"2507.18081","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Manual inspection of five Java projects yields a seven-category taxonomy of similar identifier names and preliminary per-project frequency counts.","lead":"This paper proposes a seven-category taxonomy of similar identifier names in software, based on manual inspection of five open-source Java projects. It reports preliminary frequencies, such as 90.5% of Spring Petclinic's analyzed identifiers falling into some similarity category.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RQ2 prevalence figures are computed over a non-random subset of identifiers selected for suspected similarity; sample-size arguments cannot make them representative of the project populations.","rationale":"The taxonomy itself (RQ1) is supported by concrete, plausible examples and is valuable as an exploratory qualitative contribution; the artifact availability is a point in favor. The load-bearing weakness is in RQ2: the frequency estimates are derived from an undescribed, evidently similarity-directed subset, and the paper's statistical argument is an appeal to sample size rather than to random sampling. The reader's weakest_assumption already names the representativeness of the analyzed identifier subset, so the same concern is identified. The reader's CONDITIONAL verdict is the appropriate disposition: the paper should reframe the frequency table as descriptive of the reviewed subsets and supply the missing sampling and reliability details. My concern does not change that verdict, hence UNCHANGED.","tokens_in":9803,"tokens_out":4974,"duration_ms":52572,"concrete_test":"Reconstruct the subset-selection rule from the artifact at [38]. Specifically, determine whether the analyzed identifiers were sampled uniformly from the extracted inventories or were placed into \"sets of potentially similar identifiers\" based on lexical similarity. If the latter, recompute the variant counts using the full extracted inventory as the denominator (e.g., Petclinic 267/494 = 54.0% vs. 267/295 = 90.5%) or, preferably, classify a fresh simple random sample of 300 identifiers per project and compare Table I. A material change in the percentages would show that the statistical-significance claim did not make the results representative.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section II.C describes the analysis as manual review of \"sets of potentially similar identifiers,\" but never specifies how those sets—or the 295/837/1,167/1,206/724 identifiers counted in Table I—were selected from the extracted inventories in Section II.B (494/15,688/1,697/21,876/7,590). The same section claims the analyzed count \"exceeded the sample size necessary for statistical significance at a 95% confidence level with a 5% margin of error,\" and that this \"ensures that our findings are representative of the overall identifier population in each project.\" This inference is invalid: a large sample only supports population estimates if it is drawn by a probability mechanism. If the subsets were formed by first identifying lexically similar candidates and then classifying them, the variant counts in Table I are systematically inflated—e.g., Spring Petclinic's 267/295 (90.5%) \"variants\" may simply reflect that the 295 were chosen for their similarity, not that 90.5% of the project's identifiers are similar. Because RQ2's prevalence distribution is the paper's stated quantitative contribution (Section I.B), the table should be presented as describing the reviewed subsets, not the projects.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an exploratory taxonomy of identifier name similarities in Java, derived from manual inspection of five open-source projects. Seven categories with subcategories are defined and illustrated with code examples. RQ2 reports per-project counts and percentages of identifiers classified as variants (Table I), and the paper discusses implications for researchers, developers, tool vendors, and educators.","tokens_in":10010,"tokens_out":6231,"duration_ms":67536,"significance":"The qualitative taxonomy is a plausible and potentially useful starting point, and the paper gives concrete code examples for every category, which aids comprehension and reuse. The artifact package linked in the paper is a positive step for reproducibility. If the prevalence results were properly supported, the frequency distribution across projects would be a useful empirical contribution. As it stands, the quantitative claims are not reliable because of an unspecified sampling mechanism, an internally inconsistent ranking, and the absence of reliability measures for the manual classification.","major_comments":[{"comment":"The analyzed counts (295, 837, 1,167, 1,206, 724 identifiers) are presented as sufficient for statistical significance at a 95% confidence level and a 5% margin of error, but Section II.C states that reviewers analyzed 'sets of potentially similar identifiers.' The extracted inventories in Section II.B are much larger (494, 15,688, 1,697, 21,876, 7,590), and no random or representative sampling procedure is described. Without a probability sampling mechanism, the sample-size argument cannot establish representativeness, and the variant percentages in Table I are likely inflated if the subsets were enriched for similarity before classification. The authors should either specify and justify the selection mechanism or explicitly reframe Table I as describing the reviewed candidate subsets rather than the full identifier populations.","section":"Section II.C and Table I"},{"comment":"The text states that Apache Nutch had the lowest percentage of identifier variants, with 93 (12.85%) instances, but Dromara Sureness has 129/1,167 = 11.05%, which is lower. This internal contradiction invalidates the comparative ranking and the adjacent discussion of which projects have low similarity rates. Please correct the statement and re-check all percentages and rankings in Table I.","section":"Section III, Table I"},{"comment":"The taxonomy was iteratively refined and finalized on the same five projects whose identifiers are then counted using that taxonomy in Section III. This makes the frequency results descriptive of the taxonomy-building sample rather than an independent test of the categories. For an exploratory study this circularity is acceptable if clearly acknowledged, but RQ2's phrasing, which asks how frequently categories occur in real-world projects, should be softened to describe the projects used to develop the taxonomy, and the Threats to Validity section should state explicitly that the counts cannot validate the taxonomy categories.","section":"Section II.C"},{"comment":"The unit of counting is ambiguous: the text says the team calculated 'the number of occurrences for each category,' while Table I reports a total 'identifier similarities count' and per-category percentages. Because an identifier or an identifier pair can plausibly belong to more than one category (for example, an abbreviated name that also differs in cardinality), the denominator and the mutual exclusivity of categories need to be defined. Without this, the category percentages cannot be interpreted or reproduced from the reported totals.","section":"Table I"},{"comment":"Section II.C describes independent review and discussion to resolve disagreements, but no inter-rater reliability statistic (e.g., Cohen's kappa) is reported for the manual classification. Given that the taxonomy and all frequency counts depend on subjective semantic interpretation, the absence of a reliability measure substantially weakens confidence in both the categories and the counts. Reporting an agreement statistic on at least a subsample of identifiers would materially strengthen the claims.","section":"Section II.C"}],"minor_comments":[{"comment":"Reference [32] for Spring Petclinic points to the Dromara Sureness repository; it should point to the Spring Petclinic repository.","section":"References"},{"comment":"The text refers to identifiers 'COUNT 2' and 'COUNT 3', but the code shows COUNT_2 and COUNT_3; the text should match the code.","section":"Section III, Listing 11"},{"comment":"Table I uses the label 'Name Collisions' while Section III uses 'Colliding Names' for the same category; the terminology should be consistent throughout.","section":"Table I"},{"comment":"Listing 9 contains 'rolesTmp = null ? null : ...', which is not valid Java; it should be 'rolesTmp == null'.","section":"Section III, Listing 9"},{"comment":"The project name is written 'ThymeLeaf' in one place and 'Thymeleaf' elsewhere; please standardize the spelling.","section":"Section II.C"}],"recommendation":"major_revision","confidential_remarks":"The paper's stated quantitative contribution is currently not reportable as population prevalence because of the unspecified sampling procedure and the internal ranking inconsistency. The qualitative taxonomy, however, is a reasonable exploratory seed for an MSR-style paper. If the authors re-analyze with a proper random sample or reframe the RQ2 results as descriptive of the reviewed subsets, the paper could become acceptable. I would also encourage the editor to remind the authors to fix the reference URL and the code typo before any revision is considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the taxonomy is the real contribution, and it's genuinely useful. The RQ2 prevalence numbers are not to be trusted as project-level estimates, because the analyzed subsets were selected non-randomly and the paper's statistical-significance argument doesn't hold. That said, the paper is clearly written, honest, and a good starting point for the identifier-naming subfield.\n\nWhat's new: the seven-category taxonomy (Standardized Repetitive, Inconsistent Semantic, Colliding, Type-Based, Derivational, Numerically Distinguished, Concise Variants) with subcategories is not something I've seen assembled this way. The examples are concrete and helpful, and the distinction between categories that are risky (colliding, inconsistent, concise) and those that are often intentional (type-based, derivational) is a nice nuance. The paper also acknowledges that not all similarity is a problem, which is more balanced than much of the readability literature.\n\nWhere it gets soft: Section II.C describes analyzing \"sets of potentially similar identifiers\" but never describes how those sets were formed. The extracted inventories are 494 to 21,876 identifiers, and the analyzed counts are 295 to 1,206. Without a sampling procedure, those counts are not a random sample. The claim that the analyzed number exceeds the sample size needed for 95% confidence / 5% margin of error, and therefore is representative, is invalid: a large non-random subset can be arbitrarily unrepresentative. If the subsets were built by first picking lexically similar candidates, then the variant percentages (e.g., 90.5% for Spring Petclinic) are inflated by construction. The paper's own Threats to Validity covers project selection and manual subjectivity but misses this sampling gap. Also, no inter-rater reliability is reported, which matters for a manual classification task.\n\nNote: the reader's report mentioned overlapping categories exceeding 100%; looking at Table I, the category percentages are proportions of the variant count, and the top three sum to under 100% in each project. That concern doesn't hold up.\n\nBottom line: this is a modest exploratory paper with a useful taxonomy and an invalid prevalence-inference. The right fix is to reframe RQ2 as descriptive of the reviewed subsets and either add a real sampling plan or drop the statistical-significance language. It deserves a serious referee, not a desk reject. I'd point students to it for the taxonomy, but I wouldn't cite the prevalence numbers.","headline":"The taxonomy is the real contribution and is genuinely useful, but the RQ2 prevalence numbers are undermined by an invalid sampling-inference argument.","tokens_in":10519,"tokens_out":2281,"would_cite":false,"duration_ms":23050,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A seven-category taxonomy sorts the ways identifier names resemble each other inside a codebase, with frequency counts from five open-source Java projects.","keywords":["identifier names","name similarity","program comprehension","naming taxonomy","code quality","manual classification","open-source Java projects"],"falsifier":"Run the taxonomy on the same five projects with a pre-specified random sample of identifier pairs, two independent annotators who classify blind, and a published agreement statistic; if agreement is near chance, or if the category frequencies shift enough that Standardized Repetitive Names is no longer among the top three in every project, the taxonomy's reliability and the reported frequencies would be undermined.","tokens_in":9602,"feed_emoji":"🔤","tokens_out":8090,"duration_ms":78219,"temperature":0.7,"pith_summary":"The paper proposes a first taxonomy of the ways identifier names (the names of variables, methods, classes, and similar program elements) resemble one another within a codebase. It sorts the similarities into seven categories, from deliberate reuse of the same standardized name to abbreviated or single-character variants, and it applies the taxonomy by manually classifying identifiers in five open-source Java projects. The aim is to give researchers and tool builders a shared vocabulary so that confusingly similar names can be studied for their effect on code comprehension, maintainability, and collaboration. A sympathetic reader would take the contribution as the taxonomy itself, together with initial frequency evidence, not as a finished measurement.","feed_headline":"Seven patterns explain similar identifier names in real projects","feed_subtitle":"A manual review of five Java codebases counts how often each pattern appears and flags the risky ones.","key_machinery":"The carrying mechanism is the taxonomy itself, a seven-category classification of the relation between an identifier's name and its semantic role in the surrounding code. For each candidate pair of similar identifiers, a reviewer reads the surrounding code, judges whether the names are semantically equivalent, related, or unrelated, and assigns the pair to one of the seven categories, with a few subcategories such as Polymorphic Names and Cardinality Names. The taxonomy carries the argument by converting an unstructured inventory of extracted identifier names into frequency counts that can be compared across projects; without that classification step, raw string similarity would not distinguish harmless deliberate reuse from confusing collisions.","core_discovery":"On the paper's own terms, the discovery is that identifier-name similarity is not one phenomenon but several distinct ones: same names reused intentionally, different names for the same concept, lexically identical names with unrelated meanings, type-based and derivation-based variants, numerically distinguished variants, and concise or abbreviated forms. The paper reports that the categories occur with widely different frequencies across the five projects, from a high of 90.5% of reviewed identifiers classified as variants to a low of 12.85%, and that Standardized Repetitive Names is among the three most common categories in every project. This supports the paper's claim that most codebases deliberately reuse names for consistency, but that the mix of similarity patterns varies strongly with project size and structure.","pith_inferences":["The paper does not report an inter-rater reliability statistic, so a reader should treat the category frequencies as descriptive of the reviewed sets rather than as a stable measurement of the full identifier populations.","One test the paper does not run is whether the high-risk categories actually slow comprehension; a controlled experiment comparing code with Colliding or Concise names against renamed versions would give the taxonomy predictive teeth.","The categories may overlap in practice, and the paper does not fully specify an operational rule for assigning a pair to exactly one category; future refinements will need such a rule before automated tools can use the taxonomy directly.","If the taxonomy generalizes to dynamically typed languages, Type-Based Variants would likely shrink in importance, which would be a quick way to test whether the categories reflect universal naming behavior or Java-specific conventions."],"forward_implications":["If the taxonomy holds, development teams can audit a codebase for the specific patterns that pose the greatest risk: Colliding Names, Inconsistent Semantic Names, and Concise Variants.","Automated detectors can be built from the taxonomy, but they will need to read context and data types, not just compare name strings, to assign pairs to the right category.","The frequency evidence suggests that a high variant percentage is not itself a quality score; boilerplate-heavy codebases will naturally show more Standardized Repetitive Names.","The categories give researchers a dependent variable for studies linking naming similarity to comprehension time, bug rates, or onboarding difficulty.","The taxonomy points to concrete refactoring moves: rename colliding or abbreviated identifiers, align inconsistent names for the same concept, and replace numeric variants with collections or arrays."],"supporting_citations":[{"why":"Supplies the claim that identifiers make up roughly 70% of codebase content, which motivates studying their names.","marker":"[1]"},{"why":"Supplies the figure that developers spend about 55% of their time understanding code.","marker":"[2]"},{"why":"Grounds the link between identifier names and comprehension, and later the discussion of concise name risk.","marker":"[3]"},{"why":"Provides the project-selection service and criteria used to choose the five open-source projects.","marker":"[28]"},{"why":"Justifies the small sample size through the qualitative idea that theme saturation can be reached with few cases.","marker":"[30]"},{"why":"Supports the claim that automated techniques cannot fully capture semantic and contextual relationships, motivating manual analysis.","marker":"[36]"},{"why":"Provides a prior name-similarity checker that is likewise insufficient for semantic classification, motivating manual review.","marker":"[37]"}],"fun_headline_variants":["Up to 90% of identifiers are similar variants in code","Identifier similarity: from 12% to 90% across projects","New taxonomy categorizes similar identifier names","Similar identifier names: six patterns, not one"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that reviewers classifying the same identifier pairs by reading surrounding code would agree on the category, and that the inspected identifier sets stand in for each project's full set of identifiers.","fun_headline_variants_meta":{"raw":{"variants":["Up to 90% of identifiers are similar variants in code","Identifier similarity: from 12% to 90% across projects","New taxonomy categorizes similar identifier names","Similar identifier names: six patterns, not one"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000869,"raw_usage":{"total_tokens":3691,"prompt_tokens":798,"completion_tokens":2893,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":414,"completion_tokens_details":{"reasoning_tokens":2829}},"tokens_in":414,"tokens_out":2893,"duration_ms":21777,"temperature":1.0,"reasoning_tokens":2829,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:38:12.591278+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the taxonomy on the same five projects with a pre-specified random sample of identifier pairs, two independent annotators who classify blind, and a published agreement statistic; if agreement is near chance, or if the category frequencies shift enough that Standardized Repetitive Names is no longer among the top three in every project, the taxonomy's reliability and the reported frequencies would be undermined.","supporting_citations":[{"cited_title":"Concise and consistent naming,","cited_arxiv_id":null,"evidence_quote":"Supplies the claim that identifiers make up roughly 70% of codebase content, which motivates studying their names."},{"cited_title":"Measuring program comprehension: A large-scale field study with professionals,","cited_arxiv_id":null,"evidence_quote":"Supplies the figure that developers spend about 55% of their time understanding code."},{"cited_title":"Shorter identifier names take longer to comprehend,","cited_arxiv_id":null,"evidence_quote":"Grounds the link between identifier names and comprehension, and later the discussion of concise name risk."},{"cited_title":"Sampling projects in github for MSR studies,","cited_arxiv_id":null,"evidence_quote":"Provides the project-selection service and criteria used to choose the five open-source projects."},{"cited_title":"Code saturation ver- sus meaning saturation: how many interviews are enough?,","cited_arxiv_id":null,"evidence_quote":"Justifies the small sample size through the qualitative idea that theme saturation can be reached with few cases."},{"cited_title":"Idbench: Evaluating semantic rep- resentations of identifier names in source code,","cited_arxiv_id":null,"evidence_quote":"Supports the claim that automated techniques cannot fully capture semantic and contextual relationships, motivating manual analysis."},{"cited_title":"Namesake: A checker of lexical similarity in identifier names,","cited_arxiv_id":null,"evidence_quote":"Provides a prior name-similarity checker that is likewise insufficient for semantic classification, motivating manual review."}],"review_version":1}