{"id":"eb9ff482-23b1-4350-bb62-aed0080cd7e5","arxiv_id":"2601.23141","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Across four benchmark monoliths, hierarchical DBSCAN-based decomposition scores highest under the authors' hand-weighted aggregate metric, but most comparison data comes from prior papers.","lead":"Nine automated tools that split monolithic software into microservices were scored on four common benchmark codebases using one metric pipeline, and a simple clustering method, HDBScan, came out on top. The result is a useful but fragile comparison, because most of the scores were taken from earlier papers rather than recomputed under the supposedly unified pipeline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"HDBScan's top ranking rests on IFN/ICP values imported from [21] with a different pipeline; if comparability fails the central claim collapses.","rationale":"The reader identified the same weak assumption (Section 3 + Section 5), so my concern is not novel but is the one that matters. The paper's stated goal is a 'unified comparative evaluation,' but the unification only applies to two locally run tools; all other data points come from papers that may operationalize metrics differently. HDBScan's winning position is disproportionately carried by four imported IFN/ICP values that are orders of magnitude lower than any recomputed value. Since the aggregate score uses equal weights (SM: 3, IFN/ICP/NED: -1 each), a single extreme cell can dominate the z-score sum. The authors' own limitation (Section 5) flags exactly this. The proposed sensitivity test is simple and would settle whether the headline claim survives honest uncertainty. Thus the reader's CONDITIONAL verdict is appropriate; I would keep it unchanged. I do not see a separate internal inconsistency, though the term 'balanced' is misleading given NED values; that is a presentation issue, not the central logic.","tokens_in":9557,"tokens_out":3537,"duration_ms":36575,"concrete_test":"Recompute the aggregate scores for all benchmarks after replacing HDBScan's imported IFN and ICP values with the median IFN and ICP values of the locally recomputed tools (CHGNN, MonoEmbed) for each benchmark, or with values derived from re-running HDBScan's replicable implementation [21] under the paper's metric pipeline (Equations 2–3). If HDBScan's rank changes from first or its score drops below a-BMSC/MEM, the conclusion is an artifact of incomparable metric definitions; if it remains first, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that HDBScan 'produce[s] the most consistently balanced decompositions' rests on the aggregate Score in Table 1. HDBScan's dominant advantage comes almost entirely from its imported IFN and ICP values (e.g., IFN 0.05/1.00/0.30/0.30 and ICP 0.05/0.03/0.02/0.02 across the four benchmarks), all taken from [21]. Section 3 states that for all tools other than CHGNN and MonoEmbed, 'the metric values were taken directly from the corresponding research papers.' These cells are extreme relative to every tool recomputed under the paper's own metric pipeline. Section 5 concedes that mixing reported and recomputed values 'may affect comparability due to differences in implementations, configurations and dataset versions, particularly for ICP and IFN.' If HDBScan's definitions of interface count or inter-partition communication differ from Equations 2–3 (e.g., counting REST endpoints vs. method calls, or using dynamic traces instead of static call graphs), the z-score normalization is invalid and the ranking's top spot is an artifact. Moreover, HDBScan's NED values (0.70, 0.80, 0.90, 0.90) are poor, contradicting the abstract's 'balanced' characterization; the aggregate score still rewards HDBScan because low imported IFN/ICP dominate the weighted sum. Thus the paper's headline recommendation is only as strong as the comparability of those imported cells, which the authors themselves flag as uncertain.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a comparative evaluation of nine microservice decomposition tools (Bunch, MEM, FoSCI, CoGCN, Mono2Micro, HDBScan, a-BMSC, CHGNN, MonoEmbed) on four Java benchmark systems (JPetStore, AcmeAir, DayTrader, Plants). The authors recompute results for CHGNN and MonoEmbed through a 'unified metric computation pipeline' and combine these with metric values imported from prior publications. They compute an aggregate score by z-scoring each metric per benchmark and applying a fixed weight vector W={3,-1,-1,-1}, and on this basis conclude that HDBScan produces the most consistently balanced decompositions.","tokens_in":9868,"tokens_out":3335,"duration_ms":37106,"significance":"If the comparison were fully credible, the paper would be a useful consolidation for practitioners choosing a decomposition technique: it brings nine tools onto four common benchmarks and makes an explicit, falsifiable ranking claim. The study also has strengths: it identifies a real problem in the field (incompatible evaluation setups), it contributes locally reproduced results for two tools, and its Limitations section is unusually candid about threats to validity. However, the headline conclusion is not supported by the evidence as presented: the decisive HDBScan values are imported from another paper and are extreme outliers on the very metrics that drive the aggregate score, while the paper itself concedes that mixing reported and recomputed values may break comparability. The paper does not ship code or data, so the reproducibility promised in the abstract is only partial.","major_comments":[{"comment":"The central claim that HDBScan ranks first is load-bearing on imported cells. Section 3 states that for all tools except CHGNN and MonoEmbed, metric values were taken directly from prior papers, and Section 5 concedes that this 'may affect comparability due to differences in implementations, configurations and dataset versions, particularly for ICP and IFN.' HDBScan's IFN values (0.05, 1.00, 0.30, 0.30) and ICP values (0.05, 0.03, 0.02, 0.02) are one to two orders of magnitude lower than nearly every recomputed or reported cell from other tools. If [21] used different interface definitions or different call-graph/dynamic-trace semantics than Equations 2–3, the z-score normalization is invalid and the top ranking is an artifact. The paper must either recompute HDBScan under the same pipeline, provide a sensitivity analysis that re-ranks tools without the imported cells, or substantially w","section":"§3, §4, §5, Table 1"},{"comment":"The characterization of HDBScan as producing the 'most consistently balanced decompositions' is contradicted by its NED values: 0.70, 0.80, 0.90, 0.90 across the four benchmarks. These are among the worst NED values in the table (e.g., CoGCN on AcmeAir is 1.00, but HDBScan is worse than MEM, FoSCI, Mono2Micro, CHGNN, and MonoEmbed on several systems). The aggregate score rewards HDBScan despite its poor size balance because the imported low IFN/ICP values dominate the weighted sum. The abstract and Section 4 should be reworded to say that HDBScan achieves high aggregate scores driven by low interface and communication counts, while its service-size balance is actually a weakness.","section":"§4, Abstract, Table 1"},{"comment":"The aggregate score's construction is not robust enough to carry the paper's ranking conclusions. First, the weight vector W={3,-1,-1,-1} is taken from [20], which is the paper introducing MonoEmbed, one of the evaluated tools; using the same authors' weighting to evaluate their own tool is an endogeneity risk that should at least be tested. Second, with z-scores computed over only 8–9 tools per benchmark, a single extreme imported cell can dominate the aggregate. Third, Section 5 itself notes 'formulation ambiguities in Equation 5 that may hinder exact reproducibility.' The authors should report per-metric rankings and a sensitivity analysis over alternative weight vectors, or explicitly frame the aggregate score as exploratory rather than as the basis for the headline claim.","section":"§3, Eq. (5)"},{"comment":"The paper makes comparative claims such as 'HDBScan obtains high scores; often ranking first' and 'most consistently strong results,' but Section 5 admits there is no statistical testing, no variance reporting, and no sensitivity analysis. In Table 1, several aggregate scores are close (e.g., DayTrader: CoGCN 0.73 vs. HDBScan 0.47 vs. a-BMSC 0.49; AcmeAir: MEM 0.69 vs. HDBScan 0.53), so with single reported values the ranking could change under modest measurement error. The authors should either run repeated executions where feasible, provide confidence intervals, or downgrade the comparative conclusions to descriptive observations.","section":"§4, §5, Table 1"},{"comment":"HDBScan's Micro (number of microservices) column is missing for all four benchmarks. Without the service count, the reported NED and IFN values cannot be independently interpreted or reproduced, and the aggregate score cannot be validated from the table. This missing cell is especially problematic because the paper argues HDBScan generates 'a moderate number of services.' Please supply the service counts or explain why they are unavailable.","section":"Table 1"}],"minor_comments":[{"comment":"The text contains a formatting typo: 'T able 1' on the page before the table. Please fix.","section":"General"},{"comment":"Notation is inconsistent between metrics: Eq. (2) uses N for the total number of microservices and also as the summation index upper bound, while Eq. (3) uses M for the same concept. Also, Eq. (3) as written defines pairwise ICP_{i,j}, but the table reports a single ICP value per tool/dataset; please clarify how the pairwise values are aggregated.","section":"§2.2, Eq. (2)–(3)"},{"comment":"The NED formula uses |N| in the denominator but N elsewhere, and the 'non-extreme' size range 5 ≤ |k| ≤ 20 is asserted without justification. Please clarify the notation and cite/explain the choice of range.","section":"§2.2, Eq. (4)"},{"comment":"The sentence 'All experiments were executed on multiple developer workstations' should be scoped, since only CHGNN and MonoEmbed were actually executed in this study. As written it suggests all tools were run locally, which contradicts the later statement that values for the remaining tools were taken from papers.","section":"§3"},{"comment":"The Plants benchmark is described as 'a simplified variant derived from PlantsByWebSphere' but no repository URL or version identifier is given. Since reproducibility is a stated goal, please provide the exact dataset version and preprocessing steps.","section":"§2.3, Table 1"},{"comment":"The z-score normalization is described as being computed 'across all tools on the same benchmark,' but the table does not report the per-benchmark means and standard deviations. Adding these would make the aggregate score auditable.","section":"§3, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The paper is positioned as a unified comparative evaluation, but its practical contribution is closer to a literature-metrics consolidation with two locally reproduced tools. The central HDBScan ranking depends entirely on imported cells from [21], and the paper itself flags the comparability risk. I do not see this as an irreparable flaw — recomputing HDBScan, adding sensitivity analysis, and softening the 'balanced' wording would address my main concerns — but the required changes affect the paper's headline result, so major revision is appropriate. I would also encourage the editor to consider whether the journal expects a reproducibility artifact; the abstract promises 'reproducible evaluation' but no code or data is provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the deal. The paper is a comparative evaluation of microservice decomposition tools. The genuinely new part is small but real: the authors ran CHGNN and MonoEmbed on four standard benchmarks (JPetStore, AcmeAir, DayTrader, Plants) and computed SM, IFN, ICP, NED through their own pipeline. For every other tool—Bunch, MEM, FoSCI, CoGCN, Mono2Micro, HDBScan, a-BMSC—the metric values are taken from published papers, mostly [18] and [21]. They then z-score all values for each benchmark and combine them into a weighted score using weights borrowed from Sellami and Saied [20], the paper that introduced MonoEmbed.\n\nThe writing is clear and the authors are unusually honest: Section 5 concedes that mixing reported and recomputed values \"may affect comparability due to differences in implementations, configurations and dataset versions, particularly for ICP and IFN,\" and it lists the absence of statistical testing, variance reporting, sensitivity analysis, and ground-truth validation. That candor earns credit.\n\nThe problem is that the central claim does not survive the evidence. HDBScan's top score is driven almost entirely by IFN and ICP values imported from [21] that are extreme outliers (e.g., IFN 0.05–0.30, ICP 0.02–0.05 across benchmarks) relative to every tool run under the paper's own metric definitions. If those values came from a different counting rule—REST endpoints vs. method calls, or dynamic traces vs. static call graphs—the z-score normalization is invalid and the ranking collapses. The paper's own limitations section effectively says that. Additionally, HDBScan's NED values (0.70–0.90) are among the worst on two benchmarks, which undercuts the abstract's phrase \"balanced decompositions.\" The aggregate weight vector is also endogenous: it comes from the MonoEmbed paper, and the metric definitions mostly from other evaluated tools' papers.\n\nSo what's the honest verdict? This is a useful, well-organized survey with a small amount of new experimental data and a transparent if fragile aggregation scaffold. It is not a robust empirical ranking. The headline conclusion should be reframed as a hypothesis—\"density-based hierarchical clustering appears promising, pending a common-pipeline comparison\"—or the authors should run all nine tools in one pipeline with variance and sensitivity analysis. Given the limitations the authors themselves acknowledge, I would treat the ranking as indicative, not definitive.\n\nWho is this for? Researchers and practitioners working on microservice decomposition who want a bird's-eye view of the field and a few new data points. It deserves a serious referee—there is real content here—but the referee should push hard for a re-run of all tools or a much more cautious conclusion. I'd accept for peer review with the expectation of major revision.","headline":"The paper's headline recommendation—HDBScan as most balanced—rests on imported outlier metric values, so the ranking is a useful hypothesis rather than a measurement; the new local runs and honest limitations section are the real value.","tokens_in":10418,"tokens_out":2463,"would_cite":false,"duration_ms":26040,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A unified comparison of nine microservice decomposition tools on four Java monoliths finds that hierarchical density clustering (HDBScan) produces the most consistently balanced service boundaries.","keywords":["microservice decomposition","service boundary identification","HDBScan","hierarchical clustering","structural modularity","benchmark comparison","monolith-to-microservices"],"falsifier":"Run HDBScan on JPetStore, AcmeAir, DayTrader, and Plants using the same metric pipeline the paper applied to CHGNN and MonoEmbed, and compare its recomputed IFN/ICP with the quoted values. If HDBScan's composite score falls below a-BMSC or MEM, the central conclusion is refuted.","tokens_in":9423,"feed_emoji":"🧩","tokens_out":5588,"duration_ms":57357,"temperature":0.7,"pith_summary":"This paper attempts to settle which automated decomposition tool should be used when breaking monolithic Java applications into microservices. It brings together nine tools spanning static, dynamic, and hybrid approaches, evaluates them on four benchmark monoliths, and scores every decomposition with one composite metric. The central claim is that HDBScan, a hierarchical density-based clustering method, consistently produces the most balanced decompositions—strong structural modularity with low interface counts and low inter-service communication. A practitioner reading this would come away with a specific first choice for medium-size Java monoliths, and a reproducible pipeline for future tool comparisons.","feed_headline":"HDBScan tops microservice decomposition comparison","feed_subtitle":"Unified scoring of nine tools on four Java monoliths favors density-based hierarchical clustering over GNN and embedding methods.","key_machinery":"The ranking mechanism is a two-part apparatus. First, HDBScan is a hierarchical variant of DBSCAN that forms service clusters from class dependency graphs without requiring the number of services in advance. Second, the paper's comparison pipeline computes four quality metrics for every decomposition and converts them into a single score via z-score normalization per benchmark, weighted W={3,−1,−1,−1} to reward structural modularity and penalize interface number, inter-partition communication, and non-extreme distribution. The z-scoring is what lets a method that is merely average on SM but outstandingly low on IFN/ICP, like HDBScan, top the ranking.","core_discovery":"Across all four benchmark systems—JPetStore, AcmeAir, DayTrader, and Plants—HDBScan earns the highest or near-highest aggregate score, and the paper concludes that 'hierarchical clustering-based methods, particularly HDBScan, produce the most consistently balanced decompositions across benchmarks.' The score is computed by z-normalizing four metrics (Structural Modularity, Interface Number, Inter-partition Communication, Non-Extreme Distribution) within each benchmark and combining them with weights of +3 for SM and −1 for the other three. HDBScan's edge comes largely from very low IFN and ICP values, which offset its moderate modularity and NED, while fine-grained a-BMSC and coarse-grained","pith_inferences":["The most fragile number in the table is HDBScan's IFN/ICP, which were quoted from its original paper rather than recomputed; rerunning HDBScan under the paper's own pipeline is the direct test of the first-place conclusion.","The composite weights are manually chosen; changing the relative penalty of IFN vs NED, for example, could shuffle the ranking, so the 'balanced' conclusion is partly built into the aggregation rule.","Because ground-truth datasets such as TrainTicket were excluded, the comparison captures structural and communication proxies, not semantic correctness of service boundaries; a ground-truth evaluation could overturn the ranking.","The evidence covers only four systems, all medium-size Java monoliths; extending to larger codebases or other languages would test whether HDBScan's advantage generalizes."],"forward_implications":["Practitioners decomposing medium-size Java monoliths can treat HDBScan-style hierarchical density clustering as a strong default, since it balances cohesion, coupling, interface simplicity, and size distribution.","Fine-grained tools like a-BMSC produce more microservices and pay a coordination penalty; coarse-grained tools like Mono2Micro may yield poor size balance and higher interface counts on some systems.","Embedding-based (MonoEmbed) and graph-neural (CHGNN) methods show high dataset sensitivity and should be tuned per system rather than used out of the box.","The unified metric pipeline and composite score give future tool evaluations a reproducible template for the four benchmarks."],"fun_headline_variants":["HDBScan wins microservice split contest, low overhead key","Microservice decomposition: HDBScan beats GNN and embedding methods","Hierarchical clustering leads in four-benchmark microservice study","Density-based clustering trims interface and communication in microservices","HDBScan top in fair microservice decomposition comparison"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper treats metric values taken from other papers as directly comparable to values it recomputed itself, even though implementations, configurations, and dataset versions differ—especially for HDBScan's extreme IFN and ICP cells, which were not independently recomputed.","fun_headline_variants_meta":{"raw":{"variants":["HDBScan wins microservice split contest, low overhead key","Microservice decomposition: HDBScan beats GNN and embedding methods","Hierarchical clustering leads in four-benchmark microservice study","Density-based clustering trims interface and communication in microservices","HDBScan top in fair microservice decomposition comparison"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000864,"raw_usage":{"total_tokens":3567,"prompt_tokens":713,"completion_tokens":2854,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":2769}},"tokens_in":457,"tokens_out":2854,"duration_ms":22415,"temperature":1.0,"reasoning_tokens":2769,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T06:12:22.225646+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HDBScan on JPetStore, AcmeAir, DayTrader, and Plants using the same metric pipeline the paper applied to CHGNN and MonoEmbed, and compare its recomputed IFN/ICP with the quoted values. If HDBScan's composite score falls below a-BMSC or MEM, the central conclusion is refuted.","supporting_citations":[],"review_version":1}