REVIEW 5 major objections 6 minor 1 cited by
From Monolith to Microservices: A Comparative Evaluation of Decomposition Frameworks
T0 review · 5 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read A unified comparison of nine microservice decomposition tools on four Java monoliths finds that hierarchical density clustering (HDBScan) produces the most consistently balanced service boundaries.
desk verdict The paper's headline recommendation—HDBScan as most balanced—rests on imported outlier metric values, so the ranking is a useful hypothesis rather than a measurement; the new local runs and honest limitations section are the real value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The ranking mechanism is a two-part apparatus. First, HDBScan is a hierarchical variant of DBSCAN that forms service clusters from class dependency graphs without requiring the number of services in advance. Second, the paper's comparison pipeline computes four quality metrics for every decomposition and converts them into a single score via z-score normalization per benchmark, weighted W={3,−1,−1,−1} to reward structural modularity and penalize interface number, inter-partition communication, and non-extreme distribution. The z-scoring is what lets a method that is merely average on SM but outstandingly low on IFN/ICP, like HDBScan, top the ranking.
What would settle it
Run HDBScan on JPetStore, AcmeAir, DayTrader, and Plants using the same metric pipeline the paper applied to CHGNN and MonoEmbed, and compare its recomputed IFN/ICP with the quoted values. If HDBScan's composite score falls below a-BMSC or MEM, the central conclusion is refuted.
Extended reading notes
Core claim
Across all four benchmark systems—JPetStore, AcmeAir, DayTrader, and Plants—HDBScan earns the highest or near-highest aggregate score, and the paper concludes that 'hierarchical clustering-based methods, particularly HDBScan, produce the most consistently balanced decompositions across benchmarks.' The score is computed by z-normalizing four metrics (Structural Modularity, Interface Number, Inter-partition Communication, Non-Extreme Distribution) within each benchmark and combining them with weights of +3 for SM and −1 for the other three. HDBScan's edge comes largely from very low IFN and ICP values, which offset its moderate modularity and NED, while fine-grained a-BMSC and coarse-grained
Load-bearing premise
The paper treats metric values taken from other papers as directly comparable to values it recomputed itself, even though implementations, configurations, and dataset versions differ—especially for HDBScan's extreme IFN and ICP cells, which were not independently recomputed.
Editorial extensions
If this is right
- Practitioners decomposing medium-size Java monoliths can treat HDBScan-style hierarchical density clustering as a strong default, since it balances cohesion, coupling, interface simplicity, and size distribution.
- Fine-grained tools like a-BMSC produce more microservices and pay a coordination penalty; coarse-grained tools like Mono2Micro may yield poor size balance and higher interface counts on some systems.
- Embedding-based (MonoEmbed) and graph-neural (CHGNN) methods show high dataset sensitivity and should be tuned per system rather than used out of the box.
- The unified metric pipeline and composite score give future tool evaluations a reproducible template for the four benchmarks.
Reading between the lines
- The most fragile number in the table is HDBScan's IFN/ICP, which were quoted from its original paper rather than recomputed; rerunning HDBScan under the paper's own pipeline is the direct test of the first-place conclusion.
- The composite weights are manually chosen; changing the relative penalty of IFN vs NED, for example, could shuffle the ranking, so the 'balanced' conclusion is partly built into the aggregation rule.
- Because ground-truth datasets such as TrainTicket were excluded, the comparison captures structural and communication proxies, not semantic correctness of service boundaries; a ground-truth evaluation could overturn the ranking.
- The evidence covers only four systems, all medium-size Java monoliths; extending to larger codebases or other languages would test whether HDBScan's advantage generalizes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents a comparative evaluation of nine microservice decomposition tools (Bunch, MEM, FoSCI, CoGCN, Mono2Micro, HDBScan, a-BMSC, CHGNN, MonoEmbed) on four Java benchmark systems (JPetStore, AcmeAir, DayTrader, Plants). The authors recompute results for CHGNN and MonoEmbed through a 'unified metric computation pipeline' and combine these with metric values imported from prior publications. They compute an aggregate score by z-scoring each metric per benchmark and applying a fixed weight vector W={3,-1,-1,-1}, and on this basis conclude that HDBScan produces the most consistently balanced decompositions.
Significance. If the comparison were fully credible, the paper would be a useful consolidation for practitioners choosing a decomposition technique: it brings nine tools onto four common benchmarks and makes an explicit, falsifiable ranking claim. The study also has strengths: it identifies a real problem in the field (incompatible evaluation setups), it contributes locally reproduced results for two tools, and its Limitations section is unusually candid about threats to validity. However, the headline conclusion is not supported by the evidence as presented: the decisive HDBScan values are imported from another paper and are extreme outliers on the very metrics that drive the aggregate score, while the paper itself concedes that mixing reported and recomputed values may break comparability. The paper does not ship code or data, so the reproducibility promised in the abstract is only partial.
major comments (5)
- [§3, §4, §5, Table 1] The central claim that HDBScan ranks first is load-bearing on imported cells. Section 3 states that for all tools except CHGNN and MonoEmbed, metric values were taken directly from prior papers, and Section 5 concedes that this 'may affect comparability due to differences in implementations, configurations and dataset versions, particularly for ICP and IFN.' HDBScan's IFN values (0.05, 1.00, 0.30, 0.30) and ICP values (0.05, 0.03, 0.02, 0.02) are one to two orders of magnitude lower than nearly every recomputed or reported cell from other tools. If [21] used different interface definitions or different call-graph/dynamic-trace semantics than Equations 2–3, the z-score normalization is invalid and the top ranking is an artifact. The paper must either recompute HDBScan under the same pipeline, provide a sensitivity analysis that re-ranks tools without the imported cells, or substantially w
- [§4, Abstract, Table 1] The characterization of HDBScan as producing the 'most consistently balanced decompositions' is contradicted by its NED values: 0.70, 0.80, 0.90, 0.90 across the four benchmarks. These are among the worst NED values in the table (e.g., CoGCN on AcmeAir is 1.00, but HDBScan is worse than MEM, FoSCI, Mono2Micro, CHGNN, and MonoEmbed on several systems). The aggregate score rewards HDBScan despite its poor size balance because the imported low IFN/ICP values dominate the weighted sum. The abstract and Section 4 should be reworded to say that HDBScan achieves high aggregate scores driven by low interface and communication counts, while its service-size balance is actually a weakness.
- [§3, Eq. (5)] The aggregate score's construction is not robust enough to carry the paper's ranking conclusions. First, the weight vector W={3,-1,-1,-1} is taken from [20], which is the paper introducing MonoEmbed, one of the evaluated tools; using the same authors' weighting to evaluate their own tool is an endogeneity risk that should at least be tested. Second, with z-scores computed over only 8–9 tools per benchmark, a single extreme imported cell can dominate the aggregate. Third, Section 5 itself notes 'formulation ambiguities in Equation 5 that may hinder exact reproducibility.' The authors should report per-metric rankings and a sensitivity analysis over alternative weight vectors, or explicitly frame the aggregate score as exploratory rather than as the basis for the headline claim.
- [§4, §5, Table 1] The paper makes comparative claims such as 'HDBScan obtains high scores; often ranking first' and 'most consistently strong results,' but Section 5 admits there is no statistical testing, no variance reporting, and no sensitivity analysis. In Table 1, several aggregate scores are close (e.g., DayTrader: CoGCN 0.73 vs. HDBScan 0.47 vs. a-BMSC 0.49; AcmeAir: MEM 0.69 vs. HDBScan 0.53), so with single reported values the ranking could change under modest measurement error. The authors should either run repeated executions where feasible, provide confidence intervals, or downgrade the comparative conclusions to descriptive observations.
- [Table 1] HDBScan's Micro (number of microservices) column is missing for all four benchmarks. Without the service count, the reported NED and IFN values cannot be independently interpreted or reproduced, and the aggregate score cannot be validated from the table. This missing cell is especially problematic because the paper argues HDBScan generates 'a moderate number of services.' Please supply the service counts or explain why they are unavailable.
minor comments (6)
- [General] The text contains a formatting typo: 'T able 1' on the page before the table. Please fix.
- [§2.2, Eq. (2)–(3)] Notation is inconsistent between metrics: Eq. (2) uses N for the total number of microservices and also as the summation index upper bound, while Eq. (3) uses M for the same concept. Also, Eq. (3) as written defines pairwise ICP_{i,j}, but the table reports a single ICP value per tool/dataset; please clarify how the pairwise values are aggregated.
- [§2.2, Eq. (4)] The NED formula uses |N| in the denominator but N elsewhere, and the 'non-extreme' size range 5 ≤ |k| ≤ 20 is asserted without justification. Please clarify the notation and cite/explain the choice of range.
- [§3] The sentence 'All experiments were executed on multiple developer workstations' should be scoped, since only CHGNN and MonoEmbed were actually executed in this study. As written it suggests all tools were run locally, which contradicts the later statement that values for the remaining tools were taken from papers.
- [§2.3, Table 1] The Plants benchmark is described as 'a simplified variant derived from PlantsByWebSphere' but no repository URL or version identifier is given. Since reproducibility is a stated goal, please provide the exact dataset version and preprocessing steps.
- [§3, Eq. (5)] The z-score normalization is described as being computed 'across all tools on the same benchmark,' but the table does not report the per-benchmark means and standard deviations. Adding these would make the aggregate score auditable.
Circularity Check
No constructional circularity; central claim is an empirical ranking with disclosed comparability limits.
full rationale
The paper's central claim is an empirical comparison, not a derivation from first principles. The headline that HDBScan 'produce[s] the most consistently balanced decompositions' is obtained by z-scoring the metric cells in Table 1 and applying the fixed aggregate Score(T) with W={3,-1,-1,-1}. No metric is defined in terms of the Score, and the Score is not fitted to make HDBScan win; it is applied uniformly to all tools. The main concerns are comparability and independence, not circularity: most metric values, including HDBScan's decisive IFN/ICP cells, are imported from prior papers rather than recomputed, and the weighting convention is taken from [20], the paper introducing MonoEmbed (one of the evaluated tools). These are real limitations, and the paper itself flags them: 'The evaluation combines metric values reported in prior work with metrics recomputed under our pipeline, which may affect comparability due to differences in implementations, configurations and dataset versions, particularly for ICP and IFN' (Section 5). However, importing values from another paper and adopting a weighting scheme from the literature do not make the derivation circular unless the imported values or weights are themselves defined in terms of the present conclusion, which is not the case here. No equation reduces to another by construction, and no fitted parameter is renamed as a prediction. The score of 1 reflects the mild provenance/transparency issue of using an evaluated tool's paper for the scoring rubric, not a circular-step finding.
Assumptions & free parameters
free parameters (2)
- Aggregate score weight vector W =
SM=3, IFN=-1, ICP=-1, NED=-1
- NED non-extreme class-size range =
5 <= |k| <= 20
assumptions (4)
- domain assumption SM, IFN, ICP and NED are valid, commensurate proxies for decomposition quality.
- domain assumption Published metric values from different tools use compatible definitions, benchmark versions, and configurations.
- domain assumption The four selected monoliths are representative enough to generalize about tool quality.
- domain assumption Heterogeneous execution workstations do not affect decomposition quality metrics.
Cite this review
Pith. "Pith review of From Monolith to Microservices: A Comparative Evaluation of Decomposition Frameworks." pith.science (2026). https://pith.science/paper/KZ35QXIG
@misc{pith2026260123141,
author = {Pith},
title = {Pith review of: From Monolith to Microservices: A Comparative Evaluation of Decomposition Frameworks},
year = {2026},
howpublished = {\url{https://pith.science/paper/KZ35QXIG}},
note = {Machine review of arXiv:2601.23141}
}
read the original abstract
Software modernisation through the migration from monolithic architectures to microservices has become increasingly critical, yet identifying effective service boundaries remains a complex and unresolved challenge. Although numerous automated microservice decomposition frameworks have been proposed, their evaluation is often fragmented due to inconsistent benchmark systems, incompatible metrics, and limited reproducibility, thus hindering objective comparison. This work presents a unified comparative evaluation of state-of-the-art microservice decomposition approaches spanning static, dynamic, and hybrid techniques. Using a consistent metric computation pipeline, we assess the decomposition quality across widely used benchmark systems (JPetStore, AcmeAir, DayTrader, and Plants) using Structural Modularity (SM), Interface Number(IFN), Inter-partition Communication (ICP), Non-Extreme Distribution (NED), and related indicators. Our analysis combines results reported in prior studies with experimentally reproduced outputs from available replication packages. Findings indicate that the hierarchical clustering-based methods, particularly HDBScan, produce the most consistently balanced decompositions across benchmarks, achieving strong modularity while minimizing communication and interface overhead.
Forward citations
Cited by 1 Pith paper
-
Structural Validation of LLM-Generated Microservice Decompositions Using Source-Code Dependencies
After normalizing class-to-service mapping coverage, OpenAI o3 zero-shot and few-shot decompositions show identical structural adherence (TPD 68% PetClinic, 83.3% Bookstore).
Reference graph
Works this paper leans on
-
[18]
arXiv preprint arXiv:2402.08481 (2024), URLhttps://arxiv.org/abs/2402.08481
Saied, M.A.: Migration to microservices: A comparative study of decom- position strategies and analysis metrics. arXiv preprint arXiv:2402.08481 (2024), URLhttps://arxiv.org/abs/2402.08481
arXiv 2024
-
[21]
In: EASE, ACM (2022),https://doi.org/10.1145/3530019.3530040
Sellami, K., Saied, M.A., Ouni, A.: A hierarchical DBSCAN method for extracting microservices from monolithic applications. In: EASE, ACM (2022),https://doi.org/10.1145/3530019.3530040
arXiv 2022
-
[20]
arXiv preprint arXiv:2502.04604 (2025), URLhttps://arxiv.org/abs/2502.04604
Sellami, K., Saied, M.A.: Contrastive Learning-Enhanced Large Lan- guage Models for Monolith-to-Microservice Decomposition. arXiv preprint arXiv:2502.04604 (2025), URLhttps://arxiv.org/abs/2502.04604
arXiv 2025
-
[1]
Journal of Systems and Software203, 111704 (2023)
Abgaz, Y., McCarren, A., Elger, P., et al.: Decomposition of Monolith Ap- plications Into Microservices Architectures: A Systematic Review. Journal of Systems and Software203, 111704 (2023)
2023
-
[2]
In: 2021 2nd International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pp
Al-Debagy, O., Martinek, P.: Semantic-based microservice identification us- ing natural language processing. In: 2021 2nd International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pp. 1–6, IEEE (2021)
2021
-
[3]
arXiv preprint arXiv:2208.00682 (2022),https://doi.org/10.3233/FAIA220242
Bolanowski, M., Żak, K., et al.: Efficiency of REST and gRPC Realizing Communication Tasks in Microservice-Based Ecosystems. arXiv preprint arXiv:2208.00682 (2022),https://doi.org/10.3233/FAIA220242
arXiv 2022
-
[4]
In: IBM DeveloperWorks Technical Reports (2007)
Davis, S., et al.: IBM DayTrader: A Performance Benchmark for J2EE. In: IBM DeveloperWorks Technical Reports (2007)
2007
-
[5]
In: AAAI, vol
Desai, Utkarsh, B., et al.: Graph Neural Network to Dilute Outliers for Refactoring Monolith Application. In: AAAI, vol. 35, pp. 72–80 (2021)
2021
Show all 27 references
-
[6]
In: Perspectives of System Informatics, pp
Dragoni, N., Lanese, I., et al.: Microservices: How to make your application scale. In: Perspectives of System Informatics, pp. 293–313 (2018),https: //doi.org/10.1007/978-3-319-74313-4_8
2018 doi
-
[7]
In: ICSA, pp
Filippone, Gianluca, M., et al.: From monolithic to microservice architec- ture: An automated approach based on graph clustering and combinatorial optimization. In: ICSA, pp. 47–57, IEEE (2023),https://doi.org/10. 1109/ICSA56044.2023.00013
2023
-
[8]
IEEE Transactions on Software En- gineering47(5), 987–1007 (2021),https://doi.org/10.1109/TSE.2019
Jin, Wuxia, L., et al.: Service Candidate Identification from Monolithic Systems Based on Execution Traces. IEEE Transactions on Software En- gineering47(5), 987–1007 (2021),https://doi.org/10.1109/TSE.2019. 2910531
2021 doi
-
[9]
In: ESEC/FSE, pp
Kalia, Anup K., L., et al.: Mono2Micro: An AI-based Toolchain for Evolv- ing Monolithic Enterprise Applications to a Microservice Architecture. In: ESEC/FSE, pp. 1606–1610 (2020)
2020
-
[10]
In: ESEC/FSE, pp
Kalia, Anup K., X., et al.: Mono2Micro: A practical and effective tool for decomposing monolithic Java applications to microservices. In: ESEC/FSE, pp. 1214–1224 (2021),https://doi.org/10.1145/3468264.3473915
2021
-
[11]
In: ICSR, pp
Lecrivain, Matthéo, B., et al.: MONO2REST: Identifying and Exposing Microservices: a Reusable RESTification Approach. In: ICSR, pp. 33–43 (2025),https://doi.org/10.1109/ICSR66718.2025.00010
2025
-
[12]
arXiv preprint arXiv:2112.01317 (2021), URL https://arxiv.org/abs/2112.01317
Mathai, A., Bandyopadhyay, S., Desai, U., Tamilselvam, S.: Monolith to microservices: Representing Application Software through Heterogeneous Graph Neural Network. arXiv preprint arXiv:2112.01317 (2021), URL https://arxiv.org/abs/2112.01317
2021 arXiv
-
[13]
In: ICWS, pp
Mazlami, G., Cito, J., Leitner, P.: Extraction of Microservices from Mono- lithic Software Architectures. In: ICWS, pp. 524–531, IEEE (2017),https: //doi.org/10.1109/ICWS.2017.61 12 M. Weerasinghe et al
2017 doi
-
[14]
Soft Computing12(1), 77–93 (2008)
Mitchell, B.S., Mancoridis, S.: On the evaluation of the Bunch search-based software modularization algorithm. Soft Computing12(1), 77–93 (2008)
2008
-
[15]
Information & Software Technology143, 107732 (2025),https://doi.org/10.1016/j
Mohottige, Thakshila Imiya, P., et al.: Reengineering Software Systems into Microservices: State-of-the-Art and Future Directions. Information & Software Technology143, 107732 (2025),https://doi.org/10.1016/j. infsof.2025.107732
2025
-
[16]
O’Reilly Media (2021)
Newman, S.: Building Microservices: Designing Fine-Grained Systems. O’Reilly Media (2021)
2021
-
[17]
In: ASE, pp
Nitin, V., Asthana, S., Ray, B., Krishna, R.: Cargo: AI-guided dependency analysis for migrating monolithic applications to microservices architecture. In: ASE, pp. 1–12, Association for Computing Machinery (2022),https: //doi.org/10.1145/3551349.3556960
2022
-
[19]
In: ACM SAC, pp
Scanniello, G., D’Amico, A., D’Amico, C., D’Amico, T.: An Approach for Architectural Layer Recovery. In: ACM SAC, pp. 2198–2202, ACM (2010), https://doi.org/10.1145/1774088.1774551
2010
-
[22]
In: ICSOC, pp
Sellami, K., Saied, M.A., Ouni, A., Abdalkareem, R.: Combining static and dynamic analysis to decompose monolithic applications into microser- vices. In: ICSOC, pp. 203–218, Lecture Notes in Computer Science, Springer (2022),https://doi.org/10.1007/978-3-031-20984-0_14
2022 doi
-
[23]
Journal of Systems and Software 176, 110941 (2021)
Taibi, Davide, L., et al.: Empirical comparison of microservice identification approaches using a monolithic system. Journal of Systems and Software 176, 110941 (2021)
2021
-
[24]
Applied Sciences10(17), 5797 (2020), https://doi.org/10.3390/app10175797
Tapia, Freddy, M., et al.: From monolithic systems to microservices: A comparative study of performance. Applied Sciences10(17), 5797 (2020), https://doi.org/10.3390/app10175797
2020 doi
-
[25]
In: ICSM, pp
Wu, J., Hassan, A.E., Holt, R.C.: Comparison of Clustering Algorithms in the Context of Software Evolution. In: ICSM, pp. 525–535, IEEE (2005), https://doi.org/10.1109/ICSM.2005.31
2005 doi
-
[26]
In: 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), pp
Zhang, Y., Li, Y., Yang, Y., Chen, S., et al.: RapidMS: A Tool for Support- ing Rapid Microservices Generation and Refinement from Requirements Model. In: 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), pp. 45–49, ...
2023
-
[27]
In: ICSE Companion, pp
Zhou, Xiang, P., et al.: Benchmarking microservice systems for software engineering research. In: ICSE Companion, pp. 323–324, ACM (2018), https://doi.org/10.1145/3183440.3194991
2018
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.