Pith. sign in

REVIEW 3 major objections 6 minor 39 references

A Decomposition and Metric-Based Evaluation Framework for Microservices

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a four-metric measurement framework, computed from runtime execution traces, that lets software architects compare microservice decompositions objectively and monitor architectural erosion over time.

desk verdict A clear, honest proposal for comparing microservice decompositions with four simple metrics, but the metrics are unvalidated and the ones normalizing by class count are gameable by class splitting. read the letter →

arxiv 1908.08513 v1 pith:NNNNRV33 submitted 2019-08-22 cs.SE

Davide Taibi , Kari Systä This is my paper · ORCID
classification cs.SE
keywords microservicesdecompositionmeasurementframeworkprocessminingexecutiontracescouplingmetricarchitecturalerosionmonolithmigration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Migrating a monolithic system to microservices usually comes down to an architect's manual judgment about where to slice the code. This paper argues that the choice can be made more objective through four metrics computed from runtime execution traces: coupling between microservices, number of classes per microservice, duplicated classes, and frequency of external calls. The same metrics also allow a company to track whether an existing microservice system is eroding as it evolves. The paper combines these measures with a process-mining decomposition framework that turns log files into candidate slicing options, so that options can be generated from actually executed call paths rather than from static dependency analysis alone.

What carries the argument

The load-bearing object is the four-measure metric suite, computed from the call frequencies in log traces of the monolith (or of an existing microservice system). CBM follows the classic Coupling Between Objects idea, counting external links relative to class count; CLA is a size proxy that flags microservices that are too large or too small; DUP counts shared sub-paths that would force code duplication if placed in separate services; FEC divides the number of cross-service call instances by the class count, quantifying the performance cost of a split. The measures are fed by a decomposition pipeline: execution path analysis, frequency analysis, circular dependency removal, and identification of candidate slices, followed by metric-based ranking. What gives the machinery its force is that every number is grounded in observed runtime behavior rather than in static dependencies, so the comparison reflects the system as it actually runs.

What would settle it

Take a monolith with complete execution traces, compute the four metrics for two decomposition options, and benchmark the production latency and maintenance cost of each; if the option with lower CBM and FEC performs consistently worse, the central claim that these metrics indicate decomposition quality would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that the quality of a microservice decomposition can be evaluated with repeatable, objective measures derived from complete execution logs. Four measures are defined: Coupling Between Microservice (CBM), the number of external links per class in each microservice; CLA, the number of classes per microservice; DUP, the number of classes that would need to be duplicated across microservices; and FEC, the number of call instances per class that cross service boundaries. Because external calls are computationally much heavier than internal calls, FEC captures the run-time cost of a split, while CBM captures its coupling. Applied to alternative decompositions of the same monolith, these measures support a numerical comparison that complements or challenges the architect's intuition; applied periodically to a running system, they reveal architectural erosion as the numbers drift. The paper does not claim to automate the decision: it presents measurements plus a six-step process that still leaves the final choice to the team.

Load-bearing premise

The framework assumes complete end-to-end execution traces exist: every user action or external call must be traceable from its entry point through every internal method to the database and back, so if logs omit any part of the call path, the frequency analysis and all four metrics lose their grounding.

Editorial extensions

If this is right

  • The framework works independently of how decompositions are generated, so any two candidate splits can be compared numerically.
  • Periodic reapplication to a live microservice system yields a historical record of metric drift, giving an early warning of architectural erosion.
  • The dynamic call frequencies highlight the most-used features, allowing teams to prioritize which slices to develop first.
  • If the approach matures, the metric-defined objective could be handed to an automated optimizer, such as a genetic algorithm, to propose decompositions that maximize the quality measures.
  • Metric conflicts, such as smaller services raising coupling and call frequency, remain a team judgment call, but the trade-offs become explicit and quantifiable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • With complete traces, the same four metrics could serve as a continuous-integration gate: a commit that increases FEC or CBM beyond a chosen threshold would automatically flag a quality regression.
  • The framework implicitly assumes a fixed relative cost of external versus internal calls; the example uses a 1000x weight, and if the true ratio varies by deployment, the ranking between decomposition options could change, so sensitivity analysis on this weight is a direct test of the framework's stability.
  • The metrics treat classes as the atomic unit; in languages or systems with different granularities, the approach should transfer if execution paths can be traced, suggesting applicability beyond Java-style object-oriented monoliths.
  • Because decomposition options are still identified manually in Step 4, the framework reduces subjectivity in comparing options but not in generating them; extending it with an algorithmic candidate generator would close that gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes a measurement framework for evaluating microservice decompositions, using four metrics (CBM, CLA, DUP, FEC) derived from runtime execution traces, and presents it as an extension of the authors' prior process-mining decomposition framework. The framework is intended to support objective comparison of decomposition options and continuous evaluation of architectural erosion, and the paper illustrates the metrics on a hypothetical example with calculated values in Table 2. No empirical validation is reported; the authors explicitly state that validation is future work.

Significance. The intended contribution is a repeatable, quantitative way to compare decomposition alternatives using dynamic call data rather than expert judgment alone. The transparency and simplicity of the proposed formulas are strengths, as is the explicit integration with an existing trace-based decomposition process. However, the current formulation has a load-bearing granularity sensitivity in two of the four metrics, and the objectivity claim is unsupported by empirical evidence in this manuscript. If robust definitions and validation are provided, the framework could be genuinely useful to migration practitioners.

major comments (3)
  1. [§3.1 and §3.4, Eqs. (1) and (2)] Both CBM and FEC are normalized by the number of classes in the microservice. This makes the metrics depend on class granularity: splitting or duplicating a class can change the denominator and thus the metric value without changing the actual number or cost of external calls. Since decomposition options may already differ in how classes are grouped or duplicated (e.g., §3.3 explicitly considers duplication), the claimed objective comparability is not guaranteed. Please either normalize by absolute call/link counts and path lengths, or provide a formal justification for why class count is an appropriate denominator and why the metric is invariant under class splits.
  2. [Table 2 (Section 3.4)] The reported load value for split 0 is inconsistent with the stated assumption. With 100 external calls, 1150 internal calls, and an external call 1000 times heavier than an internal call, the load should be 100*1000 + 1150 = 101150, not 101550 as listed. This arithmetic discrepancy must be corrected, and the example should be recomputed to ensure all FEC and load values follow from the formulas.
  3. [Sections 5 and 7] The central claim that the framework "helps to decrease subjectivity" is not supported by any empirical study in this paper; the only example is hypothetical, and the authors explicitly defer validation to future work. At minimum, an evaluation with real traces (even a small case study) or a demonstration on a known open-source system is needed to support the claim about usefulness for companies and erosion detection.
minor comments (6)
  1. [Section 1] "very difficult and complex tasks" should be "task"; the sentence should be grammatically revised.
  2. [Section 3.2] "two big" should be "too big".
  3. [Section 6] "reported buy Khadka" should be "reported by Khadka".
  4. [Section 4.4] "identity different" should be "identify different".
  5. [Section 3.1] The definition of "Number of External Links" is ambiguous; please specify whether it counts distinct external services or distinct call paths, and how this interacts with the FEC measure.
  6. [Figures 3 and 4] Figures 3 and 4 are referenced but their content is not fully described in the text; please clarify what the reader should observe in Figure 4, especially the duplication scenario.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the four metrics are counting definitions over execution traces, and the prior-work self-citation is provenance rather than the load-bearing evidence for the measurement claim.

full rationale

The paper's measurement framework defines four countable metrics from execution traces: CBM = external links/classes (Sec. 3.1), CLA = classes per microservice (Sec. 3.2), DUP = duplicated classes (Sec. 3.3), and FEC = call instances/classes (Sec. 3.4). These are definitions, not derived predictions; no parameter is fitted and no empirical quantity is later 'predicted' from the framework, so there is no equation whose output equals an input by construction. The only author-self citation is the provenance of the CBM formula and of the decomposition pipeline from [36] (Sec. 3.1, Sec. 4); that earlier work is not used to force the current metrics because the formulas are stated fully in this paper and can be applied independently, exactly as the paper claims. The paper explicitly defers empirical validation ('Future works include the validation of the framework', Sec. 7), which is an evidence gap regarding the objective-comparison claim, but an unsupported claim is not a circular derivation. The reviewer concern that class-count denominators make CBM/FEC sensitive to class granularity is a validity threat, not a self-referential reduction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the availability of complete runtime traces, the coupling-cohesion inverse relation, and the heavier cost of external calls. No fitted parameters are part of the metric definitions; the 1000x relative-cost value appears only in the illustrative example.

free parameters (1)
  • relative cost of external call (1000x) = 1000 (illustrative)
    Section 3.4 and Table 2 assume an external call is 1000 times heavier than an internal call to compute example 'load' values. This hand-chosen value is not part of the metric definitions and no sensitivity analysis is given.
assumptions (4)
  • domain assumption Complete end-to-end runtime log traces are available for the system.
    Section 2.4 states the core assumption: the log must trace the entire request from entry point to database and result, including start and end events.
  • domain assumption Coupling is inversely proportional to cohesion, so low coupling implies high likelihood of high cohesion.
    Section 3.1 adopts this relation to justify CBM as a coupling proxy for a cohesion goal.
  • domain assumption External service calls are computationally substantially heavier than internal calls.
    Section 3.4 motivates FEC as a performance-oriented measure; no quantitative evidence is provided.
  • domain assumption High cohesion and low coupling are the key quality targets for microservice decompositions.
    Section 2.3 takes this as the architectural goal from microservices literature, e.g., Newman [19] and Fowler/Lewis [7].

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Decomposition and Metric-Based Evaluation Framework for Microservices." pith.science (2026). https://pith.science/paper/NNNNRV33

@misc{pith2026190808513,
  author       = {Pith},
  title        = {Pith review of: A Decomposition and Metric-Based Evaluation Framework for Microservices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NNNNRV33}},
  note         = {Machine review of arXiv:1908.08513}
}
read the original abstract

Migrating from monolithic systems into microservice is a very complex task. Companies are commonly decomposing the monolithic system manually, analyzing dependencies of the monolith and then assessing different decomposition options. The goal of our work is two-folded: 1) we provide a microservice measurement framework to objectively evaluate and compare the quality of microservices-based systems; 2) we propose a decomposition system based on business process mining. The microservice measurement framework can be applied independently from the decomposition process adopted, but is also useful to continuously evaluate the architectural evolution of a system. Results show that the decomposition framework helps companies to easily identify the different decomposition options. The measurement framework can help to decrease the subjectivity of the decision between different decomposition options and to evaluate architectural erosion in existing systems.

Figures

Figures reproduced from arXiv: 1908.08513 by the authors.

Figure 1
Figure 1. Architectures of Microservices and Monolithic systems [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The Decomposition Process (From [36]) When the log files are available, the decomposition process defined in ( [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Simplified Process Example (From [36]) 6 https://fluxicon.com/disco/ [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Simplified Process Example (From [36]) 4.2 Step 2: Frequency Analysis of the Execution Paths. In our approach, the call frequency is a major contributor for the produced recommen￾dations. Thus the frequency should be studied and analyzed. For visual inspection, the pro…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages

  1. [36]

    In: 9th International Conference on Cloud Computing and Services Science, CLOSER , 2019

    Taibi, D., Systa, K.: From monolithic systems to microservices: A decomposition framework based on process mining. In: 9th International Conference on Cloud Computing and Services Science, CLOSER , 2019. Heraklion (Greece) (05/2019 2019)

    work page 2019

  2. [1]

    In: Proceedings of the 27th International Workshop on Software Measurement and 12th International Conference on Software Process and Product Measurement

    Bogner, J., Wagner, S., Zimmermann, A.: Automatically measuring the maintainability of service- and microservice-based systems: A literature review. In: Proceedings of the 27th International Workshop on Software Measurement and 12th International Conference on Software Process and Product Measurement. pp. 107–115. IWSM Mensura ’17, ACM, New York, NY , USA...

    work page arXiv 2017

  3. [2]

    In: 2017 24th Asia-Pacific Software Engineering Conference (APSEC)

    Chen, R., Li, S., Li, Z.: From monolith to microservices: A dataflow-driven approach. In: 2017 24th Asia-Pacific Software Engineering Conference (APSEC). pp. 466–475 (Dec 2017)

    work page 2017

  4. [3]

    IEEE Trans

    Chidamber, S.R., Kemerer, C.F.: A metrics suite for object oriented design. IEEE Trans. Softw. Eng. 20(6), 476–493 (1994)

    work page 1994

  5. [4]

    In: Pahl, C., Vukovic, M., Yin, J., Yu, Q

    De Alwis, A.A.C., Barros, A., Polyvyanyy, A., Fidge, C.: Function-splitting heuristics for discovery of microservices in enterprise systems. In: Pahl, C., Vukovic, M., Yin, J., Yu, Q. (eds.) Service-Oriented Computing. pp. 37–53. Springer International Publishing, Cham (2018)

    work page 2018

  6. [5]

    In: EUROGEN’99

    Deb, K.: Evolutionary algorithms for multi-criterion optimization in engineering design. In: EUROGEN’99. pp. 135–161 (1999)

    work page 1999

  7. [6]

    Fenton, N., Bieman, J.: Software Metrics: A Rigorous and Practical Approach, Third Edition. 3rd edn. (2014)

    work page 2014

  8. [7]

    Fowler, M., Lewis, J.: Microservices (2014), http://martinfowler.com/articles/ microservices.html

    work page 2014

Show all 39 references
  1. [8]

    In: Software Engineering Aspects of Continuous Development and New Paradigms of Software Production and Deployment (2018)

    Fritzsch, J., Bogner, J., Zimmermann, A., Wagner, S.: From monolith to microservices: A classification of refactoring approaches. In: Software Engineering Aspects of Continuous Development and New Paradigms of Software Production and Deployment (2018)

    2018

  2. [9]

    In: European Conference, ESOCC 20162016

    Gysel, M., K ¨olbener, L., Giersche, W., Zimmermann, O.: Service cutter: A systematic ap- proach to service decomposition. In: European Conference, ESOCC 20162016. pp. 185–200 (2016)

    2016

  3. [10]

    Em- pirical Softw

    Jabangwe, R., B ¨orstler, J., Smite, D., Wohlin, C.: Empirical evidence on the link between object-oriented measures and external quality attributes: A systematic literature review. Em- pirical Softw. Engg. 20(3), 640–693 (2015)

    2015

  4. [11]

    In: 2016 International Conference on High Performance Com- puting Simulation (HPCS)

    Kecskemeti, G., Marosi, A.C., Kertesz, A.: The entice approach to decompose monolithic services into microservices. In: 2016 International Conference on High Performance Com- puting Simulation (HPCS). pp. 591–596 (July 2016)

    2016

  5. [12]

    In: 2013 20th Working Conference on Reverse Engineering (WCRE)

    Khadka, R., Saeidi, A., Jansen, S., Hage, J., Haas, G.P.: Migrating a large scale legacy application to soa: Challenges and lessons learned. In: 2013 20th Working Conference on Reverse Engineering (WCRE). pp. 425–432 (Oct 2013). https://doi.org/10.1109/WCRE.2013.6671318

    2013

  6. [13]

    ACM Transaction on Software Engineering Methodologies13(3), 332–358 (2004)

    Kramer, S., Kaindl, H.: Coupling and cohesion metrics for knowledge-based systems using frames and rules. ACM Transaction on Software Engineering Methodologies13(3), 332–358 (2004)

    2004

  7. [14]

    Lehman, M.M.: On understanding laws, evolution, and conservation in the large-program life cycle. J. Syst. Softw. 1, 213–221 (Sep 1984). https://doi.org/10.1016/0164-1212(79)90022- 0, http://dx.doi.org/10.1016/0164-1212(79)90022-0

    1984 doi

  8. [15]

    In: 39th International Conference on Software Engineering Companion

    Lenarduzzi, V ., Sillitti, A., Taibi, D.: Analyzing forty years of software maintenance models. In: 39th International Conference on Software Engineering Companion. ICSE-C ’17, IEEE Press (2017). https://doi.org/10.1109/ICSE-C.2017.122, https://doi.org/10. 1109/ICSE-C.2017.122

    2017 doi

  9. [16]

    In: 11th European Conference on Information Systems Man- agement (ECISM) (2017)

    Lenarduzzi, V ., Stan, C., Taibi, D., Venters, G.: A dynamical quality model to continuously monitor software maintenance. In: 11th European Conference on Information Systems Man- agement (ECISM) (2017)

    2017

  10. [17]

    Prentice Hall PTR, Upper Saddle River, NJ, USA (2003)

    Martin, R.C.: Agile Software Development: Principles, Patterns, and Practices. Prentice Hall PTR, Upper Saddle River, NJ, USA (2003)

    2003

  11. [18]

    In: Jedl- itschka, A., Kuvaja, P., Kuhrmann, M., M ¨annist¨o, T., M ¨unch, J., Raatikainen, M

    Mikkonen, T., Syst ¨a, K.: Maximizing product value: Continuous maintenance. In: Jedl- itschka, A., Kuvaja, P., Kuhrmann, M., M ¨annist¨o, T., M ¨unch, J., Raatikainen, M. (eds.) Product-Focused Software Process Improvement. pp. 298–301. Springer International Pub- lishing, Ch...

    2014

  12. [19]

    O’Reilly Media, Inc., 1st edn

    Newman, S.: Building Microservices. O’Reilly Media, Inc., 1st edn. (2015)

    2015

  13. [20]

    Parnas, D.L.: On the criteria to be used in decomposing systems into modules. Commun. ACM 15(12), 1053–1058 (1972)

    1972

  14. [21]

    https:// github.com/clowee/MicroDepGraph (2019)

    Rahman, M., Taibi, D.: Microservice dependency graph (microdepgraph). https:// github.com/clowee/MicroDepGraph (2019)

    2019

  15. [22]

    In: Proceedings of the joint Summer School on Software Evolution, Software Quality and Bug Prediction with Machine Learning

    Rahman, M., Taibi, D.: The microservice dependency graph dataset. In: Proceedings of the joint Summer School on Software Evolution, Software Quality and Bug Prediction with Machine Learning. CEUR-WS (September 2019)

    2019

  16. [23]

    Richardson, C.: Pattern: Microservice architecture (2017), http://microservices.io/ patterns/microservices.html

    2017

  17. [24]

    Richardson, C.: The scale cube (2017), https://microservices.io/articles/ scalecube.html

    2017

  18. [25]

    Journal of Systems and Software146, 215 – 232 (2018)

    Soldani, J., Tamburri, D.A., Heuvel, W.J.V .D.: The pains and gains of microservices: A sys- tematic grey literature review. Journal of Systems and Software146, 215 – 232 (2018)

    2018

  19. [26]

    In: International Workshop on Software Measurement (IWSM)

    Suonsyrj ¨a, S.: Designing an unobtrusive analytics framework for monitoring java applica- tions. In: International Workshop on Software Measurement (IWSM). pp. 160–175 (2015)

    2015

  20. [27]

    Information & Software Technology92, 223–235 (2017)

    Taibi, D., Janes, A., Lenarduzzi, V .: How developers perceive smells in source code: A repli- cated study. Information & Software Technology92, 223–235 (2017)

    2017

  21. [28]

    IEEE Software35(3), 56–62 (2018)

    Taibi, D., Lenarduzzi, V .: On the definition of microservice bad smells. IEEE Software35(3), 56–62 (2018)

    2018

  22. [29]

    In: Proceed- ings of the 21st International Conference on Evaluation and Assessment in Software Engi- neering

    Taibi, D., Lenarduzzi, V ., Ahmad, M.O., Liukkunen, K.: Comparing communication effort within the scrum, scrum with kanban, xp, and banana development processes. In: Proceed- ings of the 21st International Conference on Evaluation and Assessment in Software Engi- neering. pp. ...

    2017

  23. [30]

    In: Agile Processes in Software Engineering and Extreme Programming

    Taibi, D., Lenarduzzi, V ., Janes, A., Liukkunen, K., Ahmad, M.O.: Comparing requirements decomposition within the scrum, scrum with kanban, xp, and banana development processes. In: Agile Processes in Software Engineering and Extreme Programming. pp. 68–83 (2017)

    2017

  24. [31]

    IEEE Cloud Computing 4(5), 22–32 (2017)

    Taibi, D., Lenarduzzi, V ., Pahl, C.: Processes, motivations, and issues for migrating to mi- croservices architectures: An empirical investigation. IEEE Cloud Computing 4(5), 22–32 (2017)

    2017

  25. [32]

    8th International Conference on Cloud Computing and Services Science (CLOSER2018) (2018)

    Taibi, D., Lenarduzzi, V ., Pahl, C.: Architectural patterns for microservices: a systematic mapping study. 8th International Conference on Cloud Computing and Services Science (CLOSER2018) (2018)

    2018

  26. [33]

    Microservices - Science and Engineering

    Taibi, D., Lenarduzzi, V ., Pahl, C.: Microservices anti-patterns: A taxonomy. Microservices - Science and Engineering. Springer. 2019 (2019)

    2019

  27. [34]

    Cloud Computing and Services Science

    Taibi, D., Lenarduzzi, V ., Pahl, C.: Microservices architectural, code and organizational anti- patterns. Cloud Computing and Services Science. CLOSER 2018 Selected papers. Commu- nications in Computer and Information Science pp. 126–151 (2019)

    2019

  28. [35]

    In: XP Workshops

    Taibi, D., Lenarduzzi, V ., Pahl, C., Janes, A.: Microservices in agile software development: a workshop-based study into issues, advantages, and disadvantages. In: XP Workshops. pp. 23:1–23:5. ACM (2017)

    2017

  29. [37]

    In: ESEM

    Tosi, D., Lavazza, L., Morasca, S., Taibi, D.: On the definition of dynamic software measures. In: ESEM. pp. 39–48. ACM (2012)

    2012

  30. [38]

    In: MIPRO

    Vresk, T., Cavrak, I.: Architecture of an interoperable iot platform based on microservices. In: MIPRO. pp. 1196–1201. IEEE (2016)

    2016

  31. [39]

    Computer Science - Research and Development 32(3), 301–310 (Jul 2017)

    Zimmermann, O.: Microservices tenets. Computer Science - Research and Development 32(3), 301–310 (Jul 2017)

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.