Structural Validation of LLM-Generated Microservice Decompositions Using Source-Code Dependencies

Ademar Sousa Neto; Angelo Perkusich; Daniel Silva; Danyllo Wagner Albuquerque; Emanuel Dantas Filho; Kyller Gorg\^onio; Mirko Perkusich; Renan Alves

REVIEW 2 major objections 4 minor 19 references

Structural Validation of LLM-Generated Microservice Decompositions Using Source-Code Dependencies

T0 review · 2 major / 4 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Once class-to-service mapping coverage is equalized, zero-shot and few-shot LLM microservice decompositions show identical structural adherence to source-code dependencies.

desk verdict Narrow but clean methodological result: coverage mismatch can fake prompt-strategy winners, and after intersection normalization o3 zero-shot and few-shot tie on structural adherence. read the letter →

arxiv 2607.28331 v1 pith:NZZUDDBD submitted 2026-07-30 cs.SE

Daniel Silva , Renan Alves , Emanuel Dantas Filho , Ademar Sousa Neto , Mirko Perkusich , Danyllo Wagner Albuquerque , Kyller Gorgônio , Angelo Perkusich This is my paper · ORCID

classification cs.SE

keywords LargeLanguageModelsMicroserviceArchitectureMonolithDecompositionStaticAnalysisStructuralValidationReproducibilityPromptingStrategies

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models can propose microservice splits from textual requirements that look coherent on paper, yet it is unclear whether those splits respect the real dependency structure already in the code. This paper builds an automated pipeline that extracts a static dependency graph from two Java monoliths, maps classes onto the proposed services, and measures how many dependencies stay inside a service versus cross a boundary. After forcing both prompting strategies onto exactly the same set of mapped classes, the apparent score gaps vanish: both preserve 68 percent of dependencies on PetClinic and 83 percent on Bookstore. The result matters because raw comparisons of prompting styles can be pure measurement artifacts; any structural evaluation of AI-generated architectures must control mapping coverage before claiming one approach is better.

What carries the argument

An automated structural validation pipeline that extracts a real dependency graph via static analysis, maps monolith classes to LLM-proposed services, flags cross-boundary edges as violations, and reports Total Violated Dependencies (TVD) and Total Preserved Dependencies (TPD = 1 − TVD), recomputed on the intersection of mapped classes to remove coverage bias.

What would settle it

Re-run the same pipeline on these systems (or larger ones) with complete, independently audited class-to-service mappings that cover every domain class for both strategies; a clear TPD advantage for one prompt style under full equal coverage would falsify the equivalence claim.

Watch

Extended reading notes

Core claim

After normalizing to the intersection of class-to-service mappings, OpenAI o3 decompositions produced under zero-shot and few-shot prompting exhibit identical structural adherence on both systems—TPD 68.0 percent (TVD 32.0 percent) for PetClinic and TPD 83.3 percent (TVD 16.7 percent) for Bookstore. Differences seen in the raw metrics are entirely explained by unequal mapping coverage, not by genuine differences in architectural quality between the prompting strategies.

Load-bearing premise

That restricting the comparison to classes both strategies successfully mapped, after name heuristics and manual fixes for a few framework edges, yields a fair enough dependency set to conclude the prompting styles are structurally equivalent.

Editorial extensions

If this is right

Structural adherence metrics (TVD/TPD) should accompany semantic scores in benchmarks of LLM-based architecture generation.
Comparisons of prompting strategies must report class-to-service mapping coverage and normalize whenever coverages differ.
The pipeline can serve as a pre-migration audit that surfaces coupling hotspots and estimates structural adaptation effort.
Remaining violations are attributed more to missing structural input than to prompt style, favoring hybrid semantic-plus-structural methods over further prompt engineering alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Coverage bias may systematically hide subsystems rich in dependencies (e.g., authentication), so mapping-coverage artifacts should become required reporting in this literature.
The same partial-assignment bias can distort other LLM software-engineering evaluations that map code entities to labels incompletely.
Feeding detected violations back as constraints in an iterative generation loop is a direct, testable way to close the remaining 17–32 percent preservation gap.
Pairing the static graph with light dynamic traces would better capture reflection and framework-mediated edges that static analysis misses.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing worth knowing is the bias finding, not a new decomposer. They take existing o3 zero/few-shot decompositions for PetClinic and Bookstore, build a Tree-sitter dependency graph, map classes to services, and score TVD/TPD. Raw numbers look different by prompt; after restricting to the intersection of mapped classes, the scores are identical (PetClinic TPD 68%, Bookstore 83.3%). The Bookstore JWT omission (13 classes, ~18 edges) is a concrete, checkable explanation. That is real evaluation hygiene for anyone comparing LLM architecture outputs.

What is new is the attachable audit pipeline plus the explicit demonstration that unequal mapping coverage can manufacture a “winner.” Generation-from-requirements and code-driven decomposers already exist; this paper sits downstream and asks whether those proposals respect implemented edges. The before/after tables make the central claim easy to verify. Shipping a Zenodo package with graphs, mappings, and metrics is a genuine plus in this area.

Soft spots are mostly scope, not internal contradiction. N=2 Java systems and one model; static edges only; class-to-service mapping still needs heuristics plus manual cleanup (they flag framework-mediated cases like Vet→Specialty). Intersection normalization removes coverage bias but also shrinks the evaluated universe, so “equivalent structural adherence” is conditional on that shared set. TVD/TPD measure migration friction against the monolith graph, not overall architectural quality—they say so. Mild circularity: they audit decompositions from an overlapping author line, but the metrics are grounded in an independent static graph, so that is not load-bearing fraud.

Math is trivial proportions; data story is transparent; citations cover Newman/Fowler, reviews, and the generation/code-driven lines without obvious padding. This is for people who evaluate or adopt LLM-assisted monolith splits and want pre-migration structural checks, not for someone hunting a new clustering algorithm.

I would send it to peer review. Expect referees to push external validity and mapping subjectivity, but the narrow claim is solid enough to deserve that time. Engage if you care about evaluation protocol for AI-generated architectures; skip if you only want generation methods.

Referee Report

2 major / 4 minor

Summary. This paper proposes an automated pipeline to structurally validate LLM-generated microservice decompositions against source-code dependencies extracted by static analysis. Using OpenAI o3 decompositions of PetClinic and Bookstore under zero-shot and few-shot prompting, it defines TVD/TPD metrics over a dependency graph and shows that raw differences between strategies disappear after restricting evaluation to the intersection of class-to-service mappings. Normalized TPD is 68.0% (PetClinic) and 83.3% (Bookstore) for both strategies. The central claim is methodological: structural comparisons of LLM decompositions must control mapping coverage, or apparent prompting effects may be artifacts of unequal class sets rather than architectural quality.

Significance. If the result holds, the paper makes a useful and timely methodological contribution to LLM-assisted software architecture. It cleanly separates semantic plausibility of service boundaries from structural feasibility relative to an existing monolith, and it documents a concrete bias (unequal mapping coverage) that can invalidate prompt-strategy comparisons. Strengths include a transparent before/after robustness analysis (Tables 1–2), an auditable extract–map–violate–score pipeline with a public replication package, and appropriately cautious interpretation of TVD as migration-adaptation effort rather than absolute architectural quality. The scope is narrow (two Java systems, one model, static edges only), but the bias diagnosis is transferable and should influence how future empirical studies report structural metrics for AI-generated decompositions.

major comments (2)

[§3 Stage 3; §4.1 Table 2] §3 Stage 3 and §4.1–4.2: the equality claim rests on intersection-normalized mappings after heuristic name/package matching plus manual refinement, including framework-mediated edges (e.g., Vet→Specialty). The paper should quantify residual mapping ambiguity (how many classes required manual adjudication; inter-rater checks if any) and state more explicitly that equivalence is established only on the commonly mapped subgraph. Without that, readers cannot judge how sensitive Table 2’s identical violation counts are to mapping choices.
[§3 Stage 2; §7 Internal validity] §3 Stage 2 and §7: the evaluated decompositions are taken from prior overlapping-author studies rather than regenerated under a fixed protocol in this work. That is acceptable for a validation-focused paper, but the manuscript should more clearly bound the claim—prompting strategies are structurally equivalent for these published o3 artifacts after coverage control—not that zero-shot and few-shot are generally equivalent for o3 or other models. A short sensitivity note on whether regenerating decompositions could change coverage patterns would strengthen internal validity.

minor comments (4)

[§3 Figure 1] Figure 1 is referenced as the methodology overview but is not available in the text package; ensure the camera-ready figure clearly shows the four stages and where coverage normalization occurs.
[§3 Stage 4] Formal definitions of TVD/TPD appear after the metrics are introduced narratively; moving the set-theoretic definition earlier in §3 Stage 4 would improve readability.
[References] Several bibliography entries are still anonymized placeholders ([1], [12]); replace with full citations for production.
Minor wording inconsistencies (e.g., “OpenAI o3” vs. model naming elsewhere; “PetClinicApplication” coverage detail) should be normalized for clarity.

Circularity Check

1 steps flagged · score 1.0 of 10

No derivation circularity: TVD/TPD equality is an independent static-graph measurement; mild self-audit of prior in-group decompositions is not load-bearing.

other [§3 Stage 2; §4.1 Tables 1–2; citations [1],[2],[5]]
"The analyzed decompositions were previously generated by OpenAI o3 under zero-shot and few-shot prompting strategies in the studies of Albuquerque et al. [1] and Pereira et al. [2, 5]."

Not true derivation circularity: the objects under test are prior outputs from an overlapping author line, so the paper partly audits its own research stream. The structural metrics and the coverage-normalization argument are independently computed from source-code dependencies and do not inherit their numerical equality from those citations. Flagged only as mild self-audit dependence, not a load-bearing circular step.

full rationale

The paper’s central claim—that after intersecting class-to-service mappings, zero-shot and few-shot yield identical TPD/TVD (68.0%/32.0% PetClinic; 83.3%/16.7% Bookstore), and that raw differences were coverage artifacts—does not reduce to its inputs by construction. G_real is extracted independently via Tree-sitter static analysis; TVD/TPD are defined as the fraction of edges that cross (or stay within) the proposed partition s(c), with TPD = 1 − TVD. Normalization recomputes those counts over the same edge set; identical violation counts in Table 2 are arithmetic consequences of that control, not a fitted parameter renamed as a prediction. The evaluated decompositions come from overlapping-author prior work (Albuquerque et al., Pereira et al.), so the study partly audits its own line’s outputs, but that is ordinary choice of experimental objects, not a self-citation uniqueness theorem or ansatz that forces the structural-equivalence result. No self-definitional loop, fitted-input-as-prediction, or renamed known law is present. Score 1 only for that mild in-group object dependence; the measurement chain itself is self-contained.

Assumptions & free parameters 0 free parameters · 5 assumptions · 2 invented entities

The load-bearing claim rests on standard static-analysis practice, definitions of preserved vs violated edges under a class-to-service map, and experimental controls (same systems, model, metrics; differ only by prompt). No physical constants or fitted scaling laws. The main non-standard burdens are the completeness of static extraction, the correctness of (partly manual) mappings, and treating intersection-normalized TVD/TPD as the right comparison target for ‘structural adherence’ and prompt equivalence.

assumptions (5)

standard math A dependency (ci, cj) is preserved iff s(ci)=s(cj) and violated otherwise; TPD = 1 - TVD over the chosen edge set E.
Definitional metric construction in Stage 4 / formal display of TVD and TPD; standard partition-vs-graph cut accounting.
domain assumption Tree-sitter Java static analysis plus limited manual fixes sufficiently recovers the structural dependencies that matter for migration effort (inheritance, DI, repositories, JPA).
Stage 3 static extraction; threats section notes reflection, proxies, and framework-mediated links may be missed.
domain assumption Comparing strategies on the intersection of mapped classes removes bias without introducing selection bias that would hide true prompt differences.
Robustness analysis §4.1; central to the equivalence claim after normalization.
domain assumption TVD indicates potential structural adaptation effort and is an appropriate complement to semantic service-quality metrics, not a full architecture-quality score.
Stated explicitly when interpreting metrics and in Discussion; shapes what the equality result is allowed to mean.
ad hoc to paper Previously published o3 zero/few-shot decompositions for PetClinic and Bookstore are fair experimental objects for prompt-strategy comparison under a fixed model.
Stage 2: objects taken from Albuquerque/Pereira line rather than regenerated inside this protocol; internal-validity threat noted by authors.

invented entities (2)

TVD/TPD structural adherence metrics (as used in this pipeline) independent evidence
purpose: Quantify fraction of monolith dependencies cut vs kept by an LLM service partition.
Named and formalized for this evaluation protocol; conceptually standard cut rates on a dependency graph, not a new physical entity.
Automated structural validation pipeline (extract → map → violate → score) independent evidence
purpose: Make LLM decompositions auditable against source-code dependencies with reproducible artifacts.
Main methodological artifact; existence is demonstrated by implementation description and replication package claim.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Structural Validation of LLM-Generated Microservice Decompositions Using Source-Code Dependencies." pith.science (2026). https://pith.science/paper/NZZUDDBD

@misc{pith2026260728331,
  author       = {Pith},
  title        = {Pith review of: Structural Validation of LLM-Generated Microservice Decompositions Using Source-Code Dependencies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NZZUDDBD}},
  note         = {Machine review of arXiv:2607.28331}
}

read the original abstract

Decomposing monolithic systems into microservices is a key activity in software modernization. Although Large Language Models (LLMs) can generate semantically plausible decompositions from textual requirements, it remains unclear whether these proposals preserve the structural dependencies implemented in the source code. This paper evaluates the structural adherence of microservice decompositions generated by OpenAI o3 for the PetClinic and Bookstore systems. We propose an automated validation pipeline based on static dependency analysis and compare zero-shot and few-shot prompting using dependency preservation (TPD) and dependency violation (TVD) metrics. A robustness analysis was conducted to control for differences in class-to-service mapping coverage. After normalization, both prompting strategies produced equivalent structural adherence, achieving TPD values of 68.0% (PetClinic) and 83.3% (Bookstore). The findings demonstrate that structural evaluations of LLM-generated decompositions should explicitly control for mapping coverage, as apparent differences between prompting strategies may otherwise reflect methodological bias rather than genuine architectural quality.

Figures

Figures reproduced from arXiv: 2607.28331 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 1 linked inside Pith

[1]

Submitted for review, 2026

Anonymous.Anonymized. Submitted for review, 2026. Available at: https:// submission-2026.com/paper01

2026
[2]

In:Anais do XIX Simpósio Brasileiro de Componentes, Arquiteturas e Reutilização de Software (SBCARS), p

Pereira, Jose Renan A.et al.Toward Generating Microservice Architectures from Textual Requirements with Large Language Models. In:Anais do XIX Simpósio Brasileiro de Componentes, Arquiteturas e Reutilização de Software (SBCARS), p. 79–89, Porto Alegre, RS, 2025. 6 Validating LLM-Generated Monolith Conference’17, July 2017, Washington, DC, USA

2025
[3]

MonoEmbed: Enhancing LLM rep- resentations for monolith to microservices decomposition through contrastive learning.Empirical Software Engineering, v

Sellami, Khaled; Saied, Mohamed Aymen. MonoEmbed: Enhancing LLM rep- resentations for monolith to microservices decomposition through contrastive learning.Empirical Software Engineering, v. 31, n. 11, 2026

2026
[4]

MicroDec: Leveraging Large Language Models for Microservice Decomposition.Journal of Systems and Software, v

Alsayed, Ahmed Saeed; Dam, Hoa Khanh; Nguyen, Chau. MicroDec: Leveraging Large Language Models for Microservice Decomposition.Journal of Systems and Software, v. 203, 111622, 2024

2024
[5]

In:The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), Thessaloniki, Grécia, 2026

Pereira, José Renan A.et al.Do LLMs Agree on Microservice Decompositions? A Multi–Model Study from Textual Requirements. In:The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), Thessaloniki, Grécia, 2026

2026
[6]

Se- bastopol, CA: O’Reilly Media, 2015

Newman, Sam.Building Microservices: Designing Fine-Grained Systems. Se- bastopol, CA: O’Reilly Media, 2015

2015
[7]

Microservices: a definition of this new architectural term

Lewis, James; Fowler, Martin. Microservices: a definition of this new architectural term. ThoughtWorks, 2014. https://martinfowler.com/articles/microservices.html

2014
[8]

In:2020 IEEE International Conference on Software Architecture Companion (ICSA-C), p

Krause, Alexanderet al.Microservice Decomposition via Static and Dynamic Analysis of the Monolith. In:2020 IEEE International Conference on Software Architecture Companion (ICSA-C), p. 9–16, 2020

2020

Show all 19 references

[9]

In:European Conference on Software Architecture (ECSA), p

Matias, Tiagoet al.Determining Microservice Boundaries: A Case Study Using Static and Dynamic Software Analysis. In:European Conference on Software Architecture (ECSA), p. 315–332, 2020

2020
[10]

Abgaz, Yalemisewet al.Decomposition of Monolith Applications into Microser- vices Architectures: A Systematic Review.IEEE Transactions on Software Engi- neering, v. 49, n. 8, p. 4213–4242, 2023

2023
[11]

In:Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE ’25), ACM, 2025

Bucaioni, Alessioet al.Artificial Intelligence for Software Architecture: Literature Review and the Road Ahead. In:Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE ’25), ACM, 2025

2025
[12]

Submitted for review, 2026

Anonymous.Anonymized. Submitted for review, 2026. Available at: https:// submission-2026.com/paper02

2026
[13]

13, 2025

Ait Mansour, Nassimaet al.Semantic Approaches to Microservice Identification: A Systematic Literature Review.IEEE Access, v. 13, 2025

2025
[14]

In:39th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW ’24), p

Sooksatra, Kornet al.Using Static Analysis to Aid Monolith to Microservice System Transformation: Tuning Fuzzy c-Means in a VAE-Based GNN Approach. In:39th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW ’24), p. 43–53, 2024

2024
[15]

In:2024 L Latin American Computer Conference (CLEI), p

Saucedo, Ana Martínezet al.On the Variability of Microservice Decompositions: A Data-Driven Analysis. In:2024 L Latin American Computer Conference (CLEI), p. 1–9, 2024

2024
[16]

Saucedo, Ana Martínezet al.Exploring Alternative Microservice Decompositions using Data-driven Techniques and LLMs.CLEI Electronic Journal, v. 28, n. 3, paper 6, 2025

2025
[17]

Automated Migration of Monolithic Systems into Microservices- based Architecture

Sellami, Khaled. Automated Migration of Monolithic Systems into Microservices- based Architecture. 217 f. Tese (Doutorado em Informática) – Université Laval, Québec, Canadá, 2026

2026
[18]

Weerasinghe, Minethet al.From Monolith to Microservices: A Comparative Evaluation of Decomposition Frameworks.arXiv preprint arXiv:2601.23141, 2026

2026 arXiv
[19]

MicroRec: Leveraging Large Language Models for Microservice Recommendation

Alsayed, Ahmed Saeed; Dam, Hoa Khanh; Nguyen, Chau. MicroRec: Leveraging Large Language Models for Microservice Recommendation. In:21st International Conference on Mining Software Repositories (MSR ’24), ACM, 2024. 7

2024

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.

[1] [1]

Submitted for review, 2026

Anonymous.Anonymized. Submitted for review, 2026. Available at: https:// submission-2026.com/paper01

2026

[2] [2]

In:Anais do XIX Simpósio Brasileiro de Componentes, Arquiteturas e Reutilização de Software (SBCARS), p

Pereira, Jose Renan A.et al.Toward Generating Microservice Architectures from Textual Requirements with Large Language Models. In:Anais do XIX Simpósio Brasileiro de Componentes, Arquiteturas e Reutilização de Software (SBCARS), p. 79–89, Porto Alegre, RS, 2025. 6 Validating LLM-Generated Monolith Conference’17, July 2017, Washington, DC, USA

2025

[3] [3]

MonoEmbed: Enhancing LLM rep- resentations for monolith to microservices decomposition through contrastive learning.Empirical Software Engineering, v

Sellami, Khaled; Saied, Mohamed Aymen. MonoEmbed: Enhancing LLM rep- resentations for monolith to microservices decomposition through contrastive learning.Empirical Software Engineering, v. 31, n. 11, 2026

2026

[4] [4]

MicroDec: Leveraging Large Language Models for Microservice Decomposition.Journal of Systems and Software, v

Alsayed, Ahmed Saeed; Dam, Hoa Khanh; Nguyen, Chau. MicroDec: Leveraging Large Language Models for Microservice Decomposition.Journal of Systems and Software, v. 203, 111622, 2024

2024

[5] [5]

In:The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), Thessaloniki, Grécia, 2026

Pereira, José Renan A.et al.Do LLMs Agree on Microservice Decompositions? A Multi–Model Study from Textual Requirements. In:The 41st ACM/SIGAPP Symposium on Applied Computing (SAC ’26), Thessaloniki, Grécia, 2026

2026

[6] [6]

Se- bastopol, CA: O’Reilly Media, 2015

Newman, Sam.Building Microservices: Designing Fine-Grained Systems. Se- bastopol, CA: O’Reilly Media, 2015

2015

[7] [7]

Microservices: a definition of this new architectural term

Lewis, James; Fowler, Martin. Microservices: a definition of this new architectural term. ThoughtWorks, 2014. https://martinfowler.com/articles/microservices.html

2014

[8] [8]

In:2020 IEEE International Conference on Software Architecture Companion (ICSA-C), p

Krause, Alexanderet al.Microservice Decomposition via Static and Dynamic Analysis of the Monolith. In:2020 IEEE International Conference on Software Architecture Companion (ICSA-C), p. 9–16, 2020

2020