REVIEW 4 major objections 5 minor 33 references
LLM-Ehnanced Holonic Architecture for Ad-Hoc Scalable SoS
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper proposes a three-layer holonic architecture—reasoning, communication, capabilities—with large language models as the reasoning core of every holon, and adds supervisor, planner, task, and resource holons so that systems of…
desk verdict A clear conceptual architecture for LLM-driven holons whose central real-time claim is unverified; readable, honest, but too thin on evidence for acceptance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the holon, defined as a semi-autonomous unit that is simultaneously a whole and a part of a larger whole. The machinery is the three-layer holon: a reasoning layer whose core is a large language model (command processing, context management, decision making), a communication layer that translates the LLM's action plans into executable commands through a robotics middleware and routes messages between holons, and a capabilities layer that wraps the constituent system's resources. Around this core sit four specialised holon roles—supervisor, planner, task, and resource—each given role-specific prompt context, with the human resource holon providing the natural-language interface for people. The same machinery also does dual-level planning: a supervisor-level plan is decomposed into task holons and sub-plans, so local changes can be handled without restarting the global mission.
What would settle it
Take the paper's own 3D mobility scenario and inject a road closure after a task holon has committed to a route; if the LLM's replanning exceeds the response-time budget or yields an action plan the vehicle cannot execute, with no non-LLM fallback catching the error, then the claimed real-time adaptability is refuted.
Extended reading notes
Core claim
The paper's central claim is that specialised holons—supervisor, planner, task, and resource—use large language models within their reasoning layers to support decision-making and ensure real-time adaptability, and that the layered holon design lets each holon adapt autonomously to environmental changes while keeping behaviour coherent across the whole system of systems. The reasoning layer gives every holon the same three-part routine: preprocess an incoming command, build a context-aware prompt, and produce an action plan that the communication layer turns into concrete commands for the capabilities layer. This is presented as the first introduction of manufacturing-style specialised holons in the system-of-systems context, and the 3D mobility case study is the demonstration: a passenger's natural-language request is refined by a supervisor holon, split into driving and flying task holons, and executed by resource holons representing vehicles.
Load-bearing premise
The whole design collapses unless a large language model can reliably reason in real time—returning correct action plans quickly enough for moving vehicles—and unless unknown third-party systems can be wrapped into the holon's three layers without their vendors' cooperation; both are stated but not yet shown by measurement.
Editorial extensions
If this is right
- Heterogeneous, black-box constituent systems can be brought into the system of systems at runtime by wrapping each in the same holon facade, since the LLM interprets commands and the communication layer standardises how capabilities are invoked.
- A supervisor holon can produce a high-level plan while task holons locally replan sub-segments, so resource reallocation—for example, switching from vehicle $m_1$ to $m_3$ for a later leg—happens without rebuilding the whole mission.
- Human operators and customers can interact through natural language via the human resource holon, lowering the technical expertise needed to monitor or redirect the system of systems.
- The architecture avoids a single central controller, preserving the holonic paradigm's scalability while adding structure through specialised holon roles.
- The proposed metrics—scalability, adaptability, resource utilisation, response time, and user satisfaction—give an intended comparison against holonic architectures without LLMs and specialised holons.
Reading between the lines
- The decisive experiment is to stress the LLM reasoning core with injected failures and measure whether task holons detect and replan within the latency a moving vehicle tolerates; that would turn the architecture's central premise into evidence.
- Because all four holon roles share the same LLM core, an error in one holon's context management could propagate as contradictory plans across the holarchy; adding a lightweight validation step for each generated action plan would test this.
- The architecture's promise extends beyond transportation: any system of systems with black-box components and human operators—emergency response, logistics, manufacturing cells—could adopt the same specialised holon roles and natural-language interface.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a layered holonic architecture for system of systems, in which each holon is decomposed into reasoning, communication, and capabilities layers, and introduces four specialized holon roles (supervisor, planner, task, resource) that use LLMs for decision-making. The architecture is illustrated with a 3D mobility scenario involving UGVs and UAVs for passenger transport, and the paper proposes evaluation metrics for future empirical work. The paper explicitly identifies itself as a conceptual framework and states that implementation and evaluation are future work.
Significance. If the architecture worked as claimed, it would advance the state of the art in holonic SoS architectures by adding LLM-based reasoning to holons and by importing manufacturing-inspired specialized holon roles into the SoS domain. The layered decomposition is clearly described, the figures and sequence diagram are helpful, and the paper is transparent about the lack of implementation. The main advertised benefits, especially "real-time adaptability," however, rest on unverified assumptions about LLM reliability and latency, and the case study provides no empirical evidence.
major comments (4)
- [Section 3.1 / Section 5] The claim that the reasoning layer "ensures real-time adaptability" (abstract and Section 5) is not supported by any measurement, simulation, or feasibility argument. Section 6.1 proposes a fine-tuned GPT-3 for the reasoning layer, whose generation latency is nondeterministic and can exceed the control-loop intervals of UGVs/UAVs, yet no latency bound, timeout, or degraded-mode fallback is specified. Section 6.2 lists response time only as a future metric. Because the LLM is the "central reasoning core" of every holon, this unverified premise is load-bearing for the paper's central contribution.
- [Section 4] The case study is a narrative description with figures and a sequence diagram, not a demonstration. No simulation, prototype, or quantitative data are presented, so the claims in Section 5 about dynamic adjustment, resource reallocation, and improved interoperability are not evidenced. Section 6 explicitly states that this is a conceptual framework and that implementation in a simulated environment is future work, which means the paper's stated benefits are currently unsupported.
- [Section 3.1 / Section 1] The architecture assumes that heterogeneous black-box CS, which are "unknown at design time" (Section 1), can be wrapped into a uniform three-layer, ROS-based holon facade without vendor cooperation. No mechanism is described for generating the interface, protocol, or capability descriptions needed by the communication layer to invoke black-box CS capabilities. This assumption is load-bearing for the interoperability claim and is not substantiated in the paper.
- [Section 5] The assertion that this work is "the first to introduce" specialized holons in the SoS context is not supported by a systematic comparison. The cited ontological holonic architectures in [7] and [20] already represent CS as holons with roles and dynamic composition, so the novelty should be scoped more carefully, for example by focusing specifically on the LLM-enhanced reasoning layer rather than on the specialized holon roles per se.
minor comments (5)
- [Title / Abstract] The arXiv title contains a typo: "LLM-Ehnanced" should be "LLM-Enhanced."
- [Section 4.2] Terminology is inconsistent: Section 3.2 introduces "Planner Holon" and "Supervisor Holon," but Section 4.2 later refers to "Plan Holons" and "Supervise Holons." Please unify the terminology.
- [Section 4.2] Notation such as "S-SoS", "S-CS1", "c1", "m1", "Pa", "Ta1", and sub-plan identifiers is introduced without definitions, making the sequence diagram and text difficult to follow.
- [Section 6.2] The proposed evaluation metrics are listed but not operationalized; for example, "adaptability" and "user satisfaction" need concrete measurement procedures and comparison baselines.
- [Section 3.2] The phrase "Human Resource Holon ... encapsulates humans as holistic entities" is vague. The paper should clarify how user intent, goals, and status are captured, and how consent and privacy are handled.
Circularity Check
No equations, fits, or empirical predictions; self-citations are non-load-bearing, so no circularity.
full rationale
This is a conceptual architecture paper. There are no equations, fitted parameters, or empirical predictions, so the reduction-to-fit patterns (self-definitional claims, fitted inputs called predictions) do not apply. The closest to a derivation chain is the Section 5 assertion that the reasoning layer 'provides consistent reasoning capabilities, enabling each holon to adapt autonomously to environmental changes while maintaining coherent behaviour across the system.' That is a design claim, not a result derived from the architecture's definitions, and Section 6 explicitly defers evaluation: 'This study presents a conceptual framework, and we recognise the importance of evaluating its effectiveness through experimentation.' The paper cites its own ICSOFT 2024 predecessor [2] as the origin of the NLP concept and Sadik et al. [23] as a future simulation evaluation approach, but neither citation carries the load of the paper's main contribution; both are disclosed as prior or planned work rather than as externally forced premises. The specialized-holon taxonomy is attributed to external manufacturing work [13], and the claimed SoS novelty is a positioning statement, not a circular derivation. No equation, theorem, fitted value, or benchmark result is equivalent by construction to an output. Any concern about unverified LLM real-time reliability belongs to correctness/verification risk, not circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption An LLM in every holon's reasoning layer reliably interprets heterogeneous commands and produces correct, executable plans with latency acceptable for real-time operation (Sections 3.1, 5).
- domain assumption Heterogeneous, black-box constituent systems, unknown at design time, can be encapsulated at runtime into a uniform three-layer holon with ROS-based communication and an LLM reasoning facade (Sections 1, 3.1).
- ad hoc to paper A human can be represented as a 'human resource holon' whose LLM interface faithfully captures user intent, goals, and status (Section 3.2).
invented entities (2)
-
Human Resource Holon (encapsulates humans with an LLM natural-language interface)
-
Specialized holon roles (supervisor, planner, task, resource) transplanted from manufacturing into the SoS context
Cite this review
Pith. "Pith review of LLM-Ehnanced Holonic Architecture for Ad-Hoc Scalable SoS." pith.science (2026). https://pith.science/paper/LXEEULCR
@misc{pith2026250107992,
author = {Pith},
title = {Pith review of: LLM-Ehnanced Holonic Architecture for Ad-Hoc Scalable SoS},
year = {2026},
howpublished = {\url{https://pith.science/paper/LXEEULCR}},
note = {Machine review of arXiv:2501.07992}
}
read the original abstract
As modern system of systems (SoS) become increasingly adaptive and human centred, traditional architectures often struggle to support interoperability, reconfigurability, and effective human system interaction. This paper addresses these challenges by advancing the state of the art holonic architecture for SoS, offering two main contributions to support these adaptive needs. First, we propose a layered architecture for holons, which includes reasoning, communication, and capabilities layers. This design facilitates seamless interoperability among heterogeneous constituent systems by improving data exchange and integration. Second, inspired by principles of intelligent manufacturing, we introduce specialised holons namely, supervisor, planner, task, and resource holons aimed at enhancing the adaptability and reconfigurability of SoS. These specialised holons utilise large language models within their reasoning layers to support decision making and ensure real time adaptability. We demonstrate our approach through a 3D mobility case study focused on smart city transportation, showcasing its potential for managing complex, multimodal SoS environments. Additionally, we propose evaluation methods to assess the architecture efficiency and scalability,laying the groundwork for future empirical validations through simulations and real world implementations.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[7]
Future Generation Computer Systems157, 499–515 (2024)
Elhabbash, A., Elkhatib, Y., Nundloll, V., Marco, V.S., Blair, G.S.: Principled and automated system of systems composition using an ontological architecture. Future Generation Computer Systems157, 499–515 (2024)
work page 2024
-
[20]
In: IEEE International Conference on Informatics, IoT, and Enabling Technologies
Nundloll, V., Elkhatib, Y., Elhabbash, A., Blair, G.S.: An ontological framework for opportunistic composition of iot systems. In: IEEE International Conference on Informatics, IoT, and Enabling Technologies. pp. 614–621. IEEE (2020)
work page 2020
-
[1]
In: 1st IEEE/SMC International Conference on System of Systems En- gineering
Abbott, R.: Open at the top; open at the bottom; and continually (but slowly) evolving. In: 1st IEEE/SMC International Conference on System of Systems En- gineering. pp. 1–6. IEEE Computer Society (2006)
work page 2006
-
[2]
In: Proceedings of the 19th International Conference on Software Technologies
Ashfaq, M., Sadik, A.R., Mikkonen, T., Waseem, M., Mäkitalo, N.: Enhancing holonic architecture with natural language processing for system of systems. In: Proceedings of the 19th International Conference on Software Technologies. pp. 427–433. SciTePress (2024)
work page 2024
-
[3]
In: Proceedings of the 14th International Workshop on Adaptive and Reflective Middleware
Blair, G.S., Bromberg, Y., Coulson, G., Elkhatib, Y., Réveillère, L., Ribeiro, H.B., Rivière, E., Taïani, F.: Holons: towards a systematic approach to composing sys- tems of systems. In: Proceedings of the 14th International Workshop on Adaptive and Reflective Middleware. pp. 5:1–5:6. ACM (2015)
work page 2015
-
[4]
In: 1st IEEE/SMC International Conference on System of Systems Engineering
Boardman, J.T., Sauser, B.J.: System of systems – the meaning of of. In: 1st IEEE/SMC International Conference on System of Systems Engineering. pp. 118–
-
[5]
In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H
Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Win- ter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford...
work page 2020
-
[6]
Systems of Systems Engineering for NATO Defence Applications (STO-EN-SCI-276) pp
Dahmann, J.S.: Systems of systems characterization and types. Systems of Systems Engineering for NATO Defence Applications (STO-EN-SCI-276) pp. 1–14 (2015)
work page 2015
Show all 33 references
-
[8]
In: Pro- ceedings of the Conference on Systems Engineering Research
Gideon, J., Dagli, C.H., Miller, A.K.: Taxonomy of systems-of-systems. In: Pro- ceedings of the Conference on Systems Engineering Research. pp. 356–363. IEEE (2005)
2005
-
[9]
(ed.): Systems of Systems Engineering: Principles and Applications
Jamshidi, M. (ed.): Systems of Systems Engineering: Principles and Applications. CRC Press, 1st edn. (2008)
2008
-
[10]
Multimedia tools and applications82(3), 3713–3744 (2023)
Khurana, D., Koli, A., Khatter, K., Singh, S.: Natural language processing: state of the art, current trends and challenges. Multimedia tools and applications82(3), 3713–3744 (2023)
2023
-
[11]
Macmillan (1968) 12
Koestler, A.: The Ghost in the Machine. Macmillan (1968) 12
1968
-
[12]
Software: Practice and Experience (2024)
Koubaa, A., Ammar, A., Boulila, W.: Next-generation human-robot interaction with ChatGPT and robot operating system. Software: Practice and Experience (2024)
2024
-
[13]
Computers in industry57(2), 121–130 (2006)
Leitão, P., Restivo, F.: Adacor: A holonic architecture for agile and adaptive man- ufacturing control. Computers in industry57(2), 121–130 (2006)
2006
-
[14]
In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction
Levinson, L., Dietrich, M., Sarkisian, A., Sabanovic, S., Smart, W.D.: Privacy aware robotics. In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction. pp. 1335–1337. ACM (2024)
2024
-
[15]
In: Proceedings of the 2nd Inter- national Workshop on Smart
Lopes, F., Loss, S., Mendes, A., Batista, T., Lea, R.: SoS-centric middleware ser- vices for interoperability in smart cities systems. In: Proceedings of the 2nd Inter- national Workshop on Smart. pp. 1–6. ACM (2016)
2016
-
[16]
Systems Engineering: The Journal of the International Council on Systems Engineering1(4), 267–284 (1998)
Maier, M.W.: Architecting principles for systems-of-systems. Systems Engineering: The Journal of the International Council on Systems Engineering1(4), 267–284 (1998)
1998
-
[17]
In: International Conference on Advanced Aspects of Software Engineering
Nadira, B., Bouanaka, C., Bendjaballah, M., Djarri, A.: Towards an UML-based SoS analysis and design process. In: International Conference on Advanced Aspects of Software Engineering. pp. 1–8. IEEE (2020)
2020
-
[18]
In: IEEE/ACM 39th International Confer- ence on Software Engineering Companion
Neto, V.V.G.: A model-based approach towards the building of trustworthy software-intensive systems-of-systems. In: IEEE/ACM 39th International Confer- ence on Software Engineering Companion. pp. 425–428. IEEE (2017)
2017
-
[19]
ACM Computing Surveys48(2), 1–41 (2015)
Nielsen, C.B., Larsen, P.G., Fitzgerald, J., Woodcock, J., Peleska, J.: Systems of systems engineering: basic concepts, model-based techniques, and research direc- tions. ACM Computing Surveys48(2), 1–41 (2015)
2015
-
[21]
https://cdn.openai.com/ better-language-models/language_models_are_unsupervised_multitask_ learners.pdf (2018), last accessed 2024/11/06
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Lan- guage models are unsupervised multitask learners. https://cdn.openai.com/ better-language-models/language_models_are_unsupervised_multitask_ learners.pdf (2018), last accessed 2024/11/06
2018
-
[22]
In: Proceedings of the 14th International Conference on Software Business
Rousi, R., Samani, H., Mäkitalo, N., Vakkuri, V., Linkola, S., Kemell, K.K., Daubaris, P., Fronza, I., Mikkonen, T., Abrahamsson, P.: Business and ethical concerns in domestic conversational generative AI-empowered multi-robot systems. In: Proceedings of the 14th International...
2023
-
[23]
In: Proceedings of the 7th International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence
Sadik, A.R., Bolder, B., Subasic, P.: A self-adaptive system of systems architecture to enable its ad-hoc scalability: unmanned vehicle fleet-mission control center case study. In: Proceedings of the 7th International Conference on Intelligent Systems, Metaheuristics & Swarm I...
2023
-
[24]
CoRR abs/2306.00597 (2023)
Sadik, A.R., Ceravola, A., Joublin, F., Patra, J.: Analysis of chatgpt on source code. CoRR abs/2306.00597 (2023)
2023 arXiv
-
[25]
Future Internet9(3), 35 (2017)
Sadik, A.R., Urban, B.: Combining adaptive holonic control and isa-95 architec- tures to self-organize the interaction in a worker-industrial robot cooperative work- cell. Future Internet9(3), 35 (2017)
2017
-
[26]
Science of Computer Programming 192, 102425 (2020)
Silva, E., Batista, T., Oquendo, F.: On the verification of mission-related properties in software-intensive systems-of-systems architectural design. Science of Computer Programming 192, 102425 (2020)
2020
-
[27]
Information and Software Technology147, 106904 (2022) 13
Song, J., Kang, J., Hyun, S., Jee, E., Bae, D.H.: Continuous verification of sys- tem of systems with collaborative MAPE-K pattern and probability model slicing. Information and Software Technology147, 106904 (2022) 13
2022
-
[28]
Engineering Applications of Artificial Intelligence90, 103503 (2020)
Tchappi, I.H., Galland, S., Kamla, V.C., Kamgang, J.C., Mualla, Y., Najjar, A., Hilaire, V.: A critical review of the use of holonic paradigm in traffic and trans- portation systems. Engineering Applications of Artificial Intelligence90, 103503 (2020)
2020
-
[29]
In: IEEE International Conference on Software Architecture Companion
Teixeira, P.G., Lebtag, B.G.A., Dos Santos, R.P., Fernandes, J., Mohsin, A., Kassab, M., Neto, V.V.G.: Constituent system design: A software architecture ap- proach. In: IEEE International Conference on Software Architecture Companion. pp. 218–225. IEEE (2020)
2020
-
[30]
In: IEEE International Symposium on Software Reliability Engineering Workshops
Wudka, B., Thomas, C., Siefke, L., Sommer, V.: A reconfiguration approach for open adaptive systems-of-systems. In: IEEE International Symposium on Software Reliability Engineering Workshops. pp. 219–222. IEEE (2020)
2020
-
[31]
In: IEEE International Conference on Web Services
Zhang, Z., Elkhatib, Y., Elhabbash, A.: NLP-based generation of ontological sys- tem descriptions for composition of smart home devices. In: IEEE International Conference on Web Services. pp. 360–370. IEEE (2023)
2023
-
[32]
CoRR abs/2303.18223 (2023) 14
Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Liu, P., Nie, J., Wen, J.: A survey of large language models. CoRR abs/2303.18223 (2023) 14
2023 arXiv
-
[123]
IEEE Computer Society (2006)
2006
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.