pith. sign in

arxiv: 2604.05835 · v1 · submitted 2026-04-07 · 💻 cs.SE

Proof of Concept as a First-Class Architectural Decision Instrument

Bruno Fernando Antognolli , Fabio Petrillo This is my paper

Pith reviewed 2026-05-10 18:55 UTC · model grok-4.3

classification 💻 cs.SE
keywords proof of conceptsoftware architecturearchitectural decisionssystematic reviewdecision traceabilityanti-patternsoftware engineering
0
0 comments X

The pith

Proofs of Concept should be treated as first-class instruments for architectural decisions rather than informal experiments.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that Proofs of Concept remain poorly defined and inconsistently applied in software engineering, so they rarely leave behind usable knowledge when they shape major design choices. The authors review academic and industry sources to create a clearer definition and a three-phase process that ties validation directly to documented decisions. If adopted, this approach would turn ad hoc experiments into traceable contributions that improve how teams learn from past validations. Readers would care because many software projects base high-stakes architecture on these experiments without retaining the reasoning or outcomes.

Core claim

The central claim is that a refined definition and lightweight three-phase framework (planning, execution, decision-making) can position PoCs as deliberate architectural decision instruments, closing the gap where literature describes outcomes but not processes, while preventing the Undocumented Architectural Experiment anti-pattern in which PoCs influence decisions without producing durable architectural knowledge.

What carries the argument

The three-phase PoC framework that links technical validation to explicit decision traceability.

Load-bearing premise

That a framework synthesized from existing literature descriptions of PoC outcomes will lead teams to make architectural decisions of measurably higher quality once adopted.

What would settle it

A longitudinal comparison of architectural decision documentation and reversal rates in teams that follow the three-phase PoC process versus teams that continue with current ad hoc practices.

Figures

Figures reproduced from arXiv: 2604.05835 by Bruno Fernando Antognolli, Fabio Petrillo.

Figure 1
Figure 1. Figure 1: PoC Characteristics and their link with a definition [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: BPMN PoC 6.1 Phase 1: Planning The planning phase serves as the foundation for a PoC. It makes assumptions explicit by linking the PoC to a concrete decision it is intended to inform. Seven planning steps were identified: – Determine purpose/objectives. Clearly define the PoC’s hypothesis, whether it concerns a technology, an architectural style, or a deployment strategy. Well-defined objectives provide di… view at source ↗
read the original abstract

Proofs of Concept (PoCs) are widely adopted practices in software engineering. Despite their relevance, PoCs remain conceptually underdefined and methodologically ad hoc in both research and industry, with definitions and implementation approaches that often lack clarity and consistency. This paper investigates the concept of PoCs with two complementary goals: (1) to provide a refined definition and astructured framework for PoC development grounded in a systematic review of academic and grey literature; and (2) to position PoCs as first-class architectural decision instruments rather than informal experiments or disposable artifacts. Through a systematic review of academic and grey literature we identify the key characteristics, processes, associated with PoCs and expose a significant gap the academic literature describes PoC outcomes but rarely its process. By synthesizing insights from diverse sources we propose a refined definition and a lightweight, three-phase framework (planning, execution, decision-making) that encompasses technical validation and explicit decision traceability. We also introduce the Undocumented Architectural Experiment anti-pattern, arising when PoCs influence high-impact architectural decisions without leaving durable architectural knowledge. We argue that elevating PoCs to first-class status improves decision quality, enhances traceability, and supports more systematic learning in architectural practice.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript conducts a systematic review of academic and grey literature on Proofs of Concept (PoCs) in software engineering. It identifies that academic sources emphasize PoC outcomes over processes, proposes a refined definition of PoCs, introduces a lightweight three-phase framework (planning, execution, decision-making) that incorporates technical validation and explicit decision traceability, defines the 'Undocumented Architectural Experiment' anti-pattern, and argues that treating PoCs as first-class architectural decision instruments will improve decision quality, traceability, and systematic learning in architectural practice.

Significance. The systematic review is a clear strength, as it synthesizes diverse sources to expose a documented gap between outcome descriptions and process details in the PoC literature. The normative proposal of a structured framework and the anti-pattern concept could, if adopted and validated, help standardize ad-hoc PoC practices and improve architectural knowledge retention. However, the significance remains prospective rather than demonstrated, given the absence of any outcome measurement or implementation evidence.

major comments (2)
  1. [Abstract and framework proposal] Abstract and the section describing the proposed three-phase framework: the central claim that elevating PoCs to first-class status 'improves decision quality, enhances traceability, and supports more systematic learning' rests entirely on deductive synthesis from the literature review. No case study, pilot data, metric (e.g., decision reversal rate or knowledge retention), or controlled comparison is supplied to show that the framework produces measurable gains versus current ad-hoc usage.
  2. [Systematic review results] The systematic review results section: while the paper correctly notes that 'the academic literature describes PoC outcomes but rarely its process,' it provides no evaluation plan, proposed metrics, or falsifiable test to determine whether the new planning-execution-decision-making framework actually reduces the identified gap or the Undocumented Architectural Experiment anti-pattern in practice.
minor comments (2)
  1. [Abstract] Abstract: 'a refined definition and astructured framework' contains a missing space and should read 'a structured framework'.
  2. [Abstract] Abstract: the sentence 'expose a significant gap the academic literature describes PoC outcomes but rarely its process' is missing a connector such as 'that' or a colon/period for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and for recognizing the value of the systematic review. We respond to each major comment below, maintaining the manuscript's scope as a literature synthesis and normative proposal while addressing the noted limitations through clarification and additions.

read point-by-point responses

  1. Referee: [Abstract and framework proposal] Abstract and the section describing the proposed three-phase framework: the central claim that elevating PoCs to first-class status 'improves decision quality, enhances traceability, and supports more systematic learning' rests entirely on deductive synthesis from the literature review. No case study, pilot data, metric (e.g., decision reversal rate or knowledge retention), or controlled comparison is supplied to show that the framework produces measurable gains versus current ad-hoc usage.

    Authors: We agree that the claims are deductive, derived from the identified literature gap between outcome-focused descriptions and absent process details, along with the risks of the Undocumented Architectural Experiment anti-pattern. The manuscript presents these as logical outcomes of adopting the proposed framework rather than as empirically demonstrated results. No case studies or metrics are included because the work is a conceptual proposal based on synthesis, not an evaluation study. We will revise the abstract and framework section to explicitly qualify the benefits as hypothesized improvements pending validation, and add a limitations paragraph noting the absence of empirical testing. revision: partial

  2. Referee: [Systematic review results] The systematic review results section: while the paper correctly notes that 'the academic literature describes PoC outcomes but rarely its process,' it provides no evaluation plan, proposed metrics, or falsifiable test to determine whether the new planning-execution-decision-making framework actually reduces the identified gap or the Undocumented Architectural Experiment anti-pattern in practice.

    Authors: The systematic review section focuses on exposing the gap to ground the framework proposal; an embedded evaluation plan would exceed the paper's stated goals of definition refinement and framework introduction. We will add a new 'Future Research Directions' subsection that outlines potential metrics (e.g., decision reversal rates, traceability audits, and knowledge retention measures) and suggests falsifiable tests such as controlled adoption studies or longitudinal case analyses to assess the framework's impact on reducing ad-hoc practices. revision: yes

Circularity Check

0 steps flagged

No significant circularity; literature synthesis is independent

full rationale

The paper performs a systematic review of external academic and grey literature to extract PoC characteristics and identify process gaps, then synthesizes a refined definition, three-phase framework, and anti-pattern. The claim that elevating PoCs improves decision quality is presented as a deductive inference from those external sources rather than a quantity fitted to the paper's own outputs or reduced via self-citation. No equations, predictions, or self-definitional loops appear; the derivation chain remains anchored in independently reviewed material.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The proposal rests on the assumption that clearer process documentation will improve architectural outcomes, without new empirical data or formal axioms.

axioms (1)
  • domain assumption Systematic review of academic and grey literature can reliably identify the key characteristics and gaps in PoC practice.
    Invoked in the description of the review process and gap identification.
invented entities (1)
  • Undocumented Architectural Experiment anti-pattern no independent evidence
    purpose: To name the situation where PoCs influence high-impact decisions without leaving durable knowledge.
    Introduced as a new label for an observed practice; no independent falsifiable test is provided.

pith-pipeline@v0.9.0 · 5503 in / 1236 out tokens · 26125 ms · 2026-05-10T18:55:58.287734+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    In: 2012 20th IEEE In- ternational Requirements Engineering Conference (RE)

    Ameller, D., Ayala, C., Cabot, J., Franch, X.: How do software architects con- sider non-functional requirements: An exploratory study. In: 2012 20th IEEE In- ternational Requirements Engineering Conference (RE). pp. 41–50 (2012).https: //doi.org/10.1109/RE.2012.6345838

    work page doi:10.1109/re.2012.6345838 2012

  2. [2]

    Sams Teach Yourself – Hours Series, Pearson Education (2023),https: //books.google.ca/books?id=8bUtzwEACAAJ

    Anderson, G.: Design Thinking for Tech: Solving Problems and Realizing Value in 24 Hours. Sams Teach Yourself – Hours Series, Pearson Education (2023),https: //books.google.ca/books?id=8bUtzwEACAAJ

    work page 2023

  3. [3]

    Bales, K.: Structures and Dynamics Division Research and Technology Plans, Fis- cal Year, 1981 (1981),https://books.google.ca/books?id=AXBFAQAAIAAJ

    work page 1981

  4. [4]

    Packt Publishing Limited (2024),https://books.google.ca/ books?id=wJJQ0AEACAAJ

    Baptista, G., Abbruzzese, F.: Software Architecture with C# 12 and .NET 8: Build Enterprise Applications Using Microservices, DevOps, EF Core, and Design Patterns for Azure. Packt Publishing Limited (2024),https://books.google.ca/ books?id=wJJQ0AEACAAJ

    work page 2024

  5. [5]

    Apress (2022),https://books.google.ca/ books?id=cIoPzwEACAAJ

    Beningo, J.: Embedded Software Design: A Practical Approach to Architecture, Processes, and Coding Techniques. Apress (2022),https://books.google.ca/ books?id=cIoPzwEACAAJ

    work page 2022

  6. [6]

    O’Reilly Media (2024), https://books.google.ca/books?id=afDvEAAAQBAJ

    Brisals, S., Hedger, L.: Serverless Development on AWS. O’Reilly Media (2024), https://books.google.ca/books?id=afDvEAAAQBAJ

    work page 2024

  7. [7]

    O’Reilly Media (2018),https://books.google.ca/books?id= 5hJNDwAAQBAJ 16 Antognolli and Petrillo

    Burns, B.: Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services. O’Reilly Media (2018),https://books.google.ca/books?id= 5hJNDwAAQBAJ 16 Antognolli and Petrillo

    work page 2018

  8. [8]

    com/dictionary/english/proof-of-concept, collins English Dictionary, 5th edi- tion

    Collins: Definition of proof of concept (2024),https://www.collinsdictionary. com/dictionary/english/proof-of-concept, collins English Dictionary, 5th edi- tion

    work page 2024

  9. [9]

    AD A0132 910 ADVANCES IN SENSORS AND THEIR INTEGRATION INTO AIRCRAFT 19, 2 (1954)

    Deckert, J.C., Szalai, K.J.: Analytic redundancy management for flight control sensors. AD A0132 910 ADVANCES IN SENSORS AND THEIR INTEGRATION INTO AIRCRAFT 19, 2 (1954)

    work page 1954

  10. [10]

    In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering

    Esfahani, N., Razavi, K., Malek, S.: Dealing with uncertainty in early software ar- chitecture. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. FSE ’12, Association for Comput- ing Machinery, New York, NY, USA (2012).https://doi.org/10.1145/2393596. 2393621

    work page doi:10.1145/2393596 2012

  11. [11]

    Mar- shall & Brainerd (2010),https://books.google.ca/books?id=5UZ-AQAAQBAJ

    Fairbanks, G.: Just Enough Software Architecture: A Risk-Driven Approach. Mar- shall & Brainerd (2010),https://books.google.ca/books?id=5UZ-AQAAQBAJ

    work page 2010

  12. [12]

    O’Reilly Media (2017),https://books.google.ca/books?id= pYI2DwAAQBAJ

    Ford, N., Parsons, R., Kua, P.: Building Evolutionary Architectures: Support Constant Change. O’Reilly Media (2017),https://books.google.ca/books?id= pYI2DwAAQBAJ

    work page 2017

  13. [13]

    Fowler, M.: Sacrificial architecture (2015),https://martinfowler.com/bliki/ SacrificialArchitecture.html

    work page 2015

  14. [14]

    Fowler, M.: Architecture decision record.https://martinfowler.com/bliki/ ArchitectureDecisionRecord.html(March 2026), accessed: 2026-03-25

    work page 2026

  15. [15]

    Garousi, V., Felderer, M., M¨ antyl¨ a, M.V.: Guidelines for including grey litera- ture and conducting multivocal literature reviews in software engineering. Infor- mation and Software Technology106, 101–121 (2019).https://doi.org/https: //doi.org/10.1016/j.infsof.2018.09.006,https://www.sciencedirect.com/ science/article/pii/S0950584918301939

    work page doi:10.1016/j.infsof.2018.09.006 2019

  16. [16]

    martin- fowler.com (2023), published on martinfowler.com

    Harmel-Law, A.: Scaling the practice of architecture, conversationally. martin- fowler.com (2023), published on martinfowler.com. Accessed: 2026-03-25

    work page 2023

  17. [17]

    ASQ Quality Press (2009),https:// books.google.ca/books?id=9OqiEAAAQBAJ

    Helgeson, J.: The Software Audit Guide. ASQ Quality Press (2009),https:// books.google.ca/books?id=9OqiEAAAQBAJ

    work page 2009

  18. [18]

    Wiley (2018),https://books.google.ca/books?id=uo9jDwAAQBAJ

    Hinchey, M.: Software Technology: 10 Years of Innovation in IEEE Computer. Wiley (2018),https://books.google.ca/books?id=uo9jDwAAQBAJ

    work page 2018

  19. [19]

    Packt Publishing (2018),https: //books.google.ca/books?id=6EZsDwAAQBAJ

    Ingeno, J.: Software Architect’s Handbook: Become a successful software architect by implementing effective architecture concepts. Packt Publishing (2018),https: //books.google.ca/books?id=6EZsDwAAQBAJ

    work page 2018

  20. [20]

    Deep Science Publishing (2025)

    Lakkarasu, P.: Designing Scalable and Intelligent Cloud Architectures: An End-to- End Guide to AI Driven Platforms, MLOps Pipelines, and Data Engineering for Digital Transformation. Deep Science Publishing (2025)

    work page 2025

  21. [21]

    Safari electronic books, Prentice Hall PTR (2002),https://books.google.ca/books?id=r8i-4En_aa4C

    Larman, C.: Applying UML and Patterns: An Introduction to Object-oriented Analysis and Design and the Unified Process. Safari electronic books, Prentice Hall PTR (2002),https://books.google.ca/books?id=r8i-4En_aa4C

    work page 2002

  22. [22]

    Premier reference source, Business Science Reference (2012),https://books.google.ca/ books?id=lv3EswzXZpsC

    Mistrik, I.: Aligning Enterprise, System, and Software Architectures. Premier reference source, Business Science Reference (2012),https://books.google.ca/ books?id=lv3EswzXZpsC

    work page 2012

  23. [23]

    O’Reilly Media (2016),https://books.google.ca/books?id=BIhRDAAAQBAJ

    Morris, K.: Infrastructure as Code: Managing Servers in the Cloud. O’Reilly Media (2016),https://books.google.ca/books?id=BIhRDAAAQBAJ

    work page 2016

  24. [24]

    Toward a better future: Education and training for economic development in Singapore since pp

    Mun, C.L.: Polytechnic education. Toward a better future: Education and training for economic development in Singapore since pp. 135–148 (1965)

    work page 1965

  25. [25]

    Oxford: Definition of proof of concept noun (2024),https://www. oxfordlearnersdictionaries.com/definition/english/proof-of-concept, oxford Advanced Learner’s Dictionary, 5th edition Proof of Concept as a First-Class Architectural Decision Instrument 17

    work page 2024

  26. [26]

    In: Proceedings of the 2026 IEEE/ACM 48th International Confer- ence on Software Engineering: Companion (ICSE-Companion ’26)

    Petrillo, F., Antognolli, B.: Proofs of concept as first-class architectural decision instruments. In: Proceedings of the 2026 IEEE/ACM 48th International Confer- ence on Software Engineering: Companion (ICSE-Companion ’26). ACM, Rio de Janeiro, Brazil (April 2026).https://doi.org/10.1145/3774748.3795875

    work page doi:10.1145/3774748.3795875 2026

  27. [27]

    Prateek, S.: Poc vs mvp vs prototype: Which strategy to use & when? (2024), https://appinventiv.com/blog/poc-vs-mvp-prototype-the-best-strategy/

    work page 2024

  28. [28]

    Putra, A.: Quarkus (java) vs fiber (go): Performance benchmark in kubernetes 201 (August 2024),https://www.youtube.com/watch?v=PL0c-SvjSVg, anton Putra

    work page 2024

  29. [29]

    Packt Publishing (2022),https: //books.google.ca/books?id=zTFYEAAAQBAJ

    Shrivastava, S., Srivastav, N., Sheth, R., Karmarkar, R., Arora, K.: Solutions Architect’s Handbook: Kick-start your career as a solutions architect by learn- ing architecture design principles and strategies. Packt Publishing (2022),https: //books.google.ca/books?id=zTFYEAAAQBAJ

    work page 2022

  30. [30]

    ACM Trans

    Sobhy, D., Bahsoon, R., Minku, L., Kazman, R.: Evaluation of software architec- tures under uncertainty: A systematic literature review. ACM Trans. Softw. Eng. Methodol.30(4) (Aug 2021).https://doi.org/10.1145/3464305

    work page doi:10.1145/3464305 2021

  31. [31]

    TechBohdana, M.: Poc vs prototype vs mvp: A detailed comparison (2024),https: //www.techmagic.co/blog/poc-vs-prototype-vs-mvp/

    work page 2024

  32. [32]

    Manning (2024),https://books.google.ca/books? id=MrvuEAAAQBAJ

    Tune, N., Perrin, J.: Architecture Modernization: Socio-technical alignment of soft- ware, strategy, and structure. Manning (2024),https://books.google.ca/books? id=MrvuEAAAQBAJ

    work page 2024

  33. [33]

    Learning, Pearson Education (2013),https://books.google.ca/books? id=PkNGSW-2i-EC

    Van Hecke, W.: Learning iOS Design: A Hands-On Guide for Programmers and De- signers. Learning, Pearson Education (2013),https://books.google.ca/books? id=PkNGSW-2i-EC

    work page 2013

  34. [34]

    NASA Technical Memorandum (1963)

    Welker, J.E.: Seasat altimetry for surface height of inland seas. NASA Technical Memorandum (1963)

    work page 1963