Lessons from a Big-Bang Integration: Challenges in Edge Computing and Machine Learning

Alessandro Aneggi; Andrea Janes

REVIEW 4 major objections 5 minor 13 references

Lessons from a Big-Bang Integration: Challenges in Edge Computing and Machine Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that a one-year distributed edge-computing and machine-learning project reached only six minutes of working functionality instead of the expected 40 because all components were merged at the end, and that early mock-based…

desk verdict A candid, readable failure retrospective, but the causal claims about big-bang integration outrun the evidence. read the letter →

arxiv 2507.17270 v1 pith:JLQXFDAI submitted 2025-07-23 cs.SE

Alessandro Aneggi , Andrea Janes This is my paper · ORCID

classification cs.SE

keywords big-bangintegrationedgecomputingmachinelearningreactivesystemsdistributedteamsexperiencereportsimulation-drivenengineeringtesting

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is an experience report about a one-year project that built a distributed real-time analytics system using edge computing and machine learning, with five partners developing components separately. The project's final integration, performed in two days at a test site, produced only six minutes of working system functionality against an expected 40-minute demonstration. Using root-cause analysis, the authors attribute the shortfall to the big-bang integration approach itself, compounded by weak communication across organizations, the absence of early integration testing, and a psychological preference for fully developed components over mock-ups. The paper argues that a top-down approach—deploying the whole architecture with mock components from the start, investing in communication infrastructure as seriously as in components, and cultivating a system-level mindset—would reduce this class of failure. If the attribution is right, then traditional agile habits are not enough for multi-partner reactive systems, and integration strategy must be planned before the components exist.

What carries the argument

The central mechanism is 'big-bang integration': deferring all component merging to a single final event, here a two-day on-site test. The paper's proposed antidote is a 'mock architecture'—a deployed skeleton of the full system in which initially nonfunctional or test-data-producing components imitate the real ones and are swapped out incrementally—so that interface and communication failures surface early. The contrast between bottom-up component development and top-down system deployment carries the argument.

What would settle it

Look at the integration-day issue log: if the failures encountered during the two-day merge each trace to the absent hardware, the slow network, the missing team member, or performance degradation, rather than to mismatches between independently developed components, then the paper's attribution to big-bang integration would be falsified. A stronger test would compare a twin project that used mock-based early deployment under the same external conditions.

Watch

Extended reading notes

Core claim

The central claim is that the six-minute outcome was not a run of bad luck but a predictable consequence of the integration strategy. The authors' root-cause analysis identifies the primary drivers as psychological and organizational—bias toward finished components, lack of shared ownership of system-level risk, and communication barriers across organizational boundaries—together with the technical difficulty of producing test data and testing the full system. They therefore assert that the correct countermeasure is to invert the development order: deploy the architecture skeleton first, with mock producers and consumers, and replace mocks with real components as they become ready; fund a unified communication infrastructure at the same level as component development; and train teams to see the system as a whole. The report also argues that agile's early-integration ideal is difficult to realize in multi-partner reactive projects and proposes simulation-driven engineering and structured integration cycles as remedies.

Load-bearing premise

The entire diagnosis rests on the assumption that the big-bang integration strategy, weak communication, and psychological bias—rather than the missing hardware, unexpectedly slow network, an absent team member, and performance problems that appeared only at the test site—were the main causes of the six-minute outcome.

Editorial extensions

If this is right

A distributed project that invests in component development without investing in integration strategy can deliver near-zero usable functionality at the deadline, even if every component works in isolation.
Deploying a mock-based architecture from the start turns integration from a final gamble into a continuous process, because newly completed components enter an already-running production-like system.
Communication infrastructure in multi-partner projects should receive budget, staff, and planning time comparable to the technical components, not be treated as an afterthought.
Team incentives and culture must be reshaped so that building a mock component that exposes an interface mismatch is valued as highly as finishing a real component.
Agile methods alone do not guarantee early integration when partners work independently; a unified, mandatory cross-partner integration cycle is required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension the paper does not run: compare two similar multi-partner projects differing only in whether mock-based integration is used from the start; the paper's claim predicts a large difference in delivered working functionality.
The warning-sign list—procrastinated decisions, over-optimization of single components, absent shared prototypes, and operation near resource limits—could be operationalized as a pre-integration health checklist that project managers rate at each milestone, turning the paper's retrospective insights into prospective monitoring.
The mock-first pattern transfers beyond edge and machine learning to any distributed reactive system, such as autonomous-vehicle pipelines or large IoT fleets, where interface mismatches and data-format drift dominate integration risk.
If the causal attribution holds, the 'six minutes' metric itself could serve as a baseline measure of integration readiness: projects that cannot demonstrate even a few minutes of end-to-end flow until the final event are carrying big-bang risk.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know first: this is a candid and readable experience report about a real failure, but the headline claim—that big-bang integration and psychological bias caused the failure—is not supported by the evidence presented. The paper is still worth a look, mainly for its list of six warning signs.

What's actually new here is modest. The core recommendations—early mock-based deployment, communication infrastructure, top-down thinking—are familiar, and the authors cite Mellegard et al., so they are not claiming otherwise. The genuinely fresh contribution is the concrete catalogue of warning signs: procrastinated decisions, over-optimizing individual components, missing shared prototypes, weak feedback loops, operating near resource limits, and last-minute changes. That list could become a useful checklist for distributed integration projects.

The paper also does something right: it is honest about its own process. The web search on 'top-down bottom-up' is weak scholarship, but the authors reveal it, and the observations are written in plain language. The proposed practices (mock-based architecture from day one, simulation-driven integration) are plausible and actionable.

Now the soft spots, in proportion. The central causal claim is the weakest part. Section 1 itself lists missing hardware, slower than expected network, performance issues unseen in development, and an absent team member. None of these would be prevented by early mock integration, and the paper never attempts a counterfactual. The success metric—'expected 40 minutes' of functionality—has no documented baseline or requirement. So the jump from 'we had 6 minutes' to 'big-bang caused it' is an interpretation, not a finding. The paper should say 'we believe' rather than 'the primary drivers were.' That is a significant weakness because the abstract frames it as certainty.

A minor point: the literature review relies on blog posts and StackExchange answers. For a practitioner experience report that is acceptable, but it should not be used to draw general conclusions.

Who benefits? Project leads in multi-partner edge/ML initiatives, especially those with a hard deadline. They will see themselves in the warning signs and pick up useful practical advice. As a research artifact, it lacks evidential weight, but as a lesson learned it is genuine. A serious referee should take it: experience reports of failures are worth capturing, and with a reframed causal claim and an explicit discussion of alternative explanations, this could be a useful paper.

I'd take it for peer review but expect major revision. I would not cite it as evidence for anything, though I might cite it as a cautionary example if I were writing about integration risks.

Referee Report

4 major / 5 minor

Summary. This paper is an experience report on a one-year distributed edge-computing and machine-learning project in which components developed by multiple partners were integrated in a final 'big-bang' step. The authors report that the integrated system provided only six minutes of functionality against an expected forty, and they attribute this outcome to the big-bang strategy, weak communication, lack of early integration testing, and psychological biases toward finished components. They list six early warning signs and propose countermeasures: early mock-based deployment, prioritized communication infrastructure, top-down system thinking, and simulation-driven integration cycles. The paper frames these as lessons learned for reactive, distributed, multi-partner software projects.

Significance. If read as a hypothesis-generating experience report, the paper has value for the software-engineering community: it honestly documents a real integration failure, proposes concrete and falsifiable practices, lists warning signs that could be tested in future studies, and raises four open research questions. The paper is also refreshingly explicit about the limits of generic Agile advice in multi-partner settings. However, its significance as an empirical finding is limited by the single-case retrospective design. The central causal claim — that the big-bang integration was the decisive cause of the failure — is asserted rather than demonstrated, and the headline metric (six versus forty minutes) lacks a documented basis. The paper's strength is its actionable recommendations, not its causal evidence.

major comments (4)

[Abstract and Section 1] The causal claim that the project 'faced critical setbacks due to a big-bang integration approach' is not established by the evidence presented. Section 1 also reports that some hardware was initially missing, the testing-site network was much slower than expected, components had performance issues not seen at development sites, and an important team member was absent. These factors are plausibly independent of the integration strategy, so the observed six-minute outcome could have been driven by environmental or logistical constraints rather than by big-bang integration. Please either provide a counterfactual analysis that isolates the integration strategy or explicitly reframe the integration-strategy explanation as one plausible hypothesis among several.
[Abstract and Section 1] The metric 'only six minutes of system functionality, far below the expected 40 minutes' is never operationalized. No section defines where the 40-minute expectation came from (e.g., a requirement document, a partner commitment, an estimate), what the measurement procedure was, or what counts as 'functionality.' Without this information, the headline comparison is unverifiable and cannot support the paper's conclusions.
[Section 2, Q2] The evidence for the claim that 'practitioners hesitate to view the system as a whole' is a Google search performed on December 16, 2024, in incognito mode, reviewing the first two pages and citing StackExchange and Medium posts. This is anecdotal and not a systematic literature review or survey. Please either present a more rigorous basis for this claim or label it as an informal observation that motivated the insights, rather than as evidence about practitioner attitudes.
[Section 2, Q3] The six warning signs are presented as observed facts, but no data are given to show that they were caused by the big-bang approach rather than by resource constraints, partner incentives, or other contextual factors. For example, 'operating too close to technological limits' (bandwidth at 9.6 out of 10 GB, CPU over 90%) may reflect the system's environment rather than the integration strategy. Please clarify the evidentiary basis (meeting notes, emails, interviews, measurements) or present these signs as hypotheses for future validation.

minor comments (5)

[Section 1] Typo: 'at a location were the system could be tested' should read 'at a location where the system could be tested'; also 'som issue' should be 'some issues'.
[Section 3] Typo: 'lessions learned' should be 'lessons learned'; also 'we propose multiple action as' should be 'we propose multiple actions, such as'.
[Section 2, Q2] The search query is written as 'software engineering top-up top-down'; please correct it to a meaningful query reflecting the intended top-down/bottom-up comparison.
[Figure 1] The C4-model notation used in Figure 1 is not explained in the caption; a brief note describing what the boxes and arrows represent would improve readability.
[References] Some references are informal web sources (Medium, StackExchange, Naukri, GeeksforGeeks) with no author or date; for a journal submission, please replace or supplement them with peer-reviewed sources on top-down versus bottom-up development.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an interpretive retrospective reporting a measured outcome and external references, not a derived prediction or fitted result.

full rationale

This is an experience report, not a derivation. The central claim is a causal interpretation of a single project's failure: big-bang integration, weak communication, and psychological bias toward finished components led to only six minutes of system functionality rather than the expected 40 minutes. The six-minute figure is a reported observed outcome, and the 40-minute figure is a stated project expectation; neither is produced by fitting a parameter to data or by evaluating a model. The cited literature (e.g., the Agile effort-estimation survey, the Reactive Manifesto, and the top-down vs. bottom-up web sources) is used only as background context and does not carry the load of the conclusion; the authors do not invoke a prior result of their own to justify their interpretation. No equation is present, and no prediction is claimed to follow from a fitted input. The recommendations for early mock-based deployment, communication infrastructure, and top-down thinking are proposals derived from the narrative, but they are not presented as mathematically forced or as the unique output of a self-defined quantity. The paper's limitations—such as the absence of a counterfactual analysis and the possibility that missing hardware, slow network, or the absent team member explain the failure—are threats to causal validity, not circularity. Therefore the circularity score is zero.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper introduces no fitted parameters and no invented entities. Its central claims rest on qualitative interpretations: the attribution of failure to integration strategy and psychology, and the sufficiency of the GQM-based observations and an informal web search as evidence. These are domain assumptions, not formal axioms.

assumptions (2)

domain assumption The failure is primarily attributable to the integration strategy and organizational/psychological factors, not to the situational problems the paper also lists.
Section 1 lists missing hardware, slow network, performance issues, and an absent team member; Section 3 nevertheless attributes primary causes to big-bang integration, communication, and psychology, without counterfactual analysis.
domain assumption The GQM-based observations and the informal Google search provide adequate evidence for the proposed practices.
Section 2 conducts a Google search (December 16, 2024, first two pages) and uses it with the project narrative to justify top-down thinking. No systematic literature review or empirical validation is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Lessons from a Big-Bang Integration: Challenges in Edge Computing and Machine Learning." pith.science (2026). https://pith.science/paper/JLQXFDAI

@misc{pith2026250717270,
  author       = {Pith},
  title        = {Pith review of: Lessons from a Big-Bang Integration: Challenges in Edge Computing and Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JLQXFDAI}},
  note         = {Machine review of arXiv:2507.17270}
}

read the original abstract

This experience report analyses a one year project focused on building a distributed real-time analytics system using edge computing and machine learning. The project faced critical setbacks due to a big-bang integration approach, where all components developed by multiple geographically dispersed partners were merged at the final stage. The integration effort resulted in only six minutes of system functionality, far below the expected 40 minutes. Through root cause analysis, the study identifies technical and organisational barriers, including poor communication, lack of early integration testing, and resistance to topdown planning. It also considers psychological factors such as a bias toward fully developed components over mockups. The paper advocates for early mock based deployment, robust communication infrastructures, and the adoption of topdown thinking to manage complexity and reduce risk in reactive, distributed projects. These findings underscore the limitations of traditional Agile methods in such contexts and propose simulation-driven engineering and structured integration cycles as key enablers for future success.

Figures

Figures reproduced from arXiv: 2507.17270 by the authors.

**Figure 1.** Architecture (using the notation of a C4-model context diagram [4]) analytics subsystem fails. Producers are software components that generate data from sensors or other sources. Consumers, on the other hand, are components that use data generated either internally or from external sources. From the client’s perspective, such systems are binary: they either work or they don’t. For example, if the system processes th… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

[1]

Andrea Griffini : Is is preferable to design top down or bottom up? , https://softwareengineering.stackexchange.com/questions/134633/is-is-preferable-to-design-top-down-or-bottom-up

work page
[2]

Ashborne : Top-down and Bottom-up Approach: The Difference in Software Engineering (2024), https://medium.com/@ashbornee/top-down-and-bottom-up-approach-the-difference-in-software-engineering-c6c159389a00

work page 2024
[3]

Bonér, J., Farley, D., Kuhn, R., Thompson, M.: The Reactive Manifesto (2014), https://www.reactivemanifesto.org, [Online; accessed 24-January-2025]

work page 2014
[4]

LeanPub (2023)

Brown, S.: The C4 model for visualising software architecture . LeanPub (2023)

work page 2023
[5]

Dzikri Qalam : Software Engineering Strategy: Top-Down Approach (2024), https://dzikriqalam.medium.com/software-engineering-strategy-top-down-approach-0ae010214d54

work page 2024
[6]

Kushleen Waraich : Difference Between Top Down and Bottom up Approach (2024), https://www.naukri.com/code360/library/difference-between-top-down-and-bottom-up-approach

work page 2024
[7]

IEEE Software 37(3) (2020)

Mellegard, N., Burden, H., Levin, D., Lind, K., Magazinius, A.: Contrasting Big Bang With Continuous Integration Through Defect Reports . IEEE Software 37(3) (2020)

work page 2020
[8]

Meyers, J.: Simulation-driven design: A closer look at how engineers can save time, resources, and manpower by frontloading simulation - Industrial Machinery (October 2024), https://blogs.sw.siemens.com/industrial-machinery/2024/10/08/simulation-driven-design-a-closer-look-at-frontloading-simulation/

work page 2024

Show all 13 references

[9]

No author : Top Down Approach in Software Engineering , https://prepinsta.com/software-engineering/top-down-approach/
[10]

No author : Difference between Bottom-Up Model and Top-Down Model (Dec 2022), https://www.geeksforgeeks.org/difference-between-bottom-up-model-and-top-down-model/

2022
[11]

ACM Comput

Pasuksmit, J., Thongtanunam, P., Karunasekera, S.: A Systematic Literature Review on Reasons and Approaches for Accurate Effort Estimations in Agile . ACM Comput. Surv. 56(11) (jun 2024)

2024
[12]

, " * write output.state after.block = add.period write

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...
[13]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Andrea Griffini : Is is preferable to design top down or bottom up? , https://softwareengineering.stackexchange.com/questions/134633/is-is-preferable-to-design-top-down-or-bottom-up

work page

[2] [2]

Ashborne : Top-down and Bottom-up Approach: The Difference in Software Engineering (2024), https://medium.com/@ashbornee/top-down-and-bottom-up-approach-the-difference-in-software-engineering-c6c159389a00

work page 2024

[3] [3]

Bonér, J., Farley, D., Kuhn, R., Thompson, M.: The Reactive Manifesto (2014), https://www.reactivemanifesto.org, [Online; accessed 24-January-2025]

work page 2014

[4] [4]

LeanPub (2023)

Brown, S.: The C4 model for visualising software architecture . LeanPub (2023)

work page 2023

[5] [5]

Dzikri Qalam : Software Engineering Strategy: Top-Down Approach (2024), https://dzikriqalam.medium.com/software-engineering-strategy-top-down-approach-0ae010214d54

work page 2024

[6] [6]

Kushleen Waraich : Difference Between Top Down and Bottom up Approach (2024), https://www.naukri.com/code360/library/difference-between-top-down-and-bottom-up-approach

work page 2024

[7] [7]

IEEE Software 37(3) (2020)

Mellegard, N., Burden, H., Levin, D., Lind, K., Magazinius, A.: Contrasting Big Bang With Continuous Integration Through Defect Reports . IEEE Software 37(3) (2020)

work page 2020

[8] [8]

Meyers, J.: Simulation-driven design: A closer look at how engineers can save time, resources, and manpower by frontloading simulation - Industrial Machinery (October 2024), https://blogs.sw.siemens.com/industrial-machinery/2024/10/08/simulation-driven-design-a-closer-look-at-frontloading-simulation/

work page 2024