pith. sign in

arxiv: 2605.02859 · v1 · submitted 2026-05-04 · 💻 cs.DC · cs.AI· cs.SE

From Sensors to Insight: Rapid, Edge-to-Core Application Development for Sensor-Driven Applications

Komal Thareja , Anirban Mandal , Ewa Deelman This is my paper

Pith reviewed 2026-05-08 17:02 UTC · model grok-4.3

classification 💻 cs.DC cs.AIcs.SE
keywords sensor-driven applicationspattern-based engineeringAI-assisted developmentreusable workflow templatesedge-to-cloud continuumuser productivitydistributed infrastructuresenvironmental monitoring
0
0 comments X

The pith

Reusing a hydrophone workflow template with pattern-based engineering and AI assistance allows non-experts to rapidly develop sensor-driven applications from edge to cloud.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Scientists struggle to convert raw sensor data into insights because provisioning infrastructure and managing execution across edge and cloud systems requires expertise from multiple domains. This paper introduces a methodology that takes an existing hydrophone workflow as a template and uses pattern-based engineering combined with AI assistance to generate workflows for air quality, earthquake, and soil moisture monitoring. The method includes modular configuration to extend these workflows to edge resources. Evaluation centers on user productivity and lessons learned rather than performance benchmarks. The case studies show that this combination lowers entry barriers and supports iterative development of applications on distributed infrastructures.

Core claim

By starting with an existing hydrophone workflow as a reusable template and applying pattern-based workflow engineering along with AI-assisted development, the methodology generates and refines workflows for air quality, earthquake, and soil moisture monitoring. These workflows are extended to edge resources via modular configuration and placement, with the overall approach lowering the entry barrier for non-experts and enabling iterative exploration across distributed infrastructures.

What carries the argument

Reusable templates from pattern-based workflow engineering combined with AI assistance for adapting and placing sensor applications on distributed edge-to-core infrastructures.

If this is right

  • Non-experts can adapt the template to create workflows for multiple sensor monitoring domains.
  • Abstract workflows can be extended to run on edge resources through modular settings.
  • Development supports iterative exploration instead of fixed setups.
  • The focus on productivity provides practical guidance for applying the method to new cases.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This methodology could make sensor data processing more accessible by hiding infrastructure complexities behind patterns.
  • The use of reusable templates suggests that libraries of such patterns could be developed for various scientific domains.

Load-bearing premise

The pattern-based engineering and AI assistance can be generalized from the hydrophone template to other sensor domains via modular configuration and produce meaningful productivity gains for non-experts.

What would settle it

A demonstration that the template cannot be adapted to a new sensor type without significant additional expertise or that non-experts do not experience productivity improvements in the case studies would falsify the claim.

Figures

Figures reproduced from arXiv: 2605.02859 by Anirban Mandal, Ewa Deelman, Komal Thareja.

Figure 1
Figure 1. Figure 1: Pattern-based, AI-assisted development loop for sensor view at source ↗
Figure 2
Figure 2. Figure 2: Air quality workflow structure and representative analytical output. view at source ↗
Figure 3
Figure 3. Figure 3: Earthquake workflow structure and representative analytical output. view at source ↗
Figure 4
Figure 4. Figure 4: Soil moisture workflow structure and representative analytical output. view at source ↗
read the original abstract

Scientists increasingly rely on sensor-based data, yet transforming raw streams into insights across the edge-to-cloud continuum remains difficult. Provisioning heterogeneous infrastructure and managing execution on emerging platforms like Data Processing Units typically requires cross-domain expertise, creating significant barriers to rapid prototyping. This paper introduces an experience-driven methodology for the rapid development of sensor-driven applications. By combining pattern-based workflow engineering with AI-assisted development-implemented via Pegasus on the FABRIC testbed - we utilize an existing Orcasound hydrophone workflow as a reusable template. We introduce a pattern-based engineering methodology to generate and refine workflows for air quality, earthquake, and soil moisture monitoring. Furthermore, we show how these abstract structures are extended to edge resources through modular configuration and placement. Our evaluation focuses on user productivity and practical lessons rather than peak performance. Through these case studies, we illustrate how AI-assisted, pattern-based development lowers the entry barrier for non-experts and enables iterative exploration of sensor-driven applications across distributed infrastructures.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes an experience-driven methodology for rapid development of sensor-driven applications spanning edge-to-core infrastructures. It combines pattern-based workflow engineering with AI-assisted development, implemented via Pegasus on the FABRIC testbed, and reuses an existing Orcasound hydrophone workflow as a reusable template. The approach is applied to generate workflows for air quality, earthquake, and soil moisture monitoring, with extensions to edge resources through modular configuration and placement. Evaluation centers on qualitative assessments of user productivity and practical lessons from case studies, with the claim that this lowers entry barriers for non-experts and supports iterative exploration.

Significance. If the pattern-based and AI-assisted approach can be shown to generalize reliably and deliver measurable productivity improvements, it would offer a practical pathway for domain scientists to prototype complex sensor applications on heterogeneous distributed systems without requiring deep systems expertise. The reuse of established tools like Pegasus and the FABRIC testbed is a pragmatic strength that could accelerate adoption in environmental sensing domains.

major comments (2)
  1. [Abstract and Evaluation] Abstract and Evaluation section: The central claim that the methodology 'lowers the entry barrier for non-experts' and enables 'iterative exploration' rests on qualitative lessons from the authors' own refinements of the Orcasound template. No quantitative metrics (e.g., development time, iteration counts, error rates, or success rates by independent non-experts), baseline comparisons to conventional workflow construction, or external user studies are reported, rendering the productivity and generalization assertions anecdotal rather than demonstrative.
  2. [Case Studies] Case Studies section: The assertion that the Orcasound template generalizes to air-quality, earthquake, and soil-moisture domains 'via modular configuration' is presented without concrete details on pattern adaptation rules, validation of workflow correctness in the new domains, or handling of domain-specific sensor characteristics. This makes it difficult to evaluate whether the reusability claim holds beyond the single template.
minor comments (1)
  1. [Abstract] The abstract and introduction could more explicitly delineate the boundary between the claimed contributions (pattern reuse + AI assistance) and the supporting infrastructure (Pegasus/FABRIC), to clarify what is novel versus what is an application of existing systems.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below, clarifying the qualitative scope of the work and committing to revisions that strengthen the presentation without altering the manuscript's core contributions.

read point-by-point responses

  1. Referee: [Abstract and Evaluation] Abstract and Evaluation section: The central claim that the methodology 'lowers the entry barrier for non-experts' and enables 'iterative exploration' rests on qualitative lessons from the authors' own refinements of the Orcasound template. No quantitative metrics (e.g., development time, iteration counts, error rates, or success rates by independent non-experts), baseline comparisons to conventional workflow construction, or external user studies are reported, rendering the productivity and generalization assertions anecdotal rather than demonstrative.

    Authors: We agree that the evaluation is qualitative and draws from the authors' direct experience refining the Orcasound template for the reported case studies. The manuscript's stated focus is on practical lessons and user productivity observations rather than controlled quantitative experiments or independent user studies, which would constitute a separate research effort. To address the concern, we will revise the abstract and evaluation section to explicitly characterize the assessment as qualitative, derived from the case-study refinements, and to moderate claims about entry barriers and iterative exploration accordingly. revision: yes

  2. Referee: [Case Studies] Case Studies section: The assertion that the Orcasound template generalizes to air-quality, earthquake, and soil-moisture domains 'via modular configuration' is presented without concrete details on pattern adaptation rules, validation of workflow correctness in the new domains, or handling of domain-specific sensor characteristics. This makes it difficult to evaluate whether the reusability claim holds beyond the single template.

    Authors: The case studies illustrate generalization through modular configuration of the Orcasound template, but we acknowledge that the current text provides limited specifics on adaptation rules, correctness validation, and domain-specific sensor handling. In the revised manuscript we will expand the case studies section with concrete examples of the pattern adaptations applied to each monitoring domain, including the configuration steps used, checks performed to validate the resulting workflows, and accommodations made for sensor characteristics such as data rates and formats. revision: yes

Circularity Check

0 steps flagged

No significant circularity; practical methodology is self-contained

full rationale

The paper presents an experience-driven methodology combining pattern-based workflow engineering with AI-assisted development, reusing the Orcasound hydrophone workflow as a template and extending it via modular configuration to domains like air quality and earthquake monitoring on Pegasus/FABRIC. No equations, fitted parameters, predictions, or first-principles derivations exist that could reduce to inputs by construction. Evaluation rests on qualitative case studies and practical lessons rather than quantitative claims derived from models. References to Pegasus support implementation details without forming load-bearing self-citation chains or self-definitional loops for the productivity and generalization assertions.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that existing workflows can be generalized as templates and extended modularly to edge resources with AI help, drawing from prior tools without new free parameters or invented entities.

axioms (1)
  • domain assumption An existing sensor workflow can serve as a reusable template for other sensor types through pattern-based refinement.
    Used to generate workflows for air quality, earthquake, and soil moisture monitoring from the Orcasound hydrophone workflow.

pith-pipeline@v0.9.0 · 5475 in / 1314 out tokens · 123183 ms · 2026-05-08T17:02:59.627365+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 2 internal anchors

  1. [1]

    Pegasus, a workflow management system for science automation,

    E. Deelmanet al., “Pegasus, a workflow management system for science automation,”Future Generation Computer Systems, vol. 46, pp. 17–35, 2015

    work page 2015

  2. [2]

    Fabric: A national-scale programmable experimental network infrastructure,

    I. Baldinet al., “Fabric: A national-scale programmable experimental network infrastructure,”IEEE Internet Computing, vol. 23, no. 6, 2019

    work page 2019

  3. [3]

    BlueField-3: A high-performance DPU for data center infras- tructure,

    NVIDIA, “BlueField-3: A high-performance DPU for data center infras- tructure,” inProceedings of the Hot Chips Symposium, 2022

    work page 2022

  4. [4]

    Distributed computing in prac- tice: The condor experience,

    D. Thain, T. Tannenbaum, and M. Livny, “Distributed computing in prac- tice: The condor experience,”Concurrency and Computation: Practice and Experience, vol. 17, no. 2–4, pp. 323–356, 2005

    work page 2005

  5. [5]

    Claude (anthropic),

    Anthropic, “Claude (anthropic),” https://www.anthropic.com/claude

    work page

  6. [6]

    Evaluating Large Language Models Trained on Code

    M. Chenet al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021

    work page internal anchor Pith review arXiv 2021

  7. [7]

    OpenAQ: Open air quality data,

    “OpenAQ: Open air quality data,” https://openaq.org

    work page

  8. [8]

    USGS earthquake hazards program,

    “USGS earthquake hazards program,” https://earthquake.usgs.gov

    work page

  9. [9]

    Orcasound: An open-source project for real-time bioacoustic research and conservation,

    Orcasound Project, “Orcasound: An open-source project for real-time bioacoustic research and conservation,” inProceedings of the Marine Mammal Science Conference, 2020

    work page 2020

  10. [10]

    Soil moisture analysis and irrigation workflow using pegasus wms,

    “Soil moisture analysis and irrigation workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/ soilmoisture-workflow

    work page 2025

  11. [11]

    Makeflow: A portable abstraction for data intensive computing on clusters, clouds, and grids,

    M. Albrecht, P. Donnelly, P. Bui, and D. Thain, “Makeflow: A portable abstraction for data intensive computing on clusters, clouds, and grids,” inProceedings of the 1st ACM SIGMOD Workshop on Scalable Work- flow Execution Engines and Technologies, 2012, pp. 1–13

    work page 2012

  12. [12]

    Scientific workflow management and the ke- pler system,

    B. Lud ¨ascheret al., “Scientific workflow management and the ke- pler system,”Concurrency and Computation: Practice and Experience, vol. 18, no. 10, pp. 1039–1065, 2006

    work page 2006

  13. [13]

    Nextflow enables reproducible computational work- flows,

    P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational work- flows,”Nature Biotechnology, vol. 35, no. 4, pp. 316–319, 2017

    work page 2017

  14. [14]

    Apache airflow,

    Apache Software Foundation, “Apache airflow,” https://airflow.apache. org/

    work page

  15. [15]

    The Evolution of the Pegasus Workflow Management Software,

    E. Deelmanet al., “The Evolution of the Pegasus Workflow Management Software,”Computing in Science & Engineering, vol. 21, no. 4, 2019

    work page 2019

  16. [16]

    Workflows using pegasus: Enabling dark energy survey pipelines,

    K. Vahi, M. H. Wang, C. Chang, S. Dodelson, M. Rynge, and E. Deel- man, “Workflows using pegasus: Enabling dark energy survey pipelines,” inProceedings of the 28th Astronomical Data Analysis Software and Systems (ADASS’18), 2018, pp. 689–692

    work page 2018

  17. [17]

    Accelerating scientific workflows on hpc platforms with in situ processing,

    T. M. A. Doet al., “Accelerating scientific workflows on hpc platforms with in situ processing,” in2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), 2022, pp. 1–10

    work page 2022

  18. [18]

    Chameleon: A large-scale programmable experimental environment for computer systems research,

    D. Dupontet al., “Chameleon: A large-scale programmable experimental environment for computer systems research,”Future Generation Com- puter Systems, vol. 87, pp. 644–654, 2018

    work page 2018

  19. [19]

    Cloudlab: Flexible experimentation at scale,

    D. Dupont, M. Hibler, R. Ricci, L. Stoller, and J. Webb, “Cloudlab: Flexible experimentation at scale,” inProceedings of the USENIX Annual Technical Conference, 2019

    work page 2019

  20. [20]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,”IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016

    work page 2016

  21. [21]

    Smartnics: Programmable acceleration for network- intensive applications,

    D. Firestoneet al., “Smartnics: Programmable acceleration for network- intensive applications,” inProceedings of the ACM SIGCOMM Confer- ence, 2018

    work page 2018

  22. [22]

    Pegasus Workflow Management System API Reference,

    “Pegasus Workflow Management System API Reference,” https: //pegasus.isi.edu/documentation/reference-guide/api-reference.html, 2025

    work page 2025

  23. [23]

    Automating edge-to-cloud workflows for science: Traversing the edge-to-cloud continuum with pegasus,

    R. Tanakaet al., “Automating edge-to-cloud workflows for science: Traversing the edge-to-cloud continuum with pegasus,” 05 2022, pp. 826–833

    work page 2022

  24. [24]

    Orca sound deterction workflow using pegasus wms,

    “Orca sound deterction workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/orcasound-workflow

    work page 2025

  25. [25]

    Air quality forecasting workflow using pegasus wms,

    “Air quality forecasting workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/airquality-workflow

    work page 2025

  26. [26]

    Earthquake/seismic data workflow using pegasus wms,

    “Earthquake/seismic data workflow using pegasus wms,” 2025. [Online]. Available: https://github.com/pegasus-isi/earthquake-workflow

    work page 2025

  27. [27]

    SAGE: A software-defined sensor network,

    P. Beckmanet al., “SAGE: A software-defined sensor network,”Com- puting in Science & Engineering, vol. 23, no. 3, pp. 95–101, 2021

    work page 2021

  28. [28]

    Pegasus workflow management system via ACCESS,

    “Pegasus workflow management system via ACCESS,” 2024. [Online]. Available: https://support.access-ci.org/tools/pegasus

    work page 2024

  29. [29]

    End-to-end integration of scientific workflows on dis- tributed cyberinfrastructures: Challenges and lessons learned with an earth science application,

    C. Roaet al., “End-to-end integration of scientific workflows on dis- tributed cyberinfrastructures: Challenges and lessons learned with an earth science application,” in16th IEEE/ACM International Conference on Utility and Cloud Computing, 2024

    work page 2024

  30. [30]

    Openai codex,

    OpenAI, “Openai codex,” https://openai.com/research/codex

    work page

  31. [31]

    Gemini (google),

    Google, “Gemini (google),” https://ai.google/gemini/, accessed 2026

    work page 2026

  32. [32]

    Qwen2.5-Coder Technical Report

    Q. Team, “Qwen3-coder: Advancing open-source code language mod- els,”arXiv preprint arXiv:2409.12186, 2024, alibaba Cloud

    work page internal anchor Pith review arXiv 2024

  33. [33]

    Deploying pegasus on fabric testbed,

    “Deploying pegasus on fabric testbed,” 2026. [Online]. Available: https://tinyurl.com/fabric-artifacts

    work page 2026

  34. [34]

    Kiso: Edge to cloud workflows: Advancing workflow management in the computing continuum,

    “Kiso: Edge to cloud workflows: Advancing workflow management in the computing continuum,” 2026. [Online]. Available: https: //kiso.scitech.group/

    work page 2026