{"id":"5ece09e2-9db9-4a06-8a99-444b0b9c4f98","arxiv_id":"2506.10642","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SUNRISE is a containerized, API-driven infrastructure that provides unified remote access to virtual prototyping simulations for embedded hardware across different simulation technologies and compute back-ends.","lead":"This paper describes SUNRISE, an infrastructure that wraps virtual prototypes of embedded hardware into containers and controls them through standardized REST APIs. It targets engineers who want to run hardware/software simulations on remote or cloud compute without installing specialized simulation tools.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tool-agnostic claim rests on a single headless open-source SysDef; commercial simulators with license managers or interactive setup are not shown to fit the SysDef/EvalAPI abstraction.","rationale":"The reader's weakest assumption captures the central risk: SUNRISE's tool-agnostic claim is only as strong as the class of simulators that can be packaged as headless containers and described by SysDef. The paper's own Section IV.A makes this a requirement, and the only concrete example is a single open-source VP. I agree with the conditional verdict. The architecture is coherent, and the RISC-V example demonstrates a feasible path, but the evidence does not yet support the broad 'commercial simulation tools' claim. Adding one heterogeneous commercial integration and releasing the reference implementation would close the gap. No issue with internal consistency or integrity is apparent; the gap is evidentiary.","tokens_in":6370,"tokens_out":3256,"duration_ms":35873,"concrete_test":"Take a commercial simulator with a floating-license server and a batch/GUI mode (e.g., a vendor SystemC or Verilog simulator), and define a SysDef for it using only the schema in Listing 1 and Section IV.E. Run a complete build+run+result-retrieval cycle through EvalAPI on a clean host. If a license handshake or interactive initialization must be performed outside the declared build_command/run_command, or if the schema needs extensions, the SysDef abstraction is not sufficient for the class of tools the abstract claims to cover.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV.A states as a requirement that 'a simulator can be run in a non-interactive way (headless) inside a container.' The paper presents exactly one system integration—an open-source RISC-V/SystemC VP (Listing 1)—and no experiments or artifacts demonstrating that SysDef/EvalAPI generalizes. The central claim (tool-agnostic unified access to simulation technologies, including 'highly capable commercial simulation tools') depends on this generalization. Commercial EDA simulators commonly require floating license servers, host-specific license files, vendor launchers, or interactive first-run configuration; none of these are expressible in the SysDef fields shown (docker_image, build_command, run_command, build/run parameters, results). The conclusion that the 'decentralized execution environment has proven' is therefore unsupported: the proof-of-concept validates one open-source path, not a universal API abstraction. This is not an internal inconsistency—the paper explicitly scopes to headless containers—but it is a gap between the stated capability and the evidence provided.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SUNRISE (Scalable Unified RESTful Infrastructure for System Evaluation), a proposed modular infrastructure for deploying containerized virtual prototyping simulations through standardized REST APIs. It describes a generic workflow (setup, build, run, analyze, archive), the four architectural components (Runtime Manager, front-ends, systems, compute back-ends), the SysDef JSON format for describing simulation systems, the SysAPI and EvalAPI interfaces, and three application scenarios (processor benchmarking, HW/SW codesign for AI, and DevOps integration). The central claim is that this design provides a unified, tool-agnostic way to access and execute diverse simulation technologies while hiding the underlying simulator and compute-resource details from the user.","tokens_in":6519,"tokens_out":2381,"duration_ms":29242,"significance":"If the stated capabilities were demonstrated, SUNRISE would be a useful contribution to virtual prototyping practice: it offers a clean modular architecture, a concrete container-based abstraction (SysDef), and an explicit REST API design (Table 1) that could lower the barrier to using virtual prototypes in industrial and CI/CD workflows. The paper is openly designed, with an open-source RISC-V SystemC VP as a concrete example, and the components are described in sufficient detail to be reimplemented. However, the significance is currently conditional: the paper contains no measurements, no user study, no comparison with existing tools, and no evidence that the abstraction generalizes beyond the single open-source, headless example. The claimed independence from simulation technology and the asserted operational benefits remain unvalidated.","major_comments":[{"comment":"The conclusion states that 'The decentralized execution environment has proven to provide efficient and flexible access to virtual prototyping' and that containerization 'has proven to enable fast set-up of simulations and scaling for highly parallel execution.' No experimental data, benchmarks, performance measurements, or reproducible evaluation are presented anywhere in the manuscript. These are empirical claims about efficiency, flexibility, and speed, and they are unsupported by the evidence in the paper. The sentence should be reworded to describe the proposed capability, or the paper should include a quantitative evaluation section with measurements of setup time, parallel execution speedup, and resource utilization.","section":"Section VI (Conclusion)"},{"comment":"The central claim of tool-agnostic, vendor-agnostic integration rests on the assumption that a simulator 'can be run in a non-interactive way (headless) inside a container.' The only fully worked example is an open-source RISC-V/SystemC VP (Listing 1). The paper explicitly mentions 'highly capable commercial simulation tools' as part of the target domain, yet commercial EDA simulators often require floating license servers, host-specific license files, interactive first-run configuration, or vendor-specific launchers that are not expressible in the SysDef fields shown (docker_image, build_command, run_command, build_parameters, run_parameters, results). The paper provides no evidence that the SysDef/EvalAPI abstraction can accommodate such tools without loss of functionality. This is a gap between the stated scope and the demonstrated proof of concept; additional examples or a concrete analysis of how license-managed and interactive tools would be handled are needed.","section":"Section IV.A and Section II"},{"comment":"All three application examples (Embench benchmarking, AI HW/SW codesign, and CI/CD integration) are described as conceptual flows and are not backed by any implementation, experiment, or measured result. For instance, Figure 3 shows parallel execution of 19 benchmarks, but there is no data on whether such parallelization was actually performed, how long it took, or what overhead the framework introduced. Figure 5 depicts a CI/CD pipeline, but no concrete integration with Jenkins or GitHub Actions is demonstrated. These sections read as future-work proposals rather than validation of the presented architecture; if they are intended as application scenarios, they should be explicitly labeled as such, and the conclusion should not rely on them as evidence of proven benefit.","section":"Section V (Application Examples)"}],"minor_comments":[{"comment":"There is a typo in 'one virtual protype' that should read 'one virtual prototype'.","section":"Section IV.A"},{"comment":"The phrase 'The exact implementation of the RM is subordinary' uses 'subordinary' where 'subordinate' is intended.","section":"Section IV.B"},{"comment":"The last sentence reads 'SUNRISE provides a unified solution for executing simulations than can be used across different CI/CD platforms'; 'than' should be 'that'.","section":"Section V.C"},{"comment":"The JSON snippet uses curly quotation marks around the value of 'run_time_ms' rather than standard ASCII quotes; this could break parsing if copied directly into a JSON file.","section":"Listing 1"},{"comment":"The SysAPI is described as comprising the SysDef file and container access, but the paper does not specify the exact SysAPI methods or a formal schema for SysDef; only a partial EvalAPI excerpt is given in Table 1. Adding the full API specification would improve reproducibility.","section":"Section IV.E"}],"recommendation":"major_revision","confidential_remarks":"The paper reads more like an industrial position paper or architecture description than a completed research contribution. The modular design and the concrete SysDef example are valuable, but the lack of any evaluation means it falls short of typical journal standards for empirical validation. I would encourage the editor to send the paper back for major revision with a request to either (a) add a real evaluation of the implemented system, or (b) substantially soften the claims in the abstract and conclusion to describe a proposed architecture rather than a proven one. The authors should also discuss the commercial-tool integration question explicitly, since that is the most likely point of failure for the claimed generality."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a straightforward architecture proposal for SUNRISE, a REST/container-based remote simulation infrastructure. What is actually new is applying well-known containerization and API techniques to virtual prototyping with a standardized, vendor-agnostic interface: the SysDef JSON plus EvalAPI. That is a real design contribution for the embedded/EDA community, even if the components are established.\n\nThe paper does several things well. It cleanly separates build and run parameters, distinguishes fast and full iteration loops, and defines a sensible experiment state machine. It is explicit about non-goals: no distributed execution of a single VP, and headless-only systems. The example SysDef for the AGRA RISC-V platform is concrete and plausible. Related work is appropriately scoped against Arm IP Explorer and generic workflow tools, and the positioning is honest.\n\nThe soft spots are real but not disqualifying. First, there is no quantitative evaluation: no performance numbers, no scaling measurements, no comparison to local simulation or to the cited commercial platforms. The conclusion that the environment \"has proven to provide efficient and flexible access\" is simply unsupported by anything in the paper. Second, no code or artifacts are provided, so the proof-of-concept cannot be inspected or reproduced. Third, the tool-agnostic claim rests on a single open-source SystemC/RISC-V integration. Commercial EDA simulators often require license servers, host-specific setup, or interactive launch; none of that is shown to fit the SysDef/EvalAPI abstraction. The paper does scope to headless containers, so this is a gap between stated generality and evidence, not an internal contradiction. It is also not a fatal flaw for a design paper, provided the authors stop short of claiming universal coverage until they show more integrations.\n\nWho is this for? Practitioners who want a blueprint for building such infrastructure, and researchers working on remote VP access. It is not a scientific breakthrough, but it is a coherent, useful systems paper. A serious referee should see it, but with the expectation of heavy revision: add at least basic evaluation (latency, throughput, scaling), release the code and SysDef schema, and either add a second or third integration (ideally a commercial tool) or dial back the tool-agnostic language to \"headless containerized simulators.\"\n\nEngage with it. Send it to peer review with conditions.","headline":"A clean, well-scoped architecture description for containerized virtual prototyping; the tool-agnostic claim outruns the evidence, but the design is worth referee time if the authors add evaluation and artifacts.","tokens_in":6990,"tokens_out":1885,"would_cite":false,"duration_ms":23971,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SUNRISE makes virtual prototyping a uniform REST service across simulators and compute backends.","keywords":["Virtual Prototyping","Simulation","SystemC","RISC-V","HW/SW Codesign","Containerization","REST API","Cloud Computing"],"falsifier":"Attempt to integrate a commercial EDA simulator that requires an interactive GUI or a floating license server using only a SysDef and the SysAPI's build/run commands, and observe whether all normal simulation features remain available. A simulator that cannot be containerized and driven headlessly without losing functionality would falsify the paper's working assumption.","tokens_in":6186,"feed_emoji":"⚙️","tokens_out":5750,"duration_ms":63704,"temperature":0.7,"pith_summary":"SUNRISE is an infrastructure that makes virtual prototyping—simulating hardware/software systems before physical chips exist—available through a few stable REST interfaces instead of tool-specific local setups. The paper claims that any simulation tool, from open-source console simulators to commercial platforms, can be wrapped as a containerized \"system\" described by a standard JSON file, then configured, built, run, and analyzed through the same API regardless of the underlying simulator or compute backend. If this works, users no longer need to master a particular simulation tool to run it: they can launch experiments on local machines, on-premise clusters, or cloud backends from a script, GUI, or CI/CD pipeline. The paper demonstrates the idea with a RISC-V virtual prototype, a parallel benchmark run, and a DevOps integration example.","feed_headline":"One API runs virtual prototypes on any compute backend","feed_subtitle":"SUNRISE wraps simulators in containers and exposes them through REST, cutting local setup for teams.","key_machinery":"The load-bearing object is the System Definition (SysDef) JSON file together with the SysAPI/EvalAPI pair. SysDef records the container image, build command, run command, typed build/run parameters, and expected result artifacts, so the Runtime Manager can operate any simulator without knowing its internals; the EvalAPI then maps that description to REST endpoints that front-ends use. This abstraction does the argument's work: it turns heterogeneous simulators into uniformly manageable \"systems\" and lets compute backends be swapped.","core_discovery":"On its own terms, the paper establishes that virtual prototyping can be decomposed into a containerized \"system\" plus two API layers: a System API that operates any simulator through its build/run commands and a JSON System Definition (SysDef), and an Evaluation API that exposes experiment sessions (create, parameterize, build, run, status, fetch results) to front-ends. A central Runtime Manager tracks each experiment as a state machine, separates build parameters from run parameters to allow fast iteration, archives finished experiments with their exact configuration, and can dispatch work to any container-capable compute backend. The resulting claim is that simulation technology, model, and infrastructure become interchangeable: the same workflow and interface work for a SystemC/RISC-V model and for a commercial tool, and the user never touches simulator-specific details.","pith_inferences":["A consequence the paper leaves implicit is that virtual prototypes could become shareable, vendor-independent assets: once a SysDef exists, any team or tool that speaks the API can consume the model, separating model development from deployment.","The headless-container assumption suggests a boundary: commercial tools with interactive front-ends or license managers may need gateway adapters, and the paper's planned FPGA gateway is an early sign of that.","A straightforward testable extension is a benchmark comparing containerized simulation runtime and overhead against native execution.","The Runtime Manager's back-end choice could be extended from availability-based to cost-, latency-, or data-residency-aware scheduling, which the paper does not address."],"forward_implications":["Simulation workloads become portable across compute resources: the same experiment can run on a laptop Docker engine or a Kubernetes-backed cluster without changing user-facing code.","Teams can iterate faster because build parameters and run parameters are separated, so changing a run parameter does not require recompiling the virtual prototype.","Archived experiments freeze the exact configuration that produced results, making simulation runs reproducible and referenceable.","A stable EvalAPI lets different front-ends—GUIs, scripts, CI/CD systems—drive the same simulations, reducing integration effort across projects and product variants.","Benchmark suites can be parallelized by launching many experiments simultaneously, enabling performance databases for multiple processor cores."],"supporting_citations":[{"why":"Supplies the RISC-V virtual prototype that serves as the integrated example system in the paper's SysDef listing.","marker":"[8]"},{"why":"Provides the Embench IOT benchmark suite used to demonstrate parallel execution of 19 experiments.","marker":"[10]"},{"why":"Gives the Kubernetes container orchestration platform referenced as the scalable compute back-end option.","marker":"[9]"},{"why":"Represents the CI/CD platform category that SUNRISE plugs into for DevOps integration.","marker":"[7]"}],"fun_headline_variants":["Containers make any simulator plug-and-play via one API","SUNRISE: one API to run any virtual prototype anywhere","Swap simulators without code changes with SUNRISE","API-driven containerized sims: run on any backend","SUNRISE turns simulators into interchangeable API services"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every simulator SUNRISE supports must be runnable headlessly inside a container, with its full functionality expressible as build/run commands plus file-based parameters and result artifacts; if a tool needs a GUI, interactive session, or host-specific license setup, the SysDef abstraction may not cover it.","fun_headline_variants_meta":{"raw":{"variants":["Containers make any simulator plug-and-play via one API","SUNRISE: one API to run any virtual prototype anywhere","Swap simulators without code changes with SUNRISE","API-driven containerized sims: run on any backend","SUNRISE turns simulators into interchangeable API services"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000643,"raw_usage":{"total_tokens":2869,"prompt_tokens":768,"completion_tokens":2101,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":384,"completion_tokens_details":{"reasoning_tokens":2018}},"tokens_in":384,"tokens_out":2101,"duration_ms":15380,"temperature":1.0,"reasoning_tokens":2018,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:20:01.271950+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Attempt to integrate a commercial EDA simulator that requires an interactive GUI or a floating license server using only a SysDef and the SysAPI's build/run commands, and observe whether all normal simulation features remain available. A simulator that cannot be containerized and driven headlessly without losing functionality would falsify the paper's working assumption.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Embench IOT benchmark suite used to demonstrate parallel execution of 19 experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the Kubernetes container orchestration platform referenced as the scalable compute back-end option."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Represents the CI/CD platform category that SUNRISE plugs into for DevOps integration."}],"review_version":1}