{"id":"4eeade2e-bbd4-4dc0-882d-92e6d26ed3b1","arxiv_id":"2505.13750","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Eudoxia is a deterministic, open-source simulator for evaluating FaaS scheduling algorithms in composable lakehouses, with a TPC-H-based runtime validation on the Bauplan cloud platform.","lead":"This paper introduces Eudoxia, an open-source simulator that models how data pipelines expressed as cloud functions are scheduled in a composable data lakehouse. A systems researcher might use it to test scheduling policies cheaply before deploying them on real cloud infrastructure.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation covers only single-query runtime at a fixed 16-vCPU allocation; it never tests the CPU-scaling or contention behavior scheduling algorithms rely on, so the paper's scheduling-evaluation claim is not yet supported.","rationale":"The reader's conditional verdict is appropriate. Eudoxia is a real, released artifact with a clean plug-in API, and the runtime-model calibration against TPC-H is a useful first step. However, the paper's central claim is that developers can use Eudoxia to evaluate scheduling algorithms against their infrastructure. That claim requires fidelity not just in single-query runtime estimates but in the scheduling-relevant behaviors: how runtimes change with CPU allocation, how concurrent jobs contend, how preemption and OOM re-queueing affect completion times. Section 4.2 does not test any of these. The validation runs each query in isolation on one fixed instance size, excludes three short queries, and configures simulated pipelines with statistics similar to the measured traces, so the low average error is largely a consistency check of the calibrated runtime model rather than independent evidence about scheduling outcomes. The paper explicitly calls the validation 'preliminary,' and that is an accurate self-assessment. My concern is not that the simulator is wrong, but that the current evidence does not yet support the strongest advertised use. This does not change the reader's conditional verdict; it reinforces it. A multi-tenant replay experiment would either validate the scheduling layer or expose the gap concretely.","tokens_in":11268,"tokens_out":4899,"duration_ms":50902,"concrete_test":"Replay a real multi-tenant trace on both Bauplan and Eudoxia: submit two or three TPC-H queries concurrently with one batch pipeline to a Bauplan instance of the same c5ad.4xlarge profile, using the priority scheduler, and record per-query completion times, queue waits, and preemptions. Feed the same trace, measured CPU/RAM statistics, and identical scheduler into Eudoxia, then compare scheduling-level metrics. If the simulated makespan, per-query latencies, or the relative ranking of schedulers diverge materially from the real run, the central claim is unsupported; if they match, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 is the only quantitative support for the central claim. TPC-H queries are executed one at a time on a fixed c5ad.4xlarge with 16 vCPUs, and simulated runtimes are compared to real runtimes for pipelines 'with similar statistics.' The reported 0.44–3.08% errors (average 1.74%) therefore validate the runtime model only at the exact allocation observed, after calibration; they do not validate the CPU scaling functions at the smaller or larger allocations a scheduler would choose, nor any interaction among concurrent pipelines. The built-in schedulers (naive, priority, priority-pool) allocate slices of resources, preempt containers, and re-queue OOM failures; none of these behaviors is checked against a real multi-tenant Bauplan deployment. Because queueing delay, preemption cost, OOM handling, and I/O or network contention are precisely the phenomena that distinguish scheduling policies, the paper gives developers no evidence that scheduler rankings from Eudoxia match rankings on their infrastructure. The text's assertion that 'this is a realistic setup that can effectively represent any kind of workload in the appropriate and necessary dimensions' (Section 4.2) marks the unsupported leap: the scheduler's decision space is exactly the resource-allocation and interleaving space that the validation never exercises.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Eudoxia, a deterministic discrete-event simulator for scheduling FaaS functions in the composable lakehouse architecture embodied by Bauplan. The authors describe the Bauplan programming and execution model, formalize the scheduling problem for function DAGs with three priority classes, and introduce a modular simulator with pluggable workload generation, executor, and scheduler components. The paper demonstrates the developer API for registering custom schedulers and reports a preliminary validation in which 19 TPC-H queries at 10GB are run alone on a 16-vCPU Bauplan instance; simulated runtimes are within 0.44% to 3.08% (mean 1.74%) of measured runtimes. The paper concludes that Eudoxia is a cheap mechanism for evaluating scheduling algorithms against user infrastructure.","tokens_in":11479,"tokens_out":3165,"duration_ms":30020,"significance":"If the simulator's runtime model generalizes beyond the calibrated single-query, fixed-allocation setting, Eudoxia would be a valuable, low-cost tool for the composable data systems community. The open-source release under a permissive license, the clean separation of workload generation, executor, and scheduler interfaces, and the deterministic tick-based design are genuine strengths that lower the barrier to experimentation. The provided validation is a reasonable first step, but it only exercises the runtime model at one resource allocation with no concurrency, leaving the paper's central claim about evaluating scheduling algorithms unsupported. Because the scheduler decision space includes CPU allocation sizes, preemption, queueing, and OOM handling, the current evidence does not establish that Eudoxia can reliably rank scheduling policies against real cloud behavior.","major_comments":[{"comment":"The only quantitative validation compares single-query runtimes at a fixed 16-vCPU allocation with caching disabled. The three built-in schedulers described in Section 4.1.2 allocate fractions of resources (e.g., 10% slices), double allocations on OOM failures, preempt low-priority containers, and route work across multiple pools; none of these behaviors is compared against a real multi-tenant Bauplan deployment. Because queueing delay, preemption cost, OOM handling, and resource contention are precisely the phenomena that distinguish scheduling policies, the paper's claim that Eudoxia enables developers to evaluate scheduling algorithms against their infrastructure is not yet supported. A concrete test would be to run a mixed workload of interactive and batch pipelines concurrently on Bauplan and in Eudoxia, then compare the ranking of the built-in schedulers by metrics such as p95 latency and throughput across several CPU allocations.","section":"Section 4.2 / Figure 3"},{"comment":"The execution model assumes that an operator's runtime is fully determined by a user-supplied CPU scaling function and a peak RAM requirement, with no dependence on I/O contention, network behavior, data layout, or host interference. The validation constructs simulated pipelines 'with similar statistics' to the measured traces, so the reported 0.44-3.08% errors partly reflect input replay: the simulator is given the same CPU and RAM characteristics observed on Bauplan and then recovers the same runtimes. This is a correctness risk for the scheduling claim because scheduler decisions explore allocations and interleavings far from the single calibrated point. I recommend validating the CPU scaling functions at multiple allocation sizes (for example 2, 8, and 16 vCPUs) and testing the simulator under controlled contention (e.g., two pipelines sharing a pool) to demonstrate that the model extrapolates.","section":"Section 3.2.1 / Section 4.2"},{"comment":"The statement that the workload generator setup 'can effectively represent any kind of workload in the appropriate and necessary dimensions' is unsupported by the evidence. The validation covers only TPC-H SQL queries at 10GB, run serially on one instance, and the workload generator's distributions are user-supplied defaults rather than characterized against diverse real workloads. The abstract's claim that Eudoxia 'can simulate a wide range of workloads' is therefore overstated. Please either narrow this claim to what is demonstrated (TPC-H-style SQL functions with given CPU and RAM profiles) or add validation with additional workload types, such as Python pipelines, different dataset sizes, and bursty arrival patterns.","section":"Section 4.2, paragraph beginning 'We believe'"}],"minor_comments":[{"comment":"In the description of the priority scheduler, the word 'qery' appears twice; it should be 'query'.","section":"Section 4.1.2"},{"comment":"The phrase 'simulations as an effective tools' has a subject-verb agreement issue; it should be 'as an effective tool'.","section":"Abstract"},{"comment":"The caption says 'Distribution of percent error of simulator estimates for runtime vs. real runtime' but the plot appears to show bar charts of runtimes for a subset of queries. Please align the caption with the actual plot content or add an error-distribution panel.","section":"Figure 3"},{"comment":"Reference [12] is cited as 'Hai et. all'; this should be 'Hai et al.'","section":"References"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Eudoxia is a real, released simulator for scheduling FaaS-style data pipelines in a composable lakehouse, and the design is the best part: a deterministic tick loop with workload generator, scheduler, and executor, plus a simple scheduler interface that makes custom policies easy to plug in. That is a genuinely useful abstraction for the composable-data community, and releasing it with a permissive license is a concrete contribution. No prior work I know of targets scheduling simulation specifically for this setting. The paper also does a decent job characterizing the scheduling problem in terms of priorities, resource pools, and operator-level CPU scaling.\n\nThe soft spot is the validation, and it is the load-bearing one. Section 4.2 compares simulated runtime against 22 TPC-H queries run alone on a single 16-vCPU instance, with caching disabled and three short queries excluded. The simulated pipelines are configured with “similar statistics” to the measured traces, so the reported 0.44–3.08% error is mostly a consistency check of the runtime model at the observed allocation. It does not validate the CPU scaling functions at smaller or larger allocations, which is exactly what a scheduler needs to reason about. None of the scheduler behaviors that matter—preemption, queueing delay, OOM handling, contention among concurrent pipelines—is checked against a real multi-tenant deployment. The abstract’s claim that developers can use Eudoxia to evaluate scheduling algorithms against their infrastructure is therefore not yet supported, and the Section 4.2 sentence about “realistic setup” representing “any kind of workload” overreaches.\n\nThat said, this is not a fatal flaw in the artifact itself. The abstractions are reasonable, the implementation is modular, and the scheduling problem is well described. The fix is straightforward: validate CPU scaling across multiple allocations and run a multi-tenant validation with real traces. Also fix the typos like “qery.”\n\nRecommendation: this deserves peer review as an artifact paper—send it to a systems/DB workshop with artifact evaluation, not a full conference yet. I would not cite its validation numbers, but I would cite it as an example of simulator design for this niche.","headline":"A useful open-source simulator with a clean pluggable design, but the validation only checks runtime at one fixed allocation, so the headline claim about evaluating scheduling policies is not yet supported.","tokens_in":631,"tokens_out":913,"would_cite":false,"duration_ms":29594,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that a composable lakehouse whose workloads are all cloud functions can be scheduled and optimized cheaply with a deterministic simulator, and presents Eudoxia as that simulator with validation on TPC-H.","keywords":["FaaS scheduling","composable data systems","data lakehouse","scheduling simulator","Bauplan","serverless functions","TPC-H"],"falsifier":"Run a TPC-H workload in Eudoxia and on the same cloud instance, but start a second workload that saturates object storage or network bandwidth on the host; if simulated runtimes stay unchanged while the real queries slow down by more than the reported validation error, the CPU/RAM-only execution model is missing a load-bearing factor.","tokens_in":11027,"feed_emoji":"🧪","tokens_out":5384,"duration_ms":44526,"temperature":0.7,"pith_summary":"The paper argues that once a data lakehouse expresses every workload—SQL queries, Python pipelines, bookkeeping steps—as cloud functions, the hard optimization problem becomes one of scheduling functions onto pools of CPU and RAM. It introduces Eudoxia, a deterministic simulator that models this problem at one CPU tick per ten microseconds and lets developers plug in their own scheduling algorithms. The authors claim Eudoxia can stand in for expensive cloud experiments: when they run TPC-H queries both on a Bauplan production instance and in the simulator, predicted runtimes miss by 0.44–3.08 percent, with an average error of 1.74 percent. If that holds, scheduler design for composable lakehouses can be iterated cheaply and safely before touching real infrastructure.","feed_headline":"Simulator estimates serverless pipeline runtimes within 3 percent","feed_subtitle":"Built from the Bauplan lakehouse, Eudoxia lets developers test custom schedulers without cloud bills.","key_machinery":"The load-bearing abstraction is the Container: a simulated bundle of operators plus allocated CPUs and RAM, created by the scheduler and executed by the executor. The executor precomputes, from each operator's hidden CPU scaling function and RAM requirement, how many ticks the container needs to finish or when it will hit an out-of-memory error, so a scheduler's decisions translate deterministically into runtimes and failures. Around this loop, the workload generator emits pipeline DAGs with configurable arrival distributions and priorities, and every scheduler implementation fits a fixed signature (inspect newly arrived and failed pipelines, return suspensions and assignments), which is what makes custom scheduling algorithms a matter of writing one Python function.","core_discovery":"The central claim is that scheduling in a composable lakehouse reduces to a tractable, simulatable problem because the system's programming model makes every unit of work a function with the same shape. Eudoxia's execution model assigns each operator a RAM requirement and a CPU scaling function saying how runtime depends on allocated CPUs; the scheduler sees only queue state, while the executor uses these hidden values to decide real completion times and out-of-memory failures. The paper validates the model by running the 22 TPC-H queries at 10GB on a Bauplan production instance and matching each query's CPU/RAM profile in Eudoxia, obtaining runtimes within an average of 1.74 percent of production. The intended consequence is that developers can evaluate naive, priority, priority-pool, or user-written scheduling algorithms against their own infrastructure model without paying for cloud experiments.","pith_inferences":["A natural next step is calibrating CPU scaling functions from real telemetry per operator type (filter, join, IO-heavy read), which would let Eudoxia distinguish memory-bound from network-bound queries instead of treating every operator the same way.","The same simulator loop could be reused for capacity planning—for example, asking what arrival rate or pool size a given scheduler tolerates before interactive queries miss a latency target—since cost and utilization are already logged per tick.","Extending the executor with an explicit I/O or network contention model is the most direct way to test whether the CPU/RAM-only approximation holds when pipelines share hosts, a scenario the current validation does not exercise.","Because the paper's validation runs each query alone with caching disabled, the strongest test of the simulator would be reproducing the reported 1.74 percent average error under concurrent mixed workloads, where preemption and OOM failures actually occur."],"forward_implications":["If the validation generalizes, scheduling algorithms for FaaS lakehouses can be ranked on throughput, latency, and cost from simulation alone.","Because real traces and real CPU scaling functions can be plugged into Eudoxia, a team can replay its own production workload history to test a new scheduler before deploying it.","The three built-in schedulers illustrate the policy space—greedy single-pool, priority with preemption, and multi-pool priority—giving developers baselines for custom implementations.","The simulation's determinism means two runs with the same parameters and workload seed produce comparable numbers, which is what benchmarking different policies requires.","A single-VM 'reasonable scale' assumption is built into the model: each function runs in one container and cannot split across hosts, so the simulator is calibrated for scale-up workloads rather than distributed engines."],"supporting_citations":[{"why":"Defines the Bauplan FaaS runtime over VMs that Eudoxia models, including millisecond function spawning and scale-up constraints.","marker":"[28]"},{"why":"Motivates the 'spare parts' composable lakehouse and the logical planner that inserts system functions into user DAGs.","marker":"[31]"},{"why":"Supplies the composable data system philosophy that frames the whole scheduling problem.","marker":"[23]"},{"why":"Provides the TPC-H benchmark whose 22 queries are used to validate simulated versus production runtimes.","marker":"[33]"},{"why":"Supports the claim that interleaving interactive and batch workloads is more efficient, which motivates priority-aware scheduling.","marker":"[25]"},{"why":"Provides large-scale cluster management evidence that co-locating workloads improves efficiency, cited alongside XFaaS.","marker":"[35]"},{"why":"Supports the 'Reasonable Scale' assumption that most OLAP workloads fit below 250GB, justifying single-VM execution.","marker":"[34]"}],"fun_headline_variants":["Simulate lakehouse FaaS scheduling with 1.74% accuracy","Eudoxia lets developers test schedulers without cloud bills","Deterministic simulator for serverless data pipeline scheduling","Cut cloud experiment costs with Eudoxia scheduler simulator","Match production runtimes within 1.74% using Eudoxia"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model assumes a function's runtime depends only on how many CPUs it is given and how much RAM it needs; if real cloud runtimes depend significantly on I/O contention, network delays, data layout, or noisy neighbors, simulated scheduling decisions may not predict production behavior.","fun_headline_variants_meta":{"raw":{"variants":["Simulate lakehouse FaaS scheduling with 1.74% accuracy","Eudoxia lets developers test schedulers without cloud bills","Deterministic simulator for serverless data pipeline scheduling","Cut cloud experiment costs with Eudoxia scheduler simulator","Match production runtimes within 1.74% using Eudoxia"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1350,"prompt_tokens":880,"completion_tokens":470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":385}},"tokens_in":496,"tokens_out":470,"duration_ms":4275,"temperature":1.0,"reasoning_tokens":385,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:10:41.710374+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a TPC-H workload in Eudoxia and on the same cloud instance, but start a second workload that saturates object storage or network bandwidth on the host; if simulated runtimes stay unchanged while the real queries slow down by more than the reported validation error, the CPU/RAM-only execution model is missing a load-bearing factor.","supporting_citations":[{"cited_title":"TPC-H Homepage","cited_arxiv_id":null,"evidence_quote":"Provides the TPC-H benchmark whose 22 queries are used to validate simulated versus production runtimes."}],"review_version":1}