{"id":"25622223-c3df-4a9a-931f-2fb5b7d577eb","arxiv_id":"2501.09562","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"HarmonAIze proposes new cloud abstractions, split into micro- and macro-level control, to enable tenant-provider cooperation for more efficient and resilient AI workloads.","lead":"AI workloads in the cloud run inefficiently because tenants and providers keep each other in the dark. HarmonAIze is a proposed set of cloud abstractions that would let both sides cooperate on scheduling, communication, and checkpointing for better performance and efficiency.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed cooperative optimization rests on information sharing that is both non-leaky and useful; the paper never shows such an interface exists.","rationale":"The reader identified the same weakest assumption: the required information sharing may be impossible without violating confidentiality. I refine it: the paper's own text constrains the provider to coarse-grained updates, so the question is not just whether tenants can be trusted, but whether the sanitized signal has any utility. This is the single most load-bearing concern because every claimed benefit in Table 1 flows through information that the paper itself says must be protected. The concrete test would settle the utility half; if it fails, the vision would need a fundamentally different abstraction (e.g., trusted execution, differential privacy, or delegated decisions with no tenant visibility). The paper is a well-scoped vision paper with a clear research agenda, so the conditional verdict stands; no change needed.","tokens_in":11561,"tokens_out":4721,"duration_ms":51764,"concrete_test":"Build a discrete-event simulation of the Section 3.1 macro-level pub/sub interface with a redaction layer that emits only aggregate events (e.g., 'a GPU class became available' with no topology, tenant identifiers, or utilization details). Run the O3 scheduling and O2 collective-selection scenarios from Section 4 on a realistic cluster trace, comparing tenant decisions informed by the redacted feed against decisions with full infrastructure visibility and against a no-feedback baseline. If the redacted feed does not produce decisions within a pre-registered margin of the full-visibility outcome, the central claim depends on leaking the very details Section 3.2 forbids.","verdict_should_be":"UNCHANGED","load_bearing_attack":"HarmonAIze's central claim is that two new control-loop couplings (Section 1) unlock cooperative optimization. Both couplings require a bidirectional information channel that is simultaneously useful and non-sensitive. Section 3.1 says macro-level updates 'may be provided in aggregate or coarse-grained forms to safeguard sensitive infrastructure details,' and Section 3.2 says 'we must avoid exposing sensitive infrastructure details to tenants.' Yet the paper never characterizes the granularity at which infrastructure insight remains actionable. For example, O3 (cluster scheduling) and O2 (collective communication) depend on information such as topology, link utilization, and other tenants' activity. Aggregating or coarsening this data to hide commercially sensitive details can destroy exactly the signal the tenant control loop needs to make a decision. The benefits in Table 1 come from systems (e.g., Pollux, Section 2) that assume full visibility of both workload and infrastructure; those numbers cannot be transferred to a regime with deliberately degraded information. Without a concrete privacy-utility analysis, the two innovations asserted in Section 1 are not established: the provider-side loop may have no safe data to act on, and the tenant-side loop may have no useful data to act on. The paper acknowledges privacy as an open concern (Section 5), which is honest, but that acknowledgement also locates the load-bearing gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This position paper proposes HarmonAIze, a set of micro- and macro-level cloud abstractions intended to enable cooperative optimization between AI workload tenants and cloud providers. The central idea is that tenant control loops should be informed by infrastructure-level insights while provider control loops are guided by workload-specific requirements, with micro-level abstractions supporting API-compatible, provider-executed fine-grained adaptations and macro-level abstractions providing tenant-executed strategic adaptations via interfaces such as pub/sub events. The paper identifies seven concrete optimization opportunities (O1--O7, in Section 4) and tabulates benefits from prior systems as representative of expected gains. It does not present an implementation or measurements; Section 5 outlines an incremental roadmap and briefly discusses adoption, timing, and open challenges including privacy.","tokens_in":11787,"tokens_out":4377,"duration_ms":42749,"significance":"If the HarmonAIze vision holds, it offers a useful organizing framework for research on tenant--provider cooperation in AI clouds, with a clear division of labor between micro-level provider control and macro-level tenant control. The paper's strengths are its concrete enumeration of seven optimization opportunities with up-to-date references, the explicit identification of building blocks for a prototype (e.g., Blox, MCCS, KungFu), and its honest treatment of adoption and standardization challenges. It is a genuine position paper: it frames testable hypotheses rather than claiming demonstrated results. The main significance is as a catalyst for discussion; the claimed benefits, however, are asserted rather than shown, and the privacy--utility tension is left unresolved.","major_comments":[{"comment":"The central claim of Section 1 that HarmonAIze's two innovations are load-bearing requires an information channel that is simultaneously non-sensitive and actionable, but the paper never characterizes the granularity at which infrastructure insight remains actionable. Section 3.1 suggests aggregate or coarse-grained updates to safeguard sensitive details, and Section 3.2 reiterates that sensitive infrastructure details must not be exposed; however, for O2 (collective communication) and O3 (cluster scheduling), decisions such as algorithm selection and scaling require topology, link utilization, and other tenants' activity. Aggregating those signals to protect commercial confidentiality can remove precisely the information the tenant control loop needs. The benefits in Table 1 are from systems such as Pollux [43] and MCCS [56] that assume full visibility of both workload and infrastructure, so those numbers cannot be transferred without further argument to a regime with deliberately degraded information. This is a load-bearing gap; Section 5 lists privacy as open, but the authors should either provide a concrete privacy--utility analysis or moderate the claimed benefits accordingly.","section":"§3.1–3.2, §5"},{"comment":"Table 1 is introduced as 'select benefits that are representative of cooperative optimizations that we expect from HarmonAIze' (Section 5), but the cited systems are not instances of the HarmonAIze abstractions and in some cases require precisely the full visibility that HarmonAIze's coarse-grained interfaces deliberately withhold. For example, Pollux [43] co-adapts scheduling and batch size with direct control of cluster resources, and MCCS [56] offloads collectives as a service with infrastructure-level knowledge. As presented, the table conflates the headroom that cross-layer optimization could offer with the gains that HarmonAIze specifically would deliver. The paper should either relabel the table as evidence of headroom, or add a caveat that these numbers assume information quality that HarmonAIze's privacy constraints may not preserve.","section":"§4, Table 1"},{"comment":"The proposed pub/sub interface for macro-level updates mentions several possible mechanisms for resolving contention—priority-based allocation, reservation windows, or price-based auctioning—as though they were interchangeable (Section 3.1). These mechanisms impose very different incentive structures and information requirements: auctioning can elicit strategic behavior and reveal private valuations, reservation windows can reduce utilization, and priority rules require a notion of fairness across tenants. The paper's later claim of a 'natural alignment of incentives' (Section 5) therefore needs substantially more support; without it, the cooperative control-loop model is underspecified. At minimum, the authors should indicate which mechanism they envisage for the initial prototype described in Section 5.","section":"§3.1, macro-level abstractions"}],"minor_comments":[{"comment":"The text '22.61×%' in Section 4 (O6) contains a stray percentage sign; it should read '22.61×' or 'by a factor of 22.61'.","section":"O6"},{"comment":"In Figure 1, 'MACRO-level' appears in all caps while 'micro-level' is lowercase; use consistent casing for the two levels.","section":"Figure 1"},{"comment":"The abbreviation 'c.f.' (e.g., in Section 3) should be 'cf.' (confer), and 'check pointing' (Section 4, O6) should be 'checkpointing' for consistency.","section":"Throughout"},{"comment":"The phrase 'as much as a $30K [42]' in Section 4 (O6) is awkward; consider 'up to $30K [42]' or 'as much as $30K [42]'.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"This is a well-written vision paper that fits the APSys workshop scope. The self-citation overlap with the authors' earlier Workload Intelligence paper is noticeable but explicit and properly discussed. The main technical risk is the privacy--utility tension: the paper hand-waves the granularity at which infrastructure information remains useful after aggregation. If the authors can address that point substantively—even with a qualitative argument or a structured set of design options—the paper would be acceptable. Given the workshop format, I would not demand an implementation, but the claimed benefits should be scaled back or clearly labeled as headroom evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a position paper from APSys arguing for HarmonAIze, a set of abstractions for cooperative optimization between tenants and providers of AI clouds. The core idea: split control into micro (provider-side, fast loops) and macro (tenant-side, strategic loops), with API compatibility plus buy-in for adoption. That split is a useful synthesis, and the paper is honest about what it is: a research agenda, not a system. It does a good job laying out seven concrete opportunity areas (scheduling, collectives, checkpointing, energy, etc.) with references to prior systems that demonstrate individual gains.\n\nWhat's actually new is the packaging: the specific micro/macro division, the emphasis on drop-in API compatibility as a first-class requirement, and the application to AI workloads in multi-tenant clouds. Prior work like Workload Intelligence, MCCS, NetHint, and KungFu covers pieces, but the combination here is not in the literature. The paper also acknowledges incentives, adoption, and timing issues in Section 5, which is more self-aware than most vision papers.\n\nThe soft spot is the one you'd expect: the whole edifice rests on an information channel that is simultaneously useful and non-sensitive. The paper says macro-level updates 'may be provided in aggregate or coarse-grained forms to safeguard sensitive infrastructure details' (Section 3.1), and Section 3.2 says 'we must avoid exposing sensitive infrastructure details to tenants.' But it never characterizes the granularity at which infrastructure insight remains actionable. For O2 (collectives) and O3 (scheduling), the tenant needs topology and load information; aggregate it enough to hide commercial details and you may destroy the signal. Table 1 lists benefits from systems that assume full visibility (Pollux, MCCS), so those numbers don't directly transfer to a regime with deliberately degraded information. The paper explicitly flags privacy as an open concern in Section 5, which is honest, but that also means the central claim is not yet supported. As a position paper this is acceptable; as evidence it is not.\n\nWho this is for: systems researchers working on cloud abstractions, AI infrastructure, or tenant-provider interfaces. It deserves to be read and discussed, and I'd bring it to a reading group. For peer review, I'd send it out—at a workshop or a position-paper track, not as a full systems paper. It's a well-argued agenda, not a validated result.\n\nRecommendation: engage, but keep expectations calibrated.","headline":"A clear, well-scoped vision paper for cooperative tenant-provider abstractions, but the load-bearing assumption that useful and non-sensitive infrastructure information can coexist is acknowledged, not solved.","tokens_in":12330,"tokens_out":2495,"would_cite":true,"duration_ms":24106,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HarmonAIze argues that the root cause of AI workload inefficiency in multi-tenant clouds is the absence of abstractions for tenant–provider cooperation, and proposes a two-level abstraction design—micro-level, where providers execute…","keywords":["cloud abstractions","tenant-provider cooperation","cross-layer optimization","AI workloads","control loops","collective communication","multi-tenant cloud","workload intelligence"],"falsifier":"Run a controlled multi-tenant experiment comparing AI training jobs under HarmonAIze-style cooperative control (provider-side collectives and checkpointing, tenant-side scaling on provider events) against the same jobs on standard IaaS with static NCCL settings; if the cooperative cohort does not show measurably better job completion time, energy use, or failure recovery on realistic workloads with noisy neighbours, the core efficiency claim fails. A simpler disproof would be evidence that cloud providers cannot expose even aggregated infrastructure events at a granularity tenants can act on without leaking competitive information.","tokens_in":11379,"feed_emoji":"🤝","tokens_out":7152,"duration_ms":66189,"temperature":0.7,"pith_summary":"This position paper argues that AI workloads in multi-tenant clouds run inefficiently because tenants cannot see infrastructure state and providers cannot see workload intent. The proposed fix, HarmonAIze, is a layer of abstractions that splits control by timescale: providers execute micro-level loops over collective communication, batch size, and checkpointing using tenant-declared constraints, while tenants execute macro-level loops over scaling, parallelism, and hyperparameters using provider-supplied events. The paper argues that AI workloads' iterative, predictable, tightly synchronized character makes them the right first domain for such cooperation, and surveys seven optimization areas that would benefit. If the vision is right, shared control loops could turn one-off optimizations from research systems into standard cloud services.","feed_headline":"Cooperative cloud abstractions target AI workload inefficiency","feed_subtitle":"With shared insight, AI training can adapt to congestion, failures, and energy cost in real time.","key_machinery":"The central mechanism is a pair of abstraction levels. Micro-level abstractions keep API compatibility with existing AI libraries (such as collective communication interfaces) while letting tenants specify ranges and constraints, and they delegate data-path decisions to the provider, which can execute them in the hypervisor or on isolated infrastructure processing units. Macro-level abstractions give tenants a pub/sub interface to subscribe to infrastructure-level events (resource availability, failures) in aggregate or coarse-grained form, and allow multi-round negotiation between tenants and providers for strategic adaptations. The argument rests on the claim that AI workloads' predictability and tight synchronization make these cooperative control loops both feasible and high-value.","core_discovery":"HarmonAIze's central claim is that today's cloud fails AI workloads because tenants lack infrastructure insight and providers lack workload insight, and that the fix is a new layer of abstractions that splits control by timescale: providers execute micro-level control loops (algorithm selection, batch-size adjustment, checkpoint placement) using tenant-declared constraints, while tenants execute macro-level control loops (scaling, parallelization strategy, hyperparameters) using provider-supplied aggregated events. The paper argues this division matches who has the timely information and who has the strategic view, and surveys seven optimization opportunities (O1–O7) where cross-layer cooperation would amplify known gains. HarmonAIze deliberately focuses on AI workloads first because their iterative, accelerator-driven, and tightly synchronized nature makes them more predictable than general distributed systems, making the optimization problem tractable.","pith_inferences":["If the premise holds, the same micro/macro split is a candidate for other synchronous, iterative workloads such as HPC simulations, even though the paper deliberately restricts itself to AI.","The buy-in deployment model implies a competitive dynamic the paper leaves implicit: early-adopting tenants should see measurable performance wins, which would push providers to standardize these interfaces to avoid churn.","A testable prediction following from the micro-level design is that provider-side execution of collectives on SmartNICs or IPUs outperforms tenant-side algorithm selection, because only the provider observes real-time topology and load.","The granularity of aggregate infrastructure events is the crux the paper flags but does not resolve: too coarse and tenants cannot act, too fine and providers leak sensitive information; finding the workable middle is an empirical question."],"forward_implications":["Providers take over fine-grained, data-path decisions such as collective algorithm selection, batch-size adjustment, and checkpoint placement, using tenant-specified ranges as guardrails.","Tenants receive standardized, aggregate infrastructure events and can negotiate with providers, so scaling, parallelism, and hyperparameter choices can react to failures and congestion mid-run.","Known optimization gains—up to 2.4× collective communication speedup, 22.61× mean time between failures, and 53% energy reduction—would become attainable in public multi-tenant clouds rather than only in tightly coupled single-operator systems.","Adoption is incremental: tenants opt in by adding buy-in interfaces, while providers continue serving non-adopting tenants as today.","The first proof-of-concept can be assembled from existing open tools: scheduler simulators, distributed-job emulators, collectives-as-a-service, and runtime adaptation systems."],"supporting_citations":[{"why":"Workload Intelligence is the prior framework HarmonAIze shares goals with but departs from, supplying the baseline for cloud-cooperation abstractions.","marker":"[26]"},{"why":"MCCS demonstrates collectives-as-a-service with provider-side optimization, the model for O2 and micro-level abstractions.","marker":"[56]"},{"why":"KungFu shows runtime adaptation of training parameters, motivating O1 and the tenant-side macro-level control loop.","marker":"[37]"},{"why":"Pollux demonstrates co-adaptive scheduling benefits, motivating O3 and the need for adaptive resource allocation.","marker":"[43]"},{"why":"Parcae shows proactive liveput-optimized training on preemptible instances, motivating adaptive jobs in O1.","marker":"[15]"},{"why":"Andromeda provides a reference model for provider-side control loops placed in the hypervisor or network virtualization layer.","marker":"[14]"},{"why":"NetHint supplies evidence that white-box network information improves multi-tenant distributed ML, supporting the value of sharing infrastructure insight.","marker":"[10]"},{"why":"SuperBench shows proactive validation raising mean time between failures by 22.61×, motivating O6 and the need for workload-aware diagnostics.","marker":"[59]"},{"why":"DynamoLLM demonstrates dynamic inference cluster reconfiguration cutting energy by 53% and carbon emissions by 38%, motivating O7.","marker":"[47]"},{"why":"Unicron shows managed self-healing LLM training achieving 1.9× efficiency, motivating O5 and provider-side fault tolerance.","marker":"[23]"}],"fun_headline_variants":["HarmonAIze: split control loops to fix AI cloud inefficiency","New cloud layer lets tenants and providers cooperate in real time","Cloud abstractions rethought for AI: micro and macro control","Cooperative optimization for AI workloads on shared clouds","Cloud teamwork: tenants share goals, providers share insights"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Tenants and providers must both be willing to reveal enough—tenants their workload requirements, providers their infrastructure state—without leaking commercially sensitive details, and must trust each other enough to act on that shared information; if this mutual disclosure cannot happen, the cooperative control loops have nothing to feed on.","fun_headline_variants_meta":{"raw":{"variants":["HarmonAIze: split control loops to fix AI cloud inefficiency","New cloud layer lets tenants and providers cooperate in real time","Cloud abstractions rethought for AI: micro and macro control","Cooperative optimization for AI workloads on shared clouds","Cloud teamwork: tenants share goals, providers share insights"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000151,"raw_usage":{"total_tokens":1106,"prompt_tokens":755,"completion_tokens":351,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":371,"completion_tokens_details":{"reasoning_tokens":269}},"tokens_in":371,"tokens_out":351,"duration_ms":4473,"temperature":1.0,"reasoning_tokens":269,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:52:15.907141+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled multi-tenant experiment comparing AI training jobs under HarmonAIze-style cooperative control (provider-side collectives and checkpointing, tenant-side scaling on provider events) against the same jobs on standard IaaS with static NCCL settings; if the cooperative cohort does not show measurably better job completion time, energy use, or failure recovery on realistic workloads with noisy neighbours, the core efficiency claim fails. A simpler disproof would be evidence that cloud providers cannot expose even aggregated infrastructure events at a granularity tenants can act on without leaking competitive information.","supporting_citations":[{"cited_title":"Workload Intelligence: Punching Holes Through the Cloud Abstraction","cited_arxiv_id":"2404.19143","evidence_quote":"Workload Intelligence is the prior framework HarmonAIze shares goals with but departs from, supplying the baseline for cloud-cooperation abstractions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MCCS demonstrates collectives-as-a-service with provider-side optimization, the model for O2 and micro-level abstractions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"KungFu shows runtime adaptation of training parameters, motivating O1 and the tenant-side macro-level control loop."},{"cited_title":"Ganger, and Eric P","cited_arxiv_id":null,"evidence_quote":"Pollux demonstrates co-adaptive scheduling benefits, motivating O3 and the need for adaptive resource allocation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Parcae shows proactive liveput-optimized training on preemptible instances, motivating adaptive jobs in O1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Andromeda provides a reference model for provider-side control loops placed in the hypervisor or network virtualization layer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"NetHint supplies evidence that white-box network information improves multi-tenant distributed ML, supporting the value of sharing infrastructure insight."},{"cited_title":"In EMNLP: System Demonstrations","cited_arxiv_id":null,"evidence_quote":"SuperBench shows proactive validation raising mean time between failures by 22.61×, motivating O6 and the need for workload-aware diagnostics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DynamoLLM demonstrates dynamic inference cluster reconfiguration cutting energy by 53% and carbon emissions by 38%, motivating O7."},{"cited_title":"Unicron: Economizing Self-Healing LLM Training at Scale","cited_arxiv_id":"2401.00134","evidence_quote":"Unicron shows managed self-healing LLM training achieving 1.9× efficiency, motivating O5 and provider-side fault tolerance."}],"review_version":1}