pith. sign in

arxiv: 2511.21612 · v2 · submitted 2025-11-26 · 💻 cs.DC

Diagonal Scaling: A Multi-Dimensional Resource Model and Optimization Framework for Distributed Databases

Shahir Abdullah , Syed Rohit Zaman This is my paper

Pith reviewed 2026-05-17 04:07 UTC · model grok-4.3

classification 💻 cs.DC
keywords diagonal scalingscaling planedistributed databasesautoscalingresource optimizationcloud computingperformance trade-offslocal search
0
0 comments X

The pith

Optimal scaling trajectories in distributed databases frequently lie along diagonal paths that jointly adjust node count and per-node resources.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the Scaling Plane as a two-dimensional model in which each database configuration is a point combining node count with a vector of per-node resources. It shows that optimal scaling often requires diagonal moves that change both dimensions at once instead of scaling nodes or resources separately. A local-search algorithm called DIAGONALSCALE finds these moves while respecting service-level agreements. Experiments on synthetic models and real distributed SQL and key-value systems report lower tail latency, lower cost per query, and far less rebalancing than one-dimensional approaches.

Core claim

We show analytically and empirically that optimal scaling trajectories frequently lie along diagonal paths: sequences of joint horizontal and vertical adjustments that simultaneously exploit cluster parallelism and per-node improvements. To compute such actions, we propose DIAGONALSCALE, a discrete local-search algorithm that evaluates horizontal, vertical, and diagonal moves in the Scaling Plane and selects the configuration minimizing a multi-objective function subject to SLA constraints. Using synthetic surfaces, microbenchmarks, and experiments on distributed SQL and KV systems, we demonstrate that diagonal scaling reduces p95 latency by up to 40 percent, lowers cost-per-query by up to

What carries the argument

The Scaling Plane, a two-dimensional model in which each distributed database configuration is represented as a point (H, V), with H denoting node count and V a vector of resources, over which smooth approximations of latency, throughput, coordination overhead, and monetary cost are defined.

If this is right

  • Diagonal scaling reduces p95 latency by up to 40 percent compared to horizontal-only and vertical-only autoscaling.
  • Diagonal scaling lowers cost-per-query by up to 37 percent compared to horizontal-only and vertical-only autoscaling.
  • Diagonal scaling reduces rebalancing by 2 to 5 times compared to horizontal-only and vertical-only autoscaling.
  • The approach supplies a unified view of performance-cost trade-offs that prevents overreaction to load spikes and oscillation between suboptimal states.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Existing cloud autoscalers could adopt diagonal moves as a default policy once the required surface approximations are available.
  • The same two-dimensional framing might extend to other elastic systems such as stream processors or machine-learning training clusters.
  • Higher-dimensional generalizations that add network or storage axes could be tested on the same local-search skeleton.

Load-bearing premise

The smooth approximations of latency, throughput, coordination overhead, and monetary cost over the Scaling Plane accurately capture real system dynamics, and the discrete local-search algorithm identifies near-optimal configurations under SLA constraints.

What would settle it

A side-by-side run on the same distributed SQL and KV workloads in which diagonal trajectories produce no measurable reduction in p95 latency, cost-per-query, or rebalancing frequency compared with horizontal-only or vertical-only autoscaling.

Figures

Figures reproduced from arXiv: 2511.21612 by Shahir Abdullah, Syed Rohit Zaman.

Figure 1
Figure 1. Figure 1: The Scaling Plane, showing horizontal, vertical, and diagonal scaling [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Synthetic latency surface L(H, V ) demonstrating reduced per-node latency with larger V and increased coordination latency with larger H [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cost-per-query heatmap illustrating interior minima reachable by [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 6
Figure 6. Figure 6: Cost per unit time. DIAGONALSCALE reduces over-provisioning at low load [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Rebalance events and scaling actions. DIAGONALSCALE reduces disruptive changes by 2–5×. G. Algorithm Efficiency Each autoscaling decision takes < 5 ms due to closed-form evaluation. This is suitable for real-time control loops (1–5 seconds per iteration). VIII. DISCUSSION A. Implications for Cloud Autoscaling Diagonal scaling opens up a new class of autoscaling strategies that combine HPA-like and VPA-like… view at source ↗
read the original abstract

Modern cloud databases present scaling as a binary decision: scale-out by adding nodes or scale-up by increasing per-node resources. This one-dimensional view is limiting because database performance, cost, and coordination overhead emerge from the joint interaction of horizontal elasticity and per-node CPU, memory, network bandwidth, and storage IOPS. As a result, systems often overreact to load spikes, underreact to memory pressure, or oscillate between suboptimal states. We introduce the Scaling Plane, a two-dimensional model in which each distributed database configuration is represented as a point (H, V), with H denoting node count and V a vector of resources. Over this plane, we define smooth approximations of latency, throughput, coordination overhead, and monetary cost, providing a unified view of performance trade-offs. We show analytically and empirically that optimal scaling trajectories frequently lie along diagonal paths: sequences of joint horizontal and vertical adjustments that simultaneously exploit cluster parallelism and per-node improvements. To compute such actions, we propose DIAGONALSCALE, a discrete local-search algorithm that evaluates horizontal, vertical, and diagonal moves in the Scaling Plane and selects the configuration minimizing a multi-objective function subject to SLA constraints. Using synthetic surfaces, microbenchmarks, and experiments on distributed SQL and KV systems, we demonstrate that diagonal scaling reduces p95 latency by up to 40 percent, lowers cost-per-query by up to 37 percent, and reduces rebalancing by 2 to 5 times compared to horizontal-only and vertical-only autoscaling. Our results highlight the need for multi-dimensional scaling models and provide a foundation for next-generation autoscaling in cloud database systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper introduces the Scaling Plane, a two-dimensional model in which each distributed database configuration is a point (H, V) with H as node count and V as a vector of per-node resources (CPU, memory, network, storage). It defines smooth approximations to latency, throughput, coordination overhead, and monetary cost over this plane, analytically and empirically argues that optimal scaling trajectories frequently lie along diagonal paths, and presents DIAGONALSCALE, a discrete local-search algorithm that evaluates horizontal, vertical, and diagonal moves to minimize a multi-objective cost subject to SLA constraints. Experiments on synthetic surfaces, microbenchmarks, and real distributed SQL/KV systems report up to 40% lower p95 latency, 37% lower cost-per-query, and 2–5× fewer rebalances versus horizontal-only or vertical-only autoscaling.

Significance. If the smooth approximations prove faithful, the work supplies a principled multi-dimensional alternative to the prevailing binary scale-out/scale-up view and could improve resource efficiency in cloud databases. The first-principles construction of the model (no fitted parameters) and the combination of analytical optimality arguments with experiments on both synthetic and production-like systems are positive features. The reported quantitative gains, if reproducible, would constitute a practically relevant advance in autoscaling.

major comments (1)
  1. [Abstract / §3] Abstract and §3 (model definition): the central optimality claim and the reported 40%/37%/2–5× gains rest on the assumption that the smooth functions for latency, throughput, coordination overhead, and cost accurately represent real-system dynamics. Real distributed SQL/KV systems exhibit discrete jumps at rebalancing, leader election, and cache-warming events; if these discontinuities dominate the trajectories found by DIAGONALSCALE, the analytical argument and empirical improvements do not transfer.
minor comments (1)
  1. The manuscript would benefit from an explicit statement of the functional forms used for the smooth approximations (e.g., the exact expression for coordination overhead as a function of H and V).

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for this constructive comment on the fidelity of our smooth approximations. We respond to the major comment below and have made a partial revision to clarify the relationship between the model and real-system behavior.

read point-by-point responses

  1. Referee: [Abstract / §3] Abstract and §3 (model definition): the central optimality claim and the reported 40%/37%/2–5× gains rest on the assumption that the smooth functions for latency, throughput, coordination overhead, and cost accurately represent real-system dynamics. Real distributed SQL/KV systems exhibit discrete jumps at rebalancing, leader election, and cache-warming events; if these discontinuities dominate the trajectories found by DIAGONALSCALE, the analytical argument and empirical improvements do not transfer.

    Authors: We acknowledge that real distributed systems exhibit discrete events such as rebalancing, leader elections, and cache warming that introduce discontinuities not fully captured by our smooth approximations. The analytical arguments in §3 use these approximations to identify that diagonal paths are often optimal in the continuous model. However, DIAGONALSCALE is a discrete local-search procedure that can incorporate real measurements. Our experimental results on synthetic surfaces, microbenchmarks, and real distributed SQL/KV systems (reported in §4 and §5) were obtained by running the systems and measuring actual p95 latency, cost, and rebalancing counts, which inherently include all discrete events. The 2-5× reduction in rebalances is direct evidence that the approach mitigates the impact of such events by choosing better trajectories. We have added a new paragraph in §3.3 discussing the limitations of the smoothness assumption and how the algorithm's use of empirical feedback addresses it. This revision clarifies the relationship between the model and real dynamics. revision: partial

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained from first-principles resource model

full rationale

The paper defines the Scaling Plane as (H, V) points and introduces smooth approximations for latency, throughput, coordination overhead, and cost directly from joint interactions of horizontal and vertical resources. The analytical claim that optimal trajectories lie along diagonal paths is obtained by optimizing the multi-objective function over this explicitly constructed plane, and DIAGONALSCALE is a discrete local-search procedure that enumerates horizontal, vertical, and diagonal moves subject to SLA constraints. No equation or step reduces a claimed prediction to a fitted parameter or self-citation by construction; the model inputs are independent descriptions of database dynamics rather than quantities defined in terms of the outputs. Empirical results on synthetic surfaces and real systems are presented as external validation, not tautological confirmation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claim depends on the existence of accurate smooth performance surfaces and the effectiveness of local search over the defined plane; no explicit free parameters are stated in the abstract.

axioms (1)
  • domain assumption Smooth approximations of latency, throughput, coordination overhead, and monetary cost can be defined over the Scaling Plane.
    Invoked to provide a unified view of performance trade-offs.
invented entities (2)
  • Scaling Plane no independent evidence
    purpose: Two-dimensional representation of database configurations as (H, V) points.
    Core modeling construct introduced by the paper.
  • DIAGONALSCALE no independent evidence
    purpose: Discrete local-search algorithm that evaluates horizontal, vertical, and diagonal moves.
    Proposed method for computing optimal scaling actions.

pith-pipeline@v0.9.0 · 5590 in / 1269 out tokens · 38620 ms · 2026-05-17T04:07:12.526027+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages

  1. [1]

    Amazon aurora serverless v2: Instant, auto- scaling configuration for aurora

    Amazon Web Services. Amazon aurora serverless v2: Instant, auto- scaling configuration for aurora. https://aws.amazon.com/rds/aurora/ serverless/. Accessed 2025-11-22

    work page 2025

  2. [2]

    Franklin, Ali Ghodsi, Joseph M

    Peter Bailis, Alan Fekete, Michael J. Franklin, Ali Ghodsi, Joseph M. Hellerstein, and Ion Stoica. Highly available transactions: Virtues and limitations. InProceedings of the VLDB Endowment, volume 7, pages 181–192, 2013

    work page 2013

  3. [3]

    Hsieh, Deb- orah A

    Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deb- orah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. Bigtable: A distributed storage system for structured data.ACM Transactions on Computer Systems, 26(2):4:1–4:26, 2008

    work page 2008

  4. [4]

    Cockroachdb: Architecture and design overview

    Cockroach Labs. Cockroachdb: Architecture and design overview. https: //www.cockroachlabs.com/docs/. Accessed 2025-11-22

    work page 2025

  5. [5]

    Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears

    Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (SoCC), pages 143–154. ACM, 2010

    work page 2010

  6. [6]

    Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J

    James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J. J. Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, Shiva Naghshineh, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szyma...

    work page 2012

  7. [7]

    Dynamo: Amazon’s highly available key-value store

    Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubra- manian, Peter V osshall, and Werner V ogels. Dynamo: Amazon’s highly available key-value store. InProceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP), pages 205–220. ACM, 2007

    work page 2007

  8. [8]

    Kubernetes horizontal pod autoscaler

    Kubernetes Authors. Kubernetes horizontal pod autoscaler. https:// kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/. Ac- cessed 2025-11-22

    work page 2025

  9. [9]

    Kubernetes cluster autoscaler

    Kubernetes Autoscaling SIG. Kubernetes cluster autoscaler. https: //github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler. Ac- cessed 2025-11-22

    work page 2025

  10. [10]

    Kubernetes vertical pod autoscaler

    Kubernetes Autoscaling SIG. Kubernetes vertical pod autoscaler. https: //github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler. Accessed 2025-11-22

    work page 2025

  11. [11]

    Cassandra: A decentralized structured storage system.ACM SIGOPS Operating Systems Review, 44(2):35–40, 2010

    Avinash Lakshman and Prashant Malik. Cassandra: A decentralized structured storage system.ACM SIGOPS Operating Systems Review, 44(2):35–40, 2010

    work page 2010

  12. [12]

    Yugabytedb architecture

    Yugabyte, Inc. Yugabytedb architecture. https://docs.yugabyte.com/. Accessed 2025-11-22

    work page 2025