{"id":"817bfbdd-a421-4249-ad58-9077b07e7579","arxiv_id":"2412.16985","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"BladeDISC++ reduces GPU memory during dynamic-shape deep learning training by scheduling operations and rematerialization decisions based on symbolic shape expressions.","lead":"A compiler extension, BladeDISC++, schedules operations and chooses what to recompute or reload using symbolic shapes instead of exact tensor sizes, so memory can be optimized even when shapes are only known at runtime. It reports memory use close to static-shape training on one Llama-2 fine-tuning workload.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never quantifies how often symbolic-shape comparison succeeds versus its topology fallback, so the reported memory gains may not be attributable to the central symbolic mechanism.","rationale":"The reader identified the completeness of the symbolic shape graph as the weakest assumption. I agree: the paper's own text shows the method is best-effort and falls back to a topology heuristic on incomparable expressions, yet the evaluation does not isolate how often each path is taken or whether the fallback alone would reproduce the memory numbers. This is the most load-bearing concern because it goes directly to whether the symbolic-shape contribution, rather than existing heuristics or runtime exact-shape decisions, is what makes the method work. I also noticed concrete inconsistencies in the examples—Section 2.1 states expr1 = 11008*@S1 simplifies to 132096*@S0, which is arithmetically wrong (with @S0 = 12*@S1 it should be (11008/12)*@S0), and Section 2.2's ReadySet example appears to schedule an op after its predecessor has already been scheduled—but these are localized and could be presentation errors. The lack of coverage/ablation evidence is the broader, correctness-relevant issue. It does not refute the method, so it does not justify changing the reader's CONDITIONAL verdict; it does strengthen the need for the requested ablation and coverage measurement.","tokens_in":4996,"tokens_out":15199,"duration_ms":138584,"concrete_test":"Instrument BladeDISC++ on the Llama-2-1b/CodeAlpaca workload to log, for every op-scheduling decision and every rematerialization candidate, whether the SymbolicExpr comparison was resolved by algebraic simplification or by the topology fallback. Then run the same workload with all symbolic comparisons replaced by the topology fallback. If peak memory and the selected schedules/subgraphs are unchanged, the symbolic shape graph is not the load-bearing mechanism behind the claimed result and the central claim should be weakened accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that symbolic shapes alone let BladeDISC++ compare memory impacts of op sequences and recomputation subgraphs, and thereby match static-shape memory optimization. That claim requires the derived symbolic shape graph to be complete enough to order the memory-impact expressions that actually arise. Section 2.1 admits the comparison is a 'best-effort strategy,' Section 2.2 says that when two SymbolicExprs cannot be compared the method falls back to a graph-topology heuristic, and Section 2.3 uses a similar comparison for recomputation subgraphs without ever defining the sign convention for 'memory impact' (the example reports -11007*@S1, -11*@S1, and +1*@S1 and calls the +1 case the memory-efficient one). The evaluation reports only end-to-end peak memory for one model and one dataset. It does not report how often the symbolic comparison path succeeds versus the fallback, nor whether the claimed memory reductions survive an ablation that removes the symbolic comparison. If most decisions are made by the topology fallback or by runtime exact-shape bookkeeping, the load-bearing assertion that symbolic-shape reasoning, rather than conventional heuristics or runtime shapes, produces the comparable memory is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BladeDISC++, a compiler extension for dynamic shape graphs that performs memory optimization without exact tensor shapes. It constructs a symbolic shape graph expressing algebraic relations among shape dimensions, then uses symbolic expressions of memory impact to guide operation scheduling and to enumerate/evaluate rematerialization candidates at compile time, with final eviction decisions deferred to runtime. The evaluation reports peak memory and throughput for fine-tuning a Llama-2-1b model on CodeAlpaca-20K on one GPU, arguing that BladeDISC++ achieves memory consumption comparable to static-shape training while supporting dynamic shapes.","tokens_in":5236,"tokens_out":5388,"duration_ms":50306,"significance":"If the claims hold, the paper addresses a real and increasingly important gap: memory optimization for dynamic shape workloads without padding or recompilation. The core idea of deriving and comparing memory-impact expressions from a symbolic shape graph is concrete, falsifiable, and not circular: the memory expressions are derived from op semantics and the symbolic shape graph, not fitted to the reported measurements. The compile-time/runtime split for rematerialization is also a sensible design for shape dynamics. However, the current evidence is narrow—one model, one dataset, one GPU, one table with no variance or ablation—and the presentation of the comparison mechanism contains ambiguities that must be resolved before the central claim is fully supported.","major_comments":[{"comment":"The sign convention for the SymbolicExpr \"memory impact\" is never defined, and the two uses of the term appear inconsistent. In Section 2.2, the DotOp's impact is reported as 10996 * @S1 because scheduling it allocates %3 (11008 * @S1) and deallocates %2 (12 * @S1), which reads as allocated-minus-freed. In Section 2.3, the recomputation subgraph impacts are listed as -11007 * @S1, -11 * @S1, and +1 * @S1, and the +1 case is called \"memory-efficient,\" which reads as freed-minus-allocated (or positive savings). Since the entire scheduling and rematerialization selection is a comparison of these values, the paper must state the sign convention explicitly and use it consistently; otherwise the example cannot be checked and the load-bearing comparison mechanism is under-specified.","section":"Section 2.3"},{"comment":"The illustrative scheduling example is internally inconsistent. The text says \"The DynamicReshapeOp's memory impact, on the other hand, is 4096 * @S0 because scheduling it only involves allocation for %1,\" but DynamicReshapeOp produces %2, whose element count is 12 * @S1, not 4096 * @S0; %1 is produced by the preceding broadcast operation. This makes the sole worked example of symbolic comparison ambiguous and should be corrected to identify the op whose output is %1 and whose allocation is 4096 * @S0.","section":"Section 2.2, Listing 1"},{"comment":"The evaluation does not isolate the contribution of the symbolic comparison mechanism. The reported peak memory and throughput are end-to-end numbers, and the paper neither reports how often the symbolic comparison path succeeds versus the graph-topology fallback described in Section 2.2, nor includes an ablation with the symbolic comparison disabled. Without such an ablation or success-rate statistics, the claim that symbolic-shape reasoning, rather than the topology heuristic or runtime exact-shape bookkeeping, produces the memory comparable to static-shape training is not supported. I would also ask for repeated runs or variance information, since differences as small as 0.01 GiB (batch size 14) are reported without any measure of run-to-run variability.","section":"Section 3, Table 1"},{"comment":"The runtime rematerialization decision is described only as following \"a similar approach as outlined in [10]\" (Delta). Because the proposed method's novelty depends on the combination of compile-time symbolic subgraph search and runtime decisions, the paper should specify what symbolic information is used in the runtime branch and how the runtime decision incorporates the SymbolicExpr-based memory impacts, rather than deferring entirely to a prior system.","section":"Sections 2.3 and 3"}],"minor_comments":[{"comment":"The text switches between \"exp1\" and \"expr1\" for the same symbolic expression; please use one spelling consistently.","section":"Section 2.1"},{"comment":"The sentence \"the last consumer of %2\" is followed immediately by \"// The last consumer of %3\" in the listing without a clear narrative link; consider labeling each op in the example with its name and output tensor to make the scheduling discussion easier to follow.","section":"Section 2.2"},{"comment":"Table 1 would be clearer if it reported the GPU memory limit explicitly and if the OOM entries were marked with the batch size at which the limit was exceeded, rather than only showing \"OOM\" with no throughput.","section":"Section 3"},{"comment":"Several references are cited by URL with no version or commit information (e.g., [4], [11], [13], [14]), which makes reproducibility harder; please add versioned citations or DOIs where available.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a plausible industry-experience paper with a clear and testable central idea, but the current evidence is well below what I would expect for acceptance at a top systems venue: a single end-to-end experiment, no ablation, no repeated runs, and no artifact release. The main fixable issues are the undefined/inconsistent sign convention for memory impact, the inconsistent worked example in Section 2.2, and the lack of an ablation separating the symbolic mechanism from the fallback. I would encourage the authors to address these and resubmit; I do not see a fundamental correctness flaw that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper has a real idea, and the main weakness is not the idea but the evidence attached to it.\n\nWhat is new: using a symbolic shape graph to compare memory impacts of ops and recomputation subgraphs at compile time, then deferring the actual eviction decision to runtime through pre-enumerated branches. That is a genuine extension over static-shape schedulers and rematerializers, and over runtime-only systems like DTR and Delta. The shape relation derivation from op semantics (e.g., @S0 = 12 * @S1) is coherent, and I don't see circularity: the reported memory numbers are benchmarked against static shape training, not fitted. The self-citations to BladeDISC/DISC are appropriate substrate.\n\nWhere it goes soft: the empirical section is one model, one dataset, one GPU, one table, with no variance, no repeats, and no ablation. The stress-test note lands: the paper itself admits symbolic comparisons are best-effort and falls back to a graph-topology heuristic when expressions cannot be compared, but it never reports how often that fallback triggers. Without that, the central claim that symbolic-shape reasoning produces memory comparable to static-shape training is not backed up; the gains could come from the topology heuristic or from runtime exact-shape bookkeeping. The rematerialization example also has a confusing sign convention (-11007 * @S1 vs +1 * @S1) and the signs are never defined. That is a presentation issue, not a load-bearing flaw. The absence of an artifact is a real cost in a systems paper.\n\nWho this is for: compiler folks working on dynamic-shape memory optimization. The idea deserves a serious referee, but the current evaluation would not convince me to adopt the technique without more evidence. My recommendation: send it to review with a request for a stronger evaluation; specifically, an ablation isolating the symbolic comparison from the fallback, statistics on comparison success rates, more than one workload, and a code release.","headline":"Plausible symbolic-shape memory optimization idea, but the evaluation is too thin to show the symbolic mechanism is what carries the gains.","tokens_in":5736,"tokens_out":3461,"would_cite":true,"duration_ms":31668,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dynamic-shape compiler can optimize memory by comparing symbolic shape expressions instead of exact tensor sizes, keeping peak memory within a few percent of static-shape training.","keywords":["dynamic shape compilation","symbolic shapes","memory optimization","operator scheduling","rematerialization","recomputation","GPU memory","deep learning compiler"],"falsifier":"Construct a dynamic graph with two legal op orders whose relative memory depends on a non-affine shape relation, such as $S_0 = \\lfloor S_1 / 2 \\rfloor$, and let BladeDISC++ choose; if its symbolic comparison cannot derive the relation and it selects the order with higher peak memory than the exact-shape optimum, the claim that symbolic shapes suffice for memory optimization is falsified.","tokens_in":4806,"feed_emoji":"⚙️","tokens_out":7313,"duration_ms":62070,"temperature":0.7,"pith_summary":"BladeDISC++ sets out to prove that memory optimization does not require exact tensor shapes: a compiler can compare the memory impact of scheduling choices and recomputation subgraphs using symbolic expressions over shape variables. The paper builds a global symbolic shape graph from op semantics, derives algebraic relations among dimensions, and uses those relations to simplify and order memory-impact expressions. For rematerialization, it moves the eviction decision to runtime because peak memory varies across runs, while the search for candidates and regeneration subgraphs happens at compile time. On fine-tuning a 1B-parameter transformer with variable-length batches, the resulting peak GPU memory is close to static-shape training while avoiding recompilation and padding overhead. If correct, this makes dynamic-shape compilation more practical for large-model training.","feed_headline":"Symbolic-shape compiler matches static-shape GPU memory","feed_subtitle":"On a 1B-parameter transformer with variable-length batches, peak memory stays within 0.2–0.3 GiB of static shape training.","key_machinery":"The symbolic shape graph is the mechanism: a global structure whose nodes are symbolic dimensions ($S_0, S_1, \\dots$) of tensor shapes and whose edges record algebraic equations inferred from operators, e.g. reshape implies equality of element counts. On top of it, memory impacts are expressed as SymbolicExprs, such as $11008 \\cdot S_1$ or $1024 \\cdot S_0$, and comparisons are made by substituting equalities (e.g. replacing $S_0$ with $12 S_1$) and simplifying. This same comparison machinery drives both the op scheduler's choice among ready operations and the rematerialization search's decision whether a recomputation subgraph reduces or raises peak memory; the runtime monitor then bridges the residual gap between symbolic knowledge and the actual shapes seen in a given run.","core_discovery":"The paper's central claim is that a fixed-topology graph with unknown tensor shapes still carries enough information to make memory-conscious decisions, once those shapes are represented symbolically and tied together by semantic equations. Each unknown dimension becomes a symbolic variable, and op semantics produce equations such as $S_0 = 12 S_1$ from a dynamic reshape; tensor sizes then become symbolic expressions, and two expressions can be compared after simplification through the graph. The op scheduler uses these comparisons to choose, at each step, the ready operation with the smallest memory footprint, and falls back to a graph-topology heuristic only when expressions are not reducible to a comparable form. For rematerialization, the compiler enumerates eviction candidates and generates recomputation/reload subgraphs whose memory impacts are assessed symbolically, then inserts runtime decision points where actual memory pressure determines which tensor to evict and how to regenerate it. The reported peak-memory numbers—35.76, 37.89, and 39.18 GiB for batch sizes 14, 16, and 18 versus 35.75, 37.71, and 38.92 GiB in static-shape training—are offered as evidence that symbolic-shape optimization is on par with exact-shape optimization.","pith_inferences":["Beyond the paper, a natural extension is applying the same symbolic comparison to op fusion choices and to host-device offloading, where memory impact is likewise a symbolic function of shape.","The runtime eviction decision opens the door to online re-scheduling as actual shapes arrive: if the compile-time symbolic ranking is inconclusive, runtime could re-rank by exact sizes without recompiling.","The completeness assumption will matter most on workloads with non-affine shape relations, such as sizes derived from data-dependent sampling or geometric transforms; our reading is that those cases would degrade to the heuristic fallback, and it is untested how often that occurs in practice.","Because the evaluation covers one 1B-parameter model and one dataset, the claim of parity with static-shape memory is a proof of concept; the symbolic-comparison mechanism itself seems model-agnostic, though runtime decision overhead and fallback frequency should be measured on more workloads."],"forward_implications":["Dynamic-shape training can avoid shape specialization and input bucketing while still getting static-shape-level peak memory, cutting recompilation overhead.","Rematerialization becomes a two-phase decision: exhaustive candidate and regeneration-subgraph search at compile time, with a cheap eviction choice at runtime using monitored memory pressure.","Memory-impact comparison works across expressions with disjoint symbol sets whenever the symbolic graph supplies enough equations to relate them.","The approach generalizes to any dynamic workload whose graph topology is fixed but whose tensor sizes vary, such as variable-batch or variable-length inference.","When expressions cannot be compared, the method falls back to a topology-based heuristic, preserving a practical optimization even without proof of optimality."],"supporting_citations":[{"why":"Prior BladeDISC work that provides the dynamic shape compiler base and op fusion that BladeDISC++ builds on.","marker":"[16]"},{"why":"The DISC dynamic shape compiler in which the symbolic shape representation is rooted.","marker":"[17]"},{"why":"The XLA scheduling pass that supplies the ready-set op selection methodology this paper adapts to symbolic shapes.","marker":"[8]"},{"why":"Checkmate, whose formulation of optimal tensor rematerialization defines the recomputation subgraph search being made shape-symbolic.","marker":"[6]"},{"why":"Delta, the runtime rematerialization approach whose eviction and regeneration decision factors BladeDISC++ follows.","marker":"[10]"},{"why":"Early recomputation work that motivates regeneration subgraphs and the memory-performance tradeoff.","marker":"[3]"},{"why":"CodeAlpaca-20K, the variable-length dataset used for the dynamic-shape training evaluation.","marker":"[1]"},{"why":"The Llama-2 model family from which the 1B-parameter test model is derived.","marker":"[12]"},{"why":"The GPU instance with 40GB memory that sets the peak-memory constraint in the reported experiments.","marker":"[4]"}],"fun_headline_variants":["Symbolic shapes close memory gap for dynamic graphs","Dynamic shape compiler matches static-shape GPU memory","BladeDISC++: memory on par with static shapes","Symbolic reasoning enables memory optimizations for dynamic shapes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the symbolic shape graph captures every relation needed to compare the memory impact of any two op sequences or recomputation subgraphs that matter; if a relation is missing or non-affine, the comparison falls back to a heuristic without a guarantee.","fun_headline_variants_meta":{"raw":{"variants":["Symbolic shapes close memory gap for dynamic graphs","Dynamic shape compiler matches static-shape GPU memory","BladeDISC++: memory on par with static shapes","Symbolic reasoning enables memory optimizations for dynamic shapes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1571,"prompt_tokens":977,"completion_tokens":594,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":531}},"tokens_in":593,"tokens_out":594,"duration_ms":5847,"temperature":1.0,"reasoning_tokens":531,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:53:36.185464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a dynamic graph with two legal op orders whose relative memory depends on a non-affine shape relation, such as $S_0 = \\lfloor S_1 / 2 \\rfloor$, and let BladeDISC++ choose; if its symbolic comparison cannot derive the relation and it selects the order with higher peak memory than the exact-shape optimum, the claim that symbolic shapes suffice for memory optimization is falsified.","supporting_citations":[{"cited_title":"meta-llama/llama-2-7b, 2024","cited_arxiv_id":null,"evidence_quote":"The Llama-2 model family from which the 1B-parameter test model is derived."},{"cited_title":"Accessed: December 24, 2024","cited_arxiv_id":null,"evidence_quote":"CodeAlpaca-20K, the variable-length dataset used for the dynamic-shape training evaluation."},{"cited_title":"Bladedisc: Optimizing dynamic shape machine learning workloads via compiler approach","cited_arxiv_id":null,"evidence_quote":"Prior BladeDISC work that provides the dynamic shape compiler base and op fusion that BladeDISC++ builds on."},{"cited_title":"Disc: A dynamic shape compiler for machine learning workloads","cited_arxiv_id":null,"evidence_quote":"The DISC dynamic shape compiler in which the symbolic shape representation is rooted."},{"cited_title":"Delta: Dynamically optimizing gpu memory beyond tensor recomputation, 2022","cited_arxiv_id":null,"evidence_quote":"Delta, the runtime rematerialization approach whose eviction and regeneration decision factors BladeDISC++ follows."},{"cited_title":"Alibaba cloud ecs.gn7-c12g1.3xlarge instance, 2024","cited_arxiv_id":null,"evidence_quote":"The GPU instance with 40GB memory that sets the peak-memory constraint in the reported experiments."}],"review_version":1}