{"id":"e32c528a-6133-40cb-81bb-3a7f0eacc3aa","arxiv_id":"2506.20511","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A distributed randomized binary search lets federated learning clients find a hardware-safe shared batch size within a few rounds, speeding up training compared to a small default batch size.","lead":"This paper proposes a randomized binary search method, called RASBA, for federated learning clients to jointly find the largest batch size that fits in every participant's GPU memory. A smart generalist might read it because wrong batch sizes are a common, unglamorous cause of slow or crashed federated training runs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"b_max=64 is a hard cap that contradicts the paper's own evidence that batches 128 and 256 run on the test GPU, so RASBA cannot find the claimed physical maximum.","rationale":"The reader's weakest assumption already identified the b_max=64 cap, and my stress test agrees with that concern while sharpening it into an internal contradiction: the paper's own Table I lists successful runs at batch=128 and 256 on the same hardware, so the algorithm cannot be finding the maximum physically supported batch size because it never probes above 64. The proposed concrete test is decisive and cheap: re-run with a larger initial cap and compare the converged batch size and accuracy. If the method then selects a batch size well above 64 with degraded accuracy, the reported speed/accuracy trade-off is an artifact of the exogenous cap rather than of the search mechanism. The verdict remains CONDITIONAL: the paper presents a plausible safe-bound-search heuristic with reproducible code, but the central 'maximum' claim needs to be reframed or substantiated with experiments that do not pre-truncate the search space. The reader's conditional verdict already captures this need, so no change in verdict is recommended.","tokens_in":6198,"tokens_out":6404,"duration_ms":77343,"concrete_test":"Run the public implementation on the same MNIST setup but initialize b_max=256 (or the client dataset-size bound) instead of 64, keeping f=0.5. Record the batch size RASBA converges to and the resulting accuracy/time, and compare with Table I. If the converged batch size exceeds 64 and accuracy drops toward the 83.17% row, or the search fails to terminate in a few rounds, then the reported near-optimal result depends on the hand-chosen cap and the 'maximum physically supported' claim is untenable. Also inspect the released code to verify that b_max is never raised when a client succeeds at the current cap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central claim—that RASBA discovers the maximum physically supported shared batch size—is invalidated by the hard cap b_max=64 set in Section IV. On the paper's own GTX 1080, Table I and Figure 1 show batch sizes 128 and 256 execute successfully; batch=256 is the fastest row (342s on MNIST). Since the server aggregates b_max = min_i(b_i_max), and no client ever samples above 64, the search interval can never expand beyond 64. RASBA therefore cannot find the true physical maximum; it only certifies the largest safe size inside a hand-chosen interval. The algorithm is more honestly described as a safe-bound explorer within [b_min, b_max]. Because the paper repeatedly concludes that it determines a maximal shared batch size, the empirical results do not support the stated contribution. The cap also conflates physical support with accuracy: large batches may fit in memory yet degrade accuracy (batch=256 gives 83.17% vs. 96.09% for RASBA on MNIST), so the 'safe' batch size is determined by the authors' prior, not by the search. This is not an external-consensus disagreement; it is an internal inconsistency with Table I.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RASBA, a greedy randomized binary-search procedure run collaboratively across federated-learning clients, to choose a shared local mini-batch size. The server sends b_min and b_max to clients, each client samples a batch size in that interval and reports success or an out-of-memory failure, and the server aggregates by taking the maximum of the reported minima and the minimum of the reported maxima. Experiments on MNIST with MobileNet-v3 and CIFAR10 with ResNet18 on a single GTX 1080 GPU compare RASBA against fixed batch sizes in terms of wall-clock time and final accuracy. The paper reports that RASBA reaches near-optimal accuracy much faster than the default batch size 4 and that it finds the maximal possible shared batch size within a few rounds.","tokens_in":6446,"tokens_out":3756,"duration_ms":42472,"significance":"If the central claims held, the method would be a simple, practical tool for FL simulations, where batch-size limits are often chosen manually and conservatively. The paper has concrete strengths: the code is stated to be available, the experimental setup uses standard frameworks (Flower, PyTorch), and Table I provides a clear wall-clock comparison across fixed batch sizes. The qualitative result that larger batches reduce training time on this GPU is plausible and consistent with Figure 1. However, the paper's main contribution, discovering the maximum physically supported shared batch size, is not established by the current evidence because the search is hard-capped at 64 even though batches 128 and 256 run successfully on the same hardware. The accuracy discrepancy between RASBA and fixed batch 64 is also unexplained, and there are no repeated runs or error bars. The contribution is therefore only conditionally supported; it is a reasonable idea that needs a corrected evaluation and a re-scoped or strengthened claim.","major_comments":[{"comment":"The initial cap b_max=64 contradicts the paper's maximum-discovery claim. Table I shows that batch sizes 128 and 256 complete successfully on the same GTX 1080 (e.g., 416s and 342s on MNIST), and Figure 1 includes batch=128 and batch=256. Since Algorithm 1 only samples b in [b_min, b_max] and the server sets b_max = min_i(b_i_max), no client ever samples above 64 and the shared b_max can never exceed 64. Consequently, RASBA cannot find the maximum physically supported shared batch size; it only finds a safe value inside the hand-chosen interval [b_min, 64]. The abstract, introduction, and conclusion state that the method determines 'the maximum achievable shared batch size' or 'maximal possible shared batch size,' and those statements are not supported by the experiments. Either re-scope the claim to 'largest safe batch size within a user-specified interval' or modify the algorithm to expand the upper bound (e.g., doubling) until failure is observed.","section":"Section IV, Algorithm 1 and Table I"},{"comment":"The accuracy of 'our method' is not consistent with the method's own search behavior. On MNIST, RASBA reaches 96.09% accuracy, whereas fixed batch 64 reaches 91.21%; on CIFAR10, the corresponding values are 73.42% versus 70.62%. Because the search is capped at 64 and is claimed to converge to the shared batch size, one would expect RASBA's accuracy to be close to that of fixed batch 64 rather than substantially better. The discrepancy is not explained; it may be due to the fraction f=0.5 of clients training at b_min during the search, to random intermediate batch sizes, or to unmeasured run-to-run variance. No repeated runs or error bars are reported, so the 7.76x and 4.78x speedup figures and the accuracy numbers could be affected by noise. The paper needs either an explanation of the mechanism behind the accuracy difference or experiments with multiple seeds to establish that the advantage is real.","section":"Section V, Table I"},{"comment":"The paper claims that 'with f=0.5 we observe that our method finds the maximal possible shared batch size in at most 3 rounds,' but no evidence for this is presented. There is no plot or table showing the batch size selected by RASBA over rounds, no record of how many clients sampled which values, no count of out-of-memory failures, and no measurement of the round at which b_min and b_max converge. Without this information, the central convergence-time claim is not verifiable. The authors should report the search trajectory (sampled batch sizes and b_min/b_max progress) for at least one representative run on each dataset.","section":"Section V, Figures 2/3"},{"comment":"The evaluation uses a single homogeneous GPU (GTX 1080) with 10 simulated clients, so the federated, collaborative aspect of the method is never tested under hardware heterogeneity. The paper's motivation is that the central server is unaware of client hardware capabilities, but all clients in the experiment have identical capabilities. Consequently, the claim that RASBA helps when clients have different hardware limits is not demonstrated. A small experiment with two or three client hardware tiers (or simulated memory limits) would directly test the server's min-aggregation rule and the method's robustness to heterogeneous b_i_max.","section":"Section V, Setup"}],"minor_comments":[{"comment":"The sentence 'Due to a higher sampling rate compared to the generic randomized binary search, our method reduces the time to convergence by ((1-f)×m)^{-1} compared to the binary search' is unclear and is not derived or verified in the experiments; please clarify the definition of 'sampling rate' and the derivation of this factor.","section":"Section IV"},{"comment":"The text says clients 'randomly sample a batch size bi > bmin,' while Algorithm 1 samples b in [b_min, b_max]; these formulations are inconsistent, and the algorithm should state the exact sampling distribution (e.g., uniform over integers in the interval).","section":"Section IV, Algorithm 1"},{"comment":"There are several small presentation issues: 'Federations on MNIST run for 25, while federations on CIFAR10 run for 20 global rounds' is missing the word 'rounds'; the anonymous GitHub link is not included in the text; and Figure 1 would benefit from explicit values or error bars for memory and GPU utilization.","section":"Section V, Setup"},{"comment":"The fixed batch 128 result on CIFAR10 (936s) is slower than fixed batch 64 (607s) and even fixed batch 32 (820s), which is not discussed; this non-monotonicity is worth a brief explanation because it affects the claim that larger batches always reduce time.","section":"Section V, Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper reads like a workshop-style empirical note rather than a fully developed journal article. The main issue is not the absence of a proof but an internally inconsistent claim: the evaluation itself shows batch sizes 128 and 256 work, while the algorithm's hard cap prevents it from ever considering them. This is fixable by re-scoping the contribution or by extending the search range, but it is load-bearing, so the current version should not be accepted. The accuracy discrepancy between RASBA and fixed batch 64 also needs a real explanation before the results can be trusted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jan,\n\nThis is a small paper with a real but narrow idea, and a central claim that doesn't survive contact with its own Table I. The idea is to run a randomized binary search over batch sizes during the first rounds of federated training, let each client catch out-of-memory errors, and have the server aggregate the per-client lower/upper bounds to pick a shared safe batch size. That's novel enough—I don't know prior work that probes hardware limits this way. The experiments are reproducible: code on GitHub, Flower + PyTorch, and they report GPU memory/utilization figures. That counts.\n\nThe soft spot is not subtle. The paper says its goal is to find the maximum physically supported shared batch size, and the conclusion repeats that. But Section IV sets b_max=64 as an initial cap, and the algorithm never samples above it. Table I and Figure 1 show batches of 128 and 256 run fine on the same GTX 1080—batch 256 is actually the fastest row on MNIST. So RASBA cannot possibly find the physical maximum. It just certifies the best safe value inside the hand-chosen interval [b_min, 64]. That is an inconsistency with the stated contribution, not a stylistic overclaim. The fix is to reframe the contribution as safe-bound discovery within a user-defined budget, or to make the cap adaptive.\n\nOther soft spots: no repeated runs or error bars, so I don't know whether the 96.09% MNIST accuracy is signal or luck. The method ends up more accurate than any fixed batch size, which is odd and unexplained. The algorithm is underspecified: what is b_min, how exactly does the fraction f of clients train during the search, and what does the claimed speed-up formula mean? All fixable.\n\nI'd send this to reviewers, not desk-reject it. The idea is simple, the experiments are honest in placement (simulation on a single GPU), and the cap problem is exactly what a good referee would catch. After a revision that narrows the claims and adds seeds, this could be a decent workshop or short-conference paper. Worth a look if you work on FL systems; not a must-read.","headline":"A useful but overclaimed batch-size search: the hard cap at 64 means the paper can't support its 'physical maximum' claim.","tokens_in":6961,"tokens_out":2483,"would_cite":false,"duration_ms":25530,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated randomized binary search can determine the largest safe shared batch size in a few rounds, speeding up training by up to 7.8x versus a default batch size of 4.","keywords":["federated learning","batch size optimization","randomized binary search","GPU memory constraints","convergence speed","hyperparameter optimization","RASBA","hardware-aware training"],"falsifier":"Run the same federations on the same GPU with $b_{\\max}$ initialized to 256 instead of 64 and record whether the search ever converges to a shared batch size above 64 with faster wall-clock completion; if it does, the capped configuration is not finding the physical maximum the method claims to find.","tokens_in":6008,"feed_emoji":"⚙️","tokens_out":8712,"duration_ms":84431,"temperature":0.7,"pith_summary":"The paper argues that federations can discover a safe shared batch size themselves instead of relying on a hand-picked configuration. It proposes RASBA, a greedy randomized binary search in which each client samples a candidate batch size each round, a successful update raises the lower bound, and an out-of-memory failure lowers the upper bound, with the server merging all clients' bounds. On MNIST and CIFAR10, the search settles on a workable common batch size within at most three rounds, cutting wall-clock time by 7.76x and 4.78x compared with the batch-size-4 default while reaching accuracies of 96.09% and 73.42%. The practical point is that the hardware-limited batch size can be set automatically without risking a client crash or a lost training round.","feed_headline":"Batch-size search finds safe limit, speeds federated learning 7.8x","feed_subtitle":"On MNIST and CIFAR10, a few search rounds find a working shared batch size without hurting accuracy.","key_machinery":"The carrying mechanism is the joint randomized binary-search protocol: the server broadcasts the current batch-size interval, each client samples one value from it, trains one local step, and reports success or failure, and the server shrinks the interval by taking the maximum of the clients' lower bounds and the minimum of their upper bounds. A fraction $f$ of clients train with the current safe minimum during the search, so no round is entirely wasted while the interval is being narrowed. This parallelizes a binary search over batch sizes across the federation and makes the search a side effect of normal federated training.","core_discovery":"RASBA identifies the maximum batch size that every client in a federation can physically support, without clients disclosing their hardware and without the federation failing on an out-of-memory error. In each round, every client samples a batch size uniformly from the current shared interval $[b_{\\min}, b_{\\max}]$; a successful local update makes the sampled value the client's new lower bound, while an out-of-memory error makes it a new upper bound. The server then sets $b_{\\min}$ to the maximum of the reported lower bounds and $b_{\\max}$ to the minimum of the reported upper bounds, narrowing the interval until all clients converge on one common value. In the reported experiments the adaptive run finishes about 7.8x faster than batch size 4 on MNIST and about 4.8x faster on CIFAR10, with final test accuracies of 96.09% and 73.42%, close to the best fixed batch sizes.","pith_inferences":["The initial cap $b_{\\max}=64$ means the search only explores a preselected range; if the true hardware limit lies above 64, the reported 'maximal' batch size is a property of the cap, not of the hardware.","On heterogeneous hardware the lowest-common-denominator choice underutilizes faster clients, so grouping clients by their reported limits and running one search per tier would likely recover additional speedup; the paper notes this direction as future work.","The experiments run on a single GPU model, so the search's convergence time under genuinely distributed or memory-diverse clients remains untested; a multi-GPU test with varying VRAM sizes would show how the protocol scales."],"forward_implications":["A federation can start with a conservative default and automatically enlarge its shared batch size until some client's hardware objects, which removes the need for a manual hardware survey.","Because the search lives entirely in the local training step and the server-side bound aggregation, it can be combined with any aggregation strategy or client-side training procedure.","Researchers running federated simulations on a shared GPU can skip the common pre-experiment step of testing each possible batch size by hand.","The reported 15x speedup on MNIST between batch size 4 and 256 shows that the batch size is a first-order factor in total training time, so finding the feasible maximum is worth several search rounds.","The shared batch size equals the lowest common denominator of all clients, which keeps every client in the federation but can leave faster hardware partially idle."],"supporting_citations":[{"why":"Establishes the federated learning formulation and aggregation setup the search runs inside.","marker":"[1]"},{"why":"Motivates large batch sizes as a route to faster training, which the method exploits.","marker":"[4]"},{"why":"Documents accuracy benefits of small batch sizes that justify capping the search range.","marker":"[6]"},{"why":"Adds evidence on small-batch training stability used to set the initial upper bound.","marker":"[7]"},{"why":"Supplies the federated learning simulation framework used in the experiments.","marker":"[15]"},{"why":"Provides the MNIST benchmark dataset for the first experiment.","marker":"[17]"},{"why":"Provides the CIFAR10 benchmark dataset for the second experiment.","marker":"[19]"},{"why":"Specifies the Dirichlet split that creates the non-IID client datasets in the experiments.","marker":"[20]"}],"fun_headline_variants":["Adaptive batch-size search speeds federated learning by 7.8x","RASBA finds the max safe shared batch size in federated learning","Greedy search tunes batch size: 7.8x faster FL, near-peak accuracy","No hardware leaks: FL batch-size search speeds up 7.8x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The search is capped by a hand-chosen initial upper bound $b_{\\max}=64$, so the claim that it finds the maximum physically supported shared batch size holds only if the true maximum is at or below that cap.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive batch-size search speeds federated learning by 7.8x","RASBA finds the max safe shared batch size in federated learning","Greedy search tunes batch size: 7.8x faster FL, near-peak accuracy","No hardware leaks: FL batch-size search speeds up 7.8x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000578,"raw_usage":{"total_tokens":2687,"prompt_tokens":870,"completion_tokens":1817,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":1742}},"tokens_in":486,"tokens_out":1817,"duration_ms":14262,"temperature":1.0,"reasoning_tokens":1742,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:47:12.757495+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same federations on the same GPU with $b_{\\max}$ initialized to 256 instead of 64 and record whether the search ever converges to a shared batch size above 64 with faster wall-clock completion; if it does, the capped configuration is not finding the physical maximum the method claims to find.","supporting_citations":[{"cited_title":"Control batch size and learning rate to generalize well: Theoretical and empirical evidence,","cited_arxiv_id":null,"evidence_quote":"Documents accuracy benefits of small batch sizes that justify capping the search range."},{"cited_title":"The mnist database of handwritten digit images for machine learning research [best of the web],","cited_arxiv_id":null,"evidence_quote":"Provides the MNIST benchmark dataset for the first experiment."}],"review_version":1}