{"id":"f3522178-e2df-44b2-aa4f-98e2bc5118a6","arxiv_id":"2507.14802","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ACME customizes Transformer models through a hierarchical cloud-edge-device loop, reducing uploaded data to 6% of centralized systems and improving accuracy by about 10% on CIFAR-100 and Stanford Cars.","lead":"ACME is a distributed method that customizes Transformer models by splitting them into a shared backbone and a personalized header, tuned across cloud, edge, and device layers. It reports cutting data uploads to 6% of centralized customization while improving accuracy by about 10% on vision benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 6% data-transmission claim likely omits ACME's required shared-data copies and iterative importance-set uploads; the real transmission advantage is unestablished.","rationale":"The paper's central value proposition is that ACME customizes models in a distributed system with less data transfer and better cost-accuracy trade-offs. The '6%' figure is a crisp, checkable headline, and it appears contradicted by the protocol's own data requirements. This is an internal accounting issue rather than a matter of disagreement with external consensus. The reader's energy-model objection is valid and should remain a condition for acceptance, but I regard the transmission-accounting gap as the more load-bearing concern because it directly targets a quantified headline claim and is falsifiable from the text alone. A revised paper must either include the shared-data or feature transfer in Table I, or define it as pre-existing and reconcile that definition with the privacy assertions in Section III-D. The system may still be viable, so I do not move the overall verdict; it remains conditional pending recomputation of the transmission volume and release of the underlying accounting details.","tokens_in":16802,"tokens_out":13547,"duration_ms":165635,"concrete_test":"Enumerate all communication in the ACME protocol from Algorithm 1 and Algorithm 2: (i) statistical parameters from edge to cloud; (ii) the shared 10-20% data subset per device (or the extracted features) used by Eq. (19); (iii) T iterations of importance-set uploads Q_n and downloads Q'_n; (iv) backbone and header downloads. Sum these into 'ACME total bytes' and compare with a CS baseline that uploads full local datasets to the cloud. Recompute Table I for N=10,20,30,40 under this accounting; if the ACME-to-CS ratio exceeds 6% by a meaningful margin (e.g., greater than 10%), the headline data-volume claim should be revised or explicitly justified as excluding the shared subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim that ACME reduces data transmission volume to 6% of a centralized system rests on Table I's 'Upload Data' column. That accounting appears incomplete. Section IV.A states that each edge server ss stores 10% to 20% of the data for its devices as a shared dataset, and Phase 2-1 uses this data for NAS. Phase 2-2 (Eqs. 19-21) computes Wasserstein distances between device data distributions using \\tilde{D_i}, a 'tiny portion of data randomly sampled from D_i' (or features extracted from it); this information must reside at the edge server. In addition, Algorithm 2 has every device upload its importance set Q_n at every iteration, for T iterations, and download Q'_n. None of these transfers are included in the 6% calculation. If the 10-20% shared subsets must be moved from devices to edge servers, the true ACME upload is at least roughly 10-20% of each local dataset plus T importance-vector exchanges; versus a CS that uploads full datasets, the ratio would likely be far above 6%. If the shared subsets are assumed to already exist at the edge, the statement 'we avoid uploading local data' (Section III-D) is contradicted and the privacy motivation is undermined. In either reading, the 'reduced to 6%' claim is not supported by the reported numbers.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"ACME proposes a hierarchical cloud-edge-device framework for customizing Transformer-based large models (instantiated with ViT) under heterogeneous device constraints. The method decomposes customization into two phases: (1) backbone generation and Pareto-front selection under model size and energy constraints (Phase 1, Eqs. (4), (10)-(13)), and (2) header customization via edge-side NAS followed by device-side importance-set refinement with Wasserstein-distance-based personalized aggregation (Phase 2, Eqs. (14)-(21)). The paper reports on CIFAR-100 and Stanford Cars that ACME reduces data transmission to 6% of a centralized system, improves average accuracy by about 10%, and improves a composite trade-off metric by nearly 30%.","tokens_in":17072,"tokens_out":3494,"duration_ms":46185,"significance":"If the quantitative claims were supported, ACME would be a practically useful contribution: it targets a real deployment problem, combines backbone and header customization in a distributed setting, and explicitly addresses device and data heterogeneity. The strengths are the clear decomposition of the optimization problem and the inclusion of two datasets and several lightweight-ViT baselines. However, the headline numbers currently rest on an unvalidated energy model, an incomplete communication-cost accounting, and an evaluation without error bars or the most closely related federated-NAS baselines. These issues are load-bearing for the central claims, so the current evidence is not yet sufficient to establish the paper's main conclusions.","major_comments":[{"comment":"The 'Trade-off Score' used to claim a 28.9% improvement is defined as Ln(θn, Dn) + En(θn) + ζ(θn), where En(θn) is exactly the paper's own approximate energy model from Eqs. (1)-(2). The same En is used as an optimization objective in P1 and Eq. (10). Therefore the reported trade-off improvement is largely a self-consistency result: the optimizer selects models that score well on the very metric being reported. To support the cost-efficiency claim, the energy model should be validated against hardware measurements (or at least against a held-out measured energy/latency dataset), and the trade-off comparison should be recomputed with measured rather than modeled energy.","section":"§IV.B.3 and Eq. (1)-(2)"},{"comment":"The claim that ACME reduces data transmission volume to 6% of a centralized system is not supported by the reported accounting. Section IV.A states that each edge server stores 10%-20% of its devices' data as a shared dataset, and Phase 2-1 uses this data at the edge. Phase 2-2 additionally requires each device to provide a sampled subset \\tilde{D}_i for Wasserstein-distance computation, and Algorithm 2 has every device upload its importance set Q_n at every iteration for T iterations and download Q'_n. None of these transfers appear in Table I's 'Upload Data' column. If the shared subsets must first be uploaded from devices, the true ACME upload volume is at least roughly 10%-20% of each local dataset plus T importance-set exchanges, which would likely be far above 6%. If the shared subsets are assumed to already reside at the edge, then the statement in §III-D that ACME 'avoid[s] uploading local data' is contradicted and the privacy motivation is weakened. The 6% claim needs a complete communication budget that includes all of these transfers, with the assumptions stated explicitly.","section":"§IV.B.1, Table I, and §IV.A"},{"comment":"The accuracy comparison omits the most relevant baselines from the paper's own related-work section. Federated NAS methods such as FedNAS and CFDNAS [42], [43] are cited in §V as existing approaches for model customization on heterogeneous devices, but they are not included in any experiment. Without these comparisons, the claim that ACME achieves better accuracy than existing distributed customization approaches is not established. The authors should either add these baselines (or a clear explanation of why they cannot be run) and report accuracy with multiple independent runs, standard deviations, or confidence intervals, since the current figures show single-point comparisons.","section":"§IV.B.2 and Fig. 7"},{"comment":"The energy model itself is an assumption that power and latency scale linearly with the number of Transformer layers and with w_B d_B, with constants proportional to base GPU power and latency. No empirical validation of this model is provided, and the experimental setup does not report any energy or latency measurements on the actual devices. Since this model underpins both the Pareto-front selection in Phase 1 and the reported trade-off improvements, its accuracy on real hardware is a correctness risk. A concrete test would be to measure energy and latency for a few selected architectures on the deployed VMs and compare them with Eq. (1)-(2) predictions.","section":"Eq. (2) and §IV.A"}],"minor_comments":[{"comment":"The notation is ambiguous: S denotes both the set of edge servers and its cardinality, and N denotes both the set of devices and its size; Eq. (3) uses 1/S and 1/|N_s| without making these distinctions explicit.","section":"§II.A and Eq. (3)"},{"comment":"The phrase 'importance of headers' in the text should read 'importance of heads'; the paper elsewhere uses 'header' for the classification module, so the terminology should be consistent.","section":"§III.B.1"},{"comment":"The formula for K on line 6 references f^l_s(\\tilde{\\theta}^*_s) and f^l_s(\\tilde{\\theta}^-_s), but these ideal and worst-case points are not explicitly defined before this line; their definitions should be stated.","section":"Algorithm 1"},{"comment":"The Trade-off Score is defined as a sum of loss, energy, and model size, so lower values are presumably better, but the text says 'improves the final Trade-off Score by at least 28.9%' without specifying the direction; please state whether lower is better and clarify the sign convention.","section":"§IV.B.3"},{"comment":"The paper reports 'vCPUs configured from 3 to 7' and storage capacities in MB, but it does not specify the GPU configuration of the edge servers and devices; since energy and latency are central to the method, the hardware setup should be described in more detail.","section":"§IV.A"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid architectural idea but the current version presents headline numbers (6% transmission, 10% accuracy, 30% trade-off) that are not yet substantiated. The energy-model circularity and the incomplete communication accounting are fixable in principle, but they require additional experiments or a major rewriting of the claims. The lack of comparison with FedNAS/CFDNAS and the absence of error bars further weaken the empirical case. I would not recommend rejection, but the revision must address these points with concrete measurements or explicitly restricted claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a coherent system proposal for customizing ViT models across cloud, edge, and devices, and the writing is clear. The novel part is the three-layer loop: cloud handles backbone generation and Pareto-front selection, edge servers handle NAS-based header search, and devices refine headers locally with importance sets that get aggregated using Wasserstein-distance weighting. That decomposition is sensible and the motivation is real.\n\nThe main problems are in the evaluation, and they are serious enough that the headline claims should not be taken at face value. The 6% data-transmission number in Table I simply omits the \"shared dataset\" that each edge server stores (10-20% of device data, Section IV.A) and the per-iteration importance-set uploads in Algorithm 2. If that shared data has to come from devices, the true upload is larger than what ACME reports, possibly close to or above the centralized baseline. If it is already at the edge, then the \"we avoid uploading local data\" statement is misleading. Either way, the 6% claim is not backed by the numbers.\n\nThe trade-off metric is also suspiciously self-referential: the energy model in Eq. (1)-(2) is used both to optimize the architecture and to compute the reported 30% improvement. The model is a linear approximation and is not validated on the simulated hardware. There are no error bars, no multiple runs, and no code or data release, so the accuracy improvements look like single points. The related work section cites FedNAS and CFDNAS as relevant federated NAS methods, but the experiments compare against neither, which makes the claim of superiority over distributed NAS unsubstantiated.\n\nOn the positive side, the system design is thoughtful and the paper gives enough detail to reproduce the method conceptually. The backbone/header split is a useful way to separate device-attribute matching from data matching, and the use of Wasserstein distance for aggregation is a reasonable choice that the small ablation supports.\n\nThis is a paper that deserves a referee, but it needs major revision before acceptance. The authors should redo the transmission accounting, validate or calibrate the energy model, add the missing federated NAS baselines, and report variance. If those are fixed, the paper could be a solid systems contribution.","headline":"Plausible system design for distributed model customization, but the headline 6% transmission claim is not supported by the paper's own accounting.","tokens_in":17643,"tokens_out":2073,"would_cite":false,"duration_ms":26214,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ACME's central claim is that a cloud–edge–device pipeline can customize Transformer models per device, cutting data transmission to 6% while improving trade-off metrics by nearly 30%.","keywords":["distributed model customization","Transformer","edge computing","neural architecture search","Pareto front","personalized aggregation","Wasserstein distance","model deployment"],"falsifier":"Measure, on the same GPU type used in the evaluation, the actual power draw and per-epoch latency of ViT backbones across a grid of widths and depths, and compare these measurements with Eq. (1)–(2). If the measured values do not follow the linear scaling assumed there, then the Pareto-front selections made by ACME and the nearly 30% trade-off improvement, both computed from that model, would need to be re-derived.","tokens_in":16558,"feed_emoji":"📡","tokens_out":7163,"duration_ms":78380,"temperature":0.7,"pith_summary":"ACME is a distributed system that customizes pre-trained Transformer models for individual edge devices by splitting the work across a cloud server, edge servers, and the devices themselves. The paper's central claim is that this division makes model customization cost-efficient: compared with centralized customization, it reduces data transmission to about 6%, raises average accuracy by about 10% over baseline lightweight models, and improves a combined accuracy–energy–size trade-off metric by nearly 30%. The system first matches a backbone architecture to each device cluster's storage and compute limits using a Pareto-front search, then designs and refines a task-specific header using neural architecture search and data-distribution-aware aggregation. If correct, ACME would let privacy-sensitive applications deploy large models on heterogeneous local hardware without uploading local data.","feed_headline":"Edge model tuning cuts uploads to 6% and lifts accuracy 10%","feed_subtitle":"ACME splits backbone and header design across cloud, edge, and devices so local data never leaves the device.","key_machinery":"The load-bearing machinery is the bidirectional single-loop distributed system: a cloud layer exchanges statistical parameters with edge servers to generate and select backbone architectures, and each edge server runs an iterative loop with its devices to refine header architectures. Within that system, three components carry the argument: (1) a grid-based Pareto-front construction (borrowed from constrained decomposition) that maps each device cluster's storage and energy limits to a small set of candidate width–depth pairs; (2) a progressive neural architecture search with an LSTM controller that generates a coarse header architecture for each backbone; and (3) an importance-set mechanism that quantifies each header parameter's contribution via a first-order Taylor expansion and aggregates importance sets across devices with weights derived from Wasserstein distances between local feature distributions.","core_discovery":"ACME's core claim is that a Transformer model can be customized for a heterogeneous device fleet without paying the usual communication and search costs of centralized customization. The paper argues that by treating the backbone (the layer stack that extracts general features) and the header (the task-specific output network) as separately optimizable pieces, and by running their optimization at different levels of a cloud–edge–device hierarchy, one can hit a better accuracy–energy–size trade-off than any single centralized process. In experiments with ViT-B on CIFAR-100, ACME reports a 10% accuracy gain over lightweight baselines at similar model sizes, a 6% data-transmission volume relative to centralized systems, and a nearly 30% improvement in the combined trade-off metric.","pith_inferences":["Because the reported trade-off gains are computed with the same approximate energy model used to select the Pareto front, a direct measurement of GPU power and latency across the width–depth grid could show whether the ~30% improvement is preserved under real hardware scaling.","The communication-saving design could be reused outside Transformer customization: any federated system that can summarize local knowledge as importance vectors or gradients rather than data might achieve similar reductions.","The Wasserstein similarity step assumes that a pre-trained feature extractor makes local distributions comparable; testing ACME with different feature extractors or distribution distances would reveal whether this assumption is essential.","The efficiency gains may depend on the fraction of data each edge server holds as a shared dataset (10–20%); ablating that fraction could show a privacy–performance trade-off not explored in the paper."],"forward_implications":["Privacy-sensitive edge deployments become feasible because raw local data never leaves devices, so models can be customized on data that cannot be uploaded to the cloud.","Communication savings scale with the number of clusters: the 6% data-transmission figure follows from uploading only statistical parameters and importance sets rather than local datasets.","After the Pareto-front grid is built, selecting a model for a device cluster is fast, cutting selection latency by about 71% compared with greedy methods.","The method transfers to other datasets and Transformer variants: on Stanford Cars, ACME reports an average accuracy improvement of 3.94% under the same storage constraints, and 14.43% on header redesign.","The two-phase decomposition lets simple backbones pair with complex headers and vice versa, improving accuracy on both small and large backbone architectures."],"supporting_citations":[{"why":"Supplies the energy model (power and latency scale with layer count and width–depth product) used to evaluate cost-efficiency and select models.","marker":"[15]"},{"why":"Provides the dynamic width–depth backbone reduction idea (DynaBERT) that ACME adapts for backbone generation via importance-based pruning.","marker":"[17]"},{"why":"Gives the constrained decomposition with grids method that ACME uses to construct the Pareto-front grid for backbone selection.","marker":"[18]"},{"why":"Introduces progressive neural architecture search, the basis for ACME's block-level header search space.","marker":"[20]"},{"why":"Supplies the LSTM controller and search strategy used by the edge server to generate coarse header architectures.","marker":"[23]"},{"why":"Provides the parameter-sharing NAS scheme in federated edge systems that ACME's header search builds on.","marker":"[24]"},{"why":"Source of the Taylor-expansion importance estimation used to score and prune header parameters on each device.","marker":"[27]"},{"why":"Supplies the Wasserstein-distance-based similarity computation used to aggregate importance sets across devices.","marker":"[28]"},{"why":"Provides the four traditional header architectures used as baselines in the accuracy comparison with NAS-generated headers.","marker":"[37]"}],"fun_headline_variants":["ACME: distributed tuning cuts uploads to 6%, lifts accuracy 10%","Split backbone and header: 6% data, 10% accuracy, 30% trade-off gain","Distributed ACME: Pareto-optimal models with 6% data, 10% accuracy","ACME's cloud-edge-device tuning reduces data to 6%, improves accuracy 10%","Edge tuning with ACME: 6% uploads, 10% accuracy, 30% better trade-off"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The cost-efficiency and Pareto-front claims rest on the assumption that a model's power consumption and per-epoch latency scale linearly with its number of Transformer layers and its width–depth product; if real hardware follows a different scaling law, the models ACME selects may not be the most cost-efficient and the reported trade-off improvements would need to be recomputed.","fun_headline_variants_meta":{"raw":{"variants":["ACME: distributed tuning cuts uploads to 6%, lifts accuracy 10%","Split backbone and header: 6% data, 10% accuracy, 30% trade-off gain","Distributed ACME: Pareto-optimal models with 6% data, 10% accuracy","ACME's cloud-edge-device tuning reduces data to 6%, improves accuracy 10%","Edge tuning with ACME: 6% uploads, 10% accuracy, 30% better trade-off"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000471,"raw_usage":{"total_tokens":2345,"prompt_tokens":948,"completion_tokens":1397,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":1272}},"tokens_in":564,"tokens_out":1397,"duration_ms":13325,"temperature":1.0,"reasoning_tokens":1272,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:48:04.128745+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on the same GPU type used in the evaluation, the actual power draw and per-epoch latency of ViT backbones across a grid of widths and depths, and compare these measurements with Eq. (1)–(2). If the measured values do not follow the linear scaling assumed there, then the Pareto-front selections made by ACME and the nearly 30% trade-off improvement, both computed from that model, would need to be re-derived.","supporting_citations":[{"cited_title":"Energy-efficient inference ser- vice of transformer-based deep learning models on gpus,","cited_arxiv_id":null,"evidence_quote":"Supplies the energy model (power and latency scale with layer count and width–depth product) used to evaluate cost-efficiency and select models."},{"cited_title":"Dyn- aBERT: Dynamic bert with adaptive width and depth,","cited_arxiv_id":null,"evidence_quote":"Provides the dynamic width–depth backbone reduction idea (DynaBERT) that ACME adapts for backbone generation via importance-based pruning."},{"cited_title":"A constrained decomposition approach with grids for evolutionary multiobjective optimization,","cited_arxiv_id":null,"evidence_quote":"Gives the constrained decomposition with grids method that ACME uses to construct the Pareto-front grid for backbone selection."},{"cited_title":"Progressive neural architecture search,","cited_arxiv_id":null,"evidence_quote":"Introduces progressive neural architecture search, the basis for ACME's block-level header search space."},{"cited_title":"Learning transferable architectures for scalable image recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM controller and search strategy used by the edge server to generate coarse header architectures."},{"cited_title":"ENASFL: A federated neural architecture search scheme for heterogeneous deep models in distributed edge computing systems,","cited_arxiv_id":null,"evidence_quote":"Provides the parameter-sharing NAS scheme in federated edge systems that ACME's header search builds on."},{"cited_title":"Importance estimation for neural network pruning,","cited_arxiv_id":null,"evidence_quote":"Source of the Taylor-expansion importance estimation used to score and prune header parameters on each device."},{"cited_title":"Data valuation and detections in federated learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the Wasserstein-distance-based similarity computation used to aggregate importance sets across devices."},{"cited_title":"Multi-exit vision trans- former for dynamic inference,","cited_arxiv_id":null,"evidence_quote":"Provides the four traditional header architectures used as baselines in the accuracy comparison with NAS-generated headers."}],"review_version":1}