{"id":"1d375dd6-834c-4554-807b-e89114cdbdf3","arxiv_id":"2506.11391","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Combining conformal risk control with an order-statistic delay bound, the paper proposes fixed and channel-adaptive model selection for wireless edge inference that claims guaranteed loss and deadline violation probability.","lead":"An edge-AI framework selects which image compression setting and classifier to run so that both accuracy and deadline guarantees hold, using conformal prediction on black-box models. It targets 6G applications such as automated driving, where inference must be reliable and delivered within milliseconds.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The latency guarantee is not established: Lemma 3's rank-uniformity holds only marginally over the random calibration set, but Algorithm 1 fixes U and selects the model minimizing the same U-based bound; the selected bound need not control the true violation probability.","rationale":"The conformal risk-control half and Lemma 2's channel lower bound appear internally coherent; the issue is specifically the deadline-violation bound. Lemma 3's rank-uniformity claim is only a marginal statement over the random calibration set, not a conditional statement for a realized set, and the algorithms require the conditional statement because they fix U and then select on the resulting bound. This is load-bearing because Proposition 1 is the only formal support for constraint (6c) in both the fixed and dynamic schemes. The paper's own Section III note that learn-then-test would be needed for joint model/threshold search confirms the gap; no such correction is applied in Algorithms 1 and 2. No code or formal verification is provided, so the numerical ImageNet results cannot substitute for the missing proof. The reader's weakest assumption identifies the same Lemma 3 issue, and the selection effect reinforces it. Therefore the verdict REJECT stands, with the loss half potentially salvageable but the advertised end-to-end strict guarantee unsupported.","tokens_in":20503,"tokens_out":13483,"duration_ms":171525,"concrete_test":"Simulate Algorithm 1 with a known data-size distribution (e.g., independent Bernoulli with P(D=0)=P(D=1)=1/2), the paper's channel model, NU=1000 calibration samples, and a large fresh test set. For each of 1000 independent draws of U, compute the Proposition 1 bound for the model Algorithm 1 selects and the exact true Pr(Ttot > T) for that selected model. If the bound is below the true probability in more than a nominal fraction of draws (e.g., >1%), Lemma 3 is not a valid conditional bound and the selection step invalidates the claimed beta guarantee.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim of strict statistical guarantees on latency rests on Proposition 1, whose proof depends on Lemma 3 (Appendix C). Lemma 3 states Pr(Dul <= D_ul,(n), Ddl <= D_dl,(m) | gl,k) >= (n+m)/(NU+1)-1, using the conformal rank identity Pr(D <= D_(n)) = n/(NU+1). This identity is correct only when the probability averages over both the new sample X and the random calibration set U. For the realized U used in Algorithms 1 and 2 it is generally false: conditional on U, Pr(D <= D_(n)|U) = F(D_(n)), which is unknown and can be arbitrarily far from n/(NU+1). Proposition 1 and Algorithm 1 nevertheless treat U as fixed when computing the bound, and then select the model g* that minimizes that same U-dependent bound over L x K candidates. Optimizing a bound on the calibration data is an in-sample selection: the returned bound is a minimum over correlated estimates and need not be an upper bound on the true Pr(Ttot > T) of the selected model. The paper explicitly defers learn-then-test for joint model/threshold search (Section III), but Algorithms 1 and 2 need exactly that correction. Proposition 2 inherits the same flaw through Lemma 3, so the dynamic scheme is affected as well. Thus the deadline-violation half of the central claim is unsupported; the numerical ImageNet split shows empirical behavior but does not supply the missing conditional or selection-adjusted guarantee.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper considers wireless edge inference where a sensor transmits an encoded observation to an edge server, which runs a black-box classifier and sends a prediction set back within a frame deadline. It proposes to select, from L encoder/decoder pairs and K inference models, the combination that minimizes the prediction-set size subject to an expected-loss constraint, handled by conformal risk control, and a deadline-violation constraint, handled by order-statistic bounds on uplink and downlink message sizes. Two schemes are presented: a fixed offline selection and a channel-adaptive dynamic selection, the latter with a variant allowing prediction-set truncation. The numerical evaluation uses ImageNet with WebP compression and EfficientNetV2 classifier models.","tokens_in":20737,"tokens_out":16888,"duration_ms":205324,"significance":"If valid, the framework would be a useful contribution to 6G edge-AI literature: it is model-agnostic, handles variable compression rates and random channel conditions, and the numerical setup is realistic. The conformal risk-control step for a single pre-specified composite model is standard and correctly stated, and the use of separate labeled and unlabeled calibration sets is a sensible design. However, the paper's main novelty is the deadline-violation bound, and that bound is not mathematically valid as stated. The flaws are load-bearing: Proposition 1 and Proposition 2 fail, and the selection algorithms do not preserve the conformal guarantee. The contribution therefore reduces to an empirical study accompanied by a guarantee that is not established.","major_comments":[{"comment":"The rank-uniform identity used in Lemma 3 is false conditional on the realized calibration set. The statement Pr(Dul,t ≤ \\bar D_ul(n) | gl,k) = n/(NU+1) holds only when the probability is taken jointly over the new sample X and the random calibration set U; for the fixed U used in Proposition 1, the conditional probability is F_{gl,k}(\\bar D_ul(n)), which is unknown and need not equal n/(NU+1). Because Algorithm 1 computes the Proposition 1 bound for a particular U and then selects the model minimizing that U-dependent bound, the returned \\bar P_{l,k} is not a valid upper bound on Pr(Ttot,t > T | gl,k). Proposition 2 uses the same Lemma 3, so the dynamic scheme inherits the error. Consequently, the deadline constraint in Eq. (6c) is unproven for both schemes.","section":"Appendix C, Lemma 3; Proposition 1"},{"comment":"Even setting Lemma 3 aside, the selection rules in Algorithms 1 and 2 do not preserve the conformal guarantee. The thresholds λ_{l,k} are computed from the labeled set D, and the same thresholds enter the U-based delay bounds and prediction-set-size estimates used for selection; the selected model is therefore a function of D as well as U. Lemma 1 is a marginal guarantee for a pre-specified model and does not automatically hold for a model chosen by a D-dependent rule. Section III explicitly defers learn-then-test for joint model/threshold search, but that is precisely the correction needed here. Without a selection-adjusted argument, the expected-loss constraint in Eq. (6b) is also not guaranteed for the returned model.","section":"Algorithms 1 and 2; Section III"},{"comment":"Lemma 4 is not valid when the uplink transmission consumes the entire available time. The chosen maximizer φ = T − τul,l − Dul,t/Rul,t lies outside the feasible interval [τfk, T − τul,l] whenever Dul,t/Rul,t ≥ T − τul,l − τfk; in that regime the conditional probability of meeting the deadline is 0, while the exponential lower bound in Lemma 4 exceeds 1. Proposition 2 then reports 1 − e^{\\hat β}(...) which can be negative, and a negative number is not a valid upper bound on a probability. This affects the dynamic scheme at low SNR.","section":"Appendix D, Lemma 4; Proposition 2"}],"minor_comments":[{"comment":"The displayed derivative of 2^{D/(B(T−φ−τ))} is not correct: it should contain a factor ln(2), not log2(Dul,t) or log2(Ddl,t). Since the derivative is used only as motivation and not in the final bound, this appears to be a typographical error, but it should be corrected.","section":"Appendix B, Eq. (22)"},{"comment":"In the paragraph after Eq. (21), the sentence stating that 'Tdl,t is independent of Ddl,t' should presumably say that Tdl,t is independent of Dul,t; as written it is inconsistent with the conditioning used in the expansion.","section":"Appendix B, proof of Lemma 2"},{"comment":"The text says the proposed schemes achieve the smallest possible deadline violation probability at low SNRs, but this is only an empirical observation. It does not compensate for the missing mathematical guarantee, and the caption/abstract should not imply a strict bound in the regime where no candidate satisfies the requirement.","section":"Section VI-B, Fig. 2(b)"}],"recommendation":"reject","confidential_remarks":"The main theoretical contribution is invalid: the order-statistic delay bound is false for a fixed calibration set, and the selection rules use the same data for bound construction and model choice. The loss guarantee is also not established after selection. Repairing these issues would require a substantially different framework, such as split-conformal calibration with a learn-then-test selection adjustment, well beyond a local revision. I recommend rejection of the current version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou should know: the novel part is real, but the latency guarantee doesn't hold. The paper combines conformal risk control with a non-parametric delay-violation bound to select among black-box encoder/inference model pairs. The loss side is standard conformal risk control, correctly applied with a separate labeled calibration set. The delay side is where it breaks.\n\nThe core issue is Lemma 3 in Appendix C. The proof claims that for a fixed calibration set U, a new sample's rank is uniform, so Pr(D ≤ D_(n)) = n/(NU+1). That identity is only valid jointly over the random calibration set and the new sample. Conditioned on the realized U, the probability is the unknown CDF evaluated at the order statistic, and can be arbitrarily far from the claimed value. Both Proposition 1 and Proposition 2 depend on Lemma 3, so the deadline-violation bound is not valid for the U actually used in Algorithms 1 and 2.\n\nWorse, the algorithms select the model that minimizes this U-dependent bound over L×K candidates. That's in-sample selection: the minimum of correlated bound estimates is not an upper bound on the selected model's true violation probability. The paper mentions learn-then-test as future work, but that is precisely the correction needed.\n\nThe numerical results are encouraging on one ImageNet split, but they don't supply the missing conditional or selection-adjusted guarantee. So the central claim of 'strict statistical guarantees' on latency is unsupported. The loss-guarantee half is solid.\n\nCredit where due: the problem is well motivated, the system model is clean, the fixed/dynamic schemes are sensible, and the handling of the conditional loss bound via ε = α(1−β) is correct. The paper is readable and the experiments are honest about the low-SNR regime where no model meets the requirement.\n\nThis deserves a serious referee, not because it's correct, but because the flaw is subtle and the idea is worth salvaging. The fix is clear: either state the bound as marginal over U and avoid selecting on U, or apply a multiple-testing correction. With that, the framework could be a real contribution. I would not cite this version.\n\nRecommendation: send to review, expect heavy revision.","headline":"A nice integration of conformal risk control and wireless delay modeling, but the latency guarantee rests on a conditional/marginal confusion in the order-statistic lemma, and the model selection makes it worse.","tokens_in":21323,"tokens_out":3182,"would_cite":false,"duration_ms":34135,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Black-box edge AI model selection can be certified to meet both expected-loss and deadline-violation requirements.","keywords":["edge AI","conformal risk control","model selection","deadline guarantee","wireless channel","6G","order statistics","image classification"],"falsifier":"Take a fixed unlabeled calibration set, compute Proposition 1's bound for a chosen model, then measure the empirical deadline violation probability on a large fresh set of independent samples; if the measured rate systematically exceeds the bound, the fixed-set application of the rank-uniform argument fails.","tokens_in":20215,"feed_emoji":"📡","tokens_out":9017,"duration_ms":94229,"temperature":0.7,"pith_summary":"This paper proposes a way to choose, from a collection of black-box compression and inference models, the combination that meets two hard requirements for wireless edge AI: the expected loss of the returned prediction set, conditioned on meeting the deadline, is at most $\\alpha$, and the probability of missing the deadline is at most $\\beta$. The selection is made without knowing the data distribution or model internals, using a labeled calibration set for the loss guarantee and an unlabeled calibration set for the delay guarantee. The paper presents a fixed scheme that picks one encoder/decoder and one edge model from channel statistics, and a dynamic scheme that re-selects the edge model after observing the instantaneous uplink rate. If correct, this gives deadline-constrained applications such as automated driving and robotics a principled way to trade compression quality against model complexity while keeping formal end-to-end guarantees.","feed_headline":"Provable loss and deadline guarantees for black-box edge AI","feed_subtitle":"Conformal risk control and an order-statistic delay bound certify each model combination before deployment.","key_machinery":"The central objects are composite black-box models $g_{l,k}$ that chain one encoder/decoder pair to one inference model; each combination is calibrated independently because the intermediate representations differ across encoder/decoder pairs. The loss guarantee comes from the conformal risk control threshold of Lemma 1, which makes the empirical loss on the labeled calibration set small enough to compensate for finite calibration size. The deadline guarantee comes from Proposition 1, which combines a Rayleigh-fading conditional success probability with a Boole's-inequality lower bound $\\frac{n+m}{N_U+1}-1$ on the joint probability that a new sample's uplink and downlink message sizes fall below order statistics of the unlabeled calibration set. The dynamic scheme uses Proposition 2, the same bound conditioned on the observed uplink rate, to re-pick the edge model at inference time.","core_discovery":"For each composite black-box model $g_{l,k}=f_k(d_l(e_l(X)))$ that chains an encoder/decoder pair to an inference model, the paper calibrates a threshold $\\lambda_{l,k}$ so that the expected loss satisfies $\\mathbb{E}[\\ell(\\Gamma(X),Y)]/(1-\\beta)\\le\\alpha$, which implies the conditional requirement $\\mathbb{E}[\\ell(\\Gamma(X),Y)\\mid T_{\\mathrm{tot}}\\le T]\\le\\alpha$. It then proves an upper bound on the deadline violation probability from the order statistics of the uplink and downlink message sizes on an unlabeled calibration set, giving $\\Pr(T_{\\mathrm{tot}}>T)\\le\\min_{n,m}(1-e^{\\bar\\beta_{\\mathrm{cal}}(l,k,n,m)}(\\frac{n+m}{N_U+1}-1))$. The fixed scheme selects the model combination with the smallest estimated prediction-set size among those satisfying the bound, and the dynamic scheme conditions the same bound on the observed uplink rate and re-selects the edge model at inference time. Numerical results on an ImageNet classification task with WebP compression and EfficientNetV2 classifiers show the guarantees met where feasible, with prediction-set size decreasing as SNR increases.","pith_inferences":["I read Lemma 3 as a joint statement over the random calibration set and the new sample; the algorithms fix $U$ in advance, so strictly the deadline bound is an approximation for any particular calibration set. A resampling or jackknife correction over $U$ would turn the guarantee into a conditional one, but that step is not in the paper.","The paper's own numerical results note that the delay bounds are conservative because they treat uplink and downlink delays as correlated; an independence-aware bound could select larger models at high SNR, where the reported violation probability sits well below $\\beta$.","Because the method treats each composite model as a black box, the same selection procedure extends to any encoder or feature extractor and any inference model that emits confidence scores, including models with variable-length outputs beyond the ImageNet classification setting."],"forward_implications":["An operator can certify before deployment, without any white-box knowledge, whether a given model combination meets both the loss and deadline requirements; the certificate is computed offline from calibration data and channel statistics.","The dynamic scheme keeps the conformal thresholds precomputed offline, so the only online work is evaluating the delay bound and comparing the $K$ edge models against the observed uplink rate.","Because the loss guarantee applies to any loss that cannot increase when the prediction set is enlarged, the same framework covers misclassification probability and other risk measures such as false negative rate.","When no available combination meets the deadline requirement, the procedures still return the combination with the smallest provable violation probability, providing graceful degradation at low SNR.","At higher SNR the schemes select larger models and produce smaller prediction sets, turning favorable channel conditions directly into more informative predictions."],"supporting_citations":[{"why":"Supplies the conformal risk control procedure used in Lemma 1 to calibrate thresholds and give the expected-loss guarantee.","marker":"[8]"},{"why":"Provides the conformal prediction foundation and the rank-uniformity fact used in Lemma 3 for the order-statistic delay bound.","marker":"[7]"},{"why":"Supplies the ImageNet 2012 dataset used to calibrate and evaluate the image-classification scenario.","marker":"[31]"},{"why":"Defines WebP, which implements the encoder/decoder pairs in the numerical experiments.","marker":"[32]"},{"why":"Provides the EfficientNetV2 classifier models used as the black-box inference models with the reported computation times.","marker":"[33]"}],"fun_headline_variants":["Conformal guarantees for real-time edge AI model selection","Black-box edge AI: provable latency and loss guarantees","Selecting edge AI models with certified deadline and accuracy","Certified model choice for 6G edge inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The deadline guarantee depends on the unlabeled calibration set being a random draw at the moment the bound is used; the algorithms, however, use a single fixed set, so the numerical certificate is only as reliable as that random-draw assumption.","fun_headline_variants_meta":{"raw":{"variants":["Conformal guarantees for real-time edge AI model selection","Black-box edge AI: provable latency and loss guarantees","Selecting edge AI models with certified deadline and accuracy","Certified model choice for 6G edge inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1376,"prompt_tokens":1001,"completion_tokens":375,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":311}},"tokens_in":617,"tokens_out":375,"duration_ms":3997,"temperature":1.0,"reasoning_tokens":311,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:11:52.764885+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed unlabeled calibration set, compute Proposition 1's bound for a chosen model, then measure the empirical deadline violation probability on a large fresh set of independent samples; if the measured rate systematically exceeds the bound, the fixed-set application of the rank-uniform argument fails.","supporting_citations":[{"cited_title":"Conformal risk control,","cited_arxiv_id":null,"evidence_quote":"Supplies the conformal risk control procedure used in Lemma 1 to calibrate thresholds and give the expected-loss guarantee."},{"cited_title":"Conformal prediction: A gentle introduction,","cited_arxiv_id":null,"evidence_quote":"Provides the conformal prediction foundation and the rank-uniformity fact used in Lemma 3 for the order-statistic delay bound."},{"cited_title":"ImageNet large scale visual recognition chal- lenge,","cited_arxiv_id":null,"evidence_quote":"Supplies the ImageNet 2012 dataset used to calibrate and evaluate the image-classification scenario."},{"cited_title":"WebP image format,","cited_arxiv_id":null,"evidence_quote":"Defines WebP, which implements the encoder/decoder pairs in the numerical experiments."},{"cited_title":"EfficientNetV2: Smaller models and faster training,","cited_arxiv_id":null,"evidence_quote":"Provides the EfficientNetV2 classifier models used as the black-box inference models with the reported computation times."}],"review_version":1}