{"id":"fb439112-7746-4776-92cd-a27add9d4537","arxiv_id":"2507.23461","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RAF, resolution-adaptive federated learning, uses multi-resolution knowledge distillation to stop resolution differences between clients from degrading federated keypoint detection.","lead":"Federated learning models trained on private device data degrade when different devices send images at different resolutions, a problem the authors call resolution-drift. The proposed method, RAF, has each device also train on downsampled copies of its images and uses higher-resolution outputs as teachers so the shared model stays accurate at every image size.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The convergence proof's claimed local equivalence (Eqs. 13-14) is algebraically false: the gradient of the surrogate Eq. (11) at w_t does not equal that of Eq. (8), so Theorem 1's O(1/T) guarantee is unsupported.","rationale":"The paper's central contribution is the RAF method plus the claim that extensive experiments and theoretical analysis demonstrate its benefit. The theoretical component is explicitly cited in the abstract, so a broken convergence proof is load-bearing: it undercuts the claimed guarantee, even though the empirical results may still be valid. I re-derived the key local-equivalence step from the equations in the manuscript and found a definite algebraic error, not merely an unverified assumption. The reader's weakest_assumption already pointed at the surrogate inconsistency and the feature-converged regime; my analysis strengthens the former into a concrete falsehood that can be settled by recomputation. I also considered whether the lack of error bars or the ambiguity of the backbone comparison between baseline and RAF might be more central, but those are secondary: the local-equivalence error is deterministic, located in the proof, and directly invalidates Theorem 1. The empirical contribution remains plausible, and the flaw is fixable by correcting the quadratic expansion or by removing the theorem, so a conditional accept is still the appropriate verdict rather than a rejection.","tokens_in":18415,"tokens_out":12294,"duration_ms":123948,"concrete_test":"Recompute Eqs. (10)-(14) from Eq. (8) with a simple scalar analogue: take a fixed teacher b w_t, student a w, and upsampler identity. Let l(w) = (b w_t - a w)^2 and s(w) = w a (b - a) w. At w = w_t, l(w_t) = (b-a)^2 w_t^2 and s(w_t) = a(b-a) w_t^2; l'(w_t) = 2a(a-b) w_t and s'(w_t) = 2a(b-a) w_t. These agree only in degenerate cases. Then re-run the proof with the actual KD loss expanded as \\|B w_t - A w\\|^2; if the authors cannot derive Eqs. (13)-(14) from this expansion, Theorem 1 should be revised or removed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section IV.C, the proof of Theorem 1 depends on Eqs. (13)-(14), which assert that the linearized loss Lk (Eqs. 7-9) and the surrogate \\bar Lk (Eqs. 10-12) are locally equivalent at w = wt. This is not merely unverified; it is false. For one KD term, set A = U_i^{i-1}(\\psi^{(i)})^T and B = (\\psi^{(i-1)})^T. The original KD loss is \\|B w_t - A w\\|^2, so at w_t its gradient is 2 A^T(A - B) w_t, while the surrogate term w^T A^T(B - A) w has gradient [A^T(B-A) + (B-A)^T A] w_t. Equality of these two gradients would require A^T(B-A) to be symmetric, which is not generally true; the function values also differ: \\|(B-A) w_t\\|^2 vs w_t^T A^T(B-A) w_t. Thus the central step of the proof fails, and the O(1/T) convergence guarantee in Theorem 1 is unsupported. The smoothness constant in Proposition 1 is computed for the surrogate, not for the actual loss whose teacher term is frozen at w_t; Proposition 2 and the invocation of [43] therefore do not apply to RAF as run. This is an internal inconsistency, not a matter of consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies federated learning for human pose estimation when clients hold images of different resolutions. It defines 'resolution-drift' as the performance degradation observed when aggregating models trained on heterogeneous input resolutions, and proposes Resolution Adaptive Federated Learning (RAF). RAF augments standard FedAvg/FedProx local training with a multi-resolution knowledge-distillation loss: each client downsamples its own images to lower resolutions and trains its model so that low-resolution heatmap predictions match (detached) higher-resolution teacher predictions. The ViTPose backbone is modified with ResFormer-style convolution-based positional embeddings to accommodate variable input sizes. Experiments on MPII with simulated resolution-heterogeneous clients show consistent gains for RAF over the baselines at seen and unseen inference resolutions. Section IV.C provides a convergence analysis claiming an O(1/T) rate by invoking FedAvg's convergence theorem on a surrogate quadratic loss.","tokens_in":18701,"tokens_out":10512,"duration_ms":108163,"significance":"If the empirical results are reproducible, RAF is a practically valuable and modular contribution to FL for high-resolution regression: it requires no changes to server aggregation, works with both FedAvg and FedProx, and yields large gains at low and unseen resolutions in the reported MPII simulations. The resolution-drift axis is a useful reframing of non-IID data for non-classification tasks, and the paper's experiments on centralized versus federated settings help isolate the regularizing effect of distillation. The main weakness is the theoretical section, whose central equivalence is algebraically incorrect; for this reason the convergence guarantee, as stated, is not established. The empirical part, if separated from the flawed proof, is still a reasonable contribution.","major_comments":[{"comment":"The claimed local equivalence between L_k and \\bar L_k is false. For one KD term, write A = U_i^{i-1}(\\psi^{(i)})^T and B = (\\psi^{(i-1)})^T. The original term (8) is ||B w_t - A w||^2, whose value at w_t is ||(B-A)w_t||^2 and whose gradient is 2 A^T(A-B)w_t. The surrogate term (11) is w^T A^T(B-A)w, whose value at w_t is w_t^T A^T(B-A)w_t and whose gradient is [A^T(B-A)+(B-A)^T A]w_t. These quantities are equal only under special symmetry conditions on A^T(B-A), which do not hold in general. Equations (13) and (14) are therefore not valid. Propositions 1-3 are proven for \\bar L_k, and Theorem 1 does not apply to the loss actually minimized by RAF.","section":"IV.C, Eqs. (8), (11), (13), (14)"},{"comment":"The gradient formula stated for the KD term is incorrect for both losses under consideration. The gradient of (8) is 2A^T(A w - B w_t), not 2A^T(A-B)w, because the teacher is frozen at w_t; the gradient of (11) contains the extra symmetrized term (B-A)^T A. As written, Eq. (18) does not derive the smoothness constant of either objective. The constant may be recoverable for the surrogate, but the proof needs to be rewritten.","section":"IV.C, Proposition 1, Eq. (18)"},{"comment":"The analysis is premised on a 'feature-converged' regime in which only the last affine layer changes. The paper does not validate this assumption for the ViTPose-S models used in the experiments (e.g., by measuring relative updates of backbone versus head weights or checking that the linearized loss tracks the true loss). Since this is the setup for the convergence result, the assumption must either be checked or the theorem explicitly presented as conditional on it.","section":"IV.C, feature-converged assumption"}],"minor_comments":[{"comment":"No uncertainty quantification is reported; please provide means and standard deviations over multiple seeds, given the small per-client dataset size (4,000 images).","section":"V.A, Tables II-III and Figures 6, 8, 10"},{"comment":"The local update 'w_k ← w − η ∇_{w_k} L_k(w)' is ambiguous about whether w on the right-hand side is the global broadcast or the local iterate; please clarify the variables.","section":"Algorithm 1, line 15"},{"comment":"The entry 'Bilnear' should be 'Bilinear'; also, the claim that 57.2 is an 'inherent accuracy limit' should be justified or phrased as an empirical observation rather than a fundamental bound.","section":"Table III"},{"comment":"For the lowest-resolution client, the paper states that no further downsampling is possible, yet it attributes gains to RAF for this client; please clarify how MRKD operates for the lowest-resolution client and how its benefit arises through the global model.","section":"V.E and Figure 8c"},{"comment":"The heading 'Anlaysis' is a typo for 'Analysis'.","section":"V.G, heading a"},{"comment":"The generalizability claims to segmentation, depth estimation, and super-resolution are supported only by a t-SNE visualization on a Torchvision FCN; consider softening these claims or adding quantitative transfer experiments.","section":"Abstract and Section VI"}],"recommendation":"major_revision","confidential_remarks":"The paper is an honest empirical study whose main deficiency is the convergence proof, which contains a genuine algebraic error. If the authors can correct the proof or remove the unsupported theoretical claims, the empirical contribution is publishable. The use of the external MPII benchmark and standard ViTPose baselines is appropriate, and I saw no signs of data fabrication. Please ask the authors to address the theoretical issue head-on and to add error bars before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou should know two things before reading this one. First, the empirical core is genuinely useful: the paper names 'resolution-drift' as an axis of non-IID data in federated keypoint detection, and shows a simple local multi-resolution knowledge-distillation regularizer consistently improves over FedAvg and FedProx on MPII across seen and unseen resolutions. Second, the convergence proof, as written, does not hold together. The claims in Eqs. (13)-(14) that L_k and \\bar{L}_k are locally equivalent are algebraically false for the KD term: the gradient of the original loss with the teacher frozen at w_t is 2A^T(A-B)w_t, while the surrogate gives (A^T(B-A)+(B-A)^T A)w_t, which match only if A^T(B-A) is symmetric. Function values also differ. So Proposition 1's smoothness constant is computed for the surrogate, not for the loss RAF actually optimizes, and Theorem 1's O(1/T) guarantee is unsupported. The lazy-training assumption that the ViT backbone is feature-converged in late training is also unverified for the ViTPose models actually used.\n\nThat said, the paper deserves a serious referee, not a desk reject. The resolution-drift framing is a real problem statement, extending ResFormer-style convolution-based positional embeddings to high-resolution regression in FL is a sensible move, and the experiments are coherent: gains of 5+ points on low-resolution inference, benefits shown for low-, mid-, and high-resolution clients, and a clean comparison against both FedAvg and FedProx. The t-SNE analysis is suggestive rather than conclusive, but that does not carry the paper's weight.\n\nThe soft spots beyond the proof are addressable. No error bars or seeds are reported, and no code is released, so the empirical gains could be within run-to-run noise; the magnitudes are large enough that I suspect they are real, but I cannot verify. The client-benefit analysis is a useful practical check, though the comparison to centralized learning is not perfectly controlled since FL uses three times the data.\n\nWho should read this: people working on federated learning for non-classification vision tasks, and anyone deploying FL on heterogeneous camera feeds. My recommendation: send it to peer review with explicit instructions that the convergence analysis must be corrected or deleted. The empirical contribution stands on its own; the incorrect theory should not be the reason it is accepted, nor the reason it is rejected. If the authors fix the proof or drop it, this could be a solid venue paper. My own verdict would be conditional accept after revision.","headline":"RAF's empirical story on resolution-drift is solid and worth engaging; its convergence proof has a genuine algebraic flaw and should be corrected or removed before publication.","tokens_in":734,"tokens_out":1715,"would_cite":true,"duration_ms":31461,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Resolution-drift, a newly named failure mode in federated learning, degrades keypoint-detection models when clients hold images at different resolutions, and the paper's RAF method—multi-resolution knowledge distillation—restores accuracy…","keywords":["federated learning","resolution drift","human pose estimation","keypoint detection","multi-resolution knowledge distillation","vision transformer","non-IID data","high-resolution regression"],"falsifier":"Re-run the RAF experiments while recording the norm of the backbone weight updates during the last rounds; if those updates are not close to zero relative to the final-layer updates, the feature-converged premise behind the O(1/T) theorem fails, and the local-equivalence of the surrogate loss at the current iterate would need to be checked at neighbouring iterates as well.","tokens_in":18183,"feed_emoji":"🎯","tokens_out":8186,"duration_ms":81268,"temperature":0.7,"pith_summary":"This paper identifies a failure mode it calls resolution-drift in federated learning for non-classification vision tasks: when different clients hold images at different resolutions, the aggregated global keypoint-detection model overfits the resolutions it saw and loses accuracy on others, even though each client's data is privately held. The paper proposes RAF (resolution-adaptive federated learning), which adds a multi-resolution knowledge-distillation loss to each client's local training: the model's own higher-resolution heatmap output acts as a teacher for its lower-resolution output, with gradients stopped on the teacher. RAF is claimed to act as a resolution-aware regularizer that improves accuracy across both seen and unseen resolutions, and it is designed to plug into existing aggregation schemes such as FedAvg or FedProx without changing the server. The paper reports gains up to 27.6% at the lowest test resolution and presents a convergence analysis claiming the same O(1/T) rate as standard FedAvg. A sympathetic reader would care because resolution heterogeneity is common in real camera networks, and the proposed fix is local, modular, and aimed at a previously unnamed axis of non-IID data.","feed_headline":"Distillation stops resolution drift in federated pose models","feed_subtitle":"One global keypoint model stays accurate across resolutions, including unseen ones.","key_machinery":"The load-bearing object is the multi-resolution knowledge-distillation (MRKD) loss, Eq. (5): for each client and each downsampled level, the higher-resolution heatmap output (teacher) is detached from the gradient and compared, under a fixed linear upsampling operator, with the lower-resolution heatmap output (student), and the squared error is added to the task's heatmap MSE with weight α. This makes the local objective a regularized, resolution-aware quadratic form in the late-training regime, where the surrogate loss is claimed to agree with the original loss in value and gradient at each round's iterate. Two supporting mechanisms carry the practical claim: convolution-based global and local positional embeddings (from ResFormer) replace ViTPose's absolute positional embeddings so the same network can take inputs of any resolution, and the stop-gradient teacher makes distillation one-directional, from high to low resolution. The convergence argument runs through three bounds—L-smoothness, γ-strong convexity, and bounded gradient variance—which plug the RAF loss into a standard FedAvg convergence theorem to obtain O(1/T) for the optimality gap.","core_discovery":"The central claim is that resolution variability across federated clients is not a minor version of class-label heterogeneity but a distinct drift mechanism for high-resolution regression tasks. Because keypoint models must preserve spatial structure, a ViT trained at one resolution learns position-dependent features that do not transfer; averaging weights from clients trained at different resolutions produces a global model that is uncertain about scale and underperforms at every resolution, including the ones used in training. RAF counters this by having each client create downsampled copies of its own images and distill the high-resolution heatmap prediction into the lower-resolution heatmap predictions, minimizing their difference after a fixed upsampling operator. This forces low-resolution outputs to carry the same spatial information as high-resolution outputs, which the paper argues prevents overfitting to any single scale. The paper also replaces ViTPose's fixed absolute positional embeddings with convolution-based global and local positional embeddings so one backbone can process arbitrary resolutions, and it proves, under a feature-converged late-training assumption, that the RAF objective inherits FedAvg's O(1/T) convergence rate. The experiments on human pose estimation with MPII show RAF improving accuracy on seen and unseen resolutions relative to FedAvg and FedProx baselines.","pith_inferences":["The paper leaves implicit that MRKD is effectively a scale-equivariance regularizer; a natural comparison would be to explicit multi-scale augmentation or a resolution-conditional normalization, which might achieve similar robustness with less compute.","The convergence theorem is only as strong as the unverified feature-converged regime; an easy experiment is to track the norm of backbone weight updates in late training for the actual ViTPose model, which the paper does not report.","One testable extension is to apply the same MRKD recipe to depth estimation or super-resolution: if resolution-drift is a shared property of high-resolution regression, RAF should produce the same pattern of gains on unseen scales in those tasks.","The interpolation-at-inference result suggests a deployment recipe: rather than retraining for every camera resolution, operators can upsample low-resolution feeds to a canonical mid-high resolution before inference, and the paper's numbers give a concrete map of the accuracy/resolution trade-off."],"forward_implications":["If RAF's central claim is right, a single federated keypoint model can serve clients whose cameras produce very different resolutions, with accuracy on unseen resolutions no longer collapsing.","A low-resolution client can improve its own inference by interpolating its low-resolution image up to a mid-range resolution before feeding the model, because RAF-trained features behave well at those scales.","The method works as a local-training add-on, so existing FL aggregation logic (FedAvg or FedProx) stays untouched; the reported gains are not due to a new server algorithm.","The t-SNE evidence suggests the same resolution-robustness story extends to other spatial-detail tasks such as semantic segmentation, since high-resolution representation models cluster features by input resolution without the regularizer."],"supporting_citations":[{"why":"Defines FedAvg, the baseline aggregation algorithm that RAF builds on and keeps unchanged.","marker":"[6]"},{"why":"Provides FedProx, the heterogeneity-aware baseline that RAF is compared against and combined with.","marker":"[8]"},{"why":"Supplies ViTPose, the architecture, training recipe, and heatmap MSE task loss used in all experiments.","marker":"[11]"},{"why":"Contributes the convolution-based global and local positional embeddings that let the ViT accept multi-resolution inputs.","marker":"[17]"},{"why":"Motivates the stop-gradient operator used on the teacher branch of the knowledge-distillation loss.","marker":"[39]"},{"why":"Provides the FedAvg convergence analysis whose rate RAF claims to inherit through its smoothness, convexity, and variance bounds.","marker":"[43]"},{"why":"Supplies the MPII human-pose dataset used for every empirical evaluation.","marker":"[44]"}],"fun_headline_variants":["Distillation erases resolution drift in federated pose models","Adaptive distillation tackles resolution drift in FL keypoint nets","Federated keypoint: Distill to stop resolution drift","RAF distills resolutions to fix federated pose drift","Resolution-adaptive distillation halts FL pose drift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof's load-bearing premise is that in the late phase of training the ViT backbone has effectively stopped changing, so only the final layer's weights move; the paper does not verify this for its own ViTPose-trained models, and if the backbone is still changing, the claimed O(1/T) convergence guarantee is not established.","fun_headline_variants_meta":{"raw":{"variants":["Distillation erases resolution drift in federated pose models","Adaptive distillation tackles resolution drift in FL keypoint nets","Federated keypoint: Distill to stop resolution drift","RAF distills resolutions to fix federated pose drift","Resolution-adaptive distillation halts FL pose drift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000723,"raw_usage":{"total_tokens":3271,"prompt_tokens":1004,"completion_tokens":2267,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":2188}},"tokens_in":620,"tokens_out":2267,"duration_ms":16401,"temperature":1.0,"reasoning_tokens":2188,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T10:44:15.749093+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the RAF experiments while recording the norm of the backbone weight updates during the last rounds; if those updates are not close to zero relative to the final-layer updates, the feature-converged premise behind the O(1/T) theorem fails, and the local-equivalence of the surrogate loss at the current iterate would need to be checked at neighbouring iterates as well.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the baseline aggregation algorithm that RAF builds on and keeps unchanged."},{"cited_title":"Federated optimization in heterogeneous networks,","cited_arxiv_id":null,"evidence_quote":"Provides FedProx, the heterogeneity-aware baseline that RAF is compared against and combined with."},{"cited_title":"Vitpose: Simple vision transformer baselines for human pose estimation,","cited_arxiv_id":null,"evidence_quote":"Supplies ViTPose, the architecture, training recipe, and heatmap MSE task loss used in all experiments."},{"cited_title":"Resformer: Scaling vits with multi-resolution training,","cited_arxiv_id":null,"evidence_quote":"Contributes the convolution-based global and local positional embeddings that let the ViT accept multi-resolution inputs."},{"cited_title":"Bootstrap your own latent – a new approach to self- supervised learning,","cited_arxiv_id":null,"evidence_quote":"Motivates the stop-gradient operator used on the teacher branch of the knowledge-distillation loss."},{"cited_title":"On the convergence of fedavg on non-iid data,","cited_arxiv_id":null,"evidence_quote":"Provides the FedAvg convergence analysis whose rate RAF claims to inherit through its smoothness, convexity, and variance bounds."},{"cited_title":"2d human pose estimation: New benchmark and state of the art analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the MPII human-pose dataset used for every empirical evaluation."}],"review_version":1}