{"id":"811c6f71-341d-4d7d-a41b-53e025b87ad0","arxiv_id":"2507.01581","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A hierarchical federated-learning DNN reaches indoor localization accuracy close to centralized learning on UJIIndoorLoc, but the reported error metric is questionable.","lead":"The authors test whether decentralized training called federated learning can locate people inside buildings from Wi-Fi signal strengths without sending raw data to a central server. They report that the federated model matches the accuracy of a centralized model on the standard UJIIndoorLoc dataset, while keeping data on each floor.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3)'s '2D-MAE' zeroes every sample with a wrong building/floor prediction; the 10.86/10.81 m figures are therefore not mean localization errors, so the 3.44% FL-vs-CL claim rests on an invalid metric.","rationale":"The reader's rejection is based on the same Eq. (3) issue, and it is indeed the load-bearing weakness. The paper's conclusion, a 'marginal increase of 3.44%' is read directly from Table IV, whose only metric definition is Eq. (3). A metric that zeroes all wrong-building/floor samples is not an MAE and can arbitrarily deflate errors; because the two models may misclassify different numbers of samples, the relative claim is not transferable to a correct metric. Independent support is limited: no code release, no seeds/repeats, and the KNN baseline is unverified, so there is no external check on the numbers. However, I do not see a need to move the verdict: the reader already recommends REJECT, and this stress-test confirms that. If the authors supply recomputed plain-MAE results and the FL/CL gap persists, the empirical comparison could be credible; but as written, the central claim is unsupported.","tokens_in":7771,"tokens_out":6851,"duration_ms":74880,"concrete_test":"Re-evaluate the trained FL and CL models on the 1111 validation fingerprints using plain Euclidean localization error: MAE_plain = (1/1111) * sum_i sqrt((x_i - xhat_i)^2 + (y_i - yhat_i)^2), without the Kronecker factors in Eq. (3). Also report building/floor classification accuracy so the number of zero-weighted samples is visible. If MAE_plain remains 10.86 vs 10.81 and the relative difference stays about 3.44%, the central comparison survives; if MAE_plain is materially larger or the FL/CL gap changes sign or magnitude, Table IV's headline numbers are artifacts of the metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (3) defines the evaluation metric as 2D-MAE = (1/N) * sum_i [delta(B_i, Bhat_i) * delta(F_i, Fhat_i) * sqrt((x_i - xhat_i)^2 + (y_i - yhat_i)^2)], with delta(a,b)=1 if a=b and 0 otherwise. Thus every validation sample whose predicted building or floor differs from ground truth contributes zero to the numerator while still counting in the denominator N. The paper's prose says the MAE 'represents the average distance between the predicted and actual geographic coordinates,' which is not what this equation computes. Since the model output is described in Sec. III as a 2D coordinate vector with no building/floor classification head, the Bhat_i and Fhat_i terms are not even defined by the stated architecture. The 10.86 m (FL) and 10.81 m (CL) values in Table IV therefore cannot be interpreted as mean localization errors, and the 3.44% relative gap could be distorted whenever the two models misclassify building/floor at different rates: more misclassifications means more zeroed samples, making a worse model look better. The KNN entry (12.81 m) is also evaluated under this same non-standard metric. The relative FL-vs-CL claim is the paper's central quantitative result, so it is unsupported until the metric is corrected.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a three-tier hierarchical federated learning (FL) scheme for Wi-Fi RSSI-based indoor localization. Floor-level DNNs are trained locally, aggregated into building-level regional models, and then into a global model, using the UJIIndoorLoc dataset. The authors report a validation MAE of 10.86 m for FL versus 10.81 m for a centralized DNN with the same architecture and 12.81 m for KNN, and conclude that FL performs comparably to centralized learning while preserving data privacy and reducing bandwidth. The central numerical claim is currently undermined by the evaluation metric in Eq. (3), which multiplies the Euclidean distance error by Kronecker deltas for building and floor, so samples with misclassified building or floor contribute zero error.","tokens_in":7982,"tokens_out":6294,"duration_ms":70248,"significance":"If the numerical results survive correction, the hierarchical FL design is a reasonable way to reduce communication and keep raw fingerprints local, and the use of a public benchmark with a temporally displaced validation set is a strength. The central quantitative claim, that FL is within 3.44% of centralized learning, is not supported as written because the reported errors are not true mean distances, the building/floor indicators in the metric are undefined for the stated architecture, and no uncertainty quantification is provided. The paper also does not currently provide enough algorithmic detail to reproduce the regional and global training procedure. The idea is worth pursuing, but the evidence presented here is insufficient for publication in its present form.","major_comments":[{"comment":"Equation (3) defines the reported error as (1/N) Σ δ(B_i, B̂_i) δ(F_i, F̂_i) √((x_i−x̂_i)²+(y_i−ŷ_i)²), so any sample whose predicted building or floor does not match the ground truth contributes zero error to the sum while still counting in N. The prose immediately after the equation states that the MAE 'reflects the average distance between the predicted and actual geographic coordinates,' which is not what the formula computes. Moreover, the architecture described in §III-B outputs only a two-dimensional coordinate vector and has no building/floor classification head, so B̂_i and F̂_i are not defined by the stated model. Because Table IV and the 3.44% conclusion in §VI are based on this metric, the main quantitative claim is unsupported. Please report plain mean Euclidean distance error, with building/floor misclassification rates given separately, and re-run the FL/CL comparison.","section":"§III-A, Eq. (3)"},{"comment":"The reported validation errors of 10.86 m and 10.81 m come from single runs; no seeds, repeated trials, or confidence intervals are given. The 0.05 m gap underlying the '3.44% increase' claim is far smaller than the run-to-run variability typical of DNN training, so it cannot be interpreted without repeated runs and a significance test. Please provide results over multiple random seeds and report mean ± standard deviation or confidence intervals.","section":"§V, Table IV"},{"comment":"Algorithm 1 is ambiguous about what happens at the regional and global levels. Lines 12–16 state that the building model is trained using the aggregated floor weights, and lines 17–20 state that the global model is trained using the aggregated regional weights; standard FederatedAveraging performs only weighted averaging and does not introduce an additional training pass on aggregated weights. If additional training is performed, the data used for that training and the privacy guarantees need to be specified. This is essential for reproducibility and for the claim that raw data never leaves the floors.","section":"§III-B, Algorithm 1"},{"comment":"The system treats each floor as a single FL user ('We considered each floor as a single user due to the limitation of data'), so training data for an entire floor is pooled before the first local update. The privacy guarantee is therefore at the floor level, not the device level: an honest-but-curious floor server could see all individual fingerprints on that floor. The abstract's statement that 'the data remains localized and confined within each device' and the related privacy framing should be revised to match the actual system, or the system should be evaluated with per-device clients.","section":"§II and §III-A"},{"comment":"The KNN comparison is not trustworthy as reported. The text states that the KNN baseline for UJIIndoorLoc is about 8.5 m in the literature while their implementation produces 12.81 m, and attributes the discrepancy to implementation differences. Without a precise description of the KNN configuration (number of neighbors, distance measure, preprocessing, and whether building/floor classification is used), the benchmark does not support the claim that FL outperforms KNN. Either align the KNN setup with the published baseline or report full configuration details and re-run.","section":"§IV, Table IV"}],"minor_comments":[{"comment":"Table III lists Adam exponential decay rates β1 = 0.1 and β2 = 0.99; the standard Adam defaults are 0.9 and 0.999. Please clarify whether this is a typo or an intentional non-standard configuration.","section":"§III-A, Table III"},{"comment":"The table header uses 'MDE' while the text consistently uses 'MAE'; please make the terminology uniform and define the acronym at first use.","section":"Table IV"},{"comment":"The sentence 'deleting all those columns where there is no data recorded at all i-e; cell values with 100 dBm' is ambiguous: it could mean columns where every value is 100 dBm, or columns containing any 100 dBm value. Please specify precisely the deletion criterion.","section":"§III-A"},{"comment":"The choices τ = 0.98, β = e, and minRSSI = −105 dBm are introduced without sensitivity analysis or citation beyond the general method in [15]. Since these choices affect all models equally, this is not a fatal issue, but a brief justification or a reference to a specific configuration would improve reproducibility.","section":"§III-A"},{"comment":"Figure 3 should include axis labels and a legend so the reader can identify which curve corresponds to FL and which to CL; the current description does not make this clear.","section":"§V, Fig. 3"},{"comment":"The notation W^c_{r,k}, β_c, and B_c is used in the algorithm but not defined; in particular, B_c appears to be a data batch rather than a building, which is confusing given that B is also used for buildings.","section":"§III-B, Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an empirical study with a constructive idea, but the main numerical claim is invalidated by the metric in Eq. (3) and the lack of uncertainty quantification. The paper could become publishable after a proper re-evaluation with a standard mean-distance error, repeated runs, and a clarified privacy model. The authors should also either fix the KNN baseline or remove that comparison. The fit with the journal's scope is acceptable if the experimental evidence is made reliable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: the reported 10.86 m vs 10.81 m MAE values are not what they appear to be. Equation (3) multiplies the Euclidean distance by Kronecker deltas for building and floor, so any sample with a wrong building or floor prediction contributes zero to the sum. The model, as described, outputs only a 2D coordinate and has no classification head, so the delta terms are not even defined. That means the headline comparison between FL and CL, and the 3.44% gap, is unsupported until the metric is fixed.\n\nWhat the paper does well: it formulates a clean three-tier hierarchical FL setup on a public dataset, keeps each floor's data local, and honestly notes in the text that its KNN baseline (12.81 m) is far from the published ~8.5 m for UJIIndoorLoc. The question of whether hierarchical FL can match centralized learning is worth asking, and the paper gives a straightforward benchmark structure.\n\nThe soft spots, in order of severity: (1) the metric flaw above is load-bearing—if the two models misclassify building/floor at different rates, a worse model can look better; (2) the KNN baseline is admitted to be far off, yet it is still used as a comparison point; (3) there are no error bars or repeated runs, so the 0.05 m difference between FL and CL is within noise; (4) the novelty is thin—[12], with a shared co-author, already does hierarchical FL for the same dataset, and the only twist here is keeping clients at the floor level rather than shuffling data.\n\nI agree with the reader's verdict. The central claim is not supported as stated. That said, the idea is not nonsensical; with a corrected metric and more careful evaluation, the relative FL-vs-CL comparison could be salvaged.\n\nWho this is for: someone working on federated indoor localization who wants to see a specific architecture attempt, and who can look past the numbers. But as a standalone result, it should not be cited for the claimed accuracy.\n\nRecommendation: I would not desk-reject out of hand, because the topic is relevant and the fix is straightforward. Send it to a serious referee only if the authors can redo the evaluation with plain MAE and report building/floor accuracy separately. As is, the paper needs major revision before it deserves publication.","headline":"The headline MAE numbers are unreliable because Eq. (3) zeroes out every sample with a misclassified building/floor, so the 3.44% FL-vs-CL claim rests on a flawed metric; the paper is an incremental hierarchical FL benchmark that needs a corrected evaluation before it can be trusted.","tokens_in":8574,"tokens_out":2015,"would_cite":false,"duration_ms":24800,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hierarchical federated learning system for indoor localization achieves nearly the same accuracy as a centralized model while keeping raw Wi-Fi data on each floor.","keywords":["hierarchical federated learning","indoor localization","privacy preservation","deep neural network","UJIIndoorLoc","RSSI fingerprinting","mean absolute error","centralized learning"],"falsifier":"Re-run the same trained FL and centralized models on the UJIIndoorLoc validation set and compute the ordinary mean Euclidean distance between predicted and true longitude-latitude coordinates without any building or floor gates, then check whether the 10.86 versus 10.81 meter gap still stands and whether the 3.44 percent increase remains consistent when the metric is changed.","tokens_in":1594,"feed_emoji":"📶","tokens_out":7602,"duration_ms":94963,"temperature":0.7,"pith_summary":"The paper tries to show that a hierarchical federated learning (FL) system can localize users indoors almost as accurately as a centralized deep neural network, while keeping raw Wi-Fi signal data on each floor and sharing only model weights. Using the UJIIndoorLoc public dataset, it reports a validation mean absolute error of 10.86 meters for FL versus 10.81 meters for centralized learning, a difference it frames as a marginal 3.44 percent increase. The point of the comparison is that the privacy, bandwidth, and server-reliability gains of FL can be bought with only a small accuracy cost, making FL a practical alternative for IoT location services. The paper also benchmarks against K-nearest neighbors and argues that machine-learning approaches beat it on the metrics that matter for real deployments.","feed_headline":"Federated localization trails centralized by just 3.44 percent","feed_subtitle":"Three-tier weight sharing keeps raw Wi-Fi on each floor, losing only 5 cm of validation accuracy.","key_machinery":"The load-bearing mechanism is the hierarchical aggregation of model weights rather than data. Each floor independently trains a DNN with three ReLU hidden layers, then each building combines its floors' weights into a regional model, and finally the global server aggregates the regional weights as a weighted average proportional to dataset sizes, following the update rule $W_{r+1} = \\frac{1}{\\sum_c |D_c|} \\sum_c |D_c| W_c^r$. Two data transformations support the pipeline: deletion of access points present in more than 98 percent of missing measurements, leaving 248 of 520 original APs, and a powered RSSI normalization that maps signal strengths to positive values before training. The stated performance measure is a validation mean absolute error computed over the training loop, with the central model used as the direct benchmark under identical architecture and hyperparameters.","core_discovery":"The central claim is that a three-tier hierarchical federated learning scheme—floor models trained locally, building-level regional models aggregated from those floors, and a global model aggregated from the regions—achieves localization performance close to a centralized model on the same DNN architecture. When both models are trained for 1000 epochs on the UJIIndoorLoc data, the validation error of the FL central model is 10.86 meters and the centralized model is 10.81 meters, which the paper interprets as a 3.44 percent relative increase. Beyond the raw numbers, the paper argues that this small accuracy gap is accompanied by substantial practical benefits: raw RSSI measurements never leave the floor, only weight updates travel over the network, and no single server holds all the data. The study therefore asserts that privacy-preserving federated localization is not a compromise that sacrifices accuracy but a viable deployment option for real indoor environments.","pith_inferences":["The evaluation metric in Eq. (3) multiplies the Euclidean distance error by Kronecker deltas for building and floor classification, so any fingerprint whose building or floor is mispredicted contributes zero to the sum; the reported 10.86 and 10.81 meter numbers are therefore not plain mean distance errors in the usual sense, and the gap between FL and centralized learning could change if the metr","The paper itself notes that its KNN result of 12.81 meters differs from a published UJIIndoorLoc KNN baseline near 8.5 meters, so the KNN column in the benchmark should be treated as implementation-specific rather than a fixed property of the algorithm.","A natural testable extension is to retrain the identical models on the same validation set while computing the ordinary mean Euclidean distance between predicted and true coordinates, and to report the error separately for correct-building, correct-floor, and misclassified samples, which would show whether the hierarchical FL advantage survives under stricter scoring.","The validation set was collected four months after the training set, giving a realistic temporal-displacement test, yet the paper reports only a single validation pass; a stronger claim would track validation error across multiple rounds and show that the 3.44 percent gap is stable rather than a snapshot."],"forward_implications":["If the reported parity holds, a building or campus can run indoor localization without aggregating raw fingerprints, so user movement data is not exposed to a central authority.","The communication cost drops from transmitting full datasets to transmitting only weight vectors, which matters when many floors or buildings participate in a city-scale deployment.","Because the same DNN architecture serves as floor, regional, and global model, the approach extends naturally to other radio or optical signal sources, and the paper names visible-light communication as the next target.","The measured gap of about five centimeters of validation error suggests that privacy-preserving deployment choices do not force a meaningful accuracy sacrifice in this setting.","The comparison to KNN indicates that learned models, FL included, produce lower localization error than a classic fingerprint-matching baseline, reinforcing the motivation for federated training."],"supporting_citations":[{"why":"Supplies the UJIIndoorLoc multi-building, multi-floor Wi-Fi fingerprint dataset that the whole experiment runs on.","marker":"[13]"},{"why":"Provides the powered RSSI transformation used to normalize and positivate signal strength inputs before DNN training.","marker":"[15]"},{"why":"Establishes the prior hierarchical federated localization approach whose privacy handling the paper improves by keeping floor models local.","marker":"[12]"},{"why":"Publishes the KNN baseline of about 8.5 meters on UJIIndoorLoc that the paper references when explaining its own higher KNN result.","marker":"[17]"},{"why":"Gives the standard federated learning formulation of weighted local losses that the paper adapts to its three-tier architecture.","marker":"[14]"}],"fun_headline_variants":["Hierarchical FL keeps localization private with only 3.44% error rise","Federated indoor localization: 5 cm accuracy drop, full privacy gain","Three-tier federated learning localizes indoors without exposing raw data","Privacy-preserving indoor location: minimal accuracy loss via FL hierarchy"],"cache_read_input_tokens":10624,"weakest_assumption_plain":"The reported accuracy numbers are trustworthy only if the metric in Eq. (3), which zeroes out the error whenever the predicted building or floor is wrong, is a valid way to measure localization error; if plain distance error is the intended measure, all the reported errors and the FL-versus-centralized comparison need to be recomputed.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical FL keeps localization private with only 3.44% error rise","Federated indoor localization: 5 cm accuracy drop, full privacy gain","Three-tier federated learning localizes indoors without exposing raw data","Privacy-preserving indoor location: minimal accuracy loss via FL hierarchy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000596,"raw_usage":{"total_tokens":2750,"prompt_tokens":867,"completion_tokens":1883,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":1806}},"tokens_in":483,"tokens_out":1883,"duration_ms":16064,"temperature":1.0,"reasoning_tokens":1806,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:48:08.765664+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same trained FL and centralized models on the UJIIndoorLoc validation set and compute the ordinary mean Euclidean distance between predicted and true longitude-latitude coordinates without any building or floor gates, then check whether the 10.86 versus 10.81 meter gap still stands and whether the 3.44 percent increase remains consistent when the metric is changed.","supporting_citations":[{"cited_title":"UJIIndoorLoc: A New Multi-Building and Multi-Floor Database for WLAN Fingerprint-based Indoor Localization Problems,","cited_arxiv_id":null,"evidence_quote":"Supplies the UJIIndoorLoc multi-building, multi-floor Wi-Fi fingerprint dataset that the whole experiment runs on."},{"cited_title":"Comprehensive Analysis of Distance and Similarity Measures for Wi- Fi Fingerprinting Indoor Positioning Systems,","cited_arxiv_id":null,"evidence_quote":"Provides the powered RSSI transformation used to normalize and positivate signal strength inputs before DNN training."},{"cited_title":"Federated Learning based Hierarchical 3D Indoor Localization,","cited_arxiv_id":null,"evidence_quote":"Establishes the prior hierarchical federated localization approach whose privacy handling the paper improves by keeping floor models local."},{"cited_title":"A realistic evaluation of indoor positioning systems based on wi-fi fingerprinting: The 2015 evaal–etri competition,","cited_arxiv_id":null,"evidence_quote":"Publishes the KNN baseline of about 8.5 meters on UJIIndoorLoc that the paper references when explaining its own higher KNN result."},{"cited_title":"The Future of Digital Health with Federated Learning,","cited_arxiv_id":null,"evidence_quote":"Gives the standard federated learning formulation of weighted local losses that the paper adapts to its three-tier architecture."}],"review_version":1}