{"id":"89398d42-4d13-4728-bc16-3ac35069e748","arxiv_id":"2505.08788","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Fine-tuning a GNN precoder on real-world CSI from a 33-AP testbed improves cell-free massive MIMO sum rate by about 8.2 bits/channel use over the synthetic-only pretrained model.","lead":"A wireless communication model trained on simulated radio data was fine-tuned using real measurements from a 33-antenna indoor testbed, improving downlink data rate by about 8.2 bits per channel use at high signal-to-noise ratio. The work shows a practical way to make graph-based precoding algorithms work in real cell-free massive MIMO systems.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Four-user 'real-world' test set is assembled from only the 44 strongest single-user channels with a random tuple-level split, so training and test share the same measured channel vectors; this can inflate the reported 8.2 bit/channel use fine-tuning gain.","rationale":"The paper's central claim is not that GNN precoding beats zero-forcing, but that fine-tuning a GNN on real CSI yields a large improvement over the synthetic-only pretrained model. For that claim to hold, the evaluation must measure generalization to real multi-user propagation. The four-user dataset fails this test in a specific, fixable way: it is generated by combining single-user measurements and then randomly splitting the combinations, so the test users are not truly unseen. Because only 44 channel vectors underlie all 124,750 four-user tuples, a large fraction of test tuples share component channels with training tuples. A GNN with 8 layers and edge features can memorize the statistics of these 44 vectors; its test sum rate can then be high without representing genuine adaptation to the real channel distribution. The top-44 amplitude selection compounds this by making the evaluation distribution artificially favorable and removing the path-loss diversity that the synthetic pretraining saw, which can make the pretrained model look worse and the fine-tuned model look better than in an unselected deployment. The freeze-depth selection on the same test set (Section V.A) is a second, independent inflation mechanism: reporting the best of nine configurations evaluated on the final test set overestimates the expected gain of the chosen configuration. These issues do not invalidate the idea of sim-to-real fine-tuning, nor the value of releasing a real single-user CSI dataset, but they do invalidate the specific 8.2 bit/channel use number as a measure of transfer-learning benefit. A position-disjoint split with validation-based freeze selection would settle whether the effect survives. The reader's conditional judgment is appropriate; the concern sharpens the reason for conditionality rather than overturning the paper's direction.","tokens_in":7343,"tokens_out":6193,"duration_ms":62309,"concrete_test":"Recompute the four-user experiment with a position-disjoint split: train on 4-tuples whose constituent positions come from one subset (e.g., 22 of the 44 strong positions) and test on 4-tuples from the other 22 positions, with the freeze depth chosen on a validation split rather than on the test set. If the fine-tuned-minus-pretrained gain at 20 dB SNR (currently 8.2 bits/channel use) or the ZF-gap reduction (15.7 to 7.5 bits) changes materially, the headline is inflated by tuple-level leakage and test-set-based model selection. An even stronger check is to build the four-user test set from all 500 positions, without the top-44 filter and with position-disjoint splits, which directly tests whether the strong-channel selection biases the gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV.A constructs the four-user dataset by keeping only the 44 single-user channel vectors with largest norm and then randomly sampling 124,750 4-tuples (C(44,4)=135,751). Section V.A then selects the freeze depth by evaluating all l in {0,...,8} on the four-user real-world test set, and the chosen Freeze-4 model is used to report Fig. 6(b). Two concrete problems follow. First, these are not genuine multi-user measurements: each 4-user channel matrix is an independent concatenation of single-user vectors measured at different times and positions, so realistic inter-user correlation and joint fading are absent; selecting the strongest 44 positions also restricts the test distribution to high-channel-gain users. Second, and more load-bearing, because all tuples are drawn from the same 44 base vectors and the split is random at the tuple level, the same single-user channel vector appears in both training and test tuples. The GNN can memorize these 44 measured vectors, so the fine-tuned model's test sum rate reflects interpolation over a tiny set of measured channels rather than generalization to new user positions. The 8.2 bit/channel use gain at 20 dB SNR and the reduction of the zero-forcing gap from 15.7 to 7.5 bits are therefore not secure evidence for sim-to-real transfer; the freeze-depth selection on the same test set further inflates the reported gain.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a graph neural network (GNN) precoder for cell-free massive MIMO, first pre-trained on synthetic channels generated with an Indoor Hotspot path-loss model and Rayleigh fading, then fine-tuned on real CSI collected from the Techtile testbed. To adapt the pretrained model, the authors freeze the first l of eight GNN layers and retrain the remaining layers on real CSI, evaluating every l in {0,...,8}. They construct two-user and four-user datasets by forming tuples of single-user measured channel vectors. The reported results claim that the Freeze-4 model gains approximately 8.2 bit/channel use over the pretrained model at 20 dB SNR on real CSI (about 15.7% relative improvement) and reduces the gap to zero-forcing from 15.7 to 7.5 bit/channel use. The paper also compares against zero-forcing, conjugate beamforming, and a randomly initialized model trained directly on real data.","tokens_in":7725,"tokens_out":4905,"duration_ms":49243,"significance":"If the findings were obtained under a leakage-free evaluation, the paper would provide a useful empirical data point on sim-to-real transfer for GNN-based precoding, with a publicly stated dataset and an unsupervised objective matched to sum-rate maximization. The central direction and the choice of baselines (ZF, CB, no-pretraining) are sensible, and the layer-freezing idea is clearly motivated by limited real data. However, the current evaluation protocol has a load-bearing flaw: the freeze depth is selected on the same test set used for the headline numbers, and the constructed real-world datasets reuse the same measured single-user channel vectors in training and testing. These issues mean the specific quantitative claims are not yet established, even though the general direction may be correct.","major_comments":[{"comment":"The four-user real-world dataset is assembled from 4-tuples of the same 44 strongest single-user channel vectors, and the random 80/10/10 split is performed at the tuple level. Because each of the 44 base vectors appears in many training and test tuples, the same measured channel vectors occur on both sides of the split. The reported 8.2 bit/channel use gain at 20 dB SNR and the reduction of the ZF gap from 15.7 to 7.5 bits therefore reflect interpolation over a very small set of measured channels rather than generalization to unseen user positions. The evaluation should be repeated with a position-level split in which the set of single-user positions used for fine-tuning is disjoint from the set used for testing.","section":"§IV.A and §V.C (Fig. 6(b))"},{"comment":"The freeze-depth parameter l is selected by exhaustively evaluating all l in {0,...,8} on the same four-user real-world CSI test set that is later used to report every headline performance number. This makes the choice of Freeze-4 and the 8.2 bit/channel use gain a selected maximum over nine configurations evaluated on the test set, which biases the result upward. The freezing strategy should be chosen on the validation split or via nested cross-validation, with the final test set used only once.","section":"§V.A"},{"comment":"The constructed multi-user datasets concatenate single-user channel vectors measured at different positions and times, and the four-user set keeps only the 44 largest-norm single-user channels. Such matrices do not contain realistic inter-user correlation, joint shadowing/fading, or user scheduling effects, and the strong-channel selection restricts the test distribution to high-gain users. The statement in Section V.C that fine-tuning learns from real-world interference patterns is therefore not supported by this dataset construction; the results should be framed as a proof-of-concept on constructed multi-user samples, not as validation on genuine multi-user real-world measurements.","section":"§IV.A and §V.C"}],"minor_comments":[{"comment":"Equation (10) is typeset as a single equation with two branches but no clear delimiter; it should be split into two separate labeled equations for CB and ZF.","section":"Eq. (10)"},{"comment":"References [7] and [8] are identical; the duplicate should be removed and the reference list should be checked for other duplicates.","section":"References"},{"comment":"The dataset availability statement mentions 'Real-world CSI Dataset' without an actual URL or repository identifier; a working link should be provided.","section":"§IV.A"},{"comment":"The phrase 'corresponding to a relative improvement of approximately 15.7%' is ambiguous because the immediately preceding numbers are the ZF gap reduction (15.7 to 7.5 bits), while the 15.7% refers to the sum-rate gain of the fine-tuned model over the pretrained model; please clarify this wording.","section":"§V.C"},{"comment":"If Freeze 8 freezes all layers, the model is identical to the pretrained network, so the relation between the Freeze 8 curve in Fig. 4 and the 'GNN Pretrained (real CSI)' curve in Fig. 5 should be stated explicitly to avoid apparent inconsistency.","section":"§V.A and Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The weakness is methodological rather than conceptual. The paper can be salvaged by redoing the evaluation with a position-level split, selection of the freeze depth on validation data, and an honest framing of the constructed datasets as paired single-user measurements. I would not recommend rejection of the research direction, but the current quantitative claims should not appear as stated without those corrections."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look, with a caveat. The paper does something the GNN-precoding literature has only gestured at: it fine-tunes a GNN precoder on real measured CSI from a 33-AP testbed and releases the dataset. That is a genuine step, and the layer-freezing study is a sensible way to handle the small real dataset. The qualitative finding — freeze early layers, adapt later ones — is believable and consistent with how transfer learning tends to behave.\n\nThe quantitative headline, though, is not secure. Two problems cut into the 8.2 bits/channel use gain. First, the freeze depth l is chosen by exhaustive evaluation on the same four-user real-CSI test set that is later used to report the gain. That is model selection on the test set; the reported number is a selected maximum. Second, and more serious, the four-user dataset is assembled from only the 44 strongest single-user channel vectors, and the train/test split is random at the tuple level. Since every tuple is drawn from those same 44 vectors, the same measured channel appears in both training and test. The model can memorize those 44 channels. Test performance then measures interpolation over a tiny set of measured vectors, not generalization to new user positions. The reduction of the ZF gap from 15.7 to 7.5 bits is therefore not secure evidence for sim-to-real transfer.\n\nThe two-user dataset has the same tuple-level overlap, though with 500 base vectors it is less severe. The paper also reports no error bars or repeated runs, and the strong-channel selection restricts the test distribution to high-gain users. None of this kills the central direction: fine-tuning on real data should help, and the authors deserve credit for building the testbed dataset and running the study. But the precise improvement is inflated, and the current evaluation cannot support the strength of the abstract's claim.\n\nWho should read it: anyone working on learning-based precoding or sim-to-real for wireless. The dataset alone is worth having. It deserves peer review, but with major revision: choose the freeze depth on a validation split, split at the user/position level rather than tuple level, and report variance across seeds or data subsets.","headline":"A credible first step on sim-to-real GNN precoding with a public CSI dataset, but the headline gain is inflated by test-set model selection and tuple-level leakage in the four-user dataset.","tokens_in":8198,"tokens_out":2555,"would_cite":true,"duration_ms":23653,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning a GNN precoder on real-world channel measurements, with the first four of eight layers frozen, yields an 8.2 bits/channel use sum-rate gain over the synthetic-only pretrained model at 20 dB SNR.","keywords":["cell-free massive MIMO","graph neural network","precoding","transfer learning","fine-tuning","real-world CSI","layer freezing","sum rate"],"falsifier":"Collect true simultaneous two-user and four-user CSI on the same 33-access-point testbed, run the same pretrain-and-Freeze-4 fine-tuning protocol, and compare sum rates; if the fine-tuned GNN no longer gains roughly 8.2 bits per channel use over the pretrained model at 20 dB SNR, or no longer cuts the zero-forcing gap from 15.7 to about 7.5 bits per channel use, the reported gain is an artifact of the constructed dataset.","tokens_in":7117,"feed_emoji":"📶","tokens_out":7081,"duration_ms":58946,"temperature":0.7,"pith_summary":"The paper claims that a graph neural network precoder trained on synthetic channel data can be made to work on real-world channels by fine-tuning on a limited set of measured channel vectors, with the first four of eight layers frozen during fine-tuning. In the four-user experiment on a 33-access-point testbed, fine-tuning improves sum rate by about 8.2 bits per channel use at 20 dB transmit SNR over the pretrained model, a relative gain of roughly 15.7 percent, and reduces the gap to zero-forcing precoding from 15.7 to 7.5 bits per channel use. The result matters because learned precoders are usually validated only in simulation, and this is a concrete demonstration that transfer learning can carry one from synthetic to measured channels.","feed_headline":"Fine-tuned GNN precoder gains 8.2 bits per channel use on real CSI","feed_subtitle":"Freezing the first four layers during fine-tuning shrinks the gap to zero-forcing from 15.7 to 7.5 bits on measured channels.","key_machinery":"The load-bearing object is an eight-layer edge-centric graph neural network operating on a bipartite graph whose vertices are access points and users and whose edges carry the CSI as attributes. Each layer updates an edge representation by combining its previous embedding with mean-aggregated messages from the two endpoint nodes, applies a LeakyReLU nonlinearity, and the final output is power-normalized and trained unsupervised to maximize sum rate. The transfer mechanism is layer freezing: the first four layers keep their pretrained weights during fine-tuning and only the remaining layers are updated, which the authors find optimal after comparing all freezing depths from zero to eight.","core_discovery":"The central claim is that unsupervised pretraining on synthetic CSI followed by unsupervised fine-tuning on real CSI makes an edge-centric GNN precoder generalize to a physical cell-free massive MIMO environment, whereas the same model trained only on synthetic CSI degrades when evaluated on real measurements. In the four-user setting, the fine-tuned model outperforms the pretrained model by 8.2 bits per channel use at 20 dB SNR and narrows the gap to zero-forcing from 15.7 to 7.5 bits per channel use; a model trained from scratch on the real data alone does not match this performance. The authors attribute the improvement to pretraining providing a favorable initialization and to freezing early layers preserving generalizable representations while later layers specialize to the real channel statistics.","pith_inferences":["The authors leave implicit that the optimal freezing depth likely depends on the size of the real dataset and the severity of the distribution shift, so the Freeze-4 choice should be re-tuned rather than reused in other deployments.","Because the four-user real dataset is constructed by pairing single-user channel vectors and keeping only the 44 strongest channels, the reported gain may differ on genuinely simultaneous multi-user measurements; that is a testable extension, not a claim in the paper.","The same pretrain-then-freeze pipeline could plausibly transfer to other learned MIMO components, such as channel estimation or RIS beamforming, which face the same sim-to-real gap.","A natural next experiment is to vary the number of real channel samples used for fine-tuning; the paper uses one fixed real dataset and does not report how the 8.2 bits per channel use gain scales with data volume."],"forward_implications":["A synthetic-to-real transfer recipe exists for GNN precoding: pretrain on simulated channels, then fine-tune on a modest set of measured channels.","Freezing early layers outperforms full fine-tuning in this setup, so preserving pretrained representations matters more than full adaptation when real data are scarce.","The fine-tuned GNN remains 7.5 bits per channel use below zero-forcing on real four-user channels, so learned precoding is closer to, but not yet a replacement for, zero-forcing.","Fine-tuning improves real-data performance at the cost of some synthetic-data performance, confirming a domain-adaptation tradeoff rather than universal improvement."],"supporting_citations":[{"why":"supplies the edge-centric GNN message-passing precoding architecture that the paper adapts.","marker":"[4]"},{"why":"gives the InH NLOS path-loss model used to generate the synthetic pretraining channels.","marker":"[9]"},{"why":"describes the physical testbed from which the real CSI measurements were collected.","marker":"[10]"},{"why":"formalizes covariate shift, the distribution mismatch that motivates the fine-tuning strategy.","marker":"[11]"},{"why":"provides the zero-forcing and conjugate beamforming baselines against which the fine-tuned GNN is compared.","marker":"[12]"}],"fun_headline_variants":["GNN precoder: 8.2-bit gain from real-CSI fine-tuning","Fine-tuning closes GNN precoder gap to zero-forcing by 8.2 bits","Real CSI fine-tuning boosts GNN precoder 15.7% over synthetic","Transfer learning lets GNN precoder handle real channels, +8.2 bits","Synthetic-to-real transfer gives GNN precoder 8.2 bits per use"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The four-user real-world data are not actual simultaneous four-user measurements; they are built by pairing single-user channel vectors and keeping only the 44 strongest channels, so the reported gain assumes those constructed matrices behave like genuine multi-user propagation.","fun_headline_variants_meta":{"raw":{"variants":["GNN precoder: 8.2-bit gain from real-CSI fine-tuning","Fine-tuning closes GNN precoder gap to zero-forcing by 8.2 bits","Real CSI fine-tuning boosts GNN precoder 15.7% over synthetic","Transfer learning lets GNN precoder handle real channels, +8.2 bits","Synthetic-to-real transfer gives GNN precoder 8.2 bits per use"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000275,"raw_usage":{"total_tokens":1656,"prompt_tokens":968,"completion_tokens":688,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":578}},"tokens_in":584,"tokens_out":688,"duration_ms":6534,"temperature":1.0,"reasoning_tokens":578,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:45:31.257644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect true simultaneous two-user and four-user CSI on the same 33-access-point testbed, run the same pretrain-and-Freeze-4 fine-tuning protocol, and compare sum rates; if the fine-tuned GNN no longer gains roughly 8.2 bits per channel use over the pretrained model at 20 dB SNR, or no longer cuts the zero-forcing gap from 15.7 to about 7.5 bits per channel use, the reported gain is an artifact of the constructed dataset.","supporting_citations":[{"cited_title":"Study on channel model for frequencies from 0.5 to 100 ghz (release 17),","cited_arxiv_id":null,"evidence_quote":"gives the InH NLOS path-loss model used to generate the synthetic pretraining channels."},{"cited_title":"Techtile: Open 6G R&D testbed for communica- tion, positioning, sensing, WPT and federated learning,","cited_arxiv_id":null,"evidence_quote":"describes the physical testbed from which the real CSI measurements were collected."},{"cited_title":"Toward energy-efficient massive MIMO: Graph neural network precoding for mitigating non- linear PA distortion,","cited_arxiv_id":null,"evidence_quote":"supplies the edge-centric GNN message-passing precoding architecture that the paper adapts."},{"cited_title":"Quinonero-Candela, M","cited_arxiv_id":null,"evidence_quote":"formalizes covariate shift, the distribution mismatch that motivates the fine-tuning strategy."},{"cited_title":"Optimal Design of Energy-Efficient Multi-User MIMO Systems: Is Massive MIMO the Answer?","cited_arxiv_id":null,"evidence_quote":"provides the zero-forcing and conjugate beamforming baselines against which the fine-tuned GNN is compared."}],"review_version":1}