REVIEW 5 major objections 9 minor 19 references
Synesthesia of Machines (SoM)-Aided FDD Precoding with Sensing Heterogeneity: A Vertical Federated Learning Approach
T0 review · 5 major / 9 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A vertical federated learning loop using eight pilots and heterogeneous vehicle sensors matches or beats perfect-CSI precoding baselines.
desk verdict Good idea, broken loss-function sign makes the central numbers unsupported as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the H-MVMM scheme, a vertical federated learning stack that splits the sum-rate optimization across vehicles. It rests on three preprocessing transforms that convert raw sensing into precoding-relevant features: a high-frequency encoding of GPS-derived line-of-sight angles, a YOLOv8-based building detector that turns camera images into one-hot multipath-angle vectors plus an encoded vehicle heading, and a voxelized bird's-eye-view compression of LiDAR point clouds. These sensing features are concatenated with features extracted from the received pilot vector and integrated by a local network that outputs the precoding vector. A rate-gap term in the loss accelerates convergence for weaker users, and an online training strategy retrains the local branches when the user count changes, saving at least 90 epochs relative to training from scratch in the reported $K=5$ to $K=6$ transition.
What would settle it
Train the identical H-MVMM scheme on a different city-block scenario or at a different carrier frequency and check whether the sum-rate gap to WMMSE with perfect CSI remains small at $L_P=8$; the claim fails if the scheme degrades to roughly pilot-only performance in the new setting.
Extended reading notes
Core claim
The central claim is that sensing heterogeneity can be turned into an advantage for FDD precoding instead of a nuisance. Under the vertical federated learning setup, each vehicle's local network maps the received pilot vector plus its available modality representations to a quantized precoding vector, and the base station reconstructs the precoding matrix and computes a sum-rate loss. The paper reports that H-MVMM with $L_P=8$ pilots approaches the sum rate of zero-forcing with ground-truth CSI when $K<5$ and surpasses it as the number of users grows, and that with sufficiently large feedback quantization it approximates the WMMSE benchmark with ground-truth CSI while beating a base-station neural network that receives full channel estimates. The complexity of the learned precoder is $O(L_P N)$, compared with $O(K^2N^3)$ for one WMMSE iteration. The per-modality results rank LiDAR ahead of RGB images and GPS in sum-rate contribution, with the H-MVMM combination retaining performance close to the best single modality.
Load-bearing premise
Everything rests on the assumption that a vehicle's GPS position, the buildings its cameras can see, and its LiDAR scan of the street are a reliable stand-in for the radio channel at 4.95 GHz in this scenario; if that sensing-to-channel link weakens in a new city or band, the near-perfect-CSI performance is not guaranteed.
Editorial extensions
If this is right
- At $L_P=8$ pilots, the H-MVMM scheme approaches zero-forcing with perfect CSI for $K<5$ and exceeds it for larger $K$ at 30 dB SNR in the tested scenario.
- With sufficiently many feedback bits, the scheme approximates WMMSE with perfect CSI at high $K$ and outperforms a base-station neural network given full channel estimates.
- The time complexity of producing one precoding vector is $O(L_P N)$, effectively linear in antenna count and independent of $K$, against $O(K^2N^3)$ for one WMMSE iteration.
- The vertical federated learning framework transmits only about 0.5 KB per vehicle per training epoch, versus hundreds of megabytes for centralized learning, with a small sum-rate penalty from quantization.
- The online training strategy adapts the network from five to six users in at least 90 fewer epochs than offline retraining, enabling response to fluctuating user counts.
Reading between the lines
- Beyond the paper: the same pipeline should be retrained and tested in a second city or at a different carrier frequency before the near-perfect-CSI result is treated as general, since the sensing-to-channel link is the unproven transfer point.
- Beyond the paper: comparing H-MVMM against compressed-sensing or other low-pilot channel estimators at the same $L_P=8$ would isolate how much of the gain comes from vehicle sensing rather than from end-to-end learned precoding.
- Beyond the paper: the claim that sufficiently large feedback approaches WMMSE suggests sweeping the feedback bit budget $B$ and plotting the sum-rate curve, which would show where quantization ceases to be the bottleneck.
- Beyond the paper: the online training result is shown for one user-count change at three SNRs; a sensitivity analysis over sensor configurations and user arrivals would define when the 90-epoch saving persists.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes H-MVMM, a vertical federated learning (VFL) framework for FDD massive MIMO precoding in which vehicles with heterogeneous on-board sensors (GPS, RGB cameras, LiDAR) and short pilot sequences (LP=8) compute local precoding vectors from their multi-modal data and received pilots, feed the quantized vectors back to the base station, and receive gradients for training. Three hand-crafted preprocessing pipelines convert raw sensing data into precoding-relevant features, and a customized loss and an online training strategy are introduced to handle data heterogeneity and changes in the number of users. Numerical results on the M3SC city-block scenario are reported to show that H-MVMM with limited feedback approaches ZF with perfect CSIT and, with sufficiently large feedback, approaches WMMSE with perfect CSIT while using far fewer pilots. The paper also compares VFL with centralized learning in terms of transmitted data volume and sum rate.
Significance. If the reported numerical results are reproducible, the paper addresses a timely problem: reducing pilot overhead and precoding complexity in FDD massive MIMO by exploiting multi-modal sensing. The use of a public dataset, strong perfect-CSI baselines (ZF, WMMSE), and an explicit VFL formulation are strengths, and the claimed reduction from N=128 pilot dimensions to LP=8 is substantial. However, the evidence is currently a single-scenario, single-run simulation with no error bars or released code. More importantly, the advertised customized-loss mechanism is not effective as written, and the handling of missing sensor modalities in the heterogeneous configuration is underspecified. The numerical results also do not clearly demonstrate an advantage of heterogeneity, since the homogeneous Pilot-LiDAR scheme outperforms the heterogeneous H-MVMM scheme at every reported K. These issues make the central claims defensible in principle but in need of substantial revision and additional verification.
major comments (5)
- [Sec. III-C, Eq. (15)] The claimed customized loss does not behave as stated. Expanding Eq. (15) with lambda=10 gives L = 9*sum_k R_k - 10K*R_T, so minimizing L maximizes the sum rate; the sign is therefore not the problem. The actual problem is that the added term is affine in R_k: its derivative with respect to each R_k is lambda-1, independent of R_T-R_k. Consequently, the statement that a large gap (R_T-R_k) results in a large derivative is false, and the loss provides no gap-dependent weighting or acceleration of convergence. Please either replace Eq. (15) with a nonlinear gap-dependent loss and re-derive the gradient, or remove the acceleration claim and provide an ablation isolating any actual benefit of the proposed term.
- [Sec. III-B and Sec. IV-A.2] The H-MVMM scheme is evaluated with vehicles having different sensor subsets (5/7 vehicles with GPS, 3/7 with RGB cameras, and 3/7 with LiDAR), but the paper does not specify how the local NN handles missing sensor modalities. Equations (13)-(14) and Fig. 5 present separate branches for GPS, RGB, LiDAR, and pilots, and the text assumes a vehicle is equipped with three sensors. There is no description of zero-padding, conditional branch selection, per-configuration architectures, or any other mechanism for variable sensor availability. This makes the training and inference procedure for the heterogeneous configuration underspecified and not reproducible.
- [Sec. III-C and Sec. IV-A.2] There is a training/inference mismatch in the feedback quantization. Section III-C states that more quantization bits can be used for uplink feedback during training, while Section IV-A.2 reports that all VFL-based schemes use B=2 quantization bits. Since the loss is evaluated on quantized precoding vectors through Eqs. (4b)-(4c), the paper must state the exact training-time bit budget and whether quantization is modeled in the forward pass. Without quantization-aware training or an explicit description, the reported 2-bit results may not reflect the actual system being evaluated.
- [Table I and Sec. IV-B] The central empirical claim rests on single runs in one M3SC scenario, with no error bars, random seeds, or code release. Moreover, Table I shows that VFL-Pilot-LiDAR outperforms H-MVMM at every reported K (e.g., 80.2 versus 75.2 bps/Hz at K=7), so the numerical results do not demonstrate that sensing heterogeneity improves sum rate; the near-WMMSE performance is driven mainly by the LiDAR modality. The abstract's unqualified statement that the approach closely approximates perfect-CSI methods also overstates the H-MVMM gap (75.2 versus 81.6 bps/Hz at K=7). Please report statistics over repeated training runs and qualify the claims according to the actual gaps and the sensor configurations used.
- [Sec. IV-B and Sec. IV-D] The complexity and data-volume comparisons are incomplete. The claim that H-MVMM has complexity O(L_P N) omits the NN forward pass, YOLOv8 building detection, and LiDAR BEV compression, which dominate the per-inference cost; the phrase constant time complexity increase with respect to N and L_P is inaccurate for a linear-order term. Similarly, the data-volume formula W_FL = N_epoch * K * 0.5KB in Section IV-D counts only the uplink precoding feedback and ignores the downlink gradient transmission required by Algorithm 1 (line 7). These comparisons should be corrected or qualified, or the claims should be limited to the pilot-processing component.
minor comments (9)
- [Sec. III-A2, Eq. (11)] The text says normalized weight and height; the first should be normalized width, and the typeset formula for q_{j,i} contains a stray symbol that should be cleaned up.
- [Sec. IV-C] The online training strategy's use of downlink channel estimation for loss calculation should be reconciled with the earlier no-explicit-CSI premise; please clarify whether online adaptation requires channel estimates and quantify the additional pilot overhead this entails.
- [Fig. 6(a) and Sec. IV-B] Please define what w/ Feedback versus w/ 2-bit Quantization mean in Fig. 6(a), give the value of B used for the sufficiently large B case, and add ZF w/ GT CSIT results to Table I so the claim of surpassing ZF can be checked numerically.
- [Sec. III-A1, Eq. (8)] The high-frequency encoding in Eq. (8) is the NeRF positional encoding; please cite the original source (Mildenhall et al.) in addition to reference [17].
- [Sec. IV-D] The statement that quantization losses are minimal is not well supported by Table II, where VFL H-MVMM gives 75.2 bps/Hz versus 79.1 for centralized learning; please quantify and qualify this claim.
- [Sec. III-B, Eq. (14)] The output dimension is written as R^{2N_t}, but the system model uses N for the number of antennas; please use consistent notation throughout.
- [Throughout] There are several typos and notation inconsistencies, including coloumn (Eq. (1)), leverging (Sec. III-B), surronding (Sec. III-A3), the the in the funding footnote, and the use of both Q and B for feedback bits.
- [Sec. III-C, after Eq. (15)] The phrase reciprocal of the sum rate appears to mean the negative sum rate; please correct the terminology.
- [Fig. 5 and Sec. IV-A.2] Please specify activation functions, normalization layers, LiDAR voxel dimensions Lx, Ly, Lz, YOLOv8 training details, and the criterion for validation loss stabilization, as these are needed for reproducibility.
Circularity Check
No significant circularity: the precoding network is trained and evaluated on the same sum-rate objective, and all external baselines are independent.
full rationale
The paper's central claim—that H-MVMM precoding with short pilots approaches WMMSE/ZF with perfect CSI—is an empirical result obtained by training a neural network to maximize sum rate and then evaluating sum rate against independent baselines (ZF, WMMSE, BS-NN). This is the intended objective, not a hidden circularity. The use of the authors' own M3SC dataset and SoM/MMFF references is contextual and does not force the result: the sensing modalities are ablated against pilot-only schemes, and the baselines use ground-truth CSI that is not derived from the model's outputs. The customized loss in Eq. (15) contains a sign inconsistency with the stated gradient descent update (minimizing it would reduce, not increase, sum rate), but this is a correctness or reproducibility concern, not a circularity: it does not make the reported result equivalent to its inputs by construction. No self-citation is load-bearing for the numerical conclusions. Score 1 reflects the presence of normal self-citations without any circular dependency.
Assumptions & free parameters
free parameters (6)
- Loss threshold R_T =
0.3
- Loss weight lambda =
10
- GPS/orientation encoding dimension L =
5 for angles, 10 for vehicle orientation
- RGB multipath angle binning interval Delta_omega =
not specified
- LiDAR voxel grid dimensions Lx, Ly, Lz =
not specified
- Vehicle sensor configuration =
5 GPS, 3 RGB, 3 LiDAR among 7 vehicles
assumptions (6)
- standard math Downlink received pilot model y_k = h_k^H X + z_k and sum rate formula in Eq. (2)
- domain assumption M3SC city-block dataset realistically represents a vehicular FDD propagation environment
- domain assumption YOLOv8 detector and camera intrinsics convert images to accurate multipath angle indicators
- domain assumption GPS noise model with standard deviation 5 m and dead-reckoning during outage capture real localization errors
- domain assumption The BS can compute and transmit exact gradients partial L / partial v_k to each vehicle during training
- ad hoc to paper The customized loss in Eq. (15) accelerates convergence of low-rate users
Cite this review
Pith. "Pith review of Synesthesia of Machines (SoM)-Aided FDD Precoding with Sensing Heterogeneity: A Vertical Federated Learning Approach." pith.science (2026). https://pith.science/paper/J3LAEZAA
@misc{pith2026250110941,
author = {Pith},
title = {Pith review of: Synesthesia of Machines (SoM)-Aided FDD Precoding with Sensing Heterogeneity: A Vertical Federated Learning Approach},
year = {2026},
howpublished = {\url{https://pith.science/paper/J3LAEZAA}},
note = {Machine review of arXiv:2501.10941}
}
read the original abstract
High complexity in precoding design for frequency division duplex systems necessitates streamlined solutions. Guided by Synesthesia of Machines (SoM), this paper introduces a heterogeneous multi-vehicle, multi-modal sensing aided precoding scheme within a vertical federated learning (VFL) framework, which significantly minimizes pilot sequence length while optimizing the system's sum rate. We address the challenges posed by local data heterogeneity due to varying on-board sensor configurations through a meticulously designed VFL training procedure. To extract valuable channel features from multi-modal sensing, we employ three distinct data preprocessing methods that convert raw data into informative representations relevant for precoding. Additionally, we propose an online training strategy based on VFL framework, enabling the scheme to adapt dynamically to fluctuations in user numbers. Numerical results indicate that our approach, utilizing short pilot sequences, closely approximates the performance of traditional optimization methods with perfect channel state information.
Figures
Reference graph
Works this paper leans on
-
[1]
Massive MIMO for next generation wireless systems,
E. G. Larsson, O. Edfors, F. Tufvesson, and T. L. Marzetta, “Massive MIMO for next generation wireless systems,” IEEE Commun. Mag., vol. 52, no. 2, pp. 186–195, Feb. 2013
work page 2013
-
[2]
J. G. Andrews, et al., “What will 5G be?, ” IEEE J. Sel. Areas Commun. , vol. 32, no. 6, pp. 1065–1082, Jun. 2014
work page 2014
-
[3]
Integrated Sensing and Com- munications (ISAC) for Vehicular Communication Networks (VCN),
X. Cheng, D. Duan, S. Gao, and L. Yang, “Integrated Sensing and Com- munications (ISAC) for Vehicular Communication Networks (VCN),” IEEE Internet Things J. , vol. 9, no. 23, pp. 23441–23451, Dec. 2022
work page 2022
-
[4]
Radar Integrated MIMO Communications for Multi-Hop V2V Networking,
Y . Fan, S. Gao, D. Duan, X. Cheng, and L. Yang, “Radar Integrated MIMO Communications for Multi-Hop V2V Networking,” IEEE Wire- less Commun. Lett. , vol. 12, no. 2, pp. 307-–311, Feb. 2023
work page 2023
-
[5]
Linear transmit processing in MIMO communications systems,
M. Joham, W. Utschick, and J. A. Nossek, “Linear transmit processing in MIMO communications systems,” IEEE Trans. Signal Process. , vol. 53, no. 8, pp. 2700–2712, Aug. 2005
work page 2005
-
[6]
An iteratively weighted MMSE approach to distributed sum-utility maximization for a MIMO interfering broadcast channel,
Q. Shi et al. , “An iteratively weighted MMSE approach to distributed sum-utility maximization for a MIMO interfering broadcast channel,” IEEE Trans. Signal Process. , vol. 59, no. 9, pp. 4331–4340, Sep. 2011
2011
-
[7]
Constrained channel decomposition-based hybrid beamforming for mmWave massive MIMO systems,
G. M. Zilli and W. P. Zhu, “Constrained channel decomposition-based hybrid beamforming for mmWave massive MIMO systems,” IEEE Open J. Commun. Soc. , vol. 1, pp. 1707–1720, 2020
work page 2020
-
[8]
Hybrid precoding and combining design for millimeter-wave multi-user MIMO based on SVD,
A. Li and C. Masouros, “Hybrid precoding and combining design for millimeter-wave multi-user MIMO based on SVD,” in Proc. IEEE Int. Conf. Commun. (ICC), 2017, pp. 1-–6
work page 2017
Show all 19 references
-
[9]
Low-complexity downlink channel estimation for millimeter-wave FDD massive MIMO systems,
X. Wu, G.. Yang, F. Hou, and S. Ma, “Low-complexity downlink channel estimation for millimeter-wave FDD massive MIMO systems,” IEEE Wireless Commun. Lett. , vol. 8, no. 4, pp. 1103–1107, Mar. 2019
2019
-
[10]
Bayesian matching pursuit-based channel esti- mation for millimeter wave communication,
Y . You and L. Zhang, “Bayesian matching pursuit-based channel esti- mation for millimeter wave communication,” IEEE Wireless Commun. Lett., vol. 24, no. 2, pp. 344–348, Nov. 2019
2019
-
[11]
Deep learning for distributed channel feedback and multiuser precoding in FDD massive MIMO,
F. Sohrabi, K. M. Attiah, and W. Yu, “Deep learning for distributed channel feedback and multiuser precoding in FDD massive MIMO,” IEEE Trans. Wireless Commun., vol. 20, no. 7, pp. 4044–4057, Jul. 2021
2021
-
[12]
Deep learning for multi-user MIMO systems: Joint design of pilot, limited feedback, and precoding,
J. Jang, H. Lee, I.-M. Kim, and I. Lee, “Deep learning for multi-user MIMO systems: Joint design of pilot, limited feedback, and precoding,” IEEE Trans. Commun. , vol. 70, no. 11, pp. 7279–7293, Nov. 2022
2022
-
[13]
Federated learning for precoding design in cell-free massive MIMO systems,
D. Wang, M. Tao, X. Zeng,and J. Liang, “Federated learning for precoding design in cell-free massive MIMO systems,” IEEE Open J. Commun. Soc. , vol. 4, pp. 1567–1582, 2023
2023
-
[14]
FLASH: Federated learning for automated selection of high-band mmWave sectors,
B. Salehi, J. Gu, D. Roy, and K. Chowdhury, “FLASH: Federated learning for automated selection of high-band mmWave sectors,” in Proc. IEEE Conf. Comput. Commun. (INFOCOM) , London, U.K., May 2022, pp. 1719–1728
2022
-
[15]
Integrated sensing and communications toward proactive beamforming in mmWave V2I via multi-modal feature fusion (MMFF),
H. Zhang, S. Gao, X. Cheng and L. Yang, “Integrated sensing and communications toward proactive beamforming in mmWave V2I via multi-modal feature fusion (MMFF),” IEEE Trans. Wireless Commun. , vol. 23, no. 11, pp. 15721–15735, Nov. 2024
2024
-
[16]
Intelligent multi-modal sensing-communication inte- gration: Synesthesia of machines,
X. Cheng et al. , “Intelligent multi-modal sensing-communication inte- gration: Synesthesia of machines,” IEEE Commun. Surveys Tuts., vol. 26, no. 1, pp. 258–301, 1st Quart., 2024
2024
-
[17]
Attention is all you need,
A. Vaswani et al., “Attention is all you need,” in Proc. Int. Conf. Neural Inf. Process. Syst. , 2017, pp. 6000–6010
2017
-
[18]
Real-time multi class helmet violation detection using few-shot data sampling technique and YOLOv8,
A. Aboah, B. Wang, U. Bagci, and Y . Adu-Gyamfi, “Real-time multi class helmet violation detection using few-shot data sampling technique and YOLOv8,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2023, pp. 5349–5357
2023
-
[19]
M 3SC: A Generic Dataset for Mixed Multi-Modal (MMM) Sensing and Communication Integration,
X. Cheng et al. , “M 3SC: A Generic Dataset for Mixed Multi-Modal (MMM) Sensing and Communication Integration,” China Commun. , vol. 20, no. 11, pp. 13–29, Nov. 2023
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.