A Hybrid RTS-BP Algorithm for Improved Detection of Large-MIMO M-QAM Signals

Low-complexity near-optimal detection of large-MIMO signals has attracted recent research. Recently, we proposed a local neighborhood search algorithm, namely `reactive tabu search' (RTS) algorithm, as well as a factor-graph based `belief propagation' (BP) algorithm for low-complexity large-MIMO detection. The motivation for the present work arises from the following two observations on the above two algorithms: $i)$ RTS works for general M-QAM. Although RTS was shown to achieve close to optimal performance for 4-QAM in large dimensions, significant performance improvement was still possible for higher-order QAM (e.g., 16- and 64-QAM). ii) BP also was shown to achieve near-optimal performance for large dimensions, but only for $\{\pm 1\}$ alphabet. In this paper, we improve the large-MIMO detection performance of higher-order QAM signals by using a hybrid algorithm that employs RTS and BP. In particular, motivated by the observation that when a detection error occurs at the RTS output, the least significant bits (LSB) of the symbols are mostly in error, we propose to first reconstruct and cancel the interference due to bits other than LSBs at the RTS output and feed the interference cancelled received signal to the BP algorithm to improve the reliability of the LSBs. The output of the BP is then fed back to RTS for the next iteration. Our simulation results show that in a 32 x 32 V-BLAST system, the proposed RTS-BP algorithm performs better than RTS by about 3.5 dB at $10^{-3}$ uncoded BER and by about 2.5 dB at $3\times 10^{-4}$ rate-3/4 turbo coded BER with 64-QAM at the same order of complexity as RTS. We also illustrate the performance of large-MIMO detection in frequency-selective fading channels.