Joint Tomlinson-Harashima source and linear relay precoder design in amplify-and-forward MIMO relay systems via MMSE criterion

Fan Shuo Tseng*, Min Yao Chang, Wen-Rong Wu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

29 Scopus citations


Existing minimum-mean-square-error (MMSE) transceiver designs in amplify-and-forward (AF) multiple-inputmultiple-output (MIMO) relay systems all assume a linear precoder at the source. Nonlinear precoders in such a system have yet to be considered. In this paper, we study a nonlinear transceiver in AF MIMO relay systems in which a Tomlinson-Harashima (TH) precoder is used at the source, a linear precoder is used at the relay, and an MMSE receiver is used at the destination. Since two precoders and three links are involved, the transceiver design, which is formulated as an optimization problem, is difficult to solve. We first propose an iterative method to overcome the problem. In the method, the two precoders are separately optimized in an iterative step. To further improve the performance, we then propose a non-iterative method that can yield closed-form solutions for the precoders. This method uses the primal decomposition technique in which the original optimization can first be decomposed into a master and a subproblem optimization. In the subproblem, the optimum source precoder is solved as a function of the relay precoder. In the master problem, the optimization is then transferred to a relay-precoder-only problem. However, the optimization is not convex, and the primal decomposition cannot be directly applied. We then propose cascading a unitary precoder after the TH precoder so that the optimization in the subproblem and the master problem can be conducted. Furthermore, using a relay precoder structure, we can transfer the master problem to a convex optimization problem and obtain a closed-form solution by the KaruchKuhnTucker (KKT) conditions. Simulations show that the proposed transceivers can significantly outperform existing linear transceivers.

Original languageEnglish
Article number5728944
Pages (from-to)1687-1698
Number of pages12
JournalIEEE Transactions on Vehicular Technology
Issue number4
StatePublished - 1 May 2011


  • Amplify-and-forward (AF)
  • joint source/relay precoders
  • KaruchKuhnTucker (KKT) conditions
  • minimum-mean-square- error (MMSE)
  • multiple-input multiple-output (MIMO)
  • primal decomposition approach
  • Tomlinson-Harashima precoding (THP)


Dive into the research topics of 'Joint Tomlinson-Harashima source and linear relay precoder design in amplify-and-forward MIMO relay systems via MMSE criterion'. Together they form a unique fingerprint.

Cite this