Keyhole Effect in Dual-Hop MIMO AF Relay Transmission with Space-Time Block Codes

In this paper, the effect of keyhole on the performance of multiple-input multiple-output (MIMO) amplify-and-forward (AF) relay networks with orthogonal space-time block codes (OSTBCs) transmission is investigated. In particular, we analyze the asymptotic symbol error probability (SEP) performance of a downlink communication system where the amplifying processing at the relay can be implemented by either the linear or squaring approach. Our tractable asymptotic SEP expressions enable us to obtain both diversity and array gains. Our finding reveals that with condition n(S) > min(n(R), n(D)), the linear approach can provide the full achievable diversity gain of min(n(R), n(D)) when only the second hop suffers from the keyhole effect, i.e., single keyhole effect (SKE), where n(S), n(R), and n(D) are the number of antennas at source, relay, and destination, respectively. However, for the case that both the source-relay and relay-destination links experience the keyhole effect, i.e., double keyhole effect (DKE), the achievable diversity order is only one regardless of the number of antennas. In contrast, utilizing the squaring approach, the overall diversity gain can be achieved as min(n(R), n(D)) for both SKE and DKE. An important observation corroborated by our studies is that for satisfying the tradeoff between performance and complexity, we should use the linear approach for SKE and the squaring approach for DKE.