Activation of gamma-aminobutyric acid B (GABA(B)) and 5-hydroxytryptamine (5-HT) receptors produces presynaptic inhibition at glutamatergic terminals in the rat neocortex. To evaluate interactions between these metabotropic receptors, field potentials were recorded in layer 2/3 of somatosensory cortex. In addition, the paired pulse (PP) protocol was used to measure changes in the ratio of the second/first extracellular synaptic potentials (S-2/S-1 ratio) as an index of glutamate release probability in the area. Lowering extracellular [Ca2+](o) to 0.5 mM, increased the S-2/S-1 ratio by 318 +/- 134%. 5-HT (1 muM) increased the S-2/S-1 ratio by 61 +/- 15%. In presence of the GABA(A) antagonist bicuculline (10 muM), 5-HT increased the S-2/S-1 ratio by 98 +/- 15%. This effect did not desensitize after two consecutive applications of the amine, and was dose dependent in the concentration range between 0.03-1 muM (EC50 = 2.36 x 10(-7) mol/L). The increase of S-2/S-1 ratio induced by 5-HT (1 muM) was blocked reversibly by the 5-HT1A antagonist NAN-190 (10-30 muM), but was unaffected by the selective GABA(B) antagonist CGP 52432 (1 muM). The action of 5-HT was mimicked by the 5-HT1A/7 agonist 8OH-DPAT (10 muM), increasing the S-2/S-1 ratio by 84 +/- 2%, a response that was unaffected by the 5-HT2/7 antagonist ritanserin (2 muM). The 5-HT1B agonist CP93129 (10 muM) had no effect. The GABA(B) agonist baclofen (1 muM) increased the S-2/S-1 ratio up to 308 +?- 33%, which is similar to that produced by low [Ca2+](o). When the effect of baclofen was maximal, application of 5-HT (1 muM) reversed the S-2/S-1 ratio back to 78 +/- 27%, a result that was blocked by the 5-HT2/7 antagonist ritanserin (100 nM). Notably, the interaction between the GABAB agonist and 5-HT was order dependent, because enhancement of the S-2/S-1 ratio elicited by baclofen was not inhibited if 5-HT was applied first. These results suggest a complex interaction between GABA(B), 5-HT1A, and 5-HT2 receptors in layer 2/3 of rat somatosensory cortex. Activation of GABA(B) receptors induces PP facilitation (inhibits glutamate release) more efficiently than does activation of 5-HT1A receptors. When the effect of GABA(B) receptor activation is maximal, however, the influence of 5-HT changes to the opposite direction, inhibiting PP facilitation (increasing glutamate release) through activation of 5-HT2 receptors. (C) 2004 Wiley-Liss, Inc.