It is well known that buoyancy suppresses, and can even laminarise, turbulence in upward heated pipe flow. Heat transfer seriously deteriorates in this case. A new direct numerical simulation model is established to simulate flow-dependent heat transfer in an upward heated pipe. The model shows good agreement with experimental results. Three flow states are simulated for different values of the buoyancy parameter C: shear turbulence, laminarisation and convective turbulence. The latter two regimes correspond to the heat transfer deterioration regime and the heat transfer recovery regime, respectively (Jackson & Li 2002; Bae et al., Phys. Fluids, vol. 17, issue 10, 2005; Zhang et al., Appl. Energy, vol. 269, 2020, 114962). We confirm that convective turbulence is driven by a linear instability (Su & Chung, J. Fluid Mech., vol. 422, 2000, pp. 141-166) and that the deteriorated heat transfer within convective turbulence is related to a lack of rolls near the wall, which leads to weak mixing between the flow near the wall and the centre of the pipe. Having surveyed the fundamental properties of the system, we perform a nonlinear non-modal stability analysis, which seeks the minimal perturbation that triggers a transition from the laminar state. Given the differences between shear and convective turbulence, we aim to determine how the nonlinear optimal (NLOP) changes as the buoyancy parameter C increases. We find that at first, the NLOP becomes thinner and closer to the wall. Most importantly, the critical initial energy E0 required to trigger turbulence keeps increasing, implying that attempts to trigger it artificially may not be an efficient means to improve heat transfer at larger C. At C = 6, a new type of NLOP is discovered, capable of triggering convective turbulence from lower energy, but over a longer time. It is active only in the centre of the pipe. We next compare the transition processes, from linear instability and by the nonlinear non-modal excitation. At C = 4, linear instability leads to a state that approaches a travelling wave solution or periodic solutions, while the minimal seed triggers shear turbulence before decaying to convective turbulence. Deeper into the parameter space for convective turbulence, at C = 6, the new nonlinear optimal triggers convective turbulence directly. Detailed analysis of the periodic solution at C = 4 reveals three stages: growth of the unstable eigenfunction, the formation of streaks, and the decay of the streaks. The stages of the cycle correspond to changes in the linear instability of the turbulent mean velocity profile. Unlike the self-sustaining process for classical shear flows, where the streak is disrupted via instability, here, decay of the streak is more closely linked to suppression of the linear instability of the mean flow, and hence suppression of the rolls. Flow visualisations at C up to 10 also show similar processes, suggesting that the convective turbulence in the heat transfer recovery regime is sustained by these three typical processes.
