High-order solutions of motion near triangular libration points for arbitrary value of μ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\mu }}$$\end{document}

In this paper, a new methodology is proposed to derive the high-order approximations of motions near them in three cases, i.e., the mass ratio is greater than, smaller than and equal to Gascheau and Routh critical value. A preliminary analysis on the phase space structure of triangular libration points in the first case is accomplished from the point of view of the dynamical system theory, demonstrating that they have two-dimensional stable/unstable manifolds and zero-dimensional center manifolds, referred to as 2+2+0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2+2+0$$\end{document} type. Further investigations show the topological type of triangular libration points evolves from center–center type to 2+2+0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2+2+0$$\end{document} type as mass ratio increases above Gascheau and Routh critical value. The high-order approximations of motion near triangular libration points are constructed using a modified method of variation of parameters. The methodology developed in this paper can deal with triangular libration points of all three topological types, which remains unsolved by the traditional Lindstedt–Poincaré method. The simulation results demonstrate that the expansions up to 11th order are a good replacement of numerical computation with a tolerate error. The validity regions of initial position deviation are presented, and a modification of the algorithm is performed to guarantee the continuity near Gascheau and Routh critical value. Furthermore, the high-order solutions of two-dimensional stable/unstable manifolds are employed to search for homoclinic connections of triangular libration points of 2577 Litva.