In this paper, the electrostatic and electromagnetic instabilities of a magnetized proton-DT plasma relevant to the fast ignition scheme of inertial fusion were examined. The magnetic field and proton beam define a plane and the magnetic field vector has the orientation angle of theta. The dispersion relation of two-stream and filamentation modes was first derived using the conservation, Euler, and Maxwell equations. Then, the more general case describing the wave propagation in an arbitrary direction in the two-dimensional case was derived symbolically. It is demonstrated that in a magnetized beam-plasma system, the two-stream instability depicts two separated growth curves, one larger than the other and there exists a stable frequency band between them. This zone will grow as the magnetic field inclination angle increases such that the smaller two-stream peak vanishes at the limiting angle of theta ->pi/2. The splitting in the growth rate curve becomes obvious at magnetic field parameter Omega(e)>0.5. The pure filamentation instability of the beam-plasma is about two orders of magnitude smaller than the two-stream instability and it just contributes to magnetic field orientation theta = 0. In a two-dimensional case, the mixed electrostatic/electromagnetic instabilities contribute to the final growth rate curve producing an X-like shape. The interface point of two growth curves is characterized by coordinate (similar to 1.0, tan(theta)) in the zx-plane.