We construct the most general disorder operator for SU(N) lattice gauge theory in (2 + 1) dimensions by using exact duality transformations. These disorder operators, defined on the plaquettes and characterized by (N - 1) angles, are the creation and annihilation or the shift operators for the SU(N) magnetic vortices carrying (N - 1) types of magnetic fluxes. They are dual to the SU(N) Wilson loop order operators which, on the other hand, are the creation-annihilation or shift operators for the (N - 1) electric fluxes on their loops. The new order-disorder algebra involving SU(N) Wigner D matrices is derived and discussed. The ZN( is an element of SU(N)) 't Hooft operator is obtained as a special limit. In this limit we also recover the standard Wilson -'t Hooft order-disorder algebra. The partition function representation and the free energies of these SU(N) magnetic vortices are discussed.