The short-range order of melt-spun and mechanically alloyed Zr-Fe-B and Zr-Fe-Si amorphous alloys has been studied by means of Fe-57-Mossbauer effect and magnetization measurements. It is shown, that the Fe-surrounding for Fe concentrations up to 40 at.% is nearly independent from additional B alloying, whereas significant changes are obtained for Si additions. It is indicated, that B mainly occupies interstitial sites with pure Zr surroundings, whereas Si substitutes on Fe sites. The complete determination of the electric field gradient via magnetic distortion leads to the conclusion, that the same well-defined structural unit is present as observed for binary Zr-Fe glasses for Fe contents up to 33 at.%. Beyond this concentration, evidence is obtained that separation into a paramagnetic and a ferromagnetic amorphous phase takes place. The ferromagnetic component shows strong induced magnetization in high external fields far above the Curie temperature, preventing the complete determination of the electric field gradient tensor.