Many envision that in the near future the application of Unmanned Aerial Vehicles (UAVs) will impact the civil engineering industry. Use of UAVs is currently experiencing tremendous growth, primarily in military and homeland security applications. It is only a matter of time until UAVs will be widely accepted as platforms for implementing monitoring/surveillance and inspection in other fields. Most UAVs already have payloads as well as hardware/software capabilities to incorporate a number of non-contact remote sensors, such as high resolution cameras, multi-spectral imaging systems, and laser ranging systems (LIDARs). Of critical importance to realizing the potential of UAVs within the infrastructure realm is to establish how (and the extent to which) such information may be used to inform preservation and renewal decisions. Achieving this will depend both on our ability to quantify information from images (through, for example, optical metrology techniques) and to fuse data from the array of non-contact sensing systems. Through a series of applications to both laboratory-scale and field implementations on operating infrastructure, this paper will present and evaluate (through comparison with conventional approaches) various image processing and data fusion strategies tailored specifically for the assessment of highway bridges. Example scenarios that guided this study include the assessment of delaminations within reinforced concrete bridge decks, the quantification of the deterioration of steel coatings, assessment of the functionality of movement mechanisms, and the estimation of live load responses (inclusive of both strain and displacement).