We introduce a framework for simultaneously investigating the structure and luminosity evolution of early-type gravitational lens galaxies. The method is based on the fundamental plane, which we interpret using the aperture mass-radius relations derived from lensed image geometries. We apply this method to our previous sample of 22 lens galaxies with measured redshifts and excellent photometry. Modeling the population with a single mass profile and evolutionary history, we find that early-type galaxies are nearly isothermal (logarithmic density slope n = 2.06 coproduct 0.17, 68% C.L.) and that their stars evolve at a rate of d log (M/L)(B)/dz = -0.50 +/- 0.19 (68% C.L.) in the rest-frame B band. For a Salpeter initial mass function and a concordance cosmology, this implies a mean star formation redshift of < z(f)> > 1.5 at 95% confidence. While this model can neatly describe the mean properties of early-type galaxies, it is clear that the scatter of the lens sample is too large to be explained by observational uncertainties alone. We therefore consider statistical models in which the galaxy population is described by a distribution of star formation redshifts. We find that stars must form over a significant range of redshifts (Delta z(f) > 1.7, 68% C.L.), which can extend as low as z(f) similar to 1 for some acceptable models. However, the typical galaxy will still have an old stellar population (< z(f)> > 1.5). The lens sample therefore favors early star formation in field ellipticals, even if we make no a priori assumption regarding the shape of the mass distribution in lenses and include the range of possible deviations from homology in the uncertainties. Our evolution results call into question several recent claims that early-type galaxies in low-density environments have much younger stars than those in rich clusters.