Integrated opto-chemical sensors present great advantages in comparison to the current lab equipment. They bring portability, reduced costs, facilitate in-situ measurements, as well as largely reduced sample volumes. In this quest, standard processing protocols over established materials, such as silicon nitride, silicon, silicon dioxide, titanium oxide, and even a wide variety of polymers have so far been the key toward on-chip devices. However, if very specific materials in terms of composition and tailored properties are required, the deposition via a solution represents a viable alternative. In this review, we highlight the role of sol-gel chemistry and top-down processing of sol-gel thin film layers in the design of waveguide-based optical sensors. In particular, we stress the advantages of porous sol-gel based materials as a new approach to increase sensitivity and selectivity, first when used as claddings, and, more recently, as waveguides with enhanced light-analyte interaction. We finally discuss the future perspectives of such devices to increase specificity in complex matrices, which is of utmost importance for bio-sensing.