We fabricated Al/Al2O3/SiO2/Si and Al/HfO2/Si structures to optimize the passivation layer of a backside-illuminated (BSI) complementary metal oxide semiconductor (CMOS) image sensor (CIS), with the key properties of the newly developed high-k passivation layer analyzed via border traps, interface traps, and fixed charges. In the first experiment using Al2O3/SiO2 bilayer-based structures, different thicknesses of SiO2 were applied from 0 to 15 nm. The improvement in their properties was confirmed by applying forming gas annealing (FGA), a type of post-treatment, to all experimental systems. The first experiment results indicated that both the SiO2 layer and FGA were effective for chemical passivation. However, a tradeoff occurred in the degree of improvement of the interface trap density (D-it) and fixed-charge density (Q(f)) according to the SiO2 layer thickness. Subsequently, in the second experiment using HfO2 single-layer-based structures, FGA improved the border trap to a relatively poor extent compared to the first experiment. Nevertheless, FGA improved the electrical characteristics of the HfO2 films without any side effects and results in optimal Dit and |Q(f)/q| values of 2.59 x 10(11) eV(-1) cm(-2) and 1.00 x 10(12) cm(-2), respectively, demonstrating its potential for the passivation layer in BSI CIS applications.