We study dynamics of a nonlinear three-level Lambda system describing Bose-Einstein condensates of atoms and diatomic molecules coupled by a two-color laser field. The system has a nonlinear dark state which is a generalization of the usual atomic dark state. In the recent paper [A. P. Itin and S. Watanabe, Phys. Rev. Lett. 99, 223903 (2007)], nonlinear instabilities of the dark state due to 1:1 and 1:2 resonances were discussed in the model without mean-field collisional interactions. Here we investigate the dark state of a model with collisional interactions. We show that nonlinear instabilities can be used, in particular, for precise determination of the scattering lengths.