Real-time micro environmental observation with virtual reality

In this paper, the observation technique on the micro environments with a real-time virtual reality camera system, which is constructed with a dynamic focusing lens and a smart vision sensor using the "depth from focus" criteria, has been discussed In the operation with a microscope, such as the micro-surgery, DNA operation and etc., the small depth of a focus makes bad controllability to actuate SD micro objects with actuators. For example, if the focus is on the object, the actuator could not be seen in the microscope. On the other hand, if the focus is on the actuator, the object could not be observed. In this sense, the image, which includes all image informations at each focus values, i.e. "all-in-focus image" is useful to actuate objects in micro environments, However, one drawback on the all-in-focus image is that there is no information about the depth of objects. Then, it is also important to reconstruct a micro 3D environment in real-time to actuate objects in the micro virtual environments. Important factors to realize real-time system with the "depth from focus" criteria, which can obtain the all-in-focus image and micro 3D reconstruction, simultaneously, had been discussed. Finally, the real-time system is constructed by a dynamic focusing lens, which can change the focus in high frequency, and a smart vision system, which is capable in capturing and processing the image data in high speed with SIMD architecture.