To address the problem that dynamic objects, sparse environmental features, and blurred images in smart manufacturing workshops cause the performance degradation of robotic SLAM (Simultaneous Localization and Mapping) systems, semantic information and pixel-based direct method are introduced to improve the existing vision SLAM algorithm. The objects in the environment are discriminated by the target detection technique, and the results are put into the tracking thread, and the objects with high dynamic level in the results are screened twice dynamically, static points are incorporated into the matching, and dynamic points are further processed to solve the problem of effective data loss caused by the previous direct rejection of dynamic objects. To cope with the variable environment, the input data are pre-processed by an adaptive enhancement algorithm that limits the contrast, and then the camera motion is estimated by a semi-dense direct method that is insensitive to feature missing. The evaluation results on the dynamic dataset show that the error of the improved system is significantly reduced compared with ORB-SLAM2, and the estimated trajectory fits better with the real trajectory, indicating that the localization accuracy of the system is improved, and the stability and robustness are improved.