Robots are used in various types of applications in manufacturing, ranging from assembly to machine tending. Changes in production lines are becoming more common, either due to product changes, requiring changing manufacturing processes, as well as faults, caused by wear and tear of the robots. In such cases it is crucial that the robot system can adapt to the new circumstances. Recently the concept of Digital Twins (DTs) has gained a large amount of interest in both academia and industry. The idea of a DT is to have a digital copy of a physical system, where both systems exchange information which can be used to adapt the system to unforeseen changes in itself or its environment. This enables a DT to perform self-adaptation when problems arise during operation. However, development of a DT, whether for existing or future robots, is cumbersome. We therefore propose a development of modular DTs and outline our methodology for creating DTs for robot systems, their shortcomings and open challenges. Such a modular DT can be utilised to safely explore the potential adaptations of the physical system and its performance in changing environmental conditions.