Abstract

A digital twin in manufacturing system is a virtual replica of a physical system that can be used to analyze, monitor, and optimize in real-time. These digital models are important in modern smart industries, helping in terms of production planning and control decisions if they are correctly representing their physical counterparts. More specifically in operations management digital twin can be used for example for real-time monitoring, production control or production planning [1]. The digital is a mean to organize and merge past, current and future information on the production system to make the smartest decisions in the management of operations.

Many challenges occur along the lifecycle of a production system digital twin. Namely, the creation of the digital twin necessitates skills, data and time. Then, during the operational life of the twin, it has to be fed with timely consistent and accurate data to be valuable in possible decisions. Computer vision can aid in the process of Automated Digital Twins Generation and feeding by using cameras and sensors to capture data about the physical asset's movements, dimensions, and properties.

The complex environment of a factory poses hard computer vision problems that we aim to address in this work, such as 2D object recognition, 3D volume estimation, integrating information from multiple views, enriching the vision model with data from logs, and providing a flexible way to adapt the model to new objects and actions happening in the manufacturing environment.

The presentation will focus on the challenges of using computer vision for the feeding of a digital twin for an assembly system. The construction of a benchmark problem will be presented along with first attempts of resolution with state-of-the-art computer vision algorithms.