Digital twins and hardware-in-the-loop simulations are often confused for the same thing. However, the two techniques are complementary and allow for fast robot development and integration. In this blog post, both digital twins and hardware-in-the-loop simulations will be explained. How can they be used to quickly find issues in a robot system? And how can new developments benefit from using these techniques?  

Digital twin 

The term ‘digital twin’ is a buzz-word increasing in popularity in recent years.  Digital twin can be defined in multiple ways. One definition says:  “[…] digital twins are […] digital replications of living as well as nonliving entities that enable data to be seamlessly transmitted between the physical and virtual worlds. Digital twins facilitate the means to monitor, understand, and optimize the functions of all physical entities […]. “[1] 

To elaborate; a digital twin is more than a pure model of a device. It is a virtual replica of the device. This replica is usually based on a model (for instance on a CAD model or a pure software simulation), but augmented with data from the real world, e.g. from sensor readings, and possibly even with virtual data such as predictions.  

In the context of a robot system, a digital twin can be used to show the internal state of the robot. For example, it can show what the robot `thinks` and what the robot is going to do. Within Smart Robotics, the digital twin is used in the Smart Palletizer application. It is used to visualize the motions of the robot, as well as which boxes the robot thinks it has placed already, and which box it will place next. Using the digital twin allows for visually verifying the calibration of the robot system. It shows how the robot thinks the boxes are placed, whilst on the pallet it can be seen how the robot actually placed the boxes. If there is a mismatch, this may indicate that there is a problem with the system’s calibration.  

Hardware-in-the-loop simulation 

A digital twin should not to be confused with a hardware-in-the-loop (HIL) simulation. In a HIL simulation, one or more components of the system are replaced with simulated virtual components.  In more detail, a complex robot system consists of multiple different components which all ‘talk’ to each other via a given protocol. For instance, the Smart Palletizer has multiple sensors that give readings, such as a sensor indicating whether there is a box available at the Smart Formation Unit (SFU) or the vacuum sensor on the gripper, indicating whether a box is attached to the gripper.  

This data is received by the program that controls the robot and the robot acts according to it. For example: ‘If I am holding a box, I should place it on the pallet’. If you can now reproduce the data that the robot system control program expects from a specific component, for instance by using a simulated model, you can ‘trick’ the rest of the system to behave as if it received the data from the real component.  

HIL simulation example 

There are several use-cases where a HIL simulation can be useful. In the normal flow of the application, after the robot has performed a pick, it knows that it should be holding a box. However, a faulty vacuum sensor could tell the robot that it is not holding a box. In this case, the robot would not continue. To make sure that it is indeed a faulty sensor reading that leads to the problem, you can replace the physical vacuum sensor by a simulated component. The simulated vacuum sensor tells the robot the value it normally receives. If the robot continues to operate as usual with the simulated component, you know for sure that the problem lies with the sensor. 

Of course, you could achieve a similar behavior by adding additional conditions directly in the application, but this leads to messy and cumbersome code: ‘After I pick a box and the vacuum sensor is disabled, continue to place, otherwise read the vacuum sensor value and check if I am holding a box and only continue to place when I am holding a box’.   

Digital twins and HIL simulations are complementary 

Hence, using a HIL simulation allows you to check and verify the behavior of the robot in many different situations which you would otherwise only encounter occasionally. Both technologies, digital twins and HIL simulations, are complementary. A digital twin is usually based on a model and uses data from the real system to update its state. If we can now simulate the necessary input data, the digital twin can also be used for the simulated component in a HIL simulation.  

To sum up, while the digital twin allows to visualize insights of the system which cannot be seen in the real world, a HIL simulation replaces physical components of the system with simulated virtual components. 

How digital twin and HIL simulations speed up robot system development and integration 

Imagine the whole robot system being replaced with simulated components. In that case, you can already perform tests without having the hardware available. For instance, during the development of a new robot frame, the frame design can be loaded into the simulations. The digital twin can then visualize how the new frame will perform. The simulation will show whether the new frame design results into collisions during certain robot movements and whether all box place positions are still reachable. Using the digital twin leads to a reduction in hardware development time, as the hardware will only be produced if the initial simulations predict good results.  

Now imagine a customer who wants to palletize a new product with a different box size and/or a different stacking pattern. Traditionally, you would need to test an entire pallet of the new boxes to see if the robot is able to reach all positions of the new stacking pattern. However, using digital twin you can perform a full simulation and see whether the robot is able to perform all motions. If the digital twin test proves successful, you can run a test with the actual robot, but using HIL simulations for the gripper and sensors. This way, you can see how the robot performs the motions without needing the actual boxes. These steps allow you to tell a customer if his new product can be handled by the robot before he actually produced a single box. These examples show that using digital twin and hardware-in-the-loop simulations can speed up the robot development and integration process.  

A word of caution: while digital twin and a HIL simulation can give good indications of the performance of a system, they are only as good as the underlying model. If the underlying model does not represent the actual system accurately, the simulation results will not reflect the performance of the real system. Therefore, it is important to constantly improve the underlying models of the simulations to better reflect the actual system. As with all robot developments, continuous innovation is needed to ensure optimal performance of the system.  

[1] Saddik, A. El (April 2018). “Digital Twins: The Convergence of Multimedia Technologies”. IEEE MultiMedia. 25 (2): 87–92. doi:10.1109/MMUL.2018.023121167. ISSN 1070-986X.