When developing intelligent robots, you want them to be able to cope with different types of environments while keeping a high performance. Therefore, it is necessary to use strategies that are not case specific but instead allow robots to have a good degree of generalization on the desired task and to handle variance in their working environment. This can be achieved with the help of machine learning. This blog discusses the concept of machine learning and how it can be beneficial for robotics.
What is machine learning?
Machine learning is the field of study which aims at empowering systems with the ability to perform certain tasks without using hard coded rules and/or priors. Instead, all the rules necessary to solve the desired task will be derived directly from available data relevant for the matter. The machine or system performs progressively better – learns – by repeating the same task over and over with small changes from try to try. With every iteration, the mathematical model is updated in the direction that improves the performance on the desired task.
Despite having existed for a long time, machine learning techniques only became omnipresent recently due to big developments in the available computational power. To understand how useful machine learning can be in the context of robotics, an item picking robot will be used as an example.
In previous blogs we have explained the concepts of the Application Decision Engine (ADE) and decision tree. Basically, the ADE allows for developing robot applications where the expected behavior of the robot is given by a decision tree, making it deterministic. Considering an oversimplified decision tree of an item picking robot, the three main states of the robot will be:
- Determine whether an item can be picked from the ‘objects’ box.
- Pick an item
- Place an item
From hard coding towards machine learning
Although a human can do any of the above tasks without any problem, the same does not apply to robots. ‘Reverse engineering’ human behavior is extremely difficult and complex. Items come in all sorts of shapes and orientation, the environment of the robot can dramatically change in a short amount of time and the amount of variation is extremely large!
A classic way to solve this problem would be to develop an algorithm for each task based on a set of hard coded rules. For instance, to place an item, one could set priors to define the optimal place location, such as distance to the center, object size and final height. Ideally, such rules should be broad enough to cover all the possible items the robot would have to handle. This is not only difficult, but also rather inefficient and far from optimal.
What if it is possible for the robot to learn how to pick an item and find or place items at target locations the same way humans do? Better, what if it could learn directly from humans, mimicking their behavior? This would add a level of abstraction allowing the developer to not have to think about how to encode the entire world! Instead, the robot would simply ‘learn’ how to do it.
Machine learning strategies
This is where machine learning comes in. There several different strategies that can be used for machine learning. The most common ones are:
- Supervised Learning:
The system learns based on input data that has an associated expected output. Then, by computing the error between expected output and the actual output of the system, it is possible to continuously improve performance. For instance, an item picking robot could learns by receiving images in which the existing items are labeled. By analyzing the same images multiple times, the model learns to understand where certain items are placed based on an image.
- Unsupervised learning:
The system learns only from input data. It is quite useful to find hidden relationships between the input samples. For example, it can be used if one wants to classify the types of objects the robot is picking, based only on the product features that are available.
- Reinforcement learning:
The system learns by interacting directly with the environment. Positive behaviors are reinforced, while negative behaviors are penalized to avoid being selected again. For instance, a robot can learn how to pick an item based on trying it multiple times and receiving feedback about its performance each time. A very interesting example can be found at https://openai.com/blog/learning-dexterity/.
Implementing machine learning strategies within robotics can be very useful. However, there also some downfalls that should be taken into account:
- Machine learning requires large amounts of data:
To achieve a good level of generality, it is necessary to collect large amounts of data. This can be quite expensive in terms of time, especially if the data needs to be annotated, e.g. with supervised learning techniques.
- The learning procedure can take a long time:
Finding the correct parameters for the model is usually a long and slow procedure and quite dependent on the power of the used hardware
- Less understanding of the decisions made by the system:
Since the rules to perform a certain task are derived directly from the data, it might be difficult to visualize and understand the ‘why’ for decisions made by the system
Fortunately, the frameworks used to implement machine learning techniques, such as TensorFlow, are becoming more and more robust. Building and training a model becomes a smoother and faster process. At the same time, more powerful Graphical Processing Units (GPUs) arrive at the market, speeding up training times and bringing machine learning to real-time systems.
The key to successful machine learning
Hence, they key to successfully using machine learning in robotic applications is to do a trade-off between the complexity of the task and the amount of time available to create a solution. Due to the expected long time required for data collection and labeling, training and deployment, machine learning should be employed mostly for relatively complex tasks, where the abstraction needed to come up with robust solutions is high.
This blog post has shown how machine learning can be used to add a layer of abstraction to robot application development and to remove the need to explicitly use instructions. This is extremely useful when dealing with problems where there is no clear and systematic approach for solving them. Machine learning already plays an important role in the world of robotics, empowering the robots with more humanoid skills to solve tasks. Continuous improvement of these skills will lead to a world where robotics can easily be implemented in many different ways! Flexibility is key.
The machine learning images are generated using the Keras implementation of MaskRCNN object detection from our development partner Fizyr.