One of the main challenges in robotics is the precise estimation of the position and orientation of objects located in a three-dimensional space. This task in known as 6D pose estimation and is key to enable the manipulation of objects, navigation, and interaction. Many challenges arise due to variabilities in lighting, partially occluded objects, indistinct textures, and sensor noise in real life scenarios. Each of these issues hinders the robotic capabilities for flexible manipulation.

This task can be further divided into two domains: pose estimation of known objects and the pose estimation of unknown objects. An object is known if properties are defined before the execution of the task, generally through a CAD model.

The HARTU project addresses these challenges by implementing powerful state-of-the-art vision hardware and software systems. The 6D pose estimation of known objects is explored with a mixed approach that leverages both classical algorithms, such as fast point feature histograms and iterative closest point, and innovative neural networks architectures to achieve millimetric-range precision. This approach is used in TOFAS kitting operation, PCL line feeding, and INFAR fixtureless assembly use cases, where the objects to manipulate are limited and known.

In the unknown object domain, classical best fit algorithms are enhanced by neural network models to classify geometric primitives, like planes, cylinders or cubes, to robustly estimate the position and orientation of a wide array of objects. This approach is used in TOFAS order preparation, TCA vegetables packing, and ULMA order preparation. In each of these scenarios, the objects to handle are unknown and can attain many different shapes and sizes.

These solutions have been built as blocks for the HARTU App Manager using ROS2. This implementation facilitates the deployment of our solutions into each use case and the integration alongside the developments made by each partner.

Thanks to these developments, the system solution provided by the HARTU project will be capable of handling objects ranging from vegetables in a packing line to consumer goods parts in a manufacturing line.