Thursday, February 22, 2024
HomeTechnologyAI & Machine LearningA robot that can pick up objects and drop them in a...

A robot that can pick up objects and drop them in a desired location in an unfamiliar house

Credit: arXiv (2024). DOI: 10.48550/arxiv.2401.12202

A team of roboticists at New York University, working with a colleague from AI at Meta, has developed a robot that is capable of picking up designated objects in an unfamiliar room and placing them in a new designated location. In their paper posted on the arXiv preprint server, the team describes how the robot was programmed and how well it performed when tested in multiple real-word environments.

The researchers noted that visual language models (VLMs) have progressed a great deal over the past several years and have become very good at recognizing objects based on language prompts. They also pointed out that robot skills have improved as well—they can grasp things without breaking them, carry them to desired locations and set them down. But, thus far, little has been done to combine VLMs with skilled robots.

For this new study, the researchers attempted to do just this, with a robot sold by Hello Robot. It has wheels, a pole and retractable arms with claspers for hands. The research team gave it a previously trained VLM and dubbed it OK-Robot.

They then carried it to 10 volunteer homes where they created 3D videos using an iPhone and fed them to the robot to give it an overall feel for the layout of a given home. They then asked it to perform some simple moving tasks—”move the pink bottle on the shelf to the trash can,” for example.

In all, they asked the robot to carry out 170 such tasks—it was able to do them successfully 58% of the time. The researchers found they could improve its success rate to as high as 82% by decluttering the workspace.

The research team points out that their system uses a zero-shot algorithm, which means the robot was not trained in the environment in which it was working. They also suggest that the success rate they achieved proves that VLM-based robot systems are viable.

They suspect the success rate could be improved with tweaking and perhaps by using a more sophisticated robot. They conclude by suggesting that their work could be the first step toward advanced VLM-based robots.

More information:
Peiqi Liu et al, OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics, arXiv (2024). DOI: 10.48550/arxiv.2401.12202


Journal information:

© 2024 Science X Network


Post Disclaimer

The information provided in our posts or blogs are for educational and informative purposes only. We do not guarantee the accuracy, completeness or suitability of the information. We do not provide financial or investment advice. Readers should always seek professional advice before making any financial or investment decisions based on the information provided in our content. We will not be held responsible for any losses, damages or consequences that may arise from relying on the information provided in our content.


Most Popular

Recent Comments

error: Content is protected !!