Can A Robot Be Used To Locate Misplaced Items?

MIT researchers have developed a robotic system capable of sorting through clutter to locate an object.

Researchers at the Massachusetts Institute of Technology have built a fully integrated robotic arm capable of locating and retrieving lost things by fusing visual data from a camera and radio frequency (RF) information from an antenna. It is capable of locating objects that are buried or hidden from view. The system is termed RFusion, and the prototype makes use of small RFID tags that are battery-free and reflect signals transmitted by an antenna. Because radio frequency signals can pass through most materials, RFusion can locate a tagged item even if it is buried.

According to the researchers, RFusion may be used to sort through heaps in a warehouse to fulfil orders or identify and install components in an auto manufacturing plant.

How does RFusion function?

The method makes use of commercially available RFID tags connected to target products. It begins by querying the RFID on the target item selectively using its wrist-mounted antenna. It then utilises our state-of-the-art RFID positioning technology to compute the round-trip distance to the tag based on the RFID’s measured response. Due to the fact that a single round-trip distance is insufficient for localisation, the robot combines RF and optical information to efficiently locate, navigate toward, and grab the target object. It works in three stages.


Sign up for your weekly dose of what's up in emerging technology.

1) RF-Visual Geometric Fusion: The robot maps the round-trip distance to the RFID to a spherical ring centred on the wrist-mounted antenna. It then intersects this spherical ring geometrically with the RGB-D data collected from the wrist-mounted camera to generate a list of probable places.

2) RF-Visual Reinforcement Learning: Following that, the robot’s gripper must be relocated to take additional RF and optical data. To accomplish this, we built a reinforcement learning network that uses the history of RF and visual measurements to estimate the gripper’s ideal future vantage position. The robot then moves to this new place, takes measurements, and repeats this process until it is confident in the estimated location of the RFID-tagged item.

3) RF-Visual Grasping: Once an object’s location has been calculated with sufficient accuracy, RFusion uses the location estimate to grasp it. Following clutching, the wrist-mounted antenna can perform another RFID measurement to confirm that the target object was really gripped successfully, and if not, to attempt additional grasps until the item of interest is picked up.


The researchers trained a neural network using reinforcement learning to optimise the robot’s course to an object.

Tara Boroushaki, a research assistant at Signal Kinetics, MIT Media Lab, states, “This is also how our brain acquires knowledge. We are rewarded by our professors, our parents, or a computer game, among others. In reinforcement learning, the same phenomenon occurs.”

When the number of moves required to localise the item and the distance travelled to pick it up was reduced, the optimisation process in RFusion was awarded.


The researchers evaluated the system in a variety of scenarios, including one in which a keychain was buried in a cluttered box, and a remote control was concealed beneath a mound of objects on a couch. To avoid overwhelming the system, they summarised the RF measurements and limited the visual data to the region directly in front of the robot. This resulted in a success rate of 96 per cent for retrieving objects completely hidden beneath a pile.

According to Matt S. Reynolds, co-founder of CoMotion and an associate professor of electrical and computer engineering at the University of Washington, “The RFusion approach paves the way for autonomous robots that can sift through a pile of mixed goods and sort them based on the data stored in the RFID tags. The RFusion technique is a significant step forward for robotics functioning in complex supply chains, where rapidly and precisely recognising and ‘picking’ the correct item is critical to fulfilling orders on time and keeping demanding consumers happy.”

The researchers intend to improve the system’s speed in the future, rather than frequently halting to take measurements. This would permit the employment of RFusion in a fast-paced industrial or warehouse environment. Additionally, the researchers stated that, in addition to its industrial applications, such a system might be included in future smart homes to assist residents with various daily duties.

More Great AIM Stories

Dr. Nivash Jeevanandam
Nivash holds a doctorate in information technology and has been a research associate at a university and a development engineer in the IT industry. Data science and machine learning excite him.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM