Integrating Machine Learning and Motion Planning Techniques for Multi-Object Search in Unknown Household Environments

Abstract

Robot navigation and object search within household environments are foundational tasks in the robotics community, yet made challenging as robots will often find themselves without a full map of the environment. The difficulty of these tasks increases when robots need to locate multiple objects or complete complex multi-stage objectives. To address this, we use a framework that translates complex input specifications into a structured sequence of goals and actions, which the robot uses to systematically search for objects in a specified order via an algorithm called PO-TLP. This approach allows for planning despite missing knowledge, leveraging "exploratory actions'' that define opportunities for the robot to discover necessary objects. Further, the PO-TLP planner integrates learning to predict where objects are likely to be found in unknown space and generates an efficient plan to look for missing task-relevant objects.  Our research connects this PO-TLP algorithm to household domains by applying it to ProcTHOR, a tool for procedural generation of realistic home-like environments. To validate our approach, we performed a comprehensive analysis of planning costs associated with various PO-TLP implementations and their success in real-world scenarios. Our research applies an existing planning framework to a new application of open-set object search in household environments. By improving the performance and reliability of robotic systems in this domain, our work opens the door to new capabilities that were previously challenging in household domains, potentially leading to advancements in household robotics, automation, and beyond.