IP Library Granted Patent US 11,170,220
Granted Patent B2
US 11,170,220 · App. 16/742,526 · Granted Nov 9, 2021

Delegation of object and pose detection

Inventors: Nareshkumar Rajkumar (Cupertino, CA); Stefan Hinterstoisser (Munich, DE)
Assignee: X DEVELOPMENT LLC
G06K9/00664G06K9/00979G06K9/6296G06T7/33G06T7/73G06T2207/10021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,170,220
App. No.
16/742,526
Granted
Nov 9, 2021
Kind
B2
Abstract

Methods, apparatus, systems, and computer-readable media are provided for delegating object type and/or pose detection to a plurality of “targeted object recognition modules.” In some implementations, a method may be provided that includes: operating an object recognition client to facilitate object recognition for a robot; receiving, by the object recognition client, sensor data indicative of an observed object in an environment; providing, by the object recognition client, to each of a plurality of remotely-hosted targeted object recognition modules, data indicative of the observed object; receiving, by the object recognition client, from one or more of the plurality of targeted object recognition modules, one or more inferences about an object type or pose of the observed object; and determining, by the object recognition client, information about the observed object, such as its object type and/or pose, based on the one or more inferences.

Claims (35)

1. A method implemented using one or more processors, comprising:

operating an object recognition client to facilitate object recognition for a robot;

receiving, by the object recognition client, sensor data indicative of an observed object in an environment;

generating, by the object recognition client, a soft classifier of a type of the observed object, wherein the soft classifier is generated based on both the sensor data and a type of location in which the object is observed;

providing, by the object recognition client, to a root object recognition server operating remotely from the robot, data indicative of the observed object, wherein the data indicative of the observed object includes the soft classifier;

receiving, by the object recognition client, from one or more targeted object recognition modules selected by the root object recognition server based on the soft classifier, one or more inferences about an object type or pose of the observed object; and

determining, by the object recognition client, information about the observed object based on the one or more inferences.

2. The method of claim 1 , wherein the type of location in which the object is observed is obtained from an offline resource that is available to the robot while the robot is not in network communication with the root object recognition server.

3. The method of claim 1 , wherein the soft classifier is generated further based on a location of the robot.

4. The method of claim 1 , wherein the soft classifier is generated further based on a time of day at the robot's location.

5. The method of claim 1 , wherein the soft classifier is generated further based on an attribute of a task being performed by the robot.

6. The method of claim 1 , wherein the one or more processors are integral with the robot.

7. The method of claim 1 , wherein the one or more processors are communicatively coupled with the robot via a local area or personal area network.

8. A robot comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:

operate an object recognition client to facilitate object recognition for the robot;

receive, by the object recognition client, sensor data indicative of an observed object in an environment;

generate, by the object recognition client, a soft classifier of a type of the observed object, wherein the soft classifier is generated based on the sensor data and a type of location in which the object is observed;

provide, by the object recognition client, to a root object recognition server operating remotely from the robot, data indicative of the observed object, wherein the data indicative of the observed object includes the soft classifier;

receive, by the object recognition client, from one or more targeted object recognition modules selected by the root object recognition server based on the soft classifier, one or more inferences about an object type or pose of the observed object; and

determine, by the object recognition client, information about the observed object based on the one or more inferences.

9. The robot of claim 8 , wherein the type of location in which the object is observed obtained from an offline resource that is available to the robot while the robot is not in network communication with the root object recognition server.

10. The robot of claim 8 , wherein the soft classifier is generated further based on a location of the robot.

11. The robot of claim 8 , wherein the soft classifier is generated further based on a time of day at the robot's location.

12. The robot of claim 8 , wherein the soft classifier is generated further based on an attribute of a task being performed by the robot.

13. The robot of claim 8 , wherein the one or more processors are integral with the robot.

14. The robot of claim 8 , wherein the one or more processors are communicatively coupled with the robot via a local area or personal area network.

15. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:

operating an object recognition client to facilitate object recognition for a robot;

receiving, by the object recognition client, sensor data indicative of an observed object in an environment;

generating, by the object recognition client, a soft classifier of a type of the observed object, wherein the soft classifier is generated based on the sensor data and a type of location in which the object is observed;

providing, by the object recognition client, to a root object recognition server operating remotely from the robot, data indicative of the observed object, wherein the data indicative of the observed object includes the soft classifier;

receiving, by the object recognition client, from one or more targeted object recognition modules selected by the root object recognition server based on the soft classifier, one or more inferences about an object type or pose of the observed object; and

determining, by the object recognition client, information about the observed object based on the one or more inferences.

16. The at least one non-transitory computer-readable medium of claim 15 , wherein the type of location in which the object is observed is obtained from an offline resource that is available to the robot while the robot is not in network communication with the root object recognition server.

17. The at least one non-transitory computer-readable medium of claim 15 , wherein the soft classifier is generated further based on a location of the robot.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2025
From: GOOGLE LLC
To: GDM HOLDING LLC
Reel/Frame 071109/0342 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2023
From: X DEVELOPMENT LLC
To: GOOGLE LLC
Reel/Frame 063992/0371 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 4, 2020
From: RAJKUMAR, NARESHKUMAR; HINTERSTOISSER, STEFAN
To: GOOGLE INC.
Reel/Frame 052558/0681 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 4, 2020
From: GOOGLE INC.
To: X DEVELOPMENT LLC
Reel/Frame 052558/0739 →