IP Library Granted Patent US 10,981,270
Granted Patent B1
US 10,981,270 · App. 16/530,711 · Granted Apr 20, 2021

Operating multiple testing robots based on robot instructions and/or environmental parameters received in a request

Inventors: Peter Pastor Sampedro (Oakland, CA); Mrinal Kalakrishnan (Mountain View, CA); Ali Yahya Valdovinos (Palo Alto, CA); Adrian Li (San Francisco, CA); Kurt Konolige (Menlo Park, CA); Vincent Dureau (San Francisco, CA)
Assignee: X DEVELOPMENT LLC
B25J9/0084B25J9/163
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,981,270
App. No.
16/530,711
Granted
Apr 20, 2021
Kind
B1
Abstract

Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.

Claims (42)

1. A method, comprising:

receiving, via one or more network interfaces, a request that includes environmental parameters and machine learning model properties;

wherein the machine learning model properties specify a machine learning model to be utilized in operation of a robot, and

wherein the environmental parameters indicate properties of environmental objects that are external to the robot and that are to be interacted with by the robot during operation of the robot based on the machine learning model;

based on receiving the request:

configuring a simulated operating space of a robot simulator based on the environmental parameters of the request so that the simulated operating space includes at least one simulated object having at least one of the properties of the environmental parameters, wherein the robot simulator is implemented on one or more computing devices;

causing the robot simulator to operate based on the machine learning model and with the simulated operating space that is configured based on the environmental parameters;

training the machine learning model, based on simulated data, to generate an updated machine learning model, wherein the simulated data is generated by the robot simulator during the operation of the robot simulator based on the machine learning model and with the simulated operating space that is configured based on the environmental parameters; and

providing, via one or more of the network interfaces, the updated machine learning model.

2. The method of claim 1 , wherein the properties indicated by the environmental parameters include one or more pose constraints for the environmental objects and wherein configuring the simulated operating space of the robot simulator based on the environmental parameters comprises:

positioning the simulated object in the simulated operating space based on at least one of the pose constraints.

3. The method of claim 1 , wherein the properties indicated by the environmental parameters include a class having a plurality of the environmental objects as members and wherein configuring the simulated operating space of the robot simulator based on the environmental parameters comprises:

selecting the simulated object based on the simulated object being a member of the class.

4. The method of claim 1 , wherein training the machine learning model, based on the simulated data, to generate the updated machine learning model, comprises performing backpropagation on the machine learning model based on training example output generated based on the simulated data.

5. The method of claim 1 , wherein the machine learning model is a neural network model.

6. A method, comprising:

receiving user interface input generated via a user interface input device of a client computing device;

generating, based on the user interface input, a request that includes robot instructions, properties of a machine learning model, and environmental parameters,

wherein the robot instructions in combination with the properties of the machine learning model specify at least part of operation of the robot, and

wherein the environmental parameters indicate one or more properties of environmental objects that are external to the robot and that are to be interacted with by the robot during operation of the robot based on the robot instructions;

transmitting the request via one or more network interfaces of the client computing device; and

receiving, by the client computing device or an additional computing device in response to the request:

an updated machine learning model that conforms to the properties of the request, and that is trained based on simulated data generated by a robot simulator operated based on the robot instructions and the properties of the request and with a simulated operating space configured based on the environmental parameters of the request.

7. The method of claim 6 , wherein the properties indicated by the environmental parameters include one or more pose constraints for the environmental objects and wherein the simulated operating space is configured based on at least one of the pose constraints.

8. The method of claim 6 , wherein the properties indicated by the environmental parameters include a class having a plurality of the environmental objects as members and wherein the simulated operating space is configured based on the class.

9. The method of claim 6 , wherein the robot instructions comprise:

object code that is executable by the robot simulator, or

source code, wherein a compilation of the source code is executable by the robot simulator.

10. The method of claim 6 , wherein the machine learning model is a neural network model.

11. A method, comprising:

receiving, by one or more processors via one or more network interfaces, a request that includes robot instructions and that specifies a machine learning model,

wherein the robot instructions in combination with the machine learning model specify at least part of operation of a robot;

operating a robot simulator based on the robot instructions and the machine learning model;

storing, by one or more of the processors in one or more computer readable media, simulated data generated by the robot simulator during operation of the robot simulator based on the robot instructions and the machine learning model;

training the machine learning model based on the simulated data generated by the robot simulator during operation of the robot simulator based on the robot instructions and the machine learning model, wherein training the machine learning model creates an updated machine learning model; and

providing the updated machine learning model in response to the request.

12. The method of claim 11 , wherein the request further includes environmental parameters, and further comprising:

configuring a simulated operating space of the robot simulator based on the environmental parameters of the request.

13. The method of claim 11 , wherein the environmental parameters indicate one or more pose constraints for environmental objects simulated in the simulated operating space.

14. The method of claim 11 , wherein the environmental parameters indicate a class for environmental objects simulated in the simulated operating space.

15. The method of claim 11 , wherein training the machine learning model based on the simulated data generated by the robot simulator comprises performing backpropagation on the machine learning model.

16. The method of claim 15 , wherein the machine learning model is a neural network model.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2025
From: GOOGLE LLC
To: GDM HOLDING LLC
Reel/Frame 071109/0342 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2023
From: X DEVELOPMENT LLC
To: GOOGLE LLC
Reel/Frame 063992/0371 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 2, 2019
From: PASTOR SAMPEDRO, PETER; KALAKRISHNAN, MRINAL; VALDOVINOS, ALI YAHYA; LI, ADRIAN; KONOLIGE, KURT; DUREAU, VINCENT
To: GOOGLE INC.
Reel/Frame 049946/0786 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 2, 2019
From: GOOGLE INC.
To: X DEVELOPMENT LLC
Reel/Frame 049949/0218 →
Cited By (1)
US 12,589,494