Robot learning

1 . A method comprising:

coordinating, using a machine learned robotic control application, one or more robots to perform a mission;

obtaining event information from the one or more robots during performance of the mission by the one or more robots, the event information including (1) contextual information generated based at least on the one or more robots locally processing sensor data and (2) robot state information indicating a current state of the one or more robots;

storing the event information in a log;

replaying a sequence of events from the log based on an order and cadence in which the events were stored in the log to simulate generating one or more decisions to control the one or more robots; and

updating the machine learned robotic control application based at least on results of the replaying.

2 . The method of claim 1 , wherein the performing the one or more operations to simulate the generating the one or more decisions comprises simulating the event information being received again.

3 . The method of claim 1 , wherein the one or more determinations include one or more objects being detected based at least on the sensor data.

4 . The method of claim 1 , wherein the event information indicates at least one of one or more actions performed using the one or more robots, one or more states of the one or more robots, or one or more reasons that one or more events occurred.

5 . The method of claim 1 , wherein the obtaining and storing operations are performed using a communication interface between the one or more robots and a robotics control application that generates the one or more decisions to control the one or more robots.

6 . The method of claim 1 , wherein the one or more operations to simulate the generating the one or more decisions begins from an initial state of the one or more robots.

7 . The method of claim 1 , wherein the one or more operations to simulate the generating the one or more decisions begins from a state of the one or more robots subsequent to an initial state of the one or more robots.

8 . The method of claim 1 , further comprising modifying a robotics control application that generates the one or more decisions based at least on the event information.

9 . The method of claim 1 , further comprising performing one or more operations to train a machine learning model based at least on the event information.

10 . The method of claim 1 , further comprising detecting an error during the mission based at least on the event information and a trained machine learning model.

11 . The method of claim 1 , further comprising receiving additional event information generated using one or more actors participating in the mission, wherein the one or more actors are not robots.

12 . The method of claim 1 , wherein the method is performed by a processor comprised in at least one of:

an infotainment system for an autonomous or semi-autonomous machine;

a system for performing one or more simulation operations;

a system for performing one or more digital twin operations;

a system for performing light transport simulation;

a system for performing collaborative content creation for 3D assets;

a system for performing one or more deep learning operations;

a system implemented using an edge device;

a system implemented using the one or more robots;

a system for generating or presenting virtual reality, augmented reality, or mixed reality content;

a system for performing one or more conversational AI operations;

a system implementing one or more large language models (LLMs);

a system for generating synthetic data;

a system for performing one or more generative AI operations;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

13 . A method comprising:

receiving, at a communication interface between one or more robots and a robotics control application, event information including (1) contextual information derived from sensor data generated based at least on the one or more robots locally processing sensor data and (2) robot state information indicating a current state of the one or more robots;

storing the event information in a log;

replaying a sequence of events from the log at the one or more robots, via the communication interface, during one or more simulations, based on an order and cadence in which the events were stored in the log; and

updating the robotics control application based at least on results of the replaying.

14 . The method of claim 13 , wherein the one or more determinations include one or more objects being detected based at least on sensor data.

15 . The method of claim 13 , wherein the event information indicates at least one of one or more actions performed using the one or more robots, one or more states of the one or more robots, or one or more reasons that one or more events occurred.

16 . The method of claim 13 , further comprising modifying the robotics control application based at least on the event information.

17 . The method of claim 13 , further comprising performing one or more operations to train a machine learning model to detect errors based at least on the event information.

18 . The method of claim 13 , further comprising performing one or more operations to train a machine learning model to generate missions for controlling one or more robots based at least on the event information.

19 . The method of claim 18 , wherein the machine learning model comprises a large language model (LLM).

20 . A system comprising:

at least one memory having executable instructions stored thereon; and

one or more processors configured to execute the executable instructions in order to cause the system to:

replay, at one or more robots, logged event information being received at a communication interface between the one or more robots and a robotics control application, wherein the logged event information includes (1) contextual information generated based at least on the one or more robots locally processing sensor data and (2) robot state information indicating a current state of the one or more robots; and

update a machine learned robotic control application based at least on results of the replaying.