Generating simulation scenarios and digital media using digital twins
A computer-implemented method includes receiving a request for generating a simulation scenario or digital media that includes one or more entities and one or more assets in an environment, the request including contextual information. The method further includes selecting three-dimensional (3D) digital twins based on the contextual information, wherein one or more of the 3D digital twins correspond to the assets, one or more of the 3D digital twins correspond to the one or more entities, and one of the 3D digital twins corresponds to the environment. The method further includes generating the simulation scenario that initializes the 3D digital twins and establishes a spatial relationship between the 3D digital twins based on the contextual information. The method further includes controlling behavior of the 3D digital twins in the simulation scenario based on the contextual information.
1. A computer-implemented method comprising:
performing by one or more processors, operations including:
receiving a request for generating a simulation scenario or digital media that includes one or more entities and one or more assets in an environment, the request including contextual information;
selecting three-dimensional (3D) digital twins based on the contextual information, wherein one or more of the 3D digital twins correspond to the assets, one or more of the 3D digital twins correspond to the one or more entities, and one of the 3D digital twins corresponds to the environment;
generating the simulation scenario that initializes the 3D digital twins and establishes a spatial relationship between the 3D digital twins based on the contextual information; and
controlling behavior of the 3D digital twins in the simulation scenario based on the contextual information.
2. The method of claim 1 , further comprising:
generating, with one or more virtual sensors, the digital media based on the simulation scenario.
3. The method of claim 2 , further comprising:
providing the digital media and style reference media as input to a style transfer machine-learning model; and
outputting, with the style transfer machine-learning model, stylized digital media that reflects the digital media in a style of the style reference media.
4. The method of claim 2 , wherein the assets are products, the digital media is a digital media advertisement and further comprising:
providing a link of the digital media advertisement to a web client.
5. The method of claim 4 , wherein the digital media is delivered in real time via a hypertext transfer protocol secure (HTTPS) media request.
6. The method of claim 1 , wherein the request includes a scripted prompt that describes the contextual information and further comprising:
providing the scripted prompt as input to a large language model (LLM); and
outputting, with the LLM, the simulation scenario based on the contextual information in the scripted prompt.
7. The method of claim 1 , further comprising:
providing training set data from the simulated scenario that includes the contextual information to a machine-learning model; and
training the machine-learning model based on the training set data to output additional simulated scenarios based on receiving other contextual information as input.
8. The method of claim 1 , wherein the request for generating the simulation scenario is for testing a robotic system and the simulation scenario includes a corresponding digital twin of the robotic system in the digital twin of the environment that predicts behavior of the robotic system in a real-world environment.
9. The method of claim 8 , wherein the robotic system is an autonomous system and the contextual information includes sensor data of a real-world event that occurred involving the autonomous system.
10. The method of claim 8 , wherein the simulation scenario includes multiple robotics systems, human operated systems, and simulated entities in a coordinated workflow or process.
11. A device comprising:
one or more processors; and
a memory coupled to the one or more processors, with instructions stored thereon that, when executed by the processor, cause the processor to perform operations comprising:
receiving a request for generating a simulation scenario or digital media that includes one or more entities and one or more assets in an environment, the request including contextual information;
selecting three-dimensional (3D) digital twins based on the contextual information, wherein one or more of the 3D digital twins correspond to the assets, one or more of the 3D digital twins correspond to the one or more entities, and one of the 3D digital twins corresponds to the environment;
generating the simulation scenario that initializes the 3D digital twins and establishes a spatial relationship between the 3D digital twins based on the contextual information; and
controlling behavior of the 3D digital twins in the simulation scenario based on the contextual information.
12. The device of claim 11 , wherein the operations further comprise:
generating, with one or more virtual sensors, the digital media based on the simulation scenario.
13. The device of claim 12 , wherein the operations further comprise:
providing the digital media and style reference media as input to a style transfer machine-learning model; and
outputting, with the style transfer machine-learning model, stylized digital media that reflects the digital media in a style of the style reference media.
14. The device of claim 12 , wherein the assets are products, the digital media is a digital media advertisement, and the operations further comprise:
providing a link of the digital media advertisement to a web client.
15. The device of claim 14 , wherein the digital media is delivered in real time via a hypertext transfer protocol secure (HTTPS) media request.
16. A non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more computers, cause the one or more computers to perform operations, the operations comprising:
receiving a request for generating a simulation scenario or digital media that includes one or more entities and one or more assets in an environment, the request including contextual information;
selecting three-dimensional (3D) digital twins based on the contextual information, wherein one or more of the 3D digital twins correspond to the assets, one or more of the 3D digital twins correspond to the one or more entities, and one of the 3D digital twins corresponds to the environment;
generating the simulation scenario that initializes the 3D digital twins and establishes a spatial relationship between the 3D digital twins based on the contextual information; and
controlling behavior of the 3D digital twins in the simulation scenario based on the contextual information.
17. The computer-readable medium of claim 16 , wherein the operations further comprise:
generating, with one or more virtual sensors, the digital media based on the simulation scenario.
18. The computer-readable medium of claim 17 , wherein the operations further comprise: providing the digital media and style reference media as input to a style transfer machine-learning model; and
outputting, with the style transfer machine-learning model, stylized digital media that reflects the digital media in a style of the style reference media.
19. The computer-readable medium of claim 17 , wherein the assets are products, the digital media is a digital media advertisement, and the operations further comprise:
providing a link of the digital media advertisement to a web client.
20. The computer-readable medium of claim 19 , wherein the digital media is delivered in real time via a hypertext transfer protocol secure (HTTPS) media request.