Data augmentation for model training

1 . One or more processors comprising:

one or more circuits to:

determine lighting information associated with a light distribution map obtained using a high dynamic range (HDR) panoramic view frame;

determine a virtual scene;

determine a position and orientation for an asset based at least on the virtual scene;

determine that the asset is overlapped by an object in the virtual scene, by at least a threshold amount, based on the position and orientation of the asset; and

render, on a frame, and based at least on the virtual scene, the asset being overlapped in the virtual scene, and the light distribution map, a shadow corresponding to the asset.

2 . The one or more processors of claim 1 , wherein, when rendering the asset and the shadow, the one or more circuits are to:

determine a random position and orientation for the asset based at least on the virtual scene, and

render the asset and the shadow based at least on the random position.

3 . The one or more processors of claim 1 , wherein, when rendering the asset and the shadow, the one or more circuits are to:

determine a region of interest based at least on the virtual scene;

determine a position and orientation for the asset based at least on the region of interest; and

render the asset and the shadow based at least on the position and orientation for the asset.

4 . The one or more processors of claim 1 , wherein, when rendering the asset and the shadow, the one or more circuits are to:

determine that the asset is occluded by the object in the frame based at least on the position and orientation for the asset and a position of the object in the virtual scene;

determine a second position and orientation for the asset based at least on determining that the asset is occluded in the frame; and

render the asset and the shadow based at least on the second position and orientation for the asset.

5 . The one or more processors of claim 1 , wherein, when rendering the asset and the shadow, the one or more circuits are to:

determine a second position and orientation for the asset based at least on determining that the asset is occluded in the frame; and

render the asset and the shadow based at least on the second position and orientation for the asset.

6 . The one or more processors of claim 1 , wherein, when determining the light distribution map, the one or more circuits are to:

generate a panoramic view frame based at least on the frame; and

determine the light distribution map based at least on the panoramic view frame and the HDR panoramic view frame, where a first portion of the light distribution map corresponds to at least a portion of the panoramic view frame, and where a second portion of the light distribution map corresponds to at least a portion of the HDR panoramic view frame.

7 . The one or more processors of claim 6 , wherein, when generating the panoramic view frame, the one or more circuits are to:

generate the panoramic view frame based at least on determining at least one of a set of intrinsic parameters or a set of extrinsic parameters associated with one or more cameras.

8 . The one or more processors of claim 6 , wherein, when generating the panoramic view frame, the one or more circuits are to:

determine that a set of frames satisfy an overlap threshold; and

generate the panoramic view frame based at least on determining that the set of frames satisfy the overlap threshold.

9 . The one or more processors of claim 1 , wherein, when determining the virtual scene, the one or more circuits are to:

determine the virtual scene to comprise a set of 3D cuboids or a neural radiance field (NeRF).

10 . The one or more processors of claim 1 , wherein the one or more processors is comprised in at least one of:

a control system for an autonomous or semi-autonomous machine;

a perception system for the autonomous or semi-autonomous machine;

a system for performing simulation operations;

a system for performing digital twin operations;

a system for performing light transport simulation;

a system for performing collaborative content creation for 3D assets;

a system for performing deep learning operations;

a system for presenting at least one of augmented reality content, virtual reality content, or mixed reality content;

a system for hosting one or more real-time streaming applications;

a system implemented using an edge device;

a system implemented using a robot;

a system for performing conversational AI operations;

a system that implements one or more large language models (LLMs);

a system for generating synthetic data;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

11 . A system comprising:

one or more processors to perform operations comprising:

determining lighting information associated with a light distribution map obtained using one or more high dynamic range (HDR) frames;

determining a virtual scene corresponding to an asset;

determining a position and orientation for the asset based at least on the virtual scene;

determining that the asset is overlapped by an object in the virtual scene, by at least a threshold amount, based at least on the position and orientation for the asset; and

rendering, within a frame, and based at least on the virtual scene, the asset being overlapped in the virtual scene, and the light distribution map, a shadow corresponding to the asset.

12 . The system of claim 11 , wherein, when rendering the asset and the shadow, the one or more processors perform the operations of:

determining a random position and orientation for the asset based at least on the virtual scene, and

rendering the asset and the shadow based at least on the random position and orientation.

13 . The system of claim 11 , wherein, when rendering the asset and the shadow, the one or more processors perform the operations of:

determining a region of interest based at least on the virtual scene;

determining a position and orientation for the asset based at least on the region of interest; and

rendering the asset and the shadow based at least on the position and orientation for the asset.

14 . The system of claim 11 , wherein, when rendering the asset and the shadow, the one or more processors perform the operations of:

determining that the asset is occluded by the object in the frame based at least on the position and orientation for the asset and a position of the object in the virtual scene;

determining a second position and orientation for the asset based at least on determining that the asset is occluded in the frame; and

rendering the asset and the shadow based at least on the second position and orientation for the asset.

15 . The system of claim 11 , wherein, when rendering the asset and the shadow, the one or more processors perform the operations of:

determining a second position and orientation for the asset based at least on determining that the asset is occluded in the frame; and

rendering the asset and the shadow based at least on the second position and orientation for the asset.

16 . The system of claim 11 , wherein, when determining the light distribution map, the one or more processors perform the operation of:

determining the light distribution map based at least on the frame and the one or more HDR frames, where a first portion of the light distribution map corresponds to at least a portion of the frame and a second portion of the light distribution map corresponds to at least a portion of the one or more HDR frames.

17 . The system of claim 11 , wherein, when determining the virtual scene, the one or more processors perform the operation of:

determining the virtual scene to comprise a set of 3D cuboids or a neural radiance field (NeRF).

18 . The system of claim 11 , wherein the determining of the virtual scene for the asset includes using the light distribution map to render the asset within the virtual scene.

19 . The system of claim 11 , wherein the system is comprised in at least one of:

a control system for an autonomous or semi-autonomous machine;

a perception system for an autonomous or semi-autonomous machine;

a system for performing simulation operations;

a system for performing digital twin operations;

a system for performing light transport simulation;

a system for performing collaborative content creation for 3D assets;

a system for performing deep learning operations;

a system implemented using an edge device;

a system implemented using a robot;

a system for performing conversational AI operations;

a system for generating synthetic data;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

20 . A method comprising:

estimating lighting information corresponding to one or more initial frames;

recreating a virtual scene corresponding to the one or more initial frames using the lighting information;

placing one or more assets within the virtual scene;

rendering one or more asset frames from within the virtual scene, at least one asset frame of the one or more asset frames including the one or more assets;

determining a position and orientation for the one or more assets based at least on the virtual scene;

determining that the one or more assets are overlapped by an object in the virtual scene, by at least a threshold amount, based at least on the position and orientation for the one or more assets; and

rendering an augmented frame including an original frame associated with the one or more initial frames augmented with the one or more assets from the one or more asset frames.