Spatial masking for stitched images and surround view visualizations
In various examples, updates to a dynamic seam placement and/or fitted 3D bowl may be at least partially concealed using spatial masking. A future time in which a predicted change in dynamic seam placement and/or fitted 3D bowl exceeds some threshold may be determined, and a predicted dynamic seam movement and/or fitted 3D bowl update may be spatially masked by triggering a viewport switch to coincide with (a) the predicted dynamic seam placement and/or fitted 3D bowl update and/or (b) a relaxation or disabling of temporal filtering. Additionally or alternatively to predicting that a future change will exceed a threshold, the determination of the change may occur based on a change between a current and previous frame. In some embodiments that employ viewport switching to spatially mask visualization updates, the switch may be to one of a plurality of candidate viewports for an applicable scene maintained in a scene catalog.
1 . A processor comprising:
one or more processing units to:
determine a change exceeding a threshold to at least one of: (a) a dynamic seam placement that is based at least on one or more detected objects in an environment around an ego-object, or (b) a three-dimensional (3D) bowl that adaptively models the environment with a shape based at least on one or more distances to the one or more detected objects; and
generate a visualization representing the environment based at least on triggering a viewport change to coincide with the change to the at least one of the dynamic seam placement or the 3D bowl.
2 . The processor of claim 1 , wherein the triggering of the viewport change spatially masks the change to the at least one of the dynamic seam placement or the 3D bowl.
3 . The processor of claim 1 , the one or more processing units further to determine the change based at least on predicting that the change will occur in a future time slice corresponding to one or more processed intervals of sensor data used to generate the visualization representing the environment.
4 . The processor of claim 1 , the one or more processing units further to determine the change based at least on predicting one or more future states of the environment and determining one or more future time slices in which the dynamic seam placement or the 3D bowl for the one or more future states is predicted to differ from a current seam placement or a current 3D bowl by more than a threshold amount.
5 . The processor of claim 1 , the one or more processing units further to determine the change based at least on: tracking one or more positions of the one or more detected objects, using a detected trajectory of the ego-object to predict one or more future locations of the ego-object in one or more future time slices, and determining a seam placement or the 3D bowl for the one or more future time slices based at least on the one or more positions of the one or more detected objects and the one or more future locations of the ego-object.
6 . The processor of claim 1 , the one or more processing units further to at least partially conceal the change based at least on temporarily disabling temporal filtering that smooths one or more changes to a previous dynamic seam placement or a previous 3D bowl.
7 . The processor of claim 1 , the one or more processing units further to determine the change based at least on a difference between a current time slice and a preceding time slice, and mask the change in the visualization representing the environment for the current time slice.
8 . The processor of claim 1 , the one or more processing units further to maintain a scene catalog that maps scenes to candidate viewports, and mask the change in the visualization based at least on: identifying an applicable scene from the scene catalog and switching to a viewport selected from the candidate viewports for the applicable scene.
9 . The processor of claim 1 , wherein the processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
10 . A system comprising:
one or more processing units to:
determine a change exceeding a threshold to at least one of: (a) a dynamic seam placement that is based at least on one or more detected objects in an environment around an ego-object or (b) a three-dimensional (3D) bowl that adaptively models the environment with a shape based at least on one or more distances to the one or more detected objects; and
generate a visualization representing the environment based at least on triggering a viewport change to coincide with the change to the at least one of the dynamic seam placement or the 3D bowl.
11 . The system of claim 10 , the one or more processing units further to determine the change based at least on predicting that the change will occur in a future time slice corresponding to one or more processed intervals of sensor data used to generate the visualization representing the environment.
12 . The system of claim 10 , the one or more processing units further to determine the change based at least on predicting one or more future states of the environment and determining one or more future time slices in which the dynamic seam placement or the 3D bowl for the one or more future states is predicted to differ from a current dynamic seam placement or a current 3D bowl by more than a threshold amount.
13 . The system of claim 10 , the one or more processing units further to determine the change based at least on: tracking one or more positions of the one or more detected objects, using a detected trajectory of the ego-object to predict one or more future locations of the ego-object in one or more future time slices, and determining the dynamic seam placement or the 3D bowl for the one or more future time slices based at least on the one or more positions of the one or more detected objects and the one or more future locations of the ego-object.
14 . The system of claim 10 , the one or more processing units further to trigger the change based at least on temporarily preventing temporal filtering that smooths one or more changes to a previous dynamic seam placement or a previous 3D bowl.
15 . The system of claim 10 , the one or more processing units further to determine the change based at least on a difference between a current time slice and a preceding time slice, and at least partially conceal the change in the visualization representing the environment for the current time slice.
16 . The system of claim 10 , the one or more processing units further to maintain a scene catalog that maps scenes to candidate viewports, and trigger the viewport change based at least on: identifying an applicable scene from the scene catalog and switching to a viewport selected from the candidate viewports for the applicable scene.
17 . The system of claim 10 , wherein the system is comprised in at least one of a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
18 . A method comprising:
determining a change exceeding a threshold to at least one of: (a) a dynamic seam placement that is based at least on one or more detected objects in an environment around an ego-object or (b) a three-dimensional (3D) bowl that adaptively models the environment with a shape that is based at least on one or more distances to the one or more detected objects; and
generating a visualization representing the environment based at least on triggering a viewport change to coincide with the change to the at least one of the dynamic seam placement or the 3D bowl.
19 . The method of claim 18 , wherein the triggering of the viewport change spatially masks the change to the at least one of the dynamic seam placement or the 3D bowl.
20 . The method of claim 18 , wherein the method is performed by at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system for performing digital twin operations;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for generating synthetic data; or
a system implemented at least partially using cloud computing resources.