System and method for training and utilizing an object-centric occupancy estimation model
A method of updating parameters of an occupancy estimation model. The method includes receiving images that are two-dimensional. An occupancy estimation model is utilized to generate a voxel based on the images. An occupancy loss is determined by comparing the voxel to an occupancy ground truth corresponding to the voxel. An object loss is determined by comparing the voxel to an object ground truth. The object loss is combined with the occupancy loss to determine a total loss for the voxel. Parameters of the occupancy estimation model are updated to reduce the total loss determined.
1 . A method of updating parameters of an occupancy estimation model, the method comprising:
receiving a plurality of images, wherein the plurality of images are two-dimensional;
utilizing the occupancy estimation model to generate a voxel based on the plurality of images;
determining an occupancy loss by comparing the voxel to an occupancy ground truth corresponding to the voxel;
determining an object loss by comparing the voxel to an object ground truth;
combining the object loss with the occupancy loss to determine a total loss for the voxel; and
updating parameters of the occupancy estimation model to reduce the total loss determined.
2 . The method of claim 1 , including applying a weight to the object loss when combined with the occupancy loss to determine the total loss.
3 . The method of claim 1 , wherein comparing the voxel to the occupancy ground truth is performed on a voxel-wise cross-entropy basis.
4 . The method of claim 1 , wherein the object loss is multiplied by a weight before being combined with the occupancy loss to determine the total loss.
5 . The method of claim 1 , wherein the object loss includes an object detection loss and utilizing an object detection model to identify detected objects in the voxel.
6 . The method of claim 5 , wherein the object detection loss includes comparing an object ground truth for the voxel to the detected objects.
7 . The method of claim 6 , wherein comparing the object ground truth for the voxel to the detected objects includes at least one of cross-entropy or shape regression.
8 . The method of claim 1 , wherein the object loss includes an object fullness loss and determining the object fullness loss includes comparing a set of occupied voxels defined within a bounding box from the object ground truth compared to a probability of occupancy estimated by the occupancy estimation model.
9 . A non-transitory computer-readable storage medium embodying programmed instructions which, when executed by a processor, are operable for performing a method comprising:
receiving a plurality of images, wherein the plurality of images are two-dimensional;
utilizing an occupancy estimation model to generate a voxel based on the plurality of images;
determining an occupancy loss by comparing the voxel to an occupancy ground truth corresponding to the voxel;
determining an object loss by comparing the voxel to an object ground truth;
combining the object loss with the occupancy loss to determine a total loss for the voxel; and
updating parameters of the occupancy estimation model to reduce the total loss determined.
10 . The computer-readable storage medium of claim 9 , wherein the method includes applying a weight to the object loss when combining with the occupancy loss to determine the total loss.
11 . The computer-readable storage medium of claim 9 , wherein comparing the voxel to the occupancy ground truth is performed on a voxel-wise cross-entropy basis.
12 . The computer-readable storage medium of claim 9 , wherein the object loss is multiplied by a weight before being combined with the occupancy loss to determine the total loss.
13 . The computer-readable storage medium of claim 9 , wherein the object loss includes an object detection loss and utilizing an object detection model to identify detected objects in the voxel.
14 . The computer-readable storage medium of claim 13 , wherein the object detection loss includes comparing an object ground truth for the voxel to the detected objects.
15 . The computer-readable storage medium of claim 14 , wherein comparing the object ground truth for the voxel to the detected objects includes at least one of cross-entropy or shape regression.
16 . The computer-readable storage medium of claim 9 , wherein the object loss includes an object fullness loss and determining the object fullness loss includes comparing a set of occupied voxels defined within a bounding box from the object ground truth compared to a probability of occupancy estimated by the occupancy estimation model.
17 . A vehicle system comprising:
at least one optical sensor and a controller in data communication with the at least one optical sensor, wherein the controller configured to:
receive a plurality of images, wherein the plurality of images are two-dimensional optical images;
perform an occupancy estimation on the plurality of images with an occupation estimation model to generate a voxel;
perform a three-dimensional object detection on the plurality of images to identify at least one region associated with an object of interest; and
vary a probability threshold for identifying an object in the voxel based on the three-dimensional object detection, wherein the probability threshold for identifying objects is reduced for a portion of the voxel that corresponds to the at least one region associated with the object of interest when compared to the probability threshold for identifying the objects outside of the at least one region associated with the object of interest.
18 . The vehicle system of claim 17 , wherein the occupation estimation model is trained by:
receiving a plurality of training images, wherein the plurality of training images are two-dimensional;
utilizing an occupancy estimation model to generate a voxel based on the plurality of training images;
determining an occupancy loss by comparing the voxel to an occupancy ground truth corresponding to the voxel;
determining an object loss by comparing the voxel to an object ground truth;
combining the object loss with the occupancy loss to determine a total loss for the voxel; and
updating parameters of the occupancy estimation model to reduce the total loss determined.
19 . The vehicle system of claim 18 , wherein
the object loss includes an object detection loss and utilizing an object detection model to identify detected objects in the voxel;
the object detection loss includes comparing an object ground truth for the voxel to the detected objects; and
comparing the object ground truth for the voxel to the detected objects includes at least one of cross-entropy or shape regression.
20 . The vehicle system of claim 18 , wherein the object loss includes an object fullness loss and determining the object fullness loss includes comparing a set of occupied voxels defined within a bounding box from the object ground truth compared to a probability of occupancy estimated by the occupancy estimation model.