Excluding objects during 3D model generation

1. A method comprising:

at an electronic device having a processor:

obtaining depth data of an environment, the depth data comprising depth values for pixels of a first image, the environment comprising an object;

obtaining a segmentation mask associated with a second image of the environment, the segmentation mask identifying a portion of the second image associated with the object;

generating a 3D model of the environment that excludes the object based on the depth data and the segmentation mask, wherein the 3D model excludes the object based on excluding depth data associated with the identified portion of the second image associated with the object; and

updating the 3D model based on multiple depth images and corresponding segmentation masks obtained over a period of time, wherein the updating of the 3D model comprises updating a first portion of the 3D model corresponding to the identified portion associated with the object at a higher rate than a second portion of the 3D model.

2. The method of claim 1 , wherein the first image and the second image each comprise a plurality of pixel locations, wherein each pixel in the first image and each pixel in the second image are located at one of the plurality of pixel locations, wherein pixel locations in the first image are spatially correlated and are aligned with pixel locations in the second image.

3. The method of claim 1 , wherein the object the segmentation mask identifies in the portion of the second image is associated with a person.

4. The method of claim 1 , wherein the object the segmentation mask identifies in the portion of the second image is associated with an animal.

5. The method of claim 1 , wherein the object the segmentation mask identifies in the portion of the second image is associated with motion.

6. The method of claim 1 , wherein the segmentation mask is generated by:

determining whether to associate a category from a set of categories for each pixel in the second image based on characteristics the pixel exhibits; and

determining, for each determined pixel in the second image that is associated with a category, a confidence value based on the characteristics the pixel exhibits in the second image.

7. The method of claim 1 , wherein the updating is based on determining motion of the object.

8. The method of claim 1 , wherein the segmentation mask uses a machine learning model that uses a representation of the second image as input.

9. The method of claim 8 , wherein the machine learning model is a neural network configured to be executed by a neural engine/circuits on a processor chip tuned to accelerate artificial intelligence software.

10. A device comprising:

a non-transitory computer-readable storage medium; and

one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the device to perform operations comprising:

obtaining depth data of an environment, the depth data comprising depth values for pixels of a first image, the environment comprising an object;

obtaining a segmentation mask associated with a second image of the environment, the segmentation mask identifying a portion of the second image associated with the object;

generating a 3D model of the environment that excludes the object based on the depth data and the segmentation mask, wherein the 3D model excludes the object based on excluding depth data associated with the identified portion of the second image associated with the object; and

updating the 3D model based on multiple depth images and corresponding segmentation masks obtained over a period of time, wherein the updating of the 3D model comprises updating a first portion of the 3D model corresponding to the identified portion associated with the object at a higher rate than a second portion of the 3D model.

11. The device of claim 10 , wherein the object the segmentation mask identifies in the portion of the second image is associated with motion.

12. The device of claim 10 , wherein the segmentation mask is generated by:

determining whether to associate a category from a set of categories for each pixel in the second image based on characteristics the pixel exhibits; and

determining, for each determined pixel in the second image that is associated with a category, a confidence value based on the characteristics the pixel exhibits in the second image.

13. The device of claim 10 , wherein the updating is based on determining motion of the object.

14. A non-transitory computer-readable storage medium, storing computer-executable program instructions on a computer to perform operations comprising:

obtaining depth data of an environment, the depth data comprising depth values for pixels of a first image, the environment comprising an object;

obtaining a segmentation mask associated with a second image of the environment, the segmentation mask identifying a portion of the second image associated with the object;

generating a 3D model of the environment that excludes the object based on the depth data and the segmentation mask, wherein the 3D model excludes the object based on excluding depth data associated with the identified portion of the second image associated with the object; and

updating the 3D model based on multiple depth images and corresponding segmentation masks obtained over a period of time, wherein the updating of the 3D model comprises updating a first portion of the 3D model corresponding to the identified portion associated with the object at a higher rate than a second portion of the 3D model.