Object segmentation and feature tracking

1. An apparatus for image processing, the apparatus comprising:

a memory; and

one or more processors coupled to the memory and configured to:

receive an image of an environment captured by an image sensor, wherein the image depicts at least a static portion of the environment and a dynamic object in the environment;

identify a portion of the image that includes a depiction of the dynamic object;

generate a masked image at least by masking the portion of the image that includes the depiction of the dynamic object;

identify one or more features of the static portion of the environment in the masked image; and

track the one or more features of the static portion of the environment between the masked image and one or more additional images of the environment.

2. The apparatus of claim 1 , wherein the one or more processors are configured to:

determine a location of a first feature of the one or more features of the static portion of the environment based on tracking of the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment; and

update a map of the environment based on the location of the first feature.

3. The apparatus of claim 2 , wherein, to update the map of the environment based on the location, the one or more processors are configured to add the location of the first feature to the map.

4. The apparatus of claim 2 , wherein, to update the map of the environment based on the location, the one or more processors are configured to modify a prior location of the first feature in the map based on the location of the first feature.

5. The apparatus of claim 1 , wherein the one or more processors are configured to:

determine a pose of the apparatus within the environment based on tracking of the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment, wherein the pose of the apparatus within the environment includes at least one of a location of the apparatus, a pitch of the apparatus, a roll of the apparatus, or a yaw of the apparatus.

6. The apparatus of claim 1 , wherein the one or more processors are configured to:

generate a downscaled image at least by downscaling the image, wherein identifying the portion of the image that includes the depiction of the dynamic object includes identifying a portion of the downscaled image that includes the depiction of the dynamic object.

7. The apparatus of claim 1 , wherein the one or more processors are configured to:

generate a greyscale image at least by desaturating color in the image, wherein identifying the portion of the image that includes the depiction of the dynamic object includes identifying a portion of the greyscale image that includes the depiction of the dynamic object.

8. The apparatus of claim 1 , wherein, to identify the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to:

analyze each pixel of a plurality of pixels corresponding to the image to identify a subset of the plurality of pixels that depicts at least a portion of the dynamic object.

9. The apparatus of claim 1 , wherein, to identify the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to identify a bounding box occupying a polygonal region of the image, wherein the depiction of the dynamic object is at least partially included within the bounding box.

10. The apparatus of claim 9 , wherein, to identify the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to analyze each pixel of a plurality of pixels within the bounding box to identify a subset of the plurality of pixels within the bounding box that each depict a portion of the dynamic object.

11. The apparatus of claim 10 , wherein, to identify the bounding box, the one or more processors are configured to use at least a first trained neural network; and

wherein, to identify the subset of the plurality of pixels, the one or more processors are configured to use at least a second trained neural network.

12. The apparatus of claim 1 , wherein, to identify the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to:

identify, using at least a first trained neural network, that the image includes the depiction of the dynamic object, and

identify, using at least a second trained neural network in response to identification that the image includes the depiction of the dynamic object, the portion of the image that includes the depiction of the dynamic object.

13. The apparatus of claim 1 , wherein, to mask the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to:

generate a mask having a same shape and a same size as the portion of the image that includes the depiction of the dynamic object;

overlay the mask over the portion of the image that includes the depiction of the dynamic object; and

merge the mask with the image after overlaying the mask over the portion of the image that includes the depiction of the dynamic object.

14. The apparatus of claim 13 , wherein, to mask the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to:

blur the mask before merging the mask with the image.

15. The apparatus of claim 1 , wherein the one or more processors are configured to:

identify a second portion of the image that includes a second depiction of a second dynamic object, wherein the image depicts the second dynamic object in the environment;

wherein, to generate the masked image, the one or more processors are configured to mask the second portion of the image that includes the second depiction of the second dynamic object.

16. The apparatus of claim 1 , wherein the one or more processors are configured to:

receive a second image of the environment captured by the image sensor, the second image being one of the one or more additional images of the environment;

identify a second portion of the second image that includes a second depiction of one of the dynamic object or a second dynamic object; and

mask the second portion of the second image before tracking the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment.

17. The apparatus of claim 1 , wherein the dynamic object is a person, and wherein, to identify the portion of the image that includes the depiction of the dynamic object, the one or more processors are configured to identify a depiction of a face of the person using facial detection.

18. The apparatus of claim 1 , wherein the static portion of the environment is static relative to a position of the image sensor during capture of the image, wherein the dynamic object moves relative to a position of the image sensor during capture of the image.

19. The apparatus of claim 1 , wherein the apparatus is one of a mobile device, a wireless communication device, a robot, a vehicle, a head-mounted display, and a camera.

20. The apparatus of claim 1 , further comprising:

the image sensor.

21. A method of image processing, the method comprising:

receiving an image of an environment captured by an image sensor, wherein the image depicts at least a static portion of the environment and a dynamic object in the environment;

identifying a portion of the image that includes a depiction of the dynamic object;

generating a masked image at least by masking the portion of the image that includes the depiction of the dynamic object;

identifying one or more features of the static portion of the environment in the masked image; and

tracking the one or more features of the static portion of the environment between the masked image and one or more additional images of the environment.

22. The method of claim 21 , further comprising:

determining a location of a first feature of the one or more features of the static portion of the environment based on tracking of the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment; and

updating a map of the environment based on the location of the first feature.

23. The method of claim 21 , further comprising:

determining a pose of the image sensor within the environment based on tracking of the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment, wherein the pose of the image sensor within the environment includes at least one of a location of the image sensor, a pitch of the image sensor, a roll of the image sensor, or a yaw of the image sensor.

24. The method of claim 21 , wherein identifying the portion of the image that includes the depiction of the dynamic object includes:

analyzing each pixel of a plurality of pixels corresponding to the image to identify a subset of the plurality of pixels that depicts at least a portion of the dynamic object.

25. The method of claim 21 , wherein identifying the portion of the image that includes the depiction of the dynamic object includes:

identifying a bounding box occupying a polygonal region of the image, wherein the depiction of the dynamic object is at least partially included within the bounding box.

26. The method of claim 25 , wherein identifying the portion of the image that includes the depiction of the dynamic object includes:

analyzing each pixel of a plurality of pixels within the bounding box to identify a subset of the plurality of pixels within the bounding box that each depict a portion of the dynamic object.

27. The method of claim 21 , wherein identifying the portion of the image that includes the depiction of the dynamic object includes:

identifying, using at least a first trained neural network, that the image includes the depiction of the dynamic object, and

identifying, using at least a second trained neural network in response to identification that the image includes the depiction of the dynamic object, the portion of the image that includes the depiction of the dynamic object.

28. The method of claim 21 , further comprising:

receiving a second image of the environment captured by the image sensor, the second image being one of the one or more additional images of the environment;

identifying a second portion of the second image that includes a second depiction of one of the dynamic object or a second dynamic object; and

masking the second portion of the second image before tracking the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment.

29. The method of claim 21 , wherein the static portion of the environment is static relative to a position of the image sensor during capture of the image, wherein the dynamic object moves relative to a position of the image sensor during capture of the image.

30. The apparatus of claim 1 , wherein the one or more processors are configured to:

track movement of the apparatus relative to the static portion of the environment based on tracking of the one or more features of the static portion of the environment between the masked image and the one or more additional images of the environment.