Optimizations for dynamic object instance detection, segmentation, and structure mapping

1. A method comprising, by a computing system:

accessing an image;

generating a feature map for the image using a first neural network;

identifying a plurality of regions of interest in the feature map;

generating a plurality of regional feature maps for the plurality of regions of interest, respectively, by sampling the feature map for the image;

processing the plurality of regional feature maps using a second neural network to:

detect at least one regional feature map in the plurality of regional feature maps that corresponds to a person depicted in the image; and

generate a target region definition associated with a location of the person using the regional feature map;

generating, based on the target region definition associated with the location of the person, a target regional feature map by sampling the feature map for the image; and

generating:

a keypoint mask associated with the person by processing the target regional feature map using a third neural network; or

an instance segmentation mask associated with the person by processing the target regional feature map using a fourth neural network.

2. The method of claim 1 , wherein the instance segmentation mask and keypoint mask are both generated and are being generated concurrently.

3. The method of claim 1 , wherein the first neural network comprises four or fewer convolutional layers.

4. The method of claim 3 , wherein each of the convolutional layers uses a kernel size of 3×3 or less.

5. The method of claim 1 , wherein the first neural network comprises a total of one pooling layer.

6. The method of claim 1 , wherein the first neural network comprises three or fewer inception modules.

7. The method of claim 6 , wherein each of the inception modules performs convolutional operations with kernel sizes of 5×5 or less.

8. The method of claim 1 , wherein each of the second neural network, third neural network, and fourth neural network is configured to process an input regional feature map using a total of one inception module.

9. A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising:

accessing an image;

generating a feature map for the image using a first neural network;

identifying a plurality of regions of interest in the feature map;

generating a plurality of regional feature maps for the plurality of regions of interest, respectively, by sampling the feature map for the image;

processing the plurality of regional feature maps using a second neural network to:

detect at least one regional feature map in the plurality of regional feature maps that corresponds to a person depicted in the image; and

generate a target region definition associated with a location of the person using the regional feature map;

generating, based on the target region definition associated with the location of the person, a target regional feature map by sampling the feature map for the image; and

generating:

a keypoint mask associated with the person by processing the target regional feature map using a third neural network; or

an instance segmentation mask associated with the person by processing the target regional feature map using a fourth neural network.

10. The system of claim 9 , wherein the instance segmentation mask and keypoint mask are both generated and are being generated concurrently.

11. The system of claim 9 , wherein the first neural network comprises four or fewer convolutional layers.

12. The system of claim 11 , wherein each of the convolutional layers uses a kernel size of 3×3 or less.

13. The system of claim 9 , wherein the first neural network comprises a total of one pooling layer.

14. The system of claim 9 , wherein the first neural network comprises three or fewer inception modules.

15. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising:

accessing an image;

generating a feature map for the image using a first neural network;

identifying a plurality of regions of interest in the feature map;

generating a plurality of regional feature maps for the plurality of regions of interest, respectively, by sampling the feature map for the image;

processing the plurality of regional feature maps using a second neural network to:

detect at least one regional feature map in the plurality of regional feature maps that corresponds to a person depicted in the image; and

generate a target region definition associated with a location of the person using the regional feature map;

generating, based on the target region definition associated with the location of the person, a target regional feature map by sampling the feature map for the image; and

generating:

a keypoint mask associated with the person by processing the target regional feature map using a third neural network; or

an instance segmentation mask associated with the person by processing the target regional feature map using a fourth neural network.

16. The media of claim 15 , wherein the instance segmentation mask and keypoint mask are both generated and are being generated concurrently.

17. The media of claim 15 , wherein the first neural network comprises four or fewer convolutional layers.

18. The media of claim 17 , wherein each of the convolutional layers uses a kernel size of 3×3 or less.

19. The media of claim 15 , wherein the first neural network comprises a total of one pooling layer.

20. The media of claim 15 , wherein the first neural network comprises three or fewer inception modules.