Method and device for pixel-level object segmentation

1. A method comprising:

in a type-agnostic object segmentation system including one or more processors and non-transitory memory:

obtaining a data set, wherein the data set includes a plurality of image data frames, wherein each of the plurality of image data frames is associated with a respective segmentation mask, the respective segmentation mask demarking at least one object instance representation within a corresponding one of the plurality of image data frames;

generating a respective embedding vector for each of a plurality of pixels of the plurality of image data frames, wherein the plurality of pixels of the plurality of image data frames includes at least a portion of an image data frame;

mapping a plurality of embedding vector representations into a dataspace, wherein the dataspace is defined by a dimensionality of the plurality of embedding vector representations;

disambiguating the plurality of embedding vector representations within the dataspace in order to detect at least one object instance representation candidate in the image data frame;

comparing the at least one object instance representation candidate against the respective segmentation mask; and

adjusting the type-agnostic object segmentation system in order to satisfy an error threshold across the plurality of image data frames according to a determination that the at least one object instance representation candidate and the respective segmentation mask differ by a threshold value.

2. The method of claim 1 further comprising:

generating a label for each of the plurality of pixels in the image data frame.

3. The method of claim 1 further comprising:

generating one or more features from each of the plurality of the pixels in the image data frame.

4. The method of claim 1 further comprising:

identifying one or more features associated with one or more pixels from the each of the plurality of pixels.

5. The method of claim 1 , wherein the object instance representation candidate satisfies a recognition threshold value.

6. The method of claim 1 , wherein disambiguating the plurality of embedding vector representations further includes satisfying a disambiguation threshold value, and wherein the disambiguation threshold value corresponds to a distance between the plurality of embedding vector representations within the dataspace.

7. The method of claim 1 , wherein adjusting the type-agnostic object segmentation system includes adjusting filter weights in order to satisfy an error threshold across the plurality of image data frames.

8. The method of claim 1 , wherein adjusting the type-agnostic segmentation system if the threshold value is not satisfied further comprises:

generating a second respective embedding vector for each of a second set of a plurality of pixels of a subsequent image data frame;

mapping a second plurality of embedding vector representations into the dataspace;

disambiguating the second plurality of embedding vector representations within the dataspace in order to detect a second object instance representation candidate in the second image data frame; and

comparing the second object instance representation candidate against the respective segmentation mask.

9. The method of claim 1 , wherein adjusting the type-agnostic segmentation system if the error threshold is not satisfied further comprises:

generating a third respective embedding vector for each of a third set of a plurality of pixels of a subsequent image data frame;

mapping a third plurality embedding vector representations into the dataspace;

disambiguating the third plurality of embedding vector representations within the dataspace in order to detect a third object instance representation candidate in the third image data frame; and

comparing the third object instance representation candidate against the respective segmentation mask.

10. A type-agnostic object segmentation system comprising:

one or more processors;

a non-transitory memory; and

one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the type-agnostic object segmentation system to:

obtain a data set, wherein the data set includes a plurality of image data frames, wherein each of the plurality of image data frames is associated with a respective segmentation mask, the respective segmentation mask demarking at least one object instance representation within a corresponding one of the plurality of image data frames;

generate a respective embedding vector for each of a plurality of pixels of the plurality of image data frames, wherein the plurality of pixels of the plurality of image data frames includes at least a portion of an image data frame;

map a plurality of embedding vector representations into a dataspace, wherein the dataspace is defined by a dimensionality of the plurality of embedding vector representations;

disambiguate the plurality of embedding vector representations within the dataspace in order to detect at least one object instance representation candidate in the image data frame;

compare the at least one object instance representation candidate against the respective segmentation mask; and

adjust the type-agnostic object segmentation system in order to satisfy an error threshold across the plurality of image data frames according to a determination that the at least one object instance representation candidate and the respective segmentation mask differ by a threshold value.

11. The type-agnostic object segmentation system of claim 10 , wherein the object instance representation candidate satisfies a recognition threshold value.

12. The type-agnostic object segmentation system of claim 10 , wherein disambiguating the plurality of embedding vector representations further includes satisfying a disambiguation threshold value, and wherein the disambiguation threshold value corresponds to a distance between the plurality of embedding vector representations within the dataspace.

13. The type-agnostic object segmentation system of claim 10 , wherein adjusting the type-agnostic object segmentation system includes adjusting filter weights in order to satisfy an error threshold across the plurality of image data frames.

14. The type-agnostic object segmentation system of claim 10 , wherein the one or more programs further cause the type-agnostic object segmentation system to:

generate a label for each of the plurality of pixels in the image data frame.

15. The type-agnostic object segmentation system of claim 10 , wherein the one or more programs further cause the type-agnostic object segmentation system to:

generate one or more features from each of the plurality of the pixels in the image data frame.

16. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a type-agnostic object segmentation system, cause the type-agnostic object segmentation system to:

obtain a data set, wherein the data set includes a plurality of image data frames, wherein each of the plurality of image data frames is associated with a respective segmentation mask, the respective segmentation mask demarking at least one object instance representation within a corresponding one of the plurality of image data frames;

generate a respective embedding vector for each of a plurality of pixels of the plurality of image data frames, wherein the plurality of pixels of the plurality of image data frames includes at least a portion of an image data frame;

map a plurality of embedding vector representations into a dataspace, wherein the dataspace is defined by a dimensionality of the plurality of embedding vector representations;

disambiguate the plurality of embedding vector representations within the dataspace in order to detect at least one object instance representation candidate in the image data frame;

compare the at least one object instance representation candidate against the respective segmentation mask; and

adjust the type-agnostic object segmentation system in order to satisfy an error threshold across the plurality of image data frames according to a determination that the at least one object instance representation candidate and the respective segmentation mask differ by a threshold value.

17. The non-transitory memory of claim 16 , wherein the object instance representation candidate satisfies a recognition threshold value.

18. The non-transitory memory of claim 16 , wherein disambiguating the plurality of embedding vector representations further includes satisfying a disambiguation threshold value, and wherein the disambiguation threshold value corresponds to a distance between the plurality of embedding vector representations within the dataspace.

19. The non-transitory memory of claim 16 , wherein adjusting the type-agnostic object segmentation system includes adjusting filter weights in order to satisfy an error threshold across the plurality of image data frames.

20. The non-transitory memory of claim 16 , wherein the one or more programs further cause the type-agnostic object segmentation system to:

generate a label for each of the plurality of pixels in the image data frame.

21. The non-transitory memory of claim 16 , wherein the one or more programs further cause the type-agnostic object segmentation system to:

generate one or more features from each of the plurality of the pixels in the image data frame.