Temporally consistent superpixels

1. A method implemented in an apparatus for generating superpixels for a sequence of frames, the method comprising:

receiving a sequence of frames via an input;

separating a feature space of the sequence of frames into a color subspace and a spatial subspace with a separator;

performing a clustering with a clustering hardware unit, the clustering comprising iteratively assigning and updating, wherein the assigning uses joint distances derived from a distance to spatial cluster centers determined in the spatial subspace on a frame basis and a distance to color cluster centers determined in the color subspace on stacked frames, wherein a search window associated with a cluster is extended into the temporal dimension by creating a window on every frame centered around a spatial cluster center in that frame, the search window being a sliding window spanned around a center frame, and wherein in the updating, updated spatial cluster centers and color cluster centers are computed based on results of the assigning; and

outputting results of the clustering via an output.

2. An apparatus for generating superpixels for a sequence of frames, wherein the apparatus comprises:

an input configured to receive a sequence of frames;

a processing unit configured to separate a feature space of the sequence of frames into a color subspace and a spatial subspace and to perform a clustering comprising iterative assigning and updating, wherein the assigning uses joint distances derived from a distance to spatial cluster centers determined in the spatial subspace on a frame basis and a distance to color cluster centers determined in the color subspace on stacked frames, wherein a search window associated with a cluster is extended into the temporal dimension by creating a window on every frame centered around a spatial cluster center in that frame, the search window being a sliding window spanned around a center frame, and the updating updates spatial cluster centers and color cluster centers based on results of the assigning; and

an output configured to output results of the clustering.

3. A non-transitory computer readable storage medium having stored therein instructions enabling generating superpixels for a sequence of frames, which when executed by a processor, cause the processor to:

separate a feature space of the sequence of frames into a color subspace and a spatial subspace; and

perform a clustering using the processor comprising iteratively assigning and updating, wherein the assigning uses joint distances derived from a distance to spatial cluster centers determined in the spatial subspace on a frame basis and a distance to color cluster centers determined in the color subspace on stacked frames, wherein a search window associated with a cluster is extended into the temporal dimension by creating a window on every frame centered around a spatial cluster center in that frame, the search window being a sliding window spanned around a center frame, and wherein in the updating, updated spatial cluster centers and color cluster centers are computed based on results of the assigning.