Living skin tissue tracking in video stream
A method comprising: receiving, as input, a video sequence comprising a plurality of frames and depicting a scene; detecting, in each of said frames, one or more regions of interest (ROI) associated with an object in said scene; extracting, from each of said ROIs with respect to all of said frames, a feature set representing spectral reflectance; at a training stage, train a machine learning classifier on a training set comprising: (i) all of said feature sets, and (ii) labels indicating whether each of said ROIs depicts living skin tissue; and at an inference stage, applying said trained machine learning classifier to a target feature set extracted from a target ROI detected in a target video sequence, to determine whether said ROI depicts living skin tissue.
1. A system comprising:
at least one hardware processor; and
a non-transitory computer-readable storage medium having stored thereon program code, the program code executable by the at least one hardware processor to:
receive, as input, a video sequence comprising a plurality of frames and depicting a scene,
detect, in each of said frames, one or more regions of interest (ROI) associated with an object in said scene,
extract, from each of said ROIs with respect to all of said frames, a feature set representing spectral reflectance,
at a training stage, train a machine learning classifier on a training set comprising:
(i) all of said feature sets, and
(ii) labels indicating whether each of said ROIs depicts living skin tissue, and
at an inference stage, apply said trained machine learning classifier to a target feature set extracted from a target ROI detected in a target video sequence, to determine whether said ROI depicts living skin tissue,
wherein said extracting comprises any one of:
calculating image data values in an RGB (red-green-blue) color representation model, wherein said features comprise at least some of: frequency peak for at least one RGB channel; area under curve (AUC) for at least one RGB channel in the frequency domain; independent component analysis (ICA) for each RGB channel; maximum amplitude range for ICA in at least one RGB channel; AUC for ICA in at least one RGB channel in the time domain; frequency peak for ICA in at least one RGB channel, in the frequency domain; time index associated with an autocorrelation signal for at least one RGB channel; frequency peak associated with an autocorrelation signal for at least one RGB channel in the frequency domain; and
calculating image data values in an HSV (hue, saturation, value) color representation model, wherein said features comprise at least some of: Frequency peak for at least one HSV channel; area under curve (AUC) for at least one HSV channel in the frequency domain; maximum amplitude range for at least one HSV channel in the time domain; AUC for at least one HSV channel in the time domain; time index associated with an autocorrelation signal for at least one HSV channel; frequency peak associated with an autocorrelation signal for at least one HSV channel in the frequency domain; number of peaks above a specified threshold for at least one HSV channel in the time domain; and amplitude range for at least one HSV channel in the time domain.
2. The system of claim 1 , wherein each of said ROIs depicts at least one of: a human; part or whole of a human facial region; a skin region in a human; and a background region in said scene.
3. The system of claim 1 , wherein each of said ROIs is further segmented into a plurality of segments based on one of: pixel similarity parameter, superpixel methods, and a fixed grid.
4. The system of claim 3 , wherein said segments are tracked in said video sequence.
5. A method comprising:
receiving, as input, a video sequence comprising a plurality of frames and depicting a scene;
detecting, in each of said frames, one or more regions of interest (ROI) associated with an object in said scene;
extracting, from each of said ROIs with respect to all of said frames, a feature set representing spectral reflectance;
at a training stage, train a machine learning classifier on a training set comprising:
(i) all of said feature sets, and
(ii) labels indicating whether each of said ROIs depicts living skin tissue; and
at an inference stage, applying said trained machine learning classifier to a target feature set extracted from a target ROI detected in a target video sequence, to determine whether said ROI depicts living skin tissue,
wherein said extracting comprises any one of:
calculating image data values in an RGB (red-green-blue) color representation model, wherein said features comprise at least some of: frequency peak for at least one RGB channel; area under curve (AUC) for at least one RGB channel in the frequency domain; independent component analysis (ICA) for each RGB channel; maximum amplitude range for ICA in at least one RGB channel; AUC for ICA in at least one RGB channel in the time domain; frequency peak for ICA in at least one RGB channel, in the frequency domain; time index associated with an autocorrelation signal for at least one RGB channel; frequency peak associated with an autocorrelation signal for at least one RGB channel in the frequency domain; and
calculating image data values in an HSV (hue, saturation, value) color representation model, wherein said features comprise at least some of: Frequency peak for at least one HSV channel; area under curve (AUC) for at least one HSV channel in the frequency domain; maximum amplitude range for at least one HSV channel in the time domain; AUC for at least one HSV channel in the time domain; time index associated with an autocorrelation signal for at least one HSV channel; frequency peak associated with an autocorrelation signal for at least one HSV channel in the frequency domain; number of peaks above a specified threshold for at least one HSV channel in the time domain; and amplitude range for at least one HSV channel in the time domain.
6. The method of claim 5 , wherein each of said ROIs depicts at least one of: a human; part or whole of a human facial region; a skin region in a human; and a background region in said scene.
7. The method of claim 5 , wherein each of said ROIs is further segmented into a plurality of segments based on one of: pixel similarity parameter, superpixel methods, and a fixed grid.
8. The method of claim 7 , wherein said segments are tracked in said video sequence.
9. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:
receive, as input, a video sequence comprising a plurality of frames and depicting a scene;
detect, in each of said frames, one or more regions of interest (ROI) associated with an object in said scene;
extract, from each of said ROIs with respect to all of said frames, a feature set representing spectral reflectance,
at a training stage, train a machine learning classifier on a training set comprising:
(i) all of said feature sets, and
(ii) labels indicating whether each of said ROIs depicts living skin tissue; and
at an inference stage, apply said trained machine learning classifier to a target feature set extracted from a target ROI detected in a target video sequence, to determine whether said ROI depicts living skin tissue,
wherein said extracting comprises any one of:
calculating image data values in an RGB (red-green-blue) color representation model, wherein said features comprise at least some of: frequency peak for at least one RGB channel; area under curve (AUC) for at least one RGB channel in the frequency domain; independent component analysis (ICA) for each RGB channel; maximum amplitude range for ICA in at least one RGB channel; AUC for ICA in at least one RGB channel in the time domain; frequency peak for ICA in at least one RGB channel, in the frequency domain; time index associated with an autocorrelation signal for at least one RGB channel; frequency peak associated with an autocorrelation signal for at least one RGB channel in the frequency domain; and
calculating image data values in an HSV (hue, saturation, value) color representation model, wherein said features comprise at least some of: Frequency peak for at least one HSV channel; area under curve (AUC) for at least one HSV channel in the frequency domain; maximum amplitude range for at least one HSV channel in the time domain; AUC for at least one HSV channel in the time domain; time index associated with an autocorrelation signal for at least one HSV channel; frequency peak associated with an autocorrelation signal for at least one HSV channel in the frequency domain; number of peaks above a specified threshold for at least one HSV channel in the time domain; and amplitude range for at least one HSV channel in the time domain.
10. The computer program product of claim 9 , wherein each of said ROIs depicts at least one of: a human; part or whole of a human facial region; a skin region in a human; and a background region in said scene.
11. The computer program product of claim 9 , wherein each of said ROIs is further segmented into a plurality of segments based on one of: pixel similarity parameter, superpixel methods, and a fixed grid.