Digital video content fingerprinting using image pixel intensity and color information
View Patent ↗Techniques are described for improving accuracy and speed of video fingerprinting and identification by using video frame color and intensity information. Multiple approaches for generating a novel signature based on video frame color information are described. One is based on spatial distribution of pixel intensities with reference to derived color groups, while another is based on thresholding a joint histogram of pixel intensity, color, and spatial coordinates. These color based signatures can be deployed either as primary video content signatures, or as hash traversal signatures used to improve speed of an initial stage of video data base search procedures. They can also be used as secondary signatures during a video signature data base search with primary video content signatures, to help disambiguate possible false positive matches, and to improve marginal matches which might not be identified otherwise.
1. A method for video frame spatio-color descriptor generation, the method comprising:
determining a region of interest (ROI) within an active region of a video frame;
partitioning pixels in the ROI into multiple color groups;
subdividing the ROI into multiple subregions;
constructing a multiple dimension spatio-color descriptor for the active region by accumulating in the ROI weighted pixel intensities in each color group of the multiple color groups within each subregion of the multiple subregions;
determining median pixel values of chrominance components of a YUV color representation for each pixel of each subregion;
determining the multiple color groups and partial weighting values within a limit range based on the median pixel values; and
determining weighting values based on a multiplicative combination of the partial weighting values, wherein the weighting values are assigned to color bins associated with the multiple color groups and are used to calculate the weighted pixel intensities.
2. The method of claim 1 , further comprising:
applying the determined weighting values to each corresponding pixel luminance component to calculate weighted pixel intensity values; and
accumulating the weighted pixel intensity values in corresponding color bins, wherein each corresponding color bin represents a dimension of a multiple dimension spatio-color descriptor.
3. The method of claim 1 , wherein a number N of dimensions of the multiple dimension spatio-color descriptor is equal to the number of subregions times the number of color groups.
4. The method of claim 1 , further comprising:
converting a red, green, and blue (RGB) color video input to a YUV color representation for each region of interest pixel using a transform matrix prior to constructing the multiple dimension spatio-color descriptor.
5. A method for video frame spatio-color descriptor generation, the method comprising:
partitioning pixels in each region of interest (ROI) of a selected video frame into multiple subregions for each ROI, wherein each subregion is configured with multiple color groups and each selected video frame having multiple ROIs;
constructing a multiple dimension spatio-color descriptor for each ROI by accumulating in each ROI weighted pixel intensities in each color group of the multiple color groups within each subregion of the multiple subregions;
determining median pixel values of chrominance components of a YUV color representation for each pixel of each subregion of each ROI;
determining partial weighting values based on a function of pixel chrominance values, the median pixel values, and threshold chrominance values; and
determining weighting values based on a multiplicative combination of the partial weighting values, wherein the weighting values are assigned to color bins associated with the multiple color groups and are used to calculate the weighted pixel intensities.
6. The method of claim 5 further comprising:
applying the determined weighting values to each corresponding pixel luminance component to calculate weighted pixel intensity values; and
accumulating the weighted pixel intensity values in corresponding color bins, wherein each corresponding color bin represents a dimension of a multiple dimension spatio-color descriptor.
7. The method of claim 5 further comprising:
applying a calculated weighting value to each corresponding pixel luminance component of the pixels in each subregion of each ROI to calculate weighted pixel intensity values; and
accumulating the weighted pixel intensity values in corresponding color bins associated with the multiple color groups, wherein each color bin represents a dimension of the multiple dimension spatio-color descriptor.
8. The method of claim 5 further comprising:
comparing each dimension of the multiple dimension spatio-color descriptor of each ROI with multiple thresholds to determine a corresponding numeric value representative of a range between two of the multiple thresholds, wherein each color group represents a dimension of the multiple dimension spatio-color descriptor; and
concatenating the corresponding numeric values to create a color signature for each ROI.
9. The method of claim 8 further comprising:
selecting primary signatures derived based on video luminance components that match with reference signatures from a large signature database to determine a signature matching score; and
adjusting the signature matching score based on the color signature for each ROI.
10. A computer readable non-transitory medium encoded with computer readable program data and code, the program data and code executes to:
determine a region of interest (ROI) within an active region of a video frame;
partition pixels in the ROI into multiple color groups;
subdivide the ROI into multiple subregions;
construct a multiple dimension spatio-color descriptor for the active region by accumulating in the ROI weighted pixel intensities in each color group of the multiple color groups within each subregion of the multiple subregions;
determine median pixel values of chrominance components of a YUV color representation for each pixel of each subregion;
determine the multiple color groups and partial weighting values within a limit range based on the median pixel values; and
determine weighting values based on a multiplicative combination of the partial weighting values, wherein the weighting values are assigned to color bins associated with the multiple color groups and are used to calculate the weighted pixel intensities.
11. The computer readable non-transitory medium of claim 10 further executes to:
apply the determined weighting values to each corresponding pixel luminance component to calculate weighted pixel intensity values; and
accumulate the weighted pixel intensity values in corresponding color bins, wherein each corresponding color bin represents a dimension of a multiple dimension spatio-color descriptor.
12. The computer readable non-transitory medium of claim 10 , wherein a number N of dimensions of the multiple dimension spatio-color descriptor is equal to the number of subregions times the number of color groups.
13. The computer readable non-transitory medium of claim 10 further executes to:
convert a red, green, and blue (RGB) color video input to a YUV color representation for each region of interest pixel using a transform matrix prior to constructing the multiple dimension spatio-color descriptor.