System and method for real time insertion into video with occlusion on areas containing multiple colors
View Patent ↗A system, method and computer program product for real-time insertion of indicia (e.g., an advertisement) into a live (or taped) broadcast. Typically, the invention allow insertion with occlusion onto surfaces such as tennis courts, the wall behind home plate on a baseball field, the grass or turf on a soccer or football stadium, and the like. The occlusion processing described herein can handle multiple colors in the background image during the insertion of the indicia into a video image. The invention can thus adapt to changing light conditions in the video image. The process includes obtaining a video image from a camera. The video image is typically digitized. Next, the pixels within the video image are sampled. A plurality of background colors are then identified for the sampled pixels. An opacity value is then assigned to each pixel in the indicia based on whether the color of a positionally corresponding pixel in the video image is the same color as one of the plurality of background colors. Finally, pixels within the indicia are displayed in the video image if the color of the positionally corresponding pixel in the video image is the same color as one of the background colors and a pixel in the indicia is not displayed in the video image if the positionally corresponding pixel in the video image is not the same color of as one of the background colors.
1. A method of occlusion processing for inserting an indicia into a video image, comprising:
sampling pixels in the video image;
identifying a background color for said sampled pixels, said background color comprising first, second and third background color components;
generating first, second and third lookup tables corresponding to each of said first, second and third background color components, each lookup table mapping an input pixel color component value to an opacity value;
applying first, second and third input pixel color component values associated with an input pixel from the video image to said first, second and third lookup tables, respectively, to obtain corresponding first, second and third opacity values; and
assigning an opacity mask value to a pixel in the indicia that positionally corresponds to said input pixel from the video image based on the first, second and third opacity values.
2. The method of claim 1 , wherein said identifying includes:
(a) generating a histogram for each of first, second and third color component values assocated with each of said sampled pixels; and
(b) selecting a dominant node from each of said histograms, wherein said dominant nodes identify a background color.
3. The method of claim 1 , wherein said first, second and third background color components are Y, U and V background color components, respectively.
4. The method of claim 1 , wherein said first, second and third background color components are R, G and B background color components, respectively.
5. The method of claim 1 , further comprising identifying additional background colors.
6. The method of claim 1 , wherein said sampling comprises sampling pixels within a predetermined region within the video image.
7. The occlusion processing method of claim 6 , wherein said predetermined region within the video image is an insert location for inserting the indicia in the video image.
8. The method of claim 1 , wherein said first, second and third opacity values each have a range from transparent to opaque.
9. A method of occlusion processing for inserting an indicia into a video image, comprising:
obtaining a video image from a camera;
sampling pixels in said video image;
identifying a background colors for said sampled pixels, said background color comprising first, second, and third background color components;
generating first, second and third lookup tables corresponding to each of said first, second and third background color components, each lookup table mapping an input pixel color component value to an opacity value;
applying first, second and third input pixel color component values associated with an input pixel from the video image to said first, second and third lookup tables, respectively, to obtain corresponding first, second and third opacity values;
assigning an opacity mask value to a pixel in the indicia that positionally corresponds to said input pixel from the video image based on the first, second and third opacity values; and
displaying said video image having said pixel in the indicia inserted therein, wherein said pixel in the indicia is displayed based on said opacity mask value.