System and method to identify programs and commercials in video content via unsupervised static content identification
In one aspect, an example method includes (i) determining, by a computing system, a mean image of a set of frames of video content; (ii) extracting, by the computing system, a reference template of static content from the mean image; (iii) identifying, by the computing system, the extracted reference template of static content in a frame of the set of frames of the video content; (iv) labeling a segment within the video content as either a program segment or an advertisement segment based on the identifying of the extracted reference template of static content in the frame of the video content; and (v) generating data identifying the labeled segment.
1. A method comprising:
determining, by a computing system, a mean image of a set of frames of video content;
extracting, by a computing system, a reference template of static content from the mean image;
identifying, by the computing system, presence of the extracted reference template of static content in a frame of the set of frames of the video content;
labeling, by the computing system, a segment within the video content as either a program segment or an advertisement segment based on the identified presence of the extracted reference template of static content in the frame of the video content; and
generating, by the computing system, data identifying the labeled segment.
2. The method of claim 1 , wherein extracting the reference template of static content from the mean image comprises:
generating a binary image from the mean image; and
using the binary image as the reference template of static content.
3. The method of claim 1 , wherein extracting the reference template of static content from the mean image comprises:
generating a binary image from the mean image;
applying one or more morphological operations to the binary image so as to identify a residual component of the binary image; and
extracting a reference template of static content based on the residual component.
4. The method of claim 1 , wherein identifying presence of the extracted reference template of static content in a frame of the video content comprises:
matching the extracted reference template of static content against the frame of the video content so as to determine a match score; and
identifying presence of the extracted reference template of static content based on the match score.
5. The method of claim 1 , further comprising labeling the frame of the video content as program content rather than advertisement content based on the identified presence of the extracted reference template of static content in the frame of the video content.
6. The method of claim 1 , wherein output data identifying the labeled segment is useable to facilitate taking an action, wherein the action comprises dynamic ad insertion.
7. The method of claim 3 , wherein extracting the reference template of static content based on the residual component comprises:
determining a bounding box of the residual component; and
extracting a reference template of static content from a sub-region of the binary image corresponding to the bounding box.
8. The method of claim 5 , wherein the reference template of static content is a first reference template of static content, the method further comprising extracting by the computing system a second reference template of static content from the mean image and identifying by the computing system presence of the extracted second reference template of static content in the frame,
wherein the labeling of the frame of video content as program content rather than advertisement content is based on both the identified presence of the extracted first reference template of static content in the frame and the identified presence of the extracted second reference template of static content in the frame.
9. A computing system comprising:
one or more processors;
non-transitory data storage; and
program instructions stored in the non-transitory data storage and executable by the one or more processors to carry out operations including:
determining a mean image of a set of frames of video content,
extracting a reference template of static content from the mean image,
identifying presence of the extracted reference template of static content in a frame of the set of frames of the video content,
labeling a segment within the video content as either a program segment or an advertisement segment based on the identified presence of the extracted reference template of static content in the frame of the video content, and
generating data identifying the labeled segment.
10. The computing system of claim 9 , wherein extracting the reference template of static content from the mean image comprises:
generating a binary image from the mean image; and
using the binary image as the reference template of static content.
11. The computing system of claim 9 , wherein extracting the reference template of static content from the mean image comprises:
generating a binary image from the mean image;
applying one or more morphological operations to the binary image so as to identify a residual component of the binary image; and
extracting a reference template of static content based on the residual component.
12. The computing system of claim 9 , wherein identifying presence of the extracted reference template of static content in a frame of the video content comprises:
matching the extracted reference template of static content against the frame of the video content so as to determine a match score; and
identifying presence of the extracted reference template of static content based on the match score.
13. The computing system of claim 9 , wherein the operations further include labeling the frame of the video content as program content rather than advertisement content based on the identified presence of the extracted reference template of static content in the frame of the video content.
14. The computing system of claim 9 , wherein output data identifying the labeled segment is useable to facilitate taking an action, wherein the action comprises dynamic ad insertion.
15. The computing system of claim 11 , wherein extracting the reference template of static content based on the residual component comprises:
determining a bounding box of the residual component; and
extracting a reference template of static content from a sub-region of the binary image corresponding to the bounding box.
16. The computing system of claim 13 , wherein the reference template of static content is a first reference template of static content, the method further comprising extracting by the computing system a second reference template of static content from the mean image and identifying by the computing system presence of the extracted second reference template of static content in the frame,
wherein the labeling of the frame of video content as program content rather than advertisement content is based on both the identified presence of the extracted first reference template of static content in the frame and the identified presence of the extracted second reference template of static content in the frame.
17. A non-transitory computer-readable medium having stored thereon program instructions that, upon execution by a processor, cause performance of a set of operations comprising:
determining a mean image of a set of frames of video content;
extracting a reference template of static content from the mean image;
identifying presence of the extracted reference template of static content in a frame of the set of frames of the video content;
labeling a segment within the video content as either a program segment or an advertisement segment based on the identified presence of the extracted reference template of static content in the frame of the video content; and
generating data identifying the labeled segment.
18. The non-transitory computer-readable medium of claim 17 , wherein extracting the reference template of static content from the mean image comprises:
generating a binary image from the mean image; and
applying one or more morphological operations to the binary image so as to identify a residual component of the binary image; and
extracting a reference template of static content based on the residual component.
19. The non-transitory computer-readable medium of claim 17 , wherein identifying presence of the extracted reference template of static content in a frame of the video content comprises:
matching the extracted reference template of static content against the frame of the video content so as to determine a match score; and
identifying presence of the extracted reference template of static content based on the match score.
20. The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise labeling the frame of the video content as program content rather than advertisement content based on the identifying of the extracted reference template of static content in the frame of the video content.