Generating synthetic frame features for sentinel frame matching
Particular embodiments can refine a seed sentinel frame signature for a seed sentinel frame. The seed sentinel frame may be predictable or partially predictable content that demarks a beginning and/or end of certain content in a video program. The seed sentinel frame may be first used to detect other sentinel frames in the video program. However, other sentinel frames throughout the video program, or in other video programs, may be slightly different from the given sentinel frame due to different reasons. The seed sentinel frame signature may not detect the sentinel frames of a video program with a desired accuracy. Accordingly, particular embodiments may refine the sentinel frame signature to a synthetic sentinel frame signature. The synthetic sentinel frame signature may then be used to analyze the current video program or other video programs. The synthetic sentinel frame signature may more accurately detect the sentinel frames within the video program.
1. A method comprising:
receiving, by a computing device, a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;
analyzing, by the computing device, the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content, wherein the analyzing the plurality of frames comprises:
generating a plurality of matching measures between the first signature and the plurality of frames, and
applying a first threshold to the plurality of matching measures to select the first set of frames; and
analyzing, by the computing device, the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features;
wherein the first set of frames has a first matching measure in the plurality of matching measures that meets the first threshold, and
wherein a second set of frames has a second matching measure in the plurality of matching measures that does not meet the first threshold, wherein the second set of frames are not used to generate the second signature.
2. A method comprising:
receiving, by a computing device, a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;
analyzing, by the computing device, the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content;
analyzing, by the computing device, the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features, wherein the analyzing the first set of frames to generate the second signature comprises:
comparing a distance between the first signature and the second signature to a deviation threshold, and
when the distance is within the deviation threshold, generating the second signature; and
outputting, by the computing device, the second signature as a replacement for the first signature.
3. The method of claim 2 , further comprising:
applying, by the computing device, the second signature to the plurality of frames to generate a second set of frames that are defined as the sentinel frames of the video.
4. The method of claim 3 , wherein the second set of frames include different frames from the first set of frames.
5. The method of claim 3 , further comprising:
performing a service on the second set of frames, wherein the second set of frames are of the second type of video content.
6. The method of claim 5 , wherein the service is not performed on frames in the plurality of frame that are not the second set of frames, wherein the first set of frames are of the first type of video content.
7. The method of claim 2 , wherein analyzing the first set of frames comprises:
applying a process to generate a set of clusters from frames in the first set of frames, wherein a cluster includes one or more of the first set of frames; and
generating the second signature based on one of the set of clusters.
8. The method of claim 7 , wherein the second signature is based on a position within the one of the set of clusters.
9. The method of claim 8 , wherein the position is a center of the one of the set of clusters.
10. The method of claim 7 , wherein the second signature is defined based on features from the one of the set of clusters.
11. The method of claim 2 , wherein:
the first set of frames is selected based on applying a first threshold to the first signature, and
a second set of frames is selected based on applying a second threshold to the second signature, wherein the second threshold is different from the first threshold.
12. The method of claim 11 , wherein the second threshold is less than the first threshold.
13. The method of claim 2 , wherein the video comprises a first video, the method further comprising:
applying the second signature to a second plurality of frames of a second video to generate a second set of frames that are considered the sentinel frames of the second video.
14. The method of claim 13 , further comprising:
performing a service on the second set of frames, the second set of frames being the second type of video content.
15. A non-transitory computer-readable storage medium containing instructions that, when executed, control a computer system to be configured for:
receiving a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;
analyzing the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content;
analyzing the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features, wherein the analyzing the first set of frames to generate the second signature comprises:
comparing a distance between the first signature and the second signature to a deviation threshold, and
when the distance is within the deviation threshold, generating the second signature; and
outputting the second signature as a replacement for the first signature.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed, further control the computer system to be configured for:
applying, by the computer system, the second signature to the plurality of frames to generate a second set of frames that are defined as the sentinel frames of the video.
17. A method comprising:
receiving, by a computing device, a first signature for a video that includes a plurality of frames, wherein the first signature is based on a first set of features;
analyzing, by the computing device, the plurality of frames of the video using the first signature to identify a first set of frames from the plurality of frames that are defined as sentinel frames of the video, wherein the sentinel frames identify a transition from a first type of video content to a second type of video content;
analyzing, by the computing device, the first set of frames to generate a second signature, the second signature including a second set of features that is different from the first set of features, wherein the analyzing the first set of frames to generate the second signature comprises:
comparing a distance between the first signature and the second signature to a deviation threshold, and
when the distance is within the deviation threshold, generating the second signature;
outputting, by the computing device, the second signature as a replacement for the first signature; and
when the second signature should be applied to the video, applying, by the computing device, the second signature to the plurality of frames to generate a second set of frames that replace the first set of frames as the sentinel frames of the video.
18. The method of claim 17 , wherein the video comprises a first video, the method further comprising:
when the second signature should be applied to a second video, applying the second signature to a second plurality of frames of the second video to generate a third set of frames that are considered the sentinel frames of the second video.