Client-side video advertisement replacement using automatic content recognition
Systems and method for advertisement replacement are disclosed. A server may receive a server feed, generate a plurality of server fingerprints for a respective plurality of segments of the server feed, receive a synchronization message comprising a client fingerprint and a client time, match the client fingerprint with one of the plurality of server fingerprints, and send a synchronization response indicative of a server time associated with the one of the plurality of server fingerprints.
1. A method comprising:
receiving, at a client device, a client feed comprising media content from a source device;
generating, by the client device, a sequence of client fingerprints for segments of the received client feed, each client fingerprint comprising a corresponding segment and client feed time indicating when the corresponding segment was received by the client device;
sending a synchronization message comprising at least one client fingerprint generated from the client device to a server device, the synchronization message when received by the server device causing the server device to identify a corresponding at least one server fingerprint generated by the server device for a corresponding segment of a server feed that matches the at least one client fingerprint of the received synchronization message;
receiving, at the client device, a synchronization response from the server device, the synchronization response comprising:
data indicative of a time difference between a server feed time associated with the identified at least one server fingerprint that matches the at least one client fingerprint of the received synchronization message and the client feed time associated with the at least one client fingerprint of the received synchronization message;
an upcoming server time for an upcoming segment to be replaced; and
a preceding server fingerprint of at least one frame that immediately precedes the upcoming server segment to be replaced;
determining, by the client device, a replacement time of the client feed for the segment to be replaced based on the upcoming server time and the time difference;
determining, by the client device, when the client device generates a corresponding preceding client fingerprint of the client feed that matches the preceding server fingerprint received from the server device; and
in response to the client device generating the corresponding preceding client fingerprint that matches the preceding server fingerprint, replacing, by the client device, the segment to be replaced in the client feed with a replacement advertisement at the replacement time.
2. The method of claim 1 , wherein the server device is configured to receive the server feed from the source device, the server feed comprising the same media content as the client feed.
3. The method of claim 1 , wherein the client feed received at the client device is delayed relative to the server feed received at the server device.
4. The method of claim 1 , wherein the server feed time associated with the identified at least one server fingerprint indicates when the corresponding server segment was sent to the server device from the source device.
5. The method of claim 1 , wherein receiving the client feed comprises receiving the client feed from the source device via one of an over-the-air broadcaster or a multichannel video programming distributor.
6. The method of claim 1 , wherein generating the sequence of client fingerprints for segments of the received client feed comprises generating the sequence of client fingerprints of frames of the received client feed.
7. The method of claim 1 , further comprising, in response to generating each client fingerprint for segments of the received client feed:
sending a corresponding synchronization message comprising a most recent generated client fingerprint from the client device to the sever device; and
receiving, at the client device, a corresponding synchronization response from the server device.
8. The method of claim 1 , wherein the synchronization response further comprises a length of the segment to be replaced.
9. The method of claim 1 , further comprising:
retrieving, by the client device, the replacement advertisement from a video advertisement server in communication with the client device; and
buffering, by the client device, the replacement advertisement prior the replacement time.
10. The method of claim 1 , further comprising:
storing, by the client device, the replacement advertisement in memory hardware at the client device; and
buffering, by the client device, the replacement advertisement prior the replacement time.
11. A client device comprising:
a processing device;
memory hardware in communication with the processing device and storing in instructions that when executed by the processing device cause the processing device to perform operations comprising:
receiving a client feed comprising media content from a source device;
generating a sequence of client fingerprints for segments of the received client feed, each client fingerprint comprising a corresponding segment and client feed time indicating when the corresponding segment was received by the client device;
sending a synchronization message comprising at least one client fingerprint generated from the client device to a server device, the synchronization message when received by the server device causing the server device to identify a corresponding at least one server fingerprint generated by the server device for a corresponding segment of a server feed that matches the at least one client fingerprint of the received synchronization message;
receiving a synchronization response from the server device, the synchronization response comprising:
data indicative of a time difference between a server feed time associated with the identified at least one server fingerprint that matches the at least one client fingerprint of the received synchronization message and the client feed time associated with the at least one client fingerprint of the received synchronization message;
an upcoming server time for an upcoming segment to be replaced; and
a preceding server fingerprint of at least one frame that immediately precedes the upcoming server segment to be replaced;
determining a replacement time of the client feed for the segment to be replaced based on the upcoming server time and the time difference;
determining when the client device generates a corresponding preceding client fingerprint of the client feed that matches the preceding server fingerprint received from the server device; and
in response to the client device generating the corresponding preceding client fingerprint that matches the preceding server fingerprint, replacing the segment to be replaced in the client feed with a replacement advertisement at the replacement time.
12. The client device of claim 11 , wherein the server device is configured to receive the server feed from the source device, the server feed comprising the same media content as the client feed.
13. The client device of claim 11 , wherein the client feed received at the client device is delayed relative to the server feed received at the server device.
14. The client device of claim 11 , wherein the server feed time associated with the identified at least one server fingerprint indicates when the corresponding server segment was sent to the server device from the source device.
15. The client device of claim 11 , wherein the receiving the client feed comprises receiving the client feed from the source device via one of an over-the-air broadcaster or a multichannel video programming distributor.
16. The client device of claim 11 , wherein generating the sequence of client fingerprints for segments of the received client feed comprises generating the sequence of client fingerprints of frames of the received client feed.
17. The client device of claim 11 , wherein the operations further comprise, in response to generating each client fingerprint for segments of the received client feed:
sending a corresponding synchronization message comprising a most recent generated client fingerprint to the sever device; and
receiving a corresponding synchronization response from the server device.
18. The client device of claim 11 , wherein the synchronization response further comprises a length of the segment to be replaced.
19. The client device of claim 11 , wherein the operations further comprise:
retrieving the replacement advertisement from a video advertisement server in communication with the client device; and
buffering the replacement advertisement prior the replacement time.
20. The client device of claim 11 , wherein the operations further comprise:
storing the replacement advertisement in the memory hardware; and
buffering the replacement advertisement prior the replacement time.