Automatic content recognition search optimization
View Patent ↗A processing device and a method are disclosed in which instructions are executed to determine, from a fingerprint of television content being consumed by a user, what program the user is watching so that program information may be sent for use by a smart device streaming the content to the user. The fingerprint may be compared with a database of fingerprints to find a match. The database may include fingerprints of favorite programs or of previously watched programs. The search may be narrowed according to the user's favorite programs or according to a subset of programs most likely to be watched, e.g., from a chronological history of programs the user usually watches and/or those that viewers in general usually watch in chronological order. When narrowed searches fail to find a fingerprint match, larger subsets up to and including the entire database may be searched for a match.
1. A server comprising:
at least one processing device in communication with a smart device, the smart device configured to:
display a television program to a user, the television program comprising media content;
generate a fingerprint of the content being displayed to the user; and
display overlay content that is based on program information;
computer-readable storage medium storing a database having a set of fingerprints associated with television programs, and storing instructions;
wherein the database comprises:
a first database having a first plurality of fingerprints associated with current programs;
a second database having a second plurality of fingerprints associated with recently consumed programs at the smart device;
a third database having a third plurality of fingerprints associated with all programs; and
a probable-program-changes database having a fourth plurality of fingerprints corresponding to programs the user is most likely watching or about to watch at the smart device;
wherein the at least one processing device is configured to search a plurality of databases of fingerprints until a match to the fingerprint of the content being displayed to the user is found, wherein the at least one processing device is configured to execute the instructions to:
receive, from the smart device, the fingerprint of the content being displayed to the user;
execute a search of the first database to determine whether the fingerprint matches one of the first plurality of fingerprints;
responsive to finding a match in the first database between the fingerprint and the first plurality of fingerprints, send program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the first database, execute a search of the second database to determine whether the fingerprint matches one of the second plurality of fingerprints;
responsive to finding a match in the second database between the fingerprint and the second plurality of fingerprints, send program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the second database, execute a search of the probable-program-changes database to determine whether the fingerprint matches one of the fourth plurality of fingerprints;
responsive to finding a match in the probable-program-changes database between the fingerprint and the fourth plurality of fingerprints, send program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the probable-program-changes database, execute a search of the third database to determine whether the fingerprint matches one of the third plurality of fingerprints;
responsive to finding a match in the third database between the fingerprint and the third plurality of fingerprints, send program information of a program corresponding to the matched fingerprint to the smart device; and
responsive to not finding a match in the third database, send a notification to the smart device indicating no match was found.
2. The server of claim 1 , wherein the program information comprises an identification of the program and starting and ending times of the program.
3. The server of claim 1 , wherein the program information comprises timing of commercials before the commercials are aired during the program.
4. The server of claim 1 , wherein the at least one processing device is further configured to:
track favorite programs watched most often by the user;
narrow the search of the third database to select fingerprints, from among the third plurality of fingerprints, corresponding to the favorite programs;
responsive to finding a match during the narrowed search of the third database, send program information of a program corresponding to the matched fingerprint to the smart device; and
responsive to not finding a match during the narrowed search of the third database, execute another search of the third database to determine whether the fingerprint matches one of the third plurality of fingerprints.
5. The server of claim 1 , wherein the at least one processing device is further configured to:
track a sequence of programs as normally watched chronologically by the user;
determine a subset of the set of fingerprints corresponding to programs the user is most likely watching or about to watch based on the sequence of programs; and
populate the probable-program-changes database with the subset of the set of fingerprints.
6. The server of claim 5 , wherein the at least one processing device is further configured to:
determine a probability that the user is watching or about to watch a program from the sequence of programs; and
populate the probable-program-changes database with the subset of the set of fingerprints when the probability is above a threshold probability.
7. The server of claim 1 , wherein the at least one processing device is further configured to:
track a sequence of programs as normally watched chronologically by most viewers;
determine a subset of the set of fingerprints corresponding to programs the viewers are most likely watching or about to watch based on the sequence of programs; and
populate the probable-program-changes database with the subset of the set of fingerprints.
8. The server of claim 1 , wherein the at least one processing device is further configured to:
determine a subset of the set of fingerprints corresponding to programs the user is most likely watching or about to watch based on a time of day; and
populate the probable-program-changes database with the subset of the set of fingerprints.
9. A method comprising:
receiving, by at least one processing device and from a smart device, a fingerprint of media content being displayed to a user, wherein the least one processing device is in communication with the smart device and has access to a computer-readable storage medium, wherein the computer-readable storage medium stores a database having a set of fingerprints associated with television programs, wherein the database comprises: (i) a first database having a first plurality of fingerprints associated with current programs, (ii) a second database having a second plurality of fingerprints associated with recently consumed programs at the smart device, (iii) a third database having a third plurality of fingerprints associated with all programs, and (iv) a probable-program-changes database having a fourth plurality of fingerprints corresponding to programs the user is most likely watching or about to watch at the smart device;
executing, by the at least one processing device, a search of a plurality of databases of fingerprints until a match to the fingerprint of content being displayed to the user is found, wherein the at least one processing device is configured to perform a search of the first database to determine whether the fingerprint matches one of the first plurality of fingerprints;
responsive to finding a match in the first database between the fingerprint and the first plurality of fingerprints, sending, by the at least one processing device, program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the first database, executing, by the at least one processing device, a search of the second database to determine whether the fingerprint matches one of the second plurality of fingerprints;
responsive to finding a match in the second database between the fingerprint and the second plurality of fingerprints, sending, by the at least one processing device, program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the second database, executing, by the at least one processing device, a search of the probable-program-changes database to determine whether the fingerprint matches one of the fourth plurality of fingerprints;
responsive to finding a match in the probable-program-changes database between the fingerprint and the fourth plurality of fingerprints, sending, by the at least one processing device, program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the probable-program-changes database, executing, by the at least one processing device, a search of the third database to determine whether the fingerprint matches one of the third plurality of fingerprints;
responsive to finding a match in the third database between the fingerprint and the third plurality of fingerprints, sending, by the at least one processing device, program information of a program corresponding to the matched fingerprint to the smart device; and
responsive to not finding a match in the third database, sending, by the at least one processing device, a notification to the smart device indicating no match was found.
10. The method of claim 9 , wherein the smart device is configured to (i) display a television program to the user, the television program comprising the media content, (ii) generate the fingerprint of the media content being displayed to the user and (iii) display overlay content that is based on program information.
11. The method of claim 9 , wherein the program information comprises an identification of the program and starting and ending times of the program.
12. The method of claim 9 , wherein the program information comprises timing of commercials before the commercials are aired during the program.
13. The method of claim 9 , further comprising:
tracking, by the at least one processing device, favorite programs watched most often by the user;
narrowing, by the at least one processing device, the search of the third database to select fingerprints, from among the third plurality of fingerprints, corresponding to the favorite programs;
responsive to finding a match during the narrowed search of the third database, sending, by the at least one processing device, program information of a program corresponding to the matched fingerprint to the smart device; and
responsive to not finding a match during the narrowed search of the third database, executing, by the at least one processing device, another search of the third database to determine whether the fingerprint matches one of the third plurality of fingerprints.
14. The method of claim 9 , further comprising:
tracking, by the at least one processing device, a sequence of programs as normally watched chronologically by the user;
determining, by the at least one processing device, a subset of the set of fingerprints corresponding to programs the user is most likely watching or about to watch based on the sequence of programs; and
populating, by the at least one processing device, the probable-program-changes database with the subset of the set of fingerprints.
15. The method of claim 14 , further comprising:
determining, by the at least one processing device, a probability that the user is watching or about to watch a program from the sequence of programs; and
populating, by the at least one processing device, the probable-program-changes database with the subset of the set of fingerprints when the probability is above a threshold probability.
16. The method of claim 9 , further comprising:
tracking, by the at least one processing device, a sequence of programs as normally watched chronologically by most viewers;
determining, by the at least one processing device, a subset of the set of fingerprints corresponding to programs the viewers are most likely watching or about to watch based on the sequence of programs; and
populating, by the at least one processing device, the probable-program-changes database with the subset of the set of fingerprints.
17. The method of claim 9 , further comprising:
determining, by the at least one processing device, a subset of the set of fingerprints corresponding to programs the user is most likely watching or about to watch based on a time of day; and
populating, by the at least one processing device, the probable-program-changes database with the subset of the set of fingerprints.
18. A non-transitory computer readable medium having stored thereon instructions executable by at least one processing device to cause the at least one processing device to perform operations comprising:
receiving, from a smart device, a fingerprint of media content being displayed to a user, wherein the least one processing device is in communication with the smart device and has access to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores a database having a set of fingerprints associated with television programs, wherein the database comprises: (i) a first database having a first plurality of fingerprints associated with current programs, (ii) a second database having a second plurality of fingerprints associated with recently consumed programs at the smart device, (iii) a third database having a third plurality of fingerprints associated with all programs, and (iv) a probable-program-changes database having a fourth plurality of fingerprints corresponding to programs the user is most likely watching or about to watch at the smart device;
executing a search of a plurality of databases of fingerprints until a match to the fingerprint of content being displayed to the user is found, wherein the search comprises a search of the first database to determine whether the fingerprint matches one of the first plurality of fingerprints;
responsive to finding a match in the first database between the fingerprint and the first plurality of fingerprints, sending program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the first database, executing a search of the second database to determine whether the fingerprint matches one of the second plurality of fingerprints;
responsive to finding a match in the second database between the fingerprint and the second plurality of fingerprints, sending program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the second database, executing a search of the probable-program-changes database to determine whether the fingerprint matches one of the fourth plurality of fingerprints;
responsive to finding a match in the probable-program-changes database between the fingerprint and the fourth plurality of fingerprints, sending program information of a program corresponding to the matched fingerprint to the smart device;
responsive to not finding a match in the probable-program-changes database, executing a search of the third database to determine whether the fingerprint matches one of the third plurality of fingerprints;
responsive to finding a match in the third database between the fingerprint and the third plurality of fingerprints, sending program information of a program corresponding to the matched fingerprint to the smart device; and
responsive to not finding a match in the third database, sending a notification to the smart device indicating no match was found.
19. The non-transitory computer readable medium of claim 18 , wherein the smart device is configured to (i) display a television program to the user, the television program comprising the media content, (ii) generate the fingerprint of the media content being displayed to the user and (iii) display overlay content that is based on program information.
20. The non-transitory computer readable medium of claim 18 , the operations further comprising:
tracking a sequence of programs as normally watched chronologically by the user;
determining a subset of the set of fingerprints corresponding to programs the user is most likely watching or about to watch based on the sequence of programs; and
populating the probable-program-changes database with the subset of the set of fingerprints.
21. The non-transitory computer readable medium of claim 18 , the operations further comprising:
tracking a sequence of programs as normally watched chronologically by most viewers;
determining a subset of the set of fingerprints corresponding to programs the viewers are most likely watching or about to watch based on the sequence of programs; and
populating the probable-program-changes database with the subset of the set of fingerprints.