Method for efficient database formation and search on media devices acting synchronously with television programming
View Patent ↗Techniques for efficient database formation and search in applications embedded in a media device are provided. The search may be performed synchronously with presentation of media programming content on a nearby media presentation device. A mobile media device captures some temporal fragments of the presented audio/video content on its microphone and camera, and then generates query fingerprints for the captured fragment. A local reference database resides on the mobile media device and a master reference database resides on a remote server with a most recent chunk of reference fingerprints transferred dynamically to the local mobile media device. A chunk of the query fingerprints generated locally on the mobile media device are searched on the local reference database for continuous content search and identification. The method presented automatically switches between the local search on the mobile media device and a remote search on an external search server.
1. A method for efficiently querying a database on a media device, the method comprising:
forming a reference cache database with reference fingerprints on the media device partitioned for multiple search levels configured for query searching, wherein each search level represents a larger portion of the reference cache database than a previous search level;
searching for media content captured on the media device using fingerprints of the captured media content in response to a local query search in each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding reference fingerprints that match fingerprints of the captured media content in any search level portion of the reference cache database; and
upon finding matching fingerprints in the remote search, downloading and storing a chunk of reference media content fingerprints, from a neighborhood of matching reference fingerprints on the external search database, in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material, wherein the external search database sends down to one of the search level portions of the media device a chunk of reference fingerprints for further local querying and media content tracking if the queries for the media content captured on the media device are found in the external search database and wherein for a second level query, a reference search range is limited to a time span around a previous successful search match, plus or minus N seconds and wherein an internal second level counter of search misses is incremented if the second level query is not found in the reference cache database.
2. The method of claim 1 further comprising:
performing the local query search on multiple subsets of a full local database on the media device with multiple counters used to schedule and control the local query search.
3. The method of claim 1 further comprising:
downloading from a remote server having the external search database the chunk of reference media content fingerprints before the local query search reaches the end of the reference cache database in a local memory on the media device.
4. The method of claim 1 further comprising:
continuously capturing the media content on the media device from media programming playing on a media presentation device; and
tracking the captured media content by using reference fingerprints associated with the continuously captured media content locally stored on the media device.
5. The method of claim 1 , wherein if query fingerprints of the captured media content are not found in the reference cache database of the media device, the query fingerprints are transmitted to a remote server having the external search database for further search and content identification.
6. A method for efficiently querying a database on a media device, the method comprising:
forming a reference cache database on the media device partitioned for multiple levels of query search, wherein each search level represents a larger portion of the reference cache database than a previous search level;
querying each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding a match in any search level portion of the reference cache database; and
upon finding a match in the remote search, downloading a chunk of reference media content fingerprints from a neighborhood of matching reference fingerprints on the external search database to be stored in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material,
wherein the external search database sends down to the media device a chunk of reference fingerprints for further local querying and media content tracking if the queries are found in the external search database, and
wherein the external search database responds with a title and timing of the media content, along with related metadata, and sends these data down to the media device if the queries are found in the external search database.
7. A method for efficiently querying a database on a media device, the method comprising:
forming a reference cache database on the media device partitioned for multiple levels of query search, wherein each search level represents a larger portion of the reference cache database than a previous search level;
querying each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding a match in any search level portion of the reference cache database; and
upon finding a match in the remote search, downloading a chunk of reference media content fingerprints from a neighborhood of matching reference fingerprints on the external search database to be stored in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material,
wherein the external search database sends down to the media device a chunk of reference fingerprints for further local querying and media content tracking if the queries are found in the external search database, and
wherein query tracking a reference search range is limited to the search level portion adjacent to a previous successful search match and just slightly larger in time span than a query length.
8. A method for efficiently querying a database on a media device, the method comprising:
forming a reference cache database on the media device partitioned for multiple levels of query search, wherein each search level represents a larger portion of the reference cache database than a previous search level;
querying each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding a match in any search level portion of the reference cache database; and
upon finding a match in the remote search, downloading a chunk of reference media content fingerprints from a neighborhood of matching reference fingerprints on the external search database to be stored in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material,
wherein the external search database sends down to the media device a chunk of reference fingerprints for further local querying and media content tracking if the queries are found in the external search database, and
wherein for a second level query, the reference search range is limited to a time span around a previous successful search match, plus or minus N seconds.
9. A method for efficiently querying a database on a media device, the method comprising:
forming a reference cache database on the media device partitioned for multiple levels of query search, wherein each search level represents a larger portion of the reference cache database than a previous search level;
querying each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding a match in any search level portion of the reference cache database; and
upon finding a match in the remote search, downloading a chunk of reference media content fingerprints from a neighborhood of matching reference fingerprints on the external search database to be stored in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material,
wherein the external search database sends down to the media device a chunk of reference fingerprints for further local querying and media content tracking if the queries are found in the external search database, and
wherein for a third level query, a reference search range is the entire locally accessible set of fingerprints that have been downloaded to the media device.
10. The method of claim 9 , wherein an internal counter of the search misses is incremented if the third level query is not found in the reference cache database.
11. The method of claim 10 , wherein a count value of the internal counter is compared to a first threshold and if the count value is greater than the first threshold, then the queries are sent for a remote search on the external search database.
12. The method of claim 1 , wherein the media device's reference cache database is configured to hold a chunk of audio fingerprint data and video fingerprint data corresponding to a variable length of time.
13. A method for efficiently querying a database on a media device, the method comprising:
forming a reference cache database with reference fingerprints on the media device partitioned for multiple search levels configured for query searching, wherein each search level represents a larger portion of the reference cache database than a previous search level;
searching for media content captured on the media device using fingerprints of the captured media content in response to a local query search in each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding reference fingerprints that match fingerprints of the captured media content in any search level portion of the reference cache database; and
upon finding matching fingerprints in the remote search, downloading and storing a chunk of reference media content fingerprints, from a neighborhood of matching reference fingerprints on the external search database, in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material, wherein after a remote first count value of consecutive NOT FOUND results occurred in the remote search on the external search database, a next remote second count value of queries to the external search database for additional remote searches are suppressed.
14. A computer readable non-transitory medium encoded with computer readable program data and code, the program data and code when executed operable to:
form a reference cache database on the media device partitioned for multiple levels of query search, wherein each search level represents a larger portion of the reference cache database than a previous search level;
querying each search level in progressively larger portions of the reference cache database on the media device with automatic switching between a local search on the media device and a remote search on an external search database upon not finding a match in any search level portion of the reference cache database; and
upon finding a match in the remote search, downloading a chunk of reference media content fingerprints from a neighborhood of matching reference fingerprints on the external search database to be stored in the reference cache database on the media device, whereby the reference cache database is updated with current search area reference material, wherein the external search database sends down to the media device a chunk of reference fingerprints for further local querying and media content tracking if the queries are found in the external search database, and
wherein the external search database responds with a title and timing of the media content, along with related metadata, and sends these data down to the media device if the queries are found in the external search database.