Distributed and tiered architecture for content search and content monitoring
View Patent ↗An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.
1. A method for fast updating of a search database, the method comprising:
storing signatures of a real time database update in sequential order as received in a buffer on a user device;
sending the signatures from the user device to a remote database without locks; and
updating a remote database with the signatures of the real time database update.
2. The method of claim 1 , wherein the signatures are associated with one or more cluster keys and the buffer is a circular buffer for each of the one or more cluster keys.
3. The method of claim 1 , wherein the capacity of the buffer, configured as a circular buffer, is determined from a maximum number of signatures expected for a cluster key.
4. The method of claim 3 , wherein the maximum number of signatures in a tiered database is determined from an expected average number of signature updates associated with a tier selected for updating.
5. The method of claim 1 , wherein the capacity of the buffer, configured as a circular buffer, is determined to require an increase in size, based on a time difference between a first time stamp of a first entry in the circular buffer and a second time stamp of a second entry that overwrites the first entry.
6. The method of claim 5 , wherein if the time difference is less than a predetermined first threshold value, a counter value is incremented which if the counter value exceeds a predetermined second threshold, the capacity of the circular buffer is increased.
7. The method of claim 5 , wherein if the time difference is greater than a predetermined first threshold value, the capacity of the circular buffer is kept the same.
8. The method of claim 1 , wherein a previous signature which is to be overwritten in the buffer, configured to operate as a circular buffer, is for a lapsed event placing the previous signature to be overwritten in another search tier.
9. The method of claim 1 , wherein the buffer, configured as a circular buffer, has a first capacity that is increased by a predetermined amount to a second capacity without using locks, the predetermined amount selected to minimize loss of data due to overwriting of entries in the circular buffer when configured at the second capacity.
10. The method of claim 1 wherein a cluster index database is updated by use of the buffer, configured as a circular buffer.
11. The method of claim 1 further comprising:
receiving an update containing signatures and associated cluster keys from a search system; and
storing, without locks, the signatures into the buffer associated with the cluster key.
12. The method of claim 11 , wherein the buffer is configured to operate as a circular buffer with a capacity that minimizes loss of data due to overwriting of entries in the circular buffer.
13. The method of claim 11 , wherein the signatures represent a unit of multimedia content and a cluster index is a data structure that stores signatures that have the same cluster key.
14. The method of claim 1 , wherein the buffer is a circular buffer and said step of storing signatures of a real time database update in sequential order as received is performed without using locks.
15. The method of claim 14 , wherein the circular buffer is configured with a capacity that minimizes loss of data due to overwriting of entries in the circular buffer.
16. The method of claim 14 , wherein the capacity of the circular buffer is determined to require an increase in size, based on a time difference between a first time stamp of a first entry in the circular buffer and a second time stamp of a second entry that overwrites the first entry.
17. The method of claim 16 , wherein if the time difference is less than a predetermined first threshold value, a counter value is incremented which if the counter value exceeds a predetermined second threshold, the capacity of the circular buffer is increased.
18. The method of claim 14 further comprising:
receiving an update containing signatures and associated cluster keys from a search system; and
storing, without locks, the signatures into the circular buffer associated with the cluster key.
19. The method of claim 18 , wherein the signatures represent a unit of multimedia content and a cluster index is a data structure that stores signatures that have the same cluster key.
20. A computer readable non-transitory medium encoded with computer readable program data and code, the program data and code when executed operable to:
store signatures of a real time database update in sequential order as received in a buffer on a user device;
send the signatures from the user device to a remote database without locks; and
update a remote database with the signatures of the real time database update.