Distributed and tiered architecture for content search and content monitoring
View Patent ↗An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.
1. A method for fast updating of a search database, the method comprising:
creating two duplicate databases using an initial set of reference signatures;
searching on a first database of the two duplicate databases as the active database;
updating a second database of the two duplicate databases as the standby database with new signatures for new content without locks;
switching the standby database with the active database to create a new standby database; and
updating the new standby database with the new signatures for the new content without locks.
2. The method of claim 1 , wherein the first database and the second database are subsets of a reference database.
3. The method of claim 1 , wherein the standby database after being switched with the active database is the new active database.
4. The method of claim 3 further comprising:
searching for the new content in the new active database.
5. The method of claim 1 , wherein the initial set of reference signatures are organized in the two duplicate databases in two signature record arrays, wherein signature records are accessed in a search operation by use of a cluster key.
6. The method of claim 1 further comprising:
searching the active database with a cluster key generated from descriptors associated with query content, wherein the cluster key is used as a direct address in a cluster key array to retrieve a link reference address (LRA) to a list of signature records for reference content.
7. The method of claim 6 , wherein the list of signature records are stored in a signature record array that stores the reference content for the active database.
8. The method of claim 1 , wherein updating without locks includes updating without write locks and updating without read locks.
9. A method for fast updating of a search database on a processing system, the method comprising:
creating two duplicate databases each with a circular buffer using an initial set of reference signatures in each database;
searching on an active database of the two duplicate databases with an active circular buffer;
updating a standby circular buffer with new content added to the standby circular buffer without locks, wherein the standby circular buffer is associated with a standby database of the two duplicate databases;
updating the standby database with the new content from the standby circular buffer,
switching the standby database with the active database to create a new standby database; and
updating the new standby database with new content from the standby circular buffer without locks.
10. The method of claim 9 further comprising:
receiving the new signatures for the new content and associated cluster keys, wherein the new content is processed on the processing system to generate the new signatures and the associated cluster keys.
11. The method of claim 10 further comprising:
storing the new signatures into the standby circular buffer without locks, wherein the standby circular buffer is allocated for the associated cluster keys.
12. The method of claim 10 further comprising:
storing the new signatures into the standby circular buffer without locks, wherein the standby circular buffer is created with a capacity appropriate for a maximum number of signatures to be generated within a specified time.
13. The method of claim 9 , wherein the standby circular buffer is configured in a data memory of the processing system.
14. The method of claim 9 , wherein the standby circular buffer is configured with sufficient capacity to store the new signatures without overwriting.
15. The method of claim 9 , wherein the standby circular buffer is configured with a first capacity which is monitored for loss of data when the new content is added and wherein the loss of data is determined by signatures being overwriting with newly received signatures.
16. The method of claim 9 further comprising:
monitoring the standby circular buffer for loss of signatures;
notifying signatures have been lost in the standby circular buffer due to overwriting; and
increasing the capacity of the standby circular buffer and the active circular buffer to minimize the loss of signatures.
17. A method for fast updating of a search database on a processing system, the method comprising:
creating a first database locally on a client device and a second database locally on the client device with fingerprint content downloaded from a remote search server to the first database and to the second database, wherein the fingerprint content is duplicated on both the first database and on the second database;
searching on the first database that is operating as an active database;
updating the second database that is operating as a standby database with new signature content without locks;
switching the active database with the standby database to create a new active database and a new standby database; and
updating the new standby database with the new signature content without locks.
18. The method of claim 17 , wherein the new signature content is associated with one or more cluster keys that are used to directly access the new signature content in the new active database.
19. The method of claim 18 , wherein an integer interpretation of a cluster key is used to directly address a signature content array to retrieve matching signatures.
20. The method of claim 18 , wherein in integer interpretation of a cluster key is used to directly address a signature content array to retrieve a link reference address to signature records in the new active database where the new signature content is stored.