Distributed architecture for tracking content indexing
An improved content indexing system is disclosed herein that content indexing system combines the functionality of the backup metadata database and the content index database into a single backup and content index database to avoid the need to perform synchronization operations. By using a single backup and content index database, the content indexing system also reduces the computing performance costs that would be associated with the synchronization operations as the amount of indexed content increases, thereby solving scalability issues.
1. A networked information management system for content indexing restored secondary copies, the networked information management system comprising:
a content indexing proxy having one or more first hardware processors, wherein the content indexing proxy is configured with first computer-executable instructions that, when executed, cause the content indexing proxy to:
receive, by a first thread executing on the content indexing proxy, identification of primary data assigned to the content indexing proxy by a master content indexing proxy, wherein the master content indexing proxy monitors a status of the content indexing proxy and statuses of one or more other content indexing proxies;
transmit, by the first thread to an indexing storage system, a query for secondary copy location data corresponding to the identified primary data;
receive, by the first thread, the secondary copy location data;
transmit, by a second thread executing on the content indexing proxy that is different than the first thread, an instruction to a first computing device that executes a media agent to restore secondary copies stored at locations indicated by the secondary copy location data;
receive, by a third thread executing on the content indexing proxy that is different than the first and second threads, an acknowledgment from the first computing device that a restoration of the secondary copies is complete; and
transmit, by a fourth thread executing on the content indexing proxy that is different than the first, second, and third threads, a request to content index the restored secondary copies; and
one or more computing devices in communication with the content indexing proxy, wherein the one or more computing devices each have one or more second hardware processors, wherein the one or more computing devices are configured with second computer-executable instructions that, when executed, cause the one or more computing devices to:
receive the request to content index the restored secondary copies;
retrieve the restored secondary copies from the first computing device; and
content index the restored secondary copies.
2. The networked information management system of claim 1 , wherein the second computer-executable instructions, when executed, further cause the one or more computing devices to generate previews and extract keywords using the restored secondary copies.
3. The networked information management system of claim 2 , wherein the second computer-executable instructions, when executed, further cause the one or more computing devices to store the generated previews in a database separate from secondary copy metadata.
4. The networked information management system of claim 2 , wherein the indexing storage system comprises an index manager and a backup and content indexing database.
5. The networked information management system of claim 4 , wherein the second computer-executable instructions, when executed, further cause the one or more computing devices to transmit the extracted keywords to the index manager.
6. The networked information management system of claim 5 , wherein the index manager is configured to mark entries in the backup and content indexing database associated with the primary data to indicate that content indexing is complete.
7. The networked information management system of claim 6 , wherein the index manager is configured to mark the entries by changing one or more status flags.
8. The networked information management system of claim 1 , wherein the first computer-executable instructions, when executed, further cause the content indexing proxy to transmit, by the third thread to the fourth thread, a request for content indexing of the restored secondary copies in response to reception of the acknowledgment.
9. The networked information management system of claim 1 , wherein a first worker thread and a second worker thread execute on the content indexing proxy, and wherein the first worker thread comprises the first thread, the second thread, the third thread, and the fourth thread.
10. The networked information management system of claim 9 , wherein the primary data is assigned to the first worker thread and second primary data is assigned to the second worker thread by the master content indexing proxy.
11. A computer-implemented method for content indexing restored secondary copies, the computer-implemented method comprising:
receiving, by a first thread executing on a content indexing proxy having one or more hardware processors, identification of primary data assigned to the content indexing proxy by a master content indexing proxy, wherein the master content indexing proxy monitors a status of the content indexing proxy and statuses of one or more other content indexing proxies;
transmitting, by the first thread to an indexing storage system, a query for secondary copy location data corresponding to the identified primary data;
transmitting, by a second thread executing on the content indexing proxy that is different than the first thread, an instruction to a first computing device that executes a media agent to restore secondary copies stored at locations indicated by the secondary copy location data;
receiving, by a third thread executing on the content indexing proxy that is different than the first and second threads, an acknowledgment from the first computing device that a restoration of the secondary copies is complete;
retrieving the restored secondary copies from the first computing device; and
content indexing the restored secondary copies.
12. The computer-implemented method of claim 11 , wherein content indexing the restored secondary copies further comprises generating previews and extracting keywords using the restored secondary copies.
13. The computer-implemented method of claim 12 , further comprising storing the generated previews in a database separate from secondary copy metadata.
14. The computer-implemented method of claim 12 , wherein the indexing storage system comprises an index manager and a backup and content indexing database.
15. The computer-implemented method of claim 14 , further comprising transmitting the extracted keywords to the index manager.
16. The computer-implemented method of claim 15 , wherein the index manager is configured to mark entries in the backup and content indexing database associated with the primary data to indicate that content indexing is complete.
17. The computer-implemented method of claim 11 , further comprising transmitting, by the third thread to the fourth thread, a request for content indexing of the restored secondary copies in response to reception of the acknowledgment.
18. The computer-implemented method of claim 11 , wherein a first worker thread and a second worker thread execute on the content indexing proxy, wherein the first worker thread comprises the first thread, the second thread, the third thread, and the fourth thread, and wherein the primary data is assigned to the first worker thread and second primary data is assigned to the second worker thread by the master content indexing proxy.
19. The computer-implemented method of claim 11 , wherein the primary data is assigned to the content indexing proxy and second primary data is assigned to a second content indexing proxy by the master content indexing proxy.
20. The computer-implemented method of claim 11 , wherein transmitting a query for secondary copy location data corresponding to the identified primary data further comprises transmitting a query for secondary copy location data corresponding to emails in a first page.