Systems and methods for scalable delocalized information governance
The invention relates to electronic indexing, and more particularly, to the indexing, in a cloud, data held in a cloud. Systems and methods of the invention index data by accessing the data in place in the cloud and breaking a job into work items and sending the work items to multiple cloud processes that can each determine whether to index data associated with the work item or to create a new work item and have a different cloud process index the data. Each cloud process is proximal to an item that it indexes. This gives the system scale as well as an internal load-balancing.
1. A method for extracting content of a plurality of items in a storage system, the method comprising:
providing a cloud-based system configured to initiate a first cloud process, the first process configured to extract content of an item of the plurality of items, and configured to controllably initiate a second cloud process, said second cloud process configured to extract the content of said item;
accessing a storage system comprising the plurality of items, each item stored in one of a plurality of data centers, by accessing each item within the data center where that item is stored;
identifying, by the first process, a first item from among the plurality of items, which first item includes content comprising an embedded second item;
for said first item:
determining, by the first process, which process from among the first process and the second process to use to extract content from the first item, said process being a selected process; and
extracting content of said item, including the embedded second item, using the selected process in the cloud computer service;
collecting results from the extracting at a control system; and
providing all of said results in a format selected from a group consisting of PDF, Tiff, text, and the native format of the results.
2. The method of claim 1 , wherein accessing each item within the data center comprises using a password specific to that item.
3. The method of claim 1 , wherein the plurality of items includes email accounts; documents; folders; and items that are not text-searchable.
4. The method of claim 3 , wherein items that are not text-searchable include PDF-formatted files and image files, and further wherein extracting content of items that are not text searchable comprises performing a character recognition operation on those items to generate text and identifying character strings within the text.
5. The method of claim 1 , wherein each of the plurality of data centers comprises a separate building housing storage computers.
6. The method of claim 1 , wherein each item is stored in one of the plurality of data centers within a computer device comprising a non-transitory memory.
7. The method of claim 1 , wherein the control system comprises at least one control computer comprising a control processor coupled to a non-transitory memory.
8. The method of claim 1 , wherein the plurality of items comprises email accounts and further wherein accessing those email accounts includes use of a password for each account or an admin password to access multiple accounts.
9. The method of claim 1 , wherein the plurality of data centers are each part of the cloud computing service.
10. The method of claim 9 , wherein content of each item is extracted within the data center where that item is stored.
11. A system for extracting content of a plurality of items in a storage system, the system comprising:
a control computer comprising a processor coupled to a non-transitory memory containing instructions causing the control computer to coordinate a content extracting operation of data stored in a cloud comprising a plurality of items, each item stored in one of a plurality of data centers, by:
accessing each item within the data center where that item is stored;
identifying, by a first cloud process, a first item from the plurality of items, which first item includes content comprising an embedded second item;
determining, by the first cloud process, which process from among the first cloud process and a second cloud process, to use to extract the content from the first item, said process being a selected process;
causing selected process within that data center to extract content of that item;
collecting or compiling the content extracted from the plurality of items; and
providing all of the results in a format selected from a group consisting of PDF, Tiff, text, and the native format of the results.
12. The system of claim 11 , wherein accessing each item within the data center comprises using a password specific to that item.
13. The system of claim 11 , wherein the plurality of items includes email accounts; documents; folders; and items that are not text-searchable held in email system files or document management systems.
14. The system of claim 13 , wherein items that are not text-searchable include PDF-formatted files and image files, and further wherein extracting content of items that are not text searchable comprises performing a character recognition operation on those items to generate text and identifying character strings within the text.
15. The system of claim 11 , wherein each of the plurality of data centers comprises a separate building housing storage computers.
16. The system of claim 11 , wherein each item is stored in one of the plurality of data centers within a computer device comprising a non-transitory memory.
17. The system of claim 11 , wherein the control computer is further operable to cause each process to determine whether to extract content of an item directly or to itself cause a second process to index the item.
18. The system of claim 11 , wherein the plurality of items comprises email accounts or documents in file sharing or document management systems and further wherein accessing those email accounts and documents includes use of a password for each account.
19. The system of claim 11 , wherein the plurality of data centers are each part of a cloud computing service.
20. The system of claim 19 , wherein each process used to extract content of the items is provided by the cloud computing service.