Methods and systems relating to network based storage
Cloud storage provides for accessible interfaces, near-instant elasticity and scalability, multi-tenancy, and metered resources within a framework of distributed resources acting to provide highly fault tolerant solutions with high data durability. However, cloud storage also has drawbacks and limitations with information uploading and how information is subsequently accessed. To date the lack of automated tools for managing tens, hundreds and thousands of users and/or documents within enterprises and organizations means that for most migrating is a massive undertaking. Accordingly, knowledge workers require a human interface to the data ingested from third-party systems that manages the data in original folder contexts/locations for each knowledge worker within the interfaces. It would be further beneficial for knowledge workers to have tools for incremental ingestion of changes from data sources to a cloud storage repository as well as determining/centralizing cloud storage repository ingestion from the data sources.
1. A method comprising:
providing a system comprising one or more microprocessors, one or more non-volatile non-transitory memories;
providing one or more network interfaces supporting electronic communications by the system to and from a communications network and therein to and from a cloud storage repository also connected to the communications network; and
storing computer executable instructions within the one or more non-volatile non-transitory memories; wherein
the computer executable instructions when executed by the one or more microprocessors configure the system to:
establish at least a policy of a plurality of policies, where each policy of the plurality of policies is associated with a policy data set and relates to the ingestion of items of electronic content to the cloud storage repository from a storage repository which is also connected to the communications network;
segment the policy data set of the policy of the plurality of policies into a series of policy blocks where each policy block of the series of policy blocks represents logical groupings of item versions of the policy data set associated with the policy of the plurality of policies; and
store within at least the one or more non-volatile non-transitory memories and the cloud storage repository each policy block of the series of policy blocks as a separate record in a policy block table;
each policy of the plurality of policies is configured with a pair of attributes comprising:
a first attribute of the pair of attributes being a query defining one or more filter criteria to be applied by the policy of the plurality of policies to the item versions of the policy data set associated with the policy of the plurality of policies; and
a second attribute of the pair of attributes being an action to be applied to a set of item versions of the policy data set where the set of item versions of the policy data set are those item versions of the policy data set matching the one or more filter criteria associated with the first attribute of the pair of attributes; and
each item version of the item versions associated with the policy data set is associated with a specific item of electronic content within the cloud storage repository.
2. The method according to claim 1 , wherein
the computer executable instructions when executed by the one or more microprocessors configure the system to:
systematically evaluate the policy data set of the policy of the plurality of policies for each policy block of the series of policy blocks independent of the remaining policy blocks of the series of policy blocks; and
establish an incremental calculation of statistics relating to the cloud storage repository where the cloud storage repository is a massively scalable cloud storage repository and the statistics relate to a resulting policy data set relating to query results established in dependence upon running a policy of the plurality of policies against items of electronic content previously ingested and stored within the cloud storage repository.
3. The method according to claim 1 , wherein
the computer executable instructions when executed by the one or more microprocessors configure the system to:
identify a policy block of the series of policy blocks as dirty when at least one of data stored within the cloud storage repository is at least one of removed, modified and added to and a policy clause of a policy is modified;
upon identifying a policy block of the series of policy blocks as dirty execute a routine to update statistics is executed against the dirty blocks to refresh their statistics, where the statistics relate to a resulting policy data set relating to query results established in dependence upon running a policy of the plurality of policies against items of electronic content previously ingested and stored within the cloud storage repository.
4. The method according to claim 1 , wherein
the computer executable instructions when executed by the one or more microprocessors configure the system to:
provide near-real time reporting of a policy data set stored within the cloud storage repository independent of the scale of the stored policy data set; wherein
the policy data set is a predetermined portion of items of electronic content previously ingested and stored within the cloud storage repository from a plurality of storage repositories of which the storage repository is one of the plurality of storage repositories.
5. The method according to claim 1 , wherein
the computer executable instructions when executed by the one or more microprocessors configure the system to:
compile and deliver to a user a graphical presentation to a user upon an electronic device associated with the user connected to the communications network, the graphical representation displaying precisely what data the user has stored within the cloud storage repository.
6. The method according to claim 1 , wherein
the computer executable instructions when executed by the one or more microprocessors configure the system to:
compile and deliver to a user an overview upon an electronic device associated with the user connected to the communications network, the overview comprising a near real time overview of data stored within the cloud storage repository which falls within the scope of a query executed by the system provided by the user; wherein
the overview relates to one of a search result defined by one or more search terms defined within the query, a policy clause result defined by one or more policy blocks of a policy of the plurality of policies defined within the query, data associated with a clause defined withinthe query, and a logical grouping of data established in dependence upon the query.