Systems and methods for data management using zero-touch tagging
Systems and methods for data management using tagging rules and/or policies. The systems and methods described herein may allow users or administrators to easily label data, so as to organize the data in using any suitable terminology or parameters. Tagging rules (or tag rules) may apply or assign one or more tags to a data file or object. A tag may relate to various components of the data file or object. For example, a tag may relate to a creation date, author, size, or information within the data, such as whether the file or object includes a picture. Once the data is associated with one or more tags, policies may determine how the data is manipulated, stored, accessed, or otherwise used. Policies may relate to actions or operations to be performed with respect to data having one or more particular tags.
1. A data handling system comprising:
a tag rule database storing tag rules on non-transitory computer readable media, each tag rule defining when data should be tagged;
a policy database storing policies on non-transitory computer readable media, each policy defining when a policy operation should be performed with respect to tagged data;
a controller programmed with computer executable instructions for:
receiving data comprising at least one of a data file and a data object;
comparing the received data to a tag rule to determine if the data should be tagged with a tag;
based on the comparison, tagging the data by storing an association between the data and the tag; and
comparing the tag to a policy to determine if a policy operation should be performed with respect to the data, wherein the policy operation causes data associated with the tag to be moved from a slow storage device to a faster storage device for a period of time, and then moved back to the slow storage device after the period of time expires;
identifying when an updated policy is stored to the policy database, the updated policy having a corresponding updated policy operation;
identifying, by referencing a mapping database, data corresponding to the tag, the mapping database storing, on non-transitory computer readable media, the association between the data and the tag;
performing the updated policy operation on the data corresponding to the tag;
sending at least a portion of received data to a client for additional tagging, the client separate from the data handling system;
receiving the portion of received data from the client with additional tagging; and
comparing the additional tagging to a second policy to determine if a second policy operation should be performed with respect to the portion of received data.
2. The data handling system of claim 1 , further comprising a data storage device storing data on non-transitory computer readable media.
3. The data handling system of claim 2 , wherein the controller is further programmed with computer executable instructions for storing the received data on the data storage device.
4. The data handling system of claim 1 , wherein the controller is further programmed with computer executable instructions for, based on the comparison of the tag to the policy, performing the policy operation.
5. The data handling system of claim 1 , wherein the data comprises metadata, and the controller is additionally programmed with computer executable instructions for comparing the metadata to a policy to determine if a policy operation should be performed with respect to the data.
6. The data handling system of claim 1 , wherein the tag rule is a user defined tag rule.
7. The data handling system of claim 1 , wherein the policy is a user defined policy.
8. The data handling system of claim 7 , wherein the policy relates to at least one of moving the data and storing the data in a predetermined type of storage.
9. The data handling system of claim 1 , wherein the tag rule is an automatically generated tag rule.
10. The data handling system of claim 1 , further comprising a natural language system comprising:
a natural language processing engine programmed with computer executable instructions for:
receiving natural language user commands; and
converting the user commands to computer executable steps; and
a natural language dictionary database comprising stored correlations between natural language and computer executable steps.
11. The data handling system of claim 1 , wherein the controller sends the at least a portion of the received data via a webhook, message queue, or similar mechanism.
12. A method for data handling comprising:
receiving, over a wired or wireless network, data at a data ingest module via data receiving hardware circuitry, the data comprising at least one of a data file and a data object;
comparing the data to a stored tag rule via a tag rules engine using tag rule hardware circuitry to determine if the data should be tagged with a tag;
tagging the data by storing, on non-transitory computer readable media, an association between the data and the tag;
comparing the tag to a stored policy via a policy engine using policy hardware circuitry to determine if a policy operation should be performed with respect to the data, wherein the policy operation causes data associated with the tag to be moved from a slow storage device to a faster storage device for a period of time, and then moved back to the slow storage device after the period of time expires;
identifying when an updated policy is stored to a policy database, the updated policy having a corresponding updated policy operation;
identifying, by referencing a mapping database, data corresponding to the tag, the mapping database storing, on non-transitory computer readable media, the association between the data and the tag;
performing the updated policy operation on the data corresponding to the tag;
sending at least a portion of received data to a client for additional tagging, the client separate from a data handling system;
receiving the portion of received data from the client with additional tagging; and
comparing the additional tagging to a second policy to determine if a second policy operation should be performed with respect to the portion of received data.
13. The method of claim 12 , further comprising storing the received data on a data storage device.
14. The method of claim 12 , further comprising, based on the comparison of the tag to the policy, performing the policy operation.
15. The method of claim 12 , wherein the tag rule is a user defined tag rule.
16. The method of claim 12 , wherein the policy is a user defined policy.
17. The method of claim 16 , wherein the policy relates to at least one of moving the data and storing the data in a predetermined type of storage.
18. A data handling system comprising:
a data ingest module comprising data receiving hardware circuitry for receiving data comprising at least one of a data file and a data object;
a tag rules engine comprising tag rules hardware circuitry for comparing the received data to a stored tag rule to determine if the data should be tagged with a tag;
a policy engine comprising policy hardware circuitry for comparing the tag to a stored policy to determine if a policy operation should be performed with respect to the data, wherein the policy operation causes data associated with the tag to be moved from a slow storage device to a faster storage device for a period of time, and then moved back to the slow storage device after the period of time expires; and
a mapping database storing, on non-transitory computer readable media, associations between data and tags,
wherein the policy engine is to:
identify when an updated policy is stored to a policy database, the updated policy having a corresponding updated policy operation;
identify by referencing a mapping database, data corresponding to the tag the mapping database storing, on non-transitory computer readable media, the association between the data and the tag;
perform the updated policy operation on the data corresponding to the tag;
send at least a portion of received data to a client for additional tagging, the client separate from the data handling system,
receive the portion of received data from the client with additional tagging; and
compare the additional tagging to a second policy to determine if a second policy operation should be performed with respect to the portion of received data.
19. The data handling system of claim 18 , further comprising a natural language system comprising:
a natural language processing engine programmed with computer executable instructions for:
receiving natural language user commands; and
converting the user commands to computer executable steps; and
a natural language dictionary database comprising stored correlations between natural language and computer executable steps.