Processing event messages for data objects to determine data to redact from a database

1. A computer program product for determining data to redact in an object, wherein the computer program product comprises a computer readable storage medium having computer readable program code embodied therein that when executed performs operations, the operations comprising:

receiving event messages for objects from a source server managing the objects in a storage, wherein the event messages include facets for data instances in the objects resulting from deep data inspection of the objects, wherein the facets associate data instances in the objects with redaction relevant attributes of the associated data instances that are used to determine whether the data instances for which the facets are provided should be redacted, wherein a facet indicates a redaction relevant attribute associated with a data instance in one of the objects;

adding the event messages to a message queue;

adding information to a database for the event messages in the message queue to include the facets for the objects in the event messages;

maintaining redaction rules indicating data types to redact;

querying the database to determine an object having a facet with a redaction relevant attribute matching one of the data types in the redaction rules;

determining facets in the determined object having redaction relevant attributes that match the data types indicated in the redaction rules to redact; and

redacting the data instances in the determined object indicated in the determined facets as associated with redaction relevant attributes that match the data types indicated in the redaction rules to redact.

2. The computer program product of claim 1 , wherein a facet that indicates data to redact comprises at least one of sensitive data, personally identifiable information, and financial information.

3. The computer program product of claim 1 , wherein an event message in a message queue includes metadata for an object indicating an object is an updated object, wherein the operations further comprise:

adding the metadata from the event message for the object to the database, wherein the querying the database comprises querying the database for updated objects having facets with redaction relevant attributes matching the data types in the redaction rules.

4. The computer program product of claim 1 , wherein the object determined from querying the database comprises a first object, wherein the operations further comprise:

receiving an update on a new data type to redact;

querying the database to determine a second object having a facet having a redaction relevant attribute that matches the new data type; and

redacting an instance of data from the determined second object the facet associates with the redaction relevant attribute matching the new data type.

5. The computer program product of claim 4 , wherein the operations further comprise:

adding the new data type to the redaction rules.

6. The computer program product of claim 1 , wherein an event message includes metadata on an updated object, wherein the metadata on the updated object updates metadata for the object in the database, wherein deep data inspection is performed on the updated object to update facets for the updated object that is added to the database.

7. The computer program product of claim 6 , wherein event messages are generated immediately after a modification resulting in the updated object to provide information on real-time changes to updated objects to the message queue and the database.

8. The computer program product of claim 1 , wherein there are a plurality of message queues and consumer processes, wherein the consumer processes are assigned to process event messages from one of the message queues, wherein the message queues receive event messages for objects, wherein the operations further comprise:

periodically invoking a consumer process to read the event messages in one of the message queues and add metadata and facets to the database for an object indicated in the read event messages.

9. A system for determining data to redact in an object and in communication with a source server managing objects in a storage, comprising:

a processor; and

a computer readable storage medium having computer readable program code embodied therein that when executed by the processor performs operations, the operations comprising:

receiving event messages for objects from the source server, wherein event messages include facets for data instances in the objects resulting from deep data inspection of the objects, wherein the facets associate data instances in the objects with redaction relevant attributes of data instances that are used to determine whether data instances for which the facets are provided should be redacted, wherein a facet indicates a redaction relevant attribute associated with a data instance in one of the objects;

adding the event messages to a message queue;

adding information to a database for the event messages in the message queue to include facets for the objects in the event message;

maintaining redaction rules indicating data types to redact;

querying the database to determine an object having a facet with a redaction relevant attribute matching one of the data types in the redaction rules;

determining facets in the determined object having redaction relevant attributes that match the data types indicated in the redaction rules to redact; and

redacting the data instances in the determined object indicated in the determined facets as associated with redaction relevant attributes that match the data types indicated in the redaction rules to redact.

10. The system of claim 9 , wherein a facet that indicates data to redact comprises at least one of sensitive data, personally identifiable information, and financial information.

11. The system of claim 8 , wherein the operations further comprise:

receiving an update on a new data type to redact;

querying the database to determine a second object having a facet having a redaction relevant attribute that matches the new data type; and

redacting an instance of data from the determined second object the facet associates with the redaction relevant attribute matching the new data type.

12. The system of claim 11 , wherein the operations further comprise:

adding the new data type to the redaction rules.

13. The system of claim 9 , wherein an event message includes metadata on an updated object, wherein the metadata on the updated object updates metadata for the object in the database, wherein deep data inspection is performed on the updated object to update facets for the updated object that is added to the database.

14. The system of claim 13 , wherein event messages are generated immediately after a modification resulting in the updated object to provide information on real-time changes to updated objects to the message queue and the database.

15. The system of claim 9 , wherein there are a plurality of message queues and consumer processes, wherein the consumer processes are assigned to process event messages from one of the message queues, wherein the message queues receive event messages for objects, wherein the operations further comprise:

periodically invoking a consumer process to read the event messages in one of the message queues and add metadata and facets to the database for an object indicated in each of the read event messages.

16. A method for determining data to redact in an object, comprising:

receiving event messages for objects from a source server managing the objects in a storage, wherein the event messages include facets for data instances in the objects resulting from deep data inspection of the objects, wherein the facets associate data instances in the objects with redaction relevant attributes of the associated data instances that are used to determine whether the data instances for which the facets are provided should be redacted associate data instances in the objects with redaction relevant;

adding the event messages to a message queue;

adding information to a database for the event messages in the message queue to include the facets for the objects in the event messages;

maintaining redaction rules indicating data types to redact;

querying the database to determine an object having a facet with a redaction relevant attribute matching one of the data types in the redaction rules;

determining facets in the determined object having redaction relevant attributes that match the data types indicated in the redaction rules to redact; and

redacting the data instances in the determined object indicated in the determined facets as associated with redaction relevant attributes that match the data types indicated in the redaction rules to redact.

17. The method of claim 16 , wherein a facet that indicates data to redact comprises at least one of sensitive data, personally identifiable information, and financial information.

18. The method of claim 16 , wherein the object determined from querying the database comprises a first object, further comprising:

receiving an update on a new data type to redact;

querying the database to determine a second object having a facet having a redaction relevant attribute that matches the new data type; and

redacting an instance of data from the determined second object the facet associates with the redaction relevant attribute matching the new data type.

19. The method of claim 18 , further comprising:

adding the new data type to the redaction rules.

20. The method of claim 16 , wherein an event message includes metadata on an updated object, wherein the metadata on the updated object updates metadata for the object in the database, wherein deep data inspection is performed on the updated object to update facets for the updated object that is added to the database.

21. The method of claim 20 , wherein event messages are generated immediately after a modification resulting in the updated object to provide information on real-time changes to updated objects to the message queue and the database.

22. The method of claim 16 , wherein there are a plurality of message queues and consumer processes, wherein the consumer processes are assigned to process event messages from one of the message queues, wherein the message queues receive event messages for objects, further comprising :

periodically invoking a consumer process to read the event messages in one of the message queues and add metadata and facets to the database for an object indicated in each of the read event messages.