Method and system for deriving metadata characteristics of derivative assets
In general, the invention relates to a method for managing data. The method includes obtaining, by a data management system, a derived data request from a client, and in response to the derived data request: obtaining a plurality of data sets based on the derived data request, identifying a plurality of metadata, wherein each metadata of the plurality of metadata is associated with a data set of the plurality of data sets, performing a metadata analysis on the plurality of metadata, making a determination, based on the metadata analysis, that derived data may be generated, performing a derived metadata operation on the plurality of metadata to obtain derived data, performing a derived data operation on the data sets to obtain derived data, associating the derived metadata with the derived data, and providing the derived metadata and the derived data to the client.
1. A method for managing data, the method comprising:
obtaining, by a data management system, a derived data request from a client; and
in response to the derived data request:
obtaining a plurality of data sets based on the derived data request;
identifying a plurality of metadata, wherein each metadata of the plurality of metadata is associated with a data set of the plurality of data sets;
performing a metadata analysis on the plurality of metadata;
making a determination, based on the metadata analysis, that derived data may be generated;
performing a derived metadata operation on the plurality of metadata to obtain derived metadata;
performing a derived data operation on the data sets to obtain derived data;
associating the derived metadata with the derived data;
providing the derived metadata and the derived data to the client obtaining, by the data management system, a second derived data request from a client;
in response to the second derived data request:
obtaining a second plurality of data sets;
identifying a second plurality of metadata, wherein each metadata of the second plurality of metadata is associated with a data set of the second plurality of data sets;
performing a second metadata analysis on the second plurality of metadata;
making a second determination, based on the second metadata analysis, that derived data may not be generated; and
sending, based on the second determination, a notification to the client based on the second determination.
2. The method of claim 1 , wherein each data set of the plurality of data sets is obtained from a local data system.
3. The method of claim 1 , wherein a metadata of the plurality of metadata specifies at least one of an owner of the data, security information, licensing information, and data protection information.
4. The method of claim 1 , wherein the derived data operation comprises:
identifying conflicts between each metadata of the plurality of metadata;
applying a rules set to the plurality of metadata to resolve the conflicts; and
generating a derived metadata based on the applying.
5. The method of claim 1 , wherein the derived data operation is performed by a data analyzer executing on the data management system.
6. The method of claim 1 , wherein the derived metadata operation is performed by a derived metadata engine executing on the data management system.
7. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method, the method comprising:
obtaining, by a data management system, a derived data request from a client; and
in response to the derived data request:
obtaining a plurality of data sets based on the derived data request;
identifying a plurality of metadata, wherein each metadata of the plurality of metadata is associated with a data set of the plurality of data sets;
performing a metadata analysis on the plurality of metadata;
making a determination, based on the metadata analysis, that derived data may be generated;
performing a derived metadata operation on the plurality of metadata to obtain derived metadata;
performing a derived data operation on the data sets to obtain derived data;
associating the derived metadata with the derived data;
providing the derived metadata and the derived data to the client obtaining, by the data management system, a second derived data request from a client;
in response to the second derived data request:
obtaining a second plurality of data sets;
identifying a second plurality of metadata, wherein each metadata of the second plurality of metadata is associated with a data set of the second plurality of data sets;
performing a second metadata analysis on the second plurality of metadata;
making a second determination, based on the second metadata analysis, that derived data may not be generated; and
sending, based on the second determination, a notification to the client based on the second determination.
8. The non-transitory computer readable medium of claim 7 , wherein each data set of the plurality of data sets is obtained from a local data system.
9. The non-transitory computer readable medium of claim 7 , wherein a metadata of the plurality of metadata specifies at least one of an owner of the data, security information, licensing information, and data protection information.
10. The non-transitory computer readable medium of claim 7 , wherein the derived data operation comprises:
identifying conflicts between each metadata of the plurality of metadata;
applying a rules set to the plurality of metadata to resolve the conflicts; and
generating a derived metadata based on the applying.
11. The non-transitory computer readable medium of claim 7 , wherein the derived data operation is performed by a data analyzer executing on the data management system.
12. The non-transitory computer readable medium of claim 7 , wherein the derived metadata operation is performed by a derived metadata engine executing on the data management system.
13. A system, comprising:
a processor; and
memory comprising instructions which, when executed by the processor, perform a method, the method comprising:
obtaining, by a data management system, a derived data request from a client; and
in response to the derived data request:
obtaining a plurality of data sets based on the derived data request;
identifying a plurality of metadata, wherein each metadata of the plurality of metadata is associated with a data set of the plurality of data sets;
performing a metadata analysis on the plurality of metadata;
making a determination, based on the metadata analysis, that derived data may be generated;
performing a derived metadata operation on the plurality of metadata to obtain derived metadata;
performing a derived data operation on the data sets to obtain derived data;
associating the derived metadata with the derived data;
providing the derived metadata and the derived data to the client obtaining, by the data management system, a second derived data request from a client;
in response to the second derived data request:
obtaining a second plurality of data sets;
identifying a second plurality of metadata, wherein each metadata of the second plurality of metadata is associated with a data set of the second plurality of data sets;
performing a second metadata analysis on the second plurality of metadata;
making a second determination, based on the second metadata analysis, that derived data may not be generated; and
sending, based on the second determination, a notification to the client based on the second determination.
14. The system of claim 13 , wherein each data set of the plurality of data sets is obtained from a local data system.
15. The system of claim 13 , wherein a metadata of the plurality of metadata specifies at least one of an owner of the data, security information, licensing information, and data protection information.
16. The system of claim 13 , wherein the derived data operation comprises:
identifying conflicts between each metadata of the plurality of metadata;
applying a rules set to the plurality of metadata to resolve the conflicts; and
generating a derived metadata based on the applying.
17. The system of claim 13 ,
wherein the derived data operation is performed by a data analyzer executing on the data management system, and
wherein the derived metadata operation is performed by a derived metadata engine executing on the data management system.