Method, apparatus, and computer-readable medium for data asset ranking
Systems, methods, and related techniques and apparatus containing instructions which when executed by one or more computing devices for determining dataset rankings by determining a lineage ordering requirement for a collection of datasets; determining, from the lineage order requirement, one or more first lineage level datasets from the collection of datasets; generating one or more first lineage level asset ranks respectively for each one of the one or more first lineage level datasets, determining at least one second lineage level dataset having an outflow to the one or more first lineage level datasets; and generating a first dataset rank for the at least one second lineage level dataset as a first function of the outflow and at least one of the one or more first lineage level asset ranks.
1. A method executed by one or more computing devices for determining dataset rankings, the method comprising:
determining a lineage ordering requirement for a collection of datasets;
determining, from the lineage order requirement, one or more first lineage level datasets from the collection of datasets;
generating one or more first lineage level asset ranks respectively for each one of the one or more first lineage level datasets;
determining at least one second lineage level dataset having an outflow to the one or more first lineage level datasets;
generating a first dataset rank for the at least one second lineage level dataset as a first function of the outflow and at least one of the one or more first lineage level asset ranks.
2. The method of claim 1 , wherein the lineage order requirement is a data user and the one or more first lineage level datasets are data tables accessed directly by the data user.
3. The method of claim 2 , wherein the lineage order requirement is an information processing system and the one or more first lineage level datasets are one or more human consumable reports generated by the information processing system.
4. The method of claim 1 further comprising generating a second dataset rank for at least one third lineage level dataset as a second function of at least a second outflow from the at least one third lineage level dataset to the at least one second lineage level dataset, the second function further dependent on the first dataset rank and a dampening factor.
5. The method of claim 1 , further comprising:
receiving a search criteria;
generating results by searching for data within the collection of datasets;
determining that the results contain first data from the at least one second lineage level datasets and second data from at least one third lineage level dataset;
determining that the at least one third lineage level dataset has a second dataset rank that is greater than the first dataset rank;
and providing the results such that the second data from the at least one third lineage level dataset is before the first data from the at least one second lineage level dataset.
6. The method of claim 5 , wherein the data search criteria includes a data use case associated with the lineage ordering requirement.
7. The method of claim 1 , wherein the collection of datasets includes a plurality of datasets residing in a plurality of physically separate data stores, the physically separate data stores interconnected by a network, the method further comprising:
determining that the at least one third lineage level dataset has a second dataset rank that is less than the first dataset rank; and
optimizing network resources to provide greater access to the second lineage level dataset than to the third lineage level dataset.
8. The method of claim 1 , wherein the collection of datasets include a third level lineage dataset on a high availability server, the method further comprising:
determining that the at least one third lineage level dataset has a second dataset rank that is less than the first dataset rank;
transferring the third lineage level dataset off of the high availability server; and
transferring the second lineage level dataset to the high availability server.
9. The method of claim 1 , further comprising:
determining that the first dataset rank exceeds a threshold; and
replicating, responsive to determining the first dataset rank exceeds a threshold, the at least second lineage level dataset.
10. The method of claim 1 , further comprising:
determining that the first dataset rank exceeds a threshold; and
apply, responsive to determining the first dataset rank exceeds a threshold, a security policy to the at least one second lineage level dataset.
11. An apparatus for determining dataset rankings, the apparatus comprising:
one or more processors; and
one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to:
determine a lineage ordering requirement for a collection of datasets;
determine, from the lineage order requirement, one or more first lineage level datasets from the collection of datasets;
generate one or more first linage level asset ranks respectively for each one of the one or more first lineage level datasets;
determine at least one second lineage level dataset having an outflow to the one or more first lineage level datasets;
generate a first dataset rank for the at least one second lineage level dataset as a first function of the outflow and at least one of the one or more first lineage level asset ranks.
12. The apparatus of claim 11 , wherein the instructions are further configured to cause at the least one of the one or more processors to:
receive a search criteria;
generate results by searching for data within the collection of datasets;
determine that the results contain first data from the at least one second lineage level datasets and second data from at least one third lineage level dataset;
determine that the at least one third lineage level dataset has a second dataset rank that is greater than the first dataset rank;
and provide the results such that the second data from the at least one third lineage level dataset is before the first data from the at least one second lineage level dataset.
13. The apparatus of claim 11 , wherein the collection of datasets includes a plurality of datasets residing in a plurality of physically separate data stores, the physically separate data stores interconnected by a network, and wherein the instructions are further configured to cause the at least one of the one or more processors to:
determine that the at least one third lineage level dataset has a second dataset rank that is less than the first dataset rank; and
optimize network resources to provide greater access to the second lineage level dataset than to the third lineage level dataset.
14. The apparatus of claim 11 , wherein the collection of datasets include a third level lineage dataset on a high availability server, and wherein the instructions are further configured to cause the at least one of the one or more processors to:
determine that the at least one third lineage level dataset has a second dataset rank that is less than the first dataset rank;
transfer the third lineage level dataset off of the high availability server; and
transfer the second lineage level dataset to the high availability server.
15. At least one non-transitory computer-readable medium storing computer-readable instructions that, when executed by one or more computing devices, cause at least one of the one or more computing devices to:
determine a lineage ordering requirement for a collection of datasets;
determine, from the lineage order requirement, one or more first lineage level datasets from the collection of datasets;
generate one or more first linage level asset ranks respectively for each one of the one or more first lineage level datasets;
determine at least one second lineage level dataset having an outflow to the one or more first lineage level datasets;
generate a first dataset rank for the at least one second lineage level dataset as a first function of the outflow and at least one of the one or more first lineage level asset ranks.
16. The at least one non-transitory computer-readable medium of claim 15 , wherein the computer readable instructions are further configured to cause the at least one of the one or more computer devices to:
receive a search criteria;
generate results by searching for data within the collection of datasets;
determine that the results contain first data from the at least one second lineage level datasets and second data from at least one third lineage level dataset;
determine that the at least one third lineage level dataset has a second dataset rank that is greater than the first dataset rank;
and provide the results such that the second data from the at least one third lineage level dataset is before the first data from the at least one second lineage level dataset.
17. The at least one non-transitory computer-readable medium of claim 15 , wherein the collection of datasets includes a plurality of datasets residing in a plurality of physically separate data stores, the physically separate data stores interconnected by a network, and wherein the computer readable instructions are further configured to cause the at least one of the one or more computer devices to:
determine that the at least one third lineage level dataset has a second dataset rank that is less than the first dataset rank; and
optimize network resources to provide greater access to the second lineage level dataset than to the third lineage level dataset.
18. The at least one non-transitory computer-readable medium of claim 15 , wherein the collection of datasets include a third level lineage dataset on a high availability server, and wherein the computer readable instructions are further configured to cause the at least one of the one or more computer devices to:
determine that the at least one third lineage level dataset has a second dataset rank that is less than the first dataset rank;
transfer the third lineage level dataset off of the high availability server; and
transfer the second lineage level dataset to the high availability server.
19. The at least one non-transitory computer-readable medium of claim 15 , and wherein the computer readable instructions are further configured to cause the at least one of the one or more computer devices to:
determining that the first dataset rank exceeds a threshold; and
replicating, responsive to determining the first dataset rank exceeds a threshold, the at least second lineage level dataset.
20. The at least one non-transitory computer-readable medium of claim 15 , and wherein the computer readable instructions are further configured to cause the at least one of the one or more computer devices to:
determine that the first dataset rank exceeds a threshold; and
apply, responsive to determining the first dataset rank exceeds a threshold, a security policy to the at least one second lineage level dataset.