Automated data quality servicing framework for efficient utilization of information technology resources

1. A method comprising:

determining two or more storage systems storing information technology (IT) resource utilization data for an IT infrastructure;

establishing a set of data quality rules for controlling storage of the IT resource utilization data in the one or more storage systems to reduce unallocated IT resource utilization;

analyzing the IT resource utilization data stored in the two or more storage systems to detect data discrepancies affecting allocation of IT resource utilization among a plurality of users of IT resources of the IT infrastructure, a given one of the data discrepancies resulting in at least a portion of the IT resource utilization, by a first one of the plurality of users during a given period of time, at least one of (i) being allocated to at least a second one of the plurality of users for the given period of time and (ii) not being allocated to any of the plurality of users for the given period of time;

adjusting the IT resource utilization data stored in at least one of the two or more storage systems to remedy the given data discrepancy associated with the IT resource utilization data; and

modifying the set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to prevent the given data discrepancy from occurring on subsequent storage of the IT resource utilization data in the two or more storage systems;

wherein the IT resource utilization by the first user during the given period of time is characterized by (i) information associated with one or more IT services consumed by the first user during the given period of time, (ii) information associated with one or more IT software resources and IT hardware resources used to provision the one or more IT services consumed by the first user during the given period of time, and (iii) information associated with operating the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time;

wherein detecting the given data discrepancy comprises:

identifying first and second ones of the two or more storage systems comprising first and second data sources providing first and second data sets that are combined to provide at least a portion of the information associated with operating the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time; and

determining that first information in the first data set associated with operating a given one of the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time is at least a partial duplicate of second information in the second data set associated with operating the given one of the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time;

wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 further comprising generating data profiles for one or more data domains, the data profiles characterizing quality of data in the one or more data domains, the set of data quality rules comprising rules governing storage of the data in the one or more data domains.

3. The method of claim 2 wherein generating the data profiles comprises:

determining a scope of a given IT function or process;

identifying data sources in the two or more storage systems utilized by the given IT function or process;

determining input and output data formats for the given IT function or process; and

identifying computations and associated data fields for generating measures of utilization of the IT resources by the given IT function or process.

4. The method of claim 2 wherein generating the data profiles comprises:

identifying, in at least one of the two or more storage systems, one or more tables and data fields and associated data types and sizes of the identified tables and data fields;

analyzing data objects in the identified data fields to determine size, volume and usage history;

identifying at least one of counts of and percentage of rows in the identified tables with null data and unique data for one or more specified data fields;

analyzing one or more characteristics of the identified data fields, the one or more characteristics comprising at least one of minimum value, maximum value, average value and standard deviation; and

utilizing the determined size, volume and usage history, the counts, the percentages, and the one or more characteristics to determine one or more of the data fields utilized by a given IT function or process for generating measures of utilization of the IT resources by the given IT function or process.

5. The method of claim 1 wherein the data quality rules comprise at least one of:

one or more rules for maintaining an IT service catalog mapping IT services to underlying ones of the one or more IT hardware resources and the one or more IT software resources in the IT infrastructure;

one or more rules for maintaining IT software data, including categorizing IT software, maintaining a database inventory of IT software, establishing demand tracking within IT software, and standardizing name, classification and status of IT software information across different data consumers; and

one or more rules for maintaining IT hardware data, including categorizing IT hardware, maintaining a database inventory of IT hardware, maintaining relationships between IT hardware components, and maintaining relationships between IT hardware components and IT software in the IT infrastructure.

6. The method of claim 1 wherein the data quality rules comprise at least one of:

one or more rules for maintaining organization structure information, including an internal IT structure of an organization utilizing the IT infrastructure;

one or more rules for maintaining a register of capital assets and mapping capital assets to the one or more IT hardware resources and the one or more IT software resources in the IT infrastructure; and

one or more rules for maintaining licensing costs mapped to the one or more IT hardware resources and the one or more IT software resources in the IT infrastructure.

7. The method of claim 1 wherein the data quality rules comprise one or more rules for logging labor effort mapped to appropriate tasks and ones of the one or more IT hardware resources and one or more IT software resources in the IT infrastructure.

8. The method of claim 1 wherein analyzing the IT resource utilization data stored in the two or more storage systems to detect the given data discrepancy comprises comparing the first data set from the first data source with the second data set from the second data source to identify:

one or more data fields available from the first data source that are missing from the second data source;

one or more data fields available from the second data source that are missing from the first data source; and

one or more data fields that have different attributes in the first data source and the second data source.

9. The method of claim 1 wherein analyzing the IT resource utilization data stored in the two or more storage systems to detect the given data discrepancy comprises analyzing the first and second data sets from the first and second data sources to identify:

one or more data values outside a designated range for a given data field;

one or more data values having a null value; and

functional data having values within or outside one or more specifications.

10. The method of claim 1 further comprising, responsive to detecting the given data discrepancy:

identifying at least one of the first and second data sets stored in the first and second storage systems as a given data source responsible for the given data discrepancy;

identifying an entity responsible for managing the given data source;

generating a notification detailing the given data discrepancy; and

providing the notification to the identified responsible entity.

11. The method of claim 1 wherein adjusting the IT resource utilization data stored in one or more of the storage systems to remedy the given data discrepancy associated with the IT resource utilization data comprises:

identifying at least one of the first and second data sets stored in the first and second storage systems as a given data source responsible for the given data discrepancy; and

fixing the given data discrepancy at the given data source prior to transfer of said at least one of the first and second data sets to one or more data consumers.

12. The method of claim 1 wherein modifying the set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to prevent the given data discrepancy from occurring on subsequent storage of the IT resource utilization data in the two or more storage systems comprises implementing at least one data quality check to validate new data for the first and second data sets having the given data discrepancy prior to storage of the new data for the first and second data sets in the two or more storage systems.

13. The method of claim 1 wherein modifying the set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to prevent the given data discrepancy from occurring on subsequent storage of the IT resource utilization data in the two or more storage systems comprises adjusting one or more IT processes and functions that store new data for the first and second data sets having the given data discrepancy.

14. The method of claim 13 , wherein adjusting one or more of the IT processes and functions comprises at least one of:

combining two or more IT processes and functions which store new data for the first and second data sets; and

automating data flow between two or more of the IT processes and functions which store new data for the first and second data sets.

15. The method of claim 1 further comprising:

generating a report characterizing data quality trends based on analyzing the IT resource utilization data stored in the two or more storage systems over a designated time period; and

providing, over at least one network to a client device of a user responsible for managing at least a portion of the IT infrastructure, the report characterizing the data quality trends over the designated time period;

wherein the report comprises one or more views of the data quality trends separated by at least one of a data domain, a data owner and an underlying IT process.

16. The method of claim 15 further comprising utilizing the report characterizing the data quality trends to estimate an accuracy of IT resource utilization by determining unallocated IT resource utilization.

17. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device cause the at least one processing device:

to determine two or more storage systems storing information technology (IT) resource utilization data for an IT infrastructure;

to establish a set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to reduce unallocated IT resource utilization;

to analyze the IT resource utilization data stored in the two or more storage systems to detect data discrepancies affecting allocation of IT resource utilization among a plurality of users of IT resources of the IT infrastructure, a given one of the data discrepancies resulting in at least a portion of the IT resource utilization, by a first one of the plurality of users during a given period of time, at least one of (i) being allocated to at least a second one of the plurality of users for the given period of time and (ii) not being allocated to any of the plurality of users for the given period of time;

to adjust the IT resource utilization data stored in at least one of the two or more storage systems to remedy the given data discrepancy associated with the IT resource utilization data; and

to modify the set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to prevent the given data discrepancy from occurring on subsequent storage of the IT resource utilization data in the two or more storage systems;

wherein the IT resource utilization by the first user during the given period of time is characterized by (i) information associated with one or more IT services consumed by the first user during the given period of time, (ii) information associated with one or more IT software resources and IT hardware resources used to provision the one or more IT services consumed by the first user during the given period of time, and (iii) information associated with operating the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time; and

wherein detecting the given data discrepancy comprises:

identifying first and second ones of the two or more storage systems comprising first and second data sources providing first and second data sets that are combined to provide at least a portion of the information associated with operating the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time; and

determining that first information in the first data set associated with operating a given one of the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time is at least a partial duplicate of second information in the second data set associated with operating the given one of the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time.

18. The computer program product of claim 17 wherein analyzing the IT resource utilization data stored in the two or more storage systems to detect the given data discrepancy comprises comparing the first data set from the first data source with the second data set from the second data source to identify:

one or more data fields available from the first data source that are missing from the second data source;

one or more data fields available from the second data source that are missing from the first data source; and

one or more data fields that have different attributes in the first data source and the second data source.

19. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

the at least one processing device being configured to implement a controller configured:

to determine two or more storage systems storing information technology (IT) resource utilization data for an IT infrastructure;

to establish a set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to reduce unallocated IT resource utilization;

to analyze the IT resource utilization data stored in the two or more storage systems to detect data discrepancies affecting allocation of IT resource utilization among a plurality of users of IT resources of the IT infrastructure, a given one of the data discrepancies resulting in at least a portion of the IT resource utilization, by a first one of the plurality of users during a given period of time, at least one of (i) being allocated to at least a second one of the plurality of users for the given period of time and (ii) not being allocated to any of the plurality of users for the given period of time;

to adjust the IT resource utilization data stored in at least one of the two or more storage systems to remedy the given data discrepancy associated with the IT resource utilization data; and

to modify the set of data quality rules for controlling storage of the IT resource utilization data in the two or more storage systems to prevent the given data discrepancy from occurring on subsequent storage of the IT resource utilization data in the two or more storage systems;

wherein the IT resource utilization by the first user during the given period of time is characterized by (i) information associated with one or more IT services consumed by the first user during the given period of time, (ii) information associated with one or more IT software resources and IT hardware resources used to provision the one or more IT services consumed by the first user during the given period of time, and (iii) information associated with operating the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time; and

wherein detecting the given data discrepancy comprises:

identifying first and second ones of the two or more storage systems comprising first and second data sources providing first and second data sets that are combined to provide at least a portion of the information associated with operating the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time; and

determining that first information in the first data set associated with operating a given one of the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time is at least a partial duplicate of second information in the second data set associated with operating the given one of the one or more IT software resources and IT hardware resources in the IT infrastructure during the given period of time.

20. The apparatus of claim 19 wherein analyzing the IT resource utilization data stored in the two or more storage systems to detect the given data discrepancy comprises comparing the first data set from the first data source with the second data set from the second data source to identify:

one or more data fields available from the first data source that are missing from the second data source;

one or more data fields available from the second data source that are missing from the first data source; and

one or more data fields that have different attributes in the first data source and the second data source.