Apparatus and method for parallel processing of data profiling information
A computer readable medium comprising executable instructions to process data in a data profiling system includes executable instructions to establish a plurality of attribute profiling threads, distribute columns of a selected row of a table across the plurality of attribute profiling threads, and generate data profiling information.
1 . A computer readable medium storing executable instructions to process data in a data profiling system, comprising executable instructions to:
establish a plurality of attribute profiling threads;
distribute columns of a selected row of a table across the plurality of attribute profiling threads; and
generate data profiling information.
2 . The computer readable medium of claim 1 further comprising executable instructions to add metadata to the columns of the selected row.
3 . The computer readable medium of claim 2 wherein the executable instructions to add metadata include executable instructions to add a cache flag.
4 . The computer readable medium of claim 3 further comprising executable instructions to set the cache flag when a row needs to be saved.
5 . The computer readable medium of claim 2 wherein the executable instructions to add metadata include executable instructions to add a row identification.
6 . The computer readable medium of claim 5 further comprising executable instructions to record the row identification when a cache flag is set.
7 . The computer readable medium of claim 1 wherein the executable instructions to establish a plurality of attribute profiling threads include executable instructions to establish a number of attribute profiling threads corresponding to the lower value between a minimum degree of available processing parallelism and the total number of columns to be processed.
8 . The computer readable medium of claim 1 wherein the executable instructions to generate data profiling information include executable instructions to normalize the data profiling information in a standard format.
9 . The computer readable medium of claim 8 wherein the executable instructions to normalize the data profiling information in a standard format include executable instructions to specify a data store identification, a table identification, a column identification, a row identification, a column value, and attribute information.
10 . The computer readable medium of claim 1 wherein the executable instructions to establish a plurality of attribute profiling threads include executable instructions to time slice attribute profiling threads on a single processor.
11 . The computer readable medium of claim 1 wherein the executable instructions to establish a plurality of attribute profiling threads include executable instructions to process the attribute profiling threads on multiple processors.
12 . The computer readable medium of claim 1 further comprising executable instructions to display the data profiling information.
13 . The computer readable medium of claim 12 further comprising executable instructions to facilitate the display of source information from the data profiling information.