Dynamic data stream histograms for large ranges

1. A system for creating a histogram from a data set comprising a plurality of data elements, comprising:

a processor;

a plurality of internal buckets, wherein each internal bucket of the plurality of internal buckets represent values between an internal minimum value and an internal maximum value, wherein a plurality of differences of the internal minimum value and the internal maximum value of each internal bucket are heterogeneous; and

a histogram engine configured to be executed by the processor to:

populate the plurality of internal buckets with the plurality of data elements based on the internal minimum value and the internal maximum value of each internal bucket to obtain a plurality of populated internal buckets; and

create the histogram comprising a plurality of external buckets, wherein each of the plurality of external buckets comprises a total sum of at least two of the plurality of populated internal buckets,

wherein creating the histogram comprises:

identifying a minimum value and a maximum value for the plurality of data elements,

calculating a range according to the minimum value and the maximum value,

dividing the range into the plurality of external buckets,

for each of the plurality of external buckets:

calculating a staffing internal bucket of the plurality of internal buckets and an ending internal bucket of the plurality of internal buckets for each of a plurality of external buckets using the range,

summing each value of a subset of the plurality of internal buckets between the staffing internal bucket and ending internal bucket to create a total sum, and

outputting the total sum,

wherein calculating the staffing internal bucket and the ending internal bucket is based on a number of the plurality of external buckets and the range,

wherein the number of the plurality of external buckets is determined from a parameter in a command to create the histogram.

2. The system of claim 1 , wherein each of the plurality of internal buckets comprises a counter.

3. The system of claim 2 , wherein the histogram engine is further configured to:

identify an internal bucket in the plurality of internal buckets according to the scope represented by the internal bucket for each of the plurality of data elements; and

increment the counter associated with the internal bucket.

4. The system of claim 1 , wherein the histogram engine is further configured to:

receive the plurality of data elements continually from a network,

wherein the plurality of internal buckets are populated while the plurality of data elements are received.

5. The system of claim 1 , wherein the maximum value and the minimum value are determined from at least one parameter in a command to create the histogram.

6. A computer usable storage medium comprising computer readable program code embodied therein for causing a computer system to perform a method, the method comprising:

specifying a plurality of internal buckets, wherein each internal bucket of the plurality of internal buckets represent values between an internal minimum value and an internal maximum value, wherein a plurality of differences of the internal minimum value and the internal maximum value of each internal bucket are heterogeneous;

populating the plurality of internal buckets with a plurality of data elements based on the internal minimum value and the internal maximum value of each internal bucket to obtain a plurality of populated internal buckets; and

creating a histogram comprising a plurality of external buckets, wherein each of the plurality of external buckets comprises a total sum of at least two of the plurality of populated internal buckets,

wherein creating the histogram comprises:

identifying a minimum value and a maximum value for the plurality of data elements,

calculating a range according to the minimum value and the maximum value,

dividing the range into the plurality of external buckets,

for each of the plurality of external buckets:

calculating a staffing internal bucket of the plurality of internal buckets and an ending internal bucket of the plurality of internal buckets,

summing each value of a subset of the plurality of internal buckets between the staffing internal bucket and ending internal bucket to create a total sum, and

outputting the total sum,

wherein calculating the staffing internal bucket and the ending internal bucket is based on a number of the plurality of external buckets and the range,

wherein the number of the plurality of external buckets is determined from a parameter in a command to create the histogram.

7. The computer usable storage medium of claim 6 , wherein populating the plurality of internal buckets comprises:

identifying a internal bucket in the plurality of internal buckets according to the scope represented by the internal bucket for each of the plurality of data elements; and

incrementing a counter associated with the internal bucket.

8. The computer usable storage medium of claim 6 , wherein the maximum value and the minimum value are determined from at least one parameter in a command to create the histogram.

9. The computer usable storage medium of claim 6 , wherein each of the plurality of internal buckets comprises a counter.

10. The computer usable storage medium of claim 6 , the method further comprising:

receiving the plurality of data elements continually from a network,

wherein the plurality of internal buckets are populated while the plurality of data elements are received.