Confidence-based database management systems and methods for use therewith
A method for execution by one or more database storage nodes of a database management system includes obtaining a row of data that includes a plurality of columns. The method further includes generating a data confidence value for data within one or more columns of the plurality of columns. The method further includes storing the data confidence value in a memory of the database management system.
1. A method for execution by a database management system that includes at least one processor, the method comprising:
obtaining, by the database management system, a row of data corresponding to a plurality of columns;
generating, by the database management system, a data confidence value for data within one or more columns of the plurality of columns, wherein the generating the data confidence value includes determining a data type associated with the row of data, and determining, based on the determined data type, to generate the data confidence value, and wherein the data type includes one or more of digital sensor data, network traffic data, and genomic base pairs of a genome sequence;
generating, by the database management system, a confidence data record that includes a new column added to the one or more columns, wherein the new column includes the data confidence value; and
storing, by the database management system, the confidence data record in memory of the database management system.
2. The method of claim 1 , wherein the obtaining the row of the data comprises:
retrieving a set of data records from the memory of the database management system; and
extracting a data record from the set of data records to produce the row of data.
3. The method of claim 1 further comprises:
determining whether the data confidence value associated with the one or more columns is less than a data confidence threshold; and
when the data confidence value is less than the data confidence threshold, generating the new column.
4. The method of claim 1 further comprises:
retrieving the confidence data record; and
analyzing the confidence data record to produce a result, wherein the result includes a confidence level that is based on the data confidence value.
5. The method of claim 1 , wherein the data confidence value comprises:
an estimate of an accuracy of the row of data.
6. The method of claim 1 , wherein the data confidence value comprises:
an estimate of an accuracy of data within a column of the one or more columns.
7. The method of claim 1 , wherein the data confidence value comprises:
an estimate of a reliability level of the row of data.
8. The method of claim 1 , wherein the generating the data confidence value comprises:
interpreting one or more data points of the data within the row of data to produce the data confidence value.
9. A database storage node of a database management system, the database storage node comprising:
memory;
an interface; and
at least one processing unit operably coupled to the memory and the interface, wherein the at least one processing unit is operable to:
obtain a row of data corresponding to a plurality of columns;
generate a data confidence value for data within one or more columns of the plurality of columns, wherein the generating the data confidence value includes determining a data type associated with the row of data, and determining, based on the determined data type, to generate the data confidence value, and wherein the data type includes one or more of digital sensor data, network traffic data, and genomic base pairs of a genome sequence;
generate a confidence data record that includes a new column added to the one or more columns, wherein the new column includes the data confidence value; and
store the confidence data record in memory of the database management system.
10. The database storage node of claim 9 , wherein the processing unit is further operable to obtain the row of the data by:
retrieving, via the interface, a set of data records from the memory of the database management system; and
extracting a data record from the set of data records to produce the row of data.
11. The database storage node of claim 9 , wherein the processing unit is further operable to:
determine whether the data confidence value associated with the one or more columns is less than a data confidence threshold; and
when the data confidence value is less than the data confidence threshold, generate the new column.
12. The database storage node of claim 9 , wherein the processing unit is further operable to:
retrieve, via the interface, the confidence data record; and
analyze the confidence data record to produce a result, wherein the result includes a confidence level that is based on the data confidence value.
13. The database storage node of claim 9 , wherein the data confidence value comprises:
an estimate of an accuracy of the row of data.
14. The database storage node of claim 9 , wherein the data confidence value comprises:
an estimate of an accuracy of a data within a column of the one or more columns.
15. The database storage node of claim 9 , wherein the data confidence value comprises:
an estimate of a reliability level of the row of data.
16. The database storage node of claim 9 , wherein the processing unit is operable to generate the data confidence value by:
interpreting one or more data points of the data within the row of data to produce the data confidence value.