Increasing confidence scores via DCF equation mapping and automated stream inspection
One example method includes receiving a data stream at a node of a data confidence fabric that comprises a group of nodes that are each operable to assign trust metadata to data of the data stream, inspecting the data stream to determine a data type of data in the data stream, accessing a configuration file that applies to all the nodes of the data confidence fabric, and obtaining an equation from the configuration file, mapping the equation to the data, performing a trust insertion process on the data, as specified in the equation, and generating trust metadata that is associated with the data and based on the trust insertion process.
1. A method, comprising:
receiving a data stream at a node of a data confidence fabric that comprises a plurality of nodes that are each operable to assign trust metadata to data of the data stream;
inspecting the data stream to determine a data type of data in the data stream, wherein the data type indicates a type of data generator that generated the data;
accessing a configuration file that applies to the plurality of nodes of the data confidence fabric and includes an equation column based on a data type and a child equation column based on a subtype of the data type;
in a case when a subtype of the data type is not identified in the configuration file, mapping an equation in an equation column based on the data type to the data;
in a case when a subtype of the data type is identified in the configuration file, mapping a child equation in a child equation column based on the subtype of the data to the data;
performing a trust insertion process on the data, as specified in the mapped equation; and
generating trust metadata that is associated with the data and based on the trust insertion process.
2. The method as recited in claim 1 , wherein the trust metadata comprises a data confidence score.
3. The method as recited in claim 1 , wherein each node in the plurality of nodes comprises a different respective type of data generator.
4. The method as recited in claim 1 , wherein the configuration file comprises a different respective equation for each type of data generator in the data confidence fabric.
5. The method as recited in claim 1 , wherein the trust insertion process is performed locally at the node that received the data stream.
6. The method as recited in claim 1 , wherein the trust insertion process is performed at a node remote from the node that received the data stream, and the node at which the trust insertion process is performed is identified by a query performed by the node that received the data stream.
7. The method as recited in claim 1 , further comprising parsing the mapped equation to identify trust insertion processes specified by the mapped equation, and building a data confidence fabric execution stack based on results of the parsing.
8. The method as recited in claim 1 , wherein the mapped equation specifies another trust insertion process, and the method further comprises performing the another trust insertion process on the data.
9. The method as recited in claim 1 , further comprising appending the trust metadata to the data.
10. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving a data stream at a node of a data confidence fabric that comprises a plurality of nodes that are each operable to assign trust metadata to data of the data stream;
inspecting the data stream to determine a data type of data in the data stream, wherein the data type indicates a type of data generator that generated the data;
accessing a configuration file that applies to the plurality of nodes of the data confidence fabric and includes an equation column based on a data type and a child equation column based on a subtype of the data type;
in a case when a subtype of the data type is not identified in the configuration file, mapping an equation in an equation column based on the data type to the data;
in a case when a subtype of the data type is identified in the configuration file, mapping a child equation in a child equation column based on the subtype of the data to the data;
performing a trust insertion process on the data, as specified in the mapped equation; and
generating trust metadata that is associated with the data and based on the trust insertion process.
11. The non-transitory storage medium as recited in claim 10 , wherein the trust metadata comprises a data confidence score.
12. The non-transitory storage medium as recited in claim 10 , wherein each node in the plurality of nodes comprises a different respective type of data generator.
13. The non-transitory storage medium as recited in claim 10 , wherein the configuration file comprises a different respective equation for each type of data generator in the data confidence fabric.
14. The non-transitory storage medium as recited in claim 10 , wherein the trust insertion process is performed locally at the node that received the data stream.
15. The non-transitory storage medium as recited in claim 10 , wherein the trust insertion process is performed at a node remote from the node that received the data stream, and the node at which the trust insertion process is performed is identified by a query performed by the node that received the data stream.
16. The non-transitory storage medium as recited in claim 10 , wherein the operations further comprise parsing the mapped equation to identify trust insertion processes specified by the mapped equation, and building a data confidence fabric execution stack based on results of the parsing.
17. The non-transitory storage medium as recited in claim 10 , wherein the mapped equation specifies another trust insertion process, and the operations further comprise performing the another trust insertion process on the data.
18. The non-transitory storage medium as recited in claim 10 , wherein the operations further comprise appending the trust metadata to the data.