Verification of information object attributes
Systems and methods for utilizing user-verified data for training confidence level models. An example method comprises: receiving a first attribute value and a second attribute value associated with an information object representing an entity referenced by a natural language text; receiving a first confidence level associated with the first attribute value and a second confidence level associated with the second attribute value; responsive to determining that the first confidence level falls below a threshold confidence value, displaying the first attribute value using a verification graphical user interface; responsive to receiving, via the verification graphical user interface, a first input verifying the first attribute value, performing at least one of: increasing the first confidence level by a first pre-defined value or setting the first confidence level to a second pre-defined value; displaying the second attribute value using the verification graphical user interface; and responsive to failing to receive, before a triggering event, via the verification graphical user interface, a second input verifying the second attribute value, performing at least one of: increasing the second confidence level by a third pre-defined value or setting the second confidence level to a fourth pre-defined value, wherein the third pre-defined value is less than the first pre-defined value and the fourth pre-defined value is less than the second pre-defined value.
1. A method, comprising:
receiving, by a processing device, a first attribute value and a second attribute value associated with an information object representing an entity referenced by a natural language text;
evaluating a confidence function to determine a first confidence level associated with the first attribute value and a second confidence level associated with the second attribute value, wherein the confidence function is associated with a set of production rules, wherein the first attribute value and the second attribute value are produced by the set of production rules;
responsive to determining that the first confidence level falls below a threshold confidence value, displaying the first attribute value using a verification graphical user interface;
responsive to receiving, via the verification graphical user interface, a first input confirming the first attribute value, performing at least one of: increasing the first confidence level by a first pre-defined value or setting the first confidence level to a second pre-defined value;
displaying the second attribute value using the verification graphical user interface; and
responsive to failing to receive, before a triggering event, via the verification graphical user interface, a second input verifying the second attribute value, performing at least one of: increasing the second confidence level by a third pre-defined value or setting the second confidence level to a fourth pre-defined value, wherein the third pre-defined value is less than the first pre-defined value and the fourth pre-defined value is less than the second pre-defined value.
2. The method of claim 1 , wherein the triggering event is provided by expiration of a certain timeout that has been initialized by displaying the second attribute value.
3. The method of claim 1 , wherein the triggering event is provided by receiving a third user input causing the verification graphical user interface to cease displaying the second attribute value.
4. The method of claim 1 , wherein the triggering event is provided by receiving, after displaying the second attribute value, a fourth user input causing the verification graphical user interface to terminate.
5. The method of claim 1 , further comprising:
appending, to a training data set, at least part of the natural language text referencing the information object, the first attribute value, and the first confidence level.
6. The method of claim 5 , further comprising:
determining, using the training data set, at least one parameter of the confidence function.
7. The method of claim 1 , wherein receiving the attribute value further comprises:
interpreting, using a set of production rules, a plurality of semantic structures to extract a plurality of information objects representing entities referenced by the natural language text.
8. The method of claim 1 , wherein the first input verifying the first attribute value confirms the first attribute value.
9. The method of claim 1 , wherein the first input verifying the first attribute value modifies the first attribute value.
10. A system, comprising:
a memory;
a processor, coupled to the memory, the processor configured to:
receive a first attribute value and a second attribute value associated with an information object representing an entity referenced by a natural language text;
evaluate a confidence function to determine a first confidence level associated with the first attribute value and a second confidence level associated with the second attribute value, wherein the confidence function is associated with a set of production rules, wherein the first attribute value and the second attribute value are produced by the set of production rules;
responsive to determining that the first confidence level falls below a threshold confidence value, display the first attribute value using a verification graphical user interface;
responsive to receiving, via the verification graphical user interface, a first input confirming the first attribute value, perform at least one of: increasing the first confidence level by a first pre-defined value or setting the first confidence level to a second pre-defined value;
display the second attribute value using the verification graphical user interface; and
responsive to failing to receive, before a triggering event, via the verification graphical user interface, a second input verifying the second attribute value, perform at least one of: increasing the second confidence level by a third pre-defined value or setting the second confidence level to a fourth pre-defined value, wherein the third pre-defined value is less than the first pre-defined value and the fourth pre-defined value is less than the second pre-defined value.
11. The system of claim 10 , wherein the processor is further configured to:
appending, to a training data set, at least part of the natural language text referencing the information object, the first attribute value, and the first confidence level.
12. The method of claim 11 , wherein the processor is further configured to:
determining, using the training data set, at least one parameter of the confidence function.
13. The system of claim 10 , wherein receiving the attribute value further comprises:
interpreting, using a set of production rules, a plurality of semantic structures to extract a plurality of information objects representing entities referenced by the natural language text.
14. The system of claim 10 , wherein the triggering event is provided by one of: expiration of a certain timeout that has been initialized by displaying the second attribute value, receiving a third user input causing the verification graphical user interface to cease displaying the second attribute value, or receiving, after displaying the second attribute value, a fourth user input causing the verification graphical user interface to terminate.
15. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to:
receive a first attribute value and a second attribute value associated with an information object representing an entity referenced by a natural language text;
evaluate a confidence function to determine a first confidence level associated with the first attribute value and a second confidence level associated with the second attribute value, wherein the confidence function is associated with a set of production rules, wherein the first attribute value and the second attribute value are produced by the set of production rules;
responsive to determining that the first confidence level falls below a threshold confidence value, display the first attribute value using a verification graphical user interface;
responsive to receiving, via the verification graphical user interface, a first input confirming the first attribute value, perform at least one of: increasing the first confidence level by a first pre-defined value or setting the first confidence level to a second pre-defined value;
display the second attribute value using the verification graphical user interface; and
responsive to failing to receive, before a triggering event, via the verification graphical user interface, a second input verifying the second attribute value, perform at least one of: increasing the second confidence level by a third pre-defined value or setting the second confidence level to a fourth pre-defined value, wherein the third pre-defined value is less than the first pre-defined value and the fourth pre-defined value is less than the second pre-defined value.
16. The computer-readable non-transitory storage medium of claim 15 , further comprising causing the computer system to:
append, to a training data set, at least part of the natural language text referencing the information object, the first attribute value, and the first confidence level.
17. The computer-readable non-transitory storage medium of claim 16 , further comprising executable instructions causing the computer system to:
determine, using the training data set, at least one parameter of the first confidence function.
18. The computer-readable non-transitory storage medium of claim 16 , wherein receiving the attribute value further comprises:
interpreting, using a set of production rules, a plurality of semantic structures to extract a plurality of information objects representing entities referenced by the natural language text.