Self-supervised self supervision by combining probabilistic logic with deep learning

1. A method, comprising:

receiving output from a deep probabilistic logic module in response to running an initial set of virtual evidence through the deep probabilistic logic module;

using the output to automatically determine at least one factor as new virtual evidence for use with the deep probabilistic logic module based on a score for the at least one factor, wherein the at least one factor is a function of observed data;

applying a score function to generate the score for each factor in a set of factors to use with the deep probabilistic logic module, wherein the score function is a function of the output of the deep probabilistic logic module and a factor template to add to a factor graph of the deep probabilistic logic module; and

adding the new virtual evidence to the deep probabilistic logic module based on the score.

2. The method of claim 1 , wherein the method further includes:

receiving the output from the deep probabilistic logic module in response to running the new virtual evidence and the initial set of virtual evidence through the deep probabilistic logic module;

using the output to automatically determine new factors for additional virtual evidence to use with the deep probabilistic logic module; and

adding the additional virtual evidence to the deep probabilistic logic module.

3. The method of claim 1 , wherein the method further includes:

using the score to determine a rank for each factor relative to other factors in the set of factors from a highest score to a lowest score.

4. The method of claim 3 , wherein automatically determining the at least one factor for the new virtual evidence uses one or more factors associated with the highest score.

5. The method of claim 3 , further comprising:

selecting one or more factors to send to an individual for review based on a confidence score indicating a level of uncertainty in a prediction for the one or more factors; and

generating the new virtual evidence for the one or more factors based on a response from the individual.

6. The method of claim 3 , wherein the score function is selected in response to the factor graph of the deep probabilistic logic module and a deep neural network of the deep probabilistic logic module agreeing on a prediction for the initial set of the virtual evidence.

7. The method of claim 3 , wherein the score function is selected in response to the factor graph of the deep probabilistic logic module and a deep neural network of the deep probabilistic logic module disagreeing on a prediction for the initial set of the virtual evidence.

8. The method of claim 1 , wherein the initial set of virtual evidence is a predetermined set of factors to use in training a model.

9. The method of claim 1 , wherein the at least one factor defines a probabilistic distribution over unknown label variables that is used in training a model.

10. A computer device, comprising:

a memory to store data and instructions; and

at least one processor operable to communicate with the memory, wherein the at least one processor is operable to:

receive output from a deep probabilistic logic module in communication with the computer device in response to running an initial set of virtual evidence through the deep probabilistic logic module;

use the output to automatically determine at least one factor as new virtual evidence for use with the deep probabilistic logic module based on a score for the at least one factor, wherein the at least one factor is a function of observed data;

apply a score function to generate the score for each factor in a set of factors to use with the deep probabilistic logic module, wherein the score function is a function of the output of the deep probabilistic logic module and a factor template to add to a factor graph of the deep probabilistic logic module; and

add the new virtual evidence to the deep probabilistic logic module based on the score.

11. The computer device of claim 10 , wherein the processor is further operable to:

receive the output from the deep probabilistic logic module in response to running the new virtual evidence and the initial set of virtual evidence through the deep probabilistic logic module;

use the output to automatically determine new factors for additional virtual evidence to use with the deep probabilistic logic module; and

add the additional virtual evidence to the deep probabilistic logic module.

12. The computer device of claim 10 , wherein the processor is further operable to:

use the score to determine a rank for each factor relative to other factors in the set of factors from a highest score to a lowest score.

13. The computer device of claim 12 , wherein the processor is further operable to automatically determine the at least one factor for the new virtual evidence by using one or more factors associated with the highest score.

14. The computer device of claim 12 , wherein the processor is further operable to:

select one or more factors to send to an individual for review based on a confidence score indicating a level of uncertainty in a prediction for the one or more factors; and

generate the new virtual evidence for the one or more factors based on a response from the individual.

15. The computer device of claim 12 , wherein the processor is further operable to select the score function in response to the factor graph of the deep probabilistic logic module and a deep neural network of the deep probabilistic logic module agreeing on a prediction for the initial set of the virtual evidence.

16. The computer device of claim 12 , wherein the processor is further operable to select the score function in response to the factor graph of the deep probabilistic logic module and a deep neural network of the deep probabilistic logic module disagreeing on a prediction for the initial set of the virtual evidence.

17. The computer device of claim 10 , wherein the initial set of virtual evidence is a predetermined set of factors to use in training a model.

18. The computer device of claim 10 , wherein the at least one factor defines a probabilistic distribution over unknown label variables that is used in training a model.

19. A non-transitory computer-readable medium storing instructions executable by a computer device, comprising:

at least one instruction for causing the computer device to receive output from a deep probabilistic logic module in communication with the computer device in response to running an initial set of virtual evidence through the deep probabilistic logic module;

at least one instruction for causing the computer device to use the output to automatically determine at least one factor as new virtual evidence for use with the deep probabilistic logic module based on a score for the at least one factor, wherein the at least one factor is a function of observed data;

at least one instruction for causing the computer to apply a score function to generate the score for each factor in a set of factors to use with the deep probabilistic logic module, wherein the score function is a function of the output of the deep probabilistic logic module and a factor template to add to a factor graph of the deep probabilistic logic module; and

at least one instruction for causing the computer device to add the new virtual evidence to the deep probabilistic logic module based on the score.