Electronic device and controlling method of electronic device for applying a logical constraint to a neural network model
A controlling method of an electronic device are provided. The controlling method includes obtaining, a first feature value corresponding to the input data, mapping information included in the first feature value to a symbolic value corresponding to each of a logically certain information and uncertain information, tweaking a gradient corresponding to the uncertain information, obtaining a second feature value in which the first feature value and the symbolic value are merged, obtaining temporary answer information including the certain information and the uncertain information, obtaining score information related to a level the temporary answer information matches a pre-defined logic rule, training the neural network model based on the tweaked gradient when the score information is less than a pre-set threshold value, and obtaining an output data on the input data based on the obtained second feature value when the score information is greater than or equal to the pre-set threshold value.
1. An electronic device configured to apply a logical constraint to a neural network model for determining an operation of an autonomous driving automobile, the electronic device comprising:
memory configured to store one or more computer programs, a neural network model, a symbolic value mapping module, a merge module, a temporary answer acquisition module, a rule check module, and a training determination module; and
one or more processors communicatively coupled to the memory,
wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
receive, by the electronic device, input data being input for the neural network model,
obtain, by the electronic device, based on the input data, a first feature value corresponding to the input data through a feature extraction module comprised in the neural network model,
input, by the electronic device, the first feature value to the symbolic value mapping module,
map, by the electronic device, information comprised in the first feature value to a symbolic value identifying the first feature value as one of logically true information, logically false information or uncertain information related to an action which may be taken when there is a high likelihood of a traffic accident occurring,
tweak, by the electronic device, a gradient corresponding to the uncertain information through the symbolic value mapping module to trigger an update on the uncertain information,
obtain, by the electronic device, a second feature value in which the first feature value and the symbolic value are merged by inputting the first feature value and the symbolic value to the merge module,
obtain, by the electronic device, temporary answer information by inputting the first feature value and the symbolic value to the temporary answer acquisition module,
obtain, by the electronic device, a score information related to a level the temporary answer information matches a pre-defined logic rule by inputting the temporary answer information to the rule check module,
based on the score information being less than a pre-set threshold value, train, by the electronic device, through the training determination module, the neural network model based on the tweaked gradient until the score information is greater than or equal to the pre-set threshold value, update the second feature value through the trained neural network model, and obtain output data based on the input data by inputting the updated second feature value into a classification module comprised in the neural network model, and
based on the score information being greater than or equal to the pre-set threshold value, obtain, by the electronic device, the output data based on the input data by inputting the obtained second feature value into the classification module,
wherein the output data is for controlling the electronic device or an external device to perform an operation matching the pre-defined logic rule, and
wherein the classification module identifies a category corresponding to the input data from among a plurality of categories based on the second feature value which is output by the feature extraction module.
2. The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
train the neural network model for the uncertain information to be updated based on the tweaked gradient, and
obtain, based on the uncertain information being updated and the score information being greater than or equal to a pre-set threshold value, the output data by inputting the second feature value updated through the trained neural network model to the classification module.
3. The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
tweak, through the symbolic value mapping module, a gradient corresponding to the uncertain information to a value other than 0 to trigger the update on the uncertain information, and
maintain a gradient corresponding to the logically true information or the logically false information to 0 for an update on the logically true information or the logically false information to not be carried out.
4. The electronic device of claim 1 ,
wherein the symbolic value mapping module comprises:
a condition symbolic value mapping module configured to map the first feature value to a condition symbolic value corresponding to a pre-defined condition, and
an action symbolic value mapping module configured to map the first feature value to an action symbolic value corresponding to a pre-defined action, and
wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
map, through the condition symbolic value mapping module, information comprised in the first feature value to a different condition symbolic value with respect to each of the logically true information, the logically false information, and the uncertain information related to the pre-defined condition, and
map, through the action symbolic value mapping module, information comprised in the first feature value to a different action symbolic value with respect to each of the logically true information, the logically false information and the uncertain information related to the pre-defined action.
5. The electronic device of claim 4 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
input the first feature value, the condition symbolic value, and the action symbolic value to the temporary answer acquisition module, and
obtain a temporary answer information comprising information on a condition comprised in the first feature value and information on an action corresponding to information on a condition comprised in the first feature value.
6. A controlling method of an electronic device configured to apply a logical constraint to a neural network model executed by one or more processors in the electronic device for determining an operation of an autonomous driving automobile, the method comprising:
receiving, by the electronic device, input data being input for the neural network model,
obtaining, by the electronic device, a first feature value corresponding to the input data through a feature extraction module comprised in the neural network model;
inputting, by the electronic device, the first feature value to a symbolic value mapping module executed by the one or more processors in the electronic device,
mapping, by the electronic device, information comprised in the first feature value to a symbolic value identifying the first feature value as one of logically true information, logically false information or uncertain information related to an action which may be taken when there is a high likelihood of a traffic accident occurring;
tweaking, by the electronic device, a gradient corresponding to the uncertain information through the symbolic value mapping module to trigger an update on the uncertain information;
obtaining, by the electronic device, a second feature value in which the first feature value and the symbolic value are merged by inputting the first feature value and the symbolic value to a merge module comprised in the electronic device;
obtaining, by the electronic device, temporary answer information by inputting the first feature value and the symbolic value to a temporary answer acquisition module comprised in the electronic device;
obtaining, by the electronic device, a score information related to a level the temporary answer information matches a pre-defined logic rule by inputting the temporary answer information to a rule check module comprised in the electronic device;
based on the score information being less than a pre-set threshold value, training, through a training determination module, the neural network model based on the tweaked gradient until the score information is greater than or equal to the pre-set threshold value, updating the second feature value through the trained neural network model, and obtaining output data based on the input data by inputting the updated second feature value into a classification module comprised in the neural network model; and
based on the score information being greater than or equal to the pre-set threshold value, obtaining the output data on the input data by inputting the obtained second feature value into the classification module;
wherein the output data is for controlling the electronic device or an external device to perform an operation matching the pre-defined logic rule, and
wherein the classification module identifies a category corresponding to the input data from among a plurality of categories based on the second feature value which is output by the feature extraction module.
7. The method of claim 6 , further comprising:
training the neural network model for the uncertain information to be updated based on the tweaked gradient; and
obtaining, based on the uncertain information being updated and the score information being greater than or equal to a pre-set threshold value, the output data by inputting the second feature value updated through the trained neural network model to the classification module.
8. The method of claim 6 , wherein the tweaking the gradient comprises:
tweaking, through the symbolic value mapping module, a gradient corresponding to the uncertain information to a value other than 0 to trigger the update on the uncertain information, and
maintaining a gradient corresponding to the logically true information or the logically false information to 0 for an update on the logically true information or the logically false information to not be carried out.
9. The method of claim 6 ,
wherein the symbolic value mapping module comprises:
a condition symbolic value mapping module configured to map the first feature value to a condition symbolic value corresponding to a pre-defined condition, and
an action symbolic value mapping module configured to map the first feature value to an action symbolic value corresponding to a pre-defined action, and
wherein the mapping of the symbolic value comprises:
mapping, through the condition symbolic value mapping module, information comprised in the first feature value to a different condition symbolic value with respect to each of the logically true information, the logically false information, and the uncertain information related to the pre-defined condition, and
mapping, through the action symbolic value mapping module, information comprised in the first feature value to a different action symbolic value with respect to each of the logically true information, the logically false information, and the uncertain information related to the pre-defined action.
10. The method of claim 9 , wherein the obtaining of the temporary answer information comprises:
inputting the first feature value, the condition symbolic value, and the action symbolic value to the temporary answer acquisition module, and
obtaining temporary answer information comprising information on a condition comprised in the first feature value and information on an action corresponding to information on a condition comprised in the first feature value.
11. One or more non-transitory computer-readable storage media storing one or more computer programs including computer-executable instructions that, when executed by one or more processors of an electronic device individually or collectively, cause the electronic device to perform operations, the operations comprising:
receiving, by the electronic device, input data being input for a neural network model for determining an operation of an autonomous driving automobile,
obtaining, by the electronic device, a first feature value corresponding to the input data through a feature extraction module comprised in the neural network model;
inputting, by the electronic device, the first feature value to a symbolic value mapping module comprised in the electronic device,
mapping, by the electronic device, information comprised in the first feature value to a symbolic value identifying the first feature value as one of logically true information, logically false information or uncertain information related to an action which may be taken when there is a high likelihood of a traffic accident occurring;
tweaking, by the electronic device, a gradient corresponding to the uncertain information through the symbolic value mapping module to trigger an update on the uncertain information;
obtaining, by the electronic device, a second feature value in which the first feature value and the symbolic value are merged by inputting the first feature value and the symbolic value to a merge module comprised in the electronic device;
obtaining, by the electronic device, temporary answer information by inputting the first feature value and the symbolic value to a temporary answer acquisition module comprised in the electronic device;
obtaining, by the electronic device, score information related to a level the temporary answer information matches a pre-defined logic rule by inputting the temporary answer information to a rule check module comprised in the electronic device; and
based on the score information being less than a pre-set threshold value, training, by the electronic device, through a training determination module executed by the one or more processors in the electronic device, the neural network model based on the tweaked gradient until the score information is greater than or equal to the pre-set threshold value, update the second feature value through the trained neural network model, and obtaining output data based on the input data by inputting the updated second feature value into a classification module comprised in the neural network model, and
based on the score information being greater than or equal to the pre-set threshold value, obtaining, by the electronic device, the output data based on the input data by training by inputting the obtained second feature value into the classification module,
wherein the output data is for controlling the electronic device or an external device to perform an operation matching the pre-defined logic rule, and
wherein the classification module identifies a category corresponding to the input data from among a plurality of categories based on the second feature value which is output by the feature extraction module.
12. The one or more non-transitory computer-readable storage media of claim 11 , wherein the symbolic value mapping module comprises:
a feature conversion module configured to map the information included in the first feature value to the symbolic value, and
a gradient tweaking module configured to tweak the gradient corresponding to logically uncertain information from among the information included in the first feature value.
13. The one or more non-transitory computer-readable storage media of claim 11 , wherein the training determination module is configured to determine training of the neural network model based on first score information related to a level matching the pre-defined logic rule and second score information related to a level the output data matches a learning data of the neural network model.
14. The one or more non-transitory computer-readable storage media of claim 13 , wherein the training determination module is configured to determine the training of the neural network model based on comprehensive score information that is a sum of the first score information and the second score information.
15. The one or more non-transitory computer-readable storage media of claim 13 , wherein the training determination module is configured to determine the training of the neural network model based on comprehensive score information that is a product of the first score information and the second score information.