Information processing apparatus, information processing method, and non-transitory computer-readable storage medium
An apparatus calculates a difference between a first vector extracted by a first unit and a second vector extracted by a second unit with a smaller amount of calculation than the first unit, generates a changed difference by changing a value of an element in the difference, and updates a parameter of the second unit based on the changed difference. An amount of change from a value of an element of the difference corresponding to a first element exceeding a threshold value in the first vector to a value of an element of the changed difference corresponding to the first element is larger than an amount of change from a value of an element of the difference corresponding to a second element not exceeding the threshold value in the first vector to a value of an element of the changed difference corresponding to the second element.
1 . An information processing apparatus comprising:
at least one processor; and
at least one memory storing instructions, which when executed by the processor, cause the information processing apparatus to:
calculate a first difference vector between a first feature vector extracted by a first calculation unit based on input data, and a second feature vector extracted by a second calculation unit based on the input data with a smaller amount of calculation than the first calculation unit;
generate a first changed difference vector by changing a value of an element in the first difference vector, and update a parameter of the second calculation unit based on the first changed difference vector; and
increase an increment amount by which a value of the element of the first difference vector corresponding to the first element is increased,
wherein an amount of change from a value of an element of the first difference vector corresponding to a first element exceeding a threshold value in the first feature vector to a value of an element of the first changed difference vector corresponding to the first element is larger than an amount of change from a value of an element of the first difference vector corresponding to a second element not exceeding the threshold value in the first feature vector to a value of an element of the first changed difference vector corresponding to the second element, and
wherein the increment amount by which the value of the element of the first difference vector corresponding to the first element is increased, at a timing when the number of the elements of the first difference vector corresponding to the first element no longer decreases.
2 . The information processing apparatus according to claim 1 , wherein the instructions, when executed by the processor, further cause the information processing apparatus to: generate the first changed difference vector by increasing a value of an element of the first difference vector corresponding to a first element exceeding a threshold value in the first feature vector.
3 . The information processing apparatus according to claim 1 , wherein the instructions, when executed by the processor, further cause the information processing apparatus to: generate the first changed difference vector by reducing a value of an element of the first difference vector corresponding to a second element not exceeding a threshold value in the first feature vector.
4 . The information processing apparatus according to claim 1 , wherein the instructions, when executed by the processor, further cause the information processing apparatus to: calculate a parameter of the second calculation unit that further reduces a value based on the value of the element in the first changed difference vector, and updates the parameter of the second calculation unit to the parameter thus calculated.
5 . The information processing apparatus according to claim 1 , wherein the first calculation unit inputs input data to a hierarchical neural network and acquires a feature vector extracted at an intermediate layer of the hierarchical neural network from the input data as the first feature vector.
6 . The information processing apparatus according to claim 5 , wherein an activation function of the hierarchical neural network used by the first calculation unit is a Rectified Linear Unit (ReLU).
7 . The information processing apparatus according to claim 1 , wherein the second calculation unit inputs input data to a hierarchical neural network with a smaller number of parameters than the hierarchical neural network used by the first calculation unit, and acquires a feature vector extracted at an intermediate layer of the hierarchical neural network from the input data as the second feature vector.
8 . The information processing apparatus according to claim 7 , wherein an activation function of the hierarchical neural network used by the second calculation unit is a Rectified Linear Unit (ReLU).
9 . The information processing apparatus according to claim 1 , wherein the instructions, when executed by the processor, further cause the information processing apparatus to:
acquire a second difference vector between a first intermediate feature vector extracted by the first calculation unit based on input data, and a second intermediate feature vector extracted by the second calculation unit, based on the input data, with a smaller amount of calculation than the first calculation unit; and
generate a second changed difference vector by changing a value of an element in the second difference vector, and updates the parameter of the second calculation unit based on the second changed difference vector and the first changed difference vector,
wherein an amount of change from a value of an element of the second difference vector corresponding to a third element exceeding a threshold value in the first intermediate feature vector to a value of an element of the second changed difference vector corresponding to the third element is larger than an amount of change from a value of an element of the second difference vector corresponding to a fourth element not exceeding the threshold value in the first intermediate feature vector to a value of an element of the second changed difference vector corresponding to the fourth element.
10 . The information processing apparatus according to claim 9 , wherein the instructions, when executed by the processor, further cause the information processing apparatus to: calculate a parameter of the second calculation unit that further reduces a value based on the value of the element in the first changed difference vector and a value based on the value of the element in the second changed difference vector, and updates the parameter of the second calculation unit to the parameter thus calculated.
11 . An information processing method performed by an information processing apparatus, comprising:
calculating a first difference vector between a first feature vector extracted by a first calculation unit based on input data, and a second feature vector extracted by a second calculation unit based on the input data with a smaller amount of calculation than the first calculation unit;
generating a first changed difference vector by changing a value of an element in the first difference vector, and updating a parameter of the second calculation unit based on the first changed difference vector; and
increasing an increment amount by which a value of the element of the first difference vector corresponding to the first element is increased,
wherein an amount of change from a value of an element of the first difference vector corresponding to a first element exceeding a threshold value in the first feature vector to a value of an element of the first changed difference vector corresponding to the first element is larger than an amount of change from a value of an element of the first difference vector corresponding to a second element not exceeding the threshold value in the first feature vector to a value of an element of the first changed difference vector corresponding to the second element, and
wherein the increment amount by which the value of the element of the first difference vector corresponding to the first element is increased, at a timing when the number of the elements of the first difference vector corresponding to the first element no longer decreases.
12 . A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute, as an information processing apparatus, the computer program, which when executed by the computer causes the information processing apparatus to:
calculate a first difference vector between a first feature vector extracted by a first calculation unit based on input data, and a second feature vector extracted by a second calculation unit based on the input data with a smaller amount of calculation than the first calculation unit;
generate a first changed difference vector by changing a value of an element in the first difference vector, and update a parameter of the second calculation unit based on the first changed difference vector; and
increase an increment amount by which a value of the element of the first difference vector corresponding to the first element is increased,
wherein an amount of change from a value of an element of the first difference vector corresponding to a first element exceeding a threshold value in the first feature vector to a value of an element of the first changed difference vector corresponding to the first element is larger than an amount of change from a value of an element of the first difference vector corresponding to a second element not exceeding the threshold value in the first feature vector to a value of an element of the first changed difference vector corresponding to the second element, and
wherein the increment amount by which the value of the element of the first difference vector corresponding to the first element is increased, at a timing when the number of the elements of the first difference vector corresponding to the first element no longer decreases.