Training device, object detection device, training method, object detection method, and non-transitory computer-readable storage medium

1 . A training device comprising:

a processor; and

a memory, including instructions stored thereon, which when executed by the processor causes the training device to:

acquire a degree of similarity between a correct answer region indicating a region of an object in an image and each of a plurality of anchor boxes set in advance in an image;

select, among the plurality of anchor boxes, one or more anchor boxes for which the degree of similarity is greater than or equal to a predetermined threshold, for the correct answer region; and

perform training of a neural network model for detecting the object, wherein the training is performed by inputting the image into the neural network model, obtaining an error between the correct answer region and the one or more anchor boxes selected for the correct answer region, and updating one or more parameters of the neural network model based on the obtained error,

wherein if a number of the one or more anchor boxes selected for the correct answer region reaches a maximum number, the maximum number of anchor boxes selectable for the correct answer region is changed based on the degrees of similarity acquired for the one or more anchor boxes selected for the correct answer region.

2 . The training device according to claim 1 , wherein

if the maximum number of anchor boxes have been selected for the correct answer region, the maximum number of the anchor boxes for the correct answer region is changed based on the degrees of similarity acquired for the one or more anchor boxes selected for the correct answer region and the degree of similarity acquired for an anchor box that has not yet been selected for the correct answer region.

3 . The training device according to claim 2 , wherein the instructions, when executed by the processor, further cause the training device to:

if the maximum number of anchor boxes have been selected for the correct answer region, select, for the correct answer region, among the anchor boxes that have not yet been selected for the correct answer region, an anchor box with a degree of similarity whose difference from the smallest degree of similarity among the degrees of similarity of the one or more anchor boxes selected for the correct answer region is less than or equal to a specified value.

4 . The training device according to claim 2 , wherein the instructions, when executed by the processor, further cause the training device to:

obtain a total sum of degrees of similarity for each combination of anchor boxes selected for a group of correct answer regions in an image, and

determine a combination of anchor boxes for which the total sum is the largest.

5 . The training device according to claim 1 , wherein

the training is performed such that each of the one or more anchor boxes selected for the correct answer region is converted into the correct answer region, and such that an anchor box that has not been selected for the correct answer region becomes a background region.

6 . The training device according to claim 1 , wherein the instructions, when executed by the processor, further cause the training device to:

obtain an IoU (Intersection over Union) of the correct answer region and each of the plurality of anchor boxes as the degree of similarity.

7 . The training device according to claim 1 , wherein the instructions, when executed by the processor, further cause the training device to:

obtain a GIoU (Generalized Intersection over Union) of the correct answer region and each of the plurality of anchor boxes as the degree of similarity.

8 . The training device according to claim 1 , wherein the instructions, when executed by the processor, further cause the training device to:

change the maximum number of the one or more anchor boxes based on the size of the object in the image.

9 . The training device according to claim 8 , wherein

the maximum number of the one or more anchor boxes is changed based on the size of the object in the image and a predetermined reference value determined such that the smaller the size of the object in the image is, the larger the maximum number of the one or more anchor boxes is.

10 . The training device according to claim 2 , wherein the instructions, when executed by the processor, further cause the training device to:

acquire a second degree of similarity between the correct answer region and each of the plurality of anchor boxes, and

if the difference between the degree of similarity acquired for each of the one or more anchor boxes selected for the correct answer region and the degree of similarity acquired for the anchor box that has not yet been selected for the correct answer region is less than or equal to a specified value select the anchor box that has not yet been selected based on the second degree of similarity.

11 . The training device according to claim 10 , wherein

the second degree of similarity is a degree of similarity obtained based on at least one of a center distance between the correct answer region and each of the plurality of anchor boxes and a difference in size between the correct answer region and each of the plurality of anchor boxes.

12 . The training device according to claim 10 , wherein

the second degree of similarity is the sum of a distance between a center of the correct answer region and a center of each of the plurality of anchor boxes and a difference in size between the correct answer region and each of the plurality of anchor boxes.

13 . An object detection device comprising:

a processor; and

a memory, including instructions stored thereon, which when executed by the processor causes the object detection device to:

acquire a plurality of detection frames for an object and degrees of reliability of the detection frames, the detection frames and degrees of reliability being output from a neural network model trained by a training device due to an image including the object being input to the neural network model; and

integrate a first detection frame with the highest degree of reliability in the plurality of detection frames and second detection frames that are fewer in number than a maximum number and do not include the first detection frame, in the plurality of detection frames,

wherein the training device comprises:

a processor; and

a memory, including instructions stored thereon, which when executed by the processor causes the training device to:

acquire a degree of similarity between a correct answer region indicating a region of an object in an image and each of a plurality of anchor boxes set in advance in an image;

select, among the plurality of anchor boxes, one or more anchor boxes for which the degree of similarity is greater than or equal to a predetermined threshold, for the correct answer region; and

perform training of a neural network model for detecting the object, wherein the training is performed by inputting the image into the neural network model, obtaining an error between the correct answer region and the one or more anchor boxes selected for the correct answer region, and updating one or more parameters of the neural network model based on the obtained error,

wherein if a number of the one or more anchor boxes selected for the correct answer region reaches a maximum number, the maximum number of anchor boxes selectable for the correct answer region is changed based on the degrees of similarity acquired for the one or more anchor boxes selected for the correct answer region.

14 . The object detection device according to claim 13 , wherein the instructions, when executed by the processor, further cause the object detection device to:

integrate a second detection frame for which the degree of similarity to the first detection frame is greater than or equal to a first similarity degree threshold, into the first detection frame.

15 . The object detection device according to claim 14 , wherein the instructions, when executed by the processor, further cause the object detection device to:

if the number of detection frames integrated into the first detection frame is less than the maximum number, integrate, into the first detection frame, a second detection frame for which the degree of similarity to the first detection frame is smaller than the first similarity degree threshold and greater than or equal to a second similarity degree threshold, among the detection frames for which the degree of similarity to the first detection frame is less than the first similarity degree threshold.

16 . A training method to be performed by a training device, the method comprising:

acquiring a degree of similarity between a correct answer region indicating a region of an object in an image and each of a plurality of anchor boxes set in advance in an image;

selecting, among the plurality of anchor boxes, one or more anchor boxes for which the degree of similarity is greater than or equal to a predetermined threshold, for the correct answer region;

performing training of a neural network model for detecting the object, wherein the training is performed by inputting the image into the neural network model, obtaining an error between the correct answer region and the one or more anchor boxes selected in the selection, and updating one or more parameters of the neural network model based on the obtained error; and

determining that a number of the one or more anchor boxes selected for the correct answer region has reached a maximum number, and changing the maximum number of the anchor boxes selectable for the correct answer region based on the degrees of similarity acquired in the acquisition for the one or more anchor boxes selected for the correct answer region.

17 . An object detection method to be performed by an object detection device, the method comprising:

acquiring a plurality of detection frames for an object and degrees of reliability of the detection frames, the detection frames and the degrees of reliability being output from a neural network model trained by a training device due to an image including the object being input to the neural network model; and

integrating a first detection frame with the highest degree of reliability in the plurality of detection frames and second detection frames that are fewer in number than a maximum number and do not include the first detection frame, in the plurality of detection frames, wherein the training device comprises:

a processor; and

a memory, including instructions stored thereon, which when executed by the processor causes the training device to:

acquire a degree of similarity between a correct answer region indicating a region of an object in an image and each of a plurality of anchor boxes set in advance in an image;

select, among the plurality of anchor boxes, one or more anchor boxes for which the degree of similarity is greater than or equal to a predetermined threshold, for the correct answer region; and

perform training of a neural network model for detecting the object, wherein the training is performed by inputting the image into the neural network model, obtaining an error between the correct answer region and the one or more anchor boxes selected for the correct answer region, and updating one or more parameters of the neural network model based on the obtained error,

wherein if a number of the one or more anchor boxes selected for the correct answer region reaches a maximum number, the maximum number of anchor boxes selectable for the correct answer region is changed based on the degrees of similarity acquired for the one or more anchor boxes selected for the correct answer region.

18 . A non-transitory computer-readable storage medium for storing a computer program for causing a computer to:

acquire a degree of similarity between a correct answer region indicating a region of an object in an image and each of a plurality of anchor boxes set in advance in an image;

select, among the plurality of anchor boxes, one or more anchor boxes for which the degree of similarity is greater than or equal to a predetermined threshold, for the correct answer region; and

perform training of a neural network model for detecting the object, wherein the training is performed by inputting the image into the neural network model, obtaining an error between the correct answer region and the one or more anchor boxes, and updating one or more parameters of the neural network model based on the obtained error,

wherein if a number of the one or more anchor boxes selected for the correct answer region reaches a maximum number, the maximum number of anchor boxes selectable for the correct answer region is changed based on the degrees of similarity acquired for the one or more anchor boxes selected for the correct answer region.

19 . A non-transitory computer-readable storage medium for storing a computer program for causing a computer to:

acquire a plurality of detection frames for an object and degrees of reliability of the detection frames, the detection frames and degrees of reliability being output from a neural network model trained by a training device due to an image including the object being input to the neural network model; and

integrate a first detection frame with the highest degree of reliability in the plurality of detection frames and second detection frames that are fewer in number than a maximum number and do not include the first detection frame, in the plurality of detection frames,

wherein the training device comprises:

a processor; and

a memory, including instructions stored thereon, which when executed by the processor causes the training device to:

acquire a degree of similarity between a correct answer region indicating a region of an object in an image and each of a plurality of anchor boxes set in advance in an image;

select, among the plurality of anchor boxes, one or more anchor boxes for which the degree of similarity is greater than or equal to a predetermined threshold, for the correct answer region; and

perform training of a neural network model for detecting the object, wherein the training is performed by inputting the image into the neural network model, obtaining an error between the correct answer region and the one or more anchor boxes selected for the correct answer region, and updating one or more parameters of the neural network model based on the obtained error,

wherein if a number of anchor boxes selected for the correct answer region reaches a maximum number, the maximum number of anchor boxes selectable for the correct answer region is changed based on the degrees of similarity acquired for the one or more anchor boxes selected for the correct answer region.