Method and apparatus for detecting an object based on identification information of the object in continuous images
An object detection method and apparatus are disclosed. The object detection method may include acquiring identification information of an object detected from each of N (N being a preset integer of 3 or more) images continuous in order of capture from the output of a classification layer of an object detection deep learning network and correcting, based on the identification information of the object detected from each of at least two images that have been captured early in order of time, among the N images, the identification information of the object detected from an image that has been captured later in order of time than the at least two images that have been captured early in order of time, among the N images.
1. An method for object detection with respect to continuous images, at least a portion of each step of which is performed by a processor, the method comprising:
acquiring, at the processor, identification information of an object detected from each of N (N being a preset integer of 3 or more) images from an output of a classification layer of an object detection deep learning network; and
correcting, by the processor, the identification information of at least one of the object detected from at least one image among the N images based on the identification information of the object detected from each of at least two images among the N images, wherein correcting the identification information of the at least one object detected from the at least one image among the N images includes adding a newly detected object and associated identification to a result of the at least one image among the N images responsive to a determination that the newly detected object being absent from the result of the at least one image.
2. The method according to claim 1 , further comprises correcting, by the processor, at least one of whether the object detected from any one of the N images exists, an identification label, an object ID, an identification probability, coordinates of a bounding box of the object detected, or a size of the bounding box.
3. The method according to claim 1 , further comprises correcting, by the processor a size of a bounding box of the detected object.
4. The method according to claim 1 , further comprising:
generating, by the processor, a key vector based on a result of comparison between the objects detected from two different images, among the N images; and
correcting, by the processor, the identification information based on the key vector.
5. The method according to claim 4 , further comprising:
calculating, by the processor, first similarity between bounding boxes of the objects detected from the two different images; and
generating, by the processor, the key vector based on the first similarity between the bounding boxes.
6. The method according to claim 5 , wherein the key vector is based on an information of determination as to whether corresponding objects detected from the two different images are identical to each other based on the first similarity between the bounding boxes.
7. The method according to claim 5 , further comprising:
determining, by the processor, a candidate bounding box for correcting the identification information of the object detected from at least one of the N images based on the key vector; and
determining, by the processor, whether to add or merge the candidate bounding box based on a second similarity between the candidate bounding box and a bounding box of the object detected from at least one of the N images.
8. The method according to claim 7 , wherein the first similarity and the second similarity are based on morphological information of the objects detected from at least one of the N images.
9. The method according to claim 8 , wherein the first similarity and the second similarity are based further on color information of the objects detected from at least one of the N images.
10. The method according to claim 7 , wherein, among a first image, a second image, and a third image of the N images, the first image is an image captured first in order of time, and the third image is an image captured finally in order of time.
11. The method according to claim 10 , further comprises rectifying, by the processor, the identification information of the object detected from the second image based on similarity between the objects detected from the first image and the second image and similarity between objects detected from the first image and the third image.
12. The method according to claim 11 , further comprising:
generating, by the processor, a first key vector based on the first similarity between bounding boxes of the objects detected from the first image and the second image and generating a second key vector based on the first similarity between bounding boxes of the objects detected from the first image and the third image;
determining, by the processor, a candidate bounding box for rectifying the identification information of the object detected from the second image based on the first key vector and the second key vector; and
adding, by the processor, the candidate bounding box to the second image or merging the candidate bounding box with the bounding box of the object detected from the second image based on the second similarity between the candidate bounding box and the bounding box of the object detected from the second image.
13. The method according to claim 10 , further comprises correcting, by the processor, the identification information of the object detected from the third image based on similarity between the objects detected from the second image and the first image and similarity between objects detected from the second image and the third image.
14. The method according to claim 13 , further comprises:
generating, by the processor, a first key vector based on the first similarity between bounding boxes of the objects detected from the second image and the first image and generating a second key vector based on the first similarity between bounding boxes of the objects detected from the second image and the third image;
determining, by the processor, a candidate bounding box for tracking the identification information of the object detected from the third image based on the first key vector and the second key vector; and
adding, by the processor, the candidate bounding box to the third image or merging the candidate bounding box with the bounding box of the object detected from the third image based on the second similarity between the candidate bounding box and the bounding box of the object detected from the third image.
15. The method according to claim 14 , wherein
the N images further comprise a fourth image captured after capturing of the third image, and
the method further comprises correcting the identification information of an object detected from the fourth image based on similarity between the objects detected from the third image and the second image and similarity between the objects detected from the third image and the fourth image, and excluding a detected object added to the identification information of the third image based on the first image and the second image from an object comparison target detected when correcting the identification information of the object detected from the fourth image.
16. The method according to claim 13 , further comprises tracking, by the processor, the identification information of the object detected from the third image based on similarity between detected objects that exist in a region corresponding to a preset region factor, among the first image, the second image, and the third image.
17. The method according to claim 16 , wherein the preset region factor is set based on any one of an image capturing environment, classes of detected objects, or a movement speed of a camera that has captured the N images.
18. The method according to claim 1 , further comprising changing, by the processor, a shape of a bounding box of at least one of objects detected from a target image, which is one of remaining images, based on a result of comparison between objects detected from two different images, among the N images, or adding a new bounding box to the target image.
19. An apparatus for object detection with respect to continuous images, the apparatus comprising:
a memory; and
at least one processor connected to the memory, the processor being configured to execute computer-readable commands included in the memory, wherein
the at least one processor is set:
to perform an operation of acquiring identification information of an object detected from each of N (N being a preset integer of 3 or more) images from an output of a classification layer of an object detection deep learning network; and
to perform an operation to correct the identification information of the object detected from any one of the N images based on the identification information of the object detected from each of at least two images, among the N images, wherein the identification information to be corrected by addition of a newly detected object and associated identification to a result of at least one image among the N images responsive to a determination that the newly detected object being absent from the result of the at least one image.
20. The apparatus according to claim 19 , wherein the at least one processor is set to perform an operation of changing a shape of a bounding box of the object detected from a target image, which is one of remaining images, based on a result of comparison between objects detected from two different images, among the N images, or adding a new bounding box to the target image.