Object detecting device, object detecting method, and recording medium
An object detecting device includes: a first luminance obtainer and a distance obtainer that obtain a first luminance image and a depth image, respectively; a first clustering unit that generates a group that is a set of pixels taken as the same object in the depth image, and determines that the group is a cluster when a total number of pixels included in the group is at least a first threshold; a second clustering unit that determines that the group is a cluster, when (i) the total number of pixels included in the group is less than the first threshold and (ii) luminance of a pixel group in the first luminance image which corresponds to the group is at least a second threshold; and a 3D object detector that generates 3D object information indicating an object in the depth image which is detected based on the cluster.
1 . An object detecting device comprising:
a distance obtainer that obtains a depth image;
a first luminance obtainer that obtains a first luminance image corresponding to an image-capturing area same as an image capturing area of the depth image;
a first clustering unit that generates one or more groups, each of which is a set of pixels that is within a certain range and is taken as a same object in the depth image, and determines that any of the one or more groups is a first cluster when a total number of pixels included therein is at least a first threshold;
a second clustering unit that determines that, when it is determined, by the first clustering unit that the total number of pixels included in a group among the one or more groups is less than the first threshold, determines whether luminance of a pixel group that is in the first luminance image and corresponds to the group is at least a second threshold, and when it is determined that the luminance of the pixel group is at least the second threshold, detects the group as a second cluster; and
a three-dimensional (3D) object detector that detects an object in the depth image based on at least one of the first cluster or the second cluster, and generates 3D object information indicating the object detected.
2 . The object detecting device according to claim 1 , wherein
the luminance of the pixel group in the first luminance image is an average luminance of pixels included in the pixel group.
3 . The object detecting device according to claim 1 , wherein
the luminance of the pixel group in the first luminance image is a maximum luminance of pixels included in the pixel group.
4 . The object detecting device according to claim 1 , wherein
the luminance of the pixel group in the first luminance image includes both of an average luminance and a maximum luminance of pixels included in the pixel group,
the second threshold includes a threshold for the average luminance and a threshold for the maximum luminance, and
the second clustering unit determines that the group is the second cluster, when the average luminance is at least the threshold for the average luminance and the maximum luminance is at least the threshold for the maximum luminance.
5 . The object detecting device according to claim 1 , further comprising:
a two-dimensional (2D) object detector that detects an object included in a 2D image corresponding to the image-capturing area, and generates 2D object information indicating the object detected; and
a fusion unit that merges the 3D object information and the 2D object information together.
6 . The object detecting device according to claim 5 , wherein
when an object that is among objects included in the 3D object information and corresponds to the second cluster determined by the second clustering unit overlaps the object indicated in the 2D object information, the fusion unit further determines that the object corresponding to the second cluster is a 3D object, and when the object corresponding to the second cluster determined by the second clustering unit does not overlap the object indicated in the 2D object information, the fusion unit further determines that the second cluster determined by the second clustering unit is not a cluster and that the object corresponding to the second cluster is not a 3D object.
7 . The object detecting device according to claim 5 , wherein the 2D image is the first luminance image.
8 . The object detecting device according to claim 5 , further comprising:
a second luminance obtainer that obtains a second luminance image for light having a wavelength different from a wavelength of light for which the first luminance image is obtained, wherein
the 2D image is either the second luminance image or a third luminance image obtained by combining the first luminance image and the second luminance image.
9 . The object detecting device according to claim 8 , further comprising:
a light emitter that emits infrared light;
an image sensor that includes first pixels having sensitivity to infrared light and second pixels having sensitivity to visible light; and
a signal processor that generates, using the light emitter and the image sensor, the first luminance image and the depth image from pixel values of the first pixels and the second luminance image from pixel values of the second pixels, wherein
the first luminance obtainer, the second luminance obtainer, and the distance obtainer obtain the first luminance image, the second luminance image, and the depth image, respectively, from the signal processor.
10 . The object detecting device according to claim 1 , further comprising:
a light emitter that emits infrared light;
an image sensor that receives reflected light that is a reflection of the infrared light; and
a signal processor that generates the first luminance image and the depth image, using the light emitter and the image sensor, wherein
the first luminance obtainer and the distance obtainer obtain the first luminance image and the depth image, respectively, from the signal processor.
11 . The object detecting device according to claim 1 , further comprising:
an information presenter that presents, based on the 3 D object information, information that assists a mobile object that includes the object detecting device in traveling.
12 . An object detecting method comprising:
obtaining a first luminance image corresponding to an image-capturing area, and a depth image corresponding to the image-capturing area of the first luminance image;
generating one or more groups, each of which is a set of pixels that is within a certain range and is taken as a same object in the depth image;
performing a first clustering process of determining that any of the one or more groups is a first cluster included in an object, when a total number of pixels therein is at least a first threshold;
performing a second clustering process of determining that, when it is determined, by the first clustering process that the total number of pixels included in a group among the one or more groups is less than the first threshold, determining whether luminance of a pixel group that is in the first luminance image and corresponds to the group is at least a second threshold, and when it is determined that the luminance of the pixel group is at least the second threshold, detecting the group as a second cluster; and
detecting an object in the depth image based on at least one of the first cluster or the second cluster to generate 3D object information indicating the object detected.
13 . The object detecting method according to claim 12 , wherein
the luminance of the pixel group in the first luminance image is an average luminance of pixels included in the pixel group.
14 . The object detecting method according to claim 12 , wherein
the luminance of the pixel group in the first luminance image is a maximum luminance of pixels included in the pixel group.
15 . The object detecting method according to claim 12 , wherein
the luminance of the pixel group in the first luminance image includes both of an average luminance and a maximum luminance of the pixel group,
the second threshold includes a threshold for the average luminance and a threshold for the maximum luminance, and
the performing of the second clustering process includes determining that the group is the second cluster, when the average luminance is at least the threshold for the average luminance and the maximum luminance is at least the threshold for the maximum luminance.
16 . The object detecting method according to claim 12 , further comprising:
detecting an object included in a 2D image corresponding to the image-capturing area;
generating 2D object information indicating the object detected; and
merging the 3D object information and the 2D object information together.
17 . The object detecting method according to claim 16 , further comprising:
determining that, when an object that is among objects included in the 3D object information and corresponds to the second cluster determined in the second clustering process overlaps the object indicated in the 2D object information, the object corresponding to the second cluster is a 3D object; and
determining that, when the object corresponding to the second cluster determined in the second clustering process does not overlap the object indicated in the 2D object information, the second cluster is not a cluster and that the object corresponding to the second cluster is not a 3D object.
18 . The object detecting method according to claim 16 , wherein
the 2D image is the first luminance image.
19 . The object detecting method according to claim 16 , further comprising:
obtaining a second luminance image for light having a wavelength different from a wavelength of light for which the first luminance image is obtained, wherein
the 2D image is either the second luminance image or a third luminance image obtained by combining the first luminance image and the second luminance image.
20 . A non-transitory computer-readable recording medium for use in a computer, the recording medium having recorded thereon a computer program for causing the computer to execute the object detecting method according to claim 12 .