Apparatus for controlling vehicle including camera and method for the same
An apparatus of controlling a vehicle and a method for controlling the same includes a camera, a processor operatively connected to the camera, and a storage operatively connected to the processor to store instructions executed by the processor. The processor obtains an image of a surrounding of a door of the vehicle, which is captured through the camera, obtains depth information from the obtained image, transforms the depth information into three dimensional (3D) point information, determines collision possibility of the door with an obstacle, based on distribution of the 3D point information to determine collision possibility, and warns the collision of the door when the collision possibility is present, when executing the instructions.
1 . An apparatus of controlling a vehicle, the apparatus comprising:
a camera;
a processor operatively connected to the camera; and
a storage operatively connected to the processor and configured to store instructions executed by the processor,
wherein, when executing the instructions, the processor is configured to:
obtain an image of a surrounding of a door of the vehicle, which is captured through the camera;
obtain depth information from the obtained image;
transform the depth information into three dimensional (3D) point information;
determine collision possibility of the door by determining the collision possibility of the door with an obstacle, based on distribution of the 3D point information; and
warn collision of the door, when the collision possibility is determined as being present, and
wherein, when executing the instructions, the processor is configured to:
set a collision sensing zone, having a predetermined size, wherein an endpoint of the collision sensing zone is spaced from a 3D point corresponding to a surface of the door by an offset;
determine a number of 3D points corresponding to the obstacle entering the collision sensing zone;
determine a variation in the number of 3D points corresponding to the obstacle entering the collision sensing zone during a period of time when a location of the collision sensing zone changes due to opening of the door; and
warn collision of the door, when the variation is equal to or greater than a predetermined number.
2 . The apparatus of claim 1 , wherein, when executing the instructions, the processor is configured to:
obtain the 3D point information by use of the depth information and a parameter of the camera, and
wherein the parameter includes a focal length and coordinates of a principal point.
3 . The apparatus of claim 2 , wherein, when executing the instructions, the processor is configured to:
obtain the 3D point information through following Equation 1, Equation 2, and Equation 3,
x =(( px−cx )/ fx ) pz, Equation 1,
y =(( py−cy )/ fy ) pz , and Equation 2,
z=pz , and Equation 3,
wherein ‘x’, ‘y’, and ‘z’ represent coordinates of a 3D point, ‘px’ and ‘py’ represent coordinates of each pixel contained in the depth information, ‘cx’ and ‘cy’ represent coordinates of the principal point, ‘fx’ and ‘fy’ represent the focal length, and ‘pz’ represents a depth value predicted with respect to the pixel.
4 . The apparatus of claim 1 , wherein, when executing the instructions, the processor is configured to:
produce a lattice having the predetermined size, based on the 3D point corresponding to the surface of the door; and
set, as the collision sensing zone, a space defined from a point, which is spaced from the 3D point corresponding to the surface of the door by the offset, to a point spaced from the 3D point corresponding to the surface of the door by a predetermined sensing range, in a space corresponding to the lattice.
5 . The apparatus of claim 4 , wherein, when executing the instructions, the processor is configured to:
produce a plurality of lattices having the predetermined size based on a plurality of 3D points corresponding to the surface of the door; and
warn the collision of the door, when 3D points corresponding to the obstacle are present in at least the predetermined number in the collision sensing zone of at least one lattice of the plurality of lattices.
6 . The apparatus of claim 1 , wherein, when executing the instructions, the processor is configured to:
determine the collision possibility of the door as being present, when the variation is equal to or greater than a predetermined value.
7 . The apparatus of claim 1 ,
wherein the vehicle control apparatus further includes a speaker, and
wherein, when executing the instructions, the processor is configured to:
output a warning sound through the speaker, when determining the collision possibility as being present.
8 . The apparatus of claim 1 , wherein the camera includes:
at least one of a mono-camera or a stereo-camera.
9 . A method for controlling a vehicle, the method comprising:
obtaining, by a processor, an image of a surrounding of a door of the vehicle, which is captured through a camera;
obtaining, by the processor, depth information from the obtained image;
transforming, by the processor, the depth information into 3D point information;
determining, by the processor, collision possibility of the door by determining the collision possibility of the door with an obstacle, based on distribution of the 3D point information; and
warning, by the processor, collision of the door, when the collision possibility is determined as being present,
wherein the determining of the collision possibility includes:
setting a collision sensing zone, having a predetermined size, wherein an endpoint of the collision sensing zone is spaced from a 3D point corresponding to a surface of the door by an offset;
determining a number of 3D points corresponding to the obstacle entering the collision sensing zone,
determine a variation in the number of 3D points corresponding to the obstacle entering the collision sensing zone during a period of time when a location of the collision sensing zone changes due to opening of the door, and
determining the collision possibility of the door, based on the variation.
10 . The method of claim 9 , wherein the transforming of the depth information into the 3D point information includes:
obtaining the 3D point information by use of the depth information and a parameter of the camera, and
wherein the parameter includes a focal length and coordinates of a principal point.
11 . The method of claim 10 , wherein the obtaining of the 3D point information is performed through following Equation 1, Equation 2, and Equation 3,
x =(( px−cx )/ fx )· pz, Equation 1,
y =(( py−cy )/ fy )· pz , and Equation 2,
z=pz, Equation 3,
wherein ‘x’, ‘y’, and ‘z’ represent coordinates of a 3D point, ‘px’ and ‘py’ represent coordinates of each pixel contained in the depth information, ‘cx’ and ‘cy’ represent coordinates of a principal point, ‘fx’ and ‘fy’ represent focal lengths, and ‘pz’ represents a depth value predicted with respect to the pixel.
12 . The method of claim 9 , wherein the setting of the collision sensing zone includes:
producing a lattice having the predetermined size based on the 3D point corresponding to the surface of the door; and
setting, as the collision sensing zone, a space defined from a point, which is spaced from the 3D point corresponding to the surface of the door by the offset, to a point spaced from the 3D predetermined point corresponding to the surface of the door by a predetermined sensing range, in a space corresponding to the lattice.
13 . The method of claim 12 , wherein the producing of the lattice includes:
producing a plurality of lattices having the predetermined size based on a plurality of 3D points corresponding to the surface of the door, and
wherein the warning of the collision of the door includes:
determining whether 3D points corresponding to the obstacle are present in at least the predetermined number in the collision sensing zone of at least one lattice of the plurality of lattices.
14 . The method of claim 9 , wherein the determining of the collision possibility includes:
determining the collision possibility of the door as being present, when the variation is equal to or greater than a predetermined value.
15 . The method of claim 9 , wherein the warning of the collision of the door includes:
outputting a warning sound through a speaker.
16 . The method of claim 9 , wherein the camera includes:
at least one of a mono-camera or a stereo-camera.