Systems and methods for monocular based object detection
Disclosed herein are systems, methods, and computer program products for object detection. The methods comprise performing the following operations by a computing device: obtaining an image that comprises a plurality of layers superimposed on each other; identifying a center point of a robot on a map; selecting a portion of the map contained in a geometric shape overlaid on the map so as to have a center set to the center point of the robot; obtaining map information associated with the selected portion of the map; generating at least one additional layer using the map information; superimposing the at least one additional layer onto the image to generate a modified image; and performing an object detection algorithm to detect at least one object in the modified image.
1 . A method for object detection, comprising:
obtaining, by a computing device, an image that comprises a plurality of layers superimposed on each other;
identifying, by the computing device, a center point of a robot on a map;
selecting, by the computing device, a portion of the map contained in a geometric shape overlaid on the map so as to have a center set to the center point of the robot;
obtaining, by the computing device, map information associated with the selected portion of the map;
generating, by the computing device, at least one additional layer using the map information;
superimposing, by the computing device, the at least one additional layer onto the image to generate a modified image; and
performing, by the computing device, an object detection algorithm to detect at least one object in the modified image.
2 . The method according to claim 1 , further comprising causing, by the computing device, control operations of the robot based on an output of the object detection algorithm.
3 . The method according to claim 1 , wherein the identifying the center point of the robot is based on pose information of the robot that comprises a location defined in 3D map coordinates, an angle of the robot relative to a reference point, and a pointing direction of the robot.
4 . The method according to claim 1 , wherein the selected portion of the map represents a geographic area encompassing the robot.
5 . The method according to claim 1 , further comprising performing machine learning operations to select a first combination of different types of map information when a first scenario exists for the robot and a second combination of different types of map information when a second scenario exists for the robot.
6 . The method according to claim 1 , wherein the map information comprises at least one of ground information, drivable geographical area information, ground depth information, map point distance-to-lane center information, lane direction information, and intersection information.
7 . The method according to claim 1 , wherein the at least one additional layer comprises a ground height layer, a ground depth layer, a drivable geographical area layer, a map point distance-to-lane center layer, a lane direction layer, or an intersection layer.
8 . The method according to claim 1 , wherein the generating the at least one additional layer comprises:
plotting the map information on a graph;
defining a 2D grid on the graph, the 2D grid comprising a plurality of cells with each said cell having four corners respectively associated with geometric points of the map;
using ground height values associated with the geometric points to define polygons in 3D space; and
projecting the polygons into a camera frame.
9 . The method according to claim 8 , further comprising setting a color value of each said projected polygon to (i) a color value associated with a select one of the geometric points used to define a corresponding one of the polygons or (ii) an average color value for the geometric points used to define a corresponding one of the polygons.
10 . The method according to claim 1 , wherein at least two different additional layers are generated using the map information and superimposed onto the image to generate the modified image.
11 . A system, comprising:
a processor;
a non-transitory computer-readable storage medium comprising programming instructions that are configured to cause the processor to implement a method for object detection, wherein the programming instructions comprise instructions to:
obtain an image that comprises a plurality of layers superimposed on each other;
identify a center point of a robot on a map;
select a portion of the map contained in a geometric shape overlaid on the map so as to have a center set to the center point of the robot;
obtain map information associated with the selected portion of the map;
generate at least one additional layer using the map information;
superimpose the at least one additional layer onto the image to generate a modified image; and
perform an object detection algorithm to detect at least one object in the modified image.
12 . The system according to claim 11 , wherein the programming instructions comprise instructions to control operations of the robot based on an output of the object detection algorithm.
13 . The system according to claim 11 , wherein the center point of the robot is identified based on pose information of the robot that comprises a location defined in 3D map coordinates, an angle of the robot relative to a reference point, and a pointing direction of the robot.
14 . The system according to claim 11 , wherein the selected portion of the map represents a geographic area encompassing the robot.
15 . The system according to claim 11 , wherein the programming instructions comprise instructions to perform machine learning operations to select a first combination of different types of map information when a first scenario exists for the robot and a second combination of different types of map information when a second scenario exists for the robot.
16 . The system according to claim 11 , wherein the map information comprises at least one of ground information, drivable geographical area information, ground depth information, map point distance-to-lane center information, lane direction information, and intersection information.
17 . The system according to claim 11 , wherein the at least one additional layer comprises a ground height layer, a ground depth layer, a drivable geographical area layer, a map point distance-to-lane center layer, a lane direction layer, or an intersection layer.
18 . The system according to claim 11 , wherein the at least one additional layer is generated by:
plotting the map information on a graph;
defining a 2D grid on the graph, the 2D grid comprising a plurality of cells with each said cell having four corners respectively associated with geometric points of the map;
using ground height values associated with the geometric points to define polygons in 3D space; and
projecting the polygons into a camera frame.
19 . The system according to claim 18 , wherein the programming instructions comprise instructions to set a color value of each said projected polygon to (i) a color value associated with a select one of the geometric points used to define a corresponding one of the polygons or (ii) an average color value for the geometric points used to define a corresponding one of the polygons.
20 . A non-transitory computer-readable medium that stores instructions that are configured to, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
obtaining an image that comprises a plurality of layers superimposed on each other;
identifying a center point of a robot on a map;
selecting a portion of the map contained in a geometric shape overlaid on the map so as to have a center set to the center point of the robot;
obtaining map information associated with the selected portion of the map;
generating at least one additional layer using the map information, wherein the additional layer includes a drivable geographical area layer;
superimposing the at least one additional layer onto the image to generate a modified image such that a least a portion of a non-drivable geographical area is blocked in the modified image; and
performing an object detection algorithm to detect at least one object in the modified image.