Method for tracking position of object and system for tracking position of object

1 . A method for tracking a position of an object in a scene surrounding a mobile machine based upon information acquired from monocular images, the method comprising:

acquiring at least a first image at a first time and a second image at a second time, the first image and the second image each including image data corresponding to the object and a scene feature present in the scene surrounding the mobile machine;

detecting the object in the first image and the second image using a convolutional neural network trained to recognize the object by appearance;

detecting a scene feature in the first image and the second image by applying a Harris corner detector algorithm to identify aspects of the scene feature;

matching the scene feature across the first image and the second image using a local Lucas-Kanade optical flow algorithm to provide sub-pixel measurements between the first image and the second image;

excluding outliers from the matches scene features using a RANSAC algorithm;

performing an estimation of an egomotion of the mobile machine based upon the scene feature matched across the first image and the second image, wherein estimating the egomotion of the mobile machine based upon the scene feature matched across the first image and the second image includes applying one or more of a generalized camera model and linear approach to obtain a rotation of the mobile machine from the first time to the second time and a translation of the mobile machine from the first time to the second time; and

predicting a position of the object taking into account the estimation of the egomotion of the mobile machine, wherein the prediction is performed by a modified Extended Kalman Filter algorithm which incorporates values for the rotation and translation of the mobile machine from the first time to the second time;

wherein the method further comprises correcting the first image and the second image by converting first viewpoint parameters of a first imaging device of the first image and the second image into second viewpoint parameters of a second imaging device based on conversion information associated with a virtualization record stored by an electronic control system, the conversion information including at least one of distortion compensation information, image rectification information, image refraction information, and rotational information.

2 . The method according to claim 1 , further comprising transmitting the first image and the second image to an electronic control system to correct the first image and the second image by converting first viewpoint parameters of the first imaging device of the first image and the second image into second viewpoint parameters of the second imaging device.

3 . The method according to claim 1 , further comprising estimating three-dimensional (3D) parameters of the object in the first image and the second image.

4 . The method according to claim 3 , wherein the 3D parameters of the object in the first image and the second image includes the 3D parameters corresponding to a displacement between a pixel of a group of pixels belonging to the object and one or more reference point belonging to the object, the one or more reference point being at least one of a plurality of corners of a 3D bounding box or centroids of top and bottom faces of the 3D bounding box.

5 . A system for tracking a position of an object in a scene surrounding a mobile machine, the system comprising:

one or more imaging devices configured to acquire at least a first image at a first time and a second image at a second time the first image and the second image each including image data corresponding to the object and a scene feature present in the scene surrounding the mobile machine; and

an electronic control system configured to process the first image and the second image, the electronic control system including one or more processors configured to

detect the object in the first image and the second image using a convolutional neural network trained to recognize the object by appearance,

detect a scene feature in the first image and the second image by applying a Harris corner detector algorithm to identify aspects of the scene feature,

match the scene feature across the first image and the second image using a local Lucas-Kanade optical flow algorithm to provide sub-pixel measurements between the first image and the second image,

excluding outliers from the matches scene features using a RANSAC algorithm,

perform an estimation of an egomotion of the mobile machine based upon the scene feature matched across the first image and the second image, wherein estimation of the egomotion of the mobile machine based upon the scene feature matched across the first image and the second image includes applying one or more of a generalized camera model and linear approach to obtain a rotation of the mobile machine from the first time to the second time and a translation of the mobile machine from the first time to the second time,

predict a position of the object taking into account the estimation of the egomotion of the mobile machine, wherein the prediction is performed by a modified Extended Kalman Filter algorithm which incorporates values for the rotation and translation of the mobile machine from the first time to the second time; and

correct the first image and the second image by converting first viewpoint parameters of a first imaging device of the first image and the second image into second viewpoint parameters of a second imaging device based on conversion information associated with a virtualization record stored by an electronic control system, the conversion information including at least one of distortion compensation information, image rectification information, image refraction information, and rotational information.

6 . The system according claim 5 , wherein the imaging device is a monocular camera.