Method for generating input data for a machine learning model
A method of generating input data for a machine learning model includes determining, for a sensor, a point cloud with points detected by the sensor from surfaces in the environment of the sensor, generating a preliminary target sensor point cloud for a target sensor by transforming, for the sensor, points of the determined point cloud into points from the perspective of the target sensor according to the relative position of the target sensor to the sensor, generating a target sensor point cloud for the target sensor by using the preliminary target sensor point cloud, wherein points which, due to one or more surfaces for which points exist in the preliminary target sensor point cloud, are not detectable by the target sensor are eliminated in the target sensor point cloud, and using the target sensor point cloud as input for the machine learning model.
1 . A method for generating input data for a machine learning model, comprising:
capturing data, using at least one sensor, in an environment of the at least one sensor and from a perspective of the at least one sensor;
determining, from the captured data, a point cloud including a first plurality of points corresponding to surfaces detected by the at least one sensor in the environment of the at least one sensor, the first plurality of points determined from the perspective of the at least one sensor;
generating a preliminary target sensor point cloud of a target sensor by transforming the first plurality of points of the determined point cloud into a second plurality of points from a perspective of the target sensor based on a relative position of the target sensor to the at least one sensor, the perspective of the target sensor being different from the perspective of the at least one sensor;
generating a target sensor point cloud as would be output by the target sensor by eliminating points of the second plurality of points from the preliminary target sensor point cloud, the eliminated points corresponding to surfaces, within a target range of the target sensor, but that are not detectable by the target sensor from the perspective of the target sensor; and
using the target sensor point cloud as an input for training the machine learning model when no data has been captured from the perspective of the target sensor,
wherein the eliminated points correspond to surfaces that are hidden from view of the target sensor from the perspective of the target sensor, but that are visible to the at least one sensor from the perspective of the at least one sensor,
wherein the at least one sensor is installed in the environment,
wherein the target sensor is not yet installed in the environment, and
wherein the target sensor point cloud corresponds to data that would be generated by the target sensor when the target sensor is installed in the environment at a position resulting in the perspective of the target sensor.
2 . The method according to claim 1 , wherein generating the target sensor point cloud comprises:
supplementing the preliminary target sensor point cloud with additional points which are included in the preliminary target sensor point cloud, and which correspond to a particular surface that is detectable by the target sensor from the perspective of the target sensor when the particular surface is present in the environment.
3 . The method according to claim 1 , wherein generating the target sensor point cloud comprises:
generating a depth image representing locations of points of the preliminary target sensor point cloud from the perspective of the target sensor;
morphologically opening the depth image; and
generating the target sensor point cloud according to the morphologically opened depth image.
4 . The method according to claim 1 , wherein generating the target sensor point cloud comprises:
generating a disparity image representing locations of points of the preliminary target sensor point cloud from the perspective of the target sensor;
morphologically closing the disparity image; and
generating the target sensor point cloud according to the morphologically closed disparity image.
5 . The method according to claim 1 , wherein:
the determination of the point cloud from the captured data includes using the first plurality of points detected by the at least one sensor from surfaces in a vicinity of the at least one sensor; and
generating the preliminary target sensor point cloud includes transforming the first plurality of points into the second plurality of points from the perspective of the target sensor according to the relative position of the target sensor to the at least one sensor, and assembling the second plurality of points into the preliminary target sensor point cloud.
6 . The method according to claim 1 , wherein:
generating the target sensor point cloud includes generating a respective target sensor point cloud for each target sensor included in a target sensor arrangement including a plurality of target sensors; and
using the target sensor point cloud includes using each respective target sensor point cloud as an input for the machine learning model.
7 . The method according to claim 1 , further comprising:
training the machine learning model to process sensor data from the perspective of the target sensor using the target sensor point cloud.
8 . The method according to claim 7 , further comprising:
collecting ground truth information for the first plurality of points of the point cloud determined for the at least one sensor;
transforming the ground truth information into ground truth information for the second plurality of points of the target sensor point cloud; and
training the machine learning model using supervised learning using the target sensor point cloud as training input data and the transformed ground truth information.
9 . The method according to claim 7 , wherein:
generating the preliminary target sensor point cloud includes generating a respective preliminary target sensor point cloud for each target sensor included in a target sensor arrangement including a plurality of target sensors; and
using the target sensor point cloud includes using each respective preliminary target sensor point cloud as an input for the machine learning model.
10 . The method according to claim 1 , wherein:
the at least one sensor includes a first sensor and a second sensor, and
the target sensor is located between the first sensor and the second sensor.
11 . The method according to claim 1 , wherein transforming of the first plurality of points into the second plurality of points is based on a difference between a known position of the at least one sensor and an assumed installation position of the target sensor.
12 . The method according to claim 1 , wherein transforming the first plurality of points includes immediately discarding any points of the first plurality of points that are outside of a target range of the target sensor.
13 . A sensor data processing system configured to carry out the method according to claim 1 .
14 . A non-transitory computer-readable medium which stores instructions that, when executed by a processor, cause the processor to perform the method according to claim 1 .