Method of training a machine learning system for an object recognition device
A method of training a machine learning system for an object recognition device. The method includes: providing sensing element data; and training a machine learning system, using the provided sensing element data; at least one object being recognized from the sensing element data; and signal intensities of the sensing element data being used together with a reflection and/or absorption factor associated with the object.
1 . A method of training a machine learning system including a neural network for an object recognition device, the method comprising the following steps:
emitting radiation into an environment using a sensor device having a sensor device type including at least one of: a lidar sensor device, a radar sensor device, or an ultrasonic sensor device;
generating sensing element data based on radiation reflected off of a plurality of objects in the environment and received by the sensor device in response to emitting the radiation;
storing, prior to training the machine learning system, a feature map including locations of the plurality of objects in the environment, and for each object at least one of: a reflection factor describing a reflection characteristic of a surface of the object, or an absorption factor describing an absorption characteristic of the surface of the object;
wherein the plurality of objects includes at least a first object having a metallic surface and a second object having a non-metallic surface, the stored reflection factor and/or absorption factor for the first object having the metallic surface being different from the stored reflection factor and/or absorption factor having the non-metallic surface based on different reflection and/or absorption characteristics for the metallic and non-metallic surfaces;
wherein for each object the stored reflection factor and/or absorption factor is specific to the sensor device type of the sensor device; and
training the machine learning system including the neural network to recognize the plurality of objects in the environment by providing inputs to the machine learning system including, for each object, (i) the sensing element data based on the radiation reflected from that object, (ii) the stored location of that object, and (iii) the stored reflection factor and/or absorption factor for that object.
2 . The method as recited in claim 1 , wherein the machine learning system is trained in a defined environment, wherein the training includes irradiating a known object by the emitting sensor device, and radiation reflected by the object is received, and the object is recognized based on the radiation reflected by the object.
3 . The method as recited in claim 1 , wherein to provide the sensing element data, the emitting sensor device remains static or is moved in relation to the at least one object.
4 . The method as recited in claim 1 , wherein the feature map is used for ascertaining the at least one object from the sensing element data, and the reflection factor and/or an absorption factor of the at least one object is stored in the feature map.
5 . The method as recited in claim 1 , wherein the machine learning system is trained with using sensing element data provided in different weather conditions.
6 . The method as recited in claim 1 , wherein interference in the sensing element data is filtered.
7 . The method as recited in claim 1 , wherein a selection of the data of the recognized objects is used for training a downstream machine learning system.
8 . A method for carrying out object recognition, the method comprising:
storing, prior to training a machine learning system including a neural network, a feature map including locations of a plurality of objects in an environment, and for each object at least one of: a reflection factor describing a reflection characteristic of a surface of the object, or an absorption factor describing an absorption characteristic of the surface of the object;
wherein the plurality of objects includes at least a first object having a metallic surface and a second object having a non-metallic surface, the stored reflection factor and/or absorption factor for the first object having the metallic surface being different from the stored reflection factor and/or absorption factor having the non-metallic surface based on different reflection and/or absorption characteristics for the metallic and non-metallic surfaces;
wherein for each object the stored reflection factor and/or absorption factor is specific to a sensor device type of s sensor device; and
performing the object recognition using the machine learning system including the neural network, the machine learning system trained by:
emitting radiation into an environment using the sensor device having the sensor device type including at least one of: a lidar sensor device, a radar sensor device, or an ultrasonic sensor device;
generating sensing element data based on radiation reflected off of the plurality of objects in the environment and received by the sensor device in response to emitting the radiation; and
training the machine learning system including the neural network to recognize the plurality of objects in the environment by providing inputs to the machine learning system including, for each object, (i) the sensing element data based on the radiation reflected from that object, (ii) the stored location of that object, and (iii) the stored reflection factor and/or absorption factor for that object.
9 . The method as recited in claim 8 , wherein the trained machine learning system is used to compare relevant image regions of the sensing element data to object types, object recognition based on the sensing element data is carried out in a downstream machine learning system, and reference is made to the relevant image regions.
10 . An object recognition device, comprising:
a non-transitory machine readable storage medium to store a feature map including locations of a plurality of objects in an environment, and for each object at least one of: a reflection factor describing a reflection characteristic of a surface of the object, or an absorption factor describing an absorption characteristic of the surface of the object;
wherein the plurality of objects includes at least a first object having a metallic surface and a second object having a non-metallic surface, the stored reflection factor and/or absorption factor for the first object having the metallic surface being different from the stored reflection factor and/or absorption factor having the non-metallic surface based on different reflection and/or absorption characteristics for the metallic and non-metallic surfaces;
wherein for each object the stored reflection factor and/or absorption factor is specific to a sensor device type of a sensor device; and
a machine learning system including a neural network to perform object recognition using the stored feature map including reflection factors and/or absorption factors, the machine learning system being trained to recognize the objects in the environment based on signal intensities of sensing element data and the reflection factors and/or absorption factors of the stored feature map by:
emitting radiation into an environment using the sensor device having the sensor device type including at least one of: a lidar sensor device, a radar sensor device, or an ultrasonic sensor device;
generating sensing element data based on radiation reflected off of the plurality of objects in the environment and received by the sensor device in response to emitting the radiation; and
training the machine learning system including the neural network to recognize the plurality of objects in the environment by providing inputs to the machine learning system including, for each object, (i) the sensing element data based on the radiation reflected from that object, (ii) the stored location of that object, and (iii) the stored reflection factor and/or absorption factor for that object.
11 . The object recognition device as recited in claim 10 , wherein the object recognition device is situated on an emitting sensor device or outside of the emitting sensor device.
12 . An object recognition device, comprising:
an emitting sensor device;
a stored feature map including locations of a plurality of objects in an environment, and for each object at least one of: a reflection factor describing a reflection characteristic of a surface of the object, or an absorption factor describing an absorption characteristic of the surface of the object;
wherein the plurality of objects includes at least a first object having a metallic surface and a second object having a non-metallic surface, the stored reflection factor and/or absorption factor for the first object having the metallic surface being different from the stored reflection factor and/or absorption factor having the non-metallic surface based on different reflection and/or absorption characteristics for the metallic and non-metallic surfaces;
wherein for each object the stored reflection factor and/or absorption factor is specific to a sensor device type of a sensor device; and
a machine learning system including a neural network trained to perform object recognition using the stored feature map including reflection factors and/or absorption factors, the machine learning system being trained to recognize the objects based on signal intensities of training sensing element data and the reflection factors and/or absorption factors for the objects by:
emitting radiation into an environment using the sensor device having the sensor device type including at least one of: a lidar sensor device, a radar sensor device, or an ultrasonic sensor device;
generating sensing element data based on radiation reflected off of the plurality of objects in the environment and received by the sensor device in response to emitting the radiation; and
training the machine learning system including the neural network to recognize the plurality of objects in the environment by providing inputs to the machine learning system including, for each object, (i) the sensing element data based on the radiation reflected from that object, (ii) the stored location of that object, and (iii) the stored reflection factor and/or absorption factor for that object.
13 . A non-transitory machine-readable storage medium on which is stored a computer program for training a machine learning system including a neural network for an object recognition device, the computer program, when executed by a computer, causing the computer to perform the following steps:
providing sensing element data for objects in an environment;
storing, prior to training the machine learning system, a feature map including locations of a plurality of objects in an environment, and for each object at least one of: a reflection factor describing a reflection characteristic of a surface of the object, or an absorption factor describing an absorption characteristic of the surface of the object;
wherein the plurality of objects includes at least a first object having a metallic surface and a second object having a non-metallic surface, the stored reflection factor and/or absorption factor for the first object having the metallic surface being different from the stored reflection factor and/or absorption factor having the non-metallic surface based on different reflection and/or absorption characteristics for the metallic and non-metallic surfaces;
wherein for each object the stored reflection factor and/or absorption factor is specific to a sensor device type of a sensor device; and
training the machine learning system to recognize the objects in the environment based on signal intensities of the provided sensing element data and the reflection factors and/or absorption factors for the objects from the stored feature map by:
emitting radiation into an environment using the sensor device having the sensor device type including at least one of: a lidar sensor device, a radar sensor device, or an ultrasonic sensor device;
generating sensing element data based on radiation reflected off of the plurality of objects in the environment and received by the sensor device in response to emitting the radiation; and
training the machine learning system including the neural network to recognize the plurality of objects in the environment by providing inputs to the machine learning system including, for each object, (i) the sensing element data based on the radiation reflected from that object, (ii) the stored location of that object, and (iii) the stored reflection factor and/or absorption factor for that object.
14 . The method as recited in claim 1 , further comprising:
training the machine learning system to filter out scattering or interference in the sensing element data based on comparing, for each object, an expected signal intensity according to the reflection factor and/or absorption factor of the surface of the object to the signal intensity of the sensing element data for the object.