IP Library › Granted Patent US 11,468,687
Granted Patent B2
US 11,468,687 · App. 16/762,757 · Granted Oct 11, 2022

Training and operating a machine learning system

Inventors: Masato Takami (Hildesheim, DE); Uwe Brosch (Hohenhameln, DE)
Assignee: Robert Bosch GmbH
G06V20/58B60W60/0011G06K9/6256G06N3/04G06N20/00G06T7/50G06T2207/10028G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,468,687
App. No.
16/762,757
Filed
May 8, 2020
Granted
Oct 11, 2022
Kind
B2
Art Unit
2631
USPC
382/104
Abstract

A method for training a machine learning system, in which image data are fed into a machine learning system with processing of at least a part of the image data by the machine learning system. The method includes synthetic generation of at least a part of at least one depth map that includes a plurality of depth information values. The at least one depth map is fed into the machine learning system with processing of at least a part of the depth information values of the at least one depth map. The machine learning system is then trained based on the processed image data and based on the processed depth information values of the at least one depth map, with adaptation of a parameter value of at least one parameter of the machine learning system, the adapted parameter value influencing an interpretation of input data by the machine learning system.

Claims (34)

1. A method for training a machine learning system, the method comprising the following steps:

feeding image data into a machine learning system and processing at least a part of the image data by the machine learning system;

synthetically generating at least a part of at least one depth map that has a plurality of depth information values, each of the depth information values correlating with a distance to an object;

feeding the at least one depth map into the machine learning system and processing of at least a part of the depth information values of the at least one depth map by the machine learning system; and

training the machine learning system based on the processed image data and based on the processed depth information values of the at least one depth map, with adaptation of a parameter value of at least one parameter of the machine learning system, wherein the adapted parameter value influences an interpretation of input data by the machine learning system.

2. The method as recited in claim 1 , further comprising the following step:

(i) assigning the image data to the at least one depth map; and/or

(ii) adapting the parameter value of the machine learning system as a function of the processed image data and as a function of the processed depth information values.

3. The method as recited in claim 1 , wherein the depth map includes a matrix and/or a list having entries, each of the entries of the matrix and/or list representing a pixel of a device for acquiring depth information, and a value of each of the entries being a depth information value for indicating a distance between the device and an object.

4. The method as recited in claim 1 , wherein the at least one depth map represents data of: a stereo camera, and/or a multiview camera, and/or a distance measuring device, and/or a radar-based distance measuring device, and/or an ultrasound-based distance measuring device, and/or a laser-based distance measuring device.

5. The method as recited in claim 1 , the synthetic generation of the at least one part of the at least one depth map includes defining a plurality of depth information values of the depth map, and storing the plurality of defined depth information values in the depth map.

6. The method as recited in claim 5 , wherein the defined depth information values representing values of at least one subset of entries of the depth map, the subset representing a contiguous pixel region of pixels of a device for acquiring depth information, so that through the definition of the depth information values an item of distance information is produced relating to a geometrically contiguous object in the depth map.

7. The method as recited in claim 6 , wherein the geometrically contiguous object is contained exclusively in the at least one depth map, so that through the synthetic generation of the at least one part of the at least one depth map, a discrepancy is produced between the image data and the at least one depth map.

8. The method as recited in claim 5 , wherein the plurality of depth information values is defined and/or selected such that a distance between the device and the object is in a range between 5 cm and 500 m.

9. The method as recited in claim 8 , wherein the range is between 5 cm and 200 m.

10. The method as recited in claim 1 , wherein the parameter value of the at least one parameter of the machine learning system is adapted such that, given a discrepancy between the image data and the at least one depth map, an interpretation of the depth map by the machine learning system is preferred over an interpretation of the image data.

11. The method as recited in claim 1 , further comprising the following steps:

synthetically generating, in each case, at least a part of a plurality of depth maps, and defining a plurality of depth information values of each of the depth map; and

training machine learning system including processing of the plurality of synthetically generated depth maps by the machine learning system, the defined depth information values of each of the depth maps representing in each case a contiguous pixel region of pixels of a device for acquiring depth information, so that through the definition of the depth information values of each depth map, in each case, an item of distance information is produced relating to a geometrically contiguous object in the respective depth map.

12. The method as recited in claim 11 , wherein the objects produced in the synthetically generated depth maps differ from one another with respect to a contour, and/or a dimension, and/or a position, and/or a distance.

13. The method as recited in claim 1 , wherein: (i) the machine learning system is a multilayer artificial neural network, and/or (ii) the at least one parameter of the machine learning system is a weight of a node of an artificial neural network.

14. A method for operating a machine learning system for a motor vehicle, the method comprising the following steps:

providing a trained machine learning system, the machine learning system being trained by:

feeding image data into the machine learning system and processing at least a part of the image data by the machine learning system;

synthetically generating at least a part of at least one depth map that has a plurality of depth information values, each of the depth information values correlating with a distance to an object;

feeding the at least one depth map into the machine learning system and processing of at least a part of the depth information values of the at least one depth map by the machine learning system; and

training the machine learning system based on the processed image data and based on the processed depth information values of the at least one depth map, with adaptation of a parameter value of at least one parameter of the machine learning system, wherein the adapted parameter value influences an interpretation of input data by the machine learning system;

using the trained machine learning system for object recognition in the motor vehicle; and

controlling the motor vehicle based on the object recognition by the trained machine learning system.

15. A machine learning system for recognizing objects for a motor vehicle, the machine learning system being trained by:

feeding image data into the machine learning system and processing at least a part of the image data by the machine learning system;

synthetically generating at least a part of at least one depth map that has a plurality of depth information values, each of the depth information values correlating with a distance to an object;

feeding the at least one depth map into the machine learning system and processing of at least a part of the depth information values of the at least one depth map by the machine learning system; and

training the machine learning system based on the processed image data and based on the processed depth information values of the at least one depth map, with adaptation of a parameter value of at least one parameter of the machine learning system, wherein the adapted parameter value influences an interpretation of input data by the machine learning system.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2021
From: TAKAMI, MASATO; BROSCH, UWE
To: ROBERT BOSCH GMBH
Reel/Frame 057244/0470 →
Priority Claims (1)
DE 10 2017 221 765.0 · Dec 4, 2017 · national
Continuity (1)
Related Publication 20210182577A1 · Jun 17, 2021
Cited By (4)
US 12,272,184 US 12,284,461 US 12,322,192 US 12,464,237