IP Library Granted Patent US 11,323,618
Granted Patent B2
US 11,323,618 · App. 17/260,047 · Granted May 3, 2022

Method for displaying a model of a surrounding area, control unit and vehicle

Inventors: Dirk Raproeger (Wunstorf, DE); Lidia Rosario Torres Lopez (Hildesheim, DE); Paul Robert Herzog (Hildesheim, DE); Paul-Sebastian Lauer (Hannover, DE); Uwe Brosch (Hohenhameln, DE)
Assignee: Robert Bosch GmbH
H04N5/23238B60R1/00G06T7/55G06T7/90H04N5/22525H04N7/181H04N9/67B60R2300/105B60R2300/303G06T2207/10024G06T2207/10028G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,323,618
App. No.
17/260,047
Granted
May 3, 2022
Kind
B2
Abstract

A method, including recording a first and a second camera image; the first camera image and the second camera image having an overlap region. The method includes: assigning pixels of the first camera image and pixels of the second camera image to predefined points of a three-dimensional lattice structure, the predefined points being situated in a region of the three-dimensional lattice structure, which represents the overlap region; ascertaining a color information item difference for each predefined point as a function of the assigned color information items; ascertaining a quality value as a function of the ascertained color information item difference at the specific, predefined point; determining a global color transformation matrix as a function of the color information item differences, weighted as a function of the corresponding quality value; and adapting the second camera image as a function of the determined color transformation matrix.

Claims (40)

1. A method for displaying a model of a surrounding area of a reference object, comprising the following steps:

recording a first camera image using a first camera;

recording a second camera image using a second camera, the first camera image and the second camera image having an overlap region, the first camera image and the second camera image each including a plurality of pixels having, in each instance, a color information item;

assigning pixels of the first camera image and pixels of the second camera image to at least predefined points of a three-dimensional lattice structure in accordance with, in each instance, a specified assignment rule, the predefined points being situated in a region of the three-dimensional lattice structure, which represents the overlap region;

ascertaining a corresponding color information item difference for each of the predefined points of the three-dimensional lattice structure as a function of the color information items of the pixels, of the first camera image and the second camera image, assigned to the predefined point;

ascertaining a corresponding quality value for each of the predefined points as a function of the ascertained color information item difference at the predefined point;

determining a global color transformation matrix as a function of the ascertained plurality of color information item differences, the ascertained color information item differences each being weighted as a function of the corresponding quality value of the predefined point;

adapting at least one subregion of the second camera image as a function of the determined global color transformation matrix; and

displaying a surrounding-area model of the reference object as a function of the lattice structure, of at least a subregion of the first camera image, and of at least the adapted subregion of the second camera image.

2. The method as recited in claim 1 , further comprising the following steps:

acquiring data regarding a distance between the reference object and objects in the surrounding area of the reference object;

adapting the three-dimensional lattice structure as a function of the acquired distance data; and

displaying the surrounding area model additionally as a function of the adapted lattice structure.

3. The method as recited in claim 2 , wherein: (i) each of the quality values is ascertained as a function of the acquired distance data and/or (ii) the determined global color transformation matrix is additionally determined as a function of the acquired distance data.

4. The method as recited in claim 3 , wherein the determination of the determined global color transformation matrix takes place as a function of color differences of predefined points of the three-dimensional lattice structure which lie in a range which is: (i) greater than or equal to a minimum distance from the reference object, and/or (ii) less than or equal to a specified lateral distance from the reference object.

5. The method as recited in claim 1 , further comprising the following steps:

detecting segments in the first camera image and/or in the second camera image;

wherein each of the quality values is ascertained additionally as a function of the detected segments.

6. The method as recited in claim 1 , wherein each of the color information item differences is ascertained and/or each of the quality values is ascertained additionally as a function of a plurality of color information items of the first camera image and/or the second camera image in a surrounding area of the corresponding predefined point.

7. The method as recited in claim 1 , wherein each of the color information item differences is ascertained and/or each of the quality values is ascertained additionally as a function of a plurality of color information items of the first camera image and/or the second camera image along at least a region of an epipolar line of the corresponding predefined point.

8. The method as recited in claim 1 , wherein the display of the model in a region, which represents the overlap region, includes the first camera image or the adapted second camera image or a combination of the first camera image and the second camera image.

9. A system comprising:

a control unit configured to display a model of a surrounding area of a reference object, the control unit configured to:

record a first camera image using a first camera; record a second camera image using a second camera, the first camera image and the second camera image having an overlap region, the first camera image and the second camera image each including a plurality of pixels having, in each instance, a color information item;

assign pixels of the first camera image and pixels of the second camera image to at least predefined points of a three-dimensional lattice structure in accordance with, in each instance, a specified assignment rule, the predefined points being situated in a region of the three-dimensional lattice structure, which represents the overlap region;

ascertain a corresponding color information item difference for each of the predefined points of the three-dimensional lattice structure as a function of the color information items of the pixels, of the first camera image and the second camera image, assigned to the predefined point;

ascertain a corresponding quality value for each of the predefined points as a function of the ascertained color information item difference at the predefined point;

determine a global color transformation matrix as a function of the ascertained plurality of color information item differences, the ascertained color information item differences each being weighted as a function of the corresponding quality value of the predefined point;

adapt at least one subregion of the second camera image as a function of the determined global color transformation matrix; and

display a surrounding-area model of the reference object as a function of the lattice structure, of at least a subregion of the first camera image, and of at least the adapted subregion of the second camera image.

10. A vehicle, comprising:

a control unit configured to display a model of a surrounding area of a reference object, the control unit configured to:

record a first camera image using a first camera;

record a second camera image using a second camera, the first camera image and the second camera image having an overlap region, the first camera image and the second camera image each including a plurality of pixels having, in each instance, a color information item;

assign pixels of the first camera image and pixels of the second camera image to at least predefined points of a three-dimensional lattice structure in accordance with, in each instance, a specified assignment rule, the predefined points being situated in a region of the three-dimensional lattice structure, which represents the overlap region;

ascertain a corresponding color information item difference for each of the predefined points of the three-dimensional lattice structure as a function of the color information items of the pixels, of the first camera image and the second camera image, assigned to the predefined point;

ascertain a corresponding quality value for each of the predefined points as a function of the ascertained color information item difference at the predefined point;

determine a global color transformation matrix as a function of the ascertained plurality of color information item differences, the ascertained color information item differences each being weighted as a function of the corresponding quality value of the predefined point;

adapt at least one subregion of the second camera image as a function of the determined global color transformation matrix; and

display a surrounding-area model of the reference object as a function of the lattice structure, of at least a subregion of the first camera image, and of at least the adapted subregion of the second camera image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2021
From: RAPROEGER, DIRK; TORRES LOPEZ, LIDIA ROSARIO; HERZOG, PAUL ROBERT; LAUER, PAUL-SEBASTIAN; BROSCH, UWE
To: ROBERT BOSCH GMBH
Reel/Frame 055682/0061 →
Priority Claims (1)
DE 102018214602.0 · Aug 29, 2018 · national
Continuity (1)
Related Publication 20210321038A1 · Oct 14, 2021