Method for displaying the surroundings of a vehicle on a display device, processing unit and vehicle
A method for displaying an environment of a vehicle on a display includes: recording the environment with at least two cameras, each having a different field of view, wherein fields of view of adjacent cameras overlap; creating a panoramic image from at least two images taken by differing cameras, the images being projected into a reference plane for creating the panoramic image; ascertaining depth information pertaining to an object in the environment by triangulation from at least two differing individual images taken by the same camera; and generating an overlay structure as a function of the ascertained depth information, the overlay structure having been uniquely assigned to an imaged object; and, representing the created panoramic image, containing the at least one object, and the at least one generated overlay structure on the display such that the overlay structure is displayed on, and/or adjacent to, the assigned object.
1. A method for displaying an environment of a vehicle on a display device, the method comprising:
recording the environment with at least two cameras, wherein each of the at least two cameras has a different field of view, wherein the fields of view of adjacent ones of the at least two cameras overlap at least in some regions;
generating a panoramic image as a two-dimensional projection from at least two individual images, wherein each of the at least two individual images has been recorded by a different one of the at least two cameras, and the at least two individual images are projected into a reference plane for generating the panoramic image;
ascertaining depth information pertaining to at least one object in the recorded environment via a structure-from-motion method, wherein the depth information is ascertained by triangulation from at least two differing individual images taken by a same one of the at least two cameras, wherein the at least one object has been imaged in the at least two differing individual images, wherein each of the at least two differing individual images are recorded at different positions of the vehicle when the vehicle is moved between two positions;
generating at least one overlay structure as a function of the ascertained depth information, wherein each of the at least one overlay structures has been uniquely assigned to one of the at least one object, wherein the at least one overlay structure is displayed on the display device in a color, wherein the color is dependent on the ascertained depth information with respect to the uniquely assigned one of the at least one object; and,
presenting the generated panoramic image, containing the at least one object, and the at least one generated overlay structure on the display device such that the at least one overlay structure is displayed at least at one of on and adjacent to, the uniquely assigned one of the at least one object.
2. The method of claim 1 , wherein at least one of a bar, a polygon, and a text is displayed as the overlay structure on the display device.
3. The method of claim 2 , wherein the bar is displayed at least at one of on and adjacent to an outer edge of the uniquely assigned one of the at least one object on the display device.
4. The method of claim 3 , wherein the bar is perpendicular to an object normal of the uniquely assigned one of the at least one object, wherein the object normal is acquired from the depth information.
5. The method of claim 2 , wherein the polygon is imaged on the display device as overlay structure such that the polygon spans the uniquely assigned one of the at least one object at least partially.
6. The method of claim 1 , wherein the at least one overlay structure is displayed on the display device in a color established in advance.
7. The method of claim 6 , wherein the color of the at least one overlay structure is dependent on an object distance between the vehicle and the uniquely assigned one of the at least one object; and, the object distance is acquired from the ascertained depth information with respect to the uniquely assigned one of the at least one object.
8. The method of claim 6 , wherein at least one of the color and the type of the at least one overlay structure for the uniquely assigned one of the at least one object is dependent on a motion-indicator assigned to the at least one object; the motion-indicator specifies whether the at least one object is able to move or is permanently stationary; and, the motion-indicator is acquired from the ascertained depth information with respect to the uniquely assigned one of the at least one object.
9. The method of claim 8 , wherein at least one of an object contour and an object shape of the uniquely assigned one of the at least one object is ascertained from the depth information; and, the motion-indicator for the at least one object is derived from at least one of the object contour and the object shape via a deep-learning algorithm by comparison with at least one of known object contours and object shapes.
10. The method of claim 8 , wherein object points on the at least one object that have been imaged in the individual images are temporally tracked in order to derive the motion-indicator for the at least one object.
11. The method of claim 1 , wherein the at least one overlay structure is displayed opaquely or at least partially transparently on the display device so that the at least one overlay structure completely or at least partially covers the panoramic image at least at one of on and adjacent to the uniquely assigned one of the at least one object.
12. The method of claim 1 , wherein the display device exhibits display pixels and panoramic image points of the panoramic image are displayed on the display pixels of the display device; the at least one object contained in the panoramic image is represented on object pixels, wherein the object pixels are a subset of the display pixels; and, the overlay structure assigned to the uniquely assigned one of the at least one object is displayed at least at one of on and adjacent to the respective object pixels on the display device.
13. The method of claim 12 , wherein an overlay image with the at least one overlay structure is superimposed on the panoramic image with the at least one object on the display device such that the overlay structure assigned to the uniquely assigned one of the at least one object is displayed on the display device at least at one of on and adjacent to the respective object pixels.
14. The method of claim 12 , wherein the panoramic image contains the at least one overlay structure and the panoramic image is adapted therefor at least at one of at and adjacent to the panoramic image points on which the at least one object has been imaged such that the overlay structure assigned to the uniquely assigned one of the at least one object is displayed on the display device at least at one of on and adjacent to the respective object pixels.
15. The method of claim 1 , wherein the at least two individual images, from which the depth information is ascertained by triangulation, are recorded by the same camera from at least two differing viewpoints, wherein the vehicle is moved so as to position the same camera at the at least two differing viewpoints; and, the depth information is ascertained by triangulation as a function of a base length between the at least two viewpoints.
16. The method of claim 1 , wherein the environment is mapped within a panoramic region in the panoramic image; and, the panoramic region is larger than the fields of view of the individual cameras; a viewing angle of the panoramic region amounts to 360°; and, the panoramic image includes at least two individual images recorded at approximately a same time by differing cameras.
17. The method of claim 1 , wherein isolines assigned to the vehicle are represented in the overlay structure on the display device; and, the isolines are displayed at fixed isodistances from a vehicle exterior of the vehicle.
18. A processing unit configured to carry out the method of claim 1 .
19. A vehicle comprising:
at least two cameras each having a different field of view,
wherein the fields of view of adjacent ones of said at least two cameras overlap at least in some regions;
a display device;
a processing unit as claimed in claim 18 ,
said display device being configured to represent a created panoramic image, containing at least one object, and at least one generated overlay structure such that the at least one overlay structure is displayed at least at one of on and adjacent to the uniquely assigned one of the at least one object.
20. The vehicle of claim 19 , wherein the field of view of each of said at least two cameras has a viewing angle greater than or equal to 120°.
21. The vehicle of claim 19 , wherein the field of view of each of said at least two cameras has a viewing angle greater than or equal to 170°.