IP Library Granted Patent US 10,475,154
Granted Patent B2
US 10,475,154 · App. 15/675,224 · Granted Nov 12, 2019

Machine surround view system and method for generating 3-dimensional composite surround view using same

Inventors: Douglas Jay Husted (Secor, IL); David Lee Zwetz, II (Morton, IL); Adam Paul Roesner (Dunlap, IL); Edward Yii-Hsin Lin (Dunlap, IL); Hassen Ali (Peoria, IL); Peter Petrany (Dunlap, IL)
Assignee: Caterpillar Inc.
G06T3/0081B60R1/00G06T3/005G06T3/4038G06T7/70B60R2300/102B60R2300/105B60R2300/303
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,475,154
App. No.
15/675,224
Granted
Nov 12, 2019
Kind
B2
Abstract

A surround view system for a machine is provided. The surround view system includes a plurality of image capturing devices generating image data of surroundings of the machine, and an object detection system for detecting an object in a target field of view of the machine and generating object position data corresponding to the object. The surround view system also includes an image processing system configured to generate an initial 3-dimensional composite surround view by projecting the image data on a virtual model corresponding to the machine. The virtual model has a 3-dimensional shape based on an initial calibration position data. The image processing system is also configured to modify the 3-dimensional shape of the virtual model based on the object position data, and generate an updated 3-dimensional composite surround view by projecting the image data on the virtual model having a modified 3-dimensional shape.

Claims (61)

1. A surround view system for a machine, the surround view system including:

one or more devices configured to generate image data of surroundings of the machine;

one or more processors configured to:

generate an initial 3-dimensional composite surround view by projecting the image data on a virtual model corresponding to the machine,

wherein the virtual model has a 3-dimensional shape based on initial calibration position data;

detect a first object and a second object in a target field of view of the machine;

filter, based on a size of the first object and a size of the second object, data for the first object and the second object to determine to consider the first object and ignore the second object;

modify, to obtain a modified 3-dimensional shape, the 3-dimensional shape of the virtual model based on a position of the first object after filtering data for the first object and the second object; and

generate an updated 3-dimensional composite surround view by projecting the image data on the virtual model having the modified 3-dimensional shape.

2. The surround view system of claim 1 , wherein, when modifying the 3-dimensional shape of the virtual model, the one or more processors are configured to:

modify an inner diameter of the virtual model and an outer diameter of the virtual model independently.

3. The surround view system of claim 1 , wherein, when modifying the 3-dimensional shape of the virtual model, the one or more processors are configured to:

modify an inner diameter of the virtual model, and

modify an outer diameter of the virtual model as a function of the inner diameter.

4. The surround view system of claim 1 ,

wherein the virtual model is a bowl-shaped model, and

wherein the modified 3-dimensional shape is asymmetrical.

5. The surround view system of claim 1 , wherein at least one of a radar, a laser, LiDAR, video, or ultrasound is used to detect the first object.

6. The surround view system of claim 5 , wherein the one or more processors are configured to:

generate object position data corresponding to the position of the first object.

7. The surround view system of claim 1 , further including:

a user display configured to display the updated 3-dimensional composite surround view.

8. A method including:

generating image data of surroundings of a machine;

generating an initial 3-dimensional composite surround view by projecting the image data on a virtual model,

wherein the virtual model has a 3-dimensional shape based on an initial calibration position data;

detecting a first object and a second object in a target field of view of the machine;

filtering, based on a size of the first object and a size of the second object, data for the first object and the second object to determine to consider the first object and ignore the second object;

modifying, to obtain a modified 3-dimensional shape, the 3-dimensional shape of the virtual model based on object position data, corresponding to the first object, after filtering the data for the first object and the second object;

generating an updated 3-dimensional composite surround view by projecting the image data on the virtual model having a modified 3-dimensional shape; and

providing, for display, the updated 3-dimensional composite surround view on a user display.

9. The method of claim 8 , wherein modifying the 3-dimensional shape includes:

modifying an inner diameter of the virtual model and an outer diameter of the virtual model independently.

10. The method of claim 8 , wherein modifying the 3-dimensional shape includes:

modifying an inner diameter of the virtual model, and

modifying an outer diameter of the virtual model as a function of the inner diameter.

11. The method of claim 8 , wherein modifying the 3-dimensional shape of the virtual model includes:

generating an asymmetrical shape of the virtual model.

12. The method of claim 8 , wherein the updated 3-dimensional composite surround view is generated and displayed while the machine is in motion.

13. The method of claim 8 , further including:

generating the object position data.

14. An image processing system for generating a 3-dimensional composite surround view, the image processing system including:

a memory; and

a processor configured to:

receive image data of surroundings of a machine;

generate an initial 3-dimensional composite surround view by projecting the image data on a virtual model;

receive object position data corresponding to a first object and a second object in a target field of view of the machine;

filter, based on a size of the first object and a size of a second object, the object position data corresponding to the first object and the second object to determine to consider the first object and ignore the second object;

modify, to obtain a modified 3-dimensional shape, the 3-dimensional shape of the virtual model based on a position of the first object after filtering the object position data corresponding to the first object and the second object; and

generate an updated 3-dimensional composite surround view by projecting the image data on the virtual model having the modified 3-dimensional shape.

15. The image processing system of claim 14 , wherein, when modifying the 3-dimensional shape of the virtual model, the processor is configured to:

modify an inner diameter of the virtual model and an outer diameter of the virtual model independently.

16. The image processing system of claim 14 , wherein, when modifying the 3-dimensional shape of the virtual model, the processor is configured to:

modify an inner diameter of the virtual model, and

modify an outer diameter of the virtual model as a function of the inner diameter.

17. The image processing system of claim 14 , wherein the modified 3-dimensional shape is asymmetrical.

18. The image processing system of claim 14 , further including:

at least four cameras configured to generate the image data.

19. The image processing system of claim 18 , wherein each of the four cameras includes a fisheye lens.

20. The image processing system of claim 14 , wherein the processor is further configured to:

transmit the updated 3-dimensional composite surround view to a user display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2017
From: HUSTED, DOUGLAS; ZWETZ, DAVID; ROESNER, ADAM; LIN, EDWARD; ALI, HASSAN; PETRANY, PETER
To: CATERPILLAR INC.
Reel/Frame 043271/0827 →
Continuity (1)
Related Publication 20190050959A1 · Feb 14, 2019