IP Library Granted Patent US 9,580,014
Granted Patent B2
US 9,580,014 · App. 14/455,818 · Granted Feb 28, 2017

System, apparatus, and method of detecting and displaying obstacles and data associated with the obstacles

Inventors: Martin Lucas (Rancho Santa Margarita, CA); Blake Gasca (Aliso Viejo, CA); Edward Cheng (Hong Kong, CN); Daniel Rebelo (Newport Beach, CA); Christian Sullivan (Newport Beach, CA)
Assignee: CONVOY TECHNOLOGIES LLC
B60R1/00G06K9/00362G06K9/00805B60R2300/103B60R2300/107B60R2300/202B60R2300/802G06K9/00791
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,580,014
App. No.
14/455,818
Granted
Feb 28, 2017
Kind
B2
Abstract

A method of detecting and displaying the obstacles and data related to obstacles on a digital display starts with a detection processing unit receiving a digital video input signal from a first camera and depth data from a second camera. Digital input video input signal may include images of an area surrounding the vehicle. Digital video input signal may be analyzed to determine whether an obstacle is present in the area. When an obstacle is detected, the depth data may be analyzed to obtain a distance associated with the obstacle. Digital video output may be generated that includes the digital video input signal and a display of the distance associated with the obstacle. The display of the distance may be overlaid on the display of the obstacle. Digital video output may then be transmitted to the display device to be displayed to the user. Other embodiments are also described.

Claims (79)

1. A method of detecting obstacles surrounding a vehicle and displaying the obstacles and data related to the obstacles on a digital display inside the vehicle, the method comprising:

receiving by a processor of a detection processing unit a digital video input signal from a first camera and depth data from a second camera, wherein the digital input video input signal includes images of an area surrounding the vehicle;

analyzing by the processor the digital video input signal to determine whether at least one obstacle is present in the area surrounding the vehicle; and

when the at least one obstacle is detected,

analyzing by the processor the depth data to obtain a distance associated with the at least one obstacle,

generating by the processor a digital video output including the digital video input signal and a display of the distance associated with the at least one obstacle overlaid on the obstacle in the digital video input signal, and

transmitting by the processor the digital video output to the display device, and when a plurality of obstacles is detected,

generating by the processor a list prioritizing the plurality of obstacles based on at least one of:

(i) an identification of the at least one obstacle,

(ii) the distance associated with the at least one obstacle, and

(iii) a calculated time the at least one obstacle will enter a blind spot.

2. The method of claim 1 , wherein the at least one obstacle is a moving object or a non-moving object.

3. The method of claim 1 , wherein analyzing the depth data to obtain the distance associated with the obstacle further comprises:

capturing by the second camera a video image of infrared (IR) light on the at least one obstacle, wherein the infrared light is outputted from an IR light source mounted on the vehicle; and

calculating by the processor a distance between the IR light source and the at least one obstacle based on the IR light captured.

4. The method of claim 1 , further comprising:

receiving by the processor sensor data from 3-Dimensional (3D) sensors;

analyzing the sensor data to detect the presence of the at least one obstacle in the area surrounding the vehicle and to obtain the distance associated with the at least one obstacle.

5. The method of claim 4 , further comprising:

calculating by the processor the calculated time at which the at least one obstacle will enter the blind spot of the vehicle.

6. The method of claim 5 , wherein the digital video output further includes a display of the calculated time.

7. The method of claim 6 , further comprising:

determining by the processor a relevance of the at least one obstacle based on at least one of the sensor data, the depth data, and the digital video input signal,

wherein determining the relevance of the at least one obstacle includes by the processor performing at least one of:

(i) comparing factors associated with an image of the at least one obstacle in the digital video input signal to known factors stored in a memory storage device to identify the at least one obstacle,

(ii) comparing the distance associated with the at least one obstacle with a threshold distance, and

(iii) comparing the time at which when the at least one obstacle will enter the blind spot with a threshold time.

8. The method of claim 4 , wherein the digital video input signal, the depth data, and the sensor data are received via a wired or a wireless connection.

9. The method of claim 1 , wherein the digital video output is transmitted to the display device via a wired or a wireless connection.

10. The method of claim 1 , further comprising, when the at least one obstacle is detected, transmitting an audio output to the digital device, the audio output provides an audible alert.

11. A system of detecting obstacles surrounding a vehicle and displaying the obstacles and data related to the obstacles, the system comprising:

an infrared light source to generate an infrared light;

a first camera to generate a digital video input signal that includes images of an area surrounding the vehicle;

a second camera to generate depth data,

a detection processing unit coupled to the first and second cameras, the object detection processing unit including:

a processor,

a storage device including instructions which, when executed by the processor, causes the processor to:

analyze the digital video input signal to determine whether at least one obstacle is present in the area surrounding the vehicle,

when the at least one obstacle is detected,

(i) analyze the depth data to obtain a distance associated with the at least one obstacle, wherein the depth data generated by the second camera is associated with the at least one obstacle in the area surrounding the vehicle, wherein the depth data is a video image of the infrared (IR) light on at least one obstacle in the area surrounding the vehicle, and

(ii) generate a digital video output including the digital video input signal and a display of the distance associated with the at least one obstacle overlaid on the obstacle in the digital video input signal, and

when a plurality of obstacles is detected, a list prioritizing the plurality of obstacles based on at least one of:

(i) an identification of the at least one obstacle,

(ii) the distance associated with the at least one obstacle, and

(iii) a calculated time the at least one obstacle will enter a blind spot, and

a communication interface to

(i) receive the digital video input signal from the first camera and the depth data from the second camera, and

(ii) transmit the digital video output; and

a digital display to display the digital video output received from the communication interface of the detection processing unit.

12. The system of claim 11 , further comprising:

a plurality of 3D sensors to generate sensor data, wherein the processor analyzes the sensor data to detect the presence of the at least one obstacle in the area surrounding the vehicle.

13. An apparatus for detecting obstacles surrounding a vehicle and displaying the obstacles and data related to the obstacles, the apparatus comprising:

a processor,

a communication interface coupled to the processor to receive a digital video input signal from a first camera and depth data from a second camera; and

a storage device including instructions which, when executed by the processor, causes the processor to:

analyze the digital video input signal to determine whether at least one obstacle is present in the area surrounding the vehicle,

when the at least one obstacle is detected,

(i) analyze the depth data to obtain a distance associated with the at least one obstacle, wherein the depth data generated by the second camera is associated with the at least one obstacle in the area surrounding the vehicle, wherein the depth data is a video image of the infrared (IR) light on at least one obstacle in the area surrounding the vehicle, and

(ii) generate a digital video output including the digital video input signal and a display of the distance associated with the at least one obstacle overlaid on the obstacle in the digital video input signal, and

(iii) transmit the digital video output to a digital device via the communication interface,

when a plurality of obstacles is detected, a list prioritizing the plurality of obstacles based on at least one of:

(i) an identification of the at least one obstacle,

(ii) the distance associated with the at least one obstacle, and

(iii) a calculated time the at least one obstacle will enter a blind spot.

14. The apparatus of claim 13 , wherein analyzing the depth data to obtain the distance associated with the obstacle further comprises the processor

to signal to the second camera to capture a video image of infrared (IR) light on the at least one obstacle, wherein the infrared light is outputted from an IR light source mounted on the vehicle; and

to calculate a distance between the IR light source and the at least one obstacle based on the IR light captured.

15. The apparatus of claim 13 , wherein the instructions, when executed by the processor, further causes the processor to:

receive sensor data from 3-Dimensional (3D) sensors; and

analyze the sensor data to detect the presence of the at least one obstacle in the area surrounding the vehicle and to obtain the distance associated with the at least one obstacle.

16. The apparatus of claim 13 , wherein the instructions, when executed by the processor, further causes the processor to:

calculate the calculated time at which the at least one obstacle will enter the blind spot of the vehicle.

17. The apparatus of claim 16 , wherein the digital video output further includes a display of the calculated time.

18. The method of claim 17 , wherein the instructions, when executed by the processor, further causes the processor to:

determine a relevance of the at least one obstacle based on at least one of the sensor data, the depth data, and the digital video input signal,

wherein determining the relevance of the at least one obstacle includes at least one of:

(iv) comparing factors associated with an image of the at least one obstacle in the digital video input signal to known factors stored in a memory storage device to identify the at least one obstacle,

(v) comparing the distance associated with the at least one obstacle with a threshold distance, and

(vi) comparing the time at which when the at least one obstacle will enter the blind spot with a threshold time.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 27, 2026
From: CONVOY TECHNOLOGIES, INC.
To: PRO-VISION CONVOY, LLC
Reel/Frame 075793/0669 →
SECURITY INTEREST Recorded Jun 3, 2026
From: PRO-VISION SPARTAN RADAR, LLC; PRO-VISION CONVOY, LLC
To: BARINGS FINANCE LLC
Reel/Frame 074835/0813 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2017
From: CONVOY TECHNOLOGIES LLC
To: CONVOY TECHNOLOGIES, INC.
Reel/Frame 042394/0356 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2016
From: LUCAS, MARTIN; GASCA, BLAKE; CHENG, EDWARD; REBELO, DANIEL; SULLIVAN, CHRISTIAN
To: CONVOY TECHNOLOGIES LLC
Reel/Frame 039274/0636 →
Continuity (2)
Provisional Application 61863830 · Aug 8, 2013
Related Publication 20150043782A1 · Feb 12, 2015