IP Library › Granted Patent US 10,281,721
Granted Patent B2
US 10,281,721 · App. 15/683,970 · Granted May 7, 2019

System and method for augmented reality head up display for vehicles

Inventor: Vijyat Bhalla (Vaughan, CA)
Assignee: 8696322 Canada Inc.
G02B27/0101B60K37/06B60W40/08B60W40/09B60W50/08G06F17/30386G06T19/006G08G1/0137G08G1/0175B60W2540/28
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,281,721
App. No.
15/683,970
Granted
May 7, 2019
Kind
B2
Abstract

What is disclosed is a method for augmented reality head up display in a user vehicle comprising: capturing, by input devices, input data; receiving, by a communications manager, said captured input data from said input devices; receiving, by said communications manager, data from a communications path from at least one of a data center, and one or more external parties; providing, by said communications manager, said received captured input data from said input devices, and said data received from said communications path to a central processing unit (CPU) coupled to said communications manager; generating, by said CPU, custom data based on at least one of said received captured input data, and said data received from said communications path; and displaying, by a windshield projector, said generated custom data on a windshield of said vehicle.

Claims (170)

1. An augmented reality head up display (AR-HUD) system for a vehicle associated with a user, wherein said vehicle comprises a windshield, and said AR-HUD system comprises:

input devices comprising

a body camera worn by the user, and

one or more body-mounted audio recording devices worn by the user;

a windshield projector;

a communications path coupled to a data center and one or more external parties;

a central processing unit (CPU) communicatively coupled to

said input devices,

said communications path, and

said windshield projector

via a communications network manager;

further wherein

said input devices are used to capture input data comprising

a detected eye position,

one or more detected positions corresponding to one or more real world objects viewed through said windshield,

further wherein said one or more real world objects comprise a target vehicle,

an image of a target vehicle license plate associated with said target vehicle, and

data captured by the body camera or the one or more body mounted audio recording devices;

said communications network manager is used to

receive said captured input data from said input devices,

receive data from said communications path from at least one of said data center and said one or more external parties,

provide said received captured input data and said data received from said communications path to said CPU, further wherein

said CPU transmits one or more search requests based on said received captured input data, and

said data received from said communications path comprises one or more records retrieved based on said one or more search requests;

said CPU is used to generate custom data based on at least one of

said received captured input data, and

said data received from said communications path,

wherein said custom data comprises one or more augmented reality overlay elements, further wherein

 said one or more augmented reality overlay elements are generated based on at least one of

 said detected eye position,

 said one or more detected positions corresponding to one or more real world objects viewed through said windshield,

 said data captured by the body camera or the one or more body mounted audio recording devices, and

 said one or more records retrieved based on said one or more search requests; and

said windshield projector is used to display said generated custom data on a windshield of the vehicle, wherein

at least one of said one or more augmented reality overlay elements is associated with said target vehicle and is displayed on said windshield proximate to one or more views of said target vehicle on said windshield.

2. The system of claim 1 , wherein said one or more search requests comprise a first search request and a second search request;

said first search request is based on the image of the target vehicle license plate associated with said target vehicle; and

said second search request is based on the data captured by the body camera or the one or more body mounted audio recording devices.

3. The system of claim 1 , wherein one of said one or more search requests is based on

the image of the target vehicle license plate associated with said target vehicle, and

the data captured by the body camera or the one or more body mounted audio recording devices.

4. The system of claim 1 , wherein at least some of said data captured by the body camera or the one or more body mounted audio recording devices is related to said target vehicle.

5. The system of claim 1 , wherein at least some part of said retrieved one or more records is transmitted to said user via an audio device or an earpiece carried by said user.

6. The system of claim 2 , wherein

at least one of said one or more records is retrieved based on said second search request; and

at least one of said one or more augmented reality overlay elements is generated based on

said detected eye position,

said one or more detected positions corresponding to one or more real world objects viewed through said windshield, and

said at least one of said one or more records retrieved based on said second search request.

7. The system of claim 2 , wherein

at least some of said one or more records are retrieved based on said first and said second search request; and

at least one of said one or more augmented reality overlay elements is generated based on

said detected eye position,

said one or more detected positions corresponding to one or more real world objects viewed through said windshield, and

said at least some of said one or more records retrieved based on said first and second search request.

8. The system of claim 5 , wherein

at least one of said one or more records is retrieved based on said one of said one or more search requests; and

at least one of said one or more augmented reality overlay elements is generated based on

said detected eye position,

said one or more detected positions corresponding to one or more real world objects viewed through said windshield, and

said at least one of said one or more records retrieved based on said one of said one or more search requests.

9. The system of claim 1 , further wherein said one or more augmented reality overlay elements are generated based on said data captured by the body camera or the one or more body mounted audio recording devices.

10. The system of claim 9 , wherein at least some part of said retrieved one or more records is transmitted to said user via an audio device or a earpiece carried by said user.

11. A method for augmented reality head up display in a user vehicle comprising:

capturing, by input devices, input data,

said input devices comprising

a body camera worn by the user, and

one or more body-mounted audio recording devices worn by the user, and said input data comprising

a detected eye position,

one or more detected positions corresponding to one or more real world objects viewed through said windshield,

further wherein said one or more real world objects comprise a target vehicle,

an image of a target vehicle license plate associated with said target vehicle, and

data captured by the body camera or the one or more body mounted audio recording devices;

receiving, by a communications network manager, said captured input data from said input devices;

providing, by said communications network manager, said received captured input data to a central processing unit (CPU) coupled to said communications network manager;

transmitting, by said CPU, one or more search requests based on said received captured input data;

receiving, by said communications manager, data from a communications path from at least one of

a data center, and

one or more external parties;

providing, by said communications manager, said data received from said communications path, to said CPU, wherein

said data received from said communications path comprises one or more records retrieved based on said one or more search requests;

generating, by said CPU, custom data based on at least one of

said received captured input data, and

said data received from said communications path,

wherein said custom data comprises one or more augmented reality overlay elements, further wherein

said one or more augmented reality overlay elements are generated based on at least one of

 said detected eye position,

 said one or more detected positions corresponding to one or more real world objects viewed through said windshield,

 said data captured by the body camera or the one or more body mounted audio recording devices, and

 said one or more records retrieved based on said one or more search requests; and

displaying, by a windshield projector, said generated custom data on a windshield of said vehicle, wherein

at least one of said one or more augmented reality overlay elements is associated with said target vehicle and is displayed on said windshield proximate to one or more views of said target vehicle on said windshield.

12. The method of claim 11 , further wherein

said one or more real world objects comprise a first person of interest;

said input data comprises an image associated with said first person of interest;

said data received from said communications path comprises records corresponding to one or more persons of interest transmitted by said data center; and

further wherein at least one of said one or more augmented reality overlay elements is

associated with said first person of interest,

generated based on information obtained by said CPU cross referencing said image associated with said first person of interest with said records corresponding to one or more persons of interest, and

displayed on said windshield proximate to one or more views of said first person of interest.

13. The method of claim 12 , further wherein

said one or more real world objects comprises a vehicle different to said target vehicle;

said input data comprises an image of a vehicle license plate associated with said different vehicle;

said data received from said communications path comprises records corresponding to one or more vehicles of interest transmitted by said data center,

wherein said transmission of said records corresponding to one or more vehicles of interest occurs without a search request being transmitted to said data center; and

further wherein at least one of said one or more augmented reality overlay elements is

associated with said different vehicle,

generated based on information obtained by said CPU cross referencing said image of said vehicle license plate associated with said different vehicle with said records corresponding to one or more vehicles of interest, and

displayed on said windshield proximate to one or more views of said different vehicle.

14. The method of claim 13 , further wherein:

said one or more real world objects comprise a second person of interest;

said input data comprises an image of said second person of interest;

said transmitted one or more search requests comprises a first search request;

said first search request comprises said image of said second person of interest to said data center via said coupled communications network manager and said communications path;

said data received from said communications path by said communications network manager comprises one or more records retrieved based on said first search request; and

further wherein said at least one of said one or more augmented reality overlay elements are

associated with said second person of interest,

generated based on said retrieved one or more records, and

displayed on said windshield proximate to one or more views of said second person of interest on said windshield.

15. The system of claim 14 , wherein at least two of

said at least one of said one or more augmented reality overlay elements associated with said target vehicle,

said at least one of said one or more augmented reality overlay elements associated with said first person of interest,

said at least one of said one or more augmented reality overlay elements associated with said different vehicle, and

said at least one of said one or more augmented reality overlay elements associated with said second person of interest,

are displayed on the windshield.

16. The method of claim 15 , wherein said data received from said communications path comprises at least one of

an email;

an instant message;

a broadcast message; and

a telephone call.

17. The method of claim 16 , wherein said custom data comprises one or more of

a notification associated with the email;

a messenger icon associated with the instant message;

an email icon associated with the email;

a broadcast message icon associated with the broadcast message;

a scrolling banner, and

a telephone icon associated with the telephone call.

18. The method of claim 17 , wherein said input data comprises a speed corresponding to said target vehicle; and

further wherein said at least one of said one or more augmented reality overlay elements comprise said speed.

19. The method of claim 18 , further wherein

said at least one of said one or more augmented reality overlay elements comprises an indication that said speed exceeds a limit for a road travelled on by said target vehicle; and

a ticket is generated based on said input data.

20. A method for an augmented reality head up display (AR-HUD) for a vehicle associated with a user, said vehicle having a windshield, said method comprising:

providing input devices comprising

a body camera worn by the user,

one or more body-mounted audio recording devices worn by the user, and

said input devices are used to capture input data comprising

a detected eye position,

one or more detected positions corresponding to one or more real world objects viewed through said windshield,

further wherein said one or more real world objects comprise a target vehicle,

an image of a target vehicle license plate associated with said target vehicle, and

data captured by the body camera or the one or more body mounted audio recording devices;

providing a communications network manager to

receive said captured input data from said input devices, and

receive data from a communications path coupled to at least one of a data center and one or more external parties;

providing a central processing unit (CPU), wherein

said communications network manager transmits said received captured input data and said data received from said communications path to said CPU,

said CPU transmits one or more search requests based on said received captured input data,

said data received from said communications path comprises one or more records retrieved based on said one or more search requests, and

said CPU generates custom data based on at least one of

said received captured input data, and

said data received from said communications path,

wherein said generated custom data comprises one or more augmented reality overlay elements, further wherein

 said one or more augmented reality overlay elements are generated based on at least one of

 said detected eye position,

 said one or more detected positions corresponding to one or more real world objects viewed through said windshield,

 said data captured by the body camera or the one or more body mounted audio recording devices, and

 said one or more records retrieved based on said one or more search requests; and

providing a windshield projector to display said generated custom data on the windshield of the vehicle, wherein

at least one of said one or more augmented reality overlay elements is associated with said target vehicle and is displayed on said windshield proximate to one or more views of said target vehicle on said windshield.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2021
From: 8696322 CANADA INC.
To: BHALLA, VIJYAT
Reel/Frame 058430/0572 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2017
From: BHALLA, VIJYAT
To: 8696322 CANADA INC.
Reel/Frame 043367/0854 →
Continuity (2)
Provisional Application 62378274 · Aug 23, 2016
Related Publication 20180059416A1 · Mar 1, 2018
Cited By (2)
US 12,472,967 US 12,515,523