IP Library › Granted Patent US 12,202,507
Granted Patent B2
US 12,202,507 · App. 17/962,070 · Granted Jan 21, 2025

Systems and methods for enhanced outdoor displays via augmented reality

Inventors: Brian M. Fields (Phoenix, AZ); Nathan L. Tofte (Downs, IL); Aaron C. Williams (Bloomington, IL); Joseph P. Harr (Bloomington, IL); Vicki King (Bloomington, IL)
Assignee: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
B60W50/14B60K35/00B60W40/02B60W40/04B60W40/08G01S1/042G02B27/0172G06T19/006G06V20/593B60K35/28B60K35/29B60K35/65B60K2360/166B60K2360/176B60K2360/177B60K2360/178B60K2360/1868B60K2360/741B60Q9/00B60W2050/146B60W2420/403B60W2540/225B60W2540/227B60W2554/4041B60W2556/40B60W2556/45G01C21/3476G02B2027/0138G02B2027/014G06F40/279G06Q50/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,202,507
App. No.
17/962,070
Granted
Jan 21, 2025
Kind
B2
Abstract

The following generally relates to using Augmented Reality (AR) to enhance the in-vehicle experience. In some examples, AR techniques are applied to provide AR indications of vehicle safety indicia to alert vehicle occupants to information that may not otherwise be perceptible. In other example, AR techniques are applied to provide emergency vehicle warnings to improve the likelihood of safe responses thereto. In yet other examples, AR techniques are applied to generated personalized outdoor displays, for example, to ameliorate conditions that may impair safe operation of a vehicle.

Claims (60)

1. A computer-implemented method for enhanced outdoor displays via Augmented Reality (AR), the method comprising:

obtaining, via one or more processors of an electronic device on-board a vehicle, contextual data associated with an occupant of the vehicle associated with an AR viewer;

determining, via the one or more processors, a position of an outdoor display relative to the vehicle;

determining, via the one or more processors, a field of view of the occupant of the vehicle; and

based upon the contextual data, the position, and the field of view, overlaying, via the one or more processors, an occupant-specific image onto the outdoor display via the AR viewer.

2. The computer-implemented method of claim 1 , wherein obtaining the contextual data comprises:

receiving, from a personal electronic device associated with the occupant, the contextual data.

3. The computer-implemented method of claim 1 , wherein obtaining the contextual data comprises:

obtaining, via the one or more processors, the contextual data from an audio sensor disposed in the vehicle.

4. The computer-implemented method of claim 1 , wherein obtaining the contextual data comprises:

obtaining, via the one or more processors, the contextual data from an image sensor configured to have a field of view oriented within the vehicle.

5. The computer-implemented method of claim 1 , wherein overlaying the occupant-specific image comprises:

inputting, via the one or more processors, the contextual data into a sentiment analysis model to obtain a condition associated with the occupant; and

obtaining, from an image database, an image associated with the condition.

6. The computer-implemented method of claim 5 , wherein:

the condition associated with the occupant is drowsiness; and

the image associated with the condition includes an indication of a hotel proximate to the vehicle.

7. The computer-implemented method of claim 1 , wherein overlaying the occupant-specific image comprises:

inputting, via the one or more processors, the contextual data into a natural language processing (NLP) model to obtain a subject relevant to the occupant; and

obtaining, from an image database, an image associated with the subject.

8. The computer-implemented method of claim 7 , wherein:

the subject relates to food; and

the image associated with the subject includes an indication of a food vendor proximate to the vehicle.

9. The computer-implemented method of claim 1 , wherein the AR viewer is at least one of a smart windshield, a smart window, or a smart mirror of the vehicle.

10. The computer-implemented method of claim 1 , wherein the AR viewer is a wearable AR viewer.

11. The computer-implemented method of claim 10 , wherein determining the field of view comprises:

obtaining, via the one or more processors, orientation data from the AR viewer.

12. The computer-implemented method of claim 1 , wherein determining the field of view comprises:

obtaining, via the one or more processors, image data generated by an image sensor configured to have a field of view oriented within the vehicle; and

based upon the image data, determining, via the one or more processors, a location of the occupant within the vehicle.

13. A system for enhanced outdoor displays via Augmented Reality (AR), the system comprising:

one or more processors of an electronic device on-board a vehicle; and

one or more non-transitory memories storing processor-executable instructions that, when executed by the one or more processors, cause the system to:

obtain contextual data associated with an occupant of the vehicle associated with an AR viewer;

determine a position of an outdoor display relative to the vehicle;

determine a field of view of the occupant of the vehicle; and

based upon the contextual data, the position, and the field of view, overlay an occupant-specific image onto the outdoor display via the AR viewer.

14. The system of claim 13 , wherein to obtain the contextual data, the instructions, when executed, cause the system to:

receive, from a personal electronic device associated with the occupant, the contextual data.

15. The system of claim 13 , wherein to obtain the contextual data, the instructions, when executed, cause the system to:

obtain the contextual data from an audio sensor disposed in the vehicle.

16. The system of claim 13 , wherein to obtain the contextual data, the instructions, when executed, cause the system to:

obtain the contextual data from an image sensor configured to have a field of view oriented within the vehicle.

17. The system of claim 13 , wherein to overlay the occupant-specific image, the instructions, when executed, cause the system to:

input the contextual data into a sentiment analysis model to obtain a condition associated with the occupant; and

obtain, from an image database, an image associated with the condition.

18. The system of claim 17 , wherein:

the condition associated with the occupant is drowsiness; and

the image associated with the condition includes an indication of a hotel proximate to the vehicle.

19. The system of claim 13 , wherein to overlay the occupant-specific image, the instructions, when executed, cause the system to:

input the contextual data into a natural language processing (NLP) model to obtain a subject relevant to the occupant; and

obtain, from an image database, an image associated with the subject.

20. The system of claim 19 , wherein:

the subject relates to food; and

the image associated with the subject includes an indication of a food vendor proximate to the vehicle.

21. A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors of an electronic device on-board a vehicle, cause the one or more processors to:

obtain contextual data associated with an occupant of the vehicle associated with an Augmented Reality (AR) viewer;

determine a position of an outdoor display relative to the vehicle;

determine a field of view of the occupant of the vehicle; and

based upon the contextual data, the position, and the field of view, overlay an occupant-specific image onto the outdoor display via the AR viewer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2022
From: FIELDS, BRIAN M.; TOFTE, NATHAN L.; WILLIAMS, AARON C.; HARR, JOSEPH P.; KING, VICKI
To: STATE FARM MUTUAL AUTOMOBILE INSURANCE COMPANY
Reel/Frame 061414/0585 →
Continuity (2)
Provisional Application 63397447 · Aug 12, 2022
Related Publication 20240054733A1 · Feb 15, 2024
References Cited (34)
US 9830748B2 · Rosenbaum · 2017 [cited by applicant]
US 9990782B2 · Rosenbaum · 2018 [cited by applicant]
US 10269190B2 · Rosenbaum · 2019 [cited by applicant]
US 10467824B2 · Rosenbaum · 2019 [cited by applicant]
US 11180159B1 · Post et al. · 2021 [cited by applicant]
US 11227452B2 · Rosenbaum · 2022 [cited by applicant]
US 11407410B2 · Rosenbaum · 2022 [cited by applicant]
US 11524707B2 · Rosenbaum · 2022 [cited by applicant]
US 11594083B1 · Rosenbaum · 2023 [cited by applicant]
US 20020050927A1 · De Moerloose · 2002 [cited by examiner]
US 20040179040A1 · Patel · 2004 [cited by examiner]
US 20100253526A1 · Szczerba · 2010 [cited by examiner]
US 20130335303A1 · Maciocci · 2013 [cited by examiner]
US 20150015479A1 · Cho · 2015 [cited by examiner]
US 20150126281A1 · Lewis · 2015 [cited by examiner]
US 20150216466A1 · Kronberg · 2015 [cited by examiner]
US 20150278370A1 · Stratvert · 2015 [cited by examiner]
US 20160120403A1 · Mochizuki · 2016 [cited by examiner]
US 20160267335A1 · Hampiholi · 2016 [cited by examiner]
US 20170155867A1 · Yokota · 2017 [cited by examiner]
US 20170213459A1 · Ogaz · 2017 [cited by applicant]
US 20180143635A1 · Zijderveld · 2018 [cited by examiner]
US 20190089999A1 · Neumeier · 2019 [cited by examiner]
US 20190102636A1 · Koravadi · 2019 [cited by applicant]
US 20220092893A1 · Rosenbaum · 2022 [cited by applicant]
US 20220340148A1 · Rosenbaum · 2022 [cited by applicant]
US 20230060300A1 · Rosenbaum · 2023 [cited by applicant]
US 20230111436A1 · Petit et al. · 2023 [cited by applicant]
EP 3239686A1 · 2017 [cited by applicant]
EP 3578433B1 · 2020 [cited by applicant]
EP 3730375B1 · 2021 [cited by applicant]
EP 3960576A1 · 2022 [cited by applicant]
EP 4190659A1 · 2023 [cited by applicant]
EP 4190660A1 · 2023 [cited by applicant]