IP Library Granted Patent US 12,282,982
Granted Patent B2
US 12,282,982 · App. 18/526,213 · Granted Apr 22, 2025

Method and system for in-vehicle sensor range and FOV AR visualization

Inventors: Jani Mantyjarvi (Oulunsalo, FI); Jussi Ronkainen (Oulu, FI); Mikko Tarkiainen (Tampere, FI)
Assignee: DRNC HOLDINGS, INC.
G06T11/00B60W50/14B60W60/001G01C21/365G05D1/0088B60W2050/146B60W2420/403B60W2420/408B60W2556/50
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,282,982
App. No.
18/526,213
Granted
Apr 22, 2025
Kind
B2
Abstract

Some embodiments of a method disclosed herein may include: receiving a predicted driving route, sensor ranges of sensors on an autonomous vehicle (AV), and sensor field-of-view (FOV) data; determining whether minimum sensor visibility requirements are met along the predicted driving route; predicting blind areas along the predicted driving route, wherein the predicted blind areas are determined to have potentially diminished sensor visibility; and displaying an augmented reality (AR) visualization of the blind areas using an AR display device.

Claims (15)

1. A method comprising:

receiving a predicted driving route of a vehicle, and sensor range and sensor field-of-view (FOV) data for sensors on the vehicle;

predicting sensor blind areas along the predicted driving route of the vehicle, wherein the predicted sensor blind areas are determined from the received sensor range and sensor field-of-view data and the predicted driving route of the vehicle; and

displaying an augmented reality (AR) visualization of the predicted sensor blind areas along the predicted driving route of the vehicle using an AR display.

2. The method of claim 1 , wherein the predicted sensor blind areas are areas along the predicted driving route where one or more of sensor range and sensor field-of-view are below a predetermined threshold.

3. The method of claim 2 , wherein the AR visualization includes the predicted driving route.

4. The method of claim 3 , wherein predicting sensor blind areas comprises determining a plurality of locations along the predicted driving route of the vehicle where the one or more sensor range and sensor field-of-view are below the predetermined threshold.

5. The method of claim 4 , wherein predicting sensor blind areas comprises continually determining locations of blind areas along the predicted driving route.

6. The method of claim 5 , wherein displaying the AR visualization of the sensor blind areas comprises projecting the AR visualization of the sensor blind areas using an in-vehicle augmented reality projection system.

7. The method of claim 6 , wherein displaying the AR visualization of the sensor blind areas comprises displaying an outline of areas on a map indicating the sensor blind areas.

8. The method of claim 7 , further comprising changing a driving mode of the vehicle based on the predicted sensor blind areas along the predicted driving route.

9. The method of claim 8 , wherein the driving mode of the vehicle is one selected from the group consisting of manual mode, driver assist mode, and fully-autonomous mode.

10. The method of claim 1 , further comprising:

receiving sensor range and sensor field-of-view (FOV) data for sensors on a second vehicle;

predicting sensor blind areas along the predicted driving route of the vehicle, wherein the predicted sensor blind areas are determined from the sensor range and sensor field-of-view data and the predicted driving route of the vehicle, and the sensor range and sensor field-of view data for sensors on the second vehicle.

Continuity (3)
Continuation 16614296
Provisional Application 62509386 · May 22, 2017
Related Publication 20240177366A1 · May 30, 2024
References Cited (50)
US 6859148B2 · Miller et al. · 2005 [cited by applicant]
US 8589014B2 · Fairfield et al. · 2013 [cited by applicant]
US 8676431B1 · Mariet et al. · 2014 [cited by applicant]
US 9170332B2 · Shimizu · 2015 [cited by applicant]
US 9367065B2 · Dolgov et al. · 2016 [cited by applicant]
US 9403482B2 · Shahraray et al. · 2016 [cited by applicant]
US 9493157B2 · Prokhorov · 2016 [cited by applicant]
US 10591924B1 · Ferguson · 2020 [cited by examiner]
US 10845202B1 · Urmson · 2020 [cited by examiner]
US 20060085131A1 · Yopp et al. · 2006 [cited by applicant]
US 20070182528A1 · Breed et al. · 2007 [cited by applicant]
US 20090153665A1 · Linsenmaier et al. · 2009 [cited by applicant]
US 20090326819A1 · Taguchi · 2009 [cited by applicant]
US 20120310466A1 · Fairfield · 2012 [cited by examiner]
US 20130080290A1 · Kamm et al. · 2013 [cited by applicant]
US 20130297199A1 · Kapp et al. · 2013 [cited by applicant]
US 20140204117A1 · Kinnebrew et al. · 2014 [cited by applicant]
US 20140214255A1 · Dolgov et al. · 2014 [cited by applicant]
US 20140236652A1 · Argue et al. · 2014 [cited by applicant]
US 20140330478A1 · Cullinane et al. · 2014 [cited by applicant]
US 20150022426A1 · Ng-Thow-Hing et al. · 2015 [cited by applicant]
US 20160003636A1 · Ng-Thow-Hing · 2016 [cited by examiner]
US 20160091323A1 · Macgougan et al. · 2016 [cited by applicant]
US 20160214647A1 · Weisswange et al. · 2016 [cited by applicant]
US 20160327953A1 · Nilsson et al. · 2016 [cited by applicant]
US 20160375911A1 · Coelingh et al. · 2016 [cited by applicant]
US 20160379411A1 · Harbach et al. · 2016 [cited by applicant]
US 20170025017A1 · Thomas et al. · 2017 [cited by applicant]
US 20170116862A1 · Inoue · 2017 [cited by applicant]
US 20170269599A1 · Ansari et al. · 2017 [cited by applicant]
US 20180328752A1 · Tomatsu et al. · 2018 [cited by applicant]
US 20190325659A1 · Harbach et al. · 2019 [cited by applicant]
US 20200361482A1 · Choi et al. · 2020 [cited by applicant]
CN 103703496A · 2014 [cited by applicant]
CN 106494309A · 2017 [cited by applicant]
CN 106502248A · 2017 [cited by applicant]
JP 2006227811A · 2006 [cited by applicant]
JP 2008293099A · 2008 [cited by applicant]
JP 2012053533A · 2012 [cited by applicant]
KR 20160134016A · 2016 [cited by applicant]
WO WO2014130049A1 · 2014 [cited by applicant]
WO WO016109635A1 · 2016 [cited by applicant]
WO WO2016210257A1 · 2016 [cited by applicant]
Sawabe, Taishi, “Comfortable Autonomous Navigation Based on Collision Prediction in Blind Occluded Regions”, Nov. 2015, Proceedings of the 2015 IEEE International Conference on Vehicular Electronics and Safety, pp. 75-8… [cited by examiner]
Ferrari, “Augmented Reality Showroom App”. Web Archive dated Mar. 23, 2016, available at: https://web.archive.org/web/20160323003356/http://zspace.com.au/Ferrari-AR-Showroom-App, 5 pages. [cited by applicant]
Lin, Che-Tsung, et al. “Front Vehicle Blind Spot Translucentization Based on Augmented Reality”. IEEE 78th Vehicular Technology Conference (2013), 6 pages. [cited by applicant]
International Preliminary Report on Patentability for PCT/US2018/032747 issued on Nov. 26, 2019, 8 pages. [cited by applicant]
Seif, Heiko G., et al., “Autonomous Driving in the iCity-HD Maps as a Key Challenge”, Science Direct, Engineering, vol. 2, No. 2, Jun. 2016, pp. 159-162. [cited by applicant]
BEO Automotive, “ibeo.HAD Feature Fusion”. Web Archive dated Nov. 18, 2016, retrieved from: hllps://web.rchive.org/web/20161118104830/http://www.ibeo-as.com/had/ibeo-feature-fusion/, 5 pages. [cited by applicant]
Gao et al., “An Entry Point Based Local Path Planning for Mobile Robot with Blind Zone”, Applied Mechanics and Materials 373 (2013): 197-200. [cited by applicant]