IP Library Granted Patent US 11,061,399
Granted Patent B2
US 11,061,399 · App. 16/228,508 · Granted Jul 13, 2021

System and method for providing information indicative of autonomous availability

Inventors: Sameer M. Bhalla (San Francisco, CA); Nicholas C. Fishwick (San Francisco, CA); Julia A. Wolinsky (San Francisco, CA); Yun Z. Wu (Los Angeles, CA)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G05D1/0061B60W40/08B60W50/14B60W50/16B60W2050/143B60W2050/146G02B27/0101G02B2027/0141G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,061,399
App. No.
16/228,508
Granted
Jul 13, 2021
Kind
B2
Abstract

One embodiment provides a system comprising at least one processor and a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including to obtain sensor data of a vehicle operating in an autonomous mode, determine a confidence level for continued operation of the vehicle in the autonomous mode based on the obtained sensor data, and selectively trigger a handoff protocol based on the confidence level determined. Movement of the vehicle is autonomously controlled in the autonomous mode. The handoff protocol comprises providing information to at least one input/output (I/O) device inside the vehicle as an alert of an upcoming transition from operating the vehicle in the autonomous mode to a manual mode.

Claims (69)

1. A system comprising:

at least one processor; and

a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including to:

obtain sensor data of a vehicle operating in an autonomous mode, wherein movement of the vehicle is autonomously controlled in the autonomous mode;

determine a confidence level for continued operation of the vehicle in the autonomous mode based on the obtained sensor data; and

trigger a handoff protocol when the confidence level is below a pre-determined threshold, wherein the handoff protocol comprises:

providing information to at least one input/output (I/O) device inside the vehicle that outputs the information as a set of alerts of an upcoming transition from operating the vehicle in the autonomous mode to a manual mode, and the information outputted includes an alert indicative of a user action to perform inside the vehicle as user preparation to take over control of the vehicle;

determining whether the user action has been performed; and

in response to determining the user action has been performed, updating the information outputted to include another alert indicative of another user action to perform inside the vehicle as user confirmation of user readiness to take over control of the vehicle.

2. The system of claim 1 , wherein the information outputted includes an alert indicative of the confidence level.

3. The system of claim 1 , wherein the at least one processor further performs operations to:

determine an amount of time remaining for the operation of the vehicle in the autonomous mode, wherein the information outputted includes a countdown timer that counts down the amount of time remaining for the operation of the vehicle in the autonomous mode, and the vehicle transitions to manual mode operation when the operation of the vehicle in the autonomous mode ends.

4. The system of claim 1 , wherein the at least one I/O device comprises at least one of the following: a display device, an audio device, or a haptic feedback device.

5. The system of claim 1 , wherein the obtained sensor data comprises data indicative of a current activity of a user inside the vehicle.

6. The system of claim 5 , wherein the at least one processor further performs operations to:

determine which of the at least one I/O device to utilize in outputting the information to increase a likelihood that the user is alerted to the upcoming transition.

7. The system of claim 6 , wherein the at least one processor further performs operations to:

determine a field of vision of the user based on the obtained sensor data;

determine one or more display devices available inside the vehicle that are substantially within the field of vision of the user; and

display one or more visual alerts on the one or more display devices to alert the user to the upcoming transition;

wherein the one or more display devices comprise at least one of the following: a head-up display, a dashboard display, an infotainment console, a light indicator, or a mobile electronic device.

8. The system of claim 6 , wherein the at least one processor further performs operations to:

determine whether an environment inside the vehicle is suitable for providing audio alerts of the upcoming transition based on the obtained sensor data; and

produce an audio alert via an audio device available inside the vehicle in response to determining the environment inside the vehicle is suitable for providing audio alerts;

wherein the audio device comprises at least one of the following: a speaker device or a mobile electronic device.

9. The system of claim 6 , wherein the at least one processor further performs operations to:

determine whether an environment inside the vehicle is suitable for haptic alerts of the upcoming transition based on the obtained sensor data; and

produce a haptic alert via a haptic feedback device available inside the vehicle in response to determining the environment inside the vehicle is suitable for haptic alerts;

wherein the haptic feedback device comprises at least one of the following: a driver seat of the vehicle, a steering wheel of the vehicle, or a mobile electronic device.

10. The system of claim 1 , wherein the at least one processor further performs operations to:

display a visual alert on a display device inside the vehicle that is substantially within a line of vision of the user in response to the vehicle transitioning from operating in the automatic mode to the manual mode, wherein the visual alert is indicative of an action for the user to perform to complete the handoff protocol, and the display device comprise at least one of a head-up display or a dashboard display.

11. The system of claim 1 , wherein the at least one processor further performs operations to:

produce one or more audio alerts via one or more audio devices inside the vehicle in response to the vehicle transitioning from operating in the automatic mode to the manual mode, wherein the one or more audio alerts are indicative of one or more actions for the user to perform to complete the handoff protocol, and the one or more audio devices comprise at least one of the following: a speaker device, or a mobile electronic device.

12. The system of claim 1 , wherein the at least one processor further performs operations to:

output a first set of information via the at least one I/O device in response to the vehicle operating in the autonomous mode, wherein the first set of information is optimized for the autonomous mode; and

output a second set of information via the at least one I/O device in response to the vehicle operating in the manual mode, wherein the second set of information is optimized for the manual mode, and the first set of information and the second set of information are different to distinguish between the vehicle operating in the autonomous mode and the vehicle operating in the manual mode.

13. The system of claim 1 , wherein the at least one processor further performs operations to:

determine an amount of distance remaining for the operation of the vehicle in the autonomous mode, wherein the information outputted includes the amount of distance remaining for the operation of the vehicle in the autonomous mode.

14. The system of claim 1 , wherein the at least one processor further performs operations to:

receive a first location and a second location; and

provide one or more routes available between the first location and the second location for selection, wherein each route includes information indicative of whether the vehicle is operable in the autonomous mode for one or more portions of the route.

15. A method comprising:

obtaining sensor data of a vehicle operating in an autonomous mode from one or more sensor devices, wherein movement of the vehicle is autonomously controlled in the autonomous mode;

determining, at a processor device, a confidence level for continued operation of the vehicle in the autonomous mode based on the obtained sensor data; and

triggering, at the processor device, a handoff protocol when the confidence level is below a pre-determined threshold, wherein the handoff protocol comprises:

providing information to at least one input/output (I/O) device inside the vehicle that outputs the information as a set of alerts of an upcoming transition from operating the vehicle in the autonomous mode to a manual mode, and the information outputted includes an alert indicative of a user action to perform inside the vehicle as user preparation to take over control of the vehicle;

determining whether the user action has been performed; and

in response to determining the user action has been performed, updating the information outputted to include another alert indicative of another user action to perform inside the vehicle as user confirmation of user readiness to take over control of the vehicle.

16. The method of claim 15 , wherein the information outputted includes an alert indicative of the confidence level.

17. The method of claim 15 , further comprising:

determining, at the processor device, a field of vision of a user inside the vehicle based on the obtained sensor data;

determining, at the processor device, one or more display devices available inside the vehicle that are substantially within the field of vision of the user; and

displaying one or more visual alerts on the one or more display devices to alert the user to the upcoming transition;

wherein the one or more display devices comprise at least one of the following: a head-up display, a dashboard display, an infotainment console, a light indicator, or a mobile electronic device.

18. The method of claim 15 , further comprising:

determining, at the processor device, whether an environment inside the vehicle is suitable for providing audio alerts of the upcoming transition based on the obtained sensor data; and

producing an audio alert via an audio device available inside the vehicle in response to determining the environment inside the vehicle is suitable for providing audio alerts;

wherein the audio device comprises at least one of the following: a speaker device or a mobile electronic device.

19. The method of claim 15 , further comprising:

determining, at the processor device, whether an environment inside the vehicle is suitable for haptic alerts of the upcoming transition based on the obtained sensor data; and

producing a haptic alert via a haptic feedback device available inside the vehicle in response to determining the environment inside the vehicle is suitable for haptic alerts;

wherein the haptic feedback device comprises at least one of the following: a driver seat of the vehicle, a steering wheel of the vehicle, or a mobile electronic device.

20. A non-transitory computer readable storage medium including instructions to perform a method comprising:

determining a field of vision of a user inside a vehicle operating in an autonomous mode based on sensor data captured, wherein movement of the vehicle is autonomously controlled in the autonomous mode;

determining one or more display devices available inside the vehicle that are substantially within the field of vision of the user;

displaying one or more visual alerts on the one or more display devices to alert the user of an upcoming transition from operating the vehicle in the autonomous mode to a manual mode, wherein the one or more visual alerts displayed includes a visual alert indicative of an action for the user to perform as preparation for the user to take over control of the vehicle;

determining whether the user has performed the action; and

in response to determining the user has performed the action, updating the one or more visual alerts displayed to include another visual alert indicative of another action for the user to perform as confirmation the user is ready to take over control of the vehicle;

wherein the one or more display devices comprise at least one of the following: a head-up display, a dashboard display, an infotainment console, a light indicator, or a mobile electronic device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2018
From: BHALLA, SAMEER M.; FISHWICK, NICHOLAS C.; WOLINSKY, JULIA A.; WU, YUN Z.
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 047836/0845 →
Continuity (3)
Provisional Application 62613243 · Jan 3, 2018
Provisional Application 62613270 · Jan 3, 2018
Related Publication 20190204827A1 · Jul 4, 2019
Cited By (1)
US 12,405,609