Multisensory gestural-audio interface to promote situational awareness for improved autonomous vehicle control

1 . A method for a multisensory gestural-audio interface system, the method comprising:

providing a haptic intersection representation of a driving environment to a driver of an ego vehicle through a haptics device by pulsing N-times at a center of a driver's palm to indicate a number of roads, drawing a line from the center of the palm toward a direction of a clock face position of a road, pulsing at an end of the line and repeating the haptic intersection representation until the driver responds;

scanning, through a multisensory gestural-audio interface device, a vehicle cabin to detect a gesture of the driver in response to the haptic intersection representation of the driving environment;

providing, through a surrounding ring of speakers of the multisensory gestural-audio interface device, a non-visual, spatial audio description of a selected vehicle control object according to the detected gesture of the driver; and

performing a vehicle control action based on the selected vehicle control object based on a confirmation from the driver to perform the vehicle control action.

2 . The method of claim 1 , in which scanning the vehicle cabin comprises tracking gestures of the driver to detect a scanning gesture.

3 . The method of claim 1 , further comprising providing a brief description of vehicle control objects in a scanning view using spatialized audio.

4 . The method of claim 1 , further comprising querying the driver to confirm the vehicle control action.

5 . The method of claim 1 , in which scanning the vehicle cabin comprises identifying a hand gesture of the driver.

6 . The method of claim 1 , in which providing the non-visual description further comprises playing an audio description of the vehicle control action.

7 . The method of claim 1 , further comprising overriding the vehicle control action if the vehicle control action may result in a collision of the ego vehicle with an external road agent detected in a surrounding scene of the ego vehicle.

8 . A non-transitory computer-readable medium having program code recorded thereon for a multisensory gestural-audio interface system, the program code being executed by a processor and comprising:

program code to provide a haptic intersection representation of a driving environment to a driver of an ego vehicle through a haptics device by pulsing N-times at a center of a driver's palm to indicate N-roads, drawing a line from the center of the palm toward a direction of a clock face position of a road, pulsing at an end of the line and repeating the haptic intersection representation until the driver responds;

program code to scan, through a multisensory gestural-audio interface device, a vehicle cabin to detect a gesture of the driver in response to the haptic intersection representation of the driving environment;

program code to provide, through a surrounding ring of speakers of the multisensory gestural-audio interface device, a non-visual, spatial audio description of a selected vehicle control object according to the detected gesture of the driver; and

program code to perform a vehicle control action based on the selected vehicle control object based on a confirmation from the driver to perform the vehicle control action.

9 . The non-transitory computer-readable medium of claim 8 , in which the program code to scan the vehicle cabin comprises program code to track gestures of the driver to detect a scanning gesture.

10 . The non-transitory computer-readable medium of claim 8 , further comprising program code to provide a brief description of vehicle control objects in a scanning view using spatialized audio.

11 . The non-transitory computer-readable medium of claim 8 , further comprising program code to query the driver to confirm the vehicle control action.

12 . The non-transitory computer-readable medium of claim 8 , in which the program code to scan the vehicle cabin comprises program code to identify a hand gesture of the driver.

13 . The non-transitory computer-readable medium of claim 8 , in which the program code to provide the non-visual description further comprises program code to play an audio description of the vehicle control action.

14 . The non-transitory computer-readable medium of claim 8 , further comprising program code to override the vehicle control action if the vehicle control action may result in a collision of the ego vehicle with an external road agent detected in a surrounding scene of the ego vehicle.

15 . A system for a multisensory gestural-audio interface system, the system comprising:

a non-visual environment overview module to provide a haptic intersection representation of a driving environment to a driver of an ego vehicle through a haptics device by pulsing N-times at a center of a driver's palm to indicate N-roads, drawing a line from the center of the palm toward a direction of a clock face position of a road, pulsing at an end of the line and repeating the haptic intersection representation until the driver responds;

a gesture tracking module to scan a vehicle cabin to detect a gesture of the driver in response to the haptic intersection representation of the driving environment;

a vehicle control selection module to provide, through a surrounding ring of speakers of the multisensory gestural-audio interface device, a non-visual, spatial audio description of a selected vehicle control object according to the detected gesture of the driver; and

a vehicle control action module to perform a vehicle control action based on the selected vehicle control object based on a confirmation from the driver to perform the vehicle control action.

16 . The system of claim 15 , in which the gesture tracking module is further to track gestures of the driver to detect a scanning gesture.

17 . The system of claim 15 , in which the gesture tracking module is further to identify a hand gesture of the driver.