See through viewing system for vehicle
A see through vision system for a vehicle includes a surround view system having exterior facing cameras. The exterior facing cameras are in communication with a controller. A driver monitoring system includes a driver monitoring sensor configured to monitor a viewing direction of a driver. The controller includes a memory storing a three dimensional model of the vehicle, a simultaneous localization and mapping (SLAM) module, a surround view module, and a data link for connecting to a wearable vision system. The controller further stores instructions for identifying a viewing direction of the driver using the driver monitoring system, identifying at least one opaque portion of the vehicle in the driver's view using the viewing direction and the three dimensional model of the vehicle, generating a cropped live video of an exterior view of the vehicle at the locations of the at least one opaque portion using the surround view module.
1 . A see through vision system for a vehicle comprising:
a surround view system including a plurality of exterior facing cameras, each of the exterior facing cameras being in communication with a controller;
a driver monitoring system including at least one driver monitoring sensor configured to monitor a viewing direction of a driver, the at least one driver monitoring sensor being in communication with the controller;
the controller including a non-transitory memory storing a three dimensional model of the vehicle, a simultaneous localization and mapping (SLAM) module, a surround view module, and a data link for connecting to a wearable vision system; and
the controller further storing instructions for identifying a viewing direction of the driver using the driver monitoring system, determine a position and orientation of the wearable vision system relative to the vehicle based at least in part on image information from the wearable vision system, identifying at least one opaque portion of the vehicle in the driver's view using the viewing direction and the three dimensional model of the vehicle, generating a cropped live video of an exterior view of the vehicle at locations of the at least one opaque portion using the surround view module, and exporting the generated cropped live video to the wearable vision system using the data link.
2 . The vehicle of claim 1 , wherein the at least one driver monitoring sensor includes an interior facing camera defining a field of view including a head of the driver.
3 . The vehicle of claim 1 , wherein the SLAM module is configured to determine the position and orientation of the wearable vision system relative to the vehicle based at least in part on the viewing direction of the driver using a simultaneous localization and mapping algorithm.
4 . The vehicle of claim 3 , wherein the SLAM module is further configured to receive at least one of sensor information and image information from the wearable vision system through the data link, and the SLAM module is configured to determine a position and orientation of the wearable vision system relative to the vehicle based at least in part on the viewing direction of the driver and the at least one of sensor information and image information from the wearable vision system.
5 . The vehicle of claim 4 , wherein the image information includes an image feed captured from a camera disposed on the wearable vision system.
6 . The vehicle of claim 5 , wherein the image information further includes object identification of objects in the image feed.
7 . The vehicle of claim 3 , wherein the simultaneous localization and mapping algorithm is performed exclusively on the controller.
8 . The vehicle of claim 1 , wherein the three dimensional model of the vehicle defines relative positions and sizes of structural elements of the vehicle and defines which structural elements of the vehicle are opaque.
9 . The vehicle of claim 1 , wherein the vehicle further includes a set of vehicle operation sensors, the vehicle operation sensors being configured to identify a vehicle driving speed, vehicle location, hazard identification, warning signals, driving guidance, and driving direction.
10 . The vehicle of claim 9 , wherein generating the cropped live video of the exterior view of the vehicle at locations of the at least one opaque portion using the surround view module includes annotating the cropped live view with at least one of the vehicle driving speed, pedestrian detection information, and hazard identification, warning signals, driving guidance.
11 . A method for generating see through view of a vehicle comprising:
determining a position and orientation of a wearable vision system within the vehicle using a vehicle controller, the vehicle controller operating a simultaneous localization and mapping (SLAM) algorithm based at least in part on sensor information from a vehicle driver monitoring system;
receiving image information from the wearable vision system, and wherein the determined position and orientation of the wearable vision system within the vehicle is further based on the image information from the wearable vision system;
determining at least one opaque portion of the vehicle in a view of a driver based on the position and orientation of the wearable vision system within the vehicle and a three dimensional model of the vehicle;
generating a cropped video feed of an exterior view of the vehicle at the position of the at least one opaque portion of the vehicle; and
outputting the cropped video feed to the wearable vision system.
12 . The method of claim 11 , further comprising displaying the cropped video feed to the driver using at least one screen of the wearable vision system.
13 . The method of claim 11 , wherein the simultaneous localization and mapping (SLAM) algorithm is performed exclusively on the vehicle controller.
14 . The method of claim 11 , wherein the received image information includes an image feed captured from a camera disposed on the wearable vision system.
15 . The method of claim 14 , wherein the image information further includes object identification of objects in the image feed.
16 . The method of claim 11 , wherein the three dimensional model of the vehicle defines relative positions and sizes of structural elements of the vehicle and defines which structural elements of the vehicle are opaque.
17 . The method of claim 11 , wherein generating the cropped video feed of an exterior view of the vehicle includes adding a driving parameter display of the vehicle to the exterior view, wherein the driving parameter display includes at least one of driving speed, navigation system directions, hazard detection and pedestrian detection.
18 . A method for generating see through view of a vehicle comprising: determining a position and orientation of a wearable vision system within the vehicle using a vehicle controller, the vehicle controller operating a simultaneous localization and mapping (SLAM) algorithm based at least in part on sensor information from a vehicle driver monitoring system;
determining at least one opaque portion of the vehicle in a view of a driver based on the position and orientation of the wearable vision system within the vehicle and a three dimensional model of the vehicle;
generating a cropped video feed of an exterior view of the vehicle at the position of the at least one opaque portion of the vehicle by at least disabling the cropped video feed in response to identifying a driving speed of the vehicle exceeding a threshold; and
outputting the cropped video feed to the wearable vision system while the driving speed of the vehicle is less than or equal to the threshold.
19 . The method of claim 18 , further comprising displaying the cropped video feed to the driver using at least one screen of the wearable vision system.
20 . The method of claim 18 , wherein the simultaneous localization and mapping (SLAM) algorithm is performed exclusively on the vehicle controller.