Detecting hazards in anticipation of opening vehicle doors
The present invention extends to methods, systems, and computer program products for detecting hazards in anticipation of opening vehicle doors. Vehicle sensors (e.g., rear viewing cameras) can be used to detect and classify traffic, for example, as pedestrians, bicyclists, skateboarders, roller skaters, wheel chair, etc., approaching on the side of a vehicle. When there is a possibility of a vehicle occupant opening a door into approaching traffic, a warning can be issued in the vehicle cabin to alert vehicle occupants of the approaching traffic. In one aspect, a vehicle prevents a door from opening if opening the door would likely cause an accident.
1. A method for use at a vehicle, the method comprising:
filtering sensor data from one or more sensors mounted to the vehicle at a heterogeneous computing platform at the vehicle to determine a speed and direction of traffic approaching the vehicle;
determining that the approaching traffic is to pass through a defined space to be occupied by a vehicle door when the vehicle door is open based on the filtered sensor data;
receiving a traffic classification for the approaching traffic from a neural network;
determining that the vehicle door poses a hazard to the approaching traffic based at least on the traffic classification, the approaching traffic passing through the defined space, and a configuration of the vehicle; and
preventing the vehicle door from opening until the approaching traffic passes the defined space.
2. The method of claim 1 , wherein filtering sensor data from one or more sensors mounted to the vehicle comprises filtering sensor data from one or more of: a camera, a LIDAR sensor, or an ultrasonic sensor mounted to the vehicle.
3. The method of claim 1 , wherein receiving a traffic classification for the approaching traffic comprises receiving an indication that the approaching traffic is classified as one of: a walking pedestrian, a bicyclist, a skate boarder, a wheel chair, or a roller skater.
4. The method of claim 1 , further comprising outputting a signal in the cabin of the vehicle to indicate the hazard.
5. The method of claim 4 , wherein outputting a signal in the cabin of the vehicle comprises outputting one of: an audible signal, a visual signal, or a haptic signal.
6. The method of claim 1 , further comprising preventing the vehicle door from opening until the approaching traffic passes the vehicle.
7. A method for use at a vehicle, the method for detecting a hazard in anticipation of opening a door of the vehicle, the method comprising:
determining that the path of approaching traffic is likely to cause the approaching traffic to travel through a defined space that is to be occupied by the door when the door is open, including:
using one or more sensors mounted to the vehicle to monitor an area in proximity to the vehicle for the approaching traffic; and
filtering data from the one or more sensors on a heterogeneous computing platform at the vehicle to determine that the approaching traffic has a speed and direction indicative of the approaching traffic passing through the defined space;
providing the filtered data as input to a neural network;
receiving a traffic classification for the approaching traffic from the neural network;
determining that the door poses a hazard to the approaching traffic based at least on the traffic classification, the approaching traffic passing through the defined space, and a configuration of the vehicle; and
preventing the door from opening until the approaching traffic passes through the defined space.
8. The method of claim 7 , wherein the vehicle has a plurality of doors that open to the side of the vehicle.
9. The method of claim 7 , wherein using one or more sensors mounted to the vehicle to monitor an area in proximity to the vehicle for the approaching traffic comprises using a plurality of sensors mounted to the vehicle to monitor the area, the plurality of sensors including: at least one camera, at least one LIDAR sensor, and at least one sound based object detector.
10. The method of claim 7 , wherein filtering data from the one or more sensors on a heterogeneous computing platform comprises filtering data from the plurality of sensors on the heterogeneous computing platform that includes: at least one Central Processing Unit (CPU), at least one Graphical Processing Unit (GPU), and at least one Field Programmable Gate Array (FPGA) in the vehicle.
11. The method of claim 7 , wherein filtering data from the one or more sensors comprises determining the optical flow of the approaching traffic.
12. The method of claim 7 , wherein receiving a traffic classification for the approaching traffic from the neural network comprises receiving an indication that the approaching traffic is classified as one of: a walking pedestrian, a bicyclist, a skate boarder, a wheel chair, or a roller skater.
13. The method of claim 7 , wherein determining that the door poses a hazard to the approaching traffic comprises detecting that a door handle in the vehicle cabin has been actuated.
14. The method of claim 7 , further comprising outputting a signal in the cabin of the vehicle to indicate a possible collision between the approaching traffic and the opening door.
15. The method of claim 14 , wherein outputting a signal in the cabin of the vehicle comprises outputting one of: an audible signal, a visual signal, or a haptic signal.
16. A vehicle, the vehicle comprising:
one or more externally mounted sensors for monitoring an area in proximity to the vehicle;
one or more processors;
system memory coupled to the one or more processors, the system memory storing instructions that are executable by the one or more processors;
the one or more processors configured to execute the instructions stored in the system memory to:
determine that the path of approaching traffic is likely to cause the approaching traffic to travel through a defined space that is to be occupied by a door of the vehicle when the door is open, including:
use the one or more sensors to monitor an area in proximity to the vehicle for the approaching traffic; and
filter data from the one or more sensors on a heterogeneous computing platform at the vehicle to determine that the approaching traffic has a speed and direction indicative of the approaching traffic passing through the defined space;
provide the filtered data as input to a neural network;
receive a traffic classification for the approaching traffic from the neural network;
determine that the door poses a hazard to the approaching traffic based at least on the traffic classification, the approaching traffic passing through the defined space, and a configuration of the vehicle; and
prevent the door from opening until the approaching traffic passes the door.
17. The vehicle of claim 16 , wherein the one or more externally mounted sensors include one or more of: a camera, a LIDAR sensor, a RADAR sensor, and an ultrasonic sensor.
18. The vehicle of claim 16 , further comprising the one or more processors configured to execute the instructions to output a signal in the cabin of the vehicle indicating the possibility of a collision between the approaching traffic and the door.
19. The method of claim 1 , wherein filtering sensor data from one or more sensors comprises determining the optical flow of the approaching traffic; and
wherein determining that the approaching traffic is to pass through a defined space to be occupied by a vehicle door comprising determining that the approaching traffic is to pass through the defined space based on the optical flow of the approaching traffic.
20. The method of claim 1 , wherein determining that the vehicle door poses a hazard to the approaching traffic comprises detecting that a door handle for the vehicle door has been actuated.