IP Library Granted Patent US 11,435,751
Granted Patent B2
US 11,435,751 · App. 16/236,084 · Granted Sep 6, 2022

Vehicle-based road obstacle identification system

Inventor: Qi Chen (Burlingame, CA)
Assignee: Beijing Voyager Technology Co., Ltd.
G05D1/024G05D1/0088G05D1/0221G05D1/0274G08G1/0112G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,435,751
App. No.
16/236,084
Granted
Sep 6, 2022
Kind
B2
Abstract

A vehicle can include an on-board data processing system that receives sensor data captured by various sensors of the vehicle. As a vehicle travels along a route, the on-board data processing system can process the captured sensor data to identify a potential vehicle stop. The on-board data processing system can then identify geographical coordinates of the location at which the potential vehicle stop occurred, use artificial intelligence to classify a situation of the vehicle at the potential stop, and determine whether the stop was caused by a road obstacle, such as a speed bump, a gutter, an unmarked crosswalk, or any other obstacle not at an intersection. If the stop was caused by the road obstacle, the on-board data processing system can generate virtual stop or yield line data corresponding to the identified geographic coordinates and transmit this data to a server over a network for processing.

Claims (50)

1. A vehicle comprising:

a plurality of sensors, wherein a first sensor in the plurality of sensors is configured to generate velocity data, and wherein a second sensor in the plurality of sensors is configured to generate location data; and

a processor configured with computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to:

detect a vehicle stop at a first time instant using the generated velocity data;

determine a location of the vehicle at the first time instant using the generated location data;

determine, using a deep neural network stored on the vehicle that receives as an input a grid map, a traffic scene surrounding the vehicle at the determined location, wherein the grid map comprises information derived from map data and having a first color being laid over information derived from sensor data having a second color;

determine, based on at least one of the determined traffic scene or the map data, that a cause of the detected vehicle stop is a road obstacle;

generate virtual stop line data in response to determining that the cause of the detected vehicle stop is the road obstacle, wherein the virtual stop line data comprises an indication of a hidden problem at the determined location; and

transmit the virtual stop line data to a server over a network via a communication array.

2. The vehicle of claim 1 , wherein the location of the vehicle comprises geographical coordinates of the vehicle at the first time instant and a lane on a road in which the vehicle was positioned at the first time instant.

3. The vehicle of claim 1 , wherein the computer-executable instructions, when executed, further cause the processor to determine the traffic scene of the vehicle based on an output of the deep neural network.

4. The vehicle of claim 3 , wherein the computer-executable instructions, when executed, further cause the processor to:

obtain the map data and at least one of light detection and ranging (LiDAR) data, radar data, or camera data; and

generate an image in which information derived from the map data is laid over information derived from at least one of the LiDAR data, the radar data, or the camera data to form the grid map.

5. The vehicle of claim 1 , wherein the computer-executable instructions, when executed, further cause the processor to train the deep neural network using a training set of grid maps.

6. The vehicle of claim 1 , wherein the computer-executable instructions, when executed, further cause the processor to:

detect a second vehicle stop at a second time instant before the first time instant; and

determine that a velocity of the vehicle increased by at least a velocity ripple value between the second time instant and the first time instant.

7. The vehicle of claim 1 , wherein the road obstacle comprises at least one of a speed bump, a gutter, an unmarked crosswalk, a pothole, a road flare, a traffic cone, a detour sign, or an uneven surface.

8. The vehicle of claim 1 , wherein the vehicle is at least one of an autonomous vehicle, a vehicle that provides one or more driver-assist features, or a vehicle used to offer location-based services.

9. A method implemented by a vehicle, the method comprising:

detecting a vehicle stop at a first time instant using velocity data measured by the vehicle;

determining a location of the vehicle at the first time instant;

determining, based in part on execution of an artificial intelligence engine running on the vehicle that receives as an input a grid map and that outputs a traffic scene surrounding the vehicle, that a cause of the detected vehicle stop is a road obstacle, wherein the grid map comprises information derived from map data and having a first color being laid over information derived from sensor data having a second color;

generating virtual stop line data in response to determining that the cause of the detected vehicle stop is the road obstacle, wherein the virtual stop line data comprises an indication of a hidden danger at the determined location; and

transmitting the virtual stop line data to a server over a network.

10. The method of claim 9 , further comprising operating in a sensitive mode at the determined location in response to accessing the map data incorporating information derived from the virtual stop line data.

11. The method of claim 9 , wherein determining that a cause of the detected vehicle stop is road obstacle further comprises:

determining the traffic scene surrounding the vehicle based on an output of the artificial intelligence engine; and

determining the cause based on at least one of the determined traffic scene or the map data.

12. The method of claim 11 , wherein generating a grip map further comprises:

obtaining the map data and at least one of light detection and ranging (LiDAR) data, radar data, or camera data; and

generating an image in which information derived from the map data is laid over information derived from at least one of the LiDAR data, the radar data, or the camera data to form the grid map.

13. The method of claim 9 , wherein the artificial intelligence engine is one of a deep neural network or a machine learning model.

14. The method of claim 9 , wherein the road obstacle comprises at least one of a speed bump, a gutter, an unmarked crosswalk, a pothole, a road flare, a traffic cone, a detour sign, or an uneven surface.

15. The method of claim 9 , wherein detecting a vehicle stop further comprises:

detecting a second vehicle stop at a second time instant before the first time instant; and

determining that a velocity of the vehicle increased by at least a velocity ripple value between the second time instant and the first time instant.

16. The method of claim 9 , wherein the vehicle is at least one of an autonomous vehicle, a vehicle that provides one or more driver-assist features, or a vehicle used to offer location-based services.

17. Non-transitory, computer-readable storage media comprising computer executable instructions for identifying a virtual stop line, wherein the computer-executable instructions, when executed by a vehicle, cause the vehicle to:

detect a vehicle stop at a first time instant using velocity data measured by the vehicle;

determine a location of the vehicle at the first time instant;

determine, based in part on execution of an artificial intelligence engine running on the vehicle that receives as an input a grid map and that outputs a traffic scene surrounding the vehicle, that a cause of the detected vehicle stop is a road obstacle, wherein the grid map comprises information derived from map data and having a first color being laid over information derived from sensor data having a second color;

generate virtual stop line data in response to determining that the cause of the detected vehicle stop is the road obstacle, wherein the virtual stop line data comprises an indication of a hidden danger at the determined location; and

transmit the virtual stop line data external to the vehicle.

18. The non-transitory, computer-readable storage media of claim 17 , wherein the computer-executable instructions, when executed, further cause the vehicle to operate in a sensitive mode at the determined location in response to accessing the map data incorporating information derived from the virtual stop line data.

19. The non-transitory, computer-readable storage media of claim 17 , wherein the computer-executable instructions, when executed, further cause the vehicle to:

determine the traffic scene surrounding the vehicle based on an output of the artificial intelligence engine; and

determine the cause based on at least one of the determined traffic scene or the map data.

20. The non-transitory, computer-readable storage media of claim 17 , wherein the artificial intelligence engine is one of a deep neural network or a machine learning model.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2020
From: DIDI RESEARCH AMERICA, LLC
To: VOYAGER (HK) CO., LTD.
Reel/Frame 052181/0607 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2020
From: VOYAGER (HK) CO., LTD.
To: BEIJING VOYAGER TECHNOLOGY CO., LTD.
Reel/Frame 052182/0307 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2019
From: CHEN, QI
To: DIDI RESEARCH AMERICA, LLC
Reel/Frame 048113/0281 →
Continuity (1)
Related Publication 20200209864A1 · Jul 2, 2020