IP Library › Granted Patent US 12,110,037
Granted Patent B2
US 12,110,037 · App. 17/096,495 · Granted Oct 8, 2024

Navigation and mapping based on detected arrow orientation

Inventors: Avigdor Eldar (Modiin, IL); Eli Peker (Jerusalem, IL); Matan Sulami (Tel Aviv, IL)
Assignee: MOBILEYE VISION TECHNOLOGIES LTD.
B60W60/001B60T7/12B60W10/18B60W10/20B60W30/12B60W30/18072B60W30/18109B62D6/001G01C21/30G01C21/3407G01C21/3626G01C21/3691G01C21/3815G01C21/3848G01C21/3885G05D1/0088G05D1/0214G05D1/0221G05D1/0246G05D1/0278G05D1/0287G06V20/56G06V20/584G06V20/588G08G1/0112G08G1/07G08G1/143G08G1/22B60W2300/123B60W2420/403B60W2520/06B60W2520/12B60W2552/05B60W2552/30B60W2552/45B60W2552/53B60W2554/60B60W2555/60B60W2556/40B60W2556/55B60W2556/65G01C21/38G01C21/3807G01C21/3841G07C5/0808
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,110,037
App. No.
17/096,495
Granted
Oct 8, 2024
Kind
B2
Abstract

A system for mapping a lane mark for use in autonomous vehicle navigation is provided. The system includes at least one processor programmed to: receive two or more location identifiers associated with a detected lane mark; associate the detected lane mark with a corresponding road segment; update an autonomous vehicle road navigation model relative to the corresponding road segment based on the two or more location identifiers associated with the detected lane mark; and distribute the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles.

Claims (86)

1. A system for mapping a directional arrow for use in autonomous vehicle navigation, the system comprising:

at least one processor included in a server and programmed to:

receive at least one location identifier associated with a directional arrow detected by a vehicle located remotely from the server, the direction arrow being on a surface of a road, and the at least one location identifier being transmitted to the server from the vehicle located remotely from the server;

receive at least one directional indicator associated with the detected directional arrow, the at least one directional indicator being transmitted to the server from the vehicle located remotely from the server;

associate the detected directional arrow with a corresponding road segment;

update an autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected directional arrow, wherein the autonomous vehicle road navigation model includes at least one target trajectory for a vehicle to follow along the corresponding road segment, the at least one target trajectory being related to the detected directional arrow; and

automatically distribute the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles via one or more networks through one or more wireless communication paths, wherein at least one of the plurality of autonomous vehicles is configured to implement at least one navigational action based on the updated autonomous vehicle road navigation model.

2. The system of claim 1 , wherein the at least one location identifier is determined based on:

acquisition, from a camera associated with a host vehicle, of at least one image representative of an environment of the host vehicle;

analysis of the at least one image to detect the directional arrow on the surface of the road in the environment of the host vehicle; and

analysis of the at least one image to determine a position of the detected directional arrow relative to at least one location associated with the host vehicle.

3. The system of claim 1 , wherein the at least one location identifier includes a location in real world coordinates of a point associated with the detected directional arrow.

4. The system of claim 3 , wherein the point coincides with a location associated with a tip of the directional arrow.

5. The system of claim 3 , wherein the point coincides with a location associated with an interior of the directional arrow.

6. The system of claim 1 , wherein the at least one directional indicator includes locations in real world coordinates of two or more points associated with a head of the detected directional arrow.

7. The system of claim 1 , wherein the at least one directional indicator includes a designation of a point associated with the detected directional arrow as a tip of the directional arrow.

8. The system of claim 1 , wherein the at least one directional indicator includes an ordered pair of points associated with the detected directional arrow.

9. The system of claim 1 , wherein the autonomous vehicle road navigation model also includes at least one target trajectory for a vehicle to follow along the corresponding road segment.

10. The system of claim 1 , wherein the at least one processor is further programmed to receive a first communication from a first host vehicle, wherein the first communication includes the at least one location identifier associated with the detected directional arrow, and receive a second communication from a second host vehicle, wherein the second communication includes at least one additional location identifier associated with the detected directional arrow, and wherein the at least one processor is programmed to refine a determination of at least one position associated with the detected directional arrow based on the at least one location identifier received in the first communication from the first host vehicle and based on the at least one location identifier received in the second communication from the second host vehicle.

11. The system of claim 1 , wherein updating of the autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected direction includes storing at least one indicator of position in real world coordinates and at least one indicator of orientation of the detected directional arrow.

12. The system of claim 1 , wherein the at least one processor is further programmed to forego the update to the autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected directional arrow if a direction of travel of a host vehicle, from which the at least one location identifier and the at least one directional indicator were received, is not consistent with an orientation of the detected directional arrow determined based, at least in part, on the at least one directional indicator associated with the detected directional arrow.

13. A method for mapping a directional arrow for use in autonomous vehicle navigation, the method comprising:

receiving, by a server, at least one location identifier associated with a directional arrow detected by a vehicle located remotely from the server, the direction arrow being on a surface of a road, and the at least one location identifier being transmitted to the server from the vehicle located remotely from the server;

receiving, by the server, at least one directional indicator associated with the detected directional arrow, the at least one directional indicator being transmitted to the server from the vehicle located remotely from the server;

associating, by the server, the detected directional arrow with a corresponding road segment;

updating, by the server, an autonomous vehicle road navigation model relative to the corresponding road segment based on the at least one location identifier and based on the at least one directional indicator associated with the detected directional arrow, wherein the autonomous vehicle road navigation model includes at least one target trajectory for a vehicle to follow along the corresponding road segment, the at least one target trajectory being related to the detected directional arrow; and

automatically distributing, by the server, the updated autonomous vehicle road navigation model to a plurality of autonomous vehicles via one or more networks through one or more wireless communication paths, wherein at least one of the plurality of autonomous vehicles is configured to implement at least one navigational action based on the updated autonomous vehicle road navigation model.

14. The method of claim 13 , wherein the at least one location identifier is determined based on:

acquisition, from a camera associated with a host vehicle, of at least one image representative of an environment of the host vehicle;

analysis of the at least one image to detect the directional arrow on the surface of the road in the environment of the host vehicle; and

analysis of the at least one image to determine a position of the detected directional arrow relative to at least one location associated with the host vehicle.

15. The method of claim 13 , wherein the at least one location identifier includes a location in real world coordinates of a point associated with the detected directional arrow.

16. The method of claim 13 , wherein the at least one directional indicator includes locations in real world coordinates of two or more points associated with a head of the detected directional arrow.

17. The method of claim 13 , wherein the at least one directional indicator includes a designation of a point associated with the detected directional arrow as a tip of the directional arrow.

18. The method of claim 13 , wherein the at least one directional indicator includes an ordered pair of points associated with the detected directional arrow.

19. A system for autonomously navigating a host vehicle along a road segment, the system comprising:

at least one processor included in a host vehicle and programmed to:

receive from an image capture device at least one image representative of an environment of the host vehicle;

detect in the at least one image a representation of an arrow present on a road surface;

identify two or more points of the arrow in the at least one image, the two or more points including at least one of a tip point, an exterior vertex point, or an interior vertex point;

determine a pointing direction of the detected arrow based on analysis of the two or more points of the arrow in the at least one image;

determine a location of the detected arrow by analyzing the at least one image;

transmit, to a server located remotely from the host vehicle, the pointing direction and the location of the detected arrow;

determine an autonomous navigational action for the host vehicle based on the determined pointing direction of the detected arrow; and

cause at least one actuator system of the host vehicle to implement the determined autonomous navigational action.

20. The system of claim 19 , wherein the determined autonomous navigational action includes braking of the host vehicle, and wherein the braking is implemented after a determination that the pointing direction of the detected arrow is opposite to a direction of travel of the host vehicle.

21. The system of claim 19 , wherein the determined autonomous navigational action includes steering of the host vehicle, and wherein the steering is implemented after a determination that the pointing direction of the detected arrow is inconsistent with a planned direction of travel for the host vehicle.

22. The system of claim 19 , wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that reduces a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

23. The system of claim 19 , wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that increases a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

24. The system of claim 19 , wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

25. A method for autonomously navigating a host vehicle along a road segment, the method comprising:

receiving, by the host vehicle, from an image capture device at least one image representative of an environment of the host vehicle;

detecting, by the host vehicle, in the at least one image a representation of an arrow present on a road surface;

determining, by the host vehicle, a pointing direction of the detected arrow based on analysis of the at least one image;

identifying, by the host vehicle, two or more points of the arrow in the at least one image, the two or more points including at least one of a tip point, an exterior vertex point, or an interior vertex point;

determining, by the host vehicle, a location of the detected arrow by analyzing the two or more points of the arrow in the at least one image;

transmitting, by the host vehicle to a server located remotely from the host vehicle, the pointing direction and the location of the detected arrow;

determining, by the host vehicle, an autonomous navigational action for the host vehicle based on the determined pointing direction of the detected arrow; and

causing, by the host vehicle, at least one actuator system of the host vehicle to implement the determined autonomous navigational action.

26. The method of claim 25 , wherein the determined autonomous navigational action includes braking of the host vehicle, and wherein the braking is implemented after a determination that the pointing direction of the detected arrow is opposite to a direction of travel of the host vehicle.

27. The method of claim 25 , wherein the determined autonomous navigational action includes steering of the host vehicle, and wherein the steering is implemented after a determination that the pointing direction of the detected arrow is inconsistent with a planned direction of travel for the host vehicle.

28. The method of claim 25 , wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that reduces a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

29. The method of claim 25 , wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that increases a difference between a heading direction of the host vehicle and the pointing direction of the detected arrow.

30. The method of claim 25 , wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

31. A system for autonomously navigating a host vehicle along a road segment, the system comprising:

at least one processor included in the host vehicle and programmed to:

receive from a server-based system an autonomous vehicle road navigation model, wherein the autonomous vehicle road navigation model includes a mapped directional arrow associated with the road segment, and wherein the mapped directional arrow includes at least one of an indication of an orientation of the directional arrow or a location of the directional arrow;

determine an autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow; and

cause at least one actuator system of the host vehicle to implement the determined autonomous navigational action,

wherein the at least one processor is further programmed to receive from an image capture device at least one image representative of an environment of the host vehicle, and

wherein the at least one processor is further programmed to determine the autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow included in the autonomous vehicle road navigation model received from the server-based system, even where a representation of the directional arrow does not appear in the at least one image representative of the environment of the host vehicle.

32. The system of claim 31 , wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that reduces a difference between a heading direction of the host vehicle and a pointing direction of the directional arrow.

33. The system of claim 31 , wherein the determined autonomous navigational action includes steering of the host vehicle in a direction that increases a difference between a heading direction of the host vehicle and a pointing direction of the directional arrow.

34. The system of claim 31 , wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

35. The system of claim 31 , wherein the autonomous vehicle road navigation model includes a target trajectory for the host vehicle and wherein the at least one processor is further programmed to receive from an image capture device at least one image representative of an environment of the host vehicle and determine a longitudinal position of the host vehicle along the target trajectory based on a location of the mapped directional arrow and based on detection of the directional arrow in the at least one image.

36. The system of claim 35 , wherein the target trajectory is represented as a three-dimensional spline.

37. The system of claim 31 , wherein the mapped directional arrow comprises data indicating the location and direction of an arrow present on a roadway associated with the road segment.

38. A method for autonomously navigating a host vehicle along a road segment, the method comprising:

receiving from a server-based system an autonomous vehicle road navigation model, wherein the autonomous vehicle road navigation model includes a mapped directional arrow associated with the road segment, and wherein the mapped directional arrow includes at least one of an indication of an orientation of the directional arrow or a location of the directional arrow;

determining an autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow; and

causing at least one actuator system of the host vehicle to implement the determined autonomous navigational action,

wherein the method further comprises:

receiving from an image capture device at least one image representative of an environment of the host vehicle; and

determining the autonomous navigational action for the host vehicle based on at least one of the orientation of the directional arrow or the location of the directional arrow included in the autonomous vehicle road navigation model received from the server-based system, even where a representation of the directional arrow does not appear in the at least one image representative of the environment of the host vehicle.

39. The method of claim 38 , wherein the determined autonomous navigational action includes maintaining a current heading direction of the host vehicle.

40. The method of claim 38 , wherein the autonomous vehicle road navigation model includes a target trajectory for the host vehicle, and the method further comprises receiving from an image capture device at least one image representative of an environment of the host vehicle and determining a longitudinal position of the host vehicle along the target trajectory based on a location of the mapped directional arrow and based on detection of the directional arrow in the at least one image.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2020
From: ELDAR, AVIGDOR
To: INTEL CORPORATION
Reel/Frame 054352/0190 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2020
From: INTEL CORPORATION
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 054352/0336 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2020
From: PEKER, ELI; SULAMI, MATAN
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 054352/0727 →
Continuity (7)
Continuation PCTUS2019032429 · May 15, 2019
Provisional Application 62805646 · Feb 14, 2019
Provisional Application 62800845 · Feb 4, 2019
Provisional Application 62797639 · Jan 28, 2019
Provisional Application 62771335 · Nov 26, 2018
Provisional Application 62671779 · May 15, 2018
Related Publication 20210064057A1 · Mar 4, 2021