IP Library Granted Patent US 11,036,986
Granted Patent B2
US 11,036,986 · App. 16/271,526 · Granted Jun 15, 2021

Systems and methods for aligning map data

Inventors: Patrick Dubroy (Munich, DE); Wolfgang Hess (Munich, DE); Clemens Marschner (Munich, DE); Emil Praun (Union City, CA); Holger Rapp (Munich, DE); Thomas Schiwietz (Munich, DE); Vinay Shet (Fremont, CA)
Assignee: Lyft, Inc.
G06K9/00671G06F16/29G06K9/00791G06T7/70G06T17/05G06T2207/30256G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,036,986
App. No.
16/271,526
Granted
Jun 15, 2021
Kind
B2
Abstract

Systems, methods, and non-transitory computer-readable media can receive a geometric map and a semantic map associated with a geographic area, the semantic map comprising semantic data associated with vehicle navigation. A first semantic position estimate associated with a first piece of semantic data contained in the semantic map is generated based on semantic data location information associated with the first piece of semantic data. A final position for the first semantic position estimate is received. One or more three-dimensional semantic labels are applied to the geometric map based on the final position of the first semantic position estimate. A warped semantic map is generated. Generating the warped semantic map comprises warping the semantic map based on the one or more three-dimensional semantic labels.

Claims (46)

1. A computer-implemented method comprising:

receiving, by a computing system, a geometric map and a semantic map associated with a geographic area, the semantic map including semantic data associated with vehicle navigation;

generating, by the computing system, a semantic position estimate based on a portion of the semantic data associated with an object, wherein the portion of the semantic data is included in the semantic map;

receiving, by the computing system, an updated position of the semantic position estimate based on aligning the semantic position estimate with one or more corresponding physical features associated with the object in the geometric map;

applying, by the computing system, one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate; and

generating, by the computing system, a warped semantic map, wherein the generating the warped semantic map is based on the one or more three-dimensional semantic labels.

2. The computer-implemented method of claim 1 , further comprising generating one or more two-dimensional semantic labels that indicate a position of the portion of the semantic data in an image associated with the geometric map based on the updated position of the semantic position estimate.

3. The computer-implemented method of claim 1 , wherein the semantic position estimate is applied to a first image of a set of images associated with the geometric map, and subsequently, used to estimates a position of the portion of the semantic data within the first image.

4. The computer-implemented method of claim 3 , wherein the receiving the updated position of the semantic position estimate comprises:

receiving an updated two-dimensional position of the semantic position estimate of the first image.

5. The computer-implemented method of claim 4 , wherein the receiving the updated position of the semantic position estimate comprises:

receiving a user input adjusting a position of the semantic position estimate of the first image.

6. The computer-implemented method of claim 4 , further comprising:

applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate, wherein the applying the one or more three-dimensional semantic labels comprises determining a region where a view ray associated with the first image intersects a surface in the geometric map, wherein the one or more three-dimensional semantic labels are applied to the geometric map based on the region where the view ray intersects the surface.

7. The computer-implemented method of claim 4 , wherein the applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate comprises translating the updated two-dimensional position of the semantic position estimate of the first image into a three-dimensional position within the geometric map.

8. The computer-implemented method of claim 7 , wherein the updated two-dimensional position of the semantic position estimate within the first image is translated into a three-dimensional position within the geometric map based on image location information associated with the first image and camera information associated with the first image, wherein the camera information comprises camera position, orientation, and direction information for a camera when the first image was captured.

9. The computer-implemented method of claim 1 , wherein the generating the warped semantic map is performed as an optimization problem that warps the semantic map based on a set of constraints.

10. The computer-implemented method of claim 1 , wherein the semantic data includes at least one of a position of a lane marker, an orientation of the lane marker, a position of a boundary, an orientations of the boundary, a lane direction, a speed limit, or a position of a road feature.

11. A system comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, cause the system to perform:

receiving a geometric map and a semantic map associated with a geographic area, the semantic map including semantic data associated with vehicle navigation;

generating a semantic position estimate based on a portion of the semantic data associated with an object, wherein the portion of the asemantic data is included in the semantic map;

receiving an updated position of the semantic position estimate based on aligning the semantic position estimate with one or more corresponding physical features associated with the object in the geometric map;

applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate; and

generating a warped semantic map, wherein the generating the warped semantic map is based on the one or more three-dimensional semantic labels.

12. The system of claim 11 , wherein the instructions, when executed by the at least one processor, further cause the system to perform:

generating one or more two-dimensional semantic labels that indicate a position of the portion of the semantic data in an image associated with the geometric map based on the updated position of the semantic position estimate.

13. The system of claim 11 , wherein the semantic position estimate is applied to a first image of a set of images associated with the geometric map, and subsequently, used to estimates a position of the portion of the semantic data within the first image.

14. The system of claim 13 , wherein the receiving the updated position of the semantic position estimate comprises:

receiving a final two-dimensional position of the semantic position estimate of the first image.

15. The system of claim 14 , wherein the receiving the updated position of the semantic position estimate comprises:

receiving a user input adjusting a position of the semantic position estimate of the first image.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform:

receiving a geometric map and a semantic map associated with a geographic area, the semantic map including semantic data associated with vehicle navigation;

generating a semantic position estimate based on a portion of the semantic data associated with an object, wherein the portion of the semantic data is included in the semantic map;

receiving an updated position of the semantic position estimate based on aligning the semantic position estimate with one or more corresponding physical features associated with the object in the geometric map;

applying one or more three-dimensional semantic labels to the geometric map based on the updated position of the semantic position estimate; and

generating a warped semantic map, wherein the generating the warped semantic map is based on the one or more three-dimensional semantic labels.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions, when executed by the at least one processor of the computing system, further cause the computing system to perform:

generating one or more two-dimensional semantic labels that indicate a position of the portion of the semantic data in an image associated with the geometric map based on the updated position of the semantic position estimate.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the semantic position estimate is applied to a first image of a set of images associated with the geometric map, and subsequently, used to estimates a position of the portion of the semantic data within the first image.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the receiving the updated position of the semantic position estimate comprises:

receiving an updated two-dimensional position of the semantic position estimate of the first image.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the receiving the updated position of the semantic position estimate comprises:

receiving a user input adjusting a position of the semantic position estimate within the first image.

Assignments (4)
CHANGE OF NAME Recorded Jun 22, 2023
From: WOVEN PLANET NORTH AMERICA, INC.
To: WOVEN BY TOYOTA, U.S., INC.
Reel/Frame 064065/0601 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2021
From: LYFT, INC.; BLUE VISION LABS UK LIMITED
To: WOVEN PLANET NORTH AMERICA, INC.
Reel/Frame 056927/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2021
From: LYFT, INC.; MAGNA AUTONOMOUS SYSTEMS, LLC
To: LYFT, INC.; MAGNA AUTONOMOUS SYSTEMS, LLC
Reel/Frame 057434/0623 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2019
From: DUBROY, PATRICK; HESS, WOLFGANG; MARSCHNER, CLEMENS; PRAUN, EMIL; RAPP, HOLGER; SCHIWIETZ, THOMAS; SHET, VINAY
To: LYFT, INC.
Reel/Frame 048309/0894 →
Continuity (1)
Related Publication 20200257900A1 · Aug 13, 2020