IP Library › Granted Patent US 11,288,520
Granted Patent B2
US 11,288,520 · App. 16/912,033 · Granted Mar 29, 2022

Systems and methods to aggregate and distribute dynamic information of crowdsourcing vehicles for edge-assisted live map service

Inventors: Qiang Liu (Charlotte, NC); Baekgyu Kim (Mountain View, CA); Rui Guo (Mountain View, CA)
Assignee: Toyota Motor Engineering & Manufacturing N.A. Inc.
G06K9/00791G06K9/46H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,288,520
App. No.
16/912,033
Granted
Mar 29, 2022
Kind
B2
Abstract

A method includes receiving sensor data, with one or more sensor of a vehicle, wherein the sensor data is associated with an environment of the vehicle, detecting an object based on the sensor data, determining pixel coordinates of the object based on the sensor data, converting the pixel coordinates of the object to world coordinates of the object, extracting a set of features associated with the object, and transmitting the set of features and the world coordinates to a remote computing device.

Claims (50)

1. A method comprising:

receiving, from a remote computing device, a first data set associated with a first set of one or more detected objects;

storing the first data set in a local database;

receiving sensor data, with one or more sensors of a vehicle, wherein the sensor data is associated with an environment of the vehicle;

detecting an object based on the sensor data;

determining pixel coordinates of the object based on the sensor data;

converting the pixel coordinates of the object to world coordinates of the object;

extracting a set of features associated with the object;

determining whether the object is among the first set of one or more detected objects based on the set of features associated with the object and the world coordinates of the object; and

upon determination that the object is not among the first set of one or more detected objects, transmitting the set of features and the world coordinates of the object from the vehicle to the remote computing device.

2. The method of claim 1 , further comprising:

determining whether the object has previously been detected based on the set of features associated with the object and the world coordinates of the object; and

upon determination that the object has not previously been detected, transmitting the set of features and the world coordinates of the object to the remote computing device.

3. The method of claim 2 , further comprising:

determining whether the object has previously been detected by comparing the set of features associated with the object to one or more sets of features associated with one or more previously detected objects and by comparing the world coordinates of the object to world coordinates of the one or more previously detected objects.

4. The method of claim 3 , further comprising:

determining a mean square error between the set of features associated with the object and the one or more sets of features associated with each of the one or more previously detected objects;

determining a geo-distance error between the world coordinates of the object and the world coordinates of each of the one or more previously detected objects; and

determining whether the object has previously been detected based on the mean square error and the geo-distance error for the object and each of the one or more previously detected objects.

5. The method of claim 1 , further comprising:

converting the pixel coordinates of the object to camera coordinates of the object; and

converting the camera coordinates of the object to the world coordinates of the object.

6. The method of claim 5 , further comprising:

receiving location data of the vehicle; and

relocalizing an absolute location of the sensors of the vehicle based on the location data.

7. The method of claim 1 , wherein the set of features associated with the object comprises a feature vector determined using an autoencoder.

8. A vehicle system comprising:

one or more sensors to receive sensor data associated with an environment of a vehicle; and

a controller programmed to:

receive, from a remote computing device, a first data set associated with a first set of one or more detected objects;

store the first data set in a local database;

detect an object based on the sensor data;

determine pixel coordinates of the object based on the sensor data;

convert the pixel coordinates of the object to world coordinates of the object;

extract a set of features associated with the object;

determine whether the object is among the first set of one or more detected objects based on the set of features associated with the object and the world coordinates of the object; and

upon determination that the object is not among the first set of one or more detected objects, transmit the set of features and the world coordinates of the object to a remote computing device.

9. The vehicle system of claim 8 , wherein the controller is programmed to:

determine whether the object has previously been detected based on the set of features associated with the object and the world coordinates of the object; and

upon determination that the object has not previously been detected, transmit the set of features and the world coordinates of the object to the remote computing device.

10. The vehicle system of claim 8 , wherein the controller is programmed to:

convert the pixel coordinates of the object to camera coordinates of the object; and

convert the camera coordinates of the object to the world coordinates of the object.

11. The vehicle system of claim 8 , wherein the set of features associated with the object comprises a feature vector determined using an autoencoder.

12. The vehicle system of claim 9 , wherein the controller is programmed to:

determine whether the object has previously been detected by comparing the set of features associated with the object to one or more sets of features associated with one or more previously detected objects and by comparing the world coordinates of the object to world coordinates of the one or more previously detected objects.

13. The vehicle system of claim 12 , wherein the controller is programmed to:

determine a mean square error between the set of features associated with the object and the one or more sets of features associated with each of the one or more previously detected objects;

determine a geo-distance error between the world coordinates of the object and the world coordinates of each of the one or more previously detected objects; and

determine whether the object has previously been detected based on the mean square error and the geo-distance error for the object and each of the one or more previously detected objects.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2022
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 059743/0825 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 29, 2020
From: LIU, QIANG; KIM, BAEKGYU; GUO, RUI
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 053066/0941 →
Continuity (1)
Related Publication 20210406558A1 · Dec 30, 2021