IP Library › Granted Patent US 11,507,084
Granted Patent B2
US 11,507,084 · App. 16/366,120 · Granted Nov 22, 2022

Collaborative 3-D environment map for computer-assisted or autonomous driving vehicles

Inventors: Sridhar G. Sharma (Palo Alto, CA); S M Iftekharul Alam (Hillsboro, OR); Nilesh Ahuja (Cupertino, CA); Avinash Kumar (Santa Clara, CA); Jason Martin (Beaverton, OR); Ignacio J. Alvarez (Portland, OR)
Assignee: Intel Corporation
G05D1/0044G01C21/32G01C21/3841G05D1/0274G06K9/6256G06K9/6277G06N3/0454G06N3/08G06N7/005G06V20/58G06V20/582G08G1/096791H04W4/46B60W2556/65G06N20/10G06T17/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,507,084
App. No.
16/366,120
Granted
Nov 22, 2022
Kind
B2
Abstract

Disclosures herein may be directed to a method, technique, or apparatus directed to a computer-assisted or autonomous driving (CA/AD) vehicle that includes a system controller, disposed in a first CA/AD vehicle, to manage a collaborative three-dimensional (3-D) map of an environment around the first CA/AD vehicle, wherein the system controller is to receive, from another CA/AD vehicle proximate to the first CA/AD vehicle, an indication of at least a portion of another 3-D map of another environment around both the first CA/AD vehicle and the another CA/AD vehicle and incorporate the at least the portion of the 3-D map proximate to the first CA/AD vehicle and the another CA/AD vehicle into the 3-D map of the environment of the first CA/AD vehicle managed by the system controller.

Claims (46)

1. An apparatus for computer-assisted or autonomous driving (CA/AD), comprising:

a system controller, disposed in a first CA/AD vehicle, to manage a three-dimensional (3-D) map of an environment around the first CA/AD vehicle, wherein the system controller is to:

receive, from a second CA/AD vehicle proximate to the first CA/AD vehicle, an indication of at least a portion of another 3-D map of another environment around both the first CA/AD vehicle and the second CA/AD vehicle;

evaluate the received indication of the at least the portion of the another 3-D map to determine whether the portion is to be incorporated into the 3-D map of the environment around the first CA/AD vehicle, wherein to evaluate the received indication, the system controller is to: evaluate a voting weight to be assigned to the received indication of the at least the portion of the another 3-D map, and based upon the evaluation of the voting weight, determine whether the portion of the another 3-D map is to be incorporated into the 3-D map of the environment around the first CA/AD vehicle; and

receive, from sensors within the first CA/AD vehicle, observations of the environment around the first CA/AD vehicle, and based upon the received observations, determine indications of at least a portion of the 3-D map of the environment around the first CA/AD vehicle;

wherein the apparatus further includes a communication subsystem, disposed in the first CA/AD vehicle and coupled with the system controller, to transmit the determined indications of the at least the portion of the 3-D map of the environment around the first CA/AD vehicle to the second CA/AD vehicle.

2. The apparatus of claim 1 , wherein the received indication is of an object, the object includes a classification selected from one of a plurality of classifications and the object is detected by the second CA/AD vehicle when the object is not detectible by the first CA/AD vehicle.

3. The apparatus of claim 1 , wherein:

the received indication is of an object; and

to evaluate the voting weight to be assigned, the system controller is to evaluate a voting weight of the received indication of the object.

4. The apparatus of claim 1 , wherein the system controller is to evaluate the voting weight to be assigned based upon a neural network or a support vector machine (SVM).

5. The apparatus of claim 1 , wherein the voting weight is determined by a detection probability.

6. The apparatus of claim 1 , wherein the indication of the at least the portion of the another 3-D map includes an indication of a visual representation.

7. The apparatus of claim 6 , wherein the indication of the visual representation includes an octree or a subset of an octree.

8. The apparatus of claim 7 , wherein the octree or the subset of the octree includes at least one of a timestamp or a frame of reference.

9. The apparatus of claim 1 , wherein the received indication is of an object and the voting weight is a function of a detection probability of the object.

10. A method to manage a collaborative 3-D map of an environment around a first computer-assisted or autonomous driving (CA/AD) vehicle, comprising:

receiving, from a second CA/AD vehicle proximate to the first CA/AD vehicle, an indication of at least a portion of another 3-D map of another environment around both the first CA/AD vehicle and the second CA/AD vehicle;

evaluating the received indication of the at least the portion of the another 3-D map to determine how much of the portion is to be incorporated into the collaborative 3-D map of the environment around the first CA/AD vehicle, wherein the evaluating the received indication comprises: evaluating a voting weight to be assigned to the received indication of the at least the portion of the another 3-D map, and based upon the evaluating of the voting weight, determining how much of the portion of the another 3-D map is to be incorporated into the collaborative 3-D map of the environment around the first CA/AD vehicle;

receiving, from sensors within the first CA/AD vehicle, observations of the environment around the first CA/AD vehicle;

based upon the received observations, determining indications of at least a portion of the collaborative 3-D map of the environment around the first CA/AD vehicle; and

transmitting, by a communication subsystem, disposed in the first CA/AD vehicle, the determined indications of the at least the portion of the collaborative 3-D map of the environment around the first CA/AD vehicle to the second CA/AD vehicle.

11. The method of claim 10 , wherein the received indication is of an object, the object includes a classification selected from one of a plurality of classifications and the object is detected by the second CA/AD vehicle when the object is not detectible by the first CA/AD vehicle.

12. The method of claim 11 , wherein the plurality of classifications are to at least one of: detect anomalies, combat adversarial attacks, or increase a training set size.

13. The method of claim 10 , wherein the receiving and the transmitting are via CA/AD vehicle to CA/AD vehicle communication.

14. The method of claim 10 , wherein:

the received indication is of an object; and

the evaluating the voting weight to be assigned includes evaluating a voting weight of the received indication of the object.

15. The method of claim 10 , wherein evaluating the voting weight to be assigned is based upon a neural network or a support vector machine (SVM).

16. The method of claim 10 , wherein the received indication is of an object and the voting weight is determined by a confidence placed in a vendor supplying a classifier of the object.

17. One or more non-transitory computer-readable media comprising instructions that cause integrated circuitry (IC) in a first computer-assisted or autonomous driving (CA/AD) vehicle, in response to execution of the instructions by the IC, to:

obtain a 3-D map of an environment around the first CA/AD vehicle based on sensors of the first CA/AD vehicle;

identify an object in the environment around the first CA/AD vehicle based on a neural network of the first CA/AD vehicle;

receive, from a second CA/AD vehicle proximate to the first CA/AD vehicle, another 3-D map of another environment around both the first CA/AD vehicle and the second CA/AD vehicle, the another 3-D map identifies an object in the another environment around both the first CA/AD vehicle and the second CA/AD vehicle based on a neural network of the second CA/AD vehicle; and

incorporate the object identified in the another 3-D map into the 3-D map of the environment around the first CA/AD vehicle, wherein a data set on which the neural network of the first CA/AD vehicle is trained is different than a data set on which the neural network of the second CA/AD vehicle is trained, wherein an accuracy of the neural network of the first CA/AD vehicle in identifying objects is different than an accuracy of the neural network of the second CA/AD vehicle in identifying objects.

18. The one or more non-transitory computer-readable media of claim 17 , wherein decision boundaries of the neural network of the first CA/AD vehicle for identifying objects are different than decision boundaries of the neural network of the second CA/AD vehicle for identifying objects.

19. The one or more non-transitory computer-readable media of claim 17 , wherein a number of layers in the neural network of the first CA/AD vehicle is different than a number of layers in the neural network of the second CA/AD vehicle.

20. The one or more non-transitory computer-readable media of claim 17 , wherein there is a diversity in decision boundaries of the neural network of the first CA/AD vehicle and the neural network of the second CA/AD vehicle for identifying objects and classifications for those objects.

21. One or more non-transitory computer-readable media (NTCRM) comprising instructions, wherein execution of the instructions by a computer in a first computer-assisted or autonomous driving (CA/AD) vehicle is to cause the computer to:

obtain a first three dimensional (3-D) map of an environment around the first CA/AD vehicle based on sensors of the first CA/AD vehicle;

identify an object in the environment around the first CA/AD vehicle based on a neural network of the first CA/AD vehicle;

receive, from a second CA/AD vehicle proximate to the first CA/AD vehicle, a second 3-D map of a second environment around both the first CA/AD vehicle and the second CA/AD vehicle, the second 3-D map identifies an object in the second environment around both the first CA/AD vehicle and the second CA/AD vehicle based on a neural network of the second CA/AD vehicle; and

incorporate the object identified in the second 3-D map into the first 3-D map of the environment around the first CA/AD vehicle, wherein a data set on which the neural network of the first CA/AD vehicle is trained is different than a data set on which the neural network of the second CA/AD vehicle is trained, and there is a diversity in decision boundaries of the neural network of the first CA/AD vehicle and the neural network of the second CA/AD vehicle for identifying objects and classifications for those objects.

22. The one or more NTCRM media of claim 21 , wherein decision boundaries of the neural network of the first CA/AD vehicle for identifying objects are different than decision boundaries of the neural network of the second CA/AD vehicle for identifying objects.

23. The one or more NTCRM media of claim 21 , wherein an accuracy of the neural network of the first CA/AD vehicle in identifying objects is different than an accuracy of the neural network of the second CA/AD vehicle for identifying objects.

24. The one or more NTCRM media of claim 21 , wherein a number of layers in the neural network of the first CA/AD vehicle is different than a number of layers in the neural network of the second CA/AD vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2019
From: SHARMA, SRIDHAR G.; ALAM, S M IFTEKHARUL; AHUJA, NILESH; KUMAR, AVINASH; MARTIN, JASON; ALVAREZ, IGNACIO J.
To: INTEL CORPORATION
Reel/Frame 048715/0279 →
Continuity (1)
Related Publication 20190220003A1 · Jul 18, 2019
Cited By (1)
US 12,454,260