IP Library Granted Patent US 12,353,216
Granted Patent B2
US 12,353,216 · App. 18/670,554 · Granted Jul 8, 2025

Removable automotive LIDAR data collection pod

Inventors: Nathaniel Gist, IV (Pittsburgh, PA); Christopher Williams (Pittsburgh, PA)
Assignee: Aurora Operations, Inc.
G05D1/0231B60R11/04G05D1/027G05D1/0276G05D1/0278G05D1/245G05D1/247G05D1/248G05D1/249G06F13/4282B60R2011/004B60R2011/0068B60R2011/007G06F2213/0002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,353,216
App. No.
18/670,554
Granted
Jul 8, 2025
Kind
B2
Abstract

A vehicle agnostic removable pod can be mounted on a vehicle using one or more legs of a pod mount. The removable pod can collect and time stamp a variety of environmental data as well as vehicle data. For example, environmental data can be collected using a sensor suite which can include an IMU, 3D positioning sensor, one or more cameras, and/or a LIDAR unit. As another example, vehicle data can be collected via a CAN bus attached to the vehicle. Environmental data and/or vehicle data can be time stamped and transmitted to a remote server for further processing by a computing device.

Claims (42)

1. A method of collecting diverse driving data, the method comprising:

for each of a plurality of removable pods each respectively attached to a unique vehicle in an environment:

collecting respective environmental data of the environment using one or more sensors of a respective removable pod,

recording the respective environmental data collected using the respective removable pod and corresponding vehicle data that corresponds to the respective environmental data,

determining time stamps for each instance of the respective environmental data and time stamps for each instance of the corresponding vehicle data using a respective computing device connected to the respective removable pod,

generating a respective aggregation of the time stamped instances of the respective environmental data and the time stamped instances of the corresponding vehicle data using the respective computing device, and

transmitting, to a remote server, the respective aggregation of the time stamped instances of the respective environmental data and the time stamped instances of the corresponding vehicle data;

aggregating, at the remote server, the time stamped instances of vehicle data and environmental data from the plurality of removable pods,

wherein each of the unique vehicles is attached to at least one removable pod;

generating training data for a machine learning model based on the time stamped instances of vehicle data and environmental data from the plurality of removable pods; and

training, based on the training data, the machine learning model, to generate control signals for an autonomous vehicle.

2. The method of claim 1 , wherein recording the respective environmental data and the corresponding vehicle data for the respective removable pod is performed using the respective computing device that is connected to the respective removable pod.

3. The method of claim 1 , wherein recording the respective environmental data and the corresponding vehicle data for the respective removable pod occurs on a separate storage device operatively connected to a vehicle cabin removable pod hardware of the respective removable pod.

4. The method of claim 1 , wherein the unique vehicles each being a non-autonomous vehicle that are collecting data in the environment with the autonomous vehicle or an additional autonomous vehicle.

5. The method of claim 4 , further comprising:

automatically labeling data collected from the autonomous vehicle or the additional autonomous vehicle using data collected from at least one removable pod, out of the plurality of removable pods, at a known location and time.

6. The method of claim 1 , wherein determining the time stamps for each instance of the respective environmental data and the time stamps for each instance of the corresponding vehicle data comprises:

determining computing device time stamps for at least one instance of the environmental or vehicle data, and

determining the time stamps for the at least one instance of the environmental or vehicle data based on having the computing device time stamps synched with time stamps from one or more sensors, or a printed circuit board (PCB), of the respective removable pod.

7. The method of claim 1 , wherein the respective aggregation of the time stamped instances of environmental data and the time stamped instances of corresponding vehicle data for the respective removable pod is generated with an external data source.

8. The method of claim 1 , wherein the unique vehicles include at least two vehicles of different vehicle types.

9. The method of claim 1 , wherein the unique vehicles include vehicles having variations and within a vehicle type.

10. The method of claim 1 , wherein the plurality of removable pods are vehicle-agnostic.

11. The method of claim 1 , wherein more than one removable pod is attached to a vehicle, of the unique vehicles.

12. The method of claim 1 , wherein the plurality of removable pods are each mounted at a valid position of a corresponding vehicle to allow sensors of the plurality of removable pods to collect data from a known predefined perspective.

13. The method of claim 1 , wherein the unique vehicles include the autonomous vehicle or an additional autonomous vehicle.

14. A method of collecting diverse driving data, the method comprising:

receiving, at a remote server and for each of a plurality of removable pods that are each respectively attached to a unique vehicle in an environment, a respective aggregation of time stamped instances of environmental data and vehicle data,

wherein each of the vehicles is attached to at least one removable pod;

aggregating, at the remote server, the time stamped instances of vehicle data and environmental data from the plurality of removable pods,

wherein prior to receiving the aggregations for the plurality of removable pods, and for a respective removable pod of the plurality of removable pods,

respective environmental data of the environment is collected using one or more sensors of the respective removable pod,

the respective environmental data and corresponding vehicle data is recorded,

the time stamps for each instance of the respective environmental data and time stamps for each instance of the corresponding vehicle data are determined using a respective computing device connected to the respective removable pod, and

a respective aggregation of the time stamped instances of respective environmental data and the time stamped instances of corresponding vehicle data is generated using the respective computing device;

generating training data for a machine learning model based on the time stamped instances of vehicle data and environmental data from the plurality of removable pods; and

training, based on the training data, the machine learning model, to generate control signals for an autonomous vehicle.

15. The method of claim 14 , wherein the respective environmental data and corresponding vehicle data for the respective removable pod is recorded using the respective computing device that is connected to the respective removable pod.

16. The method of claim 14 , wherein the respective environmental data and the corresponding vehicle data for the respective removable pod is recorded on a separate storage device operatively connected to a vehicle cabin removable pod hardware of the respective removable pod.

17. The method of claim 14 , wherein the multiple vehicles each being a non-autonomous vehicle that are collecting data in the environment with the autonomous vehicle or an additional autonomous vehicle.

18. The method of claim 17 , further comprising:

automatically labeling data collected from the autonomous vehicle or the additional autonomous vehicle using data collected from at least one removable pod, out of the plurality of removable pods, at a known location and time.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2024
From: GIST IV, NATHANIEL; WILLIAMS, CHRISTOPHER
To: AURORA INNOVATION, INC.
Reel/Frame 067888/0213 →
MERGER AND CHANGE OF NAME Recorded Jul 1, 2024
From: AVIAN U MERGER SUB CORP.; AURORA INNOVATION, INC.
To: AURORA INNOVATION OPCO, INC.
Reel/Frame 067888/0218 →
CHANGE OF NAME Recorded Jul 1, 2024
From: AURORA INNOVATION OPCO, INC.
To: AURORA OPERATIONS, INC.
Reel/Frame 068104/0903 →
Continuity (5)
Continuation 18195341 · May 9, 2023
Continuation 17516240 · Nov 1, 2021
Continuation 16271355 · Feb 8, 2019
Provisional Application 62755167 · Nov 2, 2018
Related Publication 20240310837A1 · Sep 19, 2024
References Cited (101)
US 6175799B1 · Tsutsumi et al. · 2001 [cited by applicant]
US 6322275B1 · Schmidt · 2001 [cited by applicant]
US 8849494B1 · Herbach et al. · 2014 [cited by applicant]
US 8866906B1 · Abad · 2014 [cited by applicant]
US 9020680B2 · Solyom et al. · 2015 [cited by applicant]
US 9606539B1 · Kentley et al. · 2017 [cited by applicant]
US 9734455B2 · Levinson et al. · 2017 [cited by applicant]
US 9940834B1 · Konrardy et al. · 2018 [cited by applicant]
US 10007269B1 · Gray · 2018 [cited by applicant]
US 10037613B1 · Becker et al. · 2018 [cited by applicant]
US 10514462B2 · Englard et al. · 2019 [cited by applicant]
US 10676085B2 · Smith et al. · 2020 [cited by applicant]
US 11163312B2 · Gist, IV · 2021 [cited by applicant]
US 11681298B2 · Gist, IV et al. · 2023 [cited by applicant]
US 20070219720A1 · Trepagnier et al. · 2007 [cited by applicant]
US 20100052945A1 · Breed · 2010 [cited by applicant]
US 20100180709A1 · Choi · 2010 [cited by applicant]
US 20120153087A1 · Collette et al. · 2012 [cited by applicant]
US 20130190982A1 · Nakano et al. · 2013 [cited by applicant]
US 20130218518A1 · Dimitri · 2013 [cited by examiner]
US 20140324266A1 · Zhu et al. · 2014 [cited by applicant]
US 20140347440A1 · Hatcher · 2014 [cited by applicant]
US 20140350768A1 · Filippov et al. · 2014 [cited by applicant]
US 20150112550A1 · Cho et al. · 2015 [cited by applicant]
US 20160299228A1 · Maleki et al. · 2016 [cited by applicant]
US 20170016870A1 · McPeek · 2017 [cited by applicant]
US 20170076227A1 · Elgie et al. · 2017 [cited by applicant]
US 20170270361A1 · Puttagunta et al. · 2017 [cited by applicant]
US 20180031678A1 · Singer et al. · 2018 [cited by applicant]
US 20180053102A1 · Martinson et al. · 2018 [cited by applicant]
US 20180060725A1 · Groh et al. · 2018 [cited by applicant]
US 20180113210A1 · Izadian · 2018 [cited by applicant]
US 20180136644A1 · Levinson et al. · 2018 [cited by applicant]
US 20180188032A1 · Ramanandan et al. · 2018 [cited by applicant]
US 20180188037A1 · Wheeler et al. · 2018 [cited by applicant]
US 20180188736A1 · Jian et al. · 2018 [cited by applicant]
US 20180196127A1 · Harada · 2018 [cited by applicant]
US 20180307944A1 · Li et al. · 2018 [cited by applicant]
US 20190004525A1 · Bills et al. · 2019 [cited by applicant]
US 20190012907A1 · Itahara · 2019 [cited by applicant]
US 20190147610A1 · Frossard et al. · 2019 [cited by applicant]
US 20190164049A1 · Bai et al. · 2019 [cited by applicant]
US 20190180451A1 · Kellner · 2019 [cited by applicant]
US 20190258251A1 · Ditty et al. · 2019 [cited by applicant]
US 20190354782A1 · Kee et al. · 2019 [cited by applicant]
US 20200074862A1 · Johnston et al. · 2020 [cited by applicant]
US 20200118359A1 · Sato · 2020 [cited by applicant]
US 20200142408A1 · Valois et al. · 2020 [cited by applicant]
US 20200142409A1 · Valois et al. · 2020 [cited by applicant]
US 20200142422A1 · Valois et al. · 2020 [cited by applicant]
US 20200142426A1 · Gist, IV · 2020 [cited by applicant]
US 20200209867A1 · Valois et al. · 2020 [cited by applicant]
US 20200210777A1 · Valois et al. · 2020 [cited by applicant]
US 20200391736A1 · Smith et al. · 2020 [cited by applicant]
US 20220050471A1 · Gist, IV et al. · 2022 [cited by applicant]
US 20230350417A1 · Gist, IV et al. · 2023 [cited by applicant]
CN 205395966 · 2016 [cited by applicant]
CN 113287039 · 2021 [cited by applicant]
EP 3874294 · 2021 [cited by applicant]
KR 20160000573 · 2016 [cited by applicant]
WO 2014168851 · 2014 [cited by applicant]
WO 2018102363 · 2018 [cited by applicant]
WO 2020091832 · 2020 [cited by applicant]
European Patent Office; Communication Pursuant to Article 94(3) issued for Application No. 19722303.5, 6 pages, dated Feb. 8, 2023. [cited by applicant]
China National Intellectual Property Administration; Decision of Rejection issued for Application No. 201980071453.4, 11 pages, dated Jun. 21, 2023. [cited by applicant]
Canadian Intellectual Property Office, Examination with Search Report issued in App. No. 3,118,402, 3 pages, dated Jun. 20, 2022. [cited by applicant]
Japanese Patent Office, Decision of Grant issued in App. No. 2021-523509, 5 pages, dated Aug. 1, 2022. [cited by applicant]
Korean Intellectual Property Office, Notice of Grounds for Rejection issued in App. No. 10-2021-7012268, 11 pages, dated Jul. 18, 2022. [cited by applicant]
Chinese Patent Office, Second Office Action issued in App. No. 201980071453.4, 6 pages, dated Aug. 24, 2022. [cited by applicant]
Chinese Patent Office, First Office Action issued in Application No. 201980071453.4, 26 pages, dated Feb. 21, 2022. [cited by applicant]
Japanese Patent Office, Notice of Reasons for Rejection issued in Application No. 2021-523509, 6 pages, dated May 17, 2022. [cited by applicant]
United States Patent Office, Office Action in U.S. Appl. No. 16/271,355 dated Apr. 19, 2021. [cited by applicant]
United States Patent Office, Notice of Allowance in U.S. Appl. No. 16/271,355 dated Aug. 2, 2021. [cited by applicant]
360 Fly; The Cameras That Miss Nothing. and Change Everything. https://www.369fly.com/shop/cameras, dated Jun. 5, 2018. [cited by applicant]
Bojarski, et al.; End to End Learning for Self-Driving Cars, dated Apr. 25, 2016. [cited by applicant]
Google; Accessorize Your Camera—Google Street View, https://www.google.com/streetview/publish/mounts/, dated Jun. 5, 2018. [cited by applicant]
Google; Create Street View in a Snap—Google Street View, https://www.google.com/streetview/publish/, dated Jun. 5, 2018. [cited by applicant]
Insta360; Insta360 Pro, https://mall.insta360.com/product/pro, dated Jun. 5, 2018. [cited by applicant]
NC Tech; iSTAR Camera, Amazing Results with the iSTAR Panoramic Camera, https://www.nctechimaging.com/stall, dated Jun. 5, 2018. [cited by applicant]
NC Tech; The iSTAR Pulsar big data capture system, https://www.nctechimaging.com/istar-pulsar-techspec/, dated Jun. 5, 2018. [cited by applicant]
Matterport; Matterport Prot 3D Camera Specifications, https://supportmatterport.com/hc/en-us/articles/115004093167, dated Jun. 5, 2018. [cited by applicant]
Ricoh Company; Theta V, https://theta360.com/en/about/theta/v.html, dated Jun. 5, 2018. [cited by applicant]
amazon.com; RIDIC Accessories, Large Street View Ready Camera Mount: Camera & Photo; https://www.amazon.aom/dp/B075CYB215, dated Jun. 5, 2018. [cited by applicant]
amazon.com; Delkin DDMNT-TRIPLE Fat Gecko Three-Arm Suction Mount (Black): Professional Video Tripods: 3amera & Photo, https://www.amazon.com/gp/product/B00A86MMY8, dated Jun. 5, 2018. [cited by applicant]
YI Technology; YI 360 VR Camera, https://www.yitechnology.com/yi-360-vr-camera-specs, dated Jun. 5, 2018. [cited by applicant]
International Searching Authority; Search Report and Written Opinion for PCT Application No. PCT/US2019/024947; 12 pages; dated Jun. 25, 2019. [cited by applicant]
Japanese Patent App. No. 2021-523509 entered national stage on Apr. 28, 2021. [cited by applicant]
Chinese Patent App. No. 201980071453.4 entered national stage on Apr. 28, 2021. [cited by applicant]
Korean Patent App. No. 10-2021-7012268 entered national stage on Apr. 23, 2021. [cited by applicant]
Canadian Patent App. No. 3,118,402 entered national stage on Apr. 30, 2021. [cited by applicant]
European Patent App. No. 19722303.5 entered national stage on May 27, 2021. [cited by applicant]
European Patent Office, International Preliminary Report on Patentability for PCT/US2019/024947 dated Apr. 27, 2021. [cited by applicant]
European Patent Office, Communication pursuant to Rules 161(1) and 162EPC for app. No. 19722303.5 dated Jun. 11, 2021. [cited by applicant]
Desjardins, C. et al.; Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach; IEEEE, vol. 12, No. 4; pp. 1248-1260; dated Dec. 2011. [cited by applicant]
Ye, H. et al.: Machine LEarning for Vehicular Networks: Recent Advances and Application Examples; IEEE, vol. 13, No. 2; pp. 94-101; dated Jun. 1, 2018. [cited by applicant]
Dresner, et al.; A Multiagent Approach to Autonomous Intersection Management, Journal of Artificial Intelligence Research 31 (2008) 591-656, dated Mar. 2018. [cited by applicant]
Cortica Automotive; It's Not Enough to Recognize a Traffic Light 99% of the Time, http://www.cortica.com/automotive/index.html, dated Jun. 5, 2018. [cited by applicant]
Michels, Jeff, et al.; High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning, Somputer Science Department, Stanford University, Proceedings of the 22nd International Conference on Machine _earn… [cited by applicant]
Dresner, Kurt, et al.; Stone P. (2006) Multiagent Traffic Management: Opportunities for Multiagent Learning. In: Tuyls K., Hoen P.J., Verbeeck K., Sen S. (eds) Learning and Adaption in Multi-Agent Systems. LAMAS; Lectur… [cited by applicant]
Pongpunwattana, Anawat, et al.; Real-Time Planning for Multiple Autonomous Vehicles in Dynamic Jncertain Environments. Journal of Aerospace Computing Information and Communication—J Aerosp Comput Inf 7,0mmun. 1. 580-604… [cited by applicant]
Roderick, Norman, et al.; Reinforcement Learning for Autonomous Vehicles; dated 2002. [cited by applicant]
Cited By (2)
US 1,099,718 US 1,140,753