Systems and methods for managing vehicle data
The present disclosure provides methods and systems for managing autonomous vehicle data. The method may comprise: (a) collecting said autonomous vehicle data from the autonomous vehicle, wherein the autonomous vehicle data has a size of at least 1 terabyte; (b) processing the autonomous vehicle data to generate metadata corresponding to the autonomous vehicle data, wherein the autonomous vehicle data is stored in a database; (c) using at least a portion of the metadata to retrieve a subset of the autonomous vehicle data from the database, which subset of the autonomous vehicle data has a size less than the autonomous vehicle data; and (d) storing or transmitting the subset of the autonomous vehicle data.
1. A data orchestrator for managing vehicle data for a vehicle, comprising:
a data storage unit storing (i) application data related to one or more applications that generate one or more subsets of said vehicle data, and (ii) entity data related to one or more remote entities that request said one or more subsets of said vehicle data, wherein said data storage unit is local to said vehicle where said vehicle data is collected or generated;
a database, local to said vehicle and accessible to the data orchestrator, and configured as a knowledge base storing a machine learning-based predictive model, trained and tested using metadata and said vehicle data, and user-defined rules for determining a data transmission rule comprising at least one of: (i) a selected portion of said vehicle data to be transmitted; (ii) a timing for transmitting said selected portion of said vehicle data; and (iii) a remote entity of said one or more remote entities that is designated to receive said selected portion of said vehicle data, wherein (a) said vehicle data comprises sensor data processed by a pipeline engine, (b) said metadata is generated from said sensor data, and (c) wherein additional metadata is generated when said vehicle data is processed by said pipeline engine; and
a transmission module configured to transmit said selected portion of said vehicle data based at least in part on said application data, said entity data, and said data transmission rule.
2. The data orchestrator of claim 1 , wherein said transmission module is provided onboard said vehicle.
3. The data orchestrator of claim 1 , wherein said one or more remote entities comprise a cloud application, a data center, a third-party server, or another different vehicle.
4. The data orchestrator of claim 1 , wherein said data storage unit is further configured to store supplemental data indicative of an availability of said one or more subsets of said vehicle data, a transmission timing delay, a data type of said one or more subsets of data, or a transmission protocol.
5. The data orchestrator of claim 1 , wherein said machine learning-based predictive model is based at least in part on a model tree structure.
6. The data orchestrator of claim 5 , wherein said model tree structure is used to represent one or more relationships between one or more models comprising said machine learning-based predictive model.
7. The data orchestrator of claim 5 , wherein at least one node of said model tree structure is used to represent said machine learning-based predictive model, and wherein said node includes at least one of a model architecture, a set of model parameters, a training dataset, or a test dataset.
8. The data orchestrator of claim 1 , wherein said machine learning-based predictive model is configured to be generated by a model creator located in a data center remote to said vehicle.
9. The data orchestrator of claim 8 , wherein said model creator is configured to generate one or more machine learning-based predictive models that are usable by or customized for said vehicle.
10. The data orchestrator of claim 1 , wherein said metadata is associated with a sensor that captures said sensor data.
11. The data orchestrator of claim 1 , wherein the pipeline engine comprises one or more functional components.
12. The data orchestrator of claim 11 , wherein at least one of said one or more functional components is selected from a set of functions via a user interface.
13. The data orchestrator of claim 11 , wherein at least one of said one or more functional components is configured to create a scenario data object, wherein said scenario data object is usable for specifying a use scenario for said metadata.
14. The data orchestrator of claim 1 , wherein said metadata is usable to retrieve a subset of said vehicle data from said remote entity for training said machine learning-based predictive model.
15. The data orchestrator of claim 1 , where said knowledge base is configured to store one or more machine learning-based predictive models that are usable by or customized for said vehicle.
16. The data orchestrator of claim 1 , wherein said selected portion of said vehicle data comprises an aggregation of two or more of said subsets of said vehicle data.
17. The data orchestrator of claim 1 , wherein said vehicle is (i) a connected vehicle, (ii) a connected and automated vehicle, or (iii) a connected and autonomous vehicle.