Methods and systems for managing data storage in vehicle operations
View Patent ↗This document discloses system, method, and computer program product embodiments for managing data generated by one or more systems of a vehicle. In various embodiments, a processor onboard a vehicle receives messages generated by one or more onboard systems of the vehicle. The system saves a first set of the messages to a first storage location on the vehicle according to a first data logging policy. The system processes a second set of the messages to reduce data elements and yield offboard data that is designated for offboard use. The first and second sets of messages may or may not overlap with each other. The system saves the offboard data to a second storage location that is onboard the vehicle and subject to a second data logging policy. The second data logging policy differs from the first data logging policy.
1. A method of managing data generated by one or more systems of a vehicle, the method comprising:
by one or more systems that are onboard a vehicle, wherein the one or more systems comprise a perception system of the vehicle, a forecasting system of the vehicle, a motion planning system of the vehicle, or a control system of the vehicle:
generating data elements during and relating to operation of the vehicle, and
generating messages at a first frequency, wherein each of the messages comprises a plurality of the data elements; and
by a processor that is onboard the vehicle:
receiving the plurality of messages generated by the one or more systems onboard the vehicle,
saving a first set of the received messages to a first storage location that is onboard the vehicle as onboard messages according to a first data logging policy that defines one or more parameters for storage of the onboard messages, wherein each parameter comprises a condition that, when satisfied, will signal that the processor may delete the onboard message,
processing a second set of the received messages to reduce a total number of the data elements in the second set by downsampling the second set of messages to yield a subset of messages corresponding to a second frequency that is lower than the first frequency, yielding offboard data that is designated for offboard use, and
saving the offboard data to a second storage location that is onboard the vehicle in which the offboard data is subject to a second data logging policy that defines one or more parameters for storage of the offboard data, wherein;
the second data logging policy differs from the first data logging policy;
the first data logging policy includes a first storage period;
the second data logging policy includes a second storage period; and
the second storage period is longer than the first storage period, and
deleting each of the onboard messages after the first storage period.
2. The method of claim 1 further comprising transferring the offboard data to an offboard system.
3. The method of claim 1 , wherein:
the data elements are associated with a plurality of categories,
each of the data elements is associated with one of the categories, and
processing the second set of messages to reduce a total number of the data elements in the second set comprises the data elements for at least one, but not all, of the categories.
4. The method of claim 1 , wherein processing the second set of messages to reduce the total number of the data elements in the second set comprises:
identifying one or more groups of semantically similar data elements of multiple messages in the messages of the second set; and
for each of the groups of semantically similar data elements in the messages of the second set, including fewer than all of the semantically similar data elements in the offboard data.
5. The method of claim 1 , wherein:
the first data logging policy comprises a condition that each onboard message must satisfy in order to be retained onboard the vehicle; and
the second data logging policy does not include the condition.
6. The method of claim 1 , wherein the offboard data is in the form of a set of offboard messages.
7. The method of claim 1 , wherein the second set of received messages includes one or more messages that are also in the first set of received messages.
8. A system, comprising:
a vehicle comprising:
one or more systems that are configured to generate data elements during and relating to operation of the vehicle, wherein the one or more systems comprise a perception system of the vehicle, a forecasting system of the vehicle, a motion planning system of the vehicle, or a control system of the vehicle;
at least one processor;
a first storage location; and
programming instructions stored in a memory and configured to cause the processor to, in response to receiving a plurality of messages generated by one or more of the systems onboard the vehicle, wherein each of the received messages comprises a plurality of the data elements and is received at a first frequency:
save a first set of the messages to the first storage location as onboard messages according to a first data logging policy that defines one or more parameters for storage of the onboard messages, wherein each parameter comprises a condition that, when satisfied, will signal that the processor may delete the onboard message,
process a second set of the messages to reduce a total number of the data elements in the second set by downsampling the second set to yield a subset of messages corresponding to a second frequency that is lower than the first frequency, yielding offboard data that is designated for offboard use,
save the offboard data to an offboard second storage location subject to a second data logging policy that defines one or more parameters for storage of the offboard data, wherein:
the second data logging policy differs from the first data logging policy;
the first data logging policy includes a first storage period;
the second data logging policy includes a second storage period; and
the second storage period is longer than the first storage period, and delete each of the onboard messages after the first storage period.
9. The system of claim 8 , further comprising additional programming instructions that are configured to cause the processor to transfer the offboard data to an offboard system.
10. The system of claim 8 , wherein the instructions to process the second set of messages to reduce the total number of data elements in the second set comprise instructions to downsample the data elements for at least one, but not all, of the categories that are associated with the data elements.
11. The system of claim 8 , wherein the instructions to process the second set of messages to reduce the total number of data elements in the second set comprise instructions to:
identify one or more groups of semantically similar data elements of multiple messages in the second set of messages; and
for each of the groups of semantically similar data elements in the messages of the second set, include fewer than all of the semantically similar data elements in the offboard data.
12. The system of claim 8 , wherein:
the first data logging policy comprises a condition that each onboard message must satisfy in order to be retained in the first storage location; and
the second data logging policy does not include the condition.
13. The system of claim 8 , wherein the offboard data is in the form of a set of offboard messages.
14. A computer program product comprising a non-transitory computer-readable medium that stores instructions that, when executed by a computing device that is onboard a vehicle, will cause the computing device to perform operations comprising:
generate data elements during and relating to operation of the vehicle, wherein the generating is done by a perception system of the vehicle, a forecasting system of the vehicle, a motion planning system of the vehicle, or a control system of the vehicle; generate a plurality of messages at a first frequency, wherein each of the messages comprises a plurality of the data elements;
save a first set of the messages to a first storage location as onboard messages according to a first data logging policy that defines one or more parameters for storage of the messages of the first set, wherein each parameter comprises a condition that, when satisfied, will signal that the processor may delete the onboard message;
process a second set of the received messages to reduce a total number of the data elements in the second set by downsampling the second set to yield a subset of messages corresponding to a second frequency that is lower than the first frequency, yielding offboard data that is designated for offboard use; and
save the offboard data to a second storage location subject to a second data logging policy that defines one or more parameters for storage of the offboard data, wherein;
the second data logging policy differs from the first data logging policy,
the first data logging policy includes a first storage period,
the second data logging policy includes a second storage period, and
the second storage period is longer than the first storage period; and
delete each of the onboard messages after the first storage period.
15. The computer program product of claim 14 wherein the instructions to process the second set of messages to reduce the total number of data elements in the second set comprise further instructions to perform one or more of the following:
downsample the data elements for at least one, but not all, of the categories that are associated with the data elements; or
identify one or more groups of semantically similar data elements of multiple messages in the second set of messages and, for each of the groups of semantically similar data elements in the messages of the second set, include fewer than all of the semantically similar data elements in the offboard data.
16. A method of managing data generated by one or more systems of a vehicle, the method comprising:
by one or more systems that are onboard a vehicle, wherein the one or more systems comprise a perception system of the vehicle, a forecasting system of the vehicle, a motion planning system of the vehicle, or a control system of the vehicle:
generating data elements during and relating to operation of the vehicle, and
generating messages at a first frequency, wherein each of the messages comprises a plurality of the data elements; and
by a processor that is onboard the vehicle:
receiving the plurality of messages generated by the one or more systems onboard the vehicle,
saving a first set of the received messages to a first storage location that is onboard the vehicle as onboard messages according to a first data logging policy that defines one or more parameters for storage of the onboard messages, wherein each parameter comprises a condition that, when satisfied, will signal that the processor may delete the onboard message,
processing a second set of the received messages to reduce a total number of the data elements in the second set by downsampling the messages comprising data elements for at least one, but not all, of the categories in the messages of the second set to yield a subset of messages corresponding to a second frequency that is lower than the first frequency, yielding offboard data that is designated for offboard use, and
saving the offboard data to a second storage location that is onboard the vehicle in which the offboard data is subject to a second data logging policy that defines one or more parameters for storage of the offboard data, wherein the second data logging policy differs from the first data logging policy.
17. The method of claim 16 further comprising transferring the offboard data to an offboard system.
18. The method of claim 16 , wherein processing the second set of messages to reduce the total number of the data elements in the second set comprises:
identifying one or more groups of semantically similar data elements of multiple messages in the messages of the second set; and
for each of the groups of semantically similar data elements in the messages of the second set, including fewer than all of the semantically similar data elements in the offboard data.
19. The method of claim 1 , wherein:
the first data logging policy comprises a condition that each onboard message must satisfy in order to be retained onboard the vehicle; and
the second data logging policy does not include the condition.
20. A system, comprising:
a vehicle comprising:
one or more systems that are configured to generate data elements during and relating to operation of the vehicle, wherein the one or more systems comprise a perception system of the vehicle, a forecasting system of the vehicle, a motion planning system of the vehicle, or a control system of the vehicle,
at least one processor,
a first storage location; and
programming instructions stored in a memory and configured to cause the processor to, in response to receiving a plurality of messages generated by one or more the systems onboard the vehicle, wherein each of the received messages comprises a plurality of the data elements and is received at a first frequency:
save a first set of the messages to the first storage location as onboard messages according to a first data logging policy that defines one or more parameters for storage of the onboard messages, wherein each parameter comprises a condition that, when satisfied, will signal that the processor may delete the onboard message,
process a second set of the messages to reduce a total number of the data elements in the second set by downsampling the messages comprising data elements for at least one, but not all, of the categories in the messages of the second set to yield a subset of messages corresponding to a second frequency that is lower than the first frequency, yielding offboard data that is designated for offboard use, and
save the offboard data to an offboard second storage location subject to a second data logging policy that defines one or more parameters for storage of the offboard data, wherein the second data logging policy differs from the first data logging policy.
21. The system of claim 20 , further comprising additional programming instructions that are configured to cause the processor to transfer the offboard data to an offboard system.
22. The system of claim 20 , wherein the instructions to process the second set of messages to reduce the total number of data elements in the second set comprise instructions to:
identify one or more groups of semantically similar data elements of multiple messages in the second set of messages; and
for each of the groups of semantically similar data elements in the messages of the second set, include fewer than all of the semantically similar data elements in the offboard data.
23. The system of claim 20 , wherein:
the first data logging policy comprises a condition that each onboard message must satisfy in order to be retained in the first storage location; and
the second data logging policy does not include the condition.
24. A computer program product comprising a non-transitory computer-readable medium that stores instructions that, when executed by a computing device that is onboard a vehicle, will cause the computing device to perform operations comprising:
generate data elements during and relating to operation of the vehicle, wherein the generating is done by a perception system of the vehicle, a forecasting system of the vehicle, a motion planning system of the vehicle, or a control system of the vehicle;
generate a plurality of messages at a first frequency, wherein each of the messages comprises a plurality of the data elements;
save a first set of the messages to a first storage location as onboard messages according to a first data logging policy that defines one or more parameters for storage of the messages of the first set, wherein each parameter comprises a condition that, when satisfied, will signal that the processor may delete the onboard message;
process a second set of the received messages to reduce a total number of the data elements in the second set by downsampling the messages comprising data elements for at least one, but not all, of the categories in the messages of the second set to yield a subset of messages corresponding to a second frequency that is lower than the first frequency, yielding offboard data that is designated for offboard use; and
save the offboard data to a second storage location subject to a second data logging policy that defines one or more parameters for storage of the offboard data, wherein the second data logging policy differs from the first data logging policy.