Vehicle data collection system and method including reliability information for a storage unit for storing partial log data
A vehicle data collection system includes a sensor configured to collect data; a non-transitory computer readable medium configured to store instructions, an application log, and a system log thereon; and a processor. The processor is configured to execute the instructions for receiving the collected data from the sensor, the application log or the system log; and generating log data based on the collected data. The processor is configured to execute the instructions for separating the log data into a plurality of partial log data; and generating metadata for each of the plurality of partial log data. The metadata includes reliability information for a portion of a storage unit where corresponding partial log data of the plurality of partial log data will be stored. The processor is configured to execute the instructions for instructing the storage unit to store each of the plurality of partial log data in the storage unit.
1 . A vehicle data collection system comprising:
a sensor configured to collect data;
a non-transitory computer readable medium configured to store instructions, and at least one of an application log or a system log thereon; and
a processor connected to the non-transitory computer readable medium, wherein the processor is configured to execute the instructions for:
receiving the collected data from the sensor and the at least one of the application log or the system log;
generating log data based on the collected data;
separating the log data into a plurality of partial log data;
generating metadata for each of the plurality of partial log data, wherein the metadata comprises reliability information for a first portion of a storage unit, wherein the first portion of the storage unit includes a first set of memory cells at a first location in the storage unit where corresponding partial log data of the plurality of partial log data is to be stored, the reliability information indicates a risk of corruption or loss of data stored at the first portion of the storage unit, the first portion of the storage unit has a first risk of corruption or loss of data, a second portion of the storage unit has a second risk, different from the first risk, of corruption or loss of data, and the second portion of the storage unit includes a second set of memory cells at a second location in the storage unit different from the first location; and
instructing the storage unit to store each of the plurality of partial log data in the corresponding portion of the storage unit.
2 . The vehicle data collection system of claim 1 , wherein the first risk of corruption or loss of data is based on a number of times the first set of memory cells is erased or overwritten, and the second risk of corruption or loss of data is based on a number of times the second set of memory cells is erased or overwritten.
3 . The vehicle data collection system of claim 1 , wherein the reliability information comprises at least one of a number of times the first portion of the storage unit is erased or a number of times to which the first portion of the storage unit is written.
4 . The vehicle data collection system of claim 1 , wherein the processor is further configured to execute the instructions for:
identifying a first partial log data of the plurality of partial log data having a risk of (a) additional data copy or (b) being lost being above a predetermined threshold based on the reliability information for the first partial log data.
5 . The vehicle data collection system of claim 4 , wherein the processor is further configured to execute the instructions for:
instructing a transmitter to transmit the first partial log data prior to another scheduled transmission in response to a determination that the first partial log data has a risk of additional data copy or being lost above the predetermined threshold.
6 . The vehicle data collection system of claim 1 , wherein the processor is further configured to execute the instructions for:
receiving a data request from a server, wherein the processor is configured to generate the log data based on the collected data and the received data request.
7 . The vehicle data collection system of claim 1 , wherein the processor is further configured to execute the instructions for:
determining whether an entirety of the generated log data is transmitted; and
instructing a transmitter to transmit a notification to a server in response to a determination that the entirety of the generated log data is transmitted.
8 . The vehicle data collection system of claim 1 , wherein the sensor is one of a plurality of sensors, and the processor is further configured to execute the instructions for:
generating stream identification (ID) information for each of the plurality of partial log data based on which the sensor of the plurality of sensors collected the data associated with the corresponding partial log data of the plurality of partial log data.
9 . The vehicle data collection system of claim 8 , wherein the processor is further configured to execute the instructions for:
determining the reliability information based on the stream ID information for each of the plurality of partial log data.
10 . A method comprising:
receiving collected data from a sensor and at least one of an application log or a system log;
generating log data based on the collected data;
separating the log data into a plurality of partial log data;
generating metadata for each of the plurality of partial log data, wherein the generated metadata comprises reliability information for a first portion of a storage unit, wherein the first portion of the storage unit includes a first set of memory cells at a first location in the storage unit is where corresponding partial log data of the plurality of partial log data is to be stored, the reliability information indicates a risk of corruption or loss of data stored at the first portion of the storage unit, the first portion of the storage unit has a first risk of corruption or loss of data, a second portion of the storage unit has a second risk, different from the first risk, of corruption or loss of data, and the second portion of the storage unit includes a second set of memory cells at a second location in the storage unit different from the first location; and
storing each of the plurality of partial log data in the corresponding portion of the storage unit.
11 . The method of claim 10 , wherein the first risk of corruption or loss of data is based on a number of times the first set of memory cells is erased or overwritten, and the second risk of corruption or loss of data is based on a number of times the second set of memory cells is erased or overwritten.
12 . The method of claim 10 , wherein the reliability information comprises write time information.
13 . The method of claim 10 , further comprising:
identifying a first partial log data of the plurality of partial log data having a risk of (a) additional data copy or (b) being lost being above a predetermined threshold based on the reliability information for the first partial log data.
14 . The method of claim 13 , further comprising:
transmitting the first partial log data prior to another scheduled transmission in response to a determination that the first partial log data has a risk of additional data copy or being lost above the predetermined threshold.
15 . The method of claim 10 , further comprising:
receiving a data request from a server; and
generating the log data based on the collected data and the received data request.
16 . The method of claim 10 , further comprising:
determining whether an entirety of the generated log data is transmitted; and
transmitting a notification to a server in response to a determination that the entirety of the generated log data is transmitted.
17 . The method of claim 10 , wherein receiving collected data comprises receiving the collected data from a plurality of sensors including the sensor, and the method further comprises:
generating stream identification (ID) information for each of the plurality of partial log data based on which the sensor of the plurality of sensors collected the data associated with the corresponding partial log data of the plurality of partial log data.
18 . The method of claim 17 , further comprising:
determining the reliability information based on the stream ID information for each of the plurality log data.