Data recorders of autonomous vehicles
Systems, methods and apparatus to collect sensor data generated in an autonomous vehicle. Sensors of the vehicle generate a sensor data stream that is buffered, in parallel and in a cyclic way, in a first cyclic buffer and a larger second cyclic buffer respectively. An advanced driver assistance system of the vehicle generates an accident signal when detecting or predicting an accident and provides a training signal when detecting a fault in object detection, recognition, identification or classification. The accident signal causes a sensor data stream segment to be copied from the first cyclic buffer into a slot of a non-volatile memory, selected from a plurality of slots in a round robin way. The training signal causes a sensor data stream segment to be copied from the second cyclic buffer into an area of the non-volatile memory outside of the slots reserved for the first cyclic buffer.
1. A device, comprising:
an interface operable to receive:
a sensor data stream from sensors configured on a vehicle during autonomous operations of the vehicle on a road; and
signals of a first type and a second type;
a non-volatile memory;
a first cyclic buffer having a first capacity;
a second cyclic buffer having a second capacity larger than the first capacity, wherein, in absence of a signal of the first type and a signal of the second type, the first cyclic buffer and the second cyclic buffer are configured to buffer the sensor data stream in parallel and cyclically within the first capacity and the second capacity respectively; and
a controller configured to:
copy, in response to the signal of the first type, first sensor data from the first cyclic buffer into the non-volatile memory; and
copy, in response to the signal of the second type, second sensor data from the second cyclic buffer into the non-volatile memory.
2. The device of claim 1 , wherein the first capacity is configured to buffer the first sensor data generated by the sensors up to a first period of time; and the second capacity is configured to buffer the second sensor data generated by the sensors up to a second period of time that is longer than the first period of time.
3. The device of claim 2 , wherein an advanced driver assistance system is configured to operate the vehicle on the road based on the sensor data stream.
4. The device of claim 3 , wherein the sensors include a digital camera, a radar, a lidar, or an ultrasound sonar, or any combination thereof.
5. The device of claim 3 , wherein the signal of the first type is from the advanced driver assistance system to indicate an accident involving the vehicle; and the signal of the second type is from the advanced driver assistance system to indicate an opportunity to train the advanced driver assistance system.
6. The device of claim 5 , wherein the advanced driver assistance system includes an artificial neural network and is configured to generate the signal of the second type in response to a mismatch between a first identification of an object determined by the artificial neural network using the sensor data stream and a second identification of the object determined without using the sensor data stream.
7. The device of claim 6 , further configured to transmit the second sensor data from the non-volatile memory to a remote server configured to update the artificial neural network.
8. The device of claim 5 , wherein the first sensor data is entirely generated by the sensors prior to the signal of the first type; and the second sensor data includes a first portion that is generated by the sensors prior to the signal of the second type and a second portion that is generated by the sensors during a predetermined period of time after the signal of the second type.
9. The device of claim 8 , wherein the controller is further configured to:
stop, in response to the signal of the first type, buffering data from the sensor data stream into the first cyclic buffer before at least a portion of the first sensor data is copied from the first cyclic buffer into the non-volatile memory; and
continue, for the predetermined period of time and in response to the signal of the second type, buffering data from the sensor data stream into the second cyclic buffer before the controller stops buffering further data from the sensor data stream into the second cyclic buffer.
10. A method, comprising:
receiving, in an interface of a device, a sensor data stream from sensors configured on a vehicle during autonomous operations of the vehicle on a road, the interface operable to receive signals of a first type and a second type;
in absence of a signal of the first type and a signal of the second type:
buffering, by a first cyclic buffer having a first capacity, the sensor data stream; and
buffering, by a second cyclic buffer having a second capacity larger than the first capacity, the sensor data stream in parallel with the first cyclic buffer;
copying, by a controller of the device and in response to the signal of the first type, first sensor data from the first cyclic buffer into a non-volatile memory of the device; and
copying, by the controller of the device and in response to the signal of the second type, second sensor data from the second cyclic buffer into the non-volatile memory of the device.
11. The method of claim 10 , wherein the first capacity is configured to buffer the first sensor data generated by the sensors up to a first period of time; and the second capacity is configured to buffer the second sensor data generated by the sensors up to a second period of time that is longer than the first period of time.
12. The method of claim 11 , further comprising:
operating, by an advanced driver assistance system, the vehicle on the road based on the sensor data stream, wherein the sensors include a digital camera, a radar, a lidar, or an ultrasound sonar, or any combination thereof.
13. The method of claim 12 , wherein the signal of the first type is from the advanced driver assistance system to indicate an accident involving the vehicle; and the signal of the second type is from the advanced driver assistance system to indicate an opportunity to train the advanced driver assistance system.
14. The method of claim 13 , wherein the advanced driver assistance system includes an artificial neural network and is configured to generate the signal of the second type in response to a mismatch between a first identification of an object determined by the artificial neural network using the sensor data stream and a second identification of the object determined without using the sensor data stream.
15. The method of claim 13 , wherein the first sensor data is entirely generated by the sensors prior to the signal of the first type; and the second sensor data includes a first portion that is generated by the sensors prior to the signal of the second type and a second portion that is generated by the sensors during a predetermined period of time after the signal of the second type.
16. The method of claim 10 , further comprising:
stopping, in response to the signal of the first type, buffering data from the sensor data stream into the first cyclic buffer before at least a portion of the first sensor data is copied from the first cyclic buffer into the non-volatile memory; and
continuing, for a predetermined period of time and in response to the signal of the second type, buffering data from the sensor data stream into the second cyclic buffer before the controller stops buffering further data from the sensor data stream into the second cyclic buffer.
17. A non-transitory computer storage medium storing instructions which, when executed in a device, cause the device to perform a method, comprising:
receiving, via an interface of the device, a sensor data stream from sensors configured on a vehicle during autonomous operations of the vehicle on a road, the interface operable to receive signals of a first type and a second type;
in absence of a signal of the first type and a signal of the second type:
buffering, by a first cyclic buffer having a first capacity, the sensor data stream; and
buffering, by a second cyclic buffer having a second capacity larger than the first capacity, the sensor data stream in parallel with the first cyclic buffer;
copying, by a controller of the device and in response to the signal of the first type, first sensor data from the first cyclic buffer into a non-volatile memory of the device; and
copying, by the controller of the device and in response to the signal of the second type, second sensor data from the second cyclic buffer into the non-volatile memory of the device.
18. The non-transitory computer storage medium of claim 17 , wherein the method further comprises:
stopping, in response to the signal of the first type, buffering data from the sensor data stream into the first cyclic buffer before at least a portion of the first sensor data is copied from the first cyclic buffer into the non-volatile memory; and
continuing, for a predetermined period of time and in response to the signal of the second type, buffering data from the sensor data stream into the second cyclic buffer before the controller stops buffering further data from the sensor data stream into the second cyclic buffer.
19. The non-transitory computer storage medium of claim 17 , wherein the method further comprises:
operating, by an advanced driver assistance system, the vehicle on the road based on the sensor data stream, wherein the sensors include a digital camera, a radar, a lidar, or an ultrasound sonar, or any combination thereof;
wherein the signal of the first type is from the advanced driver assistance system to indicate an accident involving the vehicle; and the signal of the second type is from the advanced driver assistance system to indicate an opportunity to train the advanced driver assistance system.
20. The non-transitory computer storage medium of claim 17 , wherein the first sensor data is entirely generated by the sensors prior to the signal of the first type; and the second sensor data includes a first portion that is generated by the sensors prior to the signal of the second type and a second portion that is generated by the sensors during a predetermined period of time after the signal of the second type.