Management of vehicle system information using a deep learning device
Methods, systems, and devices for management of vehicle system information using a deep learning device are described. The deep learning device of a vehicle (such as a deep learning accelerator (DLA)) may receive information associated with an environment of the vehicle from one or more sensors of the vehicle. The DLA may perform one or more operations using one or more machine learning models. For example, the DLA may compress the information which may reduce a resolution associated with the information, a frame rate associated with the information, or both. The DLA may generate, as part of a run-time operation, a first set of analytics associated with operation of the vehicle using the compressed information. Additionally, or alternatively, the DLA may generate, as part of a post-processing operation, a second set of analytics using the compressed or an uncompressed version of the information.
1 . A method, comprising:
receiving, at a deep learning device directly coupled with a non-volatile memory device of a vehicle, information associated with an environment of the vehicle from one or more sensors of the vehicle, the deep learning device for performing one or more operations using one or more machine learning models;
performing, at the deep learning device, a compression operation on the information based at least in part on receiving the information, wherein the compression operation is performed to satisfy a predetermined latency constraint for real-time vehicle operation;
generating, at the deep learning device, a set of run-time analytics associated with operation of the vehicle based at least in part on the compressed information, wherein the set of run-time analytics is generated within a predetermined duration from receiving the information;
generating, at the deep learning device, a set of post-processing analytics based on uncompressed information, wherein the set of post-processing analytics provides additional analysis than the set of run-time analytics; and
outputting the set of run-time analytics, the set of post-processing analytics, and the compressed information to one or more storage components associated with the vehicle.
2 . The method of claim 1 , further comprising:
generating the set of post-processing analytics after outputting the set of run-time analytics and the compressed information to the one or more storage components, wherein the set of post-processing analytics are associated with a post-processing analysis of the operation of the vehicle and unassociated with the duration.
3 . The method of claim 1 , further comprising:
reducing, based at least in part on the duration, a resolution associated with the information, a frame rate associated with the information, or a combination thereof, wherein the set of run-time analytics are generated within the duration based at least in part on the reducing.
4 . The method of claim 3 , further comprising:
generating the set of post-processing analytics using a non-reduced version of the information based at least in part on being associated with post-processing analysis of the operation of the vehicle.
5 . The method of claim 1 , further comprising:
encrypting, at the deep learning device, the set of run-time analytics and the compressed information, wherein outputting the set of run-time analytics and the compressed information to the one or more storage components associated with the vehicle is based at least in part on the encrypting.
6 . The method of claim 1 , further comprising:
storing the information from the one or more sensors of the vehicle directly to the non-volatile memory device of the vehicle;
receiving, at the deep learning device, the information stored to the non-volatile memory device; and
generating, at the deep learning device, a second set of analytics associated with the operation of the vehicle based at least in part on the information stored to the non-volatile memory device.
7 . The method of claim 1 , further comprising:
performing, before the compression operation, a second compression operation on the information using one or more video compression operations.
8 . The method of claim 1 , wherein the set of run-time analytics comprises identification of one or more objects associated with the environment of the vehicle, location information associated with the one or more objects, a speed of the vehicle, a respective speed of the one or more objects, an acceleration of the vehicle, a respective acceleration of the one or more objects, an object type of the one or more objects, one or more portions of the vehicle associated with a collision with the one or more objects, an estimated force experienced by the vehicle or a passenger of the vehicle based at least in part on the collision, a prediction of the collision, or a combination thereof.
9 . The method of claim 1 , wherein the one or more sensors comprise one or more cameras, one or more light detection and ranging sensors, one or more radar sensors, one or more sonar sensors, a speedometer, an accelerometer, one or more infrared light detectors, a geographic location device, or a combination thereof.
10 . An apparatus, comprising:
one or more sensors of a vehicle;
a volatile memory device configured to receive information associated with an environment of the vehicle from the one or more sensors;
a non-volatile memory device; and
a deep learning device directly coupled with the non-volatile memory device and configured to perform one or more operations using one or more machine learning models, wherein the deep learning device is further configured to:
receive, from the volatile memory device, the information;
perform a compression operation on the information based at least in part on receiving the information at the deep learning device, wherein the compression operation is performed to satisfy a predetermined latency constraint for real-time vehicle operation;
generate a set of run-time analytics associated with operation of the vehicle based at least in part on the compressed information, wherein the set of run-time analytics is generated within a predetermined duration from receiving the information;
generate a set of post-processing analytics based on uncompressed information, wherein the set of post-processing analytics provides additional analysis than the set of run-time analytics; and
output the set of run-time analytics, the set of post-processing analytics, and the compressed information to the non-volatile memory device.
11 . The apparatus of claim 10 , wherein the deep learning device and the volatile memory device are included in a same memory die of the apparatus.
12 . The apparatus of claim 10 , wherein the deep learning device is included in a first memory die of the apparatus and the volatile memory device is included in a second memory die of the apparatus, the first memory die coupled with the second memory die.
13 . The apparatus of claim 10 , wherein the deep learning device is included in a first memory die of the apparatus and the non-volatile memory device is associated to a second memory die of the apparatus, and the first memory die and the second memory die are hybrid bonded.
14 . The apparatus of claim 10 , wherein the deep learning device is further configured to:
generate the set of post-processing analytics after outputting the set of run-time analytics and the compressed information to the non-volatile memory device, wherein the set of post-processing analytics are associated with a post-processing analysis of the operation of the vehicle and unassociated with the duration.
15 . The apparatus of claim 10 , wherein the deep learning device is further configured to:
reduce, based at least in part on the duration, a resolution associated with the information, a frame rate associated with the information, or a combination thereof, wherein the set of run-time analytics are generated within the duration based at least in part on the reducing.
16 . The apparatus of claim 15 , wherein the deep learning device is further configured to:
generate the set of post-processing analytics using a non-reduced version of the information based at least in part on being associated with post-processing analysis of the operation of the vehicle.
17 . The apparatus of claim 10 , wherein the one or more sensors, the volatile memory device, the non-volatile memory device, and the deep learning device are included in a zonal computing system of the vehicle, the zonal computing system further comprising:
a central processor configured to communicate with a remote server and a plurality of zones associated with the zonal computing system; and
a gateway processor coupled with the central processor and associated with a zone of the plurality of zones, wherein the gateway processor is configured to route communications between the central processor and components of the zonal computing system, and wherein the volatile memory device is configured to receive the information via the gateway processor.
18 . An apparatus, comprising:
a deep learning device directly coupled with a non-volatile memory device; and
a controller coupled with the deep learning device and operable to cause the apparatus to:
receive, at the deep learning device of a vehicle, information associated with an environment of the vehicle from one or more sensors of the vehicle, the deep learning device for performing one or more operations using one or more machine learning models;
compress, at the deep learning device, the information based at least in part on receiving the information at the deep learning device, wherein the compression is performed to satisfy a predetermined latency constraint for real-time vehicle operation;
generate, at the deep learning device, a set of run-time analytics associated with operation of the vehicle based at least in part on the compressed information, wherein the set of run-time analytics is generated within a predetermined duration from receiving the information;
generate, at the deep learning device, a set of post-processing analytics based on uncompressed information, wherein the set of post-processing analytics provides additional analysis than the set of run-time analytics; and
output, the set of run-time analytics, the set of post-processing analytics, and the compressed information to one or more storage components associated with the vehicle.