Method for classifying a tracked object
A method is provided for classifying a tracked object in an environment of a vehicle. The vehicle includes a plurality of radar sensors and a processing device configured to establish a neural network. According to the method, local radar detections are captured from an object in the environment of the vehicle via the radar sensors. Based on the local radar detections, point features and tracker features are determined. The point features are encoded via point encoding layers of the neural network, whereas the tracker features are encoded via track encoding layers of the neural network. A temporal fusion of the encoded point features and the encoded tracker features is performed via temporal fusion layers of the neural network. The tracked object is classified based on the fused encoded point and tracker features via classifying layers of the neural network.
1. A method comprising:
classifying a tracked object in an environment of a vehicle that has a plurality of radar sensors in addition to a processing device configured to establish a neural network, classifying the tracked object comprising:
capturing local radar detections from an object in the environment of the vehicle via the radar sensors;
determining point features based on the local radar detections;
determining tracker features based on the local radar detections;
encoding the point features via point encoding layers of the neural network;
encoding the tracker features via track encoding layers of the neural network;
concatenating the encoded point features and the encoded tracker features to form a common tensor;
passing the common tensor to a recurrent neural network which provides temporal fusion layers of the neural network to perform a temporal fusion of the encoded point features and the encoded tracker features via the temporal fusion layers; and
classifying the tracked object based on the fused encoded point and tracker features via classifying layers of the neural network.
2. The method according to claim 1 , further comprising:
encoding each point feature for the point encoding layers of the neural network separately; and
feature pooling the point encoding layers of the neural network.
3. The method according to claim 2 , wherein feature pooling includes performing a maximum pooling.
4. The method according to claim 2 , wherein feature pooling includes performing an average pooling.
5. The method according to claim 1 , wherein the recurrent neural network is configured as a gated recurrent unit.
6. The method according to claim 1 , wherein the recurrent neural network is configured as a long-short term memory.
7. The method according to claim 1 , wherein each layer of the neural network includes a predefined maximum number of neurons.
8. The method according to claim 1 , further comprising:
sampling a predetermined number of the local radar detections before determining the point features based on local radar detections.
9. The method according to claim 1 , wherein performing the temporal fusion includes performing a hidden state reset.
10. The method according to claim 1 , wherein the point features comprise a flexible list of point features.
11. The method according to claim 1 , wherein the tracker features comprise a flexible list of tracker features.
12. A system comprising:
a computer system for a vehicle including a plurality of radar sensors and a processor, the processor configured to execute a neural network to classify a tracked object in an environment of the vehicle by:
capturing local radar detections from an object in the environment of the vehicle via the radar sensors;
determining point features based on the local radar detections;
determining tracker features based on the local radar detections;
encoding the point features via point encoding layers of the neural network;
encoding the tracker features via track encoding layers of the neural network;
concatenating the encoded point features and the encoded tracker features to form a common tensor;
passing the common tensor to a recurrent neural network which provides temporal fusion layers of the neural network to perform a temporal fusion of the encoded point features and the encoded tracker features via the temporal fusion layers; and
classifying the tracked object based on the fused encoded point and tracker features via classifying layers of the neural network.
13. The system according to claim 12 , further comprising:
encoding each point feature for the point encoding layers of the neural network separately; and
feature pooling the point encoding layers of the neural network.
14. The system according to claim 13 , wherein feature pooling includes performing a maximum pooling.
15. The system according to claim 13 , wherein feature pooling includes performing an average pooling.
16. The system according to claim 12 , wherein the recurrent neural network is configured as a gated recurrent unit.
17. The system according to claim 12 , wherein the recurrent neural network is configured as a long-short term memory.
18. The system according to claim 12 , wherein each layer of the neural network includes a predefined maximum number of neurons.
19. The system according to claim 12 , further comprising:
sampling a predetermined number of the local radar detections before determining the point features based on local radar detections.
20. A non-transitory computer readable medium comprising instructions that when executed by a processor of a vehicle having a plurality of radar sensors, configure the processor to execute a neural network to track an object in an environment of the vehicle by:
capturing local radar detections from an object in the environment of the vehicle via the radar sensors;
determining point features based on the local radar detections;
determining tracker features based on the local radar detections;
encoding the point features via point encoding layers of the neural network;
encoding the tracker features via track encoding layers of the neural network;
concatenating the encoded point features and the encoded tracker features to form a common tensor;
passing the common tensor to a recurrent neural network which provides temporal fusion layers of the neural network to perform a temporal fusion of the encoded point features and the encoded tracker features via the temporal fusion layers; and
classifying the tracked object based on the fused encoded point and tracker features via classifying layers of the neural network.