Aerial vehicle tracking using dynamic aleatoric uncertainty
Techniques for aerial vehicle tracking using dynamic aleatoric uncertainty covariance estimation are presented. The techniques include: obtaining an image depicting at least one aerial vehicle of interest; passing the image to a first machine learning subsystem, which provides at least one feature vector; inputting the at least one feature vector to a second machine learning subsystem, where the second machine learning subsystem is trained to provide detected aerial vehicle identification data sets (including respective aerial vehicle coordinates, respective aerial vehicle bounding box dimensions, and respective dynamic aleatoric uncertainty covariance values) corresponding to input feature vectors; providing at least one detected aerial vehicle identification data set to a recursive Bayesian estimator subsystem, from which at least one filtered set of aerial vehicle coordinates, representing a real-time location of a respective aerial vehicle of interest, is obtained; and outputting the at least one filtered set of aerial vehicle coordinates.
1 . A method of aerial vehicle tracking using dynamic aleatoric uncertainty covariance estimation, the method comprising:
obtaining an image depicting at least one aerial vehicle of interest;
passing the image to a first machine learning subsystem, from which at least one feature vector is obtained, wherein the first machine learning subsystem is trained with a first training corpus comprising a set of labeled images, wherein a respective labeled image comprises a respective image and a respective label;
inputting the at least one feature vector to a second machine learning subsystem, wherein the second machine learning subsystem is trained with a second training corpus comprising a set of labeled feature vectors, wherein a respective labeled feature vector comprises a respective feature vector, respective centroid coordinates, and respective bounding box dimensions, wherein the second machine learning subsystem is trained based on a loss to provide detected aerial vehicle identification data sets corresponding to input feature vectors, wherein the loss comprises (1) a regression loss for aerial vehicle coordinates and aerial vehicle bounding box dimensions, and (2) a Gaussian loss for aleatoric uncertainty, and wherein a respective detected aerial vehicle identification data set comprises: respective aerial vehicle coordinates, respective aerial vehicle bounding box dimensions, and respective dynamic aleatoric uncertainty covariance values;
providing at least one detected aerial vehicle identification data set, output from the second machine learning subsystem in response to the inputting, to a recursive Bayesian estimator subsystem, from which at least one filtered set of aerial vehicle coordinates is obtained, wherein the recursive Bayesian estimator subsystem assigns a respective detected aerial vehicle identification data set to a respective Bayesian estimator, from which a respective filtered set of aerial vehicle coordinates is obtained, wherein a respective filtered set of aerial vehicle coordinates represent a real-time location of a respective aerial vehicle of interest; and
outputting the at least one filtered set of aerial vehicle coordinates.
2 . The method of claim 1 , wherein respective dynamic aleatoric uncertainty covariance values are obtained from a single pass of a respective feature vector through the second machine learning subsystem.
3 . The method of claim 1 , wherein respective dynamic aleatoric uncertainty covariance values comprise at least one of: a respective covariance matrix for respective aerial vehicle coordinates, or respective aerial vehicle bounding box dimensions.
4 . The method of claim 1 , wherein the second machine learning subsystem comprises a multivariate deep evidential regression network.
5 . The method of claim 1 , wherein the Gaussian loss comprises a trace of a log-Cholesky decomposition of a matrix representing dynamic aleatoric uncertainty covariance values.
6 . The method of claim 1 , wherein a respective detected aerial vehicle identification data set further comprises an aerial vehicle classification label.
7 . The method of claim 1 , wherein a respective Bayesian estimator comprises a respective Kalman filter.
8 . The method of claim 7 , wherein the recursive Bayesian estimator subsystem is configured to provide a respective Kalman filter for each of a plurality of aerial vehicles of interest detected in the image.
9 . The method of claim 1 , wherein the first machine learning subsystem comprises a convolutional neural network and a multiscale feature decoder.
10 . The method of claim 1 , wherein respective filtered aerial vehicle bounding box dimensions are obtained from the respective Bayesian estimator, and wherein the outputting further comprises outputting at least one set of filtered aerial vehicle bounding box dimensions.
11 . A system for aerial vehicle tracking using dynamic aleatoric uncertainty covariance estimation, the system comprising: a non-transitory computer readable medium comprising instructions; and at least one electronic processor that executes the instructions to perform operations comprising:
obtaining an image depicting at least one aerial vehicle of interest;
passing the image to a first machine learning subsystem, from which at least one feature vector is obtained, wherein the first machine learning subsystem is trained with a first training corpus comprising a set of labeled images, wherein a respective labeled image comprises a respective image and a respective label;
inputting the at least one feature vector to a second machine learning subsystem, wherein the second machine learning subsystem is trained with a second training corpus comprising a set of labeled feature vectors, wherein a respective labeled feature vector comprises a respective feature vector, respective centroid coordinates, and respective bounding box dimensions, wherein the second machine learning subsystem is trained based on a loss to provide detected aerial vehicle identification data sets corresponding to input feature vectors, wherein the loss comprises (1) a regression loss for aerial vehicle coordinates and aerial vehicle bounding box dimensions, and (2) a Gaussian loss for aleatoric uncertainty, and wherein a respective detected aerial vehicle identification data set comprises: respective aerial vehicle coordinates, respective aerial vehicle bounding box dimensions, and respective dynamic aleatoric uncertainty covariance values;
providing at least one detected aerial vehicle identification data set, output from the second machine learning subsystem in response to the inputting, to a recursive Bayesian estimator subsystem, from which at least one filtered set of aerial vehicle coordinates is obtained, wherein the recursive Bayesian estimator subsystem assigns a respective detected aerial vehicle identification data set to a respective Bayesian estimator, from which a respective filtered set of aerial vehicle coordinates is obtained, wherein a respective filtered set of aerial vehicle coordinates represent a real-time location of a respective aerial vehicle of interest; and
outputting the at least one filtered set of aerial vehicle coordinates.
12 . The system of claim 11 , wherein respective dynamic aleatoric uncertainty covariance values are obtained from a single pass of a respective feature vector through the second machine learning subsystem.
13 . The system of claim 11 , wherein respective dynamic aleatoric uncertainty covariance values comprise at least one of: a respective covariance matrix for respective aerial vehicle coordinates, or respective aerial vehicle bounding box dimensions.
14 . The system of claim 11 , wherein the second machine learning subsystem comprises a multivariate deep evidential regression network.
15 . The system of claim 11 , wherein the Gaussian loss comprises a trace of a log-Cholesky decomposition of a matrix representing dynamic aleatoric uncertainty covariance values.
16 . The system of claim 11 , wherein a respective detected aerial vehicle identification data set further comprises an aerial vehicle classification label.
17 . The system of claim 11 , wherein a respective Bayesian estimator comprises a respective Kalman filter.
18 . The system of claim 17 , wherein the recursive Bayesian estimator subsystem is configured to provide a respective Kalman filter for each of a plurality of aerial vehicles of interest detected in the image.
19 . The system of claim 11 , wherein the first machine learning subsystem comprises a convolutional neural network and a multiscale feature decoder.
20 . The system of claim 11 , wherein respective filtered aerial vehicle bounding box dimensions are obtained from the respective Bayesian estimator, and wherein the outputting further comprises outputting at least one set of filtered aerial vehicle bounding box dimensions.