Using temporal filters for automated real-time classification
In various examples, the present disclosure relates to using temporal filters for automated real-time classification. The technology described herein improves the performance of a multiclass classifier that may be used to classify a temporal sequence of input signals—such as input signals representative of video frames. A performance improvement may be achieved, at least in part, by applying a temporal filter to an output of the multiclass classifier. For example, the temporal filter may leverage classifications associated with preceding input signals to improve the final classification given to a subsequent signal. In some embodiments, the temporal filter may also use data from a confusion matrix to correct for the probable occurrence of certain types of classification errors. The temporal filter may be a linear filter, a nonlinear filter, an adaptive filter, and/or a statistical filter.
1. A method for assigning a classification to an input signal comprising:
computing, using a neural network, a first confidence score for a first input signal, the first confidence score corresponding to a first class represented in the first input signal;
computing, using a confusion factor between the first class and a second class, a first normalization amount corresponding to the first class, the confusion factor representative of a probability that the neural network will compute, for a second input signal associated with the second class, an output indicating that the second input signal corresponds to the first class;
generating a first normalized confidence score corresponding to the first class by adjusting the first confidence score according to the first normalization amount calculated using the confusion factor;
applying a filter to the first normalized confidence score to generate a final confidence score corresponding to the first class; and
determining a final classification for the first input signal using the final confidence score corresponding to the first class.
2. The method of claim 1 , wherein the neural network is part of a gesture control system.
3. The method of claim 1 , wherein the neural network is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for the autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a gaming system;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
4. The method of claim 1 , wherein the adjusting the first confidence score according to the first normalization amount includes subtracting the first normalization amount from the first confidence score.
5. The method of claim 1 , wherein the filter includes at least one of a temporal filter, an adaptive filter, or a temporal adaptive filter.
6. The method of claim 1 , wherein the first confidence score is generated by combining an output of a softmax calculation with an output of an angular visual hardness calculation.
7. A method for assigning a classification to an input signal comprising:
receiving a temporal series of one or more classification outputs generated by processing a temporal sequence of input signals using a neural network;
detecting a classification state change within the one or more classification outputs indicating a probable classification change from a first class to a second class;
tuning, based at least in part on the classification state change, a filter to adjust a parameter corresponding to one or more first outputs in the one or more classification outputs;
applying the filter to the one or more first outputs in the one or more classification outputs to generate a final confidence score; and
generating a final classification for the input signal using the final confidence score.
8. The method of claim 7 , wherein the parameter is a weight given to the one or more classification outputs when generating the final confidence score.
9. The method of claim 8 , wherein the parameter is adjusted to give less weight when generating the final confidence score to at least one classification output of the one or more classification outputs that was generated before at least one other classification output of the one or more classification outputs when generating the final confidence score.
10. The method of claim 9 , wherein the weight is decreased by increasing a rate of decay in a decay function within the filter.
11. The method of claim 9 , wherein the weight is decreased by using fewer than a default amount of one or more classification outputs when generating the final confidence score.
12. The method of claim 7 , further comprising:
computing, using a confusion factor between the first class and the second class, a first normalization amount corresponding to the first class, the confusion factor representative of a probability that the neural network will compute, for a second input signal associated with the second class, an output indicating that the second input signal corresponds to the first class;
generating a first normalized confidence score corresponding to the first class by adjusting a first confidence score according to the first normalization amount calculated using the confusion factor; and
using the first normalized confidence score to generate the final classification.
13. The method of claim 12 , wherein the generating the first normalized confidence score includes subtracting the first normalization amount from the first confidence score.
14. The method of claim 7 , wherein the temporal sequence of input signals is generated by an autonomous or semi-autonomous machine.
15. The method of claim 7 , wherein the neural network is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for the autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a gaming system;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
16. A method for assigning a classification to an input signal comprising:
receiving a temporal series of one or more classification outputs generated by processing a temporal sequence of input signals using a neural network;
detecting a classification state change within the one or more classification outputs indicating a probable classification change from a first class to a second class;
tuning, based at least in part on the classification state change, a filter to adjust a parameter associated with one or more older outputs in the one or more classification outputs;
computing a first normalization amount corresponding to the first class using a confusion factor between the first class and the second class, the confusion factor representative of a probability that the neural network will compute, for a generic input signal associated with the second class, an output indicating that the generic input signal corresponds to the first class; and
generating a final classification for the input signal using the filter and the first normalization amount.
17. The method of claim 16 , wherein the neural network is part of a gesture control system.
18. The method of claim 16 , wherein the neural network is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for the autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing deep learning operations;
a system implemented using an edge device;
a gaming system;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
19. The method of claim 16 , wherein the generating the final classification for the input signal using the first normalization amount includes subtracting the first normalization amount from a first confidence score.
20. The method of claim 16 , wherein the generating the final classification for the input signal using the filter includes decreasing an amount of classification outputs of the one or more classification outputs used to generate the final classification from a default amount.