System for action recognition error detection and correction using probabilistic signal temporal logic
Described is a system for action recognition error detection and correction using probabilistic signal temporal logic. The system is initiated by training an action recognition system to generate true positive (TP)/false positive (FP) axioms. Thereafter, the system ca be used to classify one or more actions in a video sequence as true action classifications by using the TP/FP axioms to remove false action classifications. With the remaining true classifications, a device can be controlled given the situation and relevant true classification.
1. A system for action recognition error detection and correction, the system comprising:
one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
training an action recognition system to generate true positive (TP)/false positive (FP) axioms;
classifying one or more actions in a video sequence as true action classifications by using the TP/FP axioms to remove false action classifications; and
controlling a device based on the true action classifications.
2. The system as set forth in claim 1 , where training the action recognition system further comprises operations of:
receiving a training video sequence;
generating object action labels from object actions within the training video sequence;
evaluating the object action labels using ground truth action labels to generate TP/FP statistics;
converting the TP/FP statistics into probabilistic signal temporal logic (PTSL) based axioms; and
generating the TP/FP axioms from the PTSL based axioms.
3. The system as set forth in claim 2 , wherein classifying one or more actions in the video sequence as true action classifications further comprises operations of:
receiving a video sequence;
generating a set of action labels from the video sequence;
evaluating the set of action labels based on the TP/FP axioms to identify false positive action labels;
removing the false positive action labels from the set of action labels, leaving true action classifications.
4. The system as set forth in claim 3 , wherein controlling the device based on the true action classifications includes causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object.
5. The system as set forth in claim 4 , wherein causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object includes causing the autonomous vehicle to steer to avoid the collision.
6. The system as set forth in claim 1 , wherein controlling the device based on the true action classifications includes causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object.
7. The system as set forth in claim 6 , wherein causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object includes causing the autonomous vehicle to steer to avoid the collision.
8. A computer program product for action recognition error detection and correction, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
training an action recognition system to generate true positive (TP)/false positive (FP) axioms;
classifying one or more actions in a video sequence as true action classifications by using the TP/FP axioms to remove false action classifications; and
controlling a device based on the true action classifications.
9. The computer program product as set forth in claim 8 , where training the action recognition system further comprises operations of:
receiving a training video sequence;
generating object action labels from object actions within the training video sequence;
evaluating the object action labels using ground truth action labels to generate TP/FP statistics;
converting the TP/FP statistics into probabilistic signal temporal logic (PTSL) based axioms; and
generating the TP/FP axioms from the PTSL based axioms.
10. The computer program product as set forth in claim 9 , wherein classifying one or more actions in the video sequence as true action classifications further comprises operations of:
receiving a video sequence;
generating a set of action labels from the video sequence;
evaluating the set of action labels based on the TP/FP axioms to identify false positive action labels;
removing the false positive action labels from the set of action labels, leaving true action classifications.
11. The computer program product as set forth in claim 10 , wherein controlling the device based on the true action classifications includes causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object.
12. The computer program product as set forth in claim 10 , wherein causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object includes causing the autonomous vehicle to steer to avoid the collision.
13. The computer program product as set forth in claim 8 , wherein controlling the device based on the true action classifications includes causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object.
14. The computer program product as set forth in claim 13 , wherein causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object includes causing the autonomous vehicle to steer to avoid the collision.
15. A computer implemented method for action recognition error detection and correction, the method comprising an act of:
causing one or more processers to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
training an action recognition system to generate true positive (TP)/false positive (FP) axioms;
classifying one or more actions in a video sequence as true action classifications by using the TP/FP axioms to remove false action classifications; and
controlling a device based on the true action classifications.
16. The computer implemented method as set forth in claim 15 , where training the action recognition system further comprises operations of:
receiving a training video sequence;
generating object action labels from object actions within the training video sequence;
evaluating the object action labels using ground truth action labels to generate TP/FP statistics;
converting the TP/FP statistics into probabilistic signal temporal logic (PTSL) based axioms; and
generating the TP/FP axioms from the PTSL based axioms.
17. The computer implemented method as set forth in claim 16 , wherein classifying one or more actions in the video sequence as true action classifications further comprises operations of:
receiving a video sequence;
generating a set of action labels from the video sequence;
evaluating the set of action labels based on the TP/FP axioms to identify false positive action labels;
removing the false positive action labels from the set of action labels, leaving true action classifications.
18. The computer implemented method as set forth in claim 15 , wherein controlling the device based on the true action classifications includes causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object.
19. The computer implemented method as set forth in claim 18 , wherein causing an autonomous vehicle to initiate a maneuver to avoid a collision with an object includes causing the autonomous vehicle to steer to avoid the collision.