Disentangling human dynamics for pedestrian locomotion forecasting with noisy supervision
A method for predicting spatial positions of several key points on a human body in the near future in an egocentric setting is described. The method includes generating a frame-level supervision for human poses. The method also includes suppressing noise and filling missing joints of the human body using a pose completion module. The method further includes splitting the poses into a global stream and a local stream. Furthermore, the method includes combining the global stream and the local stream to forecast future human locomotion.
1. A method for predicting spatial positions of several key points on a human body in the near future in an egocentric setting, comprising:
generating a frame-level supervision for human poses of a pedestrian;
suppressing noise and filling missing joints of the human body of the pedestrian using a pose completion module;
splitting the concurrent motion of the joints of the pedestrian into a global stream and a local stream, in which the global motion stream models large scale movements of a position of the pedestrian with respect to a camera of a vehicle;
encoding, by the local stream, a motion of the human body of the pedestrian with respect to the global stream;
capturing a depth change of an overall pose size and movement of different joints of the human body of the pedestrian; and
combining the global stream and the local stream to forecast future human locomotion of the pedestrian based on the depth change of the overall pose size and movement of different joints of the human body of the pedestrian.
2. The method of claim 1 , further comprising forecasting the global stream based on one joint of the human body of the pedestrian.
3. The method of claim 1 , in which an architecture of a forecaster of the local stream is different from an architecture of a forecaster of the global stream.
4. The method of claim 1 , further comprising estimating motion between consecutive frames due to a motion of an ego vehicle.
5. A non-transitory computer-readable medium having program code recorded thereon for predicting spatial positions of several key points on a human body in the near future in an egocentric setting, the program code being executed by a processor and comprising:
program code to generate a frame-level supervision for human poses of a pedestrian;
program code to suppress noise and filling missing joints of the human body of the pedestrian using a pose completion module;
program code to split the concurrent motion of the joints of the pedestrian into a global stream and a local stream, in which the global motion stream models large scale movements of a position of the pedestrian with respect to a camera of a vehicle;
program code to encode, by the local stream, a motion of the human body of the pedestrian with respect to the global stream;
program code to capture a depth change of an overall pose size and movement of different joints of the human body of the pedestrian; and
program code to combine the global stream and the local stream to forecast future human locomotion of the pedestrian based on the depth change of the overall pose size and movement of different joints of the human body of the pedestrian.
6. The non-transitory computer-readable medium of claim 5 , further comprising program code to forecast the global stream based on one joint of the human body of the pedestrian.
7. The non-transitory computer-readable medium of claim 5 , in which an architecture of a forecaster of the local stream is different from an architecture of a forecaster of the global stream.
8. The non-transitory computer-readable medium of claim 5 , further comprising program code to estimate motion between consecutive frames due to a motion of an ego vehicle.
9. A system for predicting spatial positions of several key points on a human body in the near future in an egocentric setting, the system comprising:
a memory; and
at least one processor, the at least one processor configured:
to generate a frame-level supervision for human poses of a pedestrian;
to suppress noise and filling missing joints of the human body of the pedestrian using a pose completion module;
to split the concurrent motion of the joints of the pedestrian into a global stream and a local stream, in which the global motion stream models large scale movements of a position of the pedestrian with respect to a camera of a vehicle;
to encode, by the local stream, a motion of the human body of the pedestrian with respect to the global stream;
to capture a depth change of an overall pose size and movement of different joints of the human body of the pedestrian; and
to combine the global stream and the local stream to forecast future human locomotion of the pedestrian based on the depth change of the overall pose size and movement of different joints of the human body of the pedestrian.
10. The system of claim 9 , in which the at least one processor is further configured to forecast the global stream based on one joint of the human body of the pedestrian.
11. The system of claim 9 , in which an architecture of a forecaster of the local stream is different from an architecture of a forecaster of the global stream.