Scenario-specific body-part tracking
View Patent ↗A human subject is tracked within a scene of an observed depth image supplied to a general-purpose body-part tracker. The general-purpose body-part tracker is retrained for a specific scenario. The general-purpose body-part tracker was previously trained using supervised machine learning to identify one or more general-purpose parameters to be used by the general-purpose body-part tracker to track a human subject. During a retraining phase, scenario data is received that represents a human training-subject performing an action specific to a particular scenario. One or more special-purpose parameters are identified from the processed scenario data. The special-purpose parameters are selectively used to augment or replace one or more general-purpose parameters if the general-purpose body-part tracker is used to track a human subject performing the action specific to the particular scenario.
1. A computing system, comprising:
a camera input subsystem to receive depth-camera recordings;
one or more memory devices holding instructions to retrain a general-purpose body part tracker;
one or more processors configured to execute the instructions to:
receive via the camera input subsystem, a set of different instances of scenario data, each instance of scenario data including one or more depth images representing a depth-camera recording of a human training-subject performing an action specific to a particular scenario;
iterate over the set of different instances of scenario data with the general-purpose body-part tracker, the general-purpose body-part tracker previously trained using supervised machine learning to identify one or more general-purpose parameters to be used by the general-purpose body-part tracker to track a human subject;
identify one or more special-purpose parameters to be selectively used to augment or replace the one or more general-purpose parameters if the general-purpose body-part tracker is used to track a human subject performing the action specific to the particular scenario; and
associate the one or more special-purpose parameters with a scenario identifier that identifies the particular scenario for which the special-purpose parameters are to be selectively used by the general-purpose body-part tracker to track a human subject.
2. The computing system of claim 1 , wherein the instructions are further executable by the one or more processors to:
receive via the camera input subsystem, input data representing a human subject performing an action;
receive the scenario identifier indicated as being applicable to the input data; and
analyze the input data with the general-purpose body-part tracker using the one or more special-purpose parameters associated with the scenario identifier indicated as being applicable to the input data to identify a body model representing the human subject.
3. The computing system of claim 2 , wherein the one or more special-purpose parameters influence one or more of:
a weighting of one or more classifier functions used to identify the body model of the human subject;
a selection and/or branching of one or more classifier functions used to identify the body model of the human subject;
a weighting of one or more regression functions used to identify a position of at least a portion of the body model of the human subject in two-dimensional or three-dimensional space; and/or
a selection and/or branching of one or more regression functions used to identify a position of at least a portion of the body model of the human subject in two-dimensional or three-dimensional space.
4. The computing system of claim 2 , wherein the one or more special-purpose parameters influence selection of a three-dimensional position of one or more points defining the body model of the human subject.
5. The computing system of claim 2 , wherein the instructions are further executable by the one or more processors to:
retrieve the one or more special-purpose parameters associated with the scenario identifier from a data store responsive to receiving the scenario identifier, the data store forming an element of an application program.
6. The computing system of claim 1 , wherein the instructions are further executable by the one or more processors to:
receive via the camera input subsystem, input data representing a human subject performing an action;
analyze the input data with the general-purpose body-part tracker using the one or more special-purpose parameters associated with the scenario identifier; and
output an indication that the input data is applicable to the particular scenario, the indication including the scenario identifier.
7. The computing system of claim 1 , wherein the instructions to retrain the general-purpose body-part tracker form part of the general-purpose body-part tracker or form a separate retraining shell component interfacing with the general-purpose body-part tracker at runtime.
8. The computing system of claim 1 , wherein the general-purpose body part tracker is retrained to identify the one or more special-purpose parameters without supervision, and the set of different instances of scenario data is received without supervised ground-truth annotations.
9. The computing system of claim 1 , wherein the instructions are further executable by the one or more processors to:
output an indication that one or more additional instances of scenario data are needed to retrain the general-purpose body-part tracker based on convergence and/or divergence of said iterating.
10. The computing system of claim 1 , wherein the instructions are further executable by the one or more processors to:
receive from a currently-active application program, an indication that a currently-observed human subject is performing the action specific to the particular scenario, the indication representative of the scenario identifier; and
use the one or more special-purpose parameters associated with the scenario identifier to enhance body-part tracking of the general-purpose body-part tracker if the general-purpose body-part tracker tracks the currently-observed human performing the action specific to the particular scenario.
11. A method of retraining a body-part tracker, comprising:
receiving, from a currently-active application program, an indication of a particular scenario;
receiving a set of different instances of scenario data, each instance of scenario data including one or more depth images of a human subject performing an action specific to the particular scenario while the currently-active application program is running;
iterating over the set of different instances of scenario data with a general-purpose body-part tracker, the general-purpose body-part tracker previously trained using supervised machine learning to identify one or more general-purpose parameters to be used by the general-purpose body-part tracker to track a human subject; and
identifying one or more special-purpose parameters to be subsequently used by the currently-active application program to augment or replace the one or more general-purpose parameters, the one or more special-purpose parameters based on iteration over the set of different instances of scenario data received while the currently-active application program was running.
12. The method of claim 11 , further comprising:
outputting an indication that one or more additional instances of scenario data are needed based on convergence and/or divergence of the iteration.
13. The method of claim 11 , further comprising:
associate the one or more special-purpose parameters with a scenario identifier that identifies the particular scenario for which the special-purpose parameters are to be selectively used by the general-purpose body-part tracker to track a human subject;
wherein during runtime of the currently-active application program, the currently-active application program indicates applicability of the one or more special-purpose parameters by the scenario identifier.
14. The method of claim 11 , wherein the one or more special-purpose parameters influence one or more of:
a weighting of one or more classifier functions used to identify a body model of the human subject; and/or
a selection and/or branching of one or more classifier functions used to identify the body model of the human subject;
a weighting of one or more regression functions used to identify a position of at least a portion of the body model of the human subject in two-dimensional or three-dimensional space; and/or
a selection and/or branching of one or more regression functions used to identify a position of at least a portion of the body model of the human subject in two-dimensional or three-dimensional space.
15. The method of claim 11 , wherein the one or more special-purpose parameters influence selection of a three-dimensional position of one or more points defining a body model of the human subject.
16. The method of claim 11 , further comprising:
performing said iterating and said identifying of the one or more special-purpose parameters within a runtime of the general-purpose body-part tracker or within a separate retraining shell component interfacing with the general-purpose body-part tracker at runtime.
17. The method of claim 11 , further comprising:
receiving the set of different instances of scenario data without supervised ground-truth annotations; and
identifying the one or more special-purpose parameters without supervised ground-truth annotations.
18. A computing system for tracking a human subject, comprising:
a computing device configured to:
receive via a camera input subsystem of the computing device, input data including a depth image representing a depth-camera recording of a human subject performing an action while a currently-active application program is running;
receive a scenario identifier from the currently-active application program indicated as being applicable to the input data;
retrieve one or more special-purpose parameters associated with the scenario identifier from a data store; and
analyze the input data with a general-purpose body-part tracker using the one or more special-purpose parameters that augment or replace one or more general-purpose parameters to identify a body model representing the human subject performing the action for use by the currently-active application program;
the general-purpose body-part tracker previously trained using unsupervised machine learning to identify the one or more special-purpose parameters after being initially trained using supervised machine learning to identify the one or more general-purpose parameters.
19. The computing system of claim 18 , wherein the computing device is further configured to, during previous training using unsupervised machine learning:
receive via the camera input subsystem, a set of different instances of scenario data, each instance of scenario data representing a human training-subject performing an action specific to a particular scenario;
identify, based on the set of different instances of scenario data, one or more special-purpose parameters to be selectively used to augment or replace the one or more general-purpose parameters if the general-purpose body-part tracker is used to track the human subject performing the action specific to the particular scenario; and
associate the one or more special-purpose parameters with the scenario identifier that identifies the particular scenario for which the special-purpose parameters are to be selectively used by the general-purpose body-part tracker to track the human subject.
20. The computing system claim 18 , wherein the one or more special-purpose parameters influence one or more of:
a weighting of one or more classifier functions used to identify the body model of the human subject;
a selection and/or branching of one or more classifier functions used to identify the body model of the human subject;
a weighting of one or more regression functions used to identify a position of at least a portion of the body model of the human subject in two-dimensional or three-dimensional space; and/or
a selection and/or branching of one or more regression functions used to identify a position of at least a portion of the body model of the human subject in two-dimensional or three-dimensional space.