Granted Patent B2

US 12,299,795 · App. 18/187,996 · Granted May 13, 2025

Inventors: Samuel Carl William Blades (Victoria, CA); Maryam Hajizadeh (Windsor, CA); Philippe Pasquier (Vancouver, CA)

Assignee: Orpyx Medical Technologies Inc.

G06T13/40G06T2200/08

Abstract

Claims (49)

1. A method of generating a virtual avatar representing a user using sensor data from a plurality of sensors mounted to the user, the method comprising:

obtaining sensor data from the plurality of sensors, wherein the sensor data includes:

force sensor data from a plurality of force sensors positioned underfoot of the user; and

IMU data from a plurality of inertial measurement units (IMUs) mounted at respective IMU locations on the user, wherein the IMU data includes a plurality of IMU datasets, and each IMU dataset corresponds to one of the IMU locations;

determining an activity type from the sensor data;

determining an estimated full-body posture of the user by inputting the IMU data, force sensor data, and activity type to a machine learning model trained to estimate full-body posture, wherein the estimated full-body posture specifies a limb position and a limb orientation for each limb of the user;

generating the virtual avatar based on the estimated full-body posture, wherein generating the virtual avatar comprises:

determining a plurality of skeletal lines representing the limbs of the user based on the estimated full-body posture;

generating an avatar base template from the plurality of skeletal lines; and

scaling the avatar base template using anthropometric measurement data for the user to generate the virtual avatar; and

displaying the virtual avatar.

2. The method of claim 1 , further comprising determining motion of the limbs of the user based on limb velocity data and acceleration data from the IMU data.

3. The method of claim 2 , further comprising moving the virtual avatar within a virtual environment in response to the motion of the limbs.

4. The method of claim 1 , wherein the plurality of IMUs includes a pair of foot-mounted IMUs, and the activity type is determined based on the IMU datasets from the pair of foot-mounted IMUs.

5. The method of claim 1 , wherein the machine learning model is trained using training data generated by collecting training sensor data and training motion capture data and calculating a training full-body posture using the training motion capture data.

6. The method of claim 5 , wherein the training sensor data and training motion capture data is collected from one or more users while performing a plurality of specified activity types.

7. The method of claim 5 , wherein:

the training motion capture data from a given training user is used to generate an initial training virtual avatar; and

the initial training virtual avatar is scaled using anthropometric measurement data for the given training user to generate a training virtual avatar.

8. The method of claim 5 , wherein the machine learning model is trained by:

calculating a training estimated full-body posture by inputting the training sensor data to the machine learning model; and

optimizing the machine learning model based on a correlation between the training estimated full-body posture and the training full-body posture.

9. A system for generating a virtual avatar representing a user, the system comprising:

a plurality of sensors mountable to the user, the plurality of sensors including a plurality of force sensors positionable underfoot of the user and a plurality of inertial measurement units (IMUs) mountable at respective IMU locations on the user;

one or more processors communicatively coupled to the plurality of sensors; and

a non-transitory storage memory storing a machine learning model trained to estimate a full-body posture of the user;

wherein the one or more processors is configured to:

obtain sensor data from the plurality of sensors, wherein the sensor data includes:

force sensor data from the plurality of force sensors; and

IMU data from the plurality of inertial measurement units (IMUs),

wherein the IMU data includes a plurality of IMU datasets, and each IMU dataset corresponds to one of the IMU locations;

determine an activity type from the sensor data;

determine an estimated full-body posture by inputting the IMU data, force sensor data, and activity type to the machine learning model, wherein the estimated full-body posture specifies a limb position and a limb orientation for each limb of the user;

generate the virtual avatar based on the estimated full-body posture, wherein the one or more processors is configured to generate the virtual avatar by:

determining a plurality of skeletal lines representing the limbs of the user based on the estimated full-body posture;

generating an avatar base template from the plurality of skeletal lines; and

scaling the avatar base template using anthropometric measurement data for the user to generate the virtual avatar; and

display the virtual avatar.

10. The system of claim 9 , wherein the one or more processors is configured to determine motion of the limbs of the user based on limb velocity data and acceleration data from the IMU data.

11. The system of claim 10 , wherein the one or more processors is configured to move the virtual avatar within a virtual environment in response to the motion of the limbs.

12. The system of claim 9 , wherein the plurality of IMUs includes a pair of foot-mounted IMUs, and the one or more processors is configured to determine the activity type based on the IMU datasets from the pair of foot-mounted IMUs.

13. The system of claim 9 , wherein the machine learning model is trained using training data generated by collecting training sensor data and training motion capture data and calculating a training full-body posture using the training motion capture data.

14. The system of claim 13 , wherein the training sensor data and training motion capture data is collected from one or more users while performing a plurality of specified activity types.

15. The system of claim 13 , wherein:

the training motion capture data from a given training user is used to generate an initial training virtual avatar; and

the initial training virtual avatar is scaled using anthropometric measurement data for the given training user to generate a training virtual avatar.

16. The system of claim 13 , wherein the machine learning model is trained by:

calculating a training estimated full-body posture by inputting the training sensor data to the machine learning model; and

optimizing the machine learning model based on a correlation between the training estimated full-body posture and the training full-body posture.

Assignments (3)

Continuity (2)

References Cited (38)