Method and head mounted display device for pose estimation in extended reality environment

1 . A method for pose estimation of a Head Mounted Display (HMD) device, the method comprising:

receiving motion data from one or more motion sensors provided on the HMD device, the motion data comprising relevant motion data and irrelevant motion data acquired at a time a user is interacting with at least one extended reality (XR) application using the HMD device;

obtaining motion embedding vectors for the HMD device corresponding to the at least one XR application based on the relevant motion data corresponding to the at least one XR application;

generating a filtered motion data based on the motion embedding vectors corresponding to the at least one XR application and the motion data from the one or more motion sensors; and

estimating a pose of the HMD device based on the filtered motion data.

2 . The method of claim 1 , wherein the motion embedding vectors are obtained by inputting, to a contrastive learning artificial intelligence (AI) model, the motion data and application information corresponding to the at least one XR application.

3 . The method of claim 2 , further comprises:

determining a pose error based loss between estimated pose and a ground truth pose acquired by the one or more motion sensors; and

training a refinement artificial intelligence (AI) model based on the pose error based loss and the motion embedding vectors.

4 . The method of claim 3 , wherein the refinement AI model is a light weight model in comparison to the contrastive learning AI model.

5 . The method of claim 1 , wherein each of the motion embedding vectors comprises floating point numbers that represent feature value indicating size of the respective motion embedding vector.

6 . The method of claim 2 , wherein the application information comprises a name of the at least one XR application, a category of the at least one XR application, or a version of the at least one XR application.

7 . The method of claim 2 , wherein the contrastive learning AI model comprises:

obtaining a first embedding vector for a first application based on first motion data corresponding to the HMD device;

obtaining a second embedding vector for a second application based on second motion data corresponding to the HMD device;

obtaining a distance between the first embedding vector for the first application and the second embedding vector for the second application; and

learning a degree of relativeness between the first application and the second application based on the distance between the first embedding vector for the first application and the second embedding vector for the second application.

8 . The method of claim 3 , wherein the generating the filtered motion data comprises:

inputting the received motion data and the motion embedding vectors to the refinement AI model;

determining, by the refinement AI model, at least one motion embedding distance based on average motion embedding vectors and at least one embedding vectors of plurality of applications;

determining, by the refinement AI model, at least one motion data deviation based on average of motion data received from the one or more motion sensors and the motion data received from the one or more motion sensors;

determining a motion data correction value based on the at least one motion embedding distance, the at least one motion data deviation and gradients of motion data from the refinement AI model;

determining an updated motion data correction value based on a base correction value and the motion data correction value, the base correction value determined using trained base refinement model based on previous motion data received from the one or more motion sensors; and

generating, by the refinement AI model, the filtered motion data based on the motion data received from the one or more motion sensors and the updated motion data correction value.

9 . A Head Mounted Display (HMD) device comprising:

a memory storing one or more instructions;

one or more motion sensors;

a processor connected to the memory and the one or more motion sensors; and

a pose estimation controller connected to the processor and configured to:

receive motion data from the one or more motion sensors, the motion data comprising relevant motion data and irrelevant motion data acquired at a time a user is interacting with at least one extended reality (XR) application using the HMD device;

obtain motion embedding vectors for the HMD device corresponding to the at least one XR application based on the relevant motion data corresponding to the at least one XR application;

generate a filtered motion data based on the motion embedding vectors corresponding to the at least one XR application and motion data from the one or more motion sensors; and

estimate pose of the HMD device based on the filtered motion data.

10 . The HMD device of claim 9 , wherein the motion embedding vectors are obtained by inputting, to a contrastive learning AI model, the motion data and application information corresponding to the at least one XR application.

11 . The HMD device of claim 10 , wherein the pose estimation controller is further configured to:

determine a pose error based loss between the estimated pose and a ground truth pose acquired by the one or more motion sensors; and

train a refinement artificial intelligence (AI) model based on the pose error based loss and the motion embedding vectors.

12 . The HMD device of claim 11 , wherein the refinement AI model is a light weight model in comparison to the contrastive learning AI model.

13 . The HMD device of claim 9 , wherein each of the motion embedding vectors comprises floating point numbers that represent feature value indicating size of the respective motion embedding vector.

14 . The HMD device of claim 10 , wherein the application information about the at least one XR application comprises a name of the at least one XR application, a category of the at least one XR application, and a version of the at least one XR application.

15 . The HMD device of claim 10 , wherein the contrastive learning AI model is configured to:

obtain a first embedding vector for a first application based on first motion data corresponding to the HMD device;

obtain a second embedding vector for a second application based on second motion data corresponding to the HMD device;

obtain a distance between the first embedding vector for the first application and the second embedding vector for the second application; and

learn a degree of relativeness between the first application and the second application based on the distance between the first embedding vector for the first application and the second embedding vector for the second application.

16 . The method of claim 1 , wherein the filtered motion data is obtained by providing the motion embedding vectors and the motion data as separate inputs into an artificial intelligence (AI) model.

17 . The method of claim 1 , wherein the filtered motion data is obtained by inputting the motion embedding vectors and the motion data into a refinement artificial intelligence (AI) model trained based on a pose error.

18 . The HMD device of claim 10 , wherein the filtered motion data is obtained by providing the motion embedding vectors and the motion data as separate inputs into an artificial intelligence (AI) model.

19 . The HMD device of claim 10 , wherein the filtered motion data is obtained by inputting the motion embedding vectors and the motion data into a refinement artificial intelligence (AI) model trained based on a pose error.