IP Library Granted Patent US 12,087,008
Granted Patent B2
US 12,087,008 · App. 17/326,341 · Granted Sep 10, 2024

User analytics using a camera device and associated systems and methods

Inventors: Qi Zhang (Tseung Kwan O, HK); Wing Hung Chan (Kowloon, HK); Keng Fai Lee (Cupertino, CA); Long Mak (Shatin, HK)
Assignee: NEX Team Inc.
G06T7/70G06T5/20G06T5/70G06T7/20G06T7/50G06T7/60G09B19/0038G06T2207/20081G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,087,008
App. No.
17/326,341
Granted
Sep 10, 2024
Kind
B2
Abstract

Methods and systems for performing location and movement determination in a three-dimensional environment using a device with a camera are disclosed. The methods and systems perform the steps of capturing an image of a user using the camera of the computing device, identifying the user from the image using a first algorithm, determining a first physical information associated with the user using a first algorithm, determining a second physical information of the user based on the first physical information and an input parameter of the user using a search process, and determining the location of the user in the environment based on the first physical information and the second physical information.

Claims (57)

1. A computer-implemented method for performing location and movement determination in a three-dimensional environment using a computing device having a camera, comprising:

capturing an image of a user using the camera of the computing device;

identifying the user from the image using a first algorithm;

determining a first physical information, the first physical information comprising a noisy pose data of the user, using a second algorithm;

applying a filter to the noisy pose data to generate a filtered first physical information;

determining a second physical information, the second physical information comprising a depth of the user with respect to the camera, based on the first physical information and an input parameter of the user using a search process; and

determining the location of the user in the environment based on the filtered first physical information and the second physical information.

2. The computer-implemented method of claim 1 , further comprising:

determining a first pose size of the user from a first user pose and determining a second pose size of the user from a second user pose; and

determining the depth based on one or more of the camera's center, a distance to an imaging plane, an input height of the user, the first pose size, and the second pose size.

3. The computer-implemented method of claim 2 , further comprising:

determining the first pose size based on a length of pose limbs associated with the first user pose.

4. The computer-implemented method of claim 1 , further comprising:

determining a movement associated with the user based on the first physical information.

5. The computer-implemented method of claim 1 , further comprising:

determining a direction of gravity using the computing device;

determining a second location of a portion of the user's body in the three-dimensional environment; and

determining, based on the second location, a distance between the camera and the second location.

6. The computer-implemented method of claim 5 , further comprising:

determining, based on the second physical information, a reference plane in the camera's coordinate system.

7. A non-transitory storage medium for performing location and movement determination in a three-dimensional environment using a computing device having a camera, the storage medium comprising machine-readable program code that cause a processor to:

capture an image of a user using the camera of the computing device;

identify the user from the image using a first algorithm;

determine a first physical information, the first physical information comprising a noisy pose data of the user, using a second algorithm;

apply a filter to the noisy pose data to generate a filtered first physical information;

determine a second physical information, the second physical information comprising a depth of the user with respect to the camera, based on the first physical information and an input parameter of the user using a search process; and

determine the location of the user in the environment based on the filtered first physical information and the second physical information.

8. The non-transitory storage medium of claim 7 , further comprising program code to:

determine a first pose size of the user from a first user pose and determine a second pose size of the user from a second user pose; and

determine the depth based on one or more of the camera's center, a distance to an imaging plane, an input height of the user, the first pose size, and the second pose size.

9. The non-transitory storage medium of claim 7 , further comprising program code to:

determine a movement associated with the user based on the first physical information.

10. The non-transitory storage medium of claim 7 , further comprising program code to:

determine a direction of gravity using the computing device;

determine a second location of a portion of the user's body in the three-dimensional environment; and

determine, based on the second location, a distance between the camera and the second location.

11. The non-transitory storage medium of claim 10 , further comprising program code to:

determine, based on the second physical information, a reference plane in the camera's coordinate system.

12. A computing device for performing location and movement determination in a three-dimensional environment, comprising:

a camera device;

a processor; and

a non-transitory memory storing program code thereon, the program code executable by the processor to:

capture an image of a user using the camera device of the computing device;

identify the user from the image using a first algorithm;

determine a first physical information, the first physical information comprising a noisy pose data of the user, using a second algorithm;

apply a filter to the noisy pose data to generate a filtered first physical information;

determine a second physical information, the second physical information comprising a depth of the user with respect to the camera device, based on the first physical information and an input parameter of the user using a search process; and

determine the location of the user in the environment based on the filtered first physical information and the second physical information.

13. The computing device of claim 12 , further comprising program code to:

determine a first pose size of the user from a first user pose and determining a second pose size of the user from a second user pose; and

determine the depth based on one or more of the camera device's center, a distance to an imaging plane, an input height of the user, the first pose size, and the second pose size.

14. The computing device of claim 12 , further comprising program code to:

determine a direction of gravity using the computing device;

determine the location of a portion of the user's body in the three-dimensional environment; and

determine, based on the location, a distance between the camera device and the location.

15. The computing device of claim 14 , further comprising program code to:

determine, based on the second physical information, a reference plane in the camera device's coordinate system.

Assignments (2)
SECURITY INTEREST Recorded Oct 9, 2025
From: NEX TEAM INC.
To: SILVER LAKE WATERMAN FUND III, L.P., AS AGENT
Reel/Frame 072524/0882 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2021
From: ZHANG, QI; CHAN, WING HUNG; LEE, KENG FAI; MAK, LONG
To: NEX TEAM INC.
Reel/Frame 056308/0794 →
Continuity (4)
Continuation 17124520 · Dec 17, 2020
Provisional Application 62975710 · Feb 12, 2020
Provisional Application 62950949 · Dec 20, 2019
Related Publication 20210272312A1 · Sep 2, 2021