IP Library › Granted Patent US 11,660,022
Granted Patent B2
US 11,660,022 · App. 16/949,607 · Granted May 30, 2023

Adaptive skeletal joint smoothing

Inventors: Avihay Assouline (Tel Aviv, IL); Itamar Berger (Hod Hasharon, IL); Gal Dudovitch (Tel Aviv, IL); Matan Zohar (Rishon LeZion, IL)
Assignee: Snap Inc.
A61B5/1113A61B5/744G06T5/002G06T19/006G06T2207/20182G06T2210/41
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,660,022
App. No.
16/949,607
Granted
May 30, 2023
Kind
B2
Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving a video that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the video; tracking movement of the plurality of skeletal joints across a set of frames of the video; and smoothing the movement of a first set of the plurality of skeletal joints between frames in the first set of frames independently of smoothing movement of a second set of the plurality of skeletal joints in the first set of frames.

Claims (45)

1. A method comprising:

receiving, by one or more processors, a video that includes a depiction of a body of a user;

detecting, by the one or more processors, a plurality of skeletal joints of the body depicted in the video;

tracking movement of the plurality of skeletal joints across a set of frames of the video; and

smoothing the movement of a first set of the plurality of skeletal joints between frames in the first set of frames independently of smoothing movement of a second set of the plurality of skeletal joints in the first set of frames.

2. The method of claim 1 , further comprising controlling an amount of smoothing that is applied the first set of the plurality of skeletal joints based on a signal stability parameter.

3. The method of claim 1 , wherein the first set of the plurality of skeletal joints includes a first pair adjacent joints, and wherein the second set of the plurality of skeletal joints includes a second pair adjacent joints.

4. The method of claim 1 , further comprising:

applying a first set of smoothing filters to the first set of the plurality of skeletal joints;

applying a second set of smoothing filters e second set of the plurality skeletal joints; and

adapting a first parameter of the first set of smoothing filters independently of a second parameter of the second set of smoothing filters.

5. The method of claim 4 , wherein the first and second sets of the smoothing filters includes a plurality smoothing filters.

6. The method of claim 4 , further comprising:

computing a first signal stability parameter that represents a first amount of noise resulting from the movement of the first set of the plurality of skeletal joints between the frames in the first set of frames; and

modifying the first parameter based on a value of the first signal stability parameter.

7. The method of claim 6 , further comprising modifying the first parameter to increase an amount of smoothing applied by the first set of smoothing filters to a second set of frames in response to determining that the first amount of noise represented by the first signal stability parameter exceeds a threshold value.

8. The method of claim 7 , further comprising:

computing a second signal stability parameter that represents a second amount of noise resulting from the movement of the second set of the plurality of skeletal joints between the frames in the first set of frames; and

modifying the second parameter based on a value of the second signal stability parameter.

9. The method of claim 8 , further comprising modifying the second parameter to decrease an amount of smoothing applied by the second set of smoothing filters to the second set of frames in response to determining that the second amount of noise represented by the first signal stability parameter is less than the threshold value.

10. The method of claim 4 , wherein applying the first set of smoothing filters to the first set of the plurality of skeletal joints comprises applying a plurality of smoothing filters in parallel to the first set of the plurality of skeletal joints.

11. The method of claim 10 , further comprising interpolating outputs of the plurality of smoothing filters.

12. The method of claim 11 , wherein the interpolating of the outputs is performed based on a signal stability parameter representing an amount of noise between the outputs of the plurality of smoothing filters.

13. The method of claim 1 , further comprising generating an avatar based on smoothing movement of the plurality of skeletal joints.

14. A system comprising:

a processor configured to perform operations comprising:

receiving a video that includes a depiction of a body of a user;

detecting a plurality of skeletal joints of the body depicted in the video;

tracking movement of the plurality of skeletal joints across a set of frames of the video; and

smoothing the movement of a first set of the plurality of skeletal joints between frames in the first set of frames independently of smoothing movement of a second set of the plurality of skeletal joints in the first set of frames.

15. The system of claim 14 , wherein the operations further comprise controlling an amount of smoothing that is applied the first set of the plurality of skeletal joints based on a signal stability parameter.

16. The system of claim 14 , wherein the first set of the plurality of skeletal joints includes a first pair adjacent joints, and wherein the second set of the plurality of skeletal joints includes a second pair adjacent joints.

17. The system of claim 14 , wherein the operations further comprise:

applying a first set of smoothing filters to the first set of the plurality of skeletal joints;

applying a second set of smoothing filters e second set of the plurality skeletal joints; and

adapting a first parameter of the first set of smoothing filters independently of a second parameter of the second set of smoothing filters.

18. The system of claim 17 , wherein the first and second sets of the smoothing filters includes a plurality smoothing filters.

19. The system of claim 17 , wherein the operations further comprise:

computing a first signal stability parameter that represents a first amount of noise resulting from the movement of the first set of the plurality of skeletal joints between the frames in the first set of frames; and

modifying the first parameter based on a value of the first signal stability parameter.

20. A non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving a video that includes a depiction of a body of a user;

detecting a plurality of skeletal joints of the body depicted in the video;

tracking movement of the plurality of skeletal joints across a set of frames of the video; and

smoothing the movement of a first set of the plurality of skeletal joints between frames in the first set of frames independently of smoothing movement of a second set of the plurality of skeletal joints in the first set of frames.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 20, 2023
From: ASSOULINE, AVIHAY; BERGER, ITAMAR; DUDOVITCH, GAL; ZOHAR, MATAN
To: SNAP INC.
Reel/Frame 063390/0437 →
Continuity (2)
Provisional Application 63198553 · Oct 27, 2020
Related Publication 20220125337A1 · Apr 28, 2022
Cited By (4)
US 12,229,860 US 12,318,661 US 12,364,905 US 12,646,268