Granted Patent B2

US 12,243,173 · App. 18/100,080 · Granted Mar 4, 2025

Inventors: Avihay Assouline (Tel Aviv, IL); Itamar Berger (Hod Hasharon, IL); Gal Dudovitch (Tel Aviv, IL); Matan Zohar (Rishon LeZion, IL)

Assignee: SNAP INC.

G06T19/006G06T7/20G06T7/62G06T7/70G06T19/20G06T2200/24G06T2207/10016G06T2207/30196G06T2219/2016

Abstract

Claims (64)

1. A method comprising:

receiving, by one or more processors, a video that depicts a person;

identifying a given skeletal joint of multiple skeletal joints of the person depicted in the video that moves less relative to other skeletal joints of the multiple skeletal joints;

selecting, as a three-dimensional (3D) reference point, the given skeletal joint that has been identified as moving less relative to the other skeletal joints; and

tracking 3D movement of the person within the video using the selected 3D reference point.

2. The method of claim 1 , further comprising:

computing a 3D position for placement of a virtual item relative to the 3D reference point;

causing to be displayed the virtual item within the video at the 3D position; and

updating the 3D position of the virtual item in the video as the 3D reference point changes based on the 3D movement of the person.

3. The method of claim 1 , further comprising:

determining a size of a body part corresponding to a given skeletal joint of the multiple skeletal joints of the person depicted in the video;

identifying a portion of a skeletal rig of a virtual item that corresponds to the body part; and

modifying a size of the portion of the skeletal rig to match the determined size of the body part corresponding to the given skeletal joint of the multiple skeletal joints of the person depicted in the video.

4. The method of claim 1 , further comprising:

determining that the person has moved from a first 3D position to a second 3D position between a first frame and a second frame of the video;

computing a distance and trajectory of the movement of the person from the first 3D position to the second 3D position; and

moving a virtual item from a third 3D position to a fourth 3D position based on the distance and trajectory of the movement of the person from the first 3D position to the second 3D position.

5. The method of claim 4 , wherein the video is captured by a client device, wherein the second 3D position is closer to the client device than the first 3D position, and wherein the fourth 3D position appears closer to the client device than the third 3D position.

6. The method of claim 5 , further comprising increasing a size of the virtual item proportionally by an amount of increase in size of the person from the first frame to the second frame.

7. The method of claim 4 , wherein the video is captured by a client device, wherein the second 3D position is farther from the client device than the first 3D position, and wherein the fourth 3D position appears farther from the client device than the third 3D position.

8. The method of claim 7 , further comprising decreasing a size of the virtual item proportionally by an amount of increase in size of the person from the first frame to the second frame.

9. The method of claim 1 , further comprising:

determining that the person has moved left or right by a given distance; and

in response to determining that the person has moved left or right by the given distance, moving a virtual item left or right by the given distance.

10. The method of claim 1 , further comprising:

determining that the person has turned around within the video at a given rate; and

in response to determining that the person has turned around within the video at the given rate, turning a virtual item around at the given rate.

11. The method of claim 1 , wherein the person is depicted in a first frame of the video, further comprising:

determining that a depiction of the person has failed to be detected in a second frame of the video;

in response to determining that the depiction of the person has failed to be detected in the second frame of the video, presenting a list of objects detected in the second frame;

receiving input that selects a given object from the list of objects;

computing a reference position of the given object; and

updating a 3D position of a virtual item based on the computed reference position of the given object selected by the received input from the list of objects.

12. The method of claim 1 , comprising:

displaying a first virtual item; and

causing to be displayed a second virtual item that mimics movement and placement of the first virtual item.

13. The method of claim 12 , wherein the second virtual item is placed at a specified position relative to the person and the first virtual item, further comprising receiving input that selects the 3D reference point of the person depicted in the video.

14. The method of claim 1 , further comprising:

determining a size of the person depicted in the video; and

adjusting a size of a virtual item based on the size of the person.

15. The method of claim 1 , wherein the 3D reference point is a first 3D reference point and the given skeletal joint is a first skeletal joint, and wherein a virtual item is a first virtual item, further comprising:

selecting a second 3D reference point corresponding to a second skeletal joint of the multiple skeletal joints of the person depicted in the video; and

updating a position of a second virtual item based on changes to the second 3D reference point independently of updating the 3D position of the first virtual item based on changes to the first 3D reference point.

16. The method of claim 1 , comprising:

duplicating a virtual item to generate a plurality of identical virtual items based on a user selected duplication factor;

causing the plurality of identical virtual items to be evenly distributed around the person depicted in the video; and

animating the plurality of identical virtual items based on the 3D movement of the person.

17. The method of claim 16 , further comprising:

setting a maximum limit to a number of duplicates of the virtual item based on a type associated with the virtual item, a first type of virtual items being associated with a first maximum duplication quantity and a second type of virtual items being associated with a second maximum duplication quantity.

18. A system comprising:

one or more processors configured to perform operations comprising:

receiving a video that depicts a person;

identifying a given skeletal joint of multiple skeletal joints of the person depicted in the video that moves less relative to other skeletal joints of the multiple skeletal joints;

selecting, as a three-dimensional (3D) reference point, the given skeletal joint that has been identified as moving less relative to the other skeletal joints; and

tracking 3D movement of the person within the video using the selected 3D reference point.

19. The system of claim 18 , the operations comprising:

computing a 3D position for placement of a virtual item relative to the 3D reference point;

causing to be displayed the virtual item within the video at the 3D position; and

updating the 3D position of the virtual item in the video as the 3D reference point changes based on the 3D movement of the person.

20. A non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving a video that depicts a person;

identifying a given skeletal joint of multiple skeletal joints of the person depicted in the video that moves less relative to other skeletal joints of the multiple skeletal joints;

selecting, as a three-dimensional (3D) reference point, the given skeletal joint that has been identified as moving less relative to the other skeletal joints; and

tracking 3D movement of the person within the video using the selected 3D reference point.

Assignments (1)

Continuity (2)

References Cited (400)

Cited By (1)

US 12,646,268