Granted Patent B2

US 12,374,055 · App. 17/941,612 · Granted Jul 29, 2025

Inventors: Kyle Goodrich (Venice, CA); Maxim Maximov Lazarov (Culver City, CA); Andrew James McPhee (Culver City, CA); Daniel Moreno (Los Angeles, CA)

Assignee: Snap Inc.

G06T19/006G06F3/017G06T7/20G06T7/70G06T19/20G06V10/761G06V20/20G06V40/28G06T2219/2004G06T2219/2021G06V2201/07

Abstract

Claims (75)

1. A method, comprising:

detecting, using one or more hardware processors, from a set of frames, a first gesture corresponding to an open trigger finger gesture;

detecting, using the one or more hardware processors, a location and a position of a representation of a finger from the open trigger finger gesture;

generating, using the one or more hardware processors, a first virtual object based at least in part on the location and the position of the representation of the finger, the first virtual object extending from representation of the finger;

detecting, using the one or more hardware processors, a first collision event corresponding to a first collider of the first virtual object intersecting with a second collider of a second virtual object;

detecting, using the one or more hardware processors, from a second set of frames, a second gesture corresponding to a closed trigger finger gesture;

in response to the first collision event and the detected second gesture, selecting the second virtual object;

rendering, using the one or more hardware processors, the first virtual object as attached to the second virtual object in response to the selecting; and

providing for display, using the one or more hardware processors, the rendered first virtual object as attached to the second virtual object within a first scene.

2. The method of claim 1 , wherein the open trigger finger gesture includes a particular gesture including a thumb and an index finger indicating a firearm or shooting hand sign, and the closed trigger finger gesture includes the thumb and the index finger pointing in a same direction.

3. The method of claim 2 , wherein detecting, from the set of frames, the first gesture corresponding to the open trigger finger gesture comprises:

determining a set of distances between a first set of joints from the thumb and a second set of joints from the index finger; and

determining that at least one distance from the set of the distances is greater than a distance threshold indicating the open trigger finger gesture.

4. The method of claim 2 , wherein detecting, from the second set of frames, the second gesture corresponding to the closed trigger finger gesture comprises:

determining a set of distances between a first set of joints from the thumb and a second set of joints from the index finger; and

determining that at least one distance from the set of the distances is less than a distance threshold indicating the closed trigger finger gesture.

5. The method of claim 1 , wherein the first scene comprises a first representation of a real world scene, the first virtual object, and the second virtual object, wherein the first scene comprises real world video frame data and virtual object data, the virtual object data comprising information utilized for rendering the first virtual object or the second virtual object.

6. The method of claim 1 , further comprising:

detecting a second location and a second position of the representation of the finger from the closed trigger finger gesture;

detecting a change in the second location and the second position of the representation of the finger;

moving the first virtual object to a different location and different position based on the change; and

moving the second virtual object to a second different location and second different position based on the change, wherein moving the first virtual object and moving the second virtual object occurs in concert with both virtual objects.

7. The method of claim 6 , further comprising:

rendering a set of movements based at least in part on the moving the first virtual object and the moving the second virtual object within a first scene; and

providing for display the rendered set of movements.

8. The method of claim 7 , further comprising:

based at least in part on the second different location and the second different position of the second virtual object, indicating a selection of a particular option or a particular item based on a third virtual object, the third virtual object being overlaid by the second virtual object and larger in size than the second virtual object.

9. The method of claim 8 , further comprising:

detecting from a third set of frames, a third gesture corresponding to a second open trigger finger gesture;

rendering the first virtual object as unattached to the second virtual object in response to the detecting; and

providing for display the rendered first virtual object as unattached to the second virtual object.

10. The method of claim 8 , further comprising:

in response to selecting the second virtual object and prior to rendering the set of movements, indicating a first selection of a first particular option or first particular item based on the third virtual object, the first particular option or the first particular item being different than the particular option or the particular item.

11. A system comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to perform operations comprising:

detecting, from a set of frames, a first gesture corresponding to an open trigger finger gesture;

detecting a location and a position of a representation of a finger from the open trigger finger gesture;

generating a first virtual object based at least in part on the location and the position of the representation of the finger, the first virtual object extending from representation of the finger;

detecting a first collision event corresponding to a first collider of the first virtual object intersecting with a second collider of a second virtual object;

detecting, from a second set of frames, a second gesture corresponding to a closed trigger finger gesture;

in response to the first collision event and the detected second gesture, selecting the second virtual object;

rendering the first virtual object as attached to the second virtual object in response to the selecting; and

providing for display the rendered first virtual object as attached to the second virtual object within a first scene.

12. The system of claim 11 , wherein the open trigger finger gesture includes a particular gesture including a thumb and an index finger indicating a firearm or shooting hand sign, and the closed trigger finger gesture includes the thumb and the index finger pointing in a same direction.

13. The system of claim 12 , wherein detecting, from the set of frames, the first gesture corresponding to the open trigger finger gesture comprises:

determining a set of distances between a first set of joints from the thumb and a second set of joints from the index finger; and

determining that at least one distance from the set of the distances is greater than a distance threshold indicating the open trigger finger gesture.

14. The system of claim 12 , wherein detecting, from the second set of frames, the second gesture corresponding to the closed trigger finger gesture comprises:

determining a set of distances between a first set of joints from the thumb and a second set of joints from the index finger; and

determining that at least one distance from the set of the distances is less than a distance threshold indicating the closed trigger finger gesture.

15. The system of claim 11 , wherein the first scene comprises a first representation of a real world scene, the first virtual object, and the second virtual object, wherein the first scene comprises real world video frame data and virtual object data, the virtual object data comprising information utilized for rendering the first virtual object or the second virtual object.

16. The system of claim 11 , wherein the operations further comprise:

detecting a second location and a second position of the representation of the finger from the closed trigger finger gesture;

detecting a change in the second location and the second position of the representation of the finger;

moving the first virtual object to a different location and different position based on the change; and

moving the second virtual object to a second different location and second different position based on the change, wherein moving the first virtual object and moving the second virtual object occurs in concert with both virtual objects.

17. The system of claim 16 , wherein the operations further comprise:

rendering a set of movements based at least in part on the moving the first virtual object and the moving the second virtual object within a first scene; and

providing for display the rendered set of movements.

18. The system of claim 17 , wherein the operations further comprise:

based at least in part on the second different location and the second different position of the second virtual object, indicating a selection of a particular option or a particular item based on a third virtual object, the third virtual object being overlaid by the second virtual object and larger in size than the second virtual object.

19. The system of claim 18 , wherein the operations further comprise:

detecting from a third set of frames, a third gesture corresponding to a second open trigger finger gesture;

rendering the first virtual object as unattached to the second virtual object in response to the detecting; and

providing for display the rendered first virtual object as unattached to the second virtual object.

20. A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to perform operations comprising:

detecting, from a set of frames, a first gesture corresponding to an open trigger finger gesture;

detecting a location and a position of a representation of a finger from the open trigger finger gesture;

generating a first virtual object based at least in part on the location and the position of the representation of the finger, the first virtual object extending from representation of the finger;

detecting a first collision event corresponding to a first collider of the first virtual object intersecting with a second collider of a second virtual object;

detecting, from a second set of frames, a second gesture corresponding to a closed trigger finger gesture;

in response to the first collision event and the detected second gesture, selecting the second virtual object;

rendering the first virtual object as attached to the second virtual object in response to the selecting; and

providing for display the rendered first virtual object as attached to the second virtual object within a first scene.

Assignments (3)

Continuity (1)

References Cited (163)

Cited By (1)

US 12,633,061