Controlling interactive fashion based on body gestures
Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a person wearing a fashion item; generating a segmentation of the fashion item worn by the person depicted in the video; applying one or more augmented reality elements to the fashion item worn by the person based on the segmentation of the fashion item worn by the person; detecting a gesture performed by the person in the video; and modifying the one or more augmented reality elements that have been applied to the fashion item worn by the person based on the gesture performed by the person.
1. A method comprising:
applying one or more augmented reality elements to an object depicted in a video;
determining that a gesture performed by the object causes a first portion of the object to overlap a second portion of the object; and
in response to the determining, adjusting one or more attributes of the one or more augmented reality elements.
2. The method of claim 1 , further comprising:
receiving the video that includes a depiction of the object wearing a fashion item, the gi second portion comprising the fashion item;
generating a segmentation of the fashion item; and
applying the one or more augmented reality elements to the fashion item based on the segmentation of the fashion item worn.
3. The method of claim 1 , wherein the given second portion comprises a shirt, further comprising:
detecting a hand of the object in the video;
determining a position of the hand of the object in relation to a position of the shirt; and
adjusting a color of the shirt based on the position of the hand.
4. The method of claim 1 , wherein the first portion of the object comprises a hand, further comprising:
determining that a position of the hand overlaps a first portion of a segmentation of the second portion of the object; and
in response to determining that the position of the hand overlaps the first portion, adjusting a color of the second portion to be a first color.
5. The method of claim 4 , further comprising:
determining that the position of the hand overlaps a second portion of the segmentation of the second portion of the object; and
in response to determining that the position of the hand overlaps the second portion, adjusting the color of the second portion to be a second color.
6. The method of claim 1 , further comprising:
generating a segmentation of the object depicted in the video;
identifying a background in the video based on the segmentation;
detecting a hand of the object in the video;
determining a position of the hand of the object in relation to a position of the segmentation; and
adjusting the identified background of the video based on the position of the hand of the object.
7. The method of claim 1 , wherein detecting the gesture comprises:
detecting movement of hands of the object depicted in the video along a given direction; and
in response detecting movement of hands of the object depicted in the video along the given direction, modifying an animation comprising the one or more augmented reality elements.
8. The method of claim 6 , further comprising:
moving the one or more augmented reality elements along a given direction away from a position of the hands, the one or more augmented reality elements being moved along an arm of the object.
9. The method of claim 1 , further comprising:
determining a pose of the object depicted in the video; and
adjusting a pose of the one or more augmented reality elements to match the pose of the object depicted in the video.
10. The method of claim 1 , further comprising:
detecting a physical logo or emblem on the given second portion;
determining a pose of the object depicted in the video; and
generating, as the one or more augmented reality elements, an augmented reality version of the physical logo or emblem.
11. The method of claim 10 , further comprising:
adjusting a pose of the augmented reality version of the physical logo or emblem to match the pose of the object.
12. The method of claim 11 , further comprising:
overlaying the augmented reality version of the physical logo or emblem comprising the adjusted pose on top of the physical logo or emblem on the second portion.
13. The method of claim 1 , further comprising:
detecting a pose of a hand of the object depicted in the video;
generating, as the one or more augmented reality elements, an augmented reality version of the hand in the detected pose; and
displaying the augmented reality version of the hand in the detected pose on the given second portion.
14. The method of claim 1 , further comprising:
detecting an image on the second portion; and
modifying an aspect ratio of the image on the second portion in response to detecting the gesture.
15. The method of claim 14 , wherein detecting the gesture comprises:
detecting a pair of hands positioned over the image; and
determining movement of a first hand of the pair of hands in relation to a second hand of the pair of hands, wherein movement determined to be in a first manner causes the aspect ratio of the image to be modified by a first amount, and wherein movement determined to be in a second manner causes the aspect ratio of the image to be modified by a second amount.
16. A system comprising:
at least one processor of a device; and
a memory component having instructions stored thereon, when executed by the at least one processor, causes the at least one processor to perform operations comprising:
applying one or more augmented reality elements to an object depicted in a video;
determining that a gesture performed by the object causes a first portion of the object to overlap a second portion of the object; and
in response to the determining, adjusting one or more attributes of the one or more augmented reality elements.
17. The system of claim 16 , the operations further comprising:
receiving the video that includes a depiction of the object wearing a fashion item, the second portion comprising the fashion item;
generating a segmentation of the fashion item; and
applying the one or more augmented reality elements to the fashion item based on the segmentation of the fashion item worn.
18. The system of claim 16 , wherein the second portion comprises a shirt, the operations further comprising:
detecting a hand of the object in the video;
determining a position of the hand of the object in relation to a position of the shirt; and
adjusting a color of the shirt based on the position of the hand.
19. The system of claim 16 , wherein the first portion of the object comprises a hand, the operations further comprising:
determining that a position of the hand overlaps a first portion of a segmentation of the second portion of the object; and
in response to determining that the position of the hand overlaps the first portion, adjusting a color of the second portion to be a first color.
20. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor of a device, cause the at least one processor to perform operations comprising:
applying one or more augmented reality elements to an object depicted in a video;
determining that a gesture performed by the object causes a first portion of the object to overlap a second portion of the object; and
in response to the determining, adjusting one or more attributes of the one or more augmented reality elements.