Augmented reality experiences of color palettes in a messaging system

1. A method, comprising:

receiving image data captured by a camera of a client device, the image data including a representation of a physical item, the physical item comprising a beauty product;

analyzing the image data to determine an object corresponding to the physical item, the analyzing based at least in part on a recognition process performed on the image data;

identifying a set of colors corresponding to a set of regions of the determined object;

receiving second image data captured by the camera of the client device;

analyzing second image data to detect a second object corresponding to a representation of a particular body part of a user;

generating augmented reality content based at least in part on the identified set of colors and the detected second object; and

causing display, at a client device, the augmented reality content applied to the detected second object.

2. The method of claim 1 , wherein the beauty product corresponds to a makeup palette, and the recognition process further comprises:

classifying, using a first set of image classifiers, the image data to determine that the object includes a physical identification indicator containing text, a barcode pattern, or a QR code pattern; and

classifying, using a second set of image classifiers, the image data to perform object classification of the object based on a set of 3D models or images.

3. The method of claim 2 , wherein the identified set of colors corresponds to a second set of colors included in a portion of the image data including the set of regions of the determined object.

4. The method of claim 3 , wherein each region of the set of region is associated with a particular color.

5. The method of claim 1 , where the particular body part of the user comprises a forearm.

6. The method of claim 1 , wherein identifying the set of colors further comprises:

differentiating a set of colors values of pixels in a particular region from the set of regions of the determined object; and

determining a representative color value based on an average of the set of colors.

7. The method of claim 6 , wherein generating the augmented reality content further comprises:

generating a shape based on the representative color value; and

mapping the shape to a surface of the second object corresponding to the representation of the particular body part.

8. The method of claim 1 , further comprising:

determining a set of textures based on pixels from the set of regions of the determined object.

9. The method of claim 8 , wherein generating the augmented reality content further comprises:

performing a texture transfer process based on the set of textures to a surface of the second object corresponding to the representation of the particular body part.

10. The method of claim 1 , wherein analyzing the second image data further comprises:

performing a body feature detection process to detect a first body feature; and

identifying a particular body part corresponding to the detected first body feature.

11. A system comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to perform operations comprising:

receiving image data captured by a camera of a client device, the image data including a representation of a physical item, the physical item comprising a beauty product;

analyzing the image data to determine an object corresponding to the physical item, the analyzing based at least in part on a recognition process performed on the image data;

identifying a set of colors corresponding to a set of regions of the determined object;

receiving second image data captured by the camera of the client device;

analyzing second image data to detect a second object corresponding to a representation of a particular body part of a user;

generating augmented reality content based at least in part on the identified set of colors and the detected second object; and

causing display, at a client device, the augmented reality content applied to the detected second object.

12. The system of claim 11 , wherein the beauty product corresponds to a makeup palette.

13. The system of claim 12 , wherein the identified set of colors corresponds to a second set of colors included in a portion of the image data including the set of regions of the determined object.

14. The system of claim 13 , wherein each region of the set of region is associated with a particular color.

15. The system of claim 11 , where the particular body part of the user comprises a forearm.

16. The system of claim 11 , wherein identifying the set of colors further causes the processor to perform further operations comprising:

differentiating a set of colors values of pixels in a particular region from the set of regions of the determined object; and

determining a representative color value based on an average of the set of colors.

17. The system of claim 16 , wherein generating the augmented reality content further causes the processor to perform further operations comprising:

generating a shape based on the representative color value; and

mapping the shape to a surface of the second object corresponding to the representation of the particular body part.

18. The system of claim 11 , wherein the memory includes further instructions, which further cause the processor to perform further operations comprising:

determining a set of textures based on pixels from the set of regions of the determined object.

19. The system of claim 18 , wherein generating the augmented reality content further causes the processor to perform further operations comprising:

performing a texture transfer process based on the set of textures to a surface of the second object corresponding to the representation of the particular body part.

20. A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to perform operations comprising:

receiving image data captured by a camera of a client device, the image data including a representation of a physical item, the physical item comprising a beauty product;

analyzing the image data to determine an object corresponding to the physical item;

identifying a set of colors corresponding to a set of regions of the determined object;

receiving second image data captured by the camera of the client device;

analyzing second image data to detect a second object corresponding to a representation of a particular body part of a user;

generating augmented reality content based at least in part on the identified set of colors and the detected second object; and

causing display, at a client device, the augmented reality content applied to the detected second object.