Methods and systems for an e-commerce platform with augmented reality application for display of virtual objects
Disclosed are systems and methods for augmenting a customer image with at least one virtual object. One or more customer images depicting a customer environment are received from a customer computing device and 3D feature data for the customer images is determined. A virtual object is determined which corresponds to a desired merchant item. A positioning signal is received which corresponds to a desired location of virtual object in the customer environment, and a corresponding first location in each customer image at which to overlay the virtual object using the positioning signal is determined. An appropriate size and orientation of the virtual object is determined for each customer image based on corresponding 3D feature data, causing an overlay of an appropriately sized and oriented virtual object at the corresponding first location in customer images.
1. A computer-implemented method for augmenting a plurality of customer images with at least one virtual object, the method comprising:
receiving the plurality of customer images from a first computing device, wherein the plurality of customer images depicts portions of a customer environment as captured from different perspectives;
determining 3D feature data corresponding to each of the plurality of customer images, wherein respective defined coordinate systems for 3D spaces corresponding to each of the plurality of customer images are defined and oriented relative to the coordinate systems of other customer images of the plurality of customer images;
receiving a selection of a merchant item for display;
determining the at least one virtual object corresponding to the selection of the merchant item;
receiving a positioning signal corresponding to a desired location of the at least one virtual object in the customer environment, the positioning signal identifying a position for the at least one virtual object relative to a first one of the plurality of customer images;
determining a first location in the respective defined coordinate system for the 3D space corresponding to the first one of the plurality of customer images based on the position for the at least one virtual object identified by the positioning signal;
determining an appropriate size and orientation for depicting the at least one virtual object in the first one of the plurality of customer images based on the first location and the 3D feature data corresponding to the first one of the plurality of customer images;
determining, based on the first location, first corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the plurality of customer images other than first one of the plurality of customer images based on orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the first one of the plurality of customer images;
determining appropriate sizes and orientations for depicting the at least one virtual object in the plurality of customer images other than the first one of the plurality of customer images based on the first corresponding locations and the 3D feature data corresponding to the plurality of customer images other than the first one of the plurality of customer images;
causing overlays of the appropriately sized and oriented depictions of the at least one virtual object at the first location and the first corresponding locations for each of the plurality of customer images;
receiving a re-positioning signal indicative of a desired re-positioning of the at least one virtual object from a second computing device, the re-positioning signal identifying an updated position for the at least one virtual object relative to a second one of the plurality of customer images;
determining a second location in the respective defined coordinate system for the 3D space corresponding to the second one of the plurality of customer images at which to overlay the at least one virtual object in response to the re-positioning signal;
determining, based on the second location, second corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the customer images other than second one of the plurality of customer images based on the orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the second one of the plurality of customer images;
determining appropriate sizes and orientations for depicting the at least one virtual object in the plurality of customer images based on the second location, the second corresponding locations, and the 3D feature data corresponding to the plurality of customer images; and
causing updated overlays of the appropriately sized and oriented depictions of the at least one virtual object at the second location and the second corresponding locations in the plurality of customer images.
2. The method of claim 1 , wherein the causing the overlays includes transmitting at least one of the appropriately sized and oriented depictions of the at least one virtual object or the first location to the first computing device.
3. The method of claim 1 , further comprising creating an augmented image for the first one of the plurality of customer images with the overlay of the appropriately sized and oriented depiction of the at least one virtual object at the first location for display on the first computing device.
4. The method of claim 3 , further comprising creating an augmented image for the first one of the plurality of customer images with the overlay of the appropriately sized and oriented depiction of the at least one virtual object at the first location for display on the second computing device.
5. The method of claim 4 , wherein the first computing device and the second computing device are in a peer to peer relationship.
6. The method of claim 1 , wherein the causing the updated overlays includes transmitting at least one of the appropriately sized and oriented depictions of the at least one virtual object and the second location to the first computing device.
7. The method of claim 1 , further comprising creating an updated augmented image for the second one of the plurality of customer images with the updated overlays for display on the first computing device.
8. The method of claim 1 , further comprising creating an updated augmented image for the second one of the plurality of customer images with the updated overlays for display on the second computing device.
9. The method of claim 1 , further comprising creating an augmented image for the plurality of customer images with the overlays, wherein the augmented image is created responsive to receiving the plurality of customer images from the first computing device.
10. The method of claim 9 , wherein the augmented image for the plurality of customer images is created and displayed at a different time than the receipt of the plurality of customer images from the first computing device.
11. A computer-implemented method for augmenting a plurality of images of a site with a virtual object, the method comprising:
capturing the plurality of images by a first computing device, wherein the plurality of images depicts portions of a potential site for the virtual object captured from different perspectives;
determining, by the first computing device, 3D feature data corresponding to each of the plurality of images, wherein respective defined coordinate systems for 3D spaces corresponding to each of the plurality of images are defined and oriented relative to the coordinate systems of other images of the plurality of images;
communicating the plurality of images and the 3D feature data to a second computing device, wherein the second computing device is in a peer to peer relationship with the first computing device;
determining, by the second computing device, a first location in the respective defined coordinate system for the 3D space corresponding to a first one of the plurality of images;
determining, by the second computing device, an appropriate size and orientation for depicting the virtual object in the first one of the plurality of images based on the first location and 3D feature data corresponding to the first one of the plurality of images;
determining, based on the first location, first corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the plurality of images other than first one of the plurality of images based on orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the first one of the plurality of images;
determining appropriate sizes and orientations for depicting the virtual object in the plurality of images other than the first one of the plurality of images based on the first corresponding locations and the 3D feature data corresponding to the plurality of images other than the first one of the plurality of images;
creating, by the first computing device or the second computing device, a plurality of augmented images for display on the first computing device by overlaying appropriately sized and oriented depictions of the virtual object at the first location and the first corresponding locations for each of the plurality of images;
receiving a re-positioning signal indicative of a desired re-positioning of the virtual object from the second computing device, the re-positioning signal identifying an updated position for the virtual object relative to a second one of the plurality of images;
determining a second location in the respective defined coordinate system for the 3D space corresponding to the second one of the plurality of images at which to overlay the virtual object in response to the re-positioning signal;
determining, based on the second location, second corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the images other than second one of the plurality of images based on the orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the second one of the plurality of images;
determining appropriate sizes and orientations for depicting the virtual object in the plurality of images based on the second location, the second corresponding locations, and the 3D feature data corresponding to the plurality of customer images; and
causing updated overlays of the appropriately sized and oriented depictions of the virtual object at the second location and the second corresponding locations in the plurality of customer images.
12. A computer system for augmenting a plurality of images of a site with a virtual object, the system comprising:
a processor; and
a non-transitory computer readable medium structured to store instructions that, when executed by the processor, cause the processor to:
receive the plurality of images from a first computing device, wherein the plurality of images depicts portions of a potential site for the virtual object as captured from different perspectives,
receive the virtual object for display, and receive a positioning signal corresponding to a desired location of the virtual object, the positioning signal identifying a position for the virtual object relative to a first one of the plurality of images,
determine 3D feature data corresponding to each of the plurality of images, wherein respective defined coordinate systems for 3D spaces corresponding to each of the plurality of images are defined and oriented relative to the coordinate systems of other images of the plurality of images,
determine a first location in the respective defined coordinate system for the 3D space corresponding to the first one of the plurality of images based on the position for the virtual object identified by the positioning signal,
determine an appropriate size and orientation for depicting the virtual object in the first one of the plurality of images based on the first location and the 3D feature data corresponding to the first one of the plurality of images,
determine, based on the first location, first corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the plurality of images other than first one of the plurality of images based on orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the first one of the plurality of images,
determine appropriate sizes and orientations for depicting the virtual object in the plurality of images other than the first one of the plurality of images based on the first corresponding locations and the 3D feature data corresponding to the plurality of images other than the first one of the plurality of images,
cause overlays of the appropriately sized and oriented depictions of the virtual object at the first location and the first corresponding locations in each of the plurality of images,
receive a re-positioning signal indicative of a desired re-positioning of the virtual object from a second computing device, the re-positioning signal identifying an updated position for the virtual object relative to a second one of the plurality of images,
determine a second location in the respective defined coordinate system for the 3D space corresponding to the second one of the plurality of images at which to overlay the virtual object in response to the re-positioning signal,
determining, based on the second location, second corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the images other than second one of the plurality of images based on the orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the second one of the plurality of images,
determining appropriate sizes and orientations for depicting the virtual object in the plurality of images based on the second location, the second corresponding locations, and the 3D feature data corresponding to the plurality of images, and
cause updated overlays of the appropriately sized and oriented depictions of the virtual object at the second location and the second corresponding locations in the plurality of images.
13. The system of claim 12 , wherein the computer system is structured to transmit at least one of the appropriately sized and oriented depictions of the virtual object and the first location to the first computing device.
14. The system of claim 12 , further wherein the computer system is structured to create an augmented image for the first one of the plurality of images with the overlay of the appropriately sized and oriented depiction of the virtual object at the first location for display on the first computing device.
15. The system of claim 12 , wherein the 3D feature data includes a determined coordinate system relative to a camera of the first computing device.
16. The system of claim 14 , wherein the 3D feature data includes location and size of planar surfaces.
17. A computer-implemented system for augmenting a plurality of customer images with at least one virtual object, the system comprising:
a processor; and
a non-transitory computer readable medium structured to store instructions that, when executed by the processor, cause the processor to:
receive the plurality of customer images from a first computing device, wherein the plurality of customer images each depict portions of a customer environment, as captured from different perspectives,
receive a selection of a merchant item for display,
receive a positioning signal corresponding to a desired location of the at least one virtual object in the customer environment, the positioning signal identifying a position for the at least one virtual object relative to a first one of the plurality of customer images,
determine 3D feature data corresponding to the plurality of customer images, wherein respective defined coordinate systems for 3D spaces corresponding to each of the plurality of customer images are defined and oriented relative to the coordinate systems of other customer images of the plurality of customer images,
determine the at least one virtual object corresponding to the selection of the merchant item,
determine a first location in the respective defined coordinate system for the 3D space corresponding to the first one of the plurality of customer images based on the position from the at least one virtual object identified by the positioning signal,
determine an appropriate size and orientation for depicting the at least one virtual object in the first one of the plurality of customer images based on the first location and the 3D feature data corresponding to the first one of the plurality of customer images,
determine, based on the first location, first corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the plurality of customer images other than first one of the plurality of customer images based on orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the first one of the plurality of customer images,
determine appropriate sizes and orientations for depicting the at least one virtual object in the plurality of customer images other than the first one of the plurality of customer images based on the first corresponding locations and the 3D feature data corresponding to the plurality of customer images other than the first one of the plurality of customer images,
create an augmented image for each of the plurality of customer images for display on the first computing device by overlaying the appropriately sized and oriented depictions of the at least one virtual object at the first location and the first corresponding locations for each of the plurality of customer images,
receive a re-positioning signal indicative of a desired re-positioning of the at least one virtual object from a second computing device, the re-positioning signal identifying an updated position for the at least one virtual object relative to a second one of the plurality of customer images,
determine a second location in the respective defined coordinate system for the 3D space corresponding to the second one of the plurality of at which to overlay the at least one virtual object in response to the re-positioning signal,
determine, based on the second location, second corresponding locations in the respective defined coordinate systems for the 3D spaces corresponding to the customer images other than second one of the plurality of customer images based on the orientations of those coordinate systems relative to the coordinate system for the 3D space corresponding to the second one of the plurality of customer images,
determine appropriate sizes and orientations for depicting the at least one virtual object in the plurality of customer images based on the second location, the second corresponding locations, and the 3D feature data corresponding to the plurality of customer images, and
cause updated overlays of the appropriately sized and oriented depictions of the at least one virtual object at the second location and the second corresponding locations in the plurality of customer images.
18. The system of claim 17 , wherein the augmented image is enabled for display on the second computing device.
19. The system of claim 18 , wherein the first computing device and the second computing device are in a peer to peer relationship.
20. The system of claim 17 , wherein the augmented image is created in real time with the receipt of the plurality of customer images from the first computing device.
21. The system of claim 17 , wherein the augmented image for each of the plurality of customer images is created and displayed at a different time than the receipt of the plurality of customer images from the first computing device.
22. The method of claim 10 , further comprising creating an updated augmented image for the second one of the plurality of images with the updated overlays for display on the first computing device.
23. The method of claim 10 , wherein the augmented image is created responsive to the receipt of the plurality of images from the first computing device.
24. The method of claim 10 , wherein the augmented image is created and displayed at a different time than the receipt of the plurality of images from the first computing device.