IP Library Granted Patent US 11,769,308
Granted Patent B2
US 11,769,308 · App. 17/870,400 · Granted Sep 26, 2023

Systems and methods of augmented reality guided image capture

Inventors: Nicole Allison Tan (Brookline, MA); Rachana Sreedhar (Boston, MA); Shrenik Sadalgi (Cambridge, MA); Christian Vázquez (Revere, MA); Jessica Stigile (Cambridge, MA); Ashwin Subramani (Belmont, MA); Maria Paula Gomez Muñoz (New York, NY)
Assignee: Wayfair LLC
G06T19/006G06T7/0002G06T7/70G06T7/80G06T19/003H04N9/646H04N23/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,308
App. No.
17/870,400
Granted
Sep 26, 2023
Kind
B2
Abstract

Some embodiments provide a mobile device configured to guide a user, via an augmented reality (AR) interface generated by the mobile device, to capture images of the physical object using a camera of the mobile device. The mobile device may be configured to obtain boundary information indicative of a boundary enclosing the physical object (e.g., a box enclosing the physical object). The mobile device may be configured to use the boundary information to determine positions from which the user is to capture images of the physical object. The mobile device may be configured to guide the user to capture the images using the AR interface by guiding the user to each of the positions in the AR interface (e.g., by generating on or more GUI elements in the AR interface that indicate a position from which the user is to capture an image).

Claims (106)

1. A mobile device, comprising:

at least one camera;

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform a method for guiding a user, via an interface generated by the mobile device, to capture a plurality of images of a physical object using the mobile device, the method comprising:

obtaining boundary information indicative of a boundary enclosing the physical object;

determining, using the boundary information, a plurality of positions from which the user is to capture the plurality of images, the plurality of positions including a first position from which the user is to capture a first image of the plurality of images;

guiding, using the interface, the user to capture the plurality of images, the guiding comprising:

guiding the user to the first position using the interface; and

enabling the user to capture the first image at the first position when a determination is made that the user can capture the first image; and

outputting the plurality of images.

2. The mobile device of claim 1 , wherein obtaining the boundary information comprises:

obtaining information specifying a box enclosing the physical object based on input provided by the user via the interface.

3. The mobile device of claim 2 , wherein obtaining the information specifying the box enclosing the physical object comprises:

obtaining first input indicative of locations of multiple corners of the box, the first input specified by the user tapping multiple respective locations shown in the interface; and

obtaining second input indicative of a height for the box, the second input specified by the user using a slider shown in the interface.

4. The mobile device of claim 2 , wherein determining the plurality of positions from which the user is to capture the plurality of images comprises:

determining the first position by determining:

a first angle using the box; and

a first distance to the physical object at which to capture the first image.

5. The mobile device of claim 1 , wherein the physical object comprises furniture.

6. The mobile device of claim 1 , wherein guiding the user to the first position using the interface comprises:

displaying, in the interface, a visual prompt indicating where the user is to position the mobile device to capture an image from the first position.

7. The mobile device of claim 1 ,

wherein guiding the user to the first position using the interface further comprises displaying, in the interface, a visual prompt indicating a first height, above the first position, at which the at least one camera is to be positioned; and

wherein the method comprises performing at least one quality control test comprising determining whether the at least one camera is positioned at the first height.

8. The mobile device of claim 1 , wherein the method further comprises:

performing at least one quality control test, the at least one quality control test comprising one or more of determining whether the at least one camera is positioned at the first position, determining whether the at least one camera is facing the physical object, determining whether the at least one camera is held at a desired pitch, and determining whether a measure of light intensity is above a threshold.

9. The mobile device of claim 1 , wherein enabling the user to capture the first image comprises presenting a selectable element in the interface that, when selected by the user, causes the mobile device to capture the first image.

10. The mobile device of claim 1 , wherein the method further comprises:

after the user captures the first image,

determining whether the first image is blurry; and

when it is determined that the first image is blurry, prompting the user via the interface to capture a new image to replace the first image.

11. The mobile device of claim 1 , wherein outputting the plurality of images comprises:

transmitting the plurality of images to at least one other computing device.

12. A method for guiding a user, via an interface generated by a mobile device, to capture a plurality of images of a physical object using the mobile device, the method comprising:

obtaining boundary information indicative of a boundary enclosing the physical object;

determining, using the boundary information, a plurality of positions from which the user is to capture the plurality of images, the plurality of positions including a first position from which the user is to capture a first image of the plurality of images;

guiding, using the interface, the user to capture the plurality of images, the guiding comprising:

guiding the user to the first position using the interface; and

enabling the user to capture the first image at the first position when a determination is made that the user can capture the first image; and

outputting the plurality of images.

13. The method of claim 12 , wherein obtaining the boundary information comprises:

obtaining information specifying a box enclosing the physical object based on input provided by the user via the interface.

14. The method of claim 13 , wherein determining the plurality of positions from which the user is to capture the plurality of images comprises:

determining the first position by determining:

a first angle using the box; and

a first distance to the physical object at which to capture the first image.

15. The method of claim 12 , wherein guiding the user to the first position using the interface further comprises displaying, in the interface, a visual prompt indicating a first height, above the first position, at which at least one camera is to be positioned.

16. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor of a mobile device, cause the at least one computer hardware processor to perform a method for guiding a user, via an interface generated by the mobile device, to capture a plurality of images of a physical object using the mobile device, the method comprising:

obtaining boundary information indicative of a boundary enclosing the physical object;

determining, using the boundary information, a plurality of positions from which the user is to capture the plurality of images, the plurality of positions including a first position from which the user is to capture a first image of the plurality of images;

guiding, using the interface, the user to capture the plurality of images, the guiding comprising:

guiding the user to the first position using the interface; and

enabling the user to capture the first image at the first position when a determination is made that the user can capture the first image; and

outputting the plurality of images.

17. The at least one non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises:

performing at least one quality control test, the at least one quality control test comprising one or more of determining whether at least one camera is positioned at the first position, determining whether the at least one camera is positioned at a first height above the first position, determining whether the at least one camera is facing the physical object, determining whether the at least one camera is held at a desired pitch, and determining whether a measure of light intensity is above a threshold.

18. A mobile device, comprising:

at least one camera;

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform a method for guiding a user, via an interface generated by the mobile device, to capture a plurality of images of a physical object using the mobile device, the method comprising:

obtaining boundary information indicative of a boundary enclosing the physical object;

determining, using the boundary information, a plurality of positions from which the user is to capture the plurality of images, the plurality of positions including a first position from which the user is to capture a first image of the plurality of images;

guiding, using the interface, the user to capture the plurality of images using the mobile device, the guiding comprising:

guiding the user to the first position using the interface, the guiding comprising providing the user with feedback for positioning the mobile device, the feedback generated based on a field of view of the at least one camera; and

outputting the plurality of images.

19. The mobile device of claim 18 , wherein obtaining the boundary information comprises:

obtaining information specifying a box enclosing the physical object based on input provided by the user via the interface.

20. The mobile device of claim 19 , wherein determining the plurality of positions from which the user is to capture the plurality of images comprises:

determining the first position by determining:

a first angle using the box; and

a first distance to the physical object at which to capture the first image.

21. The mobile device of claim 18 , wherein guiding the user to the first position using the interface comprises:

displaying, in the interface, a visual prompt indicating where the user is to position the mobile device to capture an image from the first position.

22. The mobile device of claim 18 ,

wherein guiding the user to the first position using the interface further comprises displaying, in the interface, a visual prompt indicating a first height, above the first position, at which the at least one camera is to be positioned; and

wherein the method comprises performing at least one quality control test comprising determining whether the at least one camera is positioned at the first height.

23. The mobile device of claim 18 , wherein the method further comprises:

performing at least one quality control test, the at least one quality control test comprising one or more of determining whether the at least one camera is positioned at the first position, determining whether the at least one camera is facing the physical object, determining whether the at least one camera is held at a desired pitch, and determining whether a measure of light intensity is above a threshold; and

providing the user with the feedback for positioning the mobile device based on a result of the at least one quality control test.

24. The mobile device of claim 18 , wherein the method further comprises:

after the user captures the first image,

determining whether the first image is blurry; and

when it is determined that the first image is blurry, prompting the user via the interface to capture a new image to replace the first image.

25. The mobile device of claim 18 , wherein outputting the plurality of images comprises:

transmitting the plurality of images to at least one other computing device.

26. A method for guiding a user, via an interface generated by a mobile device, to capture a plurality of images of a physical object using the mobile device, the method comprising:

obtaining boundary information indicative of a boundary enclosing the physical object;

determining, using the boundary information, a plurality of positions from which the user is to capture the plurality of images, the plurality of positions including a first position from which the user is to capture a first image of the plurality of images;

guiding, using the interface, the user to capture the plurality of images using the mobile device, the guiding comprising:

guiding the user to the first position using the interface, the guiding comprising providing the user with feedback for positioning the mobile device, the feedback generated based on a field of view of at least one camera; and

outputting the plurality of images.

27. The method of claim 26 , wherein obtaining the boundary information comprises:

obtaining information specifying a box enclosing the physical object based on input provided by the user via the interface.

28. The method of claim 26 , further comprising:

performing at least one quality control test, the at least one quality control test comprising one or more of determining whether the at least one camera is positioned at the first position, determining whether the at least one camera is positioned at a first height above the first position, determining whether the at least one camera is facing the physical object, determining whether the at least one camera is held at a desired pitch, and determining whether a measure of light intensity is above a threshold; and

providing the user with the feedback for positioning the mobile device based on a result of the at least one quality control test.

29. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor of a mobile device, cause the at least one computer hardware processor to perform a method for guiding a user, via an interface generated by the mobile device, to capture a plurality of images of a physical object using the mobile device, the method comprising:

obtaining boundary information indicative of a boundary enclosing the physical object;

determining, using the boundary information, a plurality of positions from which the user is to capture the plurality of images, the plurality of positions including a first position from which the user is to capture a first image of the plurality of images;

guiding, using the interface, the user to capture the plurality of images using the mobile device, the guiding comprising:

guiding the user to the first position using the interface, the guiding comprising providing the user with feedback for positioning the mobile device, the feedback generated based on a field of view of at least one camera; and

outputting the plurality of images.

30. The at least one non-transitory computer-readable storage medium of claim 29 , wherein the method further comprises:

performing at least one quality control test, the at least one quality control test comprising one or more of determining whether the at least one camera is positioned at the first position, determining whether the at least one camera is positioned at a first height above the first position, determining whether the at least one camera is facing the physical object, determining whether the at least one camera is held at a desired pitch, and determining whether a measure of light intensity is above a threshold; and

providing the user with the feedback for positioning the mobile device based on a result of the at least one quality control test.

Assignments (7)
SECURITY AGREEMENT Recorded May 20, 2026
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 075591/0399 →
SECURITY INTEREST Recorded Nov 10, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 073514/0326 →
SECURITY AGREEMENT Recorded Mar 13, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 070513/0542 →
SECURITY AGREEMENT Recorded Oct 10, 2024
From: WAYFAIR LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 069143/0102 →
SECURITY AGREEMENT Recorded Oct 10, 2024
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 069143/0399 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2022
From: TAN, NICOLE ALLISON; SREEDHAR, RACHANA; SADALGI, SHRENIK; STIGILE, JESSICA; SUBRAMANI, ASHWIN; GOMEZ MUÑOZ, MARIA PAULA
To: WAYFAIR LLC
Reel/Frame 061480/0247 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2022
From: VÁZQUEZ, CHRISTIAN
To: WAYFAIR LLC
Reel/Frame 060930/0645 →
Continuity (3)
Continuation 17201693 · Mar 15, 2021
Provisional Application 63145462 · Feb 3, 2021
Related Publication 20220366658A1 · Nov 17, 2022