IP Library Granted Patent US 10,319,150
Granted Patent B1
US 10,319,150 · App. 15/595,670 · Granted Jun 11, 2019

Object preview in a mixed reality environment

Inventors: Jason Canada (San Francisco, CA); Rupa Chaturvedi (Menlo Park, CA); Jared Corso (San Mateo, CA); Michael Patrick Cutter (Mountain View, CA); Sean Niu (San Francisco, CA); Shaun Michael Post (San Mateo, CA); Peiqi Tang (Santa Clara, CA); Stefan Vant (Seattle, WA); Mark Scott Waldo (Richmond, CA); Andrea Zehr (Palo Alto, CA)
Assignee: A9.COM, INC.
G06T19/006G06T11/60G06F3/147G06F3/1454G06T19/20G09G2340/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,319,150
App. No.
15/595,670
Granted
Jun 11, 2019
Kind
B1
Abstract

Users can view images or renderings of items placed (virtually) within a physical space. For example, a rendering of an item can be placed within a live camera view of the physical space. A snapshot of the physical space can be captured and the snapshot can be customized, shared, etc. The renderings can be represented as two-dimensional images, e.g., virtual stickers or three-dimensional models of the items. Users can have the ability to view different renderings, move those items around, and develop views of the physical space that may be desirable. The renderings can link to products offered through an electronic marketplace and those products can be consumed. Further, collaborative design is enabled through modeling the physical space and enabling users to view and move around the renderings in a virtual view of the physical space.

Claims (91)

1. A computing device, comprising:

a camera;

a display screen;

at least one processor;

a memory device including instructions that, when executed by the at least one processor, enable the computing device to:

receive a selection of a user-selectable element configured to activate an augmented reality environment, the user-selectable element associated with a product from a product catalog of an electronic marketplace;

receive first image data that includes a representation of the product;

analyze the first image data to determine a set of features associated with the representation of the product;

recognize edges of the representation of the product based at least in part on the set of features;

generate second image data that includes a modified view of the representation of the product, the modified view including a border of a predetermined width and a drop shadow, the second image data corresponding to a virtual sticker;

capture image data of a first view of a physical space, the physical space corresponding to a field of view of the camera;

display a camera view of the first view of the physical space on the display screen, the camera view including the virtual sticker of the product appearing to be in a first location in the physical space, the product being part of a category associated with the physical space; and

enable purchase of the product from the electronic marketplace.

2. The computing device of claim 1 , wherein the instructions, when executed further enable the computing device to:

capture second image data that includes a second view of the physical space; and

display at least a portion of the first view and the second view, the virtual sticker appearing to be in the first location of the physical space.

3. A computing device, comprising:

an image capture element;

a display element;

at least one processor;

a memory device including instructions that, when executed by the at least one processor, enable the computing device to:

capture image data of a physical space using the image capture element, the physical space within a field of view of the image capture element;

analyze the image data to recognize a category associated with the physical space;

determine a set of items of interest based at least in part on the category, the set of items of interest associated with respective renderings;

display the respective renderings of the set of items of interest;

enable interaction with the respective renderings;

display a camera view of the physical space on the display element, the camera view including a rendering of an item of interest of the set of items of interest appearing to be positioned in the physical space, the item of interest being part of the category associated with the physical space; and

enable purchase of the item of interest from an electronic marketplace.

4. The computing device of claim 3 , wherein the instructions, when executed further enable the computing device to:

display a product page that includes the item of interest, the item of interest being a product from a product catalog of an electronic marketplace;

receive a selection of a user-selectable element configured to activate the camera view of an augmented reality environment; and

enable the camera view.

5. The computing device of claim 3 , wherein the instructions, when executed further enable the computing device to:

enable modification of at least one property associated with the rendering, the at least one property including a size of the rendering or a position of the rendering, and wherein the rendering includes a virtual sticker representation of the item of interest.

6. The computing device of claim 3 , wherein the instructions, when executed further enable the computing device to:

receive a selection of a user-selectable element configured to activate a three-dimensional viewing mode;

obtain space dimension data for the physical space;

determine object dimension data and point of view data for an image overlay that includes a three-dimensional representation of the item of interest; and

determine scale and point of view appropriate image data for the physical space, the scale and point of view appropriate image data determined using at least the image data, the point of view data, the space dimension data, and the object dimension data,

wherein the camera view includes the image overlay for the item of interest, with the scale and point of view appropriate image data determined using at least the image data, the point of view data, the space dimension data, and the object dimension data.

7. The computing device of claim 3 , wherein the instructions, when executed further enable the computing device to:

obtain a representation of the item of interest;

analyze the representation of the item of interest to determine an outline of the representation of the item of interest;

add a border of a predetermined width to the representation of the item of interest;

add a drop shadow to the border; and

store an updated representation of the item of interest as the rendering.

8. The computing device of claim 3 , wherein the instructions, when executed further enable the computing device to:

receive a request to view a set of items of interest, the set of items of interest related to the item of interest;

display respective renderings of the set of items of interest; and

enable interaction with the respective renderings.

9. The computing device of claim 3 , wherein the instructions, when executed further enable the computing device to:

receive a search request to view a set of items of interest, the search request including a search term;

determine a set of items of interest based at least in part on the search term, the set of items of interest associated with respective renderings;

display the respective renderings of the set of items of interest; and

enable interaction with the respective renderings.

10. The computing device of claim 3 , wherein the rendering of the item of interest can be at least one of moved within, or removed from, the camera view, or resized.

11. The computing device of claim 3 , wherein the instructions, when executed to provide information further enable the computing device to:

receive user input to change a location of the rendering in the camera view from a first location to a second location; and

display the rendering in the second location in the camera view.

12. The computing device of claim 11 , wherein the instructions, when executed to provide information further enable the computing device to:

receive user input to associate the rendering with a representation of a first area of the physical space corresponding to a first image capture position;

move the image capture element to a second image capture position to capture a second area of the physical space; and

store information associating the rendering with the representation of the first area of the physical space.

13. The computing device of claim 12 , wherein the instructions, when executed to provide information further enable the computing device to:

move the image capture element to the first image capture position; and

display the rendering to appear to be positioned in the representation of the first area of the physical space.

14. A method, comprising:

capturing image data of a physical space using an image capture element of a computing device, the physical space within a field of view of the image capture element;

analyzing the image data to recognize a category associated with the physical space;

determining a set of items of interest based at least in part on the category, the set of items of interest associated with respective renderings;

displaying the respective renderings of the set of items of interest;

enabling interaction with the respective renderings;

displaying a camera view of the physical space on a display element of the computing device, the camera view including a rendering of an item of interest of the set of items of interest appearing to be positioned in the physical space, the item of interest being part of the category associated with the physical space; and

enabling purchase of the item of interest from an electronic marketplace.

15. The method of claim 14 , further including:

receiving a request to model the physical space;

causing a set of mapping data to automatically be obtained for a set of views within the physical space, the set of mapping data including at least image data and dimension data, at least one view of the set of views comprising a top view of at least a portion of the physical space;

generating, using the set of mapping data for the set of views, a three-dimensional virtual model of the physical space;

receiving a request for a view of the three-dimensional virtual model, the view corresponding to a specific viewpoint; and

rendering the view of the three-dimensional virtual model from the specific viewpoint and providing the view for display, the view having associated dimension data for the physical space such that the rendering of the item of interest to be placed within the view of the three-dimensional virtual model is able to be performed with a scale corresponding to the associated dimension data.

16. The method of claim 15 , further including:

determining a set of items corresponding to data for at least one of the physical space or a customer of the physical space;

providing data for at least a subset of the set of items for display with the three-dimensional virtual model;

receiving a request relating to a selected item of the set of items;

rendering a view of the selected item using model data for the selected item, the view of the selected item being scale-appropriate for the view of the three-dimensional virtual model; and

displaying the view of the selected item in the view of the three-dimensional virtual model.

17. The method of claim 14 , wherein the rendering includes a virtual sticker, the virtual sticker associated with a border and a drop shadow.

18. The method of claim 14 , further comprising:

displaying a product page that includes the item of interest, the item of interest being a product from a product catalog of an electronic marketplace;

receiving a selection of a user-selectable element configured to activate the camera view of an augmented reality environment; and

enabling the camera view.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2024
From: A9.COM, INC.
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 069167/0493 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2017
From: CANADA, JASON; CHATURVEDI, RUPA; CORSO, JARED; CUTTER, MICHAEL PATRICK; NIU, SEAN; POST, SHAUN MICHAEL; TANG, PEIQI; VANT, STEFAN; WALDO, MARK SCOTT; ZEHR, ANDREA
To: A9.COM, INC.
Reel/Frame 042389/0655 →
Cited By (10)
US 12,198,286 US 12,236,473 US 12,236,522 US 12,266,010 US 12,288,289 US 12,354,146 US 12,406,452 US 12,530,817 US 12,561,734 US 12,717,837