IP Library Granted Patent US 11,210,732
Granted Patent B1
US 11,210,732 · App. 17/089,034 · Granted Dec 28, 2021

Systems and methods for visualizing wall coverings in an image of a scene

Inventors: Shrenik Sadalgi (Cambridge, MA); Christian Vázquez (Revere, MA)
Assignee: Wayfair LLC
G06Q30/0643G06F16/54G06K9/6202G06N3/08G06T11/00H04M1/72403G06T2200/24G06T2210/04H04M2250/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,210,732
App. No.
17/089,034
Granted
Dec 28, 2021
Kind
B1
Abstract

Methods and apparatus for visualizing a surface covering on at least a portion of a surface in an image of a scene. The method comprises identifying, using at least one computer processor, a surface plane from the image of the scene, determining, for each pixel of a plurality of pixels corresponding to the surface plane, whether the pixel corresponds to at least a portion of the surface in the scene, and generating an updated image of the scene by overlaying on the surface plane, a visualization of a plurality of surface covering tiles on pixels along the surface plane determined to correspond to at least a portion of the surface in the scene.

Claims (68)

1. A method of visualizing a wall covering on an image of a scene, the method comprising:

receiving a selection of an image of a scene;

determining a wall plane in the image;

receiving a selection of one or more images of wall tiles to visualize in the image; and

generating an updated image of the scene by overlaying on the determined wall plane, a visualization of the wall covering based, at least in part, on the selected one or more images of wall tiles,

wherein generating the updated image of the scene comprises generating, based on the selection of one or more images of wall tiles, a tile template and overlaying the tile template on the determined wall plane,

wherein generating the tile template comprises:

determining a shape category of one or more tiles in the selected one or more images of wall tiles,

determining, based on the determined shape category, an overlap between tiles in the tile template, and

generating the tile template based on the determined shape category and the determined overlap.

2. The method of claim 1 , further comprising:

providing, on a viewer displayed in a user interface, a scene selector including indicators for a plurality of images of scenes, and wherein receiving the selection of the image of the scene comprises receiving a selection of one of the plurality of images of scenes included in the scene selector.

3. The method of claim 2 , wherein the selected image of the scene is associated with a pre-selected wall plane, and wherein determining a wall plane in the image comprises determining the wall plane as the pre-selected wall plane.

4. The method of claim 1 , wherein determining the shape category comprises determining whether a shape of the one or more tiles in the selected one or more images of wall tiles is a rectangle shape, a square shape or a hexagonal shape.

5. The method of claim 1 , wherein determining the shape category comprises:

determining that a shape of the one or more tiles is a non-standard shape;

calculating, for each of the one or more tiles, a maximum number of background pixels in a row and column direction of the tile template to determine an offset; and

determining, based, at least in part, on the offset, the overlap between tiles in the tile template.

6. The method of claim 1 , wherein generating the tile template based, at least in part, on the determined overlap comprises using an alpha blending technique to generate the tile template.

7. The method of claim 1 , further comprising:

storing the generated tile template on one or more non-transitory storage devices.

8. The method of claim 1 , wherein determining the wall plane comprises:

extracting feature points from the scene in the image; and

identifying the wall plane based on the extracted feature points.

9. The method of claim 1 , further comprising:

capturing a plurality of images from a camera feed of a smartphone, wherein the plurality of images includes the image of the scene; and

displaying a visualization of the determined wall plane on a display of the smartphone during capture of the plurality of images.

10. The method of claim 9 , further comprising:

generating, by an augmented reality application, the visualization as a virtual overlay of a wall plane on an image of a physical scene.

11. A method of visualizing a wall covering on an image of a scene, the method comprising:

receiving a selection of an image of a scene;

determining a wall plane in the image;

receiving a selection of one or more images of wall tiles to visualize in the image; and

generating an updated image of the scene by overlaying on the determined wall plane, a visualization of the wall covering based, at least in part, on the selected one or more images of wall tiles,

wherein generating the updated image comprises:

applying a shader to the wall plane to determine segments of the wall plane that include pixels corresponding to wall, and

rendering the visualization of the wall covering only on the segments of the wall plane determined to be wall.

12. A method of visualizing a wall covering on an image of a scene, the method comprising:

receiving a selection of an image of a scene;

determining a wall plane in the image;

receiving a selection of one or more images of wall tiles to visualize in the image; and

generating an updated image of the scene by overlaying on the determined wall plane, a visualization of the wall covering based, at least in part, on the selected one or more images of wall tiles,

wherein generating the updated image comprises generating the updated image to include lighting information, and

wherein generating the updated image to include lighting information comprises:

determining a luminance of each of the pixels in the image of the scene to generate a light map of the wall plane in the scene;

adjusting values of pixels in the visualization of the wall covering based, at least in part, on the generated light map to generate a light-adjusted visualization of the wall covering; and

overlaying the light-adjusted visualization on the pixels along the wall plane determined to correspond to wall in the scene.

13. A computing device configured to generate a visualization of a wall covering on an image of a scene, the computing device comprising:

at least one computer processor; and

at least one non-transitory computer-readable medium having a plurality of instructions stored thereon that, when executed by the at least one computer processor, perform a method comprising:

receiving a selection of an image of a scene;

determining a wall plane in the image;

receiving a selection of one or more images of wall tiles to visualize in the image; and

generating an updated image of the scene by overlaying on the determined wall plane, a visualization of the wall covering based, at least in part, on the selected one or more images of wall tiles,

wherein generating the updated image comprises:

applying a shader to the wall plane to determine segments of the wall plane that include pixels corresponding to wall, and

rendering the visualization of the wall covering only on the determined segments of the wall plane.

14. The computing device of claim 13 , further comprising:

an image capture device configured to capture the image of the scene, wherein receiving the image of the scene comprises receiving the image of the scene from the image capture device.

15. The computing device claim 13 , wherein the computing device is a smartphone.

16. A non-transitory computer readable medium having a plurality of computer-executable instructions stored thereon that, when executed by at least one computer processor perform a method of generating a visualization of a wall covering on an image of a scene, the method comprising:

receiving a selection of an image of a scene;

determining a wall plane in the image;

receiving a selection of one or more images of wall tiles to visualize in the image; and

generating an updated image of the scene by overlaying on the determined wall plane, a visualization of the wall covering based, at least in part, on the selected one or more images of wall tiles,

wherein generating the updated image comprises:

applying a shader to the wall plane to determine segments of the wall plane that include pixels corresponding to wall, and

rendering the visualization of the wall covering only on the determined segments of the wall plane.

Assignments (7)
SECURITY AGREEMENT Recorded May 20, 2026
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 075591/0399 →
SECURITY INTEREST Recorded Nov 10, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 073514/0326 →
SECURITY AGREEMENT Recorded Mar 13, 2025
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 070513/0542 →
SECURITY AGREEMENT Recorded Oct 10, 2024
From: WAYFAIR LLC
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 069143/0399 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2021
From: SADALGI, SHRENIK
To: WAYFAIR LLC
Reel/Frame 057790/0827 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 14, 2021
From: VÁZQUEZ, CHRISTIAN
To: WAYFAIR LLC
Reel/Frame 057478/0947 →
SECURITY AGREEMENT Recorded Mar 24, 2021
From: WAYFAIR LLC
To: CITIBANK, N.A.
Reel/Frame 055708/0832 →