IP Library Granted Patent US 10,839,608
Granted Patent B2
US 10,839,608 · App. 16/442,879 · Granted Nov 17, 2020

Smart-home device placement and installation using augmented-reality visualizations

Inventors: Adam Mittleman (Redwood City, CA); Jason Chamberlain (Campbell, CA); Jacobi Grillo (San Jose, CA); Daniel Biran (Mountain View, CA); Mark Kraz (Santa Clara, CA); Lauren Chanen (San Francisco, CA); Daniel Foran (San Francisco, CA); David Fichou (Mountain View, CA); William Dong (Redwood City, CA); Bao-Tram Phan Nguyen (Mountain View, CA); Brian Silverstein (Mountain View, CA); Yash Modi (San Mateo, CA); Alex Finlayson (Mountain View, CA); Dongeek Shin (Mountain View, CA)
Assignee: Google LLC
G06T19/006G06T15/20G06T15/30G06T19/20G06T2219/2012G06T2219/2021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,839,608
App. No.
16/442,879
Filed
Jun 17, 2019
Granted
Nov 17, 2020
Kind
B2
Art Unit
2618
USPC
345/426
Abstract

A method for optimizing the placement of smart-home devices may include receiving, by a mobile computing device, a location for a smart-home device, where the mobile computing device comprises a display and a camera; rendering a view of a virtual object that represents a field-of-view of the smart-home device, where the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

Claims (52)

1. A method for optimizing the placement of smart-home devices, the method comprising:

capturing, by a camera of a mobile computing device, a view of a smart-home device, wherein the mobile computing device comprises a display;

determining the location for the smart-home device relative to a location of the mobile computing device;

rendering, by the mobile computing device, a view of a virtual object that represents a field-of-view of the smart-home device, wherein the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and

displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

2. The method of claim 1 , wherein the virtual object comprises a cone projecting away from the location for the smart-home device.

3. The method of claim 1 , wherein the virtual object is rendered as part of a virtual 3-D scene from a perspective of a virtual camera that corresponds to the position of the mobile computing device in the 3-D virtual scene.

4. The method of claim 1 , wherein rendering the view of the virtual object comprises clipping a portion of the view of the virtual object based on an obstruction within the field-of-view of the smart-home device.

5. The method of claim 1 , wherein the smart-home device comprises a motion sensor, and the field-of-view of the smart-home device comprises an area in which the motion sensor can detect motion.

6. The method of claim 1 , further comprising:

receiving a view of the smart-home device during installation;

generating a view of instructions for installing the smart-home device; and

displaying the instructions for installing the smart-home device on the display of the mobile computing device layered over the view of the smart-home device during installation.

7. The method of claim 6 , further comprising:

connecting to a second computing device; and

receiving live installation instructions from a user through the second computing device.

8. A system comprising:

one or more processors; and

one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving a first position of a mobile computing device, wherein the mobile computing device comprises a display and a camera;

receiving a second position of the mobile computing device relative to the first position;

using the first position as the location for a smart-home device;

rendering, by the mobile computing device, a view of a virtual object that represents a field-of-view of the smart-home device, wherein the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and

displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

9. The system of claim 8 , wherein the smart-home device comprises a magnetic sensor, and the field-of-view of the smart-home device comprises an area in which the magnetic sensor can detect a magnet.

10. The system of claim 8 , wherein the smart-home device comprises a radio, and the field-of-view of the smart-home device comprises a range of the radio.

11. The system of claim 8 , wherein the smart-home device comprises a motion sensor, and the field-of-view of the smart-home device comprises an area in which the motion sensor can detect motion.

12. The system of claim 8 , wherein the smart-home device comprises a camera, and the field-of-view of the smart-home device comprises an area that can be captured by the camera.

13. The system of claim 8 , wherein the view of the virtual object is rendered in a first color when a user and/or the mobile computing device is not in the field-of-view of the smart-home device, and wherein the view of the virtual object is rendered in a second color when the user and/or mobile computing device is in the field-of-view of the smart-home device.

14. The system of claim 8 , wherein the view of the virtual object is rendered in a first color for a first portion of the field-of-view and is rendered in a second color for a second portion of the field-of-view.

15. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving a finger-tap input on a display of a mobile computing device, wherein the mobile computing device comprises a camera;

determining a location for a smart-home device based on the view of the camera of the mobile computing device and the finger-tap input;

rendering, by the mobile computing device, a view of a virtual object that represents a field-of-view of the smart-home device, wherein the view of the virtual object is rendered based on a position corresponding to a position of the mobile computing device; and

displaying, by the mobile computing device, the view of a virtual object that represents a field-of-view of the smart-home device on the display of the mobile computing device.

16. The non-transitory, computer-readable medium of claim 15 , wherein a shape of the virtual object changes according to sensitivity settings on the smart-home device.

17. The non-transitory, computer-readable medium of claim 15 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:

identifying a source in the smart-home environment that interferes with the field of view of the smart-home device;

rendering a view of a second virtual object that represents an area of interference generated by the source; and

displaying, by the mobile computing device, the view of the second virtual object with the view of the virtual object.

18. The non-transitory, computer-readable medium of claim 15 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:

receiving information descriptive of at least a portion of a smart-home environment;

determining locations in the smart-home environment where the smart-home device should or should not be installed;

rendering a view of a second virtual object that represents the locations in the smart-home environment where the smart-home device should or should not be installed; and

displaying, by the mobile computing device, the view of the second virtual object with the view of the virtual object.

19. The non-transitory, computer-readable medium of claim 15 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:

receiving a view of the smart-home device during installation;

generating a view of instructions for installing the smart-home device; and

displaying the instructions for installing the smart-home device on the display of the mobile computing device layered over the view of the smart-home device during installation.

20. The non-transitory, computer-readable medium of claim 19 , comprising additional instructions that, when executed by the one or more processors, cause the one or more processors to perform additional operations comprising:

connecting to a second computing device; and

receiving live installation instructions from a user through the second computing device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2019
From: MITTLEMAN, ADAM; CHAMBERLAIN, JASON; GRILLO, JACOBI; BIRAN, DANIEL; KRAZ, MARK; CHANEN, LAUREN; FORAN, DANIEL; FICHOU, DAVID; DONG, WILLIAM; NGUYEN, BAO-TRAM PHAN; SILVERSTEIN, BRIAN; MODI, YASH; FINLAYSON, ALEX; SHIN, DONGEEK
To: GOOGLE LLC
Reel/Frame 049862/0891 →
Continuity (2)
Provisional Application 62685806 · Jun 15, 2018
Related Publication 20190385373A1 · Dec 19, 2019
Cited By (1)
US 12,694,681