IP Library › Granted Patent US 11,120,636
Granted Patent B2
US 11,120,636 · App. 17/099,399 · Granted Sep 14, 2021

Smart-home device placement and installation using augmented-reality visualizations

Inventors: Adam Mittleman (Redwood City, CA); Jason Chamberlain (Campbell, CA); Jacobi Grillo (San Jose, CA); Daniel Biran (Mountain View, CA); Mark Kraz (Santa Clara, CA); Lauren Chanen (San Francisco, CA); Daniel Foran (San Francisco, CA); David Fichou (Mountain View, CA); William Dong (Redwood City, CA); Bao-Tram Phan Nguyen (Mountain View, CA); Brian Silverstein (Mountain View, CA); Yash Modi (San Mateo, CA); Alex Finlayson (Mountain View, CA); Dongeek Shin (Mountain View, CA)
Assignee: Google LLC
G06T19/006G06T15/20G06T15/30G06T19/20G06T2219/2012G06T2219/2021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,120,636
App. No.
17/099,399
Granted
Sep 14, 2021
Kind
B2
Abstract

A method for guiding installation of smart-home devices may include capturing, by a camera of a mobile computing device, a view of an installation location for a smart-home device; identifying a wire in the view of the installation location for the smart-home device; determining an instruction for connecting the wire to the smart-home device; and displaying the view of the installation location for a smart-home device with the instruction for connecting the wire to the smart-home device.

Claims (36)

1. A method for guiding installation of smart-home devices, the method comprising:

capturing, by a camera of a mobile computing device, a view of an installation location for a smart-home device;

identifying, by the mobile computing device, a wire in the view of the installation location for the smart-home device;

determining, by the mobile computing device, an instruction for connecting the wire to the smart-home device; and

displaying, by a display of the mobile computing device, the view of the installation location for a smart-home device with the instruction for connecting the wire to the smart-home device.

2. The method of claim 1 , wherein the instruction comprises one or more arrows pointing to a location where the wire should be connected to the smart-home device.

3. The method of claim 1 , wherein identifying the wire comprises identifying a color of a wire and determining a function for the wire based on the color of the wire.

4. The method of claim 3 , wherein determining the instruction for connecting the wire to the smart-home device comprises determining a wire connector on the smart-home device that corresponds to the function for the wire.

5. The method of claim 1 , wherein the instruction comprises an indication that the wire is not inserted in a correct wire connector of the smart-home device.

6. The method of claim 1 , wherein determining the instruction for connecting the wire to the smart-home device comprises determining that not enough of the wire is exposed from a wire insulation.

7. The method of claim 1 , wherein determining the instruction for connecting the wire to the smart-home device comprises determining that too much of the wire is exposed from a wire insulation.

8. A mobile computing device comprising:

a camera;

a display;

one or more processors;

one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

capturing, by the camera of the mobile computing device, a view of an installation location for a smart-home device;

identifying, by the mobile computing device, a wire in the view of the installation location for the smart-home device;

determining, by the mobile computing device, an instruction for connecting the wire to the smart-home device; and

displaying, by the display of the mobile computing device, the view of the installation location for a smart-home device with the instruction for connecting the wire to the smart-home device.

9. The mobile computing device of claim 8 , wherein the view of the installation location for the smart-home device is captured by the camera of the mobile computing device in real time as the view of the installation location for the smart-home device is displayed on the display of the mobile computing device.

10. The mobile computing device of claim 8 , wherein the mobile computing device comprises a smart phone.

11. The mobile computing device of claim 8 , wherein the mobile computing device comprises an augmented-reality headset.

12. The mobile computing device of claim 8 , wherein the operations further comprise rendering a 2D view of a 3D object representing the instruction for connecting the wire to the smart-home device.

13. The mobile computing device of claim 12 , wherein the 3D object comprises text in a virtual 3D environment.

14. The mobile computing device of claim 12 , wherein the operations further comprise compositing the 2D view of the 3D object representing the instruction for connecting the wire to the smart-home device with the view of the installation location for the smart-home device on the display of the mobile computing device.

15. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

capturing, by a camera of a mobile computing device, a view of an installation location for a smart-home device;

identifying, by the mobile computing device, a wire in the view of the installation location for the smart-home device;

determining, by the mobile computing device, an instruction for connecting the wire to the smart-home device; and

displaying, by a display of the mobile computing device, the view of the installation location for a smart-home device with the instruction for connecting the wire to the smart-home device.

16. The non-transitory computer-readable medium of claim 15 , wherein determining the instruction for connecting the wire to the smart-home device comprises determining that the wire should be re-trimmed before connecting the wire to the smart-home device.

17. The non-transitory computer-readable medium of claim 15 , wherein the view of the installation location for the smart-home device comprises a view of at least a portion of the smart-home device.

18. The non-transitory computer-readable medium of claim 17 , wherein the view of the at least a portion of the smart-home device comprises a plurality of wire connectors of the smart-home device.

19. The non-transitory computer-readable medium of claim 15 , wherein the smart-home device comprises a thermostat.

20. The non-transitory computer-readable medium of claim 15 , wherein the smart-home device comprises a camera or a hazard detector.

Continuity (3)
Continuation 16442879 · Jun 17, 2019
Provisional Application 62685806 · Jun 15, 2018
Related Publication 20210074073A1 · Mar 11, 2021
Cited By (1)
US 12,567,202