IP Library Granted Patent US 12,567,322
Granted Patent B2
US 12,567,322 · App. 18/744,965 · Granted Mar 3, 2026

Discovery of and connection to remote devices

Inventors: Jeffrey S. Norris (Saratoga, CA); Bruno M. Sommer (Sunnyvale, CA); Alexandre Da Veiga (San Francisco, CA)
Assignee: APPLE INC.
G08C17/02G06F18/22G06V10/22H04W8/005H04W76/10G08C2201/31
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,567,322
App. No.
18/744,965
Granted
Mar 3, 2026
Kind
B2
Abstract

A method includes receiving a signal that indicates a location of a control device that is configured to change an operating state of a controlled device. The method also includes identifying a first visible device and a second visible device in one or more images, matching the first visible device with the control device based on a location of the first visible device matching the location of the control device, matching the second visible device with the controlled device, pairing the control device with a host device, and controlling the control device using the host device to change the operating state of the controlled device.

Claims (53)

1 . A method, comprising:

receiving a signal that indicates a location of a control device, wherein the control device is configured to change an operating state of a controlled device;

identifying a first visible device and a second visible device in one or more images;

matching the first visible device with the control device based on a location of the first visible device matching the location of the control device;

matching the second visible device with the controlled device;

pairing the control device with a host device; and

controlling the control device using the host device to change the operating state of the controlled device.

2 . The method of claim 1 , wherein matching the second visible device with the controlled device comprises:

changing the operating state of the controlled device using the control device; and

detecting a change of an operating state of the second visible device in the one or more images.

3 . The method of claim 1 , wherein matching the second visible device with the controlled device is based on a user input that identifies the controlled device.

4 . The method of claim 1 , wherein the signal further indicates a device type for the control device, and matching the first visible device with the control device is further based on the device type for the control device matching a device type for the first visible device.

5 . The method of claim 4 , wherein the device type for the control device is a switch and the controlled device is a light.

6 . The method of claim 5 , wherein matching the first visible device with the control device is further based on a visible output of the control device that is recognized in the one or more images.

7 . The method of claim 1 , wherein the one or more images are captured using an imaging device that is associated with the host device.

8 . The method of claim 1 , further comprising:

defining an association between the control device and the controlled device to enable the host device to change the operating state of the controlled device.

9 . A non-transitory computer-readable storage device including program instructions executable by one or more processors that, when executed, cause the one or more processors to perform operations, the operations comprising:

receiving a signal that indicates a location of a control device, wherein the control device is configured to change an operating state of a controlled device;

identifying a first visible device and a second visible device in one or more images;

matching the first visible device with the control device based on a location of the first visible device matching the location of the control device;

matching the second visible device with the controlled device;

pairing the control device with a host device; and

controlling the control device using the host device to change the operating state of the controlled device.

10 . The non-transitory computer-readable storage device of claim 9 , wherein matching the second visible device with the controlled device comprises:

changing the operating state of the controlled device using the control device; and

detecting a change of an operating state of the second visible device in the one or more images.

11 . The non-transitory computer-readable storage device of claim 9 , wherein matching the second visible device with the controlled device is based on a user input that identifies the controlled device.

12 . The non-transitory computer-readable storage device of claim 9 , wherein the signal further indicates a device type for the control device, and matching the first visible device with the control device is further based on the device type for the control device matching a device type for the first visible device.

13 . The non-transitory computer-readable storage device of claim 12 , wherein the device type for the control device is a switch and the controlled device is a light.

14 . The non-transitory computer-readable storage device of claim 13 , wherein matching the first visible device with the control device is further based on a visible output of the control device that is recognized in the one or more images.

15 . The non-transitory computer-readable storage device of claim 9 , wherein the one or more images are captured using an imaging device that is associated with the host device.

16 . The non-transitory computer-readable storage device of claim 9 , wherein the operations further comprise:

defining an association between the control device and the controlled device to enable the host device to change the operating state of the controlled device.

17 . A system, comprising:

a memory; and

one or more processors that are configured to execute instructions that are stored in the memory, wherein the instructions, when executed, cause the one or more processors to:

receive a signal that indicates a location of a control device, wherein the control device is configured to change an operating state of a controlled device;

identify a first visible device and a second visible device in one or more images;

match the first visible device with the control device based on a location of the first visible device matching the location of the control device;

match the second visible device with the controlled device;

pair the control device with a host device; and

control the control device using the host device to change the operating state of the controlled device.

18 . The system of claim 17 , wherein the instructions that cause the one or more processors to match the second visible device with the controlled device further cause the one or more processors to:

change the operating state of the controlled device using the control device; and

detect a change of an operating state of the second visible device in the one or more images.

19 . The system of claim 17 , wherein the instructions that cause the one or more processors to match the second visible device with the controlled device further cause the one or more processors to obtain a user input that identifies the controlled device.

20 . The system of claim 17 , wherein the signal further indicates a device type for the control device, and matching the first visible device with the control device is further based on the device type for the control device matching a device type for the first visible device.

21 . The system of claim 20 , wherein the device type for the control device is a switch and the controlled device is a light.

22 . The system of claim 21 , wherein the instructions that cause the one or more processors to match the first visible device with the control device further cause the one or more processors to recognize a visible output of the control device in the one or more images.

23 . The system of claim 17 , wherein the one or more images are captured using an imaging device that is associated with the host device.

24 . The system of claim 17 , wherein the instructions, when executed, further cause the one or more processors to:

define an association between the control device and the controlled device to enable the host device to change the operating state of the controlled device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2026
From: NORRIS, JEFFREY S.; SOMMER, BRUNO M.; DA VEIGA, ALEXANDRE
To: APPLE INC.
Reel/Frame 073584/0030 →
Continuity (5)
Continuation 17980057 · Nov 3, 2022
Continuation 17557206 · Dec 21, 2021
Continuation 16878249 · May 19, 2020
Provisional Application 62850596 · May 21, 2019
Related Publication 20240339027A1 · Oct 10, 2024
References Cited (28)
US 9380621B2 · Jamal-Syed et al. · 2016 [cited by applicant]
US 11210932B2 · Norris et al. · 2021 [cited by applicant]
US 20130042190A1 · Ciloci · 2013 [cited by applicant]
US 20140096220A1 · Da Cruz Pinto et al. · 2014 [cited by applicant]
US 20140225924A1 · Loxam et al. · 2014 [cited by applicant]
US 20140226851A1 · Alberth, Jr. · 2014 [cited by examiner]
US 20150120000A1 · Coffey et al. · 2015 [cited by applicant]
US 20150133051A1 · Jamal-Syed et al. · 2015 [cited by applicant]
US 20150372754A1 · Choi et al. · 2015 [cited by applicant]
US 20160062572A1 · Yang et al. · 2016 [cited by applicant]
US 20160165170A1 · McRae · 2016 [cited by applicant]
US 20160217617A1 · Barribeau · 2016 [cited by applicant]
US 20170061692A1 · Giraldi et al. · 2017 [cited by applicant]
US 20170228935A1 · Foster · 2017 [cited by examiner]
US 20170244811A1 · Mckenzie et al. · 2017 [cited by applicant]
US 20180197009A1 · Severn et al. · 2018 [cited by applicant]
US 20180300551A1 · Luccin et al. · 2018 [cited by applicant]
US 20180314406A1 · Powderly et al. · 2018 [cited by applicant]
US 20180314416A1 · Powderly et al. · 2018 [cited by applicant]
US 20190327775A1 · Ko · 2019 [cited by examiner]
CN 101636935A · 2010 [cited by applicant]
CN 102498725A · 2012 [cited by applicant]
CN 105955043A · 2016 [cited by applicant]
CN 108475326A · 2018 [cited by applicant]
CN 108604129A · 2018 [cited by applicant]
Melnick, Kyle, “Navigate Your Smart Home Devices in AR”, VR Scout, Jul. 1, 2018, https://vrscout.com/news/smart-home-devices-in-ar/ (10 pp). [cited by applicant]
Hayo, “Hayo Transforms The Objects And Spaces Around You Into A Set of Magical Remote Controls”, https://hayo.io/, Downloaded Jan. 15, 2019 (8 pp). [cited by applicant]
Sorgalla, Jonas et. al., “ARGI: Augmented Reality for Gesture-Based Interaction In Variable Smart Environments”, In Proceedings of the 13th Intl Joint Conference on Computer Vision, Imaging and Computer Graphics Theory … [cited by applicant]