IP Library Granted Patent US 11,532,227
Granted Patent B2
US 11,532,227 · App. 17/557,206 · Granted Dec 20, 2022

Discovery of and connection to remote devices

Inventors: Jeffrey S. Norris (Saratoga, CA); Bruno M. Sommer (Sunnyvale, CA); Alexandre Da Veiga (San Francisco, CA)
Assignee: APPLE INC.
G08C17/02G06K9/6201G06V10/22H04W8/005H04W76/10G08C2201/31
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,532,227
App. No.
17/557,206
Granted
Dec 20, 2022
Kind
B2
Abstract

A method includes obtaining a location and a device type for one or more remote devices, and identifying one or more visible devices in one or more images, the one or more visible devices having a location and a device type. The method also includes matching a first visible device from the one or more visible devices with a first remote device from the one or more remote devices based on a location and a device type of the first visible device matching a location and a device type of the first remote device, obtaining a user input, and controlling a function of the first remote device based on the user input.

Claims (41)

1. A method, comprising:

obtaining a location and a device type for one or more remote devices;

identifying one or more visible devices in one or more images, the one or more visible devices having a location and a device type;

matching a first visible device from the one or more visible devices with a first remote device from the one or more remote devices based on a location and a device type of the first visible device matching a location and a device type of the first remote device;

obtaining a user input; and

controlling a function of the first remote device based on the user input.

2. The method of claim 1 , wherein the location of the one or more remote devices and the device type of the one or more remote devices are determined based on signals received from the one or more remote devices.

3. The method of claim 1 , wherein the one or more visible devices are identified in the one or more images using an object detection model, and the object detection model is configured to recognize the device type of the one or more visible devices.

4. The method of claim 1 , wherein the location of the one or more visible devices are determined based on a view direction of an imaging device used to obtain the one or more images.

5. The method of claim 1 , wherein matching the first visible device from the one or more visible devices with the first remote device from the one or more remote devices includes excluding a second remote device from the one or more remote devices as a potential match to the first visible device based on the location of the first visible device not matching a location of the second remote device.

6. The method of claim 1 , wherein matching the first visible device from the one or more visible devices with the first remote device from the one or more remote devices includes excluding a second remote device from the one or more remote devices as a potential match to the first visible device based on the device type of the first visible device not matching a device type of the second remote device.

7. The method of claim 1 , wherein the one or more images of the location are captured using an imaging device that is associated with a host device and the user input is obtained using an input device that is associated with the host device.

8. The method of claim 1 , wherein the user input is directed to the first visible device.

9. A non-transitory computer-readable storage device including program instructions executable by one or more processors that, when executed, cause the one or more processors to perform operations, the operations comprising:

obtaining a location and a device type for one or more remote devices;

identifying one or more visible devices in one or more images, the one or more visible devices having a location and a device type;

matching a first visible device from the one or more visible devices with a first remote device from the one or more remote devices based on a location and a device type of the first visible device matching a location and a device type of the first remote device;

obtaining a user input; and

controlling a function of the first remote device based on the user input.

10. The non-transitory computer-readable storage device of claim 9 , wherein the location of the one or more remote devices and the device type of the one or more remote devices are determined based on signals received from the one or more remote devices.

11. The non-transitory computer-readable storage device of claim 9 , wherein the one or more visible devices are identified in the one or more images using an object detection model, and the object detection model is configured to recognize the device type of the one or more visible devices.

12. The non-transitory computer-readable storage device of claim 9 , wherein the location of the one or more visible devices is determined based on a view direction of an imaging device used to obtain the one or more images.

13. The non-transitory computer-readable storage device of claim 9 , wherein matching the first visible device from the one or more visible devices with the first remote device from the one or more remote devices includes excluding a second remote device from the one or more remote devices as a potential match to the first visible device based on the location of the first visible device not matching a location of the second remote device.

14. The non-transitory computer-readable storage device of claim 9 , wherein matching the first visible device from the one or more visible devices with the first remote device from the one or more remote devices includes excluding a second remote device from the one or more remote devices as a potential match to the first visible device based on the device type of the first visible device not matching a device type of the second remote device.

15. The non-transitory computer-readable storage device of claim 9 , wherein the one or more images of the location are captured using an imaging device that is associated with a host device and the user input is obtained using an input device that is associated with the host device.

16. The non-transitory computer-readable storage device of claim 9 , wherein the user input is directed to the first visible device.

17. A system, comprising:

a memory; and

one or more processors that are configured to execute instructions that are stored in the memory, wherein the instructions, when executed, cause the one or more processors to:

obtain a location and a device type for one or more remote devices;

identify one or more visible devices in one or more images, the one or more visible devices having a location and a device type;

match a first visible device from the one or more visible devices with a first remote device from the one or more remote devices based on a location and a device type of the first visible device matching a location and a device type of the first remote device;

obtain a user input; and

control a function of the first remote device based on the user input.

18. The system of claim 17 , wherein the location of the one or more remote devices and the device type of the one or more remote devices are determined based on signals received from the one or more remote devices.

19. The system of claim 17 , wherein the one or more visible devices are identified in the one or more images using an object detection model, and the object detection model is configured to recognize the device type of the one or more visible devices.

20. The system of claim 17 , wherein the location of the one or more visible devices is determined based on a view direction of an imaging device used to obtain the one or more images.

21. The system of claim 17 , wherein the instructions that cause the one or more processors to match the first visible device from the one or more visible devices with the first remote device from the one or more remote devices further cause the one or more processors to exclude a second remote device from the one or more remote devices as a potential match to the first visible device based on the location of the first visible device not matching a location of the second remote device.

22. The system of claim 17 , wherein the instructions that cause the one or more processors to match the first visible device from the one or more visible devices with the first remote device from the one or more remote devices further cause the one or more processors to exclude a second remote device from the one or more remote devices as a potential match to the first visible device based on the device type of the first visible device not matching a device type of the second remote device.

23. The system of claim 17 , wherein the one or more images of the location are captured using an imaging device that is associated with a host device and the user input is obtained using an input device that is associated with the host device.

24. The system of claim 17 , wherein the user input is directed to the first visible device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2022
From: NORRIS, JEFFREY S.; SOMMER, BRUNO M.; DA VEIGA, ALEXANDRE
To: APPLE INC.
Reel/Frame 061643/0353 →
Continuity (3)
Continuation 16878249 · May 19, 2020
Provisional Application 62850596 · May 21, 2019
Related Publication 20220114882A1 · Apr 14, 2022