IP Library Granted Patent US 10,447,394
Granted Patent B2
US 10,447,394 · App. 15/706,322 · Granted Oct 15, 2019

Connection with remote internet of things (IoT) device based on field of view of camera

Inventors: Vishnu Vardhan Kasilya Sudarsan (Bangalore, IN); Ganesh Babu Ketti Gangadharan (Bangalore, IN); Rameshkumar Karuppusamy (Bengaluru, IN); Hariprasad Kadavamganam (Bangalore, IN); Somya Gunjan (Bangalore, IN)
Assignee: QUALCOMM Incorporated
H04B10/116G11B27/031G11B27/34H04M1/7253H04N7/147H04N7/185H04W4/38H04W40/246H05B37/0245
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,447,394
App. No.
15/706,322
Granted
Oct 15, 2019
Kind
B2
Abstract

This disclosure provides systems, methods, and apparatus, including computer programs encoded on computer storage media, for network communication when recording audio and video (A/V) of a subject. In one aspect, a device may determine one or more internet of things (IoT) devices that are displayed within a field of view (FoV) of a camera of the device, and that are capable of remote audio recording and audio streaming. The device may select a first IoT device having a first microphone. The device may receive an audio stream from the first IoT device after selecting the first IoT device, the audio stream including the audio of the subject remotely recorded by the first IoT device for the device. When the FoV of the camera changes, the device may dynamically switch from receiving the audio stream from a first IoT device to receiving the audio stream from a second IoT device.

Claims (89)

1. A method for network communication performed by a computing device for recording audio and video (A/V), comprising:

determining, by the computing device, a plurality of internet of things (IoT) devices that are displayed within a field of view (FoV) of a camera of the computing device and that are capable of remote audio recording and audio streaming;

determining, within the FoV of the camera of the computing device, distance measurements from each of the plurality of IoT devices displayed within the FoV to a focal point of the FoV;

selecting, by the computing device, a first IoT device from the plurality of IoT devices based, at least in part, on the distance measurements, the first IoT device having a first microphone that is capable of remotely recording audio of a subject for the computing device; and

receiving, by the computing device, an audio stream from the first IoT device after selecting the first IoT device, the audio stream including the audio of the subject remotely recorded by the first IoT device for the computing device.

2. The method of claim 1 , wherein selecting the first IoT device from the plurality of IoT devices based, at least in part, on the distance measurements comprises:

determining, based on the distance measurements, that the first IoT device from the plurality of IoT devices is located closest to the focal point of the FoV; and

selecting the first IoT device from the plurality of IoT devices based, at least in part, on determining that the first IoT device is located closest to the focal point of the FoV.

3. The method of claim 1 , further comprising:

determining a microphone type and performance characteristics associated with each microphone of each of the plurality of IoT devices; and

comparing the microphone type and performance characteristics associated with each microphone of each of the plurality of IoT devices,

wherein selecting the first IoT device from the plurality of IoT devices is further based, at least in part, on the microphone type and performance characteristics associated with the first microphone of the first IoT device.

4. The method of claim 1 , further comprising:

determining that the first IoT device and a second IoT device of the plurality of IoT devices are displayed within the focal area of the FoV of the camera of the computing device;

receiving a first audio stream from the first IoT device and a second audio stream from the second IoT device; and

comparing audio quality characteristics of the first audio stream to audio quality characteristics of the second audio stream,

wherein selecting the first IoT device is further based, at least in part, on the audio quality characteristics of the first audio stream received from the first IoT device.

5. The method of claim 1 , further comprising:

receiving, by the computing device, user input selecting the first IoT device having the first microphone,

wherein selecting the first IoT device from the plurality of IoT devices is further based, at least in part, on the user input.

6. The method of claim 1 , further comprising:

receiving, by the computing device, user input selecting a second IoT device from the plurality of IoT devices, the second IoT device having a second microphone that is capable of remotely recording the audio of the subject;

overriding a selection of the first IoT device by the computing device based, at least in part, on receiving the user input selecting the second IoT device; and

selecting the second IoT device from the plurality of IoT devices based, at least in part, on the user input.

7. The method of claim 1 , further comprising:

recording video of the subject using the camera of the computing device; and

generating an A/V recording of the subject using the video of the subject recorded by the camera of the computing device and the audio of the subject remotely recorded by the first microphone of the first IoT device.

8. The method of claim 1 , further comprising:

detecting a change in the FoV of the camera of the computing device; and

determining to switch from receiving the audio stream from the first IoT device to receiving an audio stream from a second IoT device after detecting the change in the FoV of the camera, the second IoT device having a second microphone that is capable of remotely recording the audio of the subject.

9. The method of claim 8 , wherein determining to switch from receiving the audio stream from the first IoT device to receiving the audio stream from the second IoT device after detecting the change in the FoV of the camera comprises:

determining that the second IoT device is located closer to the focal point of the FoV than the first IoT device after detecting the change in the FoV of the camera; and

determining to switch from receiving the audio stream from the first IoT device to receiving the audio stream from the second IoT device in response to determining that the second IoT device is located closer to the focal point of the FoV than the first IoT device after detecting the change in the FoV of the camera.

10. The method of claim 1 , further comprising:

determining the audio stream from the first IoT device is not received at the computing device for a period of time; and

determining to switch to recording the audio of the subject using a local microphone of the camera of the computing device in response to determining the audio stream from the first IoT device is not received at the computing device for the period of time.

11. The method of claim 1 , wherein determining the plurality of IoT devices that are displayed within the FoV of the camera of the computing device and that are capable of remote audio recording and audio streaming comprises:

obtaining a map of locations of the plurality of IoT devices in a local network from a master device of the local network;

determining that the plurality of IoT devices each have a microphone and are capable of remote audio recording and audio streaming; and

displaying a visual indicator within the FoV of the camera for each of the plurality of IoT devices, the visual indicator indicating a location of each of the plurality of IoT devices within the FoV of the camera and indicating each of the plurality of IoT devices is capable of remote audio recording and audio streaming.

12. The method of claim 1 , wherein determining the plurality of IoT devices that are displayed within the FoV of the camera of the computing device and that are capable of remote audio recording and audio streaming comprises at least one member selected from the group consisting of:

obtaining information from a master device of a local network, the information indicating which of the plurality of IoT devices are capable of remote audio recording and audio streaming; and

obtaining information from each of the plurality of IoT devices, the information indicating which of the plurality of IoT devices are capable of remote audio recording and audio streaming.

13. The method of claim 1 , wherein the computing device is a mobile phone, a tablet computer, a pair of smart glasses, a dedicated camera, or a drone.

14. The method of claim 1 , wherein determining, within the FoV of the camera of the computing device, the distance measurements from each of the plurality of IoT devices displayed within the FoV to the focal point of the FoV further comprises:

determining coordinates within the FoV of the camera for the focal point of the FoV, for the first IoT device displayed within the FoV, and for a second IoT device displayed within the FoV; and

determining the distance measurements, within the FoV of the camera, from each of the plurality of IoT devices displayed within the FoV to the focal point of the FoV based, at least in part, on the coordinates within the FoV of the camera.

15. A computing device, comprising:

a processor; and

memory having instructions stored therein which, when executed by the processor cause the computing device to:

determine a plurality of internet of things (IoT) devices that are displayed within a field of view (FoV) of a camera of the computing device and that are capable of remote audio recording and audio streaming;

determine, within the FoV of the camera of the computing device, distance measurements from each of the plurality of IoT devices displayed within the FoV to a focal point of the FoV;

select a first IoT device from the plurality of IoT devices based, at least in part, on the distance measurements, the first IoT device having a first microphone that is capable of remotely recording audio of a subject for the computing device; and

receive an audio stream from the first IoT device after selection of the first IoT device, the audio stream including the audio of the subject remotely recorded by the first IoT device for the computing device.

16. The computing device of claim 15 , wherein the instructions, when executed by the processor to select the first IoT device from the plurality of IoT devices based, at least in part, on the distance measurements, further cause the computing device to:

determine, based on the distance measurements, that the first IoT device from the plurality of IoT devices is located closest to the focal point of the FoV; and

select the first IoT device from the plurality of IoT devices based, at least in part, on a determination that the first IoT device is located closest to the focal point of the FoV.

17. The computing device of claim 15 , wherein the instructions, when executed by the processor, further cause the computing device to:

determine a microphone type and performance characteristics associated with each microphone of each of the plurality of IoT devices; and

compare the microphone type and performance characteristics associated with each microphone of each of the plurality of IoT devices,

wherein selection of the first IoT device from the plurality of IoT devices is further based, at least in part, on the microphone type and performance characteristics associated with the first microphone of the first IoT device.

18. The computing device of claim 15 , wherein the instructions, when executed by the processor, further cause the computing device to:

receive user input selecting the first IoT device having the first microphone,

wherein selection of the first IoT device from the plurality of IoT devices is further based, at least in part, on the user input.

19. The computing device of claim 15 , wherein the instructions, when executed by the processor, further cause the computing device to:

record video of the subject using the camera of the computing device; and

generate an A/V recording of the subject using the video of the subject recorded by the camera of the computing device and the audio of the subject remotely recorded by the first microphone of the first IoT device.

20. The computing device of claim 15 , wherein the instructions, when executed by the processor, further cause the computing device to:

detect a change in the FoV of the camera of the computing device; and

determine to switch from reception of the audio stream from the first IoT device to reception of an audio stream from a second IoT device after detection of the change in the FoV of the camera, the second IoT device having a second microphone that is capable of remotely recording the audio of the subject.

21. The computing device of claim 15 , wherein the instructions, when executed by the processor to determine the plurality of IoT devices that are displayed within the FoV of the camera of the computing device and that are capable of remote audio recording and audio streaming, further cause the computing device to:

obtain a map of locations of the plurality of IoT devices in a local network from a master device of the local network;

determine that the plurality of IoT devices each have a microphone and are capable of remote audio recording and audio streaming; and

display a visual indicator within the FoV of the camera for each of the plurality of IoT devices, the visual indicator indicating a location of each of the plurality of IoT devices within the FoV of the camera and indicating each of the plurality of IoT devices is capable of remote audio recording and audio streaming.

22. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor of a computing device, cause the computing device to:

determine a plurality of internet of things (IoT) devices that are displayed within a field of view (FoV) of a camera of the computing device and that are capable of remote audio recording and audio streaming;

determine, within the FoV of the camera of the computing device, distance measurements from each of the plurality of IoT devices displayed within the FoV to a focal point of the FoV;

select a first IoT device from the plurality of IoT devices based, at least in part, on the distance measurements, the first IoT device having a first microphone that is capable of remotely recording audio of a subject for the computing device; and

receive an audio stream from the first IoT device after selection of the first IoT device, the audio stream including the audio of the subject remotely recorded by the first IoT device for the computing device.

23. The non-transitory computer-readable storage medium of claim 22 , wherein the instructions, when executed by the processor, further cause the computing device to:

determine, based on the distance measurements, that the first IoT device from the plurality of IoT devices is located closest to the focal point of the FoV; and

select the first IoT device from the plurality of IoT devices based, at least in part, on a determination that the first IoT device is located closest to the focal point of the FoV.

24. The non-transitory computer-readable storage medium of claim 22 , wherein the instructions, when executed by the processor, further cause the computing device to:

determine a microphone type and performance characteristics associated with each microphone of each of the plurality of IoT devices; and

compare the microphone type and performance characteristics associated with each microphone of each of the plurality of IoT devices,

wherein selection of the first IoT device from the plurality of IoT devices is further based, at least in part, on the microphone type and performance characteristics associated with the first microphone of the first IoT device.

25. The non-transitory computer-readable storage medium of claim 22 , wherein the instructions, when executed by the processor, further cause the computing device to:

detect a change in the FoV of the camera of the computing device; and

determine to switch from reception of the audio stream from the first IoT device to reception of an audio stream from a second IoT device after detection of the change in the FoV of the camera, the second IoT device having a second microphone that is capable of remotely recording the audio of the subject.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2017
From: KASILYA SUDARSAN, VISHNU VARDHAN; KETTI GANGADHARAN, GANESH BABU; KARUPPUSAMY, RAMESHKUMAR; KADAVAMGANAM, HARIPRASAD; GUNJAN, SOMYA
To: QUALCOMM INCORPORATED
Reel/Frame 044058/0815 →
Continuity (1)
Related Publication 20190089456A1 · Mar 21, 2019
Cited By (1)
US 12,293,470