IP Library Granted Patent US 11,245,840
Granted Patent B2
US 11,245,840 · App. 16/992,970 · Granted Feb 8, 2022

Method and system for dynamically adjusting camera shots

Inventors: Bryan Kelly (Sonoma, CA); Kelly Lea O'Leary (Sonoma, CA)
Assignee: EIKON TECHNOLOGIES LLC
H04N5/23222G10L25/90H04N5/23203H04N5/23219H04N5/23245H04N5/772
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,245,840
App. No.
16/992,970
Granted
Feb 8, 2022
Kind
B2
Abstract

An approach is provided for adjusting camera shots. The approach involves receiving, via a sensor of a mobile device, an audio signal during a video recording of a subject by a camera of the mobile device. The approach also involves determining, at the mobile device, an audio level in a vicinity of the subject based on the received audio signal, wherein the audio level represents a change in pitch, pace, and cadence of a sound produced by the subject. The approach additionally involves determining that the audio level triggers a shot adjustment state. In the alternative or in addition to, the shot adjustment state is triggered by facial expression of the subject. The approach further involves dynamically adjusting, in response to the shot adjustment state, one or more camera parameters of the camera to alter a shot of the subject by the camera during the video recording. The camera parameters relate to either zoom control, aperture, lighting, or a combination thereof.

Claims (69)

1. A method comprising:

receiving, via a sensor of a mobile device, an audio signal during a video recording of a subject by a camera of the mobile device;

determining, at the mobile device, an audio level in a vicinity of the subject based on the received audio signal, wherein the audio level represents a change in pitch, pace, and cadence of a sound produced by the subject;

detecting facial expression, posture, and gesture of the subject;

determining that the audio level, the facial expression, the posture, and the gesture trigger a shot adjustment state; and

dynamically adjusting, in response to the shot adjustment state, one or more camera parameters of the camera to alter a shot of the subject by the camera during the video recording, wherein the one or more camera parameters relate to zoom control, aperture, lighting, or a combination thereof.

2. The method of claim 1 , further comprising:

establishing a wireless connection with another mobile device in the vicinity of the subject, wherein the another mobile device comprises a camera configured to concurrently record the subject and a microphone configured to generate a second audio signal during the video recording; and

receiving, by the mobile device, the second audio signal from the another mobile device, wherein the determination of the audio level is further based on the second audio signal.

3. The method of claim 2 , further comprising:

generating a camera instruction message to control the camera of the another mobile device according to the determined shot adjustment state of the camera of the mobile device.

4. The method of claim 1 , further comprising:

detecting speech of the subject;

determining pitch, pace, and cadence of the speech; and

generating a camera instruction message to change zoom level or aperture setting of the camera based on the determined pitch, pace, and cadence of the speech.

5. The method of claim 1 , further comprising:

superimposing a computer-generated image over the subject based on the facial expression of the subject, wherein the superimposed computer-generated image changes with the dynamic adjustment of the camera, the audio level, the facial expression, or a combination thereof.

6. The method of claim 1 , further comprising:

processing, using a machine learning model, audio data, video data, or a combination thereof associated with the subject to dynamically adjust the one or more camera parameters of the camera; and

identifying, using the machine learning model, the audio level, the facial expression, or a combination thereof of the subject to trigger the shot adjustment state.

7. The method of claim 1 , further comprising:

receiving, via the another sensor of the mobile device, electromagnetic waves during the video recording of the subject, wherein the electromagnetic waves specify one or more biometric data of the subject;

determining a behavior of the subject by comparing the one or more biometric data with a prescribed range of biometric parameters;

selecting one of a plurality of film modes that specify pre-set settings for the determined behavior of the subject; and

dynamically adjust the one or more camera parameters based, at least in part, on the selected film mode.

8. The method of claim 7 , wherein the biometric data comprise body temperature information, heart rate information, blood glucose level information, or a combination thereof of the subject, and wherein the prescribed range of biometric parameters comprise body temperature range, heart rate range, blood glucose level range, or a combination thereof.

9. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive, via a sensor of a mobile device, an audio signal during a video recording of a subject by a camera of the mobile device;

determine, at the mobile device, an audio level in a vicinity of the subject based on the received audio signal, wherein the audio level represents a change in pitch, pace, and cadence of a sound produced by the subject;

detect facial expression, posture, and gesture of the subject;

determine that the audio level, the facial expression, the posture, and the gesture trigger a shot adjustment state; and

dynamically adjust, in response to the shot adjustment state, one or more camera parameters of the camera to alter a shot of the subject by the camera during the video recording, wherein the one or more camera parameters relate to zoom control, aperture, lighting, or a combination thereof.

10. The apparatus of claim 9 , wherein the apparatus is further caused to:

establish a wireless connection with another mobile device in the vicinity of the subject, wherein the another mobile device comprises a camera configured to concurrently record the subject and a microphone configured to generate a second audio signal during the video recording; and

receive, by the mobile device, the second audio signal from the another mobile device, wherein the determination of the audio level is further based on the second audio signal.

11. The apparatus of claim 10 , wherein the apparatus is further caused to:

generate a camera instruction message to control the camera of the another mobile device according to the determined shot adjustment state of the camera of the mobile device.

12. The apparatus of claim 9 , wherein the apparatus is further caused to:

detect speech of the subject;

determine pitch, pace, and cadence of the speech; and

generate a camera instruction message to change zoom level or aperture setting of the camera based on the determined pitch, pace, and cadence of the speech.

13. The apparatus of claim 9 , wherein the apparatus is further caused to:

superimpose a computer-generated image over the subject based on the facial expression of the subject, wherein the superimposed computer-generated image changes with the dynamic adjustment of the camera, the audio level, the facial expression, or a combination thereof.

14. The apparatus of claim 9 , wherein the apparatus is further caused to:

process, using a machine learning model, audio data, video data, or a combination thereof associated with the subject to dynamically adjust the one or more camera parameters of the camera; and

identify, using the machine learning model, the audio level, the facial expression, or a combination thereof of the subject to trigger the shot adjustment state.

15. The apparatus of claim 9 , wherein the apparatus is further caused to:

receive, via another sensor of the mobile device, electromagnetic waves during the video recording of the subject, wherein the electromagnetic waves specify one or more biometric data of the subject;

determine a behavior of the subject by comparing the one or more biometric data with a prescribed range of biometric parameters;

select one of a plurality of film modes that specify pre-set settings for the determined behavior of the subject; and

dynamically adjust the one or more camera parameters based, at least in part, on the selected film mode.

16. A system comprising:

a mobile device configured to receive, via a sensor, an audio signal during a video recording of a subject by a camera of the mobile device;

an audio processor configured to determine an audio level in a vicinity of the subject based on the received audio signal, wherein the audio level represents a change in pitch, pace, and cadence of a sound produced by the subject;

a machine learning component configured to detect facial expression, posture, and gesture of the subject;

a shot adjustment component configured to determine that the audio level, the facial expression, the posture, and the gesture trigger a shot adjustment state, and to instruct a camera controller within the mobile device to dynamically adjust, in response to the shot adjustment state, one or more camera parameters of the camera to alter shot of the subject by the camera during the video recording,

wherein the one or more camera parameters relate to either zoom control, aperture, lighting, or a combination thereof.

17. The system of claim 16 , further comprising:

a wireless communication circuitry configured to establish a wireless connection with another mobile device in the vicinity of the subject, wherein the another mobile device comprises a camera configured to concurrently record the subject and a microphone configured to generate a second audio signal during the video recording,

wherein the wireless communication circuitry being further configured to receive the second audio signal from the another mobile device, and wherein the determination of the audio level is further based on the second audio signal.

18. The system of claim 17 , further comprising:

a shot adjustment component configured to generate a camera instruction message to control the camera of the another mobile device according to the determined shot adjustment state of the camera of the mobile device.

19. The system of claim 16 , further comprising:

a shot adjustment component configured to generate a camera instruction message to change zoom level or aperture setting of the camera based on the determined pitch, pace, and cadence of the speech.

20. The system of claim 16 , further comprising:

a sensor configured to receive electromagnetic waves during the video recording of the subject, wherein the electromagnetic waves specify one or more biometric data of the subject, wherein the one or more biometric data is used to select one of a plurality of film modes for adjustment of the one or more camera parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2021
From: KELLY, BRYAN PATRICK; O'LEARY, KELLY LEA
To: EIKON TECHNOLOGIES LLC
Reel/Frame 057863/0757 →
Continuity (3)
Continuation In Part 15970564 · May 3, 2018
Provisional Application 62639230 · Mar 6, 2018
Related Publication 20200374456A1 · Nov 26, 2020
Cited By (2)
US 12,293,026 US 12,432,443