IP Library Granted Patent US 9,875,410
Granted Patent B2
US 9,875,410 · App. 14/940,988 · Granted Jan 23, 2018

Camera system for transmitting and receiving an audio signal and operating method of the same

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,875,410
App. No.
14/940,988
Granted
Jan 23, 2018
Kind
B2
Abstract

A camera system includes: a camera configured to capture an image of a surveillance area; a microphone array which includes at least one microphone; and at least one processor configured to implement: a video processor which designates at least one subject in the image as a target; a beam-former which calculates a rotation angle of the microphone array based on a location of the subject; and a driving controller which rotates the microphone array toward the subject based on the rotation angle of the microphone array, wherein the beam-forming unit further performs signal processing on an audio input signal received through the microphone array rotated toward the subject and outputs an audio output signal corresponding to the audio input signal.

Claims (49)

1. A camera system comprising:

a camera configured to capture an image of a surveillance area;

a speaker array which comprises a plurality of speakers,

a microphone array which comprises at least one microphone; and

at least one processor configured to implement:

a video processor which designates at least one subject in the image as a target;

a beam-former which calculates a rotation angle of the microphone array based on a location of the at least one subject; and

a driving controller which rotates the microphone array in a direction toward the at least one subject based on the rotation angle of the microphone array,

wherein the beam-former further performs signal processing on an audio input signal received through the microphone array rotated in the direction toward the at least one subject,

wherein the at least one microphone comprises a plurality of microphones, and the at least one subject comprises a plurality of subjects, and

wherein the at least one processor is further configured to implement an audio processor which generates a plurality of audio output signals based on audio input signals received through the plurality of microphones and controls the beam-former to transmit the plurality of audio output signals toward the plurality of subjects through the plurality of speakers, respectively, in a time division multiplexing manner.

2. The camera system of claim 1 , wherein in response to the at least one subject not being located in a center of the image, the video processor calculates a rotation angle of a lens of the camera, and the driving controller rotates the lens of the camera based on the rotation angle of the lens so that the at least one subject is located in the center of the image, and

wherein in response to the at least one subject being located in the center of the image, the video processor designates the at least one subject as the target.

3. The camera system of claim 1 ,

wherein the beam-former determines a rotation angle of the speaker array based on the rotation angle of the microphone array, and the driving controller rotates the speaker array toward the at least one subject based on the rotation angle of the speaker array.

4. The camera system of claim 1 , wherein the beam-former calculates a distance from the camera to the at least one subject, and calculates the rotation angle of the microphone array using the calculated distance.

5. The camera system of claim 1 , wherein the video processor generates an image analysis result by performing at least one of facial recognition of the at least one subject, a behavioral pattern analysis of the at least one subject, and a situation analysis with respect to the image, and

wherein the audio processor recognizes sound of the at least one subject by matching the image analysis result with the audio input signal received through the microphone array and generates an audio output signal corresponding to the sound.

6. The camera system of claim 1 ,

wherein the beam-former calculates rotations angles of the plurality of microphones based on locations of the plurality of subjects, respectively, and

wherein the driving controller rotates the plurality of microphones toward the plurality of subjects based on the rotation angles of the plurality of microphones so that the beam-former performs the signal processing on the audio input signals received through the plurality of microphones rotated toward the at least one subject, respectively.

7. The camera system of claim 1 , wherein the audio processor generates the plurality of audio output signals which are multiplexed and different in at least one of an amplitude and a phase, respectively.

8. A method of operating a camera system comprising a camera and a microphone array which comprises at least one microphone by using at least one processor, the method comprising:

capturing an image of a surveillance area using the camera;

designating at least one subject in the image as a target;

calculating a rotation angle of the microphone array based on a location of the at least one subject;

rotating the microphone array toward the at least one subject based on the rotation angle of the microphone array; and

receiving an audio input signal through the microphone array rotated toward the at least one subject,

wherein the at least one microphone comprises a plurality of microphones, and the at least one subject comprises a plurality of subjects, and

wherein the method further comprises generating a plurality of audio output signals based on audio input signals received through the plurality of microphones, and transmitting the plurality of audio output signals towards the plurality of subjects, respectively, in a time division multiplexing manner.

9. The method of claim 8 , further comprising:

generating an audio output signal based on the audio input signal received through the microphone array; and

transmitting the audio output signal toward the at least one subject through a speaker array which comprises at least one speaker.

10. The method of claim 8 , further comprising:

determining the location of the at least one subject; and

in response to the at least one subject not being located in a center of the image, calculating a rotation angle of a lens of the camera, and rotating the lens based on the rotation angle of the lens so that the at least one subject is located in the center of the image,

wherein the designating the at least one subject as the target is performed in response to the at least one subject being located in the center of the image.

11. The method of claim 8 , wherein the camera system further comprises a speaker array which comprises at least one speaker, and

wherein the method further comprises:

determining a rotation angle of the speaker array based on the rotation angle of the microphone array, and rotating the speaker array toward the at least one subject based on the rotation angle of the speaker array; and

generating an audio output signal toward the at least one subject corresponding to the audio input signal received through the microphone array.

12. The method of claim 8 , further comprising calculating a distance from the camera to the at least one subject which is used to calculate the rotation angle of the microphone array.

13. The method of claim 8 , further comprising:

generating an image analysis result by performing at least one of facial recognition of the at least one subject, a behavioral pattern analysis of the at least one subject, and a situation analysis with respect to the image; and

recognizing sound of the at least one subject by matching the image analysis result with the audio input signal received through the microphone array and generating an audio output signal corresponding to the sound.

14. The method of claim 8 ,

wherein the calculating the rotation angle of the microphone array comprises calculating rotations angles of the plurality of microphones based on locations of the plurality of subjects, respectively, and

wherein the rotating the microphone array toward the at least one subject comprises rotating the plurality of microphones toward the plurality of subjects based on the rotation angles of the plurality of microphones to receive the audio input signals through the plurality of microphones rotated toward the at least one subject, respectively.

15. The method of claim 8 , wherein the plurality of audio output signals are multiplexed and different in at least one of an amplitude and a phase, respectively.

Assignments (5)
CHANGE OF NAME Recorded Aug 10, 2023
From: HANWHA TECHWIN CO., LTD.
To: HANWHA VISION CO., LTD.
Reel/Frame 064549/0075 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2019
From: HANWHA AEROSPACE CO., LTD.
To: HANWHA TECHWIN CO., LTD.
Reel/Frame 049013/0723 →
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION NUMBER 10/853,669. IN ADDITION PLEASE SEE EXHIBIT A PREVIOUSLY RECORDED ON REEL 046927 FRAME 0019. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Jan 17, 2019
From: HANWHA TECHWIN CO., LTD.
To: HANWHA AEROSPACE CO., LTD.
Reel/Frame 048496/0596 →
CHANGE OF NAME Recorded Aug 24, 2018
From: HANWHA TECHWIN CO., LTD
To: HANWHA AEROSPACE CO., LTD.
Reel/Frame 046927/0019 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2015
From: OH, JAEYOON
To: HANWHA TECHWIN CO., LTD.
Reel/Frame 037051/0202 →