IP Library › Granted Patent US 10,497,356
Granted Patent B2
US 10,497,356 · App. 15/572,047 · Granted Dec 3, 2019

Directionality control system and sound output control method

Inventors: Masanari Miyamoto (Fukuoka, JP); Hiroyuki Matsumoto (Fukuoka, JP); Ryouichi Yuge (Fukuoka, JP); Shintaro Yoshikuni (Fukuoka, JP); Toshimichi Tokuda (Fukuoka, JP); Naoya Tanaka (Fukuoka, JP)
Assignee: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
G10K11/17827G10K11/17885H04N7/183H04R1/406H04R3/005H04R3/12H04R29/002G10K2210/102G10K2210/12G10K2210/3045H04R2430/01H04R2499/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,497,356
App. No.
15/572,047
Granted
Dec 3, 2019
Kind
B2
Abstract

In a directionality control system, a camera device captures a video of image capture area. A microphone array device collects a sound in image capture area. A signal processing section detects a sound source of the sound in image capture area, which is collected by the microphone array device. In a case where the detected sound source is within a range of privacy area, an output control section controls the sound in image capture area, which is collected by the microphone array device and is output from speaker device.

Claims (67)

1. A directionality control system, comprising:

a camera that images a video of an image capture area and that is installed in a room;

a sound collector that collects a sound in the image capture area and that is installed in the room;

a display that displays video data in the image capture area that is captured by the camera and that is installed outside of the room;

a sound outputter that outputs the sound in the image capture area that is collected by the sound collector and that is installed outside of the room;

a first memory that stores positional information of a sound masking area, which is part of the image capture area, and which is designated with respect to the video data of the image capture area, that is displayed on the display;

a detector that detects a sound source of the sound in the image capture area that is collected by the sound collector; and

an output controller that determines whether or not the sound source of the sound is within a range of the sound masking area,

wherein the output controller controls output of the sound by the sound outputter in the image capture area, which is collected by the sound collector to mask the sound so that content of the sound cannot be heard, in a case where the sound source detected by the detection section is within a range of the sound masking area,

wherein the output controller causes the sound outputter to output the sound that is collected by the sound collector as it is, in a case where the sound source detected by the detector is outside of the range of the sound masking area, and

wherein the display displays the video data in the image capture area, including the privacy area, without video masking.

2. The directionality control system of claim 1 ,

wherein in a case where the sound source detected by the detector is within the range of the privacy area, the output controller controls the output by the sound outputter when a sound pressure of the sound in the image capture area, which is collected by the sound collector, exceeds a threshold value, and omits the control of the output by the sound outputter when sound pressure of the sound in the image capture area which is collected by the sound collector, does not exceed the threshold value.

3. The directionality control system of claim 1 ,

wherein the output controller substitutes a predetermined substitute for the sound collected by the sound collector and outputs the substituted sound from the sound outputter.

4. The directionality control system of claim 1 ,

wherein the output controller restricts the output of the sound collected by the sound collector.

5. The directionality control system of claim 1 ,

wherein the output controller performs processing on the sound collected by the sound collector and outputs the processed sound.

6. The directionality control system of claim 1 , further comprising:

a positional information setter that sets coordinates of a designated position as the positional information of the privacy area in accordance with an operation of designating the privacy area with respect to the display.

7. The directionality control system of claim 1 , further comprising:

a second memory that stores the sound in the image capture area which is collected by the sound collector together with a sound collecting time,

wherein the output controller controls the output of the sound by the sound outputter which is collected from a predetermined time before a time when the sound source is detected and is stored in the second memory, in a case where the sound source detected by the detector is within the range of the privacy area.

8. The directionality control system of claim 1 ,

wherein the first memory further stores positional information of a target designated with respect to the video data of the image capture area which is displayed on the display,

wherein the detector includes

a sound emphasizer that emphasizes a sound in a first direction toward the target from the sound collector and emphasizes a sound in a second direction toward the privacy area from the sound collector by using the pieces of positional information of the target and the privacy area, and

an utterance determiner that determines whether or not utterance has occurred in each of the target and the privacy area, on the basis of the sound in the first direction and the sound in the second direction, which are emphasized by the sound emphasizer.

9. The directionality control system of claim 8 ,

wherein the output controller adds a masked sound to the emphasized sound in the first direction toward the target, and outputs the sound obtained by the addition, in a case the utterance determiner determines that utterance has occurred in the privacy area.

10. The directionality control system of claim 9 ,

wherein the output controller generates the masked sound on the basis of the sound in the second direction, which is emphasized by the sound emphasizer.

11. The directionality control system of claim 10 ,

wherein the first memory stores a plurality of sounds collected in the privacy area in the past, and

wherein the output controller adds the plurality of past collected sounds read out from the first memory to the emphasized sound in the second direction which is emphasized by the sound emphasizer to thereby generate the masked sound.

12. The directionality control system of claim 9 ,

wherein the masked sound is stored in the first memory in advance, and

wherein the output controller reads out the masked sound from the first memory and acquires the read-out masked sound.

13. The directionality control system of claim 9 ,

wherein the output controller adjusts a sound volume of the masked sound on the basis of the pieces of positional information of the target and the privacy area.

14. The directionality control system of claim 8 ,

wherein the output controller restricts the output of the sound in the first direction into which the sound in the privacy area has leaked.

15. The directionality control system of claim 8 ,

wherein the output controller performs processing on the sound in the first direction into which the sound in the privacy area has leaked, and outputs the processed sound.

16. The directionality control system of claim 8 , further comprising:

a position setter that sets coordinates of a designated position as the positional information of the privacy area in accordance with an operation of designating the privacy area with respect to the display.

17. The directionality control system of claim 8 ,

wherein the first memory stores the sound in the image capture area which is collected by the sound collector together with a sound collecting time, and

wherein the output controller controls the output of the sound in the image capture area which is stored in the first memory, on the basis of sound collecting performed in a predetermined time before a time when the sound is detected in the privacy area.

18. A sound output control method in a directionality control system including a camera and a sound collector, the sound output control method comprising:

imaging a video of an image capture area by the camera that is installed in a room;

collecting a sound in the image capture area by the sound collector that is installed in the room;

displaying, on a display, video data in the image capture area that is captured by the camera, the display being installed outside of the room;

outputting sound, by a sound outputter, sound in the image capture area that is collected by the sound collector, the sound outputter being installed outside the room;

storing positional information of a sound masking area, which is part of the image capture area, and which is designated with respect to a display on which the video data of the image capture area is displayed, in a first memory;

detecting, by a detector, a sound source of the sound in the image capture area that is collected by the sound collector;

determining, by an output controller, whether or not the sound source of the sound is within range of the sound masking area, and

controlling, by an output controller, output of the sound by the sound outputter in the image capture area, which is collected by the sound collector to mask the sound so that content of the sound cannot be heard in a case where the detected sound source is within a range of the sound masking area,

wherein the output controller causes the sound outputter to output the sound that is collected by the sound collector as it is, in a case where the sound source detected by the detector is outside of the range of the sound masking area, and

wherein the display displays the video data in the image capture area, including the privacy area, without video masking.

19. The sound output control method of claim 18 , further comprising:

storing positional information of a target designated with respect to the display on which the video data of the image capture area is displayed, in the first memory,

wherein the detecting of the sound source of the sound in the image capture area which is collected by the sound collector includes

emphasizing a sound in a first direction toward the target from the sound collector and emphasizing a sound in a second direction toward the privacy area from the sound collector by using the pieces of positional information of the target and the privacy area which are stored in the first memory, and

determining whether or not utterance has occurred in each of the target and the privacy area, on the basis of the emphasized sound in the first direction and the emphasized sound in the second direction, and

wherein output of the sound in the first direction into which the sound in the privacy area has leaked is controlled in a case where utterance has occurred in the privacy area, as the controlling of the output of the sound in the image capture area which is collected by the sound collector, in a case where the detected sound source is within the range of the privacy area.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 28, 2018
From: MIYAMOTO, MASANARI; MATSUMOTO, HIROYUKI; YUGE, RYOUICHI; YOSHIKUNI, SHINTARO; TOKUDA, TOSHIMICHI; TANAKA, NAOYA
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 044750/0178 →
Priority Claims (2)
JP 2015-101357 · May 18, 2015 · national
JP 2016-005378 · Jan 14, 2016 · national
Continuity (1)
Related Publication 20180158446A1 · Jun 7, 2018