Millimeter wave sensor used to optimize performance of a beamforming microphone array

1. A method for operating a beamforming microphone array for use in a predetermined area comprising:

receiving acoustic audio signals at each of a plurality of microphones, converting the same to an electrical mic audio signal, and outputting each of the plurality of electrical mic audio signals;

generating a user location data signal by a wave sensor system, and outputting the user location data signal, wherein the user location data signal includes location information of one or more people within the predetermined area;

receiving both the user location data signal and plurality of mic audio signals at an adaptive beamforming device;

adapting one or more beams by the adaptive beamforming device based on the user location data signal and plurality of output electrical mic audio signals wherein each of the one or more beams acquires sound from one or more specific locations in the predetermined area;

performing acoustic echo cancellation on each of the one or more beams output from the adaptive beamforming device, and wherein the predetermined area is a conference room; and

modifying, by the adaptive beamforming circuit, if the user location data signal indicates that there are more people than beams that can be formed, one or more of the fixed beam positions to cover two or more people in the conference room such that each person is covered by at least one fixed beam.

2. The method according to claim 1 , wherein the wave sensor system comprises:

a millimeter (mm) wave transmitter; and

a wave receiver.

3. The method according to claim 1 , wherein the wave sensor system comprises:

an optical transmitter; and

an optical receiver.

4. The method according to claim 1 , further comprising:

generating, by the wave sensor system, a three dimensional image of the predetermined area and output the same as an area image data signal.

5. The method according to claim 4 , further comprising:

receiving, by the adaptive beamforming circuit, the area image data signal and the plurality of mic audio signals; and

performing adaptive beamforming on the plurality of mic audio signals that takes into account the received area image data signal to adapt one or more beams to acquire sound from one or more specific locations in the predetermined area.

6. The method according to claim 5 , further comprising:

modifying, by the adaptive beamforming circuit, the beam audio signals to reduce noise reflected off one or more objects within the predetermined area based on the area image data signal.

7. The method according to claim 4 , wherein the area image data signal comprises:

information as to where motion is occurring within the predetermined area.

8. The method according to claim 7 , wherein

the information contained within the area image data signal that motion is occurring within the predetermined area substantially eliminates objects from the area image data signal that are substantially at rest.

9. The method according to claim 7 , wherein

the information contained within the area image data signal that motion is occurring within the predetermined area does not include objects that move with a substantially constant velocity.

10. The method according to claim 4 , wherein the area image data signal comprises:

distance information between the wave sensor system and objects within the predetermined area.

11. The method according to claim 10 , wherein

the objects comprise one or more of a floor, table, walls, and other furniture.

12. The method according to claim 10 , further comprising:

adapting, by the adaptive beamforming circuit, one or more beams that takes into account the distance information generated by the wave sensor system.

13. The method according to claim 12 , further comprising:

modifying, by the adaptive beamforming circuit, one or more of a beam width, beam reception angle, and range of the beam based on the received distance information generated by the wave sensor system.

14. The method according to claim 4 , further comprising:

receiving, by the adaptive beamforming circuit, the area image data signal, the user location data signal, and the plurality of mic audio signals; and

performing adaptive beamforming on the plurality of mic audio signals that takes into account the information in the area image data signal and the user location data signal, such that

the adaptive beamforming circuit is further adapted to substantially ignore voice signals that originate from outside the areas where the users are located.

15. The method according to claim 4 , further comprising:

receiving, by the adaptive beamforming circuit, the area image data signal, the user location data signal, and the plurality of mic audio signals; and

performing adaptive beamforming on the plurality of mic audio signals that takes into account the information in the area image data signal and the user location data signal, such that

the adaptive beamforming circuit is further adapted to substantially ignore audio signals generated from one or more of a television and stereo.

16. The method according to claim 4 , wherein

the predetermined area is a conference room,

there is at least one table located in the conference room, and further wherein

the area image data signal includes information as to a location of the at least one table in the conference room, and wherein the method further comprises:

adapting, by the adaptive beamforming circuit, one or more fixed beam positions to cover a perimeter of the at least one table in the conference room.

17. The method according to claim 4 , further comprising:

determining a direction of arrival of one or more microphone generated audio signals by an acoustic audio direction of arrival algorithm stored with the adaptive beamforming circuit.

18. The method according to claim 17 , further comprising:

determining the direction of arrival of the one or more microphone generated audio signals, in the adaptive beamforming circuit, using information in the area image data signal received from the wave sensor system.

19. The method according to claim 4 , further comprising:

determining, by the wave sensor system, motion of one or more objects located in the predetermined area.

20. The method according to claim 19 , wherein

the wave sensor system can include the object motion information about the predetermined area in the area image data signal, and wherein the method further comprises:

substantially eliminating, by the adaptive beamforming circuit, fixed objects and objects moving at a substantially constant rate, from the predetermined area in the area image data signal, to determine a number of people located in the predetermined area, and output the same as a room occupancy status.

21. The method according to claim 20 , further comprising:

using the room occupancy status by other interconnected systems to control one or more of lights, temperature, and audio-video equipment in the conference room.

22. The method according to claim 20 , further comprising:

transmitting the room occupancy status to a room monitoring system.

23. The method according to claim 1 , further comprising:

generating, by the adaptive beamforming circuit, one or more beams to acquire sound from one or more specific locations in the predetermined area.

24. The method according to claim 1 , further comprising:

receiving, by a first communication device, a reference signal from a remote source;

forwarding the reference signal to each of the one or more of the acoustic echo cancellation devices; and

deleting, by each of the one or more acoustic echo cancellation devices, the reference signal from a respective one of the microphone audio signals received by the respective acoustic echo cancellation devices.

25. The method according to claim 24 , wherein

the reference signal comprises a far end audio signal.

26. The method according to claim 1 , further comprising:

generating, by the adaptive beamforming circuit, new beams no faster than a first beam formation rate; and

performing, by the acoustic echo cancellation device, echo cancellation no faster than a first echo cancellation rate, and still further wherein

the first echo cancellation rate and the first beam formation rate are substantially equivalent.

27. The method according to claim 1 , wherein

the wave sensor system is adapted to resolve distances within the predetermined area within about 1 mm and within about 1 degree.

28. The method according to claim 1 , wherein

the predetermined area is a conference room, and wherein the method further comprises:

extracting, by the adaptive beamforming circuit, location information for each person in the conference room; and

generating a respective fixed beam position for each person in the conference room.

29. The method according to claim 1 , further comprising:

adjusting, by the adaptive beamforming circuit, a beam width and shape to cover two or more people in the conference room.

30. The method according to claim 1 , wherein the method further comprises:

removing noise, by a plurality of active noise reduction circuits, one for each acoustic echo cancellation device, from an output of its respective acoustic echo cancellation device;

outputting a noise reduced audio signal;

receiving, by an auto-mixer device, the plurality of noise reduced audio signals from the plurality of active noise reductions circuits;

combining the plurality of noise reduced audio signals to output a single near end audio signal; and

receiving, by an Ethernet communication device, a reference signal from a remote source;

outputting the reference signal to one or more speakers in the predetermined area;

forwarding the reference signal to each of the one or more acoustic echo cancellation devices;

deleting, by the acoustic echo cancellation device, the reference signal from a respective one of the microphone audio signals received by the respective acoustic echo cancellation devices; and

extracting, by a power-over-Ethernet device, electrical power over one or more Ethernet communications cables and providing the electrical power to the circuits in the beamforming microphone array.

31. The method according to claim 30 , wherein

the reference signal comprises a far end audio signal.

32. The method according to claim 31 , further comprising:

recognizing, by the wave sensor system, gestures including one or more of hand motion and arm motion.

33. The method according to claim 32 , wherein

the recognized gestures can control one or more functions in the conference room, and wherein

the functions include one or more of lighting levels, audio levels, temperature levels, humidity levels, and positions of shades and/or curtains.

34. The method according to claim 30 , further comprising:

outputting, by one or more of each of light sensors, temperature sensors, and humidity sensors, status outputs from each of the sensors;

receiving the sensors outputs by the Ethernet communications device; and

transmitting, by the Ethernet communications device, the sensor outputs.