Granted Patent B2

US 12,347,421 · App. 18/093,315 · Granted Jul 1, 2025

Inventors: Aleksandrs Timofejevs (Riga, LV); Boris Maslov (Newport Beach, CA)

Assignee: PolyN Technology Limited

G10L15/16G10L25/51G10L25/78G10L2015/088

Abstract

Claims (74)

1. A hardware apparatus comprising:

a digital switch coupled to a plurality of analog neuromorphic cores, the digital switch configured to:

obtain one or more sound streams from one or more sound sources;

transmit data based on the one or more sound streams to the plurality of analog neuromorphic cores;

receive output from the plurality of analog neuromorphic cores; and

output one or more modified sound streams based on the output received from the plurality of analog neuromorphic cores,

wherein each analog neuromorphic core comprises a respective analog network of analog components and is configured to (i) receive a respective sound stream from the digital switch, (ii) perform a respective voice-related function on the respective sound stream, and (iii) transmit a respective output of the respective voice-related function to the digital switch.

2. The hardware apparatus of claim 1 , wherein the digital switch is further configured to switch on or off at least one of the plurality of analog neuromorphic cores.

3. The hardware apparatus of claim 1 , wherein the analog components for each analog network include a plurality of operational amplifiers and a plurality of resistors, each operational amplifier representing an analog neuron and each resistor representing a connection between two analog neurons.

4. The hardware apparatus of claim 1 , wherein the plurality of analog neuromorphic cores includes (i) a first core configured to detect music, voice, and acoustic events, in the one or more sound streams and (ii) a second core configured to extract and/or enhance voice in the one or more sound streams.

5. The hardware apparatus of claim 4 , wherein the digital switch is further configured to:

initially transmit data based on the one or more sound streams to the first core; and

in response to receiving, from the first core, a signal detecting a voice, subsequently transmit data based on the signal to the second core, and, in response, receive an enhanced voice signal from the second core.

6. The hardware apparatus of claim 4 , wherein the plurality of analog neuromorphic cores further includes (i) a third core configured to spot keywords in the one or more sound streams and (ii) a fourth core configured to detect wake words in the one or more sound streams.

7. The hardware apparatus of claim 1 , wherein the digital switch is further configured to:

transform the one or more sound streams to normalize volume, to obtain a sound stream suitable for processing in a first analog neuromorphic core; and

transmit the sound stream of data to the first analog neuromorphic core.

8. The hardware apparatus of claim 1 , wherein the digital switch is further configured to:

log information related to audio parameters, configuration, and one or more states of activated neural cores amongst the plurality of analog neuromorphic cores.

9. The hardware apparatus of claim 1 , wherein the digital switch is further configured to:

down-sample the one or more sound streams to 16 KHz, to obtain down-sampled data;

transmit the down-sampled data to the plurality of analog neuromorphic cores; and

up-sample audio stream output from the plurality of analog neuromorphic cores, for output.

10. The hardware apparatus of claim 1 , wherein:

the plurality of analog neuromorphic cores includes (i) a first core configured to detect music, voice, and acoustic events, in the one or more sound streams, and (ii) a second core configured to extract and/or enhance voice in the one or more sound streams; and

the digital switch is further configured to:

initially transmit data based on the one or more sound streams to the first core;

in response to receiving, from the first core, a signal detecting a voice, subsequently transmit data based on the signal to the second core and, in response, receive an enhanced voice signal from the second core with noises suppressed; and

in response to receiving, from the first core, a signal detecting no voice, subsequently output ambient noise in the one or more sound streams.

11. The hardware apparatus of claim 1 , wherein the plurality of analog neuromorphic cores includes a first core configured to detect a voice, music, or no voice, wherein the digital switch is further configured to:

initially transmit data based on the one or more sound streams to the first core; and

in response to receiving, from the first core, a signal detecting a voice, music, or no voice, select a different operating mode based on the signal.

12. The hardware apparatus of claim 1 , wherein the plurality of analog neuromorphic cores includes a first core configured to detect a voice of a user, wherein the digital switch is further configured to:

initially transmit data based on the one or more sound streams to the first core; and

in response to receiving, from the first core, a signal detecting the voice of the user, only then activate another core of the plurality of analog neuromorphic cores.

13. The hardware apparatus of claim 1 , wherein the plurality of analog neuromorphic cores includes (i) a first core configured to detect a voice, (ii) a second core configured to enhance voice signals, and (iii) a third core configured to either spot keywords or detect wake words, and the digital switch is further configured to:

initially transmit data based on the one or more sound streams to the first core;

in response to receiving, from the first core, a signal detecting a voice, subsequently transmit data based on the signal to a second core and, in response, receive an enhanced voice signal from the second core with music suppressed and voice amplified; and

in response to receiving, from the second core, the enhanced voice signal, transmit the enhanced voice signal to the third core for either spotting keywords or detecting wake words.

14. The hardware apparatus of claim 1 , wherein the plurality of analog neuromorphic cores includes one or more cores selected from the group consisting of: (i) a first core that implements a trained neural network trained to spot keywords, (ii) a second core that implements a trained neural network trained to detect wake words, (iii) a third core that implements a trained neural network trained for voice activity detection, and (iv) a fourth core that implements a trained neural network to extract voice from noisy sound streams.

15. The hardware apparatus of claim 14 , wherein (i) the first core implements a trained depth wise separable convolutional neural network (DS-CNN) trained to spot keywords, (ii) the second core implements a trained recurrent neural network (RNN) trained to detect wake words, (iii) the third core implements a trained recurrent neural network (RNN) trained for voice activity detection, and (iv) the fourth core implements a trained recurrent neural network trained to extract voice from noisy sound streams.

16. A method comprising:

at a digital switch coupled to a plurality of analog neuromorphic cores:

obtaining one or more sound streams from one or more sound sources;

transmitting data based on the one or more sound streams to the plurality of analog neuromorphic cores, wherein each analog neuromorphic core comprises a respective analog network of analog components;

receiving output from the plurality of analog neuromorphic cores; and

outputting one or more modified sound streams based on the output received from the plurality of analog neuromorphic cores; and

at each of the plurality of analog neuromorphic cores:

receiving respective input data from the digital switch;

performing a respective voice-related function on the respective input data; and

transmitting a respective output to the digital switch, for the one or more sound streams.

17. The method of claim 16 , further comprising, at the digital switch:

switching on or off at least one of the plurality of analog neuromorphic cores.

18. The method of claim 16 , wherein the analog components include a plurality of operational amplifiers and a plurality of resistors, each operational amplifier representing an analog neuron and each resistor representing a connection between two analog neurons.

19. The method of claim 16 , further comprising:

at a core of the plurality of analog neuromorphic cores:

analyzing the one or more sound streams to classify the one or more sound streams as music, voice, or acoustic events.

20. The method of claim 16 , further comprising:

at a core of the plurality of analog neuromorphic cores:

extracting and/or enhancing voice in the one or more sound streams.

21. The method of claim 16 , further comprising:

at the digital switch:

initially transmitting the data based on the one or more sound streams to a first core to classify the one or more sound streams as music, voice, or acoustic events; and

in response to receiving, from the first core, a signal detecting a voice, subsequently transmitting data based on the signal to a second core and, in response, receive an enhanced voice signal from the second core.

22. The method of claim 16 , further comprising:

at the digital switch:

initially transmitting data based on the one or more sound streams to a first core for enhancing voice signals, in the one or more sound streams; and

in response to receiving, from the first core, an enhanced voice signal, subsequently transmitting the enhanced voice signal to a second core and, in response, receiving a signal from the second core that classifies the enhanced voice signal as music, voice, or acoustic events.

23. The method of claim 16 , further comprising:

at a core of the plurality of analog neuromorphic cores:

detecting wake words in the one or more sound streams.

24. The method of claim 16 , further comprising:

at a core of the plurality of analog neuromorphic cores:

spotting keywords in the one or more sound streams.

Assignments (1)

Continuity (5)

References Cited (124)