Speech characteristic recognition and conversion

1. A method, comprising:

receiving an audio conversion request configured to initiate conversion of one or more sound characteristics of an audio stream from a first state to a second state, the first state comprising spoken words and the second state comprising sung lyrics;

generating, using an audio signature for the second state, an audio conversion model based on an image of a musical score;

converting the audio stream based on the audio conversion model that is generated based on the image or text of the musical score, the converting comprising:

determining that a first segment comprising acoustic syllables of the spoken words in audio stream is of an insufficient duration to determine a pitch value for the segment;

in response to determining that the segment is of the insufficient duration to determine the pitch value for the segment, merging a set of additional segments comprising acoustic syllables of the spoken words with the first segment to form a merged segment that is of a sufficient duration to determine the pitch value of the merged segment;

compiling an audio construct from the converted audio stream and a base audio segment, the compiling comprising aligning a first syllable of the acoustic syllables to a plurality of beats; and

causing presentation of the compiled audio construct at a client device.

2. The method of claim 1 , further comprising:

detecting a set of segments within the audio stream, each segment representing a portion of the audio stream in which voice data is present;

determining an intervening gap between two segments of the set of segments, the intervening gap representing a portion of the audio stream in which voice data is absent;

determining the intervening gap has a duration below a defined threshold;

merging the two segments to generate another merged segment in response to the duration of the intervening gap being below the defined threshold; and

assigning an interpolated pitch value of the merged segment.

3. The method of claim 1 , wherein the musical score comprises sheet music with a progression of notes, further comprising:

determining a set of acoustic syllables within the audio stream, the acoustic syllables being a uniform segment of voice data having one or more homogeneous characteristics.

4. The method of claim 3 , further comprising:

detecting a set of energy peaks within the set of acoustic syllables; and

detecting one or more minimum energy points between two energy peaks of the set of energy peaks, the one or more minimum energy point representing acoustic syllable boundaries.

5. The method of claim 4 , wherein detecting a set of energy peaks further comprises:

discarding one or more energy peaks having a value below a specified energy threshold.

6. The method of claim 4 , further comprising:

tuning one or more acoustic syllable boundaries corresponding to the one or more minimum energy points.

7. The method of claim 6 , further comprising:

for each acoustic syllable, determining one or more frames; and

for each frame, determining a pitch value.

8. The method of claim 7 , wherein determining a pitch value for each frame further comprises:

determining one or more unvoiced frames lacking a pitch value; and

generating an interpolated pitch value for the one or more unvoiced frames.

9. The method of claim 1 , wherein compiling the audio construct further comprises:

detecting a set of beats within a base audio segment; and

aligning at least a portion of the acoustic syllables with the set of beats.

10. A system, comprising:

one or more processors; and

a machine-readable non-transitory processor-readable storage medium storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving an audio conversion request configured to initiate conversion of one or more sound characteristics of an audio stream from a first state to a second state, the first state comprising spoken words and the second state comprising sung lyrics;

generating, using an audio signature for the second state, an audio conversion model based on an image of a musical score;

converting the audio stream based on the audio conversion model that is generated based on the image or text of the musical score, the converting comprising:

determining that a first segment comprising acoustic syllables of the spoken words in audio stream is of an insufficient duration to determine a pitch value for the segment;

in response to determining that the segment is of the insufficient duration to determine the pitch value for the segment, merging a set of additional segments comprising acoustic syllables of the spoken words with the first segment to form a merged segment that is of a sufficient duration to determine the pitch value of the merged segment;

compiling an audio construct from the converted audio stream and a base audio segment, the compiling comprising aligning a first syllable of the acoustic syllables to a plurality of beats; and

causing presentation of the compiled audio construct at a client device.

11. The system of claim 10 , wherein the operations further comprise:

detecting a set of segments within the audio stream, each segment representing a portion of the audio stream in which voice data is present;

determining an intervening gap between two segments of the set of segments, the intervening gap representing a portion of the audio stream in which voice data is absent;

determining the intervening gap has a duration below a defined threshold;

merging the two segments to generate another merged segment in response to the duration of the intervening gap being below the defined threshold; and

assigning an interpolated pitch value of the merged segment.

12. The system of claim 10 , wherein the musical score comprises sheet music with a progression of notes, and wherein the operations further comprise:

determining a set of acoustic syllables within the audio stream, the acoustic syllables being a uniform segment of voice data having one or more homogeneous characteristics.

13. The system of claim 12 , wherein the operations further comprise:

detecting a set of energy peaks within the set of acoustic syllables; and

detecting one or more minimum energy points between two energy peaks of the set of energy peaks, the one or more minimum energy point representing acoustic syllable boundaries.

14. The system of claim 13 , wherein the operations further comprise:

tuning one or more acoustic syllable boundaries corresponding to the one or more minimum energy points.

15. The system of claim 14 , wherein the operations further comprise:

for each acoustic syllable, determining one or more frames; and

for each frame, determining a pitch value.

16. The system of claim 15 , wherein determining a pitch value for each frame further comprises:

determining one or more unvoiced frames lacking a pitch value; and

generating an interpolated pitch value for the one or more unvoiced frames.

17. The system of claim 10 , wherein compiling the audio construct further comprises:

detecting a set of beats within a base audio segment; and

aligning at least a portion of the acoustic syllables with the set of beats, wherein a first syllable of the acoustic syllables is aligned to a plurality of beats of the set of beats.

18. A non-transitory processor-readable storage medium storing processor executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:

receiving an audio conversion request configured to initiate conversion of one or more sound characteristics of an audio stream from a first state to a second state, the first state comprising spoken words and the second state comprising sung lyrics;

generating, using an audio signature for the second state, an audio conversion model based on an image of a musical score;

converting the audio stream based on the audio conversion model that is generated based on the image or text of the musical score, the converting comprising:

determining that a first segment comprising acoustic syllables of the spoken words in audio stream is of an insufficient duration to determine a pitch value for the segment;

in response to determining that the segment is of the insufficient duration to determine the pitch value for the segment, merging a set of additional segments comprising acoustic syllables of the spoken words with the first segment to form a merged segment that is of a sufficient duration to determine the pitch value of the merged segment;

compiling an audio construct from the converted audio stream and a base audio segment, the compiling comprising aligning a first syllable of the acoustic syllables to a plurality of beats; and

causing presentation of the compiled audio construct at a client device.

19. The non-transitory processor-readable storage medium of claim 18 , wherein the operations further comprise:

detecting a set of segments within the audio stream, each segment representing a portion of the audio stream in which voice data is present;

determining an intervening gap between two segments of the set of segments, the intervening gap representing a portion of the audio stream in which voice data is absent;

determining the intervening gap has a duration below a defined threshold;

merging the two segments to generate another merged segment in response to the duration of the intervening gap being below the defined threshold; and

assigning an interpolated pitch value of the merged segment.

20. The non-transitory processor-readable storage medium of claim 18 , wherein the operations further comprise:

detecting a set of energy peaks within a set of acoustic syllables;

detecting one or more minimum energy points between two energy peaks of the set of energy peaks, the one or more minimum energy point representing acoustic syllable boundaries;

tuning one or more acoustic syllable boundaries corresponding to the one or more minimum energy points;

for each acoustic syllable, determining one or more frames; and

for each frame, determining a pitch value.