IP Library Granted Patent US 10,431,242
Granted Patent B1
US 10,431,242 · App. 15/802,145 · Granted Oct 1, 2019

Systems and methods for identifying speech based on spectral features

Inventor: Tom Médioni (Paris, FR)
Assignee: GoPro, Inc.
G10L25/78G10L15/04G10L15/22G10L25/18G10L25/21
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,431,242
App. No.
15/802,145
Granted
Oct 1, 2019
Kind
B1
Abstract

Audio information defining audio content may be accessed. The audio content may have a duration. The audio content may be segmented into audio segments. Individual audio segments may correspond to a portion of the duration. The audio segments may include a first audio segment corresponding to a first portion of the duration. Energy features, entropy features, frequency features, and/or other features of the audio segments may be determined. Energy features may characterize energy of the audio segments. Entropy features may characterize spectral flatness of the audio segments. Frequency features may characterize highest frequencies of the audio segments. One or more of the audio segments may be identified as containing speech based on the energy features, the entropy features, the frequency features, and/or other information. Storage of the identification of the one or more of the audio segments as containing speech in one or more storage media may be effectuated.

Claims (56)

1. A system that identifies speech, the system comprising:

one or more physical processors configured by machine-readable instructions to:

access audio information defining audio content, the audio content having a duration;

segment the audio content into audio segments, individual audio segments corresponding to a portion of the duration, the audio segments including a first audio segment corresponding to a first portion of the duration;

determine energy features of the audio segments, the energy features characterizing energy of the audio segments, the energy features including a first energy feature of the first audio segment;

determine entropy features of the audio segments, the entropy features characterizing spectral flatness of the audio segments, the entropy features including a first entropy feature of the first audio segment;

determine frequency features of the audio segments, the frequency features characterizing highest frequencies of the audio segments, the frequency features including a first frequency feature of the first audio segment;

identify one or more of the audio segments as containing speech based on the energy features, the entropy features, and the frequency features by generating a binary curve indicating the one or more of the audio segments as containing speech, wherein the first audio segment is identified as containing speech based on the first energy feature, the first entropy feature, and the first frequency feature, wherein the binary curve is smoothed based on a number of silent audio segments; and

effectuate storage of the identification of the one or more of the audio segments as containing speech in a storage medium.

2. The system of claim 1 , wherein the audio content is segmented into the audio segments of a same segment duration.

3. The system of claim 2 , wherein the same segment duration is twenty-five milliseconds, fifty milliseconds, two-hundred milliseconds, or one second.

4. The system of claim 1 , wherein the one or more physical processors are, to identify the one or more of the audio segments as containing speech, further configured by the machine-readable instructions to:

compare the first energy feature to an energy threshold; and

change a speech counter corresponding to the first audio segment based on the first energy feature meeting the energy threshold.

5. The system of claim 4 , wherein the one or more physical processors are, to identify the one or more of the audio segments as containing speech, further configured by the machine-readable instructions to:

compare the first entropy feature to an entropy threshold; and

change the speech counter corresponding to the first audio segment based on the first entropy feature meeting the entropy threshold.

6. The system of claim 5 , wherein the one or more physical processors are, to identify the one or more of the audio segments as containing speech, further configured by the machine-readable instructions to:

compare the first frequency feature to a frequency threshold; and

change the speech counter corresponding to the first audio segment based on the first frequency feature meeting the frequency threshold.

7. The system of claim 6 , wherein:

the speech counter corresponding to the first audio segment is initially set to zero;

the speech counter corresponding to the first audio segment is incremented by one for an individual feature of the first audio segment meeting an individual threshold; and

the first audio segment is identified as containing speech based on the speech counter corresponding to the first audio segment being more than one.

8. A method for identifying speech, the method performed by a computing system including one or more physical processors, the method comprising:

accessing, by the computing system, audio information defining audio content, the audio content having a duration;

segmenting, by the computing system, the audio content into audio segments, individual audio segments corresponding to a portion of the duration, the audio segments including a first audio segment corresponding to a first portion of the duration;

determining, by the computing system, energy features of the audio segments, the energy features characterizing energy of the audio segments, the energy features including a first energy feature of the first audio segment;

determining, by the computing system, entropy features of the audio segments, the entropy features characterizing spectral flatness of the audio segments, the entropy features including a first entropy feature of the first audio segment;

determining, by the computing system, frequency features of the audio segments, the frequency features characterizing highest frequencies of the audio segments, the frequency features including a first frequency feature of the first audio segment;

identifying, by the computing system, one or more of the audio segments as containing speech based on the energy features, the entropy features, and the frequency features by generating a binary curve indicating the one or more of the audio segments as containing speech, wherein the first audio segment is identified as containing speech based on the first energy feature, the first entropy feature, and the first frequency feature, wherein the binary curve is smoothed based on a number of silent audio segments; and

effectuating storage, by the computing system, of the identification of the one or more of the audio segments as containing speech in a storage medium.

9. The method of claim 8 , wherein the audio content is segmented into the audio segments of a same segment duration.

10. The method of claim 9 , wherein the same segment duration is twenty-five milliseconds, fifty milliseconds, two-hundred milliseconds, or one second.

11. The method of claim 8 , wherein identifying the one or more of the audio segments as containing speech includes:

comparing the first energy feature to an energy threshold; and

changing a speech counter corresponding to the first audio segment based on the first energy feature meeting the energy threshold.

12. The method of claim 11 , wherein identifying the one or more of the audio segments as containing speech includes:

comparing the first entropy feature to an entropy threshold; and

changing the speech counter corresponding to the first audio segment based on the first entropy feature meeting the entropy threshold.

13. The method of claim 12 , wherein identifying the one or more of the audio segments as containing speech includes:

comparing the first frequency feature to a frequency threshold; and

changing the speech counter corresponding to the first audio segment based on the first frequency feature meeting the frequency threshold.

14. The method of claim 13 , wherein:

the speech counter corresponding to the first audio segment is initially set to zero;

the speech counter corresponding to the first audio segment is incremented by one for an individual feature of the first audio segment meeting an individual threshold; and

the first audio segment is identified as containing speech based on the speech counter corresponding to the first audio segment being more than one.

15. A system that identifies speech, the system comprising:

one or more physical processors configured by machine-readable instructions to:

access audio information defining audio content, the audio content having a duration;

segment the audio content into audio segments of a same segment duration, individual audio segments corresponding to a portion of the duration, the audio segments including a first audio segment corresponding to a first portion of the duration;

determine energy features of the audio segments, the energy features characterizing energy of the audio segments, the energy features including a first energy feature of the first audio segment;

determine entropy features of the audio segments, the entropy features characterizing spectral flatness of the audio segments, the entropy features including a first entropy feature of the first audio segment;

determine frequency features of the audio segments, the frequency features characterizing highest frequencies of the audio segments, the frequency features including a first frequency feature of the first audio segment;

identify one or more of the audio segments as containing speech based on the energy features, the entropy features, and the frequency features by generating a binary curve indicating the one or more of the audio segments as containing speech, wherein the first audio segment is identified as containing speech based on the first energy feature, the first entropy feature, and the first frequency feature, wherein the binary curve is smoothed based on a number of silent audio segments; and

effectuate storage of the identification of the one or more of the audio segments as containing speech in a storage medium.

Assignments (6)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
SECURITY INTEREST Recorded Feb 9, 2018
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 044983/0718 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2017
From: MÉDIONI, TOM
To: GOPRO, INC.
Reel/Frame 044058/0101 →
Cited By (1)
US 12,437,774