IP Library › Granted Patent US 11,752,288
Granted Patent B2
US 11,752,288 · App. 17/071,312 · Granted Sep 12, 2023

Speech-based breathing prediction

Inventors: Aki Sakari Härmä (Eindhoven, NL); Francesco Vicario (Boston, MA); Venkata Srikanth Nallanthighal (Eindhoven, NL)
Assignee: Koninklijke Philips N.V.
A61M16/026A61M16/0003G10L15/063G10L15/16A61M2016/0021A61M2205/50A61M2230/42
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,752,288
App. No.
17/071,312
Granted
Sep 12, 2023
Kind
B2
Abstract

In an embodiment, a method is described. The method comprises obtaining an indication of a speech pattern of a subject and using the indication to determine a predicted time of inspiration by the subject. A machine learning model is used for predicting the relationship between the speech pattern and a breathing pattern of the subject. The machine learning model can then be used to determine the predicted time of inspiration by the subject. The method further comprises controlling delivery of gas to the subject based on the predicted time of inspiration by the subject.

Claims (42)

1. A method comprising:

obtaining an indication of a speech pattern of a subject;

using the indication to determine, using processing circuitry, a predicted time of inspiration by the subject based on a machine learning model for predicting a relationship between the speech pattern and a breathing pattern of the subject; and

controlling delivery of gas to the subject based on the predicted time of inspiration by the subject.

2. The method of claim 1 , comprising deriving a respiration signal from the indication and using the respiration signal as an input to the machine learning model to predict, using the processing circuitry, the time of inspiration by the subject.

3. The method of claim 1 , wherein the machine learning model is built using a neural network configured to identify any correlations between a speech signal and a corresponding respiration signal acquired from a plurality of trainers.

4. The method of claim 3 , wherein the neural network is configured to identify at least one of:

linguistic content and prosodic features of speech signals acquired from the trainers to facilitate identification of the correlation.

5. The method of claim 1 , comprising causing a ventilation system to deliver the gas to the subject during the predicted time of inspiration for a specified period of time, wherein the specified period of time is one of:

a pre-determined period of time; or

adapted according to the subject's individual needs.

6. The method of claim 5 , wherein the subject's individual needs are determined based on at least one of:

linguistic context of the subject's speech;

previous inspiration durations by the subject; and

a medical need of the subject.

7. The method of claim 1 , comprising using change-point detection to predict the time of inspiration of the subject based on the subject's respiration signal as predicted by the machine learning model based on the speech pattern of the subject.

8. Apparatus comprising processing circuitry, the processing circuitry comprising:

a prediction module configured to use an indication of a speech pattern of a monitored subject to determine a predicted time of inspiration by the subject based on a machine learning model for predicting a relationship between the speech pattern and a breathing pattern of the subject; and

a control module configured to control delivery of gas to the subject based on the predicted time of inspiration by the subject.

9. The apparatus of claim 8 , comprising a sound transducer configured to obtain a speech signal corresponding to the speech pattern of the subject.

10. A tangible machine-readable medium storing instructions which, when executed by at least one processor, cause the at least one processor to:

determine, from an indication of a speech pattern of a subject, a predicted time of inspiration by the subject based on a machine learning model for predicting a relationship between the speech pattern and a breathing pattern of the subject; and

control delivery of gas to the subject based on the predicted time of inspiration by the subject.

11. The tangible machine-readable medium of claim 10 , wherein the machine learning model is trained with a plurality of speech signals and corresponding respiration signals acquired from a plurality of trainers.

12. The tangible machine-readable medium of claim 11 , wherein an input to the machine learning model comprises:

a spectral representation of the plurality of speech signals; and

an indication of the corresponding respiration signals at specified time intervals, and

wherein the input is fed into a neural network comprising a plurality of memory layers such that, when the neural network is optimized to update network weighting based on the input, the machine learning model is updated accordingly.

13. The tangible machine-readable medium of claim 12 , wherein:

the spectral representation of each of the plurality of speech signals is obtained by:

filtering each speech signal to spectrally flatten the speech signal and boost higher frequencies compared with lower frequencies of the speech signal;

applying a Fourier transform to obtain a power spectrum corresponding to the speech signal;

applying a Mel-frequency scaling to the power spectrum to obtain a Mel spectrogram; and

selecting a plurality of time windows from the Mel spectrogram, wherein each time window is separated by a specified stride interval, and wherein:

the indication of the corresponding respiration signals at specified time intervals is obtained by:

obtaining a respiratory inductive plethysmography, RIP, signal from the training subject; and

determining an RIP signal value at the end of each time window within the specified stride interval.

14. The tangible machine-readable medium of claim 12 , wherein the neural network comprises at least one of:

a recurrent neural network, RNN;

an RNN-long short term memory, RNN-LSTM, network; and

a convolutional neural network, CNN.

15. The tangible machine-readable medium of claim 12 , wherein an attention mechanism with breathing rate as an auxiliary training parameter is used to optimize the neural network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2020
From: HARMA, AKI SAKARI; VICARIO, FRANCESCO; NALLANTHIGHAL, VENKATA SRIKANTH
To: KONINKLIJKE PHILIPS N.V.
Reel/Frame 054065/0766 →
Continuity (2)
Provisional Application 62936811 · Nov 18, 2019
Related Publication 20210146082A1 · May 20, 2021