IP Library Granted Patent US 10,459,686
Granted Patent B1
US 10,459,686 · App. 16/058,200 · Granted Oct 29, 2019

Audio stream mixing system and method

Inventors: Nils Lenke (Rheinbach, DE); Christophe Couvreur (Sint-Pieters-Kapelle, BE)
Assignee: Nuance Communications, Inc.
G06F3/165G10L15/04G10L15/22G10L25/78
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,459,686
App. No.
16/058,200
Granted
Oct 29, 2019
Kind
B1
Abstract

Provided are a system and method of mixing a second audio stream with a first audio stream in an audio output device. The system is configured to execute the method, comprising buffering and outputting the first audio stream via the audio output device as unmodified output, determining at least one insertion spot within the first audio stream, modifying the first audio stream at an insertion spot to avoid content loss, outputting the second audio stream at the insertion spot, and resuming unmodified output of the first audio stream at or near a completion of the second audio stream. Modifying the first audio stream can include pausing and/or warping the first audio stream at the insertion spot. The audio output device can be a vehicle head unit or a wireless device, such as a mobile phone.

Claims (33)

1. A method of mixing a second audio stream with a first audio stream in an audio output device, the method comprising:

buffering the first audio stream;

determining an insertion spot within the first audio stream;

outputting the buffered first audio stream, including modifying output of at least a portion of the first audio stream to accommodate output of the second audio stream at the insertion spot to minimize or avoid content degradation or loss of the first audio stream, wherein modifying the first audio stream at the insertion spot includes one of (i) modifying a time-frequency structure of the first audio stream, or (ii) warping the first audio stream at or near the insertion spot;

outputting the second audio stream at the insertion spot; and

continuing output of the buffered first audio stream after completion of the second audio stream.

2. The method of claim 1 , wherein the second audio stream is received during output of the first audio stream.

3. The method of claim 1 , wherein the buffering of the first audio stream occurs in response to receipt of the second audio stream.

4. The method of claim 1 , wherein the first audio stream is a radio stream.

5. The method of claim 1 , wherein the first audio stream is playback of content from a tangible storage medium local to the audio output device.

6. The method of claim 5 , wherein the tangible storage medium is a compact disc, unified serial bus medium, hard drive, or a computer memory.

7. The method of claim 1 , wherein the second audio stream is received by the audio output device with an urgency or maximum delay indicator.

8. The method of claim 7 , wherein the method includes:

identifying a maximum delay for output of the second audio stream based on the urgency or maximum delay indicator.

9. The method of claim 7 , further comprising:

determining the insertion spot based, at least in part, on the urgency or maximum delay indicator.

10. The method of claim 1 , wherein determining the insertion spot includes finding a gap or pause within the first audio stream.

11. The method of claim 1 , wherein determining the insertion spot includes analyzing the first audio stream using one or more speech analysis techniques to find ends of sentences, phrases, words, or other natural points of interruption.

12. The method of claim 11 , wherein the one or more speech analysis techniques includes at least one of Voice Activity Detection (VAD), Automatic Speech Recognition (ASR), and Natural Language Understanding (NLU).

13. The method of claim 1 , wherein outputting the second audio stream at the insertion spot includes modifying the second audio stream.

14. The method of claim 13 , wherein modifying the second audio stream includes modifying a time-frequency structure of the second audio stream.

15. The method of claim 13 , wherein modifying the second audio stream includes modifying a voice style of the second audio stream to improve intelligibility relative to the first audio stream.

16. The method of claim 1 , wherein the audio output device is or forms part of a vehicle head unit.

17. The method of claim 16 , wherein the second audio stream is an announcement or an alert from a vehicle navigation system, vehicle monitoring system, or a text to speech system.

18. The method of claim 1 , wherein the second audio stream is an announcement or an alert from an advertising system.

19. The method of claim 1 , wherein the audio output device is a wireless portable device comprising a mobile phone, tablet, or phablet.

20. A method of mixing a second audio stream with a first audio stream in an audio output device, the method comprising:

buffering the first audio stream;

determining an insertion spot within the first audio stream, wherein determining the insertion spot includes finding a low volume level within the first audio stream;

outputting the buffered first audio stream, including modifying output of at least a portion of the first audio stream to accommodate output of the second audio stream at the insertion spot to minimize or avoid content degradation or loss of the first audio stream;

outputting the second audio stream at the insertion spot; and

continuing output of the buffered first audio stream after completion of the second audio stream.

21. The method of claim 20 , wherein modifying the first audio stream at the insertion spot includes pausing the first audio stream at the insertion spot.

Assignments (6)
RELEASE (REEL 052935 / FRAME 0584) Recorded Jan 2, 2025
From: WELLS FARGO BANK, NATIONAL ASSOCIATION
To: CERENCE OPERATING COMPANY
Reel/Frame 069797/0818 →
SECURITY AGREEMENT Recorded Jun 15, 2020
From: CERENCE OPERATING COMPANY
To: WELLS FARGO BANK, N.A.
Reel/Frame 052935/0584 →
RELEASE OF SECURITY INTEREST Recorded Jun 12, 2020
From: BARCLAYS BANK PLC
To: CERENCE OPERATING COMPANY
Reel/Frame 052927/0335 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2020
From: NUANCE COMMUNICATIONS, INC.
To: CERENCE OPERATING COMPANY
Reel/Frame 052114/0001 →
SECURITY AGREEMENT Recorded Nov 7, 2019
From: CERENCE OPERATING COMPANY
To: BARCLAYS BANK PLC
Reel/Frame 050953/0133 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2018
From: LENKE, NILS; COUVREUR, CHRISTOPHE
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 046726/0612 →
Cited By (2)
US 12,417,762 US 12,710,915