IP Library Granted Patent US 12,260,866
Granted Patent B2
US 12,260,866 · App. 17/898,962 · Granted Mar 25, 2025

System and method for watermarking audio data for automated speech recognition (ASR) systems

Inventors: Patrick Aubrey Naylor (Reading, GB); Dushyant Sharma (Mountain House, CA); William Francis Ganong, III (Brookline, MA); Uwe Helmut Jost (Groton, MA); Ljubomir Milanovic (Vienna, AT)
Assignee: Microsoft Technology Licensing, LLC
G10L19/018G10L15/22G10L25/21G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,866
App. No.
17/898,962
Granted
Mar 25, 2025
Kind
B2
Abstract

A method, computer program product, and computing system for processing audio information associated with a speech processing system and encoding a watermark in a non-disruptive portion of the audio information.

Claims (44)

1. A computer-implemented method, executed on a computing device, comprising:

processing audio information associated with a speech processing system;

encoding a watermark in a non-disruptive portion of the audio information during storage and/or transmission of the audio information including:

determining a pair of time frequency points with time frequency points from a same channel;

generating a watermark database with a plurality of keys mapping a plurality of watermarks to particular pairs of time frequency points for determining or identifying the particular pairs of time frequency points; and

recording a key mapping the watermark in the audio information to a relative phase difference between the pair of time frequency points from the same channel in the watermark database; and

decoding the watermark using the key from the watermark database to identify the pair of time frequency points and by determining the relative phase difference between the identified pair of time frequency points.

2. The computer-implemented method of claim 1 , wherein processing the audio information associated with the speech processing system includes:

processing audio information associated with an automated speech recognition (ASR) system, thus defining ASR audio information.

3. The computer-implemented method of claim 1 , wherein encoding the watermark in the non-disruptive portion of the audio information includes:

encoding the watermark in a non-speech portion of the audio information.

4. The computer-implemented method of claim 1 , wherein encoding the watermark in the non-disruptive portion of the audio information includes:

encoding the watermark in phase information of the audio information.

5. The computer-implemented method of claim 3 , wherein encoding the watermark in the phase information of the audio information includes:

encoding the watermarking in absolute phase information of the audio information.

6. The computer-implemented method of claim 3 , wherein encoding the watermark in the phase information of the audio information includes:

encoding the watermarking in relative phase information of the audio information.

7. The computer-implemented method of claim 5 , further comprising:

determining a pair of time frequency points with time frequency points from separate channels; and

encoding the watermark in the relative phase information between the pair of time frequency points.

8. A computing system comprising:

a memory; and

a processor configured to process automated speech recognition (ASR) audio information for storage and/or transmission, wherein the processor is further configured to encode a watermark in phase information of the ASR audio information during the storage and/or transmission of the ASR audio information including: to determine a pair of time frequency points with time frequency points from a same channel, to generate a watermark database with a plurality of keys mapping a plurality of watermarks to particular pairs of time frequency points for determining or identifying the particular pairs of time frequency points, and to record a key mapping the watermark in the ASR audio information to a relative phase difference between the pair of time frequency points from the same channel in the watermark database, and wherein the processor is further configured to decode the watermark using the key from the watermark database to identify during the storage and/or transmission of the ASR audio information including the pair of time frequency points and by determining the relative phase difference between the identified pair of time frequency points.

9. The computing system of claim 8 , wherein encoding the watermark in the phase information of the ASR audio information includes at least one of:

encoding the watermarking in absolute phase information of a single channel of the ASR audio information; and

encoding the watermarking in relative phase information of a single channel of the ASR audio information.

10. The computing system of claim 9 , wherein the processor is further configured to:

determine a pair of time frequency points with time frequency points from separate channels; and

encode the watermark in the relative phase information between the pair of time frequency points.

11. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

processing automated speech recognition (ASR) audio information for storage and/or transmission;

determining a pair of time frequency points within the ASR audio information with time frequency points from a same channel; and

encoding a watermark in relative phase information of the ASR audio information between the pair of time frequency points during storage and/or transmission of the audio information including:

generating a watermark database with a plurality of keys mapping a plurality of watermarks to particular pairs of time frequency points for determining or identifying the particular pairs of time frequency points; and

recording a key mapping the watermark in the ASR audio information to a relative phase difference between the pair of time frequency points from the same channel in the watermark database; and

decoding the watermark using the key from the watermark database to identify the pair of time frequency points and by determining the relative phase difference between the identified pair of time frequency points.

12. The computer program product of claim 11 , wherein determining the pair of time frequency points within the ASR audio information includes:

determining a pair of time frequency points with time frequency points from the same channel of the ASR audio information.

13. The computer program product of claim 11 , wherein determining the pair of time frequency points within the ASR audio information includes:

determining a pair of time frequency points with time frequency points from separate channels of multichannel ASR audio information.

14. The computer program product of claim 11 , wherein determining the pair of time frequency points within the ASR audio information includes:

determining an energy level associated with the pair of time frequency points within the ASR audio information.

15. The computer program product of claim 14 , wherein determining the pair of time frequency points within the ASR audio information includes:

comparing the energy level associated with the pair of time frequency points to a predefined threshold.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2023
From: NUANCE COMMUNICATIONS, INC.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065578/0676 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2022
From: GANONG, WILLIAM FRANCIS, III; NAYLOR, PATRICK AUBREY; SHARMA, DUSHYANT; JOST, UWE HELMUT; MILANOVIC, LJUBOMIR
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 060948/0155 →