IP Library Granted Patent US 12,142,283
Granted Patent B2
US 12,142,283 · App. 17/519,831 · Granted Nov 12, 2024

Audio processing

Inventors: Fabio Cappello (London, GB); Oliver Hume (London, GB); Marina Villanueva Barreiro (Acoruña, ES)
Assignee: Sony Interactive Entertainment Inc.
G10L19/008G10L25/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,142,283
App. No.
17/519,831
Granted
Nov 12, 2024
Kind
B2
Abstract

Audio communication apparatus comprises a set of two or more audio communication nodes; each audio communication node comprising: an audio encoder controlled by encoding parameters to generate encoded audio data to represent a vocal input generated by a user of that audio communication node, the encoded data being agnostic to which user who generated the vocal input; and an audio decoder controlled by decoding parameters to generate a decoded audio signal as a reproduction of a vocal signal generated by a user of another of the audio communication nodes, the decoding parameters being specific to the user of that other of the audio communication nodes.

Claims (36)

1. An audio communication apparatus comprising

a set of two or more audio communication nodes;

each audio communication node comprising:

an audio encoder controlled by encoding parameters to generate encoded audio data to represent a vocal input generated by a user of that audio communication node, the encoded audio data being agnostic to the user that generated the vocal input; and

an audio decoder controlled by decoding parameters to generate a decoded audio signal as a reproduction of a vocal signal generated by a user of another audio communication node of the set of two or more audio communication nodes, the decoding parameters being specific to the user of the another audio communication node of the set of two or more audio communication nodes, wherein:

the audio encoder of each audio communication node of the set of two or more audio communication nodes is configured to include both a user identifier for indicating a user of that audio communication node and encoded audio data generated by that audio encoder in a given audio packet to be transmitted to another audio communication node of the set of two or more audio communication nodes,

the audio decoder of each audio communication node of the set of two or more audio communication nodes is configured to detect a user identifier in an audio packet including encoded audio data received from another audio communication node of the set of two or more audio communication nodes, and to select decoding parameters for decoding that encoded audio data included in the audio packet from two or more candidate decoding parameters associated with respective different users among a plurality of different users in dependence upon the detected user identifier, wherein the selected decoding parameters are stored in a parameter storage unit, which stores decoding parameters associated with the plurality of different users currently capable of sending audio communications to the audio communication apparatus.

2. The audio communication apparatus of claim 1 , comprising a data connection to connect the set of two or more audio communication nodes for the transmission of encoded audio data between audio communication nodes of the set.

3. The audio communication apparatus of claim 1 , in which the audio encoders of the set of two or more audio communication nodes are identical and use the same encoding parameters.

4. The audio communication apparatus of claim 1 , in which each audio communication node is configured to provide decoding parameters associated with the user of that audio communication device to another audio communication node configured to receive encoded audio data from that audio communication node.

5. The audio communication apparatus of claim 1 , in which the audio encoder and the audio decoder comprise processor-implemented artificial neural networks;

the encoding parameters comprise a first set of learned parameters; and the selected decoding parameters comprise a second set of learned parameters.

6. The audio communication apparatus of claim 1 , in which each audio communication node comprises an entertainment device configured to execute a computer game.

7. A machine-implemented method of audio communication between a set of two or more audio communication nodes, the method comprising:

at each audio communication node, generating, in dependence upon encoding parameters, encoded audio data to represent a vocal input generated by a user of that audio communication node, the encoded audio data being agnostic to the user that generated the vocal input; and

at each audio communication node, generating, in response to decoding parameters, a decoded audio signal as a reproduction of a vocal signal generated by a user of another audio communication node of the set of two or more audio communication nodes, the decoding parameters being specific to the user of the another audio communication node of the set of two or more audio communication nodes, wherein:

the generating the encoded audio data including providing, by an audio encoder of each audio communication node of the set of two or more audio communication nodes, both a user identifier for indicating a user of that audio communication node and encoded audio data generated by that audio encoder in a given audio packet to be transmitted to another audio communication node of the set of two or more audio communication nodes, and the generating the decoded audio signal including: (i) detecting, by an audio decoder of each audio communication node of the set of two or more audio communication nodes, a user identifier in an audio packet including encoded audio data received from another audio communication node of the set of two or more audio communication nodes, and (ii) selecting decoding parameters for decoding that encoded audio data included in the audio packet from two or more candidate decoding parameters associated with respective different users among a plurality of different users in dependence upon the detected user identifier, wherein the selected decoding parameters are stored in a parameter storage unit, which stores decoding parameters associated with the plurality of different users currently capable of sending audio communications to a given one of the communication nodes.

8. The machine-implemented method of claim 7 , comprising artificial neural network (ANN) training steps to provide an audio encoding and decoding function at each of the two or more audio communication nodes, the artificial neural network (ANN) training steps comprising:

training an ANN to act as a user-agnostic audio encoder; and

training an ANN to act as a user-specific audio decoder, by carrying out actions comprising:

at each of the audio communication nodes, generating the encoded audio data using the ANN trained to act as the user-agnostic audio encoder; and

at each of the audio communication nodes, generating the decoded audio signal using the ANN trained to act as the user-specific audio decoder.

9. The machine-implemented method of claim 8 , in which the artificial neural network (ANN) training steps comprise generating a set of learned parameters to control operation of at least one of the ANN to act as the user-agnostic audio encoder, and the ANN to act as the user-specific audio decoder.

10. The machine-implemented method of claim 9 , in which the step of training an ANN to act as a user-agnostic audio encoder comprises:

using a user detector to differentiate users from encoded audio data generated by the user-agnostic audio encoder; and

varying the learned parameters for the user-agnostic audio encoder to penalise the differentiation of users from encoded audio data generated by the user-agnostic audio encoder.

11. A non-transitory, machine-readable storage medium which stores computer software which, when executed by a computer, causes the computer to perform a machine-implemented method of audio communication between a set of two or more audio communication nodes, the method comprising:

at each audio communication node, generating, in dependence upon encoding parameters, encoded audio data to represent a vocal input generated by a user of that audio communication node, the encoded audio data being agnostic to the user that generated the vocal input; and

at each audio communication node, generating, in response to decoding parameters, a decoded audio signal as a reproduction of a vocal signal generated by a user of another audio communication node of the set of two or more audio communication nodes, the decoding parameters being specific to the user of the another audio communication node of the set of two or more audio communication nodes, wherein:

the generating the encoded audio data including providing, by an audio encoder of each audio communication node of the set of two or more audio communication nodes, both a user identifier for indicating a user of that audio communication node and encoded audio data generated by that audio encoder in a given audio packet to be transmitted to another audio communication node of the set of two or more audio communication nodes, and

the generating the decoded audio signal including: (i) detecting, by an audio decoder of each audio communication node of the set of two or more audio communication nodes, a user identifier in an audio packet including encoded audio data received from another audio communication node of the set of two or more audio communication nodes, and (ii) selecting decoding parameters for decoding that encoded audio data included in the audio packet from two or more candidate decoding parameters associated with respective different users among a plurality of different users in dependence upon the detected user identifier, wherein the selected decoding parameters are stored in a parameter storage unit, which stores decoding parameters associated with the plurality of different users currently capable of sending audio communications to a given one of the communication nodes.

12. The non-transitory, machine-readable storage medium of claim 11 , wherein the method further comprises artificial neural network (ANN) training steps to provide an audio encoding and decoding function at each of the two or more audio communication nodes, the artificial neural network (ANN) training steps comprising:

training an ANN to act as a user-agnostic audio encoder; and

training an ANN to act as a user-specific audio decoder by carrying out actions comprising:

at each of the audio communication nodes, generating the encoded audio data using the ANN trained to act as the user-agnostic audio encoder; and

at each of the audio communication nodes, generating the decoded audio signal using the ANN trained to act as the user-specific audio decoder.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2022
From: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 059761/0698 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2022
From: CAPPELLO, FABIO
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 059820/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2021
From: VILLANUEVA BARREIRO, MARINA
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 058041/0434 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 5, 2021
From: HUME, OLIVER
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 058030/0625 →
Priority Claims (1)
GB 2017689 · Nov 10, 2020 · national
Continuity (1)
Related Publication 20220148604A1 · May 12, 2022