Modifying audio data transmitted to a receiving device to account for acoustic parameters of a user of the receiving device
A communication system provides audio content to one or more client devices capable of playing spatialized audio content. For example, the communication system receives audio content from a client device and transmits the audio content to other client devices to be played for users. The communication system dynamically modifies audio content transmitted to different client devices based on a payload including audio parameters (e.g., local area acoustic properties, an audiogram for a user, a head related transfer function for a user, etc.) received from a client device.
1. A method comprising:
obtaining, at a communication system, audio content captured by a sending client device;
receiving, at the communication system, a payload from a receiving client device, the payload including one or more acoustic parameters of the receiving client device;
generating, by the communication system, modified audio content by modifying the audio content based on the one or more acoustic parameters of the receiving client device,
wherein the generating of the modified audio content includes transforming the audio content based on acoustic properties, of a local area that includes the receiving client device, included in the payload and wherein the transforming includes:
generating a predicted position, of the receiving client device in the local area, from information included in the payload;
determining a degree of arrival of portions of audio data to the predicted position of the receiving client device; and
applying one or more transfer functions, to the audio content, based on the predicted position and the acoustic properties of the local area; and
transmitting the modified audio content from the communication system to the receiving client device, which is configured to play the modified audio content to receiving user.
2. The method of claim 1 , wherein the one or more transfer functions include a head-related transfer function, for a head of the receiving user.
3. The method of claim 2 , wherein the transforming the audio content based on the head-related transfer function comprises retrieving the head-related transfer function, from a server, based on information included in the payload.
4. The method of claim 1 , wherein the generating, by the communication system, the modified audio content comprises adjusting frequencies of certain portions of the audio content to differ from specific frequencies identified in the payload.
5. The method of claim 1 , wherein the generating, by the communication system, the modified audio content comprises adjusting the audio content to concentrate spatial cues in a portion of the audio content configured to be played for a particular ear of the receiving user.
6. A sending client device comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
capturing audio content at the sending client device;
receiving, at the sending client device, a payload from a receiving client device, the payload including one or more acoustic parameters of the receiving client device;
generating, by the sending client device, modified audio content by modifying the audio content based on the one or more acoustic parameters of the receiving client device,
wherein the generating of the modified audio content includes transforming the audio content based on acoustic properties, of a local area that includes the receiving client device, included in the payload and
wherein the transforming includes:
generating a predicted position, of the receiving client device in the local area, from information included in the payload;
determining a degree of arrival of portions of audio data to the predicted position of the receiving client device; and
applying one or more transfer functions, to the audio content, based on the predicted position and the acoustic properties of the local area; and
transmitting the modified audio content from the communication system to the receiving client device, which is configured to play the modified audio content to receiving user.
7. The sending client device of claim 6 , wherein the one or more transfer functions include a head-related transfer function, for a head of the receiving user.
8. The sending client device of claim 7 , wherein the head-related transfer function is anonymized by removing information identifying the receiving user.
9. The sending client device of claim 7 , wherein the generating, by the sending client device, the modified audio content comprises retrieving the head-related transfer function, from the sending client device, based on information included in the payload, wherein the head related transfer function is anonymized by removing information identifying the receiving user, on the sending client device.
10. A computer program product comprising a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by one or more processors, causes the one or more processors to:
obtain, at a communication system, audio content captured by a sending client device;
receive, at the communication system, a payload from a receiving client device, the payload including one or more acoustic parameters of the receiving client device;
generate, by the communication system, modified audio content by modifying the audio content based on the one or more acoustic parameters of the receiving client device,
wherein the generating of the modified audio content includes transforming the audio content based on acoustic properties, of a local area that includes the receiving client device, included in the payload and wherein the transforming includes:
generating a predicted position, of the receiving client device in the local area, from information included in the payload;
determining a degree of arrival of portions of audio data to the predicted position of the receiving client device; and
applying one or more transfer functions, to the audio content, based on the predicted position and the acoustic properties of the local area; and
transmitting the modified audio content from the communication system to the receiving client device, which is configured to play the modified audio content to a receiving user.
11. The computer program product of claim 10 , wherein the one or more transfer functions include a head-related transfer function, for a head of the receiving user.
12. The computer program product of claim 11 , wherein the transforming the audio content comprises retrieving the head-related transfer function, from a server, based on information included in the payload.
13. The computer program product of claim 10 , wherein the generating the modified audio content comprises adjust frequencies of certain portions of the audio content to differ from specific frequencies identified in the payload.
14. The computer program product of claim 10 , wherein the generating the modified audio content comprises adjusting the audio content to concentrate spatial cues in a portion of the audio content configured to be played for a particular ear, of the receiving user, identified from the payload.