Delta models for providing privatized speech-to-text during virtual meetings

1. A system comprising:

a non-transitory computer-readable medium;

a communications interface; and

a processor communicatively coupled to the non-transitory computer-readable medium and the communications interface, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

join a virtual meeting, wherein each participant in the virtual meeting exchanges one or more audio streams with other participants in the virtual meeting via the virtual meeting;

receive, from a video conference provider, a local model for speech recognition, wherein the local model comprises a copy of a centralized model;

store the copy of the centralized model as a local recognizer; and

perform speech recognition using the local model on the one or more audio streams, wherein performing speech recognition comprises:

identifying, by the local recognizer, audio feature data within the one or more audio streams;

identifying, based on a vocabulary database, user-specific vocabulary within the audio feature data; and

generating, based on the user-specific vocabulary, a private transcription of the one or more audio streams, wherein the private transcription comprises the user-specific vocabulary.

2. The system of claim 1 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to

neutralize the private transcription, wherein neutralizing the private transcription comprises modifying the user-specific vocabulary in the private transcription;

generate a neutralized transcription based on neutralizing the private transcription; and

transmit, to the video conference provider, the neutralized transcription.

3. The system of claim 1 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

generate the vocabulary database based on locally available data, wherein the locally available data comprises one or more of:

a user's calendar;

a user's email account;

a user's contact list;

past private transcriptions of a user's audio data;

documents stored locally; and

information associated with a virtual meeting.

4. The system of claim 1 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

train the local model, wherein training the local model comprises generating one or more associations between the audio feature data and a plurality of terms or phrases in the vocabulary database.

5. The system of claim 1 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

identify one or more irrelevant terms or phrases; and

remove, from the vocabulary database, the one or more irrelevant terms or phrases.

6. The system of claim 5 , wherein the processor-executable instructions for identifying the one or more irrelevant terms or phrases cause the processor to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

identify a time duration since the one or more irrelevant terms or phrases were identified in past spoken word; and

determine the one or more irrelevant terms or phrases to be removable based on the time duration.

7. The system of claim 1 , wherein the instructions for performing speech recognition using the local model during the virtual meeting cause the processor to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive, by the video conference provider, a partial transcription of the one or more audio streams, wherein the partial transcription comprises a transcription of non-user specific content from the one or more audio streams; and

combine the partial transcription of the non-user specific content with a transcription of the user-specific vocabulary to generate the private transcription.

8. A method comprising:

joining, by a first client device, a virtual meeting having a plurality of participants, each participant of the plurality of participants exchanging one or more audio streams via the virtual meeting;

receiving, from a video conference provider, a local model for speech recognition, wherein the local model comprises a copy of a centralized model;

storing, by the first client device, the copy of the centralized model as a local recognizer;

performing, by the first client device, speech recognition using the local model on the one or more audio streams, wherein performing speech recognition comprises:

identifying, by the local recognizer, audio feature data within the one or more audio streams;

identifying, based on a vocabulary database, user-specific vocabulary within the audio feature data; and

generating, based on the user-specific vocabulary, a private transcription of the one or more audio streams, wherein the private transcription comprises the user-specific vocabulary.

9. The method of claim 8 , further comprising:

neutralizing, by the first client device, the private transcription, wherein neutralizing the private transcription comprises modifying the user-specific vocabulary in the private transcription;

generating, by the first client device, a neutralized transcription based on neutralizing the private transcription; and

transmitting, by the first client device to the video conference provider, the neutralized transcription.

10. The method of claim 9 , wherein modifying the user-specific vocabulary in the private transcription comprises at least one of:

removing at least a portion of the user-specific vocabulary from the private transcription; or

replacing at least a portion of the user-specific vocabulary in the private transcription with generic vocabulary.

11. The method of claim 9 , further comprising:

transmitting, by the first client device to the video conference provider, an indication to share the neutralized transcription with the plurality of participants.

12. The method of claim 8 , further comprising:

transmitting, to the video conference provider, the private transcription;

neutralizing, by the video conference provider, the private transcription, wherein neutralizing the private transcription comprises modifying the user-specific vocabulary in the private transcription; and

generating, by the video conference provider, a neutralized transcription based on neutralizing the private transcription.

13. The method of claim 12 , further comprising:

transmitting, by the first client device to the video conference provider, an indication to share the private transcription with one or more of the plurality of participants.

14. A non-transitory computer-readable medium comprising processor-executable instructions configured to cause one or more processors to:

join a virtual meeting having a plurality of participants, wherein each participant in the virtual meeting exchanges one or more audio streams with other participants in the virtual meeting via the virtual meeting;

receive, from a video conference provider, a local model for speech recognition, wherein the local model comprises a copy of a centralized model;

store the copy of the centralized model as a local recognizer; and

perform speech recognition using the local model during the virtual meeting, wherein performing speech recognition comprises:

identifying, by the local recognizer, audio feature data from the one or more audio streams;

identifying, based on a vocabulary database, user-specific vocabulary within the audio feature data; and

generating, based on the user-specific vocabulary, a private transcription of the one or more audio streams, wherein the private transcription comprises the user-specific vocabulary.

15. The non-transitory computer-readable medium of claim 14 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

generate the vocabulary database based on user-specific data, wherein the user-specific data comprises one or more of:

a user's calendar;

a user's email account;

a user's contact list;

past private transcriptions of a user's audio data;

documents stored locally; and

information associated with a virtual meeting.

16. The non-transitory computer-readable medium of claim 14 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

identify one or more topics associated with a virtual meeting invitation;

perform, based on the one or more topics, an internet search;

determine, based on results from the internet search, one or more terms or phrases associated with the one or more topics; and

add the one or more terms or phrases to the vocabulary database.

17. The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive, via user input, an indication to search the user's email account for user-specific vocabulary; and

perform a search through one or more emails in a user's email account for user-specific vocabulary; and

identify, based on the search, one or more user-specific terms or phrases in the one or more emails; and

add the one or more user-specific terms or phrases to the vocabulary database.

18. The non-transitory computer-readable medium of claim 17 , wherein the vocabulary database comprises:

generic user-specific information comprises at least one of:

a name of the user; or

a location corresponding to the user; and

transient user-specific information comprising one or more of:

meeting specific information;

project specific information; or

email-based information.

19. The non-transitory computer-readable medium of claim 18 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

determine a time duration for the transient user-specific information in the vocabulary database;

identify one or more terms or phrases based on the transient user-specific information; and

remove, based on the time duration, the one or more terms or phrases from the vocabulary database.

20. The non-transitory computer-readable medium of claim 18 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive, from the video conference provider, a general transcription of the one or more audio streams, wherein the general transcription comprises transcription of non-user specific content from the one or more audio streams; and

combine the general transcription with the private transcription of the user-specific vocabulary to generate a complete transcription.