IP Library Granted Patent US 11,463,328
Granted Patent B2
US 11,463,328 · App. 16/940,510 · Granted Oct 4, 2022

Training a machine learning algorithm to create survey questions

Inventors: Karthik Ranganathan (Round Rock, TX); Sathish Kumar Bikumala (Round Rock, TX); Amit Sawhney (Round Rock, TX)
Assignee: Dell Products L.P.
H04L41/5009G06F17/15G06F40/205G06N20/00G10L15/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,463,328
App. No.
16/940,510
Granted
Oct 4, 2022
Kind
B2
Abstract

In some examples, a server may determine that a case, created to address an issue of a computing device, is closed and perform an analysis of a communication session between a user and a technician and the steps taken by the technician to resolve the issue. Machine learning may be used on results of the analysis to predict potential pain points. For example, steps that take longer than average and during which particular words spoken by the user increase in pitch and/or volume may be predicted to be potential pain points. The machine learning may create questions for inclusion in a custom survey based on the potential pain points. The custom survey may be presented to the user. The answers may be correlated with the potential pain points to determine actual pain points in the steps taken to resolve the issue.

Claims (136)

1. A method comprising:

determining, by a server associated with a technical support group, that a case created to address an issue associated with a computing device is closed;

performing, by the server, a natural language processing analysis of a communication session between a user of the computing device and a technician;

performing, by the server, a process analysis of a process used to close the case;

determining, by the server, a length of time corresponding to individual steps in the process used to close the case;

determining, using a machine learning algorithm executed by the server and based at least in part on the natural language processing analysis, the process analysis, and the length of time corresponding to individual steps, potential pain points associated with the communication session;

creating, by the server, one or more questions to include in a custom survey, wherein the creating comprises:

determining that a survey database does not include a particular question associated with a particular potential pain point of the potential pain points:

creating the particular question based at least in part on the particular potential pain point;

adding the particular question to the survey database; and

including the particular question in the custom survey;

presenting, by the server, the custom survey to the user;

receiving, by the server, user answers to individual questions of the one or more questions in the custom survey;

correlating, by the server, the user answers with the potential pain points to determine actual pain points associated with the communication session; and

determining, by the server, one or more techniques to address at least one of the actual pain points.

2. The method of claim 1 , wherein the natural language processing analysis comprises:

based on determining that the communication session comprises a voice call or a video call, converting an audio portion of the communication session into text;

parsing the text corresponding to the communication session to determine parts of speech associated with individual words in the text;

performing stemming of the individual words in the text;

performing lemmatization of the individual words in the text;

performing disambiguation of the individual words in the text;

determining, using the machine learning algorithm, at least one pain word; and

determining, based on particular steps of the process in which the at least one pain word was spoken, at least one pain point.

3. The method of claim 1 , further comprising:

based on determining that the communication session comprises a voice call or a video call, extracting an audio portion of the communication session;

performing a pitch analysis of a pitch associated with which each word is spoken in the audio portion of the communication session;

performing a volume analysis of a volume at which each word is spoken in the audio portion of the communication session;

determining, based at least in part on the pitch analysis and the volume analysis, at least one pain word; and

determining, based on particular steps of the process in which the at least one pain word was spoken, at least one pain point.

4. The method of claim 1 , further comprising:

retraining the machine learning algorithm based at least in part on:

the potential pain points associated with the communication session; and

the actual pain points associated with the communication session.

5. The method of claim 1 , wherein:

the communication session between the user of the computing device and the technician comprises one of:

a video call;

an audio call; or

a chat session.

6. A server comprising:

one or more processors; and

one or more non-transitory computer readable media storing instructions executable by the one or more processors to perform operations comprising:

determining that a case created to address an issue associated with a computing device is closed;

performing a natural language processing analysis of a communication session between a user of the computing device and a technician;

performing a process analysis of a process used to close the case;

determining a length of time corresponding to individual steps in the process used to close the case;

determining, using a machine learning algorithm and based at least in part on the natural language processing analysis, the process analysis, and the length of time corresponding to individual steps, potential pain points associated with the communication session;

creating one or more questions to include in a custom survey, wherein the creating comprises:

determining that a survey database does not include a particular question associated with a particular potential pain point of the potential pain points;

creating the particular question based at least in part on the particular potential pain point;

adding the particular question to the survey database; and

including the particular question in the custom survey;

presenting the custom survey to the user;

receiving user answers to individual questions of the one or more questions in the custom survey;

correlating the user answers with the potential pain points to determine actual pain points associated with the communication session; and

determining one or more techniques to address at least one of the actual pain points.

7. The server of claim 6 , wherein the natural language processing analysis comprises:

based on determining that the communication session comprises a voice call or a video call, converting an audio portion of the communication session into text;

parsing the text corresponding to the communication session to determine parts of speech associated with individual words in the text;

performing stemming of the individual words in the text;

performing lemmatization of the individual words in the text;

performing disambiguation of the individual words in the text;

determining, using the machine learning algorithm, at least one pain word; and

determining, based on particular steps of the process in which the at least one pain word was spoken, at least one pain point.

8. The server of claim 6 , further comprising:

based on determining that the communication session comprises a voice call or a video call, extracting an audio portion of the communication session;

performing a pitch analysis of a pitch associated with which each word is spoken in the audio portion of the communication session;

performing a volume analysis of a volume at which each word is spoken in the audio portion of the communication session;

determining, based at least in part on the pitch analysis and the volume analysis, at least one pain word; and

determining, based on particular steps of the process in which the at least one pain word was spoken, at least one pain point.

9. The server of claim 6 , wherein presenting the custom survey to the user comprises one of:

presenting a web-based survey to the user; or

presenting an interactive voice response (IVR) based survey to the user.

10. One or more non-transitory computer readable media storing instructions executable by one or more processors to perform operations comprising:

determining that a case created to address an issue associated with a computing device is closed;

performing a natural language processing analysis of a communication session between a user of the computing device and a technician;

performing a process analysis of a process used to close the case;

determining a length of time corresponding to individual steps in the process used to close the case;

determining, using a machine learning algorithm and based at least in part on the natural language processing analysis, the process analysis, and the length of time corresponding to individual steps, potential pain points associated with the communication session;

creating one or more questions to include in a custom survey, wherein the creating comprises:

in response to determining that a survey database includes a particular question associated with a particular potential pain point of the potential pain points:

retrieving the particular question from the survey database;

modifying the particular question based at least in part on the process used to close the case to create a modified question; and

including the modified question in the custom survey; and

in response to determining that the survey database does not include the particular question associated with the particular potential pain point of the potential pain points:

creating the particular question based at least in part on the particular potential pain point;

adding the particular question to the survey database; and

including the particular question in the custom survey;

presenting the custom survey to the user;

receiving user answers to individual questions of the one or more questions in the custom survey;

correlating the user answers with the potential pain points to determine actual pain points associated with the communication session; and

determining one or more techniques to address at least one of the actual pain points.

11. The one or more non-transitory computer readable media of claim 10 , wherein the natural language processing analysis comprises:

based on determining that the communication session comprises a voice call or a video call, converting an audio portion of the communication session into text;

parsing the text corresponding to the communication session to determine parts of speech associated with individual words in the text;

performing stemming of the individual words in the text;

performing lemmatization of the individual words in the text;

performing disambiguation of the individual words in the text;

determining, using the machine learning algorithm, at least one pain word; and

determining, based on particular steps of the process in which the at least one pain word was spoken, at least one pain point.

12. The one or more non-transitory computer readable media of claim 10 , further comprising:

based on determining that the communication session comprises a voice call or a video call, extracting an audio portion of the communication session;

performing a pitch analysis of a pitch associated with which each word is spoken in the audio portion of the communication session;

performing a volume analysis of a volume at which each word is spoken in the audio portion of the communication session;

determining, based at least in part on the pitch analysis and the volume analysis, at least one pain word; and

determining, based on particular steps of the process in which the at least one pain word was spoken, at least one pain point.

13. The one or more non-transitory computer readable media of claim 10 , further comprising:

performing a skills analysis of the technician including:

determining a skills gap set of steps in the process to close the case, wherein each step in the skills gap set of steps took an above average amount of time to perform and is one of the actual pain points; and

determining a skills advantage set of steps in the process to close the case, wherein each step in the skills advantage set of steps took a below average amount of time to perform and is not one of the actual pain points.

14. The one or more non-transitory computer readable media of claim 13 , further comprising:

scheduling training for the technician to address the skills gap set.

15. The one or more non-transitory computer readable media of claim 13 , further comprising:

creating a custom technician survey based at least in part on the skills advantage set of steps;

determining, based on the technician's response to the custom technician survey, a particular technique that the technician used to perform at least one particular step in the skills advantage set of steps; and

sharing the particular technique with other technicians to enable the other technicians to perform the at least one particular step in the below average amount of time.

16. The method of claim 1 , wherein creating the one or more questions to include in the custom survey further comprises:

determining that a survey database includes a particular question associated with a particular potential pain point of the potential pain points;

retrieving the particular question from the survey database;

modifying the particular question based at least in part on the process used to close the case to create a modified question; and

including the modified question in the custom survey.

17. The server of claim 6 , wherein creating the one or more questions to include in the custom survey further comprises:

determining that a survey database includes a particular question associated with a particular potential pain point of the potential pain points;

retrieving the particular question from the survey database;

modifying the particular question based at least in part on the process used to close the case to create a modified question; and

including the modified question in the custom survey.

18. The method of claim 1 , wherein presenting the custom survey to the user comprises one of:

presenting a web-based survey to the user; or

presenting an interactive voice response (IVR) based survey to the user.

19. The server of claim 6 , further comprising:

retraining the machine learning algorithm based at least in part on:

the potential pain points associated with the communication session; and

the actual pain points associated with the communication session.

20. The one or more non-transitory computer readable media of claim 10 , further comprising:

retraining the machine learning algorithm based at least in part on:

the potential pain points associated with the communication session; and

the actual pain points associated with the communication session.

Assignments (9)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053574/0221) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060333/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053578/0183) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060332/0864 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053573/0535) Recorded Jun 10, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060333/0106 →
RELEASE OF SECURITY INTEREST AT REEL 053531 FRAME 0108 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0371 →
SECURITY INTEREST Recorded Aug 21, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 053578/0183 →
SECURITY INTEREST Recorded Aug 21, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 053573/0535 →
SECURITY INTEREST Recorded Aug 21, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 053574/0221 →
SECURITY AGREEMENT Recorded Aug 18, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 053531/0108 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2020
From: RANGANATHAN, KARTHIK; BIKUMALA, SATHISH KUMAR; SAWHNEY, AMIT
To: DELL PRODUCTS L. P.
Reel/Frame 053326/0619 →