Multi-modal automation for human interactive skill assessment
View Patent ↗A method for screening applicants includes remotely accessing a screening entity's web page by a potential candidate. The candidate is asked, via the pre-screening web page, to input responses to a set of queries tailored to the screening entity and the candidate's responses to the queries are stored. A phone call is automatically initiated to the candidate in response to at least one of the candidate's responses. An interactive information gathering process is started with the candidate using both the telephone and the computer. In the process, audio prompts to the candidate are generated via the telephone and the candidate's verbal responses are recorded via the telephone. A profile of the candidate is stored in a database including the verbal responses from the candidate for later access and analysis by the screening entity. The recorded responses are analyzed by the screening entity to make quality judgments about the candidate.
1. A method for screening applicants by a screening entity, which
comprises:
remotely accessing a screening entity's pre-screening online web page by a potential candidate using an Internet-accessible computer available to the candidate;
using the pre-screening web page, requesting the candidate to input responses to a set of queries into the computer that are tailored to the screening entity;
storing the candidate's responses to the queries in a profile;
at least during the response requesting step, automatically initiating a phone call to the candidate over a telephone on a telephone network in response to at least one of the candidate's responses;
and
commencing an interactive information gathering process with the candidate using both the telephone and the computer by:
generating prompts to the candidate via the telephone and the web page; and recording the candidate's verbal responses to the telephone and web page prompts via the telephone;
storing a profile of the candidate in a database including the verbal responses from the candidate for later access and analysis in a skill assessment of the candidate by the screening entity; and
analyzing the recorded responses by the screening entity to make quality and criteria judgments about the candidate to determine the hiring potential of the candidate, the analyzing step comprising automatically determining a confidence scoring of the candidate with a computerized speech recognition device by automation.
2. The method according to claim 1 , which further comprises:
hosting the web page with an external server; and
accessing an internal server that hosts an internal web page of the screening entity.
3. The method according to claim 1 , which further comprises one of:
carrying out the phone call initiating step by simultaneously initiating the phone call to the candidate; and
carrying out the phone call initiating step by initiating the phone call to the candidate after the candidate has completed responding to the queries.
4. The method according to claim 1 , which further comprises initiating the phone call to the candidate with a voice server through a private branch exchange over the telephone network.
5. The method according to claim 1 , which further comprises confirming the candidate's identity through a set of identification questions and responses via one of the telephone and the pre-screening web page.
6. The method according to claim 1 , which further comprises carrying out the profile storing step by storing the profile of the candidate one of:
during the information gathering process; and
after the information gathering process is complete.
7. The method according to claim 1 , which further comprises carrying out the profile storing step by storing the profile of the candidate through an entity-secure intranet link.
8. The method according to claim 1 , which further comprises carrying out the phone call initiating step by initiating the phone call only with outbound calls from the screening entity to, thereby, control access to the information gathering process.
9. The method according to claim 1 , which further comprises carrying out the information gathering step by controlling a number and kind of questions presented to the candidate to simulate a real call-center seat.
10. The method according to claim 1 , which further comprises carrying out the storing step one of selectively and continually.
11. The method according to claim 1 , wherein the computerized speech recognition device is programmed to parse a particular response into individual words and to either compare the parsed response to a desired responses or to transcribe the parsed response for later use and access by the screening entity.
12. The method according to claim 11 , which further comprises carrying out the automated confidence scoring step by judging at least an accuracy and an intelligibility of the candidate's speech utilizing a target phrase represented in a speech recognition grammar.
13. A method for screening candidates by a screening entity, which
comprises:
initiating a job application procedure by having a candidate remotely connect to a secured online web page of a screening entity using an Internet-accessible computer;
using the secured online web page, querying the candidate to enter candidate-specific personal identification data using the computer;
securely confirming the identification data; and
initiating a phone call to candidate after the identification data is confirmed and while the candidate is still communicating with the web page and, during the phone call, carrying out at least one of the following exercise steps:
initiating a dictation clarity exercise by instructing candidate to read a script sent to a particular web page viewable by the candidate on the computer and, when finished, recording speech from the candidate for subsequent off-line analysis and applying a confidence scoring to the recorded speech; and
initiating a graphic comprehension exercise to screen the candidate's ability to respond to a provided graphic by displaying a graphic to the candidate on the computer, requiring the candidate to answer questions regarding the graphic, recording the spoken answers over the telephone, and automatically scoring each answer with a confidence level correlated with how well the recorded phrase matches the expected pronunciation;
after a given number of exercises are complete, storing data recorded for later analysis in a skill assessment of the candidate by the screening entity and informing the candidate that an application process is complete; and
analyzing the recorded responses by the screening entity to make quality and criteria judgments about the candidate to determine the hiring potential of the candidate, the analyzing step comprising automatically determining a confidence scoring of the candidate with a computerized speech recognition device by automation.
14. The method according to claim 13 , which further comprises securely confirming the identification data.
15. The method according to claim 13 , which further comprises carrying out the secure confirming step by one of directly with the candidate and indirectly through an email sent to the candidate's email address that requires a response from the candidate.
16. The method according to claim 13 , which further comprises carrying out the phone call initiating step with a voice server immediately after the identification data is confirmed.
17. The method according to claim 13 , which further comprises carrying out at least one of the following exercise steps:
initiating a voice quality exercise over the telephone by asking the candidate to repeat spoken phrases over the telephone, recording the candidate's speech for each phrase for subsequent off-line analysis, and automatically scoring each recorded phrase with a confidence level correlated with how well the recorded phrase matches an expected pronunciation;
initiating a question-answer exercise by transmitting specific multiple-choice questions over the telephone, prompting the candidate to answer each question after it is presented, recording the candidate's speech for each answer, and automatically scoring each recorded answer with a confidence level correlated to how well the recorded phrase matches an expected pronunciation;
initiating a service knowledge exercise to determine the candidate's ability to comprehend an example of what good customer service is and whether the candidate can intelligently describe such an experience by having the candidate give a speech regarding such an experience, recording the speech for subsequent off-line analysis, and applying a subjective confidence scoring to the recorded speech; and
initiating a speaking satisfaction exercise with the candidate to evaluate the kind of experience that a potential customer will have after speaking to the candidate by giving the candidate a random one of a series of predefined situations in which candidate is required to role-play an operator answering in real-time a call from a virtual customer, recording the conversation between the candidate and the virtual customer for subsequent off-line analysis, and applying a subjective confidence scoring to the recorded conversation.
18. The method according to claim 13 , which further comprises carrying out the voice quality initiating exercise step by storing the candidate's speech for each phrase for off-line analysis.
19. The method according to claim 13 , which further comprises carrying out the dictation clarity exercise step by storing speech from the candidate for subsequent off-line analysis.
20. The method according to claim 13 , which further comprises carrying out the confidence scoring step at least one of automatically and with human judgment.
21. The method according to claim 13 , which further comprises carrying out the question-answer exercise step by storing the candidate's speech for each question for subsequent off-line analysis.
22. The method according to claim 13 , which further comprises carrying out the question-answer exercise step by transmitting the multiple-choice questions by at least one of a human voice and a machine voice.
23. The method according to claim 13 , which further comprises carrying out the service knowledge exercise step by storing the speech for subsequent off-line analysis.
24. The method according to claim 13 , which further comprises carrying out the graphic comprehension exercise step by storing the spoken answers for subsequent off-line analysis.
25. The method according to claim 13 , which further comprises carrying out the graphic comprehension exercise step by providing a map as the graphic and screening the candidate's ability to give driving directions.
26. The method according to claim 13 , which further comprises carrying out the speaking satisfaction exercise step by storing the conversation for subsequent off-line analysis.
27. The method according to claim 13 , which further comprises providing the candidate with a timeframe for hearing from the screening entity or given a call number and a date for checking on the candidate's application.
28. The method according to claim 13 , which further comprises providing the candidate with a call number and a date for checking on a status of the application.