IP Library Granted Patent US 9,378,486
Granted Patent B2
US 9,378,486 · App. 14/535,205 · Granted Jun 28, 2016

Automatic interview question recommendation and analysis

Inventors: Benjamin Taylor (Lehi, UT); Loren Larsen (Lindon, UT)
Assignee: HireVue, Inc.
G06Q10/1053
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,378,486
App. No.
14/535,205
Granted
Jun 28, 2016
Kind
B2
Abstract

Described herein are methods and systems for interview question or prompt recommendation and analysis to improve the quality and efficacy of subsequent evaluation campaigns by combining data sets are described herein. In one method, processing logic selects a first prompt from a first data set of a first candidate evaluation campaign and a second prompt from a second data set of a second candidate evaluation campaign. The processing logic determines whether a degree of similarity between the first prompt and the second prompt exceeds a threshold and combines data from the first data set with data from the second data set to create a combined data set associated with the first prompt and with the second prompt based on the determination.

Claims (57)

1. A method comprising:

receiving an input identifying a position sector;

selecting a first prompt from a first candidate evaluation campaign associated with the position sector, the first candidate evaluation campaign comprising a first data set;

selecting a second prompt from a second candidate evaluation campaign associated with the position sector, the first prompt and the second prompt being accessible in a database of a candidate evaluation system being executed by a processing device, the second candidate evaluation campaign comprising a second data set;

determining, by the processing device, whether a degree of similarity between the first prompt and the second prompt exceeds a threshold;

combining data from the first data set with data from the second data set to create a related cluster of prompts associated with the first prompt and with the second prompt based on the determination relative to the threshold;

calculating, by a prompt recommendation engine of the candidate evaluation system, a weight for each word in each prompt of the related cluster of prompts, wherein the weight of each word is calculated by mapping the respective word to at least one of an evaluation rating, an evaluation outcome or result, or an achievement index;

determining, by the prompt recommendation engine, the prompt having a highest average word weight; and

providing, by the prompt recommendation engine, a template prompt, based on the determination relative to the highest average word weight.

2. The method of claim 1 , wherein the combining data from the first data set with data from the second data set to create the combined data set is performed in response to a determination that the degree of similarity is greater than the threshold.

3. The method of claim 1 , further comprising determining that the first candidate evaluation campaign and the second candidate evaluation campaign correspond to one or more positions in the position sector.

4. The method of claim 1 , wherein the position sector is one of a sales sector, an engineering sector, an accounting sector, a legal sector, or another position sector.

5. The method of claim 1 , further comprising:

analyzing the combined data set to assess a first correlation between the first prompt and an evaluation result and a second correlation between the second prompt and the evaluation result;

ranking the first prompt and the second prompt according to the first correlation and the second correlation; and

recommending the first prompt to a campaign manager during creation of a new candidate evaluation campaign in response to a determination that the first prompt is ranked higher than the second prompt.

6. The method of claim 2 , wherein the determining whether the degree of similarity between the first prompt and the second prompt exceed the threshold further comprise calculating a ratio determined by a distance between the first prompt and the second prompt divided by an average length of the first prompt and the second prompt.

7. The method of claim 1 , wherein calculating the degree of similarity comprises counting a number of word-based edits to transform the first prompt into the second prompt.

8. The method of claim 1 , further comprising

recommending the template prompt to an evaluation designer for a third candidate evaluation campaign.

9. A computing system comprising:

a data storage device; and

a processing device, coupled to the data storage device, to execute an evaluation platform to:

receive an input identifying a position sector;

select a first prompt from a first candidate evaluation campaign associated with the position sector, the first candidate evaluation campaign comprising a first data set;

select a second prompt from a second candidate evaluation campaign associated with the position sector, the first prompt and the second prompt being accessible in a database associated with the evaluation platform, the second candidate evaluation campaign comprising a second data set;

calculate a degree of similarity based on a number of edits to modify the first prompt to be identical to the second prompt; and

combine data from the first data set with data from the second data set to create a related cluster of prompts associated with the first prompt and with the second prompt;

calculate a weight for each word in each prompt of the related cluster of prompts, wherein the weight of each word is calculated by mapping the respective word to at least one of an evaluation rating, an evaluation outcome or result, or an achievement index;

determine the prompt having a highest average word weight; and

provide a template prompt based on the determination relative to the highest average word weight.

10. The system of claim 9 , wherein the data from the first data set is combined with data from the second data set to create the combined data set in response to a determination that the degree of similarity is greater than a threshold.

11. The system of claim 10 , wherein the degree of similarity is a ratio determined by a number of edits between the first prompt and the second prompt divided by an average length of the first prompt and the second prompt.

12. The system of claim 9 , wherein the evaluation platform is to determine that the first candidate evaluation campaign and the second candidate evaluation campaign correspond to one or more positions in the position sector.

13. The system of claim 9 , wherein the processing device is to further execute the evaluation platform to:

analyze the combined data set to assess a first correlation between the first prompt and an evaluation result and a second correlation between the second prompt and the evaluation result;

rank the first prompt and the second prompt according to the first correlation and the second correlation; and

recommend the first prompt to a campaign manager in conjunction with creation of a new candidate evaluation campaign in response to a determination that a first prompt is ranked higher than the second prompt.

14. The system of claim 9 , wherein determining the number of edits comprises counting a number of word-based edits to transform the first prompt into the second prompt.

15. The system of claim 9 , wherein the evaluation platform is further to:

add at least one of the first prompt or the second prompt to a prompt cluster that comprises a plurality of prompts;

identify one of the plurality of prompts as the template prompt; and

recommend the template prompt to an evaluation designer for a third candidate evaluation campaign.

16. A method of generating a template prompt in an evaluation system, the method comprising:

retrieving a plurality of prompts associated with a position sector in the evaluation system;

identifying, by a processing device, a related cluster of prompts within the plurality of prompts, the related cluster characterized by a distance between the prompts in the related cluster, wherein the distance is a number of edits to modify a first prompt in the related cluster to be identical to a second prompt in the related cluster;

calculating, by the processing device, a weight for each word in each prompt of the related cluster of prompts, wherein the weight of each word is calculated by mapping the respective word to at least one of an evaluation rating, an evaluation outcome or result, or an achievement index;

determining the prompt having a highest average word weight; and

generating the template prompt from the related cluster based on the determination relative to the highest average word weight.

17. The method of claim 16 , wherein generating the template prompt from the related cluster of prompts further comprises:

determining a first average distance of a first prompt of the related cluster to at least one other prompt of the related cluster;

determining a second average distance of a second prompt of the related cluster to at least one other prompt of the related cluster; and

selecting the first prompt as the template prompt in response to a determination that the first average distance is less than the second average distance.

18. The method of claim 16 , further comprising:

analyzing a combined data set associated with the plurality of prompts to assess a correlation between the prompts and an evaluation result; and

ranking the prompts according to the correlation; and

wherein generating the template prompt comprises generating the template prompt based on a highest ranked prompt of the plurality of prompts.

Assignments (3)
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 4, 2023
From: HIREVUE, INC.; SASS LABS INC.; MODERN HIRE, INC.; MODERN HIRE HOLDING COMPANY, INC.
To: SIXTH STREET LENDING PARTNERS
Reel/Frame 063530/0669 →
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION NO. 12/535,205 PREVIOUSLY RECORDED AT REEL: 034127 FRAME: 0032. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Nov 11, 2014
From: TAYLOR, BENJAMIN; LARSEN, LOREN
To: HIREVUE, INC.
Reel/Frame 034206/0248 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2014
From: TAYLOR, BENJAMIN; LARSEN, LOREN
To: HIREVUE, INC.
Reel/Frame 034127/0032 →
Continuity (2)
Provisional Application 61954385 · Mar 17, 2014
Related Publication 20150262130A1 · Sep 17, 2015