Forecasting model generation for sample biased data set
A method, apparatus, system, and computer program product for creating a forecasting model for payroll records. Payroll records are received for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. A forecasting model is created that aligns the payroll records to high-level employment data. Creating the forecasting model includes identifying predictor variables from the granular data parameters of the payroll records. Creating the forecasting model includes generating a set of basis functions from the predictor variables. Creating the forecasting model includes combining the set of basis functions to create the forecasting model.
1. A method for creating a forecasting model from a sampling-biased data set, the method comprising:
receiving, by a computer system, payroll records for a group of employers, wherein the payroll records comprise granular data parameters about employees of the group of employers; and
creating, by the computer system, the forecasting model that aligns the granular data parameters of the payroll records to high-level employment data, wherein creating the forecasting model comprises a non-parametric regression analysis technique that models nonlinearities and interactions between variables using multivariate adaptive regression splines to perform the steps of:
identifying, by the computer system, predictor variables from the granular data parameters of the payroll records comprising records selected from gross pay, net pay, withholdings, benefits, and combinations thereof;
generating, by the computer system, a set of basis functions from the predictor variables; and
combining, by the computer system, the set of basis functions to create the forecasting model that forecasts the high-level employment data from the granular data parameters of the payroll records.
2. The method of claim 1 , wherein creating the forecasting model, takes a form of:
f
(
x
)
=
∑
i
=
1
k
c
i
B
i
(
x
)
wherein:
c i is a constant coefficient; and
B i (x) is a basis function selected from a constant, a hinge function, and a product of two or more hinge functions.
3. The method of claim 2 , further comprising:
using, by the computer system, the forecasting model to predict employment data at a higher level of granularity than reported in the high-level employment data that comprises statistics about employment, hours, and earnings estimates for geographic areas at detailed industry levels.
4. The method of claim 2 , further comprising:
classifying, by the computer system, the payroll records into of a plurality of cells at an industry level and a size level; and
within each of the plurality of cells, generating a time series from the payroll records;
wherein the forecasting model aligns the plurality of cells to distributions of industry and employer size reported in the high-level employment data.
5. The method of claim 4 , further comprising:
seasonally adjusting, by the computer system, the time series; and
removing, by the computer system, outliers from the seasonally-adjusted time series.
6. The method of claim 5 , wherein the seasonally adjusting and removing outlier steps are performed using an autoregressive integrated moving average (ARIMA) and seasonal adjustment decomposition (SEATS) of the time series.
7. The method of claim 5 , further comprising:
adjusting, by the computer system, the time series within each of the plurality of cells to match the distributions of industry and employer size reported in the high-level employment data, including:
interpolating the high-level employment data to determine weight values at the industry level;
extrapolating the weight values at the size level based on the time series of a corresponding one of the plurality of cells; and
computing a weighted average of the seasonally-adjusted time series within each industry based on the extrapolated weight values.
8. A computer system comprising:
a model manager that operates to:
receive payroll records for a group of employers, wherein the payroll records comprise granular data parameters about employees of the group of employers; and
create a forecasting model that aligns the granular data parameters of the payroll records to high-level employment data, wherein creating the forecasting model comprises a non-parametric regression analysis technique that models nonlinearities and interaction between variables using multivariate adaptive regression splines, the computer system further operates to:
identify predictor variables from the granular data parameters of the payroll records comprising records selected from gross pay, net pay, withholdings, benefits, and combinations thereof;
generate a set of basis functions from the predictor variables; and
combine the set of basis functions to create the forecasting model that forecasts the high-level employment data from the granular data parameters of the payroll records.
9. The computer system of claim 8 , wherein creating the forecasting model, takes a form of:
f
(
x
)
=
∑
i
=
1
k
c
i
B
i
(
x
)
wherein:
c i is a constant coefficient; and
B i (x) is a basis function selected from a constant, a hinge function, and a product of two or more hinge functions.
10. The computer system of claim 9 , wherein the computer system further operates to:
using the forecasting model to predict employment data at a higher level of granularity than reported in the high-level employment data that comprises statistics about employment, hours, and earnings estimates for geographic areas at detailed industry levels.
11. The computer system of claim 9 , wherein the computer system further operates to:
classify the payroll records into of a plurality of cells at an industry level and a size level; and
within each of the plurality of cells, generate a time series from the payroll records;
wherein the forecasting model aligns the plurality of cells to distributions of industry and employer size reported in the high-level employment data.
12. The computer system of claim 11 , wherein the computer system further operates to:
seasonally adjust the time series; and
remove outliers from the seasonally-adjusted time series.
13. The computer system of claim 12 , wherein the seasonally adjusting and removing outlier steps are performed using an autoregressive integrated moving average (ARIMA) and seasonal adjustment decomposition (SEATS) of the time series.
14. The computer system of claim 12 , wherein the computer system further operates to:
adjust the time series within each of the plurality of cells to match the distributions of industry and employer size reported in the high-level employment data, including:
interpolating the high-level employment data to determine weight values at the industry level;
extrapolating the weight values at the size level based on the time series of a corresponding one of the plurality of cells; and
computing a weighted average of the seasonally-adjusted time series within each industry based on the extrapolated weight values.
15. A computer program product for creating a forecasting model from a sampling-biased data set, the computer program product comprising:
a non-transitory computer-readable storage media;
program code, stored on the non-transitory computer-readable storage media, for receiving payroll records for a group of employers, wherein the payroll records comprise granular data parameters about employees of the group of employers; and
program code, stored on the non-transitory computer-readable storage media, for creating the forecasting model that aligns the granular data parameters of the payroll records to high-level employment data, wherein the program code for creating the forecasting model comprises a non-parametric regression analysis technique that models nonlinearities and interactions between variables using multivariate adaptive regression splines including:
program code for identifying predictor variables from the granular data parameters of the payroll records comprising records selected from gross pay, net pay, withholdings, benefits, and combinations thereof;
program code for generating a set of basis functions from the predictor variables; and
program code for combining the set of basis functions to create the forecasting model that forecasts the high-level employment data from the granular data parameters of the payroll records.
16. The computer program product of claim 15 , wherein the program code for creating the forecasting model, takes a form of:
f
(
x
)
=
∑
i
=
1
k
c
i
B
i
(
x
)
wherein:
c i is a constant coefficient; and
B i (x) is a basis function selected from a constant, a hinge function, and a product of two or more hinge function.
17. The computer program product of claim 16 , further comprising:
program code, stored on the non-transitory computer-readable storage media, for using the forecasting model to predict employment data at a higher level of granularity than reported in the high-level employment data that comprises statistics about employment, hours, and earnings estimates for geographic areas at detailed industry levels.
18. The computer program product of claim 16 , further comprising:
program code, stored on the non-transitory computer-readable storage media, for classifying the payroll records into of a plurality of cells at an industry level and a size level; and
program code, stored on the non-transitory computer-readable storage media, for generating, within each of the plurality of cells, a time series from the payroll records;
wherein the forecasting model aligns the plurality of cells to distributions of industry and employer size reported in the high-level employment data.
19. The computer program product of claim 18 , further comprising:
program code, stored on the non-transitory computer-readable storage media, for seasonally adjusting the time series; and
program code, stored on the non-transitory computer-readable storage media, for removing outliers from the seasonally-adjusted time series.
20. The computer program product of claim 19 , where in the seasonally adjusting and removing outlier steps are performed using an autoregressive integrated moving average (ARIMA) and seasonal adjustment decomposition (SEATS) of the time series.
21. The computer program product of claim 19 , further comprising:
program code, stored on the non-transitory computer-readable storage media, for adjusting the time series within each of the plurality of cells to match the distributions of industry and employer size reported in the high-level employment data, including:
program code for interpolating the high-level employment data to determine weight values at the industry level;
program code for extrapolating the weight values at the size level based on the time series of a corresponding one of the plurality of cells; and
program code for computing a weighted average of the seasonally-adjusted time series within each industry based on the extrapolated weight values.