METHOD AND SYSTEM FOR OBTAINING REPRESENTATIVE SAMPLE OF AUTOMOTIVE DATA RECORDS FOR DATA SCORING
A method and system for obtaining representative sample of automotive data records for data scoring is provided herein. The method may include the following steps: applying a sampling operator to unique IDs of automotive data records space associated with connected vehicles represented by said unique IDs and respective automotive data sources, wherein the sampling operator is selected such that it preserve the proportions in the sample compared with the space; deriving a sample of automotive data records from said automotive data records by selecting a subset of the unique IDs with the sampling operator applied thereto, based on a sampling factor; verifying that the sample of automotive data records is a valid sample of the automotive data records space; and scoring the data records and or the data sources of the sample of automotive data records based on data scoring schemes.
1 . A method of obtaining a representative sample of automotive data records for data scoring, the method comprising:
applying a sampling operator to unique IDs of automotive data records space associated with connected vehicles represented by said unique IDs and respective automotive data sources, wherein the sampling operator is selected such that it preserves the proportions in the sample compared with the space;
deriving a sample of automotive data records from said automotive data records by selecting a subset of the unique IDs with the sampling operator applied thereto, based on a sampling factor;
verifying that the sample of automotive data records is a valid sample of the automotive data records space; and
scoring the data records and or the data sources of the sample of automotive data records based on data scoring schemes.
2 . The method according to claim 1 , further comprising: repeating the deriving a sample of automotive data records on ore more time wherein in each of the one or more time, a different sampling factor is selected.
3 . The method according to claim 1 , wherein the verifying is carried out by ascertaining that similar data records groups at the sample of automotive data records retain their representation compared with their representation within the automotive data records space.
4 . The method according to claim 1 , wherein the verifying is carried out by comparing statistics of the subset automotive data records with statistics of the plurality of anonymized automotive data records.
5 . The method according to claim 1 , wherein the sampling is carried out using a modulo associated with the sampled unique ID.
6 . The method according to claim 1 , wherein the sampling factor is adjusted in each sampling based on the number of unique IDs in the automotive data records space.
7 . A server for obtaining a representative sample of automotive data records for data scoring, the server comprising:
a sampling module configured to:
apply a sampling operator to unique IDs of automotive data records space associated with connected vehicles represented by said unique IDs and respective automotive data sources, wherein the sampling operator is selected such that it preserve the proportions in the sample compared with the space;
derive a sample of automotive data records from said automotive data records by selecting a subset of the unique IDs with the sampling operator applied thereto, based on a sampling factor;
a verifying module configured to verify that the sample of automotive data records is a valid sample of the automotive data records space; and
a data scoring module configured to score the data records and or the data sources of the sample of automotive data records based on data scoring schemes.
8 . The server according to claim 7 , further comprising: repeating the deriving a sample of automotive data records on or more time wherein in each of the one or more time, a different sampling factor is selected.
9 . The server according to claim 7 , wherein the verifying is carried out by ascertaining that similar data records groups at the sample of automotive data records retain their representation compared with their representation within the automotive data records space.
10 . The server according to claim 7 , wherein the verifying is carried out by comparing statistics of the subset automotive data records with statistics of the plurality of anonymized automotive data records.
11 . The server according to claim 7 , wherein the sampling is carried out using a modulo associated with the sampled unique ID.
12 . The server according to claim 7 , wherein the sampling factor is adjusted in each sampling based on the number of unique IDs in the automotive data records space.