Vector generation for distributed data sets
In various example embodiments, a vector modeling system is configured to access a set of data distributed across client devices and stored in a structured format. The vector modeling system determines vector parameters and vector templates suitable for the set of data and transforms the set of data from the structured format into a second format including one or more vectors based on one or more transformation strategies. The vector modeling system stores the transformed data and performs machine learning analysis on the vector.
1. A method, comprising:
distributing, from a server, an integrated vector template to a first client device and a second client device, the integrated vector template being generated by the server based on a first set of data provided by the first client device and a second set of data provided by the second client device, the first and second sets of data being configured in a first format;
sending, from the server to the first client device, a first conversion instruction that causes the first client device to generate the first set of data in a second format based on the integrated vector template, the first set of data in the second format comprising a first set of vectors that have one or more vector values converted from one or more data entries in the first set of data;
receiving, at the server, a first converted set of data from the first client device, the first converted set of data comprising the first set of data in the second format;
sending, from the server to the second client device, a second conversion instruction that causes the second client device to generate the second set of data in the second format based on the integrated vector template, the second set of data in the second format comprising a second set of vectors that have one or more vector values converted from one or more data entries in the second set of data;
receiving, at the server, a second converted set of data from the second client device, the second converted set of data comprising the second set of data in the second format;
mapping, by the server, each data entry in the first and second converted sets of data to a vector value to generate a conversion map, the conversion map assisting in transforming data entries, from the first or second client devices, to vector values that have a format compatible for processing by at least one machine learning algorithm;
generating, by the server, at least one subsequent vector from an additional set of data using the conversion map;
processing, by the server, the at least one subsequent vector using the at least one machine learning algorithm; and
generating a set of probability values indicative of a probability of an event occurring with respect to an element included in the at least one subsequent vector, based on the processing of the at least one subsequent vector using the at least one machine learning algorithm.
2. The method of claim 1 , further comprising:
generating an integrated vector parameter for the integrated vector template by:
transmitting a first fit instruction to the first client device, the first fit instruction causing the first client device to determine a first vector parameter for a first vector template to represent the first set of data stored on the first client device within a first application;
transmitting a second fit instruction to the second client device, the second fit instruction causing the second client device to determine a second vector parameter for a second vector template to represent the second set of data stored on the second client device within a second application; and
receiving, by the server, the first vector parameter for the first vector template and the second vector parameter for the second vector template.
3. The method of claim 1 , further comprising:
identifying a set of elements in the first converted set of data and the second converted set of data without a corresponding data entry; and
assigning a null value to the set of elements in the first converted set of data and the second converted set of data, the null value associated with one or more null indicators in the first converted set of data and the second converted set of data.
4. The method of claim 1 , further comprising:
identifying a set of vector positions within the first set of vectors and the second set of vectors having a predetermined value;
removing the set of vector positions having the predetermined value to generate a set of modified vectors; and
storing the set of modified vectors in a database accessible by the server.
5. The method of claim 1 , wherein generating the first set of vectors and the second set of vectors further comprises:
causing presentation of a set of transform elements indicating a transform strategy for converting the first set of data and the second set of data from the first format into the second format;
receiving selection of a transform element indicating a selected transform strategy associated with the transform element; and
generating the first set of vectors and the second set of vectors according to the selected transform strategy.
6. The method of claim 1 , further comprising:
accessing, by the server, a first set of data provided by a first client device and a second set of data provided by a second client device.
7. The method of claim 1 , further comprising:
generating, by the server, the integrated vector template based on the first set of data provided by the first client device and the second set of data provided by the second client device.
8. A computer implemented system, comprising:
one or more processors;
a non-transitory machine-readable storage medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
distributing an integrated vector template to a first client device and a second client device, the integrated vector template being generated by the computer implemented system based on a first set of data provided by the first client device and a second set of data provided by the second client device, the first and second sets of data being configured in a first format;
sending, to the first client device, a first conversion instruction that causes the first client device to generate the first set of data in a second format based on the integrated vector template, the first set of data in the second format comprising a first set of vectors that have one or more vector values converted from one or more data entries in the first set of data;
receiving a first converted set of data from the first client device, the first converted set of data comprising the first set of data in the second format;
sending, to the second client device, a second conversion instruction that causes the second client device to generate the second set of data in the second format based on the integrated vector template, the second set of data in the second format comprising a second set of vectors that have one or more vector values converted from one or more data entries in the second set of data;
receiving a second converted set of data from the second client device, the second converted set of data comprising the second set of data in the second format;
mapping each data entry in the first and second converted sets of data to a vector value to generate a conversion map, the conversion map assisting in transforming data entries, from the first or second client devices, to vector values that have a format compatible for processing by at least one machine learning algorithm;
generating, by the server, at least one subsequent vector from an additional set of data using the conversion map;
processing, by the server, the at least one subsequent vector using the at least one machine learning algorithm; and
generating a set of probability values indicative of a probability of an event occurring with respect to an element included in the at least one subsequent vector, based on the processing of the at least one subsequent vector using the at least one machine learning algorithm.
9. The system of claim 8 , wherein the operations further comprise:
generating an integrated vector parameter for the integrated vector template by:
transmitting a first fit instruction to the first client device, the first fit instruction causing the first client device to determine a first vector parameter for a first vector template to represent the first set of data stored on the first client device within a first application;
transmitting a second fit instruction to the second client device, the second fit instruction causing the second client device to determine a second vector parameter for a second vector template to represent the second set of data stored on the second client device within a second application; and
receiving the first vector parameter for the first vector template and the second vector parameter for the second vector template.
10. The system of claim 8 , wherein the operations further comprise:
identifying a set of elements in the first converted set of data and the second converted set of data without a corresponding data entry; and
assigning a null value to the set of elements in the first converted set of data and the second converted set of data, the null value associated with one or more null indicators in the first converted set of data and the second converted set of data.
11. The system of claim 8 , wherein the operations further comprise:
identifying a set of vector positions within the first set of vectors and the second set of vectors having a predetermined value;
removing the set of vector positions having the predetermined value to generate a set of modified vectors; and
storing the set of modified vectors in a database accessible by the computer implemented system.
12. The system of claim 8 , wherein generating the first set of vectors and the second set of vectors further comprises:
causing presentation of a set of transform elements indicating a transform strategy for converting the first set of data and the second set of data from the first format into the second format;
receiving selection of a transform element indicating a selected transform strategy associated with the transform element; and
generating the first set of vectors and the second set of vectors according to the selected transform strategy.
13. The system of claim 8 , wherein the operations further comprise:
accessing a first set of data provided by a first client device and a second set of data provided by a second client device.
14. The system of claim 8 , wherein the operations further comprise:
generating the integrated vector template based on the first set of data provided by the first client device and the second set of data provided by the second client device.
15. A non-transitory machine-readable storage medium including instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
distributing an integrated vector template to a first client device and a second client device, the integrated vector template being generated by the machine based on a first set of data provided by the first client device and a second set of data provided by the second client device, the first and second sets of data being configured in a first format;
sending, to the first client device, a first conversion instruction that causes the first client device to generate the first set of data in a second format based on the integrated vector template, the first set of data in the second format comprising a first set of vectors that have one or more vector values converted from one or more data entries in the first set of data;
receiving a first converted set of data from the first client device, the first converted set of data comprising the first set of data in the second format;
sending, to the second client device, a second conversion instruction that causes the second client device to generate the second set of data in the second format based on the integrated vector template, the second set of data in the second format comprising a second set of vectors that have one or more vector values converted from one or more data entries in the second set of data;
receiving a second converted set of data from the second client device, the second converted set of data comprising the second set of data in the second format;
mapping each data entry in the first and second converted sets of data to a vector value to generate a conversion map, the conversion map assisting in transforming data entries, from the first or second client devices, to vector values that have a format compatible for processing by at least one machine learning algorithm;
generating, by the server, at least one subsequent vector from an additional set of data using the conversion map;
processing, by the server, the at least one subsequent vector using the at least one machine learning algorithm; and
generating a set of probability values indicative of a probability of an event occurring with respect to an element included in the at least one subsequent vector, based on the processing of the at least one subsequent vector using the at least one machine learning algorithm.
16. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:
generating an integrated vector parameter for the integrated vector template by:
transmitting a first fit instruction to the first client device, the first fit instruction causing the first client device to determine a first vector parameter for a first vector template to represent the first set of data stored on the first client device within a first application;
transmitting a second fit instruction to the second client device, the second fit instruction causing the second client device to determine a second vector parameter for a second vector template to represent the second set of data stored on the second client device within a second application; and
receiving the first vector parameter for the first vector template and the second vector parameter for the second vector template.
17. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:
identifying a set of elements in the first converted set of data and the second converted set of data without a corresponding data entry; and
assigning a null value to the set of elements in the first converted set of data and the second converted set of data, the null value associated with one or more null indicators in the first converted set of data and the second converted set of data.
18. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:
identifying a set of vector positions within the first set of vectors and the second set of vectors having a predetermined value;
removing the set of vector positions having the predetermined value to generate a set of modified vectors; and
storing the set of modified vectors in a database accessible by the machine.