Managing corroborated data using pre-computed partial corroboration results
Methods and systems for managing data used to provide computer-implemented services are disclosed. To manage the data, a request for corroborated data may be obtained from a data consumer indicating a desired information content and a threshold level of trust for the corroborated data. Based on the request, the corroborated data may be obtained from a corroborated data database. The corroborated data may be based on at least a pre-computed partial corroboration result and a new partial corroboration result having been computed dynamically at a time of corroboration of the corroborated data. At least a portion of the corroborated data may be provided to the data consumer to facilitate provisioning of the computer-implemented services.
1 . A method for managing data used to provide computer-implemented services, the method being performed by a computing device configured as a data manager and comprising:
obtaining, from a data processing system associated with a data consumer, a request for corroborated data, the request indicating a desired information content and a threshold level of trust for the corroborated data; and
in response to receiving the request from the data processing system:
obtaining, based on the request, the corroborated data from a corroborated data database, the corroborated data:
having the desired information content and being ascribed a level of trust that meets the level of trust threshold based on a level of trust schema,
being obtained by a first data source and corroborated using at least one other data source, the at least one other data source being adapted to measure a similar information content to the desired information content which the first data source is adapted to measure, and
being based on at least a pre-computed partial corroboration result and a new partial corroboration result, the new partial corroboration result having been computed dynamically at a time of corroboration of the corroborated data; and
providing at least a portion of the corroborated data to the data consumer to facilitate provisioning of the computer-implemented services,
wherein, prior to obtaining the request for the corroborated data, the method further comprises generating the corroborated data, that is stored in a corroborated database, by at least:
making a first determination that the pre-computed partial corroboration result is relevant for corroborating the corroborated data;
making a second determination, in response to the first determination, that the pre-computed partial corroboration result exits in a pre-computed partial corroboration result database; and
using the pre-computed partial corroboration result instead of generating additional ones of the new partial corroboration result to generate the corroborated data.
2 . The method of claim 1 , wherein the new partial corroboration result was generated using a first portion of data generated by a second data source of the at least one other data source based on a first corroboration schema and the pre-computed partial corroboration result was generated using a second portion of data generated by a third data source of the at least one other data source based on a second corroboration scheme.
3 . The method of claim 2 , wherein the first corroboration schema indicates that the pre-computed partial corroboration result and the new partial corroboration result were used to determine whether the corroborated data was deemed corroborated.
4 . The method of claim 2 , wherein the pre-computed partial corroboration result is stored in the pre-computed partial corroboration result database by the data manager or by another entity different from the data manager.
5 . The method of claim 4 , wherein the corroborated data from the corroborated data database is based on algorithms that preferentially utilize the pre-computed partial corroboration result database over dynamic calculation of partial corroboration results required for completion of the algorithms.
6 . The method of claim 4 , wherein the pre-computed partial corroboration result is stored in the pre-computed partial corroboration result database as a cryptographically verifiable data structure.
7 . The method of claim 1 , wherein generating the corroborated data that is stored in the corroborated database further comprises:
obtaining first data to be corroborated from the first data source, the first data having a first information content;
obtaining second data from a second data source, the second data having a second information content and the second data source attempting to measure a similar information content as the first information content;
performing a corroboration process to determine whether the first information content substantially matches the second information content;
in a first instance of the performing in which the first information content substantially matches the second information content:
concluding that the second data source corroborates the first data to obtain the corroborated data;
assigning, based on at least the second data and the level of trust schema, the level of trust for the corroborated data; and
storing the corroborated data in the corroborated data database; and
in a second instance of the performing in which the first information content does not substantially match the second information content:
concluding that the second data source does not corroborate the first data.
8 . The method of claim 7 , wherein performing the corroboration process comprises:
making an identification, based on a first corroboration schema, that at least a first partial corroboration result and a second partial corroboration result are to be used to determine whether the second data source corroborates the first data;
making a determination, based on the identification, regarding whether the first partial corroboration result and/or the second partial corroboration result are stored in the pre-computed partial corroboration result database;
in an instance of the determination in which only the first partial corroboration result is stored in the pre-computed partial corroboration result database while the second partial corroboration results are not stored in the pre-computed partial corroboration result database:
retrieving the first partial corroboration result from the pre-computed partial corroboration result database to be used as the pre-computed partial corroboration result;
generating the second partial corroboration result as the new partial corroboration result; and
obtaining, based at least on the pre-computed partial corroboration result and the new partial corroboration result, a final corroboration result for the first data and the final corroboration result indicating whether the first information content substantially matches the second information content.
9 . The method of claim 1 , wherein the level of trust schema comprises a rule set for assigning levels of trust to data based on degrees of corroboration of the data.
10 . The method of claim 9 , wherein the degrees of corroboration are based on a quantity of aspects of the data which are corroborated.
11 . The method of claim 10 , wherein the aspects comprise at least one type of aspect selected from a list of types of aspects consisting of:
a portion of a third information content of the data;
a timestamp for the data; and
a geographic location where the data was collected.
12 . The method of claim 9 , wherein the degrees of corroboration are based on a quantity of data sources which corroborate the data, and the rule set ascribes higher levels of trust with higher degrees of corroboration.
13 . The method of claim 1 , wherein the corroborated data is deemed to be corroborated based on the at least one other data source:
having provided an information content of data generated by the at least one other data source substantially matching the desired information content; and
not supplying synthetic data.
14 . The method of claim 1 , wherein the corroborated data is not synthetic data.
15 . The method of claim 1 , wherein obtaining the corroborated data from the corroborated data database comprises:
performing a lookup in the corroborated data database using the desired information content as a key to identify at least one entry; and
selecting, from one of the at least one entry, the corroborated data which both has the desired information content and meets the threshold level of trust.
16 . The method of claim 1 , wherein the corroborated data database comprises an immutable ledger comprising entries that are cryptographically verifiable.
17 . A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor of a computing device configured as a data manager, cause the processor to perform operations for managing data used to provide computer-implemented services, the operations comprising:
obtaining, from a data processing system associated with a data consumer, a request for corroborated data, the request indicating a desired information content and a threshold level of trust for the corroborated data; and
in response to receiving the request from the data processing system:
obtaining, based on the request, the corroborated data from a corroborated data database, the corroborated data:
having the desired information content and being ascribed a level of trust that meets the level of trust threshold based on a level of trust schema,
being obtained by a first data source and corroborated using at least one other data source, the at least one other data source being adapted to measure a similar information content to the desired information content which the first data source is adapted to measure, and
being based on at least a pre-computed partial corroboration result and a new partial corroboration result, the new partial corroboration result having been computed dynamically at a time of corroboration of the corroborated data; and
providing at least a portion of the corroborated data to the data consumer to facilitate provisioning of the computer-implemented services,
wherein, prior to obtaining the request for the corroborated data, the operations further comprise generating the corroborated data, that is stored in a corroborated database, by at least:
making a first determination that the pre-computed partial corroboration result is relevant for corroborating the corroborated data;
making a second determination, in response to the first determination, that the pre-computed partial corroboration result exits in a pre-computed partial corroboration result database; and
using the pre-computed partial corroboration result instead of generating additional ones of the new partial corroboration result to generate the corroborated data.
18 . The non-transitory machine-readable medium of claim 17 , wherein the new partial corroboration result was generated using a first portion of data generated by a second data source of the at least one other data source based on a first corroboration schema and the pre-computed partial corroboration result was generated using a second portion of data generated by a third data source of the at least one other data source based on a second corroboration schema.
19 . A first data processing system configured as a data manager, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing data used to provide computer-implemented services, the operations comprising:
obtaining, from a second data processing system associated with a data consumer, a request for corroborated data, the request indicating a desired information content and a threshold level of trust for the corroborated data; and
in response to receiving the request from the second data processing system:
obtaining, based on the request, the corroborated data from a corroborated data database, the corroborated data:
having the desired information content and being ascribed a level of trust that meets the level of trust threshold based on a level of trust schema,
being obtained by a first data source and corroborated using at least one other data source, the at least one other data source being adapted to measure a similar information content to the desired information content which the first data source is adapted to measure, and
being based on at least a pre-computed partial corroboration result and a new partial corroboration result, the new partial corroboration result having been computed dynamically at a time of corroboration of the corroborated data; and
providing at least a portion of the corroborated data to the data consumer to facilitate provisioning of the computer-implemented services,
wherein, prior to obtaining the request for the corroborated data, the operations further comprise generating the corroborated data, that is stored in a corroborated database, by at least:
making a first determination that the pre-computed partial corroboration result is relevant for corroborating the corroborated data;
making a second determination, in response to the first determination, that the pre-computed partial corroboration result exits in a pre-computed partial corroboration result database; and
using the pre-computed partial corroboration result instead of generating additional ones of the new partial corroboration result to generate the corroborated data.
20 . The first data processing system of claim 19 , wherein the new partial corroboration result was generated using a first portion of data generated by a second data source of the at least one other data source based on a first corroboration schema and the pre-computed partial corroboration result was generated using a second portion of data generated by a third data source of the at least one other data source based on a second corroboration schema.