Data transformations for data sources of a federated application programming interface
Methods, systems, and devices for data processing are described. A federation management service may receive a user input indicating multiple data sources as federation inputs into federated application programming interface (API). The one or more data sources may have a first metadata input that defines a target schema for a computed field and a second metadata input that defines a transformation for computing a value for the computed field from a data source schema to the target schema for the computed field. A runtime service of the federation management service may compute the value of the computed field using the second metadata input. The federation management service may generate the federated API from the data sources, where the federated API includes the computed field.
1 . A method for data processing, comprising:
receiving, via a user interface of a federation management service, a user input indicating a plurality of data sources as federation inputs into a federated application programming interface (API), one or more data sources of the plurality of data sources being associated with a first metadata input that defines a target schema for a computed field and a second metadata input that defines a transformation for computing a value for the computed field from a data source schema of the one or more data sources to the target schema for the computed field, wherein the computed field is based at least in part on one or more fields of at least two data sources of the plurality of data sources, and wherein each data source of the plurality of data sources corresponds to one of a plurality of different data source schemas;
computing, by a runtime service of the federation management service, the value of the computed field using the transformation defined by the second metadata input, wherein the transformation comprises transforming data from the data source schema to the target schema for the computed field, wherein the runtime service generates the computed field by combining the one or more fields from the at least two data sources according to the target schema without modifying the data source schema of the at least two data sources, and wherein the at least two data sources corresponds to two or more different data source schemas of the plurality of different data source schemas; and
generating, using the federation management service, the federated API from the plurality of data sources, wherein the federated API comprises the computed field.
2 . The method of claim 1 , further comprising:
applying the transformation for computing the value for the computed field prior to generating the federated API, wherein the target schema is a version of the data source schema of the at least two data sources.
3 . The method of claim 2 , wherein computing the value of the computed field comprises:
configuring the runtime service based at least in part on storing the first metadata input, the second metadata input, or both;
fetching, by the runtime service from the data source schema, the first metadata input, the second metadata input, or both; and
applying, via the runtime service, the transformation to the data source schema of the at least two data sources, wherein the value of the computed field is based at least in part on applying the transformation.
4 . The method of claim 2 , further comprising:
generating the federated API based at least in part on unifying the version of the data source schema with the plurality of data sources for the federated API.
5 . The method of claim 1 , further comprising:
applying the transformation for computing the value for the computed field after generating the federated API, wherein the target schema corresponds to the federated API.
6 . The method of claim 5 , wherein computing the value of the computed field comprises:
configuring the runtime service based at least in part on storing the first metadata input, the second metadata input, or both;
fetching, by the runtime service from the plurality of data sources, the first metadata input, the second metadata input, or both; and
applying the transformation to the federated API via the runtime service, wherein the value of the computed field is based at least in part on applying the transformation.
7 . The method of claim 5 , further comprising:
computing, based at least in part on inputting one or more fields associated with the federated API, a plurality of values corresponding to the one or more fields, the plurality of values comprising the value of the computed field.
8 . The method of claim 1 , wherein computing the value of the computed field comprises:
configuring the runtime service based at least in part on storing the first metadata input, the second metadata input, or both;
fetching, by the runtime service from the one or more data sources, the first metadata input, the second metadata input, or both; and
applying the transformation to the data source schema of the at least two data sources, the federated API, or both via the runtime service, wherein the value of the computed field is based at least in part on applying the transformation.
9 . The method of claim 1 , further comprising:
determining, at a first time prior to generating the federated API, that the one or more data sources and the federated API comprise a first set of fields at the first time; and
determining, at a second time after generating the federated API, that the one or more data sources and the federated API comprise a second set of fields at the second time, wherein the second set of fields comprises the computed field.
10 . The method of claim 9 , wherein the transformation combines a plurality of fields from the first set of fields corresponding to the computed field based at least in part on the user input.
11 . An apparatus for data processing, comprising:
at least one processor;
at least one memory coupled with the at least one processor; and
instructions stored in the at least one memory and executable by the at least one processor to cause the apparatus to:
receive, via a user interface of a federation management service, a user input indicating a plurality of data sources as federation inputs into a federated application programming interface (API), one or more data sources of the plurality of data sources being associated with a first metadata input that defines a target schema for a computed field and a second metadata input that defines a transformation for computing a value for the computed field from a data source schema of the one or more data sources to the target schema for the computed field, wherein the computed field is based at least in part on one or more fields of at least two data sources of the plurality of data sources, and wherein each data source of the plurality of data sources corresponds to one of a plurality of different data source schemas;
compute, by a runtime service of the federation management service, the value of the computed field using the transformation defined by the second metadata input, wherein the transformation comprises transforming data from the data source schema to the target schema for the computed field, wherein the runtime service generates the computed field by combining the one or more fields from the at least two data sources according to the target schema without modifying the data source schema of the at least two data sources, and wherein the at least two data sources corresponds to two or more different data source schemas of the plurality of different data source schemas; and
generate, using the federation management service, the federated API from the plurality of data sources, wherein the federated API comprises the computed field.
12 . The apparatus of claim 11 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
apply the transformation for computing the value for the computed field prior to generating the federated API, wherein the target schema is a version of the data source schema of the at least two data sources.
13 . The apparatus of claim 12 , wherein the instructions to compute the value of the computed field are executable by the at least one processor to cause the apparatus to:
configure the runtime service based at least in part on storing the first metadata input, the second metadata input, or both;
fetch, by the runtime service from the data source schema, the first metadata input, the second metadata input, or both; and
apply, via the runtime service, the transformation to the data source schema of the at least two data sources, wherein the value of the computed field is based at least in part on applying the transformation.
14 . The apparatus of claim 12 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
generate the federated API based at least in part on unifying the version of the data source schema with the plurality of data sources for the federated API.
15 . The apparatus of claim 11 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
apply the transformation for computing the value for the computed field after generating the federated API, wherein the target schema corresponds to the federated API.
16 . The apparatus of claim 15 , wherein the instructions to compute the value of the computed field are executable by the at least one processor to cause the apparatus to:
configure the runtime service based at least in part on storing the first metadata input, the second metadata input, or both;
fetch, by the runtime service from the plurality of data sources, the first metadata input, the second metadata input, or both; and
apply the transformation to the federated API via the runtime service, wherein the value of the computed field is based at least in part on applying the transformation.
17 . The apparatus of claim 15 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
compute, based at least in part on inputting one or more fields associated with the federated API, a plurality of values corresponding to the one or more fields, the plurality of values comprising the value of the computed field.
18 . The apparatus of claim 11 , wherein the instructions to compute the value of the computed field are executable by the at least one processor to cause the apparatus to:
configure the runtime service based at least in part on storing the first metadata input, the second metadata input, or both;
fetch, by the runtime service from the one or more data sources, the first metadata input, the second metadata input, or both; and
apply the transformation to the data source schema of the at least two data sources, the federated API, or both via the runtime service, wherein the value of the computed field is based at least in part on applying the transformation.
19 . The apparatus of claim 11 , wherein the instructions are further executable by the at least one processor to cause the apparatus to:
determine, at a first time prior to generating the federated API, that the one or more data sources and the federated API comprise a first set of fields at the first time; and
determine, at a second time after generating the federated API, that the one or more data sources and the federated API comprise a second set of fields at the second time, wherein the second set of fields comprises the computed field.
20 . A non-transitory computer-readable medium storing code for data processing, the code comprising instructions executable by at least one processor to:
receive, via a user interface of a federation management service, a user input indicating a plurality of data sources as federation inputs into a federated application programming interface (API), one or more data sources of the plurality of data sources being associated with a first metadata input that defines a target schema for a computed field and a second metadata input that defines a transformation for computing a value for the computed field from a data source schema of the one or more data sources to the target schema for the computed field, wherein the computed field is based at least in part on one or more fields of at least two data sources of the plurality of data sources, and wherein each data source of the plurality of data sources corresponds to one of a plurality of different data source schemas;
compute, by a runtime service of the federation management service, the value of the computed field using the transformation defined by the second metadata input, wherein the transformation comprises transforming data from the data source schema to the target schema for the computed field, wherein the runtime service generates the computed field by combining the one or more fields from the at least two data sources according to the target schema without modifying the data source schema of the at least two data sources, and wherein the at least two data sources corresponds to two or more different data source schemas of the plurality of different data source schemas; and
generate, using the federation management service, the federated API from the plurality of data sources, wherein the federated API comprises the computed field.