Distributed data processing in multi-tenant environments

1. A method for data processing within a multi-tenant distributed data system, comprising:

receiving, from a tenant user device, tenant-specific processing instructions for a requested set of data, the tenant-specific processing instructions comprising input information, transformation information, and output information, wherein the input information comprises one or more custom fields specific to the tenant user device and a data structure for the requested set of data, and wherein the transformation information is from a pre-defined set of possible transformations;

identifying, from the multi-tenant distributed data system, data that the tenant has access to based at least in part on a tenant identity (ID) associated with the tenant user device, wherein the data that the tenant has access to is used to generate the requested set of data;

receiving, from a provider user device, code that is executable to process the requested set of data using the tenant-specific processing instructions;

transmitting, for execution by a plurality of worker nodes on the data that the tenant has access to within the multi-tenant distributed data system, the tenant-specific processing instructions and the code received from the provider user device to generate the requested set of data; and

transmitting, to the tenant user device, the generated requested set of data.

2. The method of claim 1 , further comprising:

generating, in accordance with the tenant-specific processing instructions and the code received from the provider user device, the requested set of data by aggregating two or more datasets from the data that the tenant has access to, wherein the generated requested set of data comprises fields from the aggregate two or more datasets and the one or more custom fields specific to the tenant user device.

3. The method of claim 2 , wherein the two or more data sets are from respective data stores within the multi-tenant distributed data system.

4. The method of claim 1 , further comprising:

transforming two or more datasets from the data that the tenant has access to using a set of transformations from the pre-defined set of possible transformations in accordance with the transformation information, wherein the generated requested set of data comprises the transformed two or more datasets.

5. The method of claim 1 , wherein identifying data from the multi-tenant distributed data system that the tenant has access to comprises:

pre-filtering data that the tenant does not have access to based at least in part on the tenant ID.

6. The method of claim 1 , wherein identifying data from the multi-tenant distributed data system that the tenant has access to comprises:

identifying datasets within the data that the tenant has access to based at least in part on the data structure, wherein the data structure comprises a format of the datasets, or a size of the datasets, or a combination thereof.

7. The method of claim 1 , further comprising:

monitoring execution of the tenant-specific processing instructions and the code received from the provider user device execution by the plurality of worker nodes.

8. The method of claim 1 , wherein the input information describes one or more input data tables and the output information describes one or more results data tables.

9. The method of claim 1 , wherein the tenant-specific processing instructions comprise a declarative object formatted in accordance with a programming language.

10. The method of claim 1 , wherein the code received from the provider user device comprises executable code written to perform data processing and using a programming language.

11. An apparatus for data processing within a multi-tenant distributed data system, comprising:

a processor,

memory in electronic communication with the processor; and

instructions stored in the memory and executable by the processor to cause the apparatus to:

receive, from a tenant user device, tenant-specific processing instructions for a requested set of data, the tenant-specific processing instructions comprising input information, transformation information, and output information, wherein the input information comprises one or more custom fields specific to the tenant user device and a data structure for the requested set of data, and wherein the transformation information is from a pre-defined set of possible transformations;

identify, from the multi-tenant distributed data system, data that the tenant has access to based at least in part on a tenant identity (ID) associated with the tenant user device, wherein the data that the tenant has access to is used to generate the requested set of data;

receive, from a provider user device, code that is executable to process the requested set of data using the tenant-specific processing instructions;

transmit, for execution by a plurality of worker nodes on the data that the tenant has access to within the multi-tenant distributed data system, the tenant-specific processing instructions and the code received from the provider user device to generate the requested set of data; and

transmit, to the tenant user device, the generated requested set of data.

12. The apparatus of claim 11 , wherein the instructions are further executable by the processor to cause the apparatus to:

generate, in accordance with the tenant-specific processing instructions and the code received from the provider user device, the requested set of data by aggregating two or more datasets from the data that the tenant has access to, wherein the generated requested set of data comprises fields from the aggregate two or more datasets and the one or more custom fields specific to the tenant user device.

13. The apparatus of claim 12 , wherein the two or more data sets are from respective data stores within the multi-tenant distributed data system.

14. The apparatus of claim 11 , wherein the instructions are further executable by the processor to cause the apparatus to:

transform two or more datasets from the data that the tenant has access to using a set of transformations from the pre-defined set of possible transformations in accordance with the transformation information, wherein the generated requested set of data comprises the transformed two or more datasets.

15. The apparatus of claim 11 , wherein the instructions to identify data from the multi-tenant distributed data system that the tenant has access to are executable by the processor to cause the apparatus to:

pre-filter data that the tenant does not have access to based at least in part on the tenant ID.

16. The apparatus of claim 11 , wherein the instructions to identify data from the multi-tenant distributed data system that the tenant has access to are executable by the processor to cause the apparatus to:

identify datasets within the data that the tenant has access to based at least in part on the data structure, wherein the data structure comprises a format of the datasets, or a size of the datasets, or a combination thereof.

17. The apparatus of claim 11 , wherein the instructions are further executable by the processor to cause the apparatus to:

monitor execution of the tenant-specific processing instructions and the code received from the provider user device execution by the plurality of worker nodes.

18. The apparatus of claim 11 , wherein the input information describes one or more input data tables and the output information describes one or more results data tables.

19. The apparatus of claim 11 , wherein the tenant-specific processing instructions comprise a declarative object formatted in accordance with a programming language.

20. A non-transitory computer-readable medium storing code for data processing within a multi-tenant distributed data system, the code comprising instructions executable by a processor to:

receive, from a tenant user device, tenant-specific processing instructions for a requested set of data, the tenant-specific processing instructions comprising input information, transformation information, and output information, wherein the input information comprises one or more custom fields specific to the tenant user device and a data structure for the requested set of data, and wherein the transformation information is from a pre-defined set of possible transformations;

identify, from the multi-tenant distributed data system, data that the tenant has access to based at least in part on a tenant identity (ID) associated with the tenant user device, wherein the data that the tenant has access to is used to generate the requested set of data;

receive, from a provider user device, code that is executable to process the requested set of data using the tenant-specific processing instructions;

transmit, for execution by a plurality of worker nodes on the data that the tenant has access to within the multi-tenant distributed data system, the tenant-specific processing instructions and the code received from the provider user device to generate the requested set of data; and

transmit, to the tenant user device, the generated requested set of data.