Code execution and data processing pipeline
A method performed by one or more processors comprises displaying code, receiving user selection of a portion of code, determining one or more settable data items, generating a template, displaying the template, receiving a user input value for the settable data items by the template, and executing the code with each of the settable data items set to the received user input value. A data processing pipeline is configured to pass a data item to a first transformer to provide first transformed data, store the first transformed data in a temporary memory, write the first transformed data to the data storage system, and pass the transformed data from the temporary memory to a second transformer.
1. A method performed by one or more processors, the method comprising:
displaying a user interface depicting data transformation code, the data transformation code programmed to transform data items when executed;
receiving two or more user selections of code segments of the data transformation code;
determining and storing a reference to each of two or more settable items from in the data transformation code based on at least the two or more user selections; and
generating a data transformation template indicating the two or more settable items from the data transformation code, wherein the data transformation template is usable by a data transformation device to:
receive a user input value for each of the two or more settable items; and
execute, on one or more data items, the data transformation code with the two or more settable items set to the corresponding received user input value.
2. The method of claim 1 , wherein the data transformation template is further usable by the data transformation device to:
while executing the data transformation code, set at least one of the settable items by changing text characters in the data transformation code.
3. The method of claim 1 , wherein the data transformation template is further usable by the data transformation device to:
while executing the data transformation code, set at least one of the settable items by application programming interface calls.
4. A computing system of comprising:
a hardware computer processor;
a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising:
displaying a user interface depicting data transformation code, the data transformation code programmed to transform data items when executed;
receiving two or more user selectionselections of segments of the data transformation code;
determining and storing a reference to each of two or more settable items in the data transformation code based on at least the two or more user selections; and
generating a data transformation template indicating the two or more settable items from the data transformation code, wherein the data transformation template is usable by a data transformation device to:
receive a user input value for each of the two or more settable items; and
execute, on one or more data items, the data transformation code with the two or more settable items set to the corresponding received user input value.
5. The computing system of claim 4 , wherein the data transformation template is further usable by the data transformation device to:
while executing the data transformation code, set at least one of the settable items by changing text characters in the data transformation code.
6. The computing system of claim 4 , wherein the data transformation template is further usable by the data transformation device to:
while executing the data transformation code, set at least one of the settable items by application programming interface calls.
7. A non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by a hardware computer processor to cause a computing system to perform operations comprising:
displaying a user interface depicting data transformation code, the data transformation code programmed to transform data items when executed;
receiving two or more user selections of segments of the data transformation code;
determining and storing a reference to each of two or more settable items fromin the firstdata transformation code based on at least the two or more user selections; and
generating a data transformation template indicating the two or more settable items from the data transformation code, wherein the data transformation template is usable by a data transformation device to:
receive a user input value for each of the two or more settable items; and
execute, on one or more data items, the data transformation code with the two or more settable items set to the corresponding received user input value.
8. The non-transitory computer readable medium of claim 7 , wherein the data transformation template is further usable by the data transformation device to:
while executing the data transformation code, set at least one of the settable items by changing text characters in the data transformation code.
9. The non-transitory computer readable medium of claim 7 , wherein the data transformation template is further usable by the data transformation device to:
while executing the data transformation code, set at least one of the settable items by application programming interface calls.