Expression data structure dataset processing
View Patent ↗Methods, computer program products, and systems are presented. The method, computer program products, and systems can include, for instance: initiating evaluating of a tree data structure expression having nodes and edges between the nodes, wherein a plurality of nodes of the tree data structure expression express functions, and wherein a hierarchy of the tree data structure expression defines an order of execution of functions expressed by nodes of the plurality of nodes; and in response to the initiating, reading, processing and writing tile data of a dataset according to node expressed functions of the tree data structure expression.
1. A method comprising:
initiating evaluating of a tree data structure expression having nodes and edges between the nodes, wherein a plurality of nodes of the tree data structure expression express functions, and wherein a hierarchy of the tree data structure expression defines an order of execution of functions expressed by nodes of the plurality of nodes; and
in response to the initiating, reading, processing and writing tile data of a dataset according to node expressed functions of the tree data structure expression, wherein the method includes receiving by a computing node the tree data structure expression from a deployment system, and wherein the reading tile data includes reading data of the tile data from a data source, wherein the computing node, the deployment system, and the data source are in network communication with one another, and external to one another, wherein the receiving by a computing node the tree data structure expression includes receiving associated software code to automatically run the tree data structure expression on the computing node so that data of the tile data is automatically read, processed and written in response to the receiving by a computing node the tree data structure expression from the deployment system.
2. The method of claim 1 , wherein the dataset is a geospatial dataset of a geographic information system (GIS), wherein one or more nodes of the plurality of nodes of the tree data structure expression express Map Algebra commands, the Map Algebra commands being scripting language commands.
3. A method comprising:
initiating evaluating of a tree data structure expression having nodes and edges between the nodes, wherein a plurality of nodes of the tree data structure expression express functions, and wherein a hierarchy of the tree data structure expression defines an order of execution of functions expressed by nodes of the plurality of nodes; and
in response to the initiating, reading, processing and writing tile data of a dataset according to node expressed functions of the tree data structure expression, wherein the method includes deploying the tree data structure expression on a computing node external to a data source that is in network communication with the computing node, and running the tree data structure expression on the computing node external to a data source responsively to the deploying the tree data structure expression to perform the initiating so that data of the tile data is read, processed and written in response to the deploying the tree data structure expression and wherein the reading tile data includes reading data of the tile data from the data source external to the computing node.
4. The method of claim 3 , wherein the dataset is a geospatial dataset of a geographic information system (GIS).
5. The method of claim 3 , wherein nodes of the plurality of nodes of the tree data structure expression express Map Algebra commands, wherein the Map Algebra commands are scripting language commands mapped to Map Algebra commands of a compiled code library, the compiled code library having a plurality of compiled code Map Algebra commands.
6. The method of claim 3 , wherein a node of the plurality of nodes of the tree data structure expression expresses a Map Algebra command, wherein the Map Algebra command is a scripting language command linked to a Map Algebra command of a compiled code library, wherein the tree data structure expression is encoded to processes an input tile so that an output tile output using the Map Algebra command produces an output tile than inherits an address of the input tile.
7. The method of claim 3 , wherein a highest order node of the tree data structure expression specifies a write command and wherein a lowest order of the tree data structure expression specifies a read command.
8. The method of claim 3 , wherein the dataset is formatted as a tile dataset, and wherein the initiating, reading, processing and writing tile data of a dataset includes sequentially reading, processing and writing a succession of tiles of the dataset ordered by tile addresses of the dataset.
9. The method of claim 3 , wherein the dataset includes streamed tiles from a first database, and wherein a node of a plurality of nodes of the tree data structure expression expresses an ordering command, wherein the ordering command orders tile data from the dataset and a second dataset, and wherein the second dataset includes streamed tiles from a second database, the second database being external to the first database.
10. The method of claim 3 , wherein the dataset includes streamed tiles from a first database, and wherein the reading in response to the initiating evaluating includes reading tile data from the dataset and reading tile data from a second dataset, and wherein the second dataset includes streamed tiles from a second database, the second database being external to the first database.
11. The method of claim 3 , wherein the dataset includes streamed tiles from a first database, and wherein the reading in response to the initiating evaluating includes reading tile data from the dataset and reading tile data from a second dataset, and wherein the second dataset includes streamed tiles from a second database, the second database being external to the first database, and wherein a node of the tree data structure expression expresses a function which when executed results in duplicate data of a common location between the dataset and the second dataset being merged.
12. The method of claim 3 , wherein writing tile data in response to the initiating evaluating includes iteratively writing a succession of processed tiles processed using the tree data structure expression to an external database without writing of the succession of processed tiles to a local persistent storage location.
13. The method of claim 3 , wherein a node of the tree data structure expression specifies a read command which when executed results in input tile data from a database external to a computing node performing the processing being read.
14. The method of claim 3 , wherein a node of the tree data structure expression specifies a write command which when executed results in output tile data being written to an external database external to the computing node.
15. The method of claim 3 , wherein a first node of the tree data structure expression specifies a read command which when executed results in input tile data from a database external to a computing node performing the processing being read, wherein a second node of the tree data structure expression specifies a write command which when executed results in output tile data being written to an external database external to the computing node.
16. The method of claim 3 , wherein a first node of the tree data structure expression specifies a read command which when executed results in input tile data from a database external to a computing node performing the processing being read, wherein a second node of the tree data structure expression specifies a write command which when executed results in output tile data being written to a local database local to the computing node performing the processing.
17. The method of claim 3 , wherein one or more nodes of the plurality of nodes of the tree data structure expression express Map Algebra commands.
18. The method of claim 3 , wherein one or more nodes of the plurality of nodes of the tree data structure expression express Map Algebra commands, the Map Algebra commands being scripting language commands.
19. The method of claim 3 , wherein the computing node external to the data source that is in network communication with the computing node is a computing node is selected from the group consisting of a computing node external to a deployment system configured for deploying the tree data structure expression and a computing node co-located with the deployment system.
20. A method comprising:
initiating evaluating of a tree data structure expression having nodes and edges between the nodes, wherein a plurality of nodes of the tree data structure expression express functions, and wherein a hierarchy of the tree data structure expression defines an order of execution of functions expressed by nodes of the plurality of nodes; and
in response to the initiating, reading, processing and writing tile data of a dataset according to node expressed functions of the tree data structure expression, wherein the dataset includes streamed tiles from a first database, and wherein the reading in response to the initiating evaluating includes reading tile data from the dataset and reading tile data from a second dataset, and wherein the second dataset includes streamed tiles from a second database, the second database being external to the first database.