Query processing of stream objects using stream expansion
Provided herein are systems and methods for a stream object configuration, including query processing of stream objects using stream expansion. For example, a method includes decoding a query to obtain a first data processing operation and a first stream object. The first stream object is associated with a view on a base table. A first stream expansion on the first stream object is performed. The first stream expansion is based on generating a second stream object on the base table. A second stream expansion of the second stream object is performed. The second stream expansion is based on replacing the second stream object with at least a second data processing operation. The query is executed based on completing the first data processing operation and the at least a second data processing operation.
1. A system comprising:
at least one hardware processor; and
at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:
decoding a query to obtain a first query plan with a first data processing operation and a first stream object, the first stream object in the first query plan associated with a view on a base table;
performing a first stream expansion on the first stream object to generate a second query plan, the first stream expansion based on generating a second stream object on the base table within the second query plan;
performing a second stream expansion of the second stream object within the second query plan, the second stream expansion based on replacing the second stream object with at least a second data processing operation; and
executing the query based on completing the first data processing operation and the at least a second data processing operation.
2. The system of claim 1 , wherein the instructions further cause the at least one hardware processor to perform operations comprising:
determining at least a third data processing operation associated with the first stream object, the first stream object including a plurality of versions of the view on the base table and a corresponding plurality of time stamps of times the plurality of versions of the view are generated by executing the at least third data processing operation.
3. The system of claim 2 , wherein the instructions to perform the first stream expansion further cause the at least one hardware processor to perform operations comprising:
determining the plurality of versions of the view in the first stream object are based on a table scan operation on the base table.
4. The system of claim 3 , wherein the instructions to perform the first stream expansion further cause the at least one hardware processor to perform operations comprising:
replacing the first stream object with at least a filter operation and the table scan operation on the base table.
5. The system of claim 4 , wherein the instructions to perform the first stream expansion further cause the at least one hardware processor to perform operations comprising:
generating a revised query plan of the first stream object from the first stream expansion, the revised query plan including the first data processing operation, a project operation, the filter operation, and the table scan operation on the base table.
6. The system of claim 5 , wherein the instructions further cause the at least one hardware processor to perform operations comprising:
generating a third query plan based on the revised query plan and the second stream expansion; and
executing the query based on completing operations in the third query plan.
7. The system of claim 1 , wherein the instructions to perform the second stream expansion further cause the at least one hardware processor to perform operations comprising:
replacing the second stream object with a join operation.
8. The system of claim 7 , wherein the instructions further cause the at least one hardware processor to perform operations comprising:
performing a first table scan operation of the base table to obtain added data, the added data being added to the base table as a result of the executing the at least second data processing operation.
9. The system of claim 8 , wherein the instructions further cause the at least one hardware processor to perform operations comprising:
performing a second table scan operation of the base table to obtain deleted data, the deleted data being deleted from the base table as a result of the executing the at least second data processing operation.
10. The system of claim 9 , wherein the instructions further cause the at least one hardware processor to perform operations comprising:
applying the join operation to the added data and the deleted data; and
outputting a result of the applying the join operation.
11. A method comprising:
decoding, by at least one hardware processor, a query to obtain a first query plan with a first data processing operation and a first stream object, the first stream object in the first query plan associated with a view on a base table;
performing a first stream expansion on the first stream object to generate a second query plan, the first stream expansion based on generating a second stream object on the base table within the second query plan;
performing a second stream expansion of the second stream object within the second query plan, the second stream expansion based on replacing the second stream object with at least a second data processing operation; and
executing the query based on completing the first data processing operation and the at least a second data processing operation.
12. The method of claim 11 , further comprising:
determining at least a third data processing operation associated with the first stream object, the first stream object including a plurality of versions of the view on the base table and a corresponding plurality of time stamps of times the plurality of versions of the view are generated by executing the at least third data processing operation.
13. The method of claim 12 , wherein performing the first stream expansion further comprises:
determining the plurality of versions of the view in the first stream object are based on a table scan operation on the base table.
14. The method of claim 13 , wherein performing the first stream expansion further comprises:
replacing the first stream object with at least a filter operation and the table scan operation on the base table.
15. The method of claim 14 , wherein performing the first stream expansion further comprises:
generating a revised query plan of the first stream object from based on the first stream expansion, the revised query plan including the first data processing operation, a project operation, the filter operation, and the table scan operation on the base table.
16. The method of claim 15 , further comprising:
generating a third query plan based on the revised query plan and the second stream expansion; and
executing the query based on completing operations in the third query plan.
17. The method of claim 11 , wherein performing the second stream expansion further comprises:
replacing the second stream object with a join operation.
18. The method of claim 17 , further comprising:
performing a first table scan operation of the base table to obtain added data, the added data being added to the base table as a result of the executing the at least second data processing operation.
19. The method of claim 18 , further comprising:
performing a second table scan operation of the base table to obtain deleted data, the deleted data being deleted from the base table as a result of the executing the at least second data processing operation.
20. The method of claim 19 , further comprising:
applying the join operation to the added data and the deleted data; and
outputting a result of the applying the join operation.
21. A computer-storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:
decoding a query to obtain a first query plan with a first data processing operation and a first stream object, the first stream object in the first query plan associated with a view on a base table;
performing a first stream expansion on the first stream object to generate a second query plan, the first stream expansion based on generating a second stream object on the base table within the second query plan;
performing a second stream expansion of the second stream object within the second query plan, the second stream expansion based on replacing the second stream object with at least a second data processing operation; and
executing the query based on completing the first data processing operation and the at least a second data processing operation.
22. The computer-storage medium of claim 21 , the operations further comprising:
determining at least a third data processing operation associated with the first stream object, the first stream object including a plurality of versions of the view on the base table and a corresponding plurality of time stamps of times the plurality of versions of the view are generated by executing the at least third data processing operation.
23. The computer-storage medium of claim 22 , wherein the operations for performing the first stream expansion further comprise:
determining the plurality of versions of the view in the first stream object are based on a table scan operation on the base table.
24. The computer-storage medium of claim 23 , wherein the operations for performing the first stream expansion further comprise:
replacing the first stream object with at least a filter operation and the table scan operation on the base table.
25. The computer-storage medium of claim 24 , wherein the operations for performing the first stream expansion further comprise:
generating a revised query plan of the first stream object from the first stream expansion, the revised query plan including the first data processing operation, a project operation, the filter operation, and the table scan operation on the base table.
26. The computer-storage medium of claim 25 , the operations further comprising:
generating a third query plan based on the revised query plan and the second stream expansion; and
executing the query based on completing operations in the third query plan.
27. The computer-storage medium of claim 21 , wherein the operations for performing the second stream expansion further comprise:
replacing the second stream object with a join operation.
28. The computer-storage medium of claim 27 , the operations further comprising:
performing a first table scan operation of the base table to obtain added data, the added data being added to the base table as a result of the executing the at least second data processing operation.
29. The computer-storage medium of claim 28 , the operations further comprising:
performing a second table scan operation of the base table to obtain deleted data, the deleted data being deleted from the base table as a result of the executing the at least second data processing operation.
30. The computer-storage medium of claim 29 , the operations further comprising:
applying the join operation to the added data and the deleted data; and
outputting a result of the applying the join operation.