Automated maintenance of external tables in database systems

1. A method performed by a database platform executing instructions on at least one hardware processor, the method comprising:

receiving an indication of a hierarchical structure of content that is stored in an external data storage platform that is separate from the database platform and to which the database platform does not have write access, the indication of the hierarchical structure listing a plurality of content-storage locations in which the content is stored among a plurality of levels of the hierarchical structure;

defining, by the database platform, an external-data-mapping table for the content, the external-data-mapping table being stored in the database platform;

generating metadata for the external-data-mapping table, the metadata comprising information about data stored in the external-data-mapping table, the metadata for the external-data-mapping table also being stored in the database platform, the metadata for the external-table organizing the external-data-mapping table into partitions according to the plurality of content-storage locations listed in the indication of the hierarchical structure;

receiving a notification that a modification has been made to a certain content-storage location within the plurality of content-storage locations in the hierarchical structure;

identifying, from the generated metadata, a certain partition of the external-data-mapping table as corresponding to the certain content-storage location; and

refreshing, in response to receiving the notification, the metadata for the certain partition of the external-data-mapping table to reflect the modification.

2. The method of claim 1 , wherein generating the metadata comprises one or more of:

defining cumulative table metadata for the external-data-mapping table;

defining a grouping expression property for a grouping of the partitions in the external-data-mapping table;

defining an expression property for at least one of the partitions of the external-data-mapping table;

defining partition statistics for at least one of the partitions of the external-data-mapping table; and

defining a column expression property for at least one column of at least one of the partitions of the external-data-mapping table.

3. The method of claim 1 , further comprising storing the metadata in a partition storage object on a shared storage platform associated with the database platform, wherein the shared storage platform is separate from the external data storage platform.

4. The method of claim 1 , further comprising one or more of:

refreshing the metadata at threshold time periods;

refreshing the metadata in response to a threshold number of modifications being made to the content; and

refreshing the metadata in response to a request to refresh the metadata.

5. The method of claim 1 , further comprising reading multiple different file formats in the content in the external data storage platform.

6. The method of claim 1 , wherein receiving the notification comprises one or more of:

querying the external data storage platform to determine whether any modifications have been made to the content;

receiving the notification from the external data storage platform; and

receiving the notification from a client associated with the content.

7. The method of claim 1 , further comprising generating change tracking metadata for the certain partition, the change tracking metadata indicating how the certain content-storage location was modified and when the certain content-storage location was modified.

8. The method of claim 1 , the modification comprising one or more of an addition of a file, a deletion of a file, and an update to a file in the certain content-storage location.

9. The method of claim 1 , wherein:

the content-storage locations in the plurality of content-storage locations comprise folders and subfolders; and

the certain content-storage location comprises a certain subfolder.

10. One or more non-transitory computer readable storage media storing instructions that, when executed by one or more processors of a database platform, cause the one or more processors to perform operations comprising:

receiving an indication of a hierarchical structure of content that is stored in an external data storage platform that is separate from the database platform and to which the database platform does not have write access, the indication of the hierarchical structure listing a plurality of content-storage locations in which the content is stored among a plurality of levels of the hierarchical structure;

defining, by the database platform, an external-data-mapping table for the content, the external-data-mapping table being stored in the database platform;

generating metadata for the external-data-mapping table, the metadata comprising information about data stored in the external-data-mapping table, the metadata for the external-data-mapping table also being stored in the database platform, the metadata for the external-table organizing the external-data-mapping table into partitions according to the plurality of content-storage locations listed in the indication of the hierarchical structure;

receiving a notification that a modification has been made to a certain content-storage location within the plurality of content-storage locations in the hierarchical structure;

identifying, from the generated metadata, a certain partition of the external-data-mapping table as corresponding to the certain content-storage location; and

refreshing, in response to receiving the notification, the metadata for the certain partition of the external-data-mapping table to reflect the modification.

11. The non-transitory computer readable storage media of claim 10 , wherein generating the metadata comprises one or more of:

defining cumulative table metadata for the external-data-mapping table;

defining a grouping expression property for a grouping of the partitions in the external-data-mapping table;

defining an expression property for at least one of the partitions of the external-data-mapping table;

defining partition statistics for at least one of the partitions of the external-data-mapping table; and

defining a column expression property for at least one column of at least one of the partitions of the external-data-mapping table.

12. The non-transitory computer readable storage media of claim 10 , wherein the operations further comprise storing the metadata in a partition storage object on a shared storage platform associated with the database platform, wherein the shared storage platform is separate from the external data storage platform.

13. The non-transitory computer readable storage media of claim 10 , further comprising one or more of:

refreshing the metadata at threshold time periods;

refreshing the metadata in response to a threshold number of modifications being made to the content; and

refreshing the metadata in response to a request to refresh the metadata.

14. The non-transitory computer readable storage media of claim 10 , the operations further comprising reading multiple different file formats in the content in the external data storage platform.

15. The non-transitory computer readable storage media of claim 10 , wherein receiving the notification comprises one or more of:

querying the external data storage platform to determine whether any modifications have been made to the content;

receiving the notification from the external data storage platform; and

receiving the notification from a client associated with the content.

16. The non-transitory computer readable storage media of claim 10 , the operations further comprising generating change tracking metadata for the certain partition, the change tracking metadata indicating how the certain content-storage location was modified and when the certain content-storage location was modified.

17. The non-transitory computer readable storage media of claim 10 , the modification comprising one or more of an addition of a file, a deletion of a file, and an update to a file in the certain content-storage location.

18. The non-transitory computer readable storage media of claim 10 , wherein:

the content-storage locations in the plurality of content-storage locations comprise folders and subfolders; and

the certain content-storage location comprises a certain subfolder.

19. A database platform comprising:

at least one processor; and

one or more non-transitory computer readable storage media containing instructions executable by the at least one processor for causing the at least one processor to perform operations comprising:

receiving an indication of a hierarchical structure of content that is stored in an external data storage platform that is separate from the database platform and to which the database platform does not have write access, the indication of the hierarchical structure listing a plurality of content-storage locations in which the content is stored among a plurality of levels of the hierarchical structure;

defining, by the database platform, an external-data-mapping table for the content, the external-data-mapping table being stored in the database platform;

generating metadata for the external-data-mapping table, the metadata comprising information about data stored in the external-data-mapping table, the metadata for the external-data-mapping table also being stored in the database platform, the metadata for the external-table organizing the external-data-mapping table into partitions according to the plurality of content-storage locations listed in the indication of the hierarchical structure;

receiving a notification that a modification has been made to a certain content-storage location within the plurality of content-storage locations in the hierarchical structure;

identifying, from the generated metadata, a certain partition of the external-data-mapping table as corresponding to the certain content-storage location; and

refreshing, in response to receiving the notification, the metadata for the certain partition of the external-data-mapping table to reflect the modification.

20. The database platform of claim 19 , wherein generating the metadata comprises one or more of:

defining cumulative table metadata for the external-data-mapping table;

defining a grouping expression property for a grouping of the partitions in the external-data-mapping table;

defining an expression property for at least one of the partitions of the external-data-mapping table;

defining partition statistics for at least one of the partitions of the external-data-mapping table; and

defining a column expression property for at least one column of at least one of the partitions of the external-data-mapping table.

21. The database platform of claim 19 , further comprising storing the metadata in a partition storage object on a shared storage platform associated with the database platform, wherein the shared storage platform is separate from the external data storage platform.

22. The database platform of claim 19 , further comprising one or more of:

refreshing the metadata at threshold time periods;

refreshing the metadata in response to a threshold number of modifications being made to the content; and

refreshing the metadata in response to a request to refresh the metadata.

23. The database platform of claim 19 , wherein receiving the notification comprises one or more of:

querying the external data storage platform to determine whether any modifications have been made to the content;

receiving the notification from the external data storage platform; and

receiving the notification from a client associated with the content.

24. The database platform of claim 19 , the operations further comprising generating change tracking metadata for the certain partition, the change tracking metadata indicating how the certain content-storage location was modified and when the certain content-storage location was modified.

25. The database platform of claim 19 , the modification comprising one or more of an addition of a file, a deletion of a file, and an update to a file in the certain content-storage location.

26. The database platform of claim 19 , wherein:

the content-storage locations in the plurality of content-storage locations comprise folders and subfolders; and

the certain content-storage location comprises a certain subfolder.