Techniques for data retention

1. A method of managing data in a multitenant environment having a relational database and a non-relational database, the method comprising:

receiving, with one or more server computing systems that provide the multitenant environment, a set of one or more policies for field history data retention corresponding to data stored in a history table in the relational database environment, wherein the policies for data retention are defined on a tenant-by-tenant basis within the multitenant environment, wherein the one or more policies for data retention define what data is to be copied from the relational database to the non-relational database, how to copy the data, how long to retain the data in the relational database after the copy and how long to retain the data in the non-relational database after the copy;

deploying, with the one or more computing systems that provide the multitenant environment, the policies to data in the relational database on a tenant-by-tenant basis within the multitenant environment;

identifying, with the one or more computing systems that provide the multitenant environment and according to the policies, data to be copied from the relational database to the non-relational database, wherein the data to be copied comprises at least field history information indicating changes made to one or more fields in the relational database and the non-relational database provides a field history archive;

storing the data to be copied in the non-relational database while maintaining tenant isolation so that data belonging to the respective tenants is not accessible by other tenants when stored in the non-relational database utilizing a two-stage archival operation where each of the two stages are independently scheduled and executed, the first stage to copy rows from the relational database to the non-relational database and the second stage to delete rows from the relational database that have been copied to the non-relational database; and

providing access to the data from the non-relational database via a user interface that accesses both the relational database and the non-relational database, wherein searching of the data stored in the non-relational database comprises transforming a query in a relational database language to multiple parallel scans of the non-relational database to retrieve result data and merging results of the multiple parallel scans to present the result data.

2. The method of claim 1 further comprising:

enabling deletion of the data stored in the non-relational database from the relational database; and

causing the data stored in the non-relational database to be deleted from the relational database according to tenant-specific deletion policies.

3. The method of claim 1 wherein the copying the data from the relational database to the non-relational database comprises:

receiving a key prefix to identify the data to be copied;

splitting the data to be copied into at least a first batch and a second batch;

copying a first batch of data from the relational database to the non-relational database;

automatically, in response to copying the first batch, enqueuing a second batch of data to be copied from the relational database to the non-relational database.

4. A non-transitory computer-readable medium having stored therein instructions to cause one or more processors to manage data in a multitenant environment having a relational database and a non-relational database, the instructions, when executed by the one or more processors, to:

receive, with one or more server computing systems that provide the multitenant environment, a set of one or more policies for field history data retention corresponding to data stored in a history table in the relational database environment, wherein the policies for data retention are defined on a tenant-by-tenant basis within the multitenant environment, wherein the one or more policies for data retention define what data is to be copied from the relational database to the non-relational database, how to copy the data, how long to retain the data in the relational database after the copy and how long to retain the data in the non-relational database after the copy;

deploying, with the one or more computing systems that provide the multitenant environment, the policies to data in the relational database on a tenant-by-tenant basis within the multitenant environment;

identify, with the one or more computing systems that provide the multitenant environment and according to the policies, data to be copied from the relational database to the non-relational database, wherein the data to be copied comprises at least field history information indicating changes made to one or more fields in the relational database and the non-relational database provides a field history archive;

store the data to be copied in the non-relational database while maintaining tenant isolation so that data belonging to the respective tenants is not accessible by other tenants when stored in the non-relational database utilizing a two-stage archival operation where each of the two stages are independently scheduled and executed, the first stage to copy rows from the relational database to the non-relational database and the second stage to delete rows from the relational database that have been copied to the non-relational database; and

provide access to the data from the non-relational database via a user interface that accesses both the relational database and the non-relational database, wherein searching of the data stored in the non-relational database comprises transforming a query in a relational database language to multiple parallel scans of the non-relational database to retrieve result data and merging results of the multiple parallel scans to present the result data.

5. The non-transitory computer-readable medium of claim 4 further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

enable deletion of the data stored in the non-relational database from the relational database; and

cause the data stored in the non-relational database to be deleted from the relational database according to tenant-specific deletion policies.

6. The non-transitory computer-readable medium of claim 4 wherein the instructions that cause the one or more processors to copy the data from the relational database to the non-relational database comprise instructions that, when executed by the one or more processors, cause the one or more processors to:

receive a key prefix to identify the data to be copied;

split the data to be copied into at least a first batch and a second batch;

copy a first batch of data from the relational database to the non-relational database;

automatically, in response to copying the first batch, enqueue a second batch of data to be copied from the relational database to the non-relational database.

7. A system for managing data in a multitenant environment having a relational database and a non-relational database, the system comprising:

a server computing device to provide services to one or more remote client computing devices;

a database management environment to support services provided by the server computing device, the database management environment to receive a set of one or more policies for field history data retention corresponding to data stored in a history table in the relational database environment, wherein the policies for data retention are defined on a tenant-by-tenant basis within the multitenant environment, to deploy the policies to data in the relational database on a tenant-by-tenant basis within the multitenant environment, wherein the one or more policies for data retention define what data is to be copied from the relational database to the non-relational database, how to copy the data, how long to retain the data in the relational database after the copy and how long to retain the data in the non-relational database after the copy, to identify, according to the policies, data to be copied from the relational database to the non-relational database, wherein the data to be copied comprises at least field history information indicating changes made to one or more fields in the relational database and the non-relational database provides a field history archive, to store the data to be copied in the non-relational database while maintaining tenant isolation so that data belonging to the respective tenants is not accessible by other tenants when stored in the non-relational database utilizing a two-stage archival operation where each of the two stages are independently scheduled and executed, the first stage to copy rows from the relational database to the non-relational database and the second stage to delete rows from the relational database that have been copied to the non-relational database, and to provide access to the data from the non-relational database via a user interface that accesses both the relational database and the non-relational database, wherein searching of the data stored in the non-relational database comprises transforming a query in a relational database language to multiple parallel scans of the non-relational database to retrieve result data and merging results of the multiple parallel scans to present the result data.

8. The system of claim 7 , wherein the database management environment is further configured to enable deletion of the data stored in the non-relational database from the relational database, and to cause the data stored in the non-relational database to be deleted from the relational database according to tenant-specific deletion policies.

9. The system of claim 7 wherein the copying the data from the relational database to the non-relational database comprises:

receiving a key prefix to identify the data to be copied;

splitting the data to be copied into at least a first batch and a second batch;

copying a first batch of data from the relational database to the non-relational database;

automatically, in response to copying the first batch, enqueuing a second batch of data to be copied from the relational database to the non-relational database.

10. An apparatus for managing data in a multitenant environment having a relational database and a non-relational database, the apparatus comprising:

means for receiving, with one or more server computing systems that provide the multitenant environment, a set of one or more policies for field history data retention corresponding to data stored in a history table in the relational database environment, wherein the policies for data retention are defined on a tenant-by-tenant basis within the multitenant environment, wherein the one or more policies for data retention define what data is to be copied from the relational database to the non-relational database, how to copy the data, how long to retain the data in the relational database after the copy and how long to retain the data in the non-relational database after the copy;

means for deploying, with the one or more computing systems that provide the multitenant environment, the policies to data in the relational database on a tenant-by-tenant basis within the multitenant environment;

means for identifying, with the one or more computing systems that provide the multitenant environment and according to the policies, data to be copied from the relational database to the non-relational database, wherein the data to be copied comprises at least field history information indicating changes made to one or more fields in the relational database and the non-relational database provides a field history archive;

means for storing the data to be copied in the non-relational database while maintaining tenant isolation so that data belonging to the respective tenants is not accessible by other tenants when stored in the non-relational database utilizing a two-stage archival operation where each of the two stages are independently scheduled and executed, the first stage to copy rows from the relational database to the non-relational database and the second stage to delete rows from the relational database that have been copied to the non-relational database; and

means for providing access to the data from the non-relational database via a user interface that accesses both the relational database and the non-relational database, wherein searching of the data stored in the non-relational database comprises transforming a query in a relational database language to multiple parallel scans of the non-relational database to retrieve result data and merging results of the multiple parallel scans to present the result data.

11. The apparatus of claim 10 further comprising:

means for enabling deletion of the data stored in the non-relational database from the relational database; and

means for causing the data stored in the non-relational database to be deleted from the relational database according to tenant-specific deletion policies.

12. The apparatus of claim 10 wherein the means for copying the data from the relational database to the non-relational database comprises:

means for receiving a key prefix to identify the data to be copied;

means for splitting the data to be copied into at least a first batch and a second batch;

means for copying a first batch of data from the relational database to the non-relational database;

means for automatically, in response to copying the first batch, enqueuing a second batch of data to be copied from the relational database to the non-relational database.