Criteria-based API selection for data access
Methods, storage systems and computer program products implement embodiments of the present invention for data access that include identifying a set of tables in a database to be accessed by an application, and identifying first and second application programming interface (API) calls having different, respective access properties for accessing records in the tables via an API. Respective counts of the records in the tables are computed by a processor, and the set of tables are partitioned into first and second subsets responsively to the respective counts. The records in the first subset of the tables are accessed by having the application convey the first API call to the API, and the records in the second subset of the tables are accessed by having the application convey the second API call to the API.
1. A method for data access, comprising:
identifying a set of tables in a database to be accessed by an application;
identifying first and second application programming interface (API) calls having different, respective access properties for accessing records in the tables via an API;
counting, by a processor, respective numbers of the records in the tables;
partitioning the set of tables into first and second subsets responsively to the respective numbers, such that the first subset comprises a first group of the tables containing respective numbers of records that are greater than a specified threshold, while the second subset comprises a second group of the tables containing respective numbers of the records that are less than the specified threshold;
accessing the records in the first subset of the tables by conveying, by the application, the first API call to the API; and
accessing the records in the second subset of the tables by conveying, by the application, the second API call to the API.
2. The method according to claim 1 , wherein a given table comprises a customer relationship management data file.
3. The method according to claim 1 , wherein accessing the records in the first and the second subsets comprises generating a set of backup files for the set of tables.
4. The method according to claim 3 , wherein generating a given backup file comprises storing, to the given backup file, a link to a previously generated backup file.
5. The method according to claim 3 , wherein the backup files comprise structured text files.
6. The method according to claim 5 , wherein the structured text files comprise comma-separated (CSV) files.
7. The method according to claim 5 , and further comprising compressing a given structured text file so as to generate a compressed backup file.
8. The method according to claim 3 , wherein accessing, in a given table, the records with a given access property comprises conveying a request for one or more attribute values in one or more records in the given table, receiving the requested attribute values, and saving the received attribute values to a corresponding set of one or more records in a given backup file.
9. The method according to claim 1 , wherein accessing, in a given table, the records with a given access property comprises conveying a request to generate a given backup file for a given table.
10. The method according to claim 9 , wherein the request to generate a given backup file for a given table comprises a request to generate a plurality of backup files that store copies of attribute values in respective subsets of the records in the given table.
11. The method according to claim 10 , and further comprising conveying a request to combine the plurality of backup files into a single combined backup file.
12. The method according to claim 1 , wherein the first and the second subsets comprise disjoint subsets of the tables.
13. The method according to claim 12 , wherein the disjoint subsets cover the set of tables.
14. The method according to claim 1 , wherein the first API call has an API call quota, and wherein partitioning the set of tables into first and second subsets comprises assigning one or more of the tables to the first subset responsively to the quota.
15. The method according to claim 1 , wherein the tables comprise table attributes, and wherein partitioning the set of tables into first and second subsets comprises assigning one or more of the tables to the first subset responsively to the respective table attributes of the tables.
16. An apparatus for data access, comprising:
a network interface controller (NIC); and
a processor configured:
to identify a set of tables in a database to be accessed by an application,
to identify first and second application programming interface (API) calls having different, respective access properties for accessing records in the tables via an API,
to count respective numbers of the records in the tables,
to partition the set of tables into first and second subsets responsively to the respective numbers, such that the first subset comprises a first group of the tables containing respective numbers of records that are greater than a specified threshold, while the second subset comprises a second group of the tables containing respective numbers of the records that are less than the specified threshold,
to access the records in the first subset of the tables by conveying, by the application over a data network via the NIC, the first API call to the API, and
to access the records in the second subset of the tables by conveying, by the application over a data network via the NIC, the second API call to the API.
17. The apparatus according to claim 16 , wherein a given table comprises a customer relationship management data file.
18. The apparatus according to claim 16 , wherein the processor is configured to access the records in the first and the second subsets by generating a set of backup files for the set of tables.
19. The apparatus according to claim 18 , wherein the processor is configured to generate a given backup file by storing, to the given backup file, a link to a previously generated backup file.
20. The apparatus according to claim 19 , wherein the backup files comprise structured text files.
21. The apparatus according to claim 20 , wherein the structured text files comprise comma-separated (CSV) files.
22. The apparatus according to claim 20 , wherein the processor is further configured to compress the structured text files so as to generate a compressed backup file.
23. The apparatus according to claim 18 , wherein the processor is configured to access, in a given table, the records with a given access property by conveying a request for one or more attribute values in one or more records in the given table, receiving the requested attribute values, and saving the received attribute values to a corresponding set of one or more records in a given backup file.
24. The apparatus according to claim 16 , wherein the processor is configured to access, in a given table, the records with a given access property by conveying a request to generate a given backup file for a given table.
25. The apparatus according to claim 24 , wherein the request to generate a given backup file for a given table comprises a request to generate a plurality of backup files that store copies of attribute values in respective subsets of the records in the given table.
26. The apparatus according to claim 25 , wherein the processor is further configured to convey a request to combine the plurality of backup files into a single combined backup file.
27. The apparatus according to claim 16 , wherein the first and the second subsets comprise disjoint subsets of the tables.
28. The apparatus according to claim 27 , wherein the disjoint subsets cover the set of tables.
29. The apparatus according to claim 16 , wherein the first API call has an API call quota, and wherein the processor is configured to partition the set of tables into first and second subsets by assigning one or more of the tables to the first subset responsively to the quota.
30. The apparatus according to claim 16 , wherein the tables comprise table attributes, and wherein the processor is configured to partition the set of tables into first and second subsets by assigning one or more of the tables to the first subset responsively to the respective table attributes of the tables.
31. A computer software product for data access, the product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer:
to identify a set of tables in a database to be accessed by an application;
to identify first and second application programming interface (API) calls having different, respective access properties for accessing records in the tables via an API;
to count respective numbers of the records in the tables;
to partition the set of tables into first and second subsets responsively to the respective numbers, such that the first subset comprises a first group of the tables containing respective numbers of records that are greater than a specified threshold, while the second subset comprises a second group of the tables containing respective numbers of the records that are less than the specified threshold;
to access the records in the first subset of the tables by conveying, by the application, the first API call to the API; and
to access the records in the second subset of the tables by conveying, by the application, the second API call to the API.
32. The method according to claim 1 , wherein the first API call is a bulk API call, and the second API call is a REST API call.
33. The product according to claim 31 , wherein the first API call is a bulk API call, and the second API call is a REST API call.