Machine learning inference calls for database query processing
Techniques for making machine learning inference calls for database query processing are described. In some embodiments, a method of making machine learning inference calls for database query processing may include generating a first batch of machine learning requests based at least on a query to be performed on data stored in a database service, wherein the query identifies a machine learning service, sending the first batch of machine learning requests to an input buffer of an asynchronous request handler, the asynchronous request handler to generate a second batch of machine learning requests based on the first batch of machine learning requests, and obtaining a plurality of machine learning responses from an output buffer of the asynchronous request handler, the machine learning responses generated by the machine learning service using a machine learning model in response to receiving the second batch of machine learning requests.
1. A computer-implemented method comprising:
receiving a request at a database service, wherein the request includes a structured query language (SQL) query to be performed on at least a portion of data stored in the database service, and wherein the request identifies a machine learning (ML) service to be used in processing the SQL query using an application programming interface (API) call to the ML service, and wherein the ML service publishes the API to perform inference using a ML model in response to requests;
executing at least a portion of the SQL query on the data stored in the database service to generate a first batch of ML requests;
generating a second batch of ML requests based on the first batch of ML requests and based on the ML service; and
obtaining ML responses generated by the ML service using the ML model in response to receiving the second batch of ML requests.
2. The computer-implemented method of claim 1 , wherein a virtual operator of a database instance of the database service generates the first batch of ML requests.
3. The computer-implemented method of claim 2 , wherein the virtual operator is implemented as a temporary data structure that identifies database records to be sent to the ML service.
4. The computer-implemented method of claim 1 , further comprising sending the first batch of ML requests to an input buffer of an asynchronous request handler.
5. The computer-implemented method of claim 4 , wherein the asynchronous request handler generates the second batch of ML requests based on the first batch of ML requests.
6. The computer-implemented method of claim 4 , wherein the ML responses are obtained from an output buffer of the asynchronous request handler.
7. The computer-implemented method of claim 4 , wherein a batch size of the first batch of ML requests is equal to a buffer size of the input buffer of the asynchronous request handler.
8. A computer-implemented method comprising:
generating a first batch of machine learning (ML) requests by executing at least a portion of a structured query language (SQL) query on data stored in a database service of a provider network, wherein the SQL query is to be processed by a machine learning (ML) service of the provider network in response to an application programming interface (API) call to the ML service, and wherein the ML service publishes the API to perform inference using a ML model in response to requests;
generating a second batch of ML requests based on the first batch of ML requests and based on the ML service; and
obtaining ML responses generated by the ML service using the ML model in response to receiving the second batch of ML requests.
9. The computer-implemented method of claim 8 , wherein a virtual operator of a database instance of the database service generates the first batch of ML requests.
10. The computer-implemented method of claim 9 , wherein the virtual operator is implemented as a temporary data structure that identifies database records to be sent to the ML service.
11. The computer-implemented method of claim 8 , further comprising sending the first batch of ML requests to an input buffer of an asynchronous request handler.
12. The computer-implemented method of claim 11 , wherein the asynchronous request handler generates the second batch of ML requests based on the first batch of ML requests.
13. The computer-implemented method of claim 11 , wherein the ML responses are obtained from an output buffer of the asynchronous request handler.
14. The computer-implemented method of claim 11 , wherein a batch size of the first batch of ML requests is equal to a buffer size of the input buffer of the asynchronous request handler.
15. A system comprising:
a machine learning (ML) service implemented by a first one or more electronic devices in a provider network; and
a database service implemented by a second one or more electronic devices in the provider network, the second one or more electronic devices including one or more processors and memory, the database service including instructions stored in the memory that, upon execution by the one or more processors, cause the database service to:
generate a first batch of machine learning (ML) requests by executing at least a portion of a structured query language (SQL) query on data stored in the database service, wherein the SQL query is to be processed by the ML service in response to an application programming interface (API) call to the ML service, and wherein the ML service publishes the API to perform inference using a ML model in response to requests;
generate a second batch of ML requests based on the first batch of ML requests and based on the ML service; and
obtain ML responses generated by the ML service using the ML model in response to receiving the second batch of ML requests.
16. The system of claim 15 , wherein a virtual operator of a database instance of the database service generates the first batch of ML requests.
17. The system of claim 16 , wherein the virtual operator is implemented as a temporary data structure that identifies database records to be sent to the ML service.
18. The system of claim 15 , wherein the database service includes further instructions stored in the memory that, upon execution by the one or more processors, further cause the database service to send the first batch of ML requests to an input buffer of an asynchronous request handler.
19. The system of claim 18 , wherein the asynchronous request handler generates the second batch of ML requests based on the first batch of ML requests.
20. The system of claim 18 , wherein the ML responses are obtained from an output buffer of the asynchronous request handler.