Systems and methods for determining a marketing strategy for an online to offline service

1. A system for determining a marketing strategy for an online to offline service, comprising:

at least one computer-readable storage medium for storing a set of instructions in determining the marketing strategy for the online to offline service; and

at least one processor in communication with the computer-readable storage medium, wherein when executing the set of instructions, the at least one processor is directed to:

obtain a plurality of historical vectors, each of the plurality of historical vectors being associated with a state of a driver and collected from a driver terminal via a network;

determine a first value function according to a first reinforcement learning algorithm based on the plurality of historical vectors;

determine a first value under a state of a driver to provide the online to offline service based on the first value function and the state;

obtain a first regression model according to a machine learning algorithm by training an initial model, wherein the plurality of historical vectors are inputs of the initial model, and the first value under the state is a label of the initial model;

determine a second value function according to a second reinforcement learning algorithm based on the plurality of historical vectors;

determine a second value under each state based on the second value function and the state;

determine a second regression model by training the initial model, wherein the plurality of historical vectors are the inputs of the initial model, the second value under each state is a label of the initial model;

select a final model from the first regression model and the second regression model; and

utilize the final model to determine the marketing strategy for the online to offline service.

2. The system of claim 1 , wherein each of the plurality of historical vectors includes the state of the driver, at least one historical action that the system implemented to the driver under the state, and a historical reward under the state and the at least one historical action.

3. The system of claim 1 , wherein the at least one processor is further directed to:

evaluate off-line the first regression model using an importance sampling method; and thereby,

adjust at least one parameter of the first reinforcement learning algorithm or the first regression model.

4. The system of claim 3 , the at least one processor is further directed to:

determine at least one random strategy based on a plurality of historical marketing strategies, wherein each historical marketing strategy includes a historical state of a driver, a historical action that the system implemented to the driver, and a historical long-term value under the state that the driver contributed to the system after being implemented to the historical action;

obtain at least one strategy distribution by sampling the at least one random strategy according to the importance sampling method;

determine an estimated long-term value under each historical state that the driver will contribute to the system based on the first regression model; and

determine an evaluation result based on the estimated long-term value and the historical long-term value.

5. The system of claim 2 , wherein the state of the driver comprises statistical information associated with the driver in the system or a demographic attribute of the driver.

6. The system of claim 5 , wherein the statistical information comprises a registration time of the car, information of the car, or a lapse between the registration time and a first online to offline service provided by the car, and wherein the demographic attribute of the driver comprises age, sex, a cultural background, and a job, and any combination thereof.

7. The system of claim 2 , wherein the at least one historical action comprises at least one of: a short message, a coupon, an action time the system sent to the driver, or a sequence of a plurality of the at least one historical actions.

8. The system of claim 2 , wherein the historical reward comprises a Gross Merchandise Volume that the driver contributed to the system.

9. The system of claim 4 , wherein the at least one strategy distribution comprises an optimal strategy distribution.

10. The system of claim 1 , wherein the first reinforcement learning algorithm includes a Q-learning algorithm, and the inputs of the Q-learning algorithm are the plurality of historical vectors, and the output of the Q-learning algorithm is the first value function.

11. The system of claim 1 , wherein the first reinforcement learning algorithm includes a Sarsa-learning algorithm, and the inputs of the Sarsa-learning algorithm are the plurality of historical vectors, and the output of the Sarsa-learning algorithm is the first value function.

12. The system of claim 1 , wherein the initial model is a XG Boost model.

13. The system of claim 1 , wherein to obtain the plurality of historical vectors, the at least one processor is further directed to:

obtain a plurality of historical vectors based on a Markov Decision Process (MDP) model, wherein the input of the MDP model is historical marketing data.

14. The system of claim 1 , wherein to determine the final model, the at least one processor is further directed to:

determine at least one random strategy;

obtain at least one strategy distribution by sampling the at least one random strategy;

determine a first estimated long-term value under the state that the driver will contribute to the system based on the first regression model;

determine a second estimated long-term value under the state that the driver will contribute to the system based on the second regression model; and

determine the final model based on the first estimated long-term value, the second estimated long-term value, and the historical long-term value under the state.

15. The system of claim 14 , wherein the at least one processor is further directed to:

adjust at least one parameter of the first reinforcement learning algorithm or the second reinforcement learning algorithm based on the first estimated long-term value, the second estimated long-term value, and the historical long-term value under the state.

16. A method for determining a marketing strategy for an online to offline service implemented on a computing device having at least one processor, at least one computer-readable storage medium, and a communication platform connected to a network, comprising:

obtaining a plurality of historical vectors, each of the plurality of historical vectors being associated with a state of a driver and collected from a driver terminal via a network;

determining a first value function according to a first reinforcement learning algorithm based on the plurality of historical vectors;

determining a first value under a state of a driver to provide the online to offline service based on the first value function and the state;

obtaining a first regression model according to a machine learning algorithm by training an initial model, wherein the plurality of historical vectors are inputs of the initial model, and the first value under the state is a label of the initial model;

determining a second value function according to a second reinforcement learning algorithm based on the plurality of historical vectors;

determining a second value under each state based on the second value function and the state;

determining a second regression model by training the initial model, wherein the plurality of historical vectors are the inputs of the initial model, the second value under each state is a label of the initial model;

selecting a final model from the first regression model and the second regression model; and

utilize the final model to determine the marketing strategy for the online to offline service.

17. The method of claim 16 , wherein each of the plurality of historical vectors includes the state of the driver, at least one historical action that the system implemented to the driver under the state, and a historical reward under the state and the at least one historical action.

18. The method of claim 16 further comprising:

evaluating off-line the first regression model using an importance sampling method; and thereby,

adjusting at least one parameter of the first reinforcement learning algorithm or the first regression model.

19. The method of claim 18 further comprising:

determining at least one random strategy based on a plurality of historical marketing strategies, wherein each historical marketing strategy includes a historical state of a driver, a historical action that the system implemented to the driver, and a historical long-term value under the state that the driver contributed to the system after being implemented to the historical action;

obtaining at least one strategy distribution by sampling the at least one random strategy according to the importance sampling method;

determining an estimated long-term value under each historical state that the driver will contribute to the system based on the first regression model; and

determining an evaluation result based on the estimated long-term value and the historical long-term value.