Method and system to streamline return decision and optimize costs
The embodiments of present disclosure herein address unresolved problems in existing initiatives to optimize costs and streamline the return decisions which are based on legacy infrastructure and explicit rules such as a SQL database. Embodiments herein provide a method and system for streamlining return decision in a supply chain network and optimizing costs. The system is configured to create a returns decision environment using an OpenAI gym base class. Created classes lend extensibility for Reinforcement Learning (RL) applications through a supply chain management environment base class and more specific returns decision environment class. These encapsulate all of the environment functions including exploration of contextual information in the dataset.
1 . A processor-implemented method comprising:
receiving, via an input/output interface, an input data, wherein the input data includes one or more returned items, a Stock Keeping-Unit (SKU) of each one or more returned items, store numbers, vicinity stores, a regional delivery center, T−1 sales, and T−1 returns;
pre-processing, via one or more hardware processors, the received input data to remove outliers and sort values of a plurality of inputs based on a predefined format to generate a pre-processed input data and creates a data set to input to a reinforcement learning (RL) model using details such as a data, a store, an item SKU and a corresponding warehouse, a value of sales and returns to obtain outcomes that is a sales dataset being transformed for return's decision application;
forecasting, via one or more hardware processors, an actual sale and an actual return from the pre-processed input data using a predefined forecasting technique;
computing, via the one or more hardware processors, a Stock-Transfer-Return (STR) data using the pre-processed input data, wherein steps for computing STR comprises;
determining sales demand for each of the one or more returned items for the store number and the item SKU from the forecasted actual sale and T−1 sale, wherein based on the outcome of the determined sales demand of the one or more returned items is (1) re-stocked in the store, or (2) transferred to the vicinity store, or (3) returned to the regional distributor center (DC); and
determining sales demand for each of the one or more returned items in the vicinity store associated with the store number and the item SKU from the forecasted actual sale and T−1 sale, wherein based on the outcome of the determined sales demand in the vicinity store the item is (1) transferred to the vicinity store, or (2) returned to the regional distributor center (DC);
creating, via the one or more hardware processors, an environment model for the STR data to train a Reinforcement Learning (RL) agent and a model agent reward, wherein the environmental model is created using an OpenAI gym base class and initialized using the input data and set of functions are customized to implement the OpenAI gym base class for a supply chain simulation and reinforcement learning, wherein the environment model aids to model classical actors in the supply chain simulation and a return decision class interacts with the RL agent in episodic timesteps through each of the functions, among the set of functions, being available only as abstract class in a OpenAI gym tool kit,
wherein the RL model learns optima in a user dataset through exploration and exploitation of a reward function, thereby reinforcing positive outcomes and penalizing negative outcomes during RL training, and defining rewards strategy allows the RL agent to learn an exploratory search space and propagate intermediate q-value rewards for unknown outcomes to take an informed decision; and
training, via one or more hardware processors, the RL agent with the computed STR data using the OpenAI gym tool kit to generate a return decision of the one or more returned items to (1) re-stock in the store, or (2) transfer to a vicinity store, or (3) return to the regional distributor center.
2 . The processor-implemented method of claim 1 , wherein a control logic for creation and initializing the environment model for training the RL agent.
3 . The processor-implemented method of claim 1 , wherein the control logic includes a reset step to initialize a current context data frame pointer to an initial value of −1 and update state value to the value in a next context data frame of the environment model.
4 . The processor-implemented method of claim 1 , wherein a state and reward value are provided to the RL agent in a timestep episodic manner to train.
5 . The processor-implemented method of claim 1 , wherein an action value is determined by the RL agent in a timestep manner to provide to the environment.
6 . The processor-implemented method of claim 1 , wherein the control logic updates a new state value to the value in the next context data frame of the environment model.
7 . The processor-implemented method of claim 1 , wherein a STR reward is calculated to determine a reward value corresponding to the action in the environment model.
8 . The processor-implemented method of claim 1 , wherein a done value is determined by the environment model when a current pointer is at the last index of the context data frame.
9 . A system comprising:
an input/output interface, to receive an input data, wherein the input data includes one or more returned items, a Stock Keeping-Unit (SKU) of each one or more returned items, store numbers, vicinity stores, a regional delivery center, T−1 sales, and T−1 returns;
a memory in communication with the one or more hardware processors, wherein the one or more hardware processors are configured to execute programmed instructions stored in the memory to:
pre-process the received plurality of input data to remove outliers and sort values of the plurality of inputs based on a predefined format to generate a pre-processed input data and creates a data set to input to a reinforcement learning (RL) model using details such as a data, a store, an item SKU and a corresponding warehouse, a value of sales and returns to obtain outcomes that is a sales dataset being transformed for return's decision application;
forecast an actual sale and an actual return from the pre-processed input data using a predefined forecasting technique;
compute a Stock-Transfer-Return (STR) data using the pre-processed input data, wherein steps for computing STR comprises:
determining sales demand for each of the one or more returned items for the store number and the item SKU from the forecasted actual sale and T−1 sale, wherein based on the outcome of the determined sales demand of the each of the one or more returned items is (1) re-stocked in the store, or (2) transferred to the vicinity store, or (3) returned to the regional distributor center (DC); and
determining sales demand for each of the one or more returned items in the vicinity store associated with the store number and the item SKU from the forecasted actual sale and T−1 sale, wherein based on the outcome of the determined sales demand in the vicinity store the item is (1) transferred to the vicinity store, or (2) returned to the regional distributor center (DC);
create an environment model for the STR to train a Reinforcement Learning (RL) agent and a model agent reward, wherein the environmental model is created using an OpenAI gym base class and initialized using the input data and set of functions are customized to implement the OpenAI gym base class for a supply chain simulation and reinforcement learning, wherein the environment model aids to model classical actors in the supply chain simulation and a return decision class interacts with the RL agent in episodic timesteps through each of the functions, among the set of functions, being available only as abstract class in a OpenAI gym tool kit,
wherein the RL model learns optima in a user dataset through exploration and exploitation of a reward function, thereby reinforcing positive outcomes and penalizing negative outcomes during RL training, and defining rewards strategy allows the RL agent to learn an exploratory search space and propagate intermediate q-value rewards for unknown outcomes to take an informed decision; and
train the RL agent with the computed STR data using the OpenAI gym tool kit to generate a return decision of the one or more returned items to (1) re-stock in the store, or (2) transfer to a vicinity store, or (3) return to the regional distributor center.
10 . The system of claim 9 , wherein the control logic includes a reset step to initialize a current context data frame pointer to an initial value of −1 and updating state value to the value in a next context data frame of the environment model.
11 . The system of claim 9 , wherein the control logic initializes the environment model for training the RL agent and updating a new state value to the value in the next context data frame of the environment model.
12 . The system of claim 9 , wherein a state and reward value are provided to the RL agent in a timestep episodic manner to train, and wherein an action value is determined by the RL agent in a timestep manner to provide to the environment.
13 . The system of claim 9 , wherein a Stock-Transfer-Return (STR) reward is calculated to determine a reward value corresponding to the action in the environment model.
14 . The system of claim 9 , wherein a done value is determined by the environment model when a current pointer is at the last index of the context data frame.
15 . One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
receiving, via an input/output interface, an input data, wherein the input data includes one or more returned items, a Stock Keeping-Unit (SKU) of each one or more returned items, store numbers, vicinity stores, a regional delivery center, T−1 sales, and T−1 returns;
pre-processing the received input data to remove outliers and sort values of a plurality of inputs based on a predefined format to generate a pre-processed input data and creates a data set to input to a reinforcement learning (RL) model using details such as a data, a store, an item SKU and a corresponding warehouse, a value of sales and returns to obtain outcomes that is a sales dataset being transformed for return's decision application;
forecasting an actual sale and an actual return from the pre-processed input data using a predefined forecasting technique;
computing, via the one or more hardware processors, a Stock-Transfer-Return (STR) data using the pre-processed input data, wherein steps for computing STR comprises;
determining sales demand for each of the one or more returned items for the store number and the item SKU from the forecasted actual sale and T −1 sale, wherein based on the outcome of the determined sales demand of the one or more returned items is (1) re-stocked in the store, or (2) transferred to the vicinity store, or (3) returned to the regional distributor center (DC); and
determining sales demand for each of the one or more returned items in the vicinity store associated with the store number and the item SKU from the forecasted actual sale and T−1 sale, wherein based on the outcome of the determined sales demand in the vicinity store the item is (1) transferred to the vicinity store, or (2) returned to the regional distributor center (DC);
creating an environment model for the STR data to train a Reinforcement Learning (RL) agent and a model agent reward, wherein the environmental model is created using an OpenAI gym base class and initialized using the input data and set of functions are customized to implement the OpenAI gym base class for a supply chain simulation and reinforcement learning, wherein the environment model aids to model classical actors in the supply chain simulation and a return decision class interacts with the RL agent in episodic timesteps through each of the functions, among the set of functions, being available only as abstract class in a OpenAI gym tool kit,
wherein the RL model learns optima in a user dataset through exploration and exploitation of a reward function, thereby reinforcing positive outcomes and penalizing negative outcomes during RL training, and defining rewards strategy allows the RL agent to learn an exploratory search space and propagate intermediate q-value rewards for unknown outcomes to take an informed decision; and
training the RL agent with the computed STR data using the OpenAI gym tool kit to generate a return decision of the one or more returned items to (1) re-stock in the store, or (2) transfer to a vicinity store, or (3) return to the regional distributor center.
16 . The one or more non-transitory machine-readable information storage mediums of claim 15 , wherein a control logic for creation and initializing the environment model for training the RL agent, and wherein the control logic updates a new state value to the value in the next context data frame of the environment model.
17 . The one or more non-transitory machine-readable information storage mediums of claim 15 , wherein the control logic includes a reset step to initialize a current context data frame pointer to an initial value of −1 and updating state value to the value in a next context data frame of the environment model.
18 . The one or more non-transitory machine-readable information storage mediums of claim 15 , wherein a state and reward value are provided to the RL agent in a timestep episodic manner to train.
19 . The one or more non-transitory machine-readable information storage mediums of claim 15 , wherein an action value is determined by the RL agent in a timestep manner to provide to the environment.
20 . The processor-implemented method of claim 1 , wherein a STR reward is calculated to determine a reward value corresponding to the action in the environment model.