Systems and methods for network optimization in a distributed big data environment
Systems and methods for network optimization in a distributed big data environment are provided. According to an aspect of the invention, a processor performs an optimization method by dividing a data set into a plurality of partitions. For each of the partitions, the processor generates a mathematical representation of a model by associating input data with elements of the model, wherein the mathematical representation includes an objective and at least one constraint. The processor forms a master objective by combining the objectives for the partitions, and forms a set of master constraints by combining the constraints for the partitions. The processor then generates an optimized solution based on the master objective and the master constraints.
1. A method, comprising:
dividing, by a processor, a data set describing resources at a plurality of memories into a plurality of partitions;
for each of the partitions of the plurality of memories, generating a linear programming file (LP file) by associating input data with elements of a linear programming model;
generating a single master LP file to be stored in a local memory of the processor, wherein the single master LP file is a master mathematical representation of a master model, wherein the generating comprises:
forming a master objective by combining objectives for the LP files; and
forming a set of master constraints by combining constraints for the LP files;
storing the single master LP file in the local memory of the processor that is separate from the plurality of memories; and
generating, by the processor, an optimized solution to the single master LP file based on the master objective and the master constraints, the optimized solution being a terminal solution for an optimized distribution of the resources described by the data set.
2. The method of claim 1 , wherein the data set is too large to fit within the local memory of the processor, and wherein the local memory is separate from the plurality of memories, the method further comprising storing the single master LP file in the local memory in place of the data set that is too large to fit within the local memory.
3. The method of claim 1 , wherein the data set is divided into the partitions on a market-by-market basis.
4. The method of claim 1 , wherein the LP file is a mathematical representation of a linear programming model, and wherein the mathematical representation includes an objective and at least one constraint.
5. The method of claim 1 , wherein the optimized solution to the single master LP file is stored in the local memory of the processor.
6. The method of claim 1 , wherein each of the partitions is unique.
7. The method of claim 1 , wherein the master objective and the master constraints are formed by incrementally aggregating the objectives and the constraints for the partitions.
8. The method of claim 1 , wherein the master constraints are formed by aggregating constraints with the same name.
9. The method of claim 1 , wherein each of the mathematical representations includes a plurality of constraints that together form a matrix.
10. The method of claim 1 , wherein the optimized solution is generated by linear programming.
11. The method of claim 1 , wherein generating the optimized solution comprises:
determining that a solution does not exist based on the master objective and the master constraints; and
in response to determining that the solution does not exist, defining slacks that relax the master constraints, and generating the optimized solution based on the master objective and the master constraints as relaxed by the slacks.
12. The method of claim 11 , wherein the optimized solution is generated by minimizing or maximizing the master objective while minimizing a sum of the slacks.
13. The method of claim 1 , wherein the optimized solution maximizes or minimizes a target metric that is defined by the model.
14. The method of claim 13 , wherein the optimized solution maximizes revenue.
15. The method of claim 1 , wherein the optimized solution indicates a value for each of a plurality of decision variables defined by the model.
16. The method of claim 15 , wherein each of the decision variables indicates a number of units of a product to sell.
17. A system, comprising:
a local memory to store a single master LP file; and
a processor operatively coupled to the memory, the processor to:
divide a data set describing resources at a plurality of memories into a plurality of partitions;
for each of the partitions of the plurality of memories, generate a linear programming file (LP file) by associating input data with elements of a linear programming model;
generate the single master LP file to be stored in the local memory, wherein the single master LP file is a master mathematical representation of a master model, wherein the to generate the single master LP file the processor is further to:
form a master objective by combining objectives for the LP files; and
form a set of master constraints by combining constraints for the LP files;
store the single master LP file in the local memory that is separate from the plurality of memories; and
generate an optimized solution to the single master LP file based on the master objective and the master constraints, the optimized solution being a terminal solution for an optimized distribution of the resources described by the data set.
18. The system of claim 17 , wherein to generate the optimized solution the processor is further to:
determine that a solution does not exist based on the master objective and the master constraints; and
in response to determining that the solution does not exist, define slacks that relax the master constraints, and generating the optimized solution based on the master objective and the master constraints as relaxed by the slacks.
19. A non-transitory computer-readable storage medium comprising computer instructions, which when executed by a processor, cause the processor to:
divide a data set describing resources at a plurality of memories into a plurality of partitions;
for each of the partitions of the plurality of memories, generate a linear programming file (LP file) by associating input data with elements of a linear programming model;
generate a single master LP file to be stored in a local memory of the processor, wherein the single master LP file is a master mathematical representation of a master model, wherein the to generate the single master LP file the processor is further to:
form a master objective by combining objectives for the LP files; and
form a set of master constraints by combining constraints for the LP files;
store the single master LP file in the local memory that is separate from the plurality of memories; and
generate, by the processor, an optimized solution to the single master LP file based on the master objective and the master constraints, the optimized solution being a terminal solution for an optimized distribution of the resources described by the data set.
20. The non-transitory computer-readable storage medium of claim 19 , wherein to generate the optimized solution the processor is further to:
determine that a solution does not exist based on the master objective and the master constraints; and
in response to determining that the solution does not exist, define slacks that relax the master constraints, and generating the optimized solution based on the master objective and the master constraints as relaxed by the slacks.