System and method for high-speed payment matching on a processing cluster
A method and system for high-speed, high-volume payment processing for transaction environments having a large number of customers is provided. Embodiments reduce the time required to apply payments against large invoices having, for example, hundreds of thousands of invoice lines through the use of a parallel processing methodology coupled with redistribution of tasks among the various processors in order to optimize available processing power. Embodiments thereby provide a scalable payment processing system that reduces the time required to match payment entries against high volume invoices. Embodiments provide a solution to an existing issue that increasing a number of threads executed (e.g., by increasing CPU cores) for a payment-matching operation against a large invoice does not necessarily reduce processing time needed to complete the operation due to a whole payment being processed as one unit of work within the context of a single thread.
1 . A computer-implemented method for executing high-speed payment processing in a cluster of information handling systems, the method comprising:
a driver information handling system of the cluster of information handling systems, comprising a payment driver in communication with multiple executor nodes, wherein when the payment driver assigns payment instructions, such payment instructions can be checkpointed and stored in persistent storage;
pre-processing a set of payment instructions by an executor node of the cluster of information handling systems, wherein
the executor node runs multiple threads and tasks, each task operating on a specific data partition,
the executor node is configured to process data stored in one or more data partitions,
each data partition is associated with a specific task of the executor node, wherein the specific task is assigned by a driver and the executor node executes code assigned by the driver and reports state of computation to the driver, and a corresponding processor of the executor node configured to process the data stored in the data partition,
each payment instruction comprises a payment identifier, customer identifier, an invoice identifier, and a payment amount,
said pre-processing comprises
associating each payment instruction with a corresponding invoice identifier,
for each payment instruction having a same customer identifier and invoice identifier, prioritizing the payment instruction in order of receipt from a customer associated with the customer identifier,
assigning each payment instruction having a common priority to a data partition for processing in order of priority,
for each payment instruction, by the processor corresponding to the associated data partition,
locating an invoice associated with the invoice identifier, wherein the invoice is stored in a billing system database communicatively coupled to the executor node,
generating an invoice line from the invoice for each unpaid amount in the invoice,
generating a payment line, wherein a sub-payment amount is allocated to each invoice line and the sub-payment amount is a portion of the payment amount of the associated payment instruction; and
processing the set of payment lines by the executor node, wherein
the payment lines are identified for distribution among the partitions by a driver node of the cluster of information handling systems,
each payment line comprises a unique payment line identifier, the customer identifier, the payment identifier, the payment amount, an invoice line identifier, and a payment line amount,
processing comprises
generating a financial transaction record corresponding to each payment line, wherein each financial transaction comprises a unique financial transaction identifier, each field of the associated payment line, and a status of the financial transaction associated with a validation of the financial transaction,
freezing each valid financial transaction,
transmitting each frozen financial transaction to the billing system database for storage.
2 . The method of claim 1 , wherein said assigning each payment instruction having a common priority to a data partition for processing further comprises:
determining a number of payment instructions associated with the common priority; and
evenly dividing the number of payment instructions associated with the common priority among a number of data partitions available on the executor node.
3 . The method of claim 1 wherein allocating the sub-payment amount to each invoice line further comprises:
applying the payment amount to each invoice line up to the unpaid amount in the invoice line; and
generating an excess payment line comprising an excess payment amount when the payment amount exceeds a total unpaid amount from all invoice lines.
4 . The method of claim 1 wherein said pre-processing further comprises:
identifying an action for processing from each payment instruction, wherein an action can be one of create, cancel, and cancel and re-create; and
marking the action for processing in the corresponding payment line.
5 . The method of claim 1 wherein said pre-processing further comprises:
validating a payment instruction assigned to a partition against a record in the billing system database; and
marking a result of said validating in the payment instruction.
6 . The method of claim 1 wherein said pre-processing further comprises:
transmitting the partitioned payment instructions to the driver node for storage as a first checkpoint.
7 . The method of claim 1 , wherein the unique payment line identifier comprises:
a service identifier associated with the invoice line;
a partition identifier; and
a padded random number.
8 . The method of claim 1 , wherein the unique financial transaction identifier comprises:
a portion generated using the unique payment line identifier;
a partition identifier; and
a padded random number.
9 . The method of claim 1 wherein said processing further comprises:
invoking one or more billing system operations; and
transmitting the financial transaction and billing system invocation commands to a billing system node.
10 . The method of claim 1 wherein said processing further comprises:
inserting a status identifier in the financial transaction record indicating that a valid transaction is frozen subsequent to said freezing the valid financial transaction record; and
skipping freezing an invalid financial transaction record and inserting a status identifier in the financial transaction record indicating that the invalid financial transaction record is not frozen.
11 . The method of claim 1 wherein the driver node identifies the payment lines for distribution among the partitions by a method comprising:
calculating an available number of partitions using a parameter specifying a maximum number of lines per partition; and
assigning each payment line among the available number of partitions.
12 . The method of claim 1 further comprising:
post-processing a set of payment status records, wherein
each payment status record is generated from the frozen financial transaction records by the driver node and each payment status record is distributed among the partitions in accord with payment status distribution instructions received from the driver node, and
said post-processing comprises
a final status of each payment status record,
logging an error in the payment status record to the billing system database, and
performing recovery operations from the error in the payment status record.
13 . A system for executing high-speed payment processing, the system comprising:
a cluster of information handling systems, the cluster comprising
a driver information handling system configured to execute a payment driver module,
comprising a payment driver in communication with multiple executor execution information handling systems, wherein when the payment driver assigns payment instructions, such payment instructions can be checkpointed and stored in persistent storage; and
or more executor information handling systems, coupled to the driver information handling system, wherein each executor information handling system comprises
one or more data partitions configured to store data,
a payment executor module configured to execute instructions for
receiving a set of payment instructions wherein each payment instruction comprises a payment identifier, customer identifier, an invoice identifier, and a payment amount,
pre-processing the set of payment instructions, wherein said pre-processing the set of payment instructions comprises
associating each payment instruction with a corresponding invoice identifier,
for each payment instruction having a same customer identifier and invoice identifier, prioritizing the payment instruction in order of receipt from a customer associated with the customer identifier, and
assigning each payment instruction having a common priority to a data partition for processing in order of priority, wherein the data partition is associated with a specific task of an executor node, wherein the specific task is assigned by a driver and the executor node executes code assigned by the driver and reports state of computation to the driver, and
distributing to data partitions, after said pre-processing the set of payment instructions, a set of received payment lines generated from the pre-processed payment instructions, and
one or more processors communicatively coupled to the payment executor module, wherein the payment executor module runs multiple threads and tasks, each task operating on a specific data partition, each processor associated with a corresponding data partition of the one or more data partitions, and each data partition is associated with a specific task of the payment executor module,
and each processor configured to execute instructions for
pre-processing the assigned payment instructions in the associated data partition, said pre-processing the assigned payment instructions comprises
locating an invoice associated with the invoice identifier, wherein the invoice is stored in a billing system database communicatively coupled to the executor node,
generating an invoice line from the invoice for each unpaid amount in the invoice,
generating a payment line wherein a sub-payment amount is allocated to each invoice line, and the sub-payment amount is a portion of the payment amount of the associated payment instruction, and
processing the distributed payment lines in the associated data partition, said processing the distributed payment lines comprises
generating a financial transaction record corresponding to each payment line, wherein each financial transaction record comprises a unique financial transaction identifier, each field of the associated payment line, and a status of the financial transaction associated with a validation of the financial transaction,
freezing each valid financial transaction, and
transmitting each frozen financial transaction to a billing system database for storage; and
a billing system, communicatively coupled to the cluster of information handling systems, comprising the billing system database.
14 . The system of claim 13 wherein the driver information handling system is further configured to execute instructions for identifying the payment lines for distribution among the partitions, wherein the instructions are configured for
calculating an available number of partitions using a parameter specifying a maximum number of lines per partition; and
assigning each payment line among the available number of partitions.
15 . The system of claim 13 , wherein the payment executor module assigns each payment instruction having a common priority to a data partition for processing by executing further instructions for
determining a number of payment instructions associated with the common priority; and
evenly dividing the number of payment instructions associated with the common priority among a number of data partitions available on the executor node.
16 . The system of claim 13 wherein said pre-processing the set of payment instructions further comprises:
validating a payment instruction assigned to a partition against a record in the billing system database; and
marking a result of the said validating in the payment instruction.
17 . The system of claim 13 further comprising:
the payment executor module further configured to execute instructions for post-processing a set of payment status records, wherein
each payment status record is generated from the frozen financial transaction records by the driver node and each payment status record is distributed among the partitions in accord with payment status distribution instructions received from the driver node; and
the one or more processors associated with the data partitions are further configured to execute instructions for post-processing the payment status record stored in the associated partition by executing further instructions for
updating a final status of each payment status record,
logging an error in a payment status record to the billing system database, and
performing recovery operations from the error in the payment status record.
18 . The system of claim 13 wherein said processing the distributed payment lines further comprises:
invoking one or more billing system operations; and
transmitting the financial transaction and billing system invocation commands to the billing system for execution by the billing system.
19 . The system of claim 13 wherein said instructions for processing the distributed payment lines further comprises:
inserting a status identifier in the financial transaction record indicating that a valid transaction is frozen subsequent to said freezing the valid financial transaction record; and
skipping freezing an invalid financial transaction record and inserting a status identifier in the financial transaction record indicating that the invalid financial transaction record is not frozen.