Methods and systems to complete transaction date
View Patent ↗A method and system to receive transaction data; determine a gap in the transaction data; and use an algorithm to generate data to fill in the gap is described. The algorithm is selected from a group including a first algorithm and a second algorithm. The first algorithm is to determine a dominant pattern in the transaction data; identify a region within the dominant pattern that corresponds to the gap in the transaction data; and adopt data associated with the corresponding region into the gap to minimize impact on the dominant pattern. The second algorithm includes a Moore-Penrose pseudo-inverse algorithm to choose the transaction data to fill in the gap based on a set of substitute data from among a group of substitute data sets and adopts the set of substitute data into the gap.
1. A method comprising:
receiving incomplete transaction data at an interface to a processing circuit;
determining by the processing circuit a gap in the incomplete transaction data; and
using an algorithm implemented by the processing circuit to generate data to fill in the gap and to generate complete transaction data, wherein the algorithm is selected by the processing circuit from a group including at least a first algorithm and a second algorithm,
wherein the first algorithm is automatically to:
determine a dominant pattern in the transaction data;
identify a region within the dominant pattern that corresponds to the gap in the transaction data; and
adopt data associated with the corresponding region in the gap to minimize impact on the dominant pattern;
wherein the second algorithm includes a Moore-Penrose pseudo-inverse algorithm to choose at least a portion of the transaction data to fill in the gap based on a set of substitute data from among a group of substitute data sets and to adopt the set of substitute data into the gap; and
where the first algorithm includes
(i, j) referring to a j th day of an i th week, for n weeks with m days in each week,
wherein x ij includes valid numerical data, and if the data is not valid on (i,j), x ij =null,
wherein v ij includes v ij =x ij , unless x ij =0, in which case, v ij =null,
wherein w ij includes w ij =ln(v ij ) whenever v ij is not null, and w ij =null whenever v ij =null,
wherein a matrix of column differences, c ij , includes c ij =w ij+1 −w ij whenever both w ij+1 and w ij are not null, and c ij =null, otherwise,
wherein a matrix of row differences, r ij , includes r ij =w i+1j −w ij whenever both w i+1j and w ij are not null, and r ij =null, otherwise,
wherein a j th column of c ij includes at least one non-null entry, and c *j includes an average of each non-null entry in the j th column of c ij , otherwise,c *j =0,
wherein an i th row of r ij includes at least one non-null entry, and r i* includes an average of each non-null entry in the i th row of r ij , otherwise, r i* =0,
wherein C j+1 =C j +c *j ,where C 1 =0,
wherein R i+1 , =R i +r i* , where R 1 =0,
wherein u ij +R I +C j ,
wherein K includes an average of w ij −u ij over each (i, j) entry where w ij is not null,
wherein y ij =w ij whenever w ij is not null and otherwise, y ij =K+u ij ,
wherein output z ij =Round(exp(y ij )), wherein the output z ij corresponds to filling in the gap.
2. The method of claim 1 wherein the algorithm is selected based upon at least one of amount of the transaction data, forecasting module restrictions, and a ratio of valid data to gap data.
3. The method of claim 1 wherein the algorithm is selected based upon one of maximizing accuracy for filling in the gaps, and minimizing processing time for filling in the gap.
4. The method of claim 1
wherein the second algorithm includes an equation Fh=g, wherein Fh=g includes a plurality of solutions, for h, wherein a solution from the plurality of solutions that is selected to fill in the gap is the solution for h, such that ∥h∥ is minimized solving for h=F + g, wherein a pseudoinverse of F includes F + , wherein vectors h and g include:
h
=
(
r
1
⋮
r
n
c
1
⋮
c
m
)
,
g
=
(
∑
j
∈
P
1
y
1
j
⋮
∑
j
∈
P
n
y
nj
∑
i
∈
Q
1
y
i
1
⋮
∑
i
∈
Q
m
y
im
)
,
respectively,
wherein
F
=
(
∑
j
ɛ
1
j
0
⋯
0
ɛ
11
ɛ
12
⋯
ɛ
1
m
0
∑
j
ɛ
2
j
⋯
0
ɛ
21
ɛ
22
⋯
ɛ
2
m
⋮
⋮
⋰
⋮
⋮
⋮
⋰
⋮
0
0
⋯
∑
j
ɛ
nj
ɛ
n
1
ɛ
n
2
⋯
ɛ
n
m
ɛ
11
ɛ
21
⋯
ɛ
n
1
∑
i
ɛ
i
1
0
⋯
0
ɛ
12
ɛ
22
⋯
ɛ
n
2
0
∑
i
ɛ
i
2
⋯
0
⋮
⋮
⋰
⋮
⋮
⋮
⋰
⋮
ɛ
1
m
ɛ
2
m
⋯
ɛ
n
m
0
0
⋯
∑
i
ɛ
im
)
,
wherein a matrix of column differences, c ij , includes c ij =w ij+1 −w ij whenever both w ij+1 and w ij are not null, and c ij =null, otherwise, wherein a matrix of row differences, r ij , includes r ij =w i+1j −w ij whenever both w i+1j and w ij are not null, and r ij =null, otherwise, wherein (i, j) refers to a j th day of an i th week, for n weeks with m days in each week, wherein x ij includes valid numerical data, and if data is not valid on (i, j), x ij =null, wherein v ij includes v ij =x ij , unless x ij =0, in which case, v ij =null, wherein w ij includes w ij =ln(v ij ) whenever v ij is not null, and w ij =null whenever v ij =null,
wherein in the matrix F the symbol ε ij , where ε ij = 1 when y ij is not null, and ε ij =0 when y ij is null, wherein
y
ij
=
{
x
ij
-
μ
;
(
i
,
j
)
∈
S
null
;
(
i
,
j
)
∉
S
,
wherein x ij denotes entries of a logarithm matrix, X, wherein a set, S=
{
(
i
,
j
)
❘
x
ij
≠
null
}
,
wherein
μ
=
1
o
(
S
)
∑
(
i
,
j
)
∈
S
x
ij
.
5. The method of claim 1 , including forecasting future transaction activity utilizing the complete transaction data.
6. A machine-readable storage medium storing a sequence of instructions that, when executed by a computer, cause the computer to perform the method comprising:
receiving incomplete transaction data;
determining a gap in the incomplete transaction data; and
using an algorithm to generate data to fill in the gap and to generate complete transaction data, wherein the algorithm is selected from a group including a first algorithm and a second algorithm,
wherein the first algorithm is automatically to:
determine a dominant pattern in the transaction data;
identify a region within the dominant pattern that corresponds to the gap in the transaction data; and
adopt data associated with the corresponding region in the gap to minimize impact on the dominant pattern; and
wherein the second algorithm includes a Moore-Penrose pseudo-inverse algorithm to choose at least a portion of the transaction data to fill in the gap based on a set of substitute data from among a group of substitute data sets and to adopt the set of substitute data into the gap; and
wherein the first algorithm includes
(i, j) referring to a j th day of an i th week, for n weeks with m days in each week,
wherein x ij , includes valid numerical data, and if the data is not valid on (i, j), x ij =null,
wherein v ij includes v ij =x ij , unless x ij =0, in which case, v ij =null,
wherein w ij includes w ij =ln(v ij ) whenever v ij is not null, and w ij =null whenever v ij =null,
wherein a matrix of column differences, c ij , includes c ij =−w ij+1−w ij whenever both w ij+1 and w ij are not null, and c ij =null, otherwise,
wherein a matrix of row differences, r ij , includes r ij =w i+1j −w ij whenever both w i+1j and w ij are not null, and r ij =null, otherwise,
wherein a j th column of c ij includes at least one non-null entry, and c *j includes an average of each non-null entry in the j th column of c ij , otherwise, c *j =0,
wherein an i th row of r ij includes at least one non-null entry, and r i* includes an average of each non-null entry in the i th row of r ij , otherwise, r i* =0,
wherein C j+1 =C j +c *j , where C 1 =0,
wherein R i+1 =R i +r i* ,where R 1 =0,
wherein u ij =R i +C j ,
wherein K includes an average of w ij −u ij over each (i, j) entry where w ij is not null,
wherein y ij =w ij whenever w ij is not null and otherwise, y ij =K+u ij ,
wherein output z ij =Round(exp(y ij )), wherein the output z ij corresponds to filling in the gap.
7. A system comprising:
an interface to receive transaction data; and
a transaction gap processor module configured to:
determine a gap in the transaction data;
determine a dominant pattern in the transaction data;
identify a region within the dominant pattern that corresponds to the gap in the transaction data; and
adopt data associated with the corresponding region into the gap to minimize impact on the dominant pattern wherein the transaction gap processor module incorporates an algorithm that includes a formula for output z ij =Round(exp(y ij )), wherein the output z ij corresponds to filling in the gap, wherein (i, j) refers to a j th day of an i th week, for n weeks with m days in each week, wherein y ij =w ij whenever w ij is not null and otherwise, y ij =K +u ij , wherein K includes an average of w ij −u ij over each (i, j) entry where w ij is not null,
wherein C j+1 =C j +c *j , where C 1 =0, wherein R i+1 =R i +r i* , where R 1 =0, wherein u ij =R i +C j ,
wherein a matrix of column differences, c ij , includes c ij =w ij=1 −w ij whenever both w ij+1 and w ij are not null, and c ij =null, otherwise,
wherein a matrix of row differences, r ij , includes r ij =w i+1j −w ij whenever both w i+1j and w ij are not null, and r ij =null, otherwise,
wherein a j th column of c ij includes at least one non-null entry, and c *j includes an average of each non-null entry in the j th column of c ij , otherwise, c *j =0,
wherein an i th row of r ij includes at least one non-null entry, and r i* includes an average of each non-null entry in the i th row of r ij , otherwise, r i* =0, wherein x ij includes valid numerical data, and if the data is not valid on (i, j), x ij =null, wherein v ij includes v ij =x ij , unless x ij =0, in which case, v ij =null, wherein w ij includes w ij =ln(v ij ) whenever v ij is not null, and w ij =null whenever v ij =null.
8. A system comprising:
an interface to receive transaction data;
a transaction gap processor module configured to:
determine a gap in the transaction data;
use a Moore-Penrose pseudo-inverse algorithm to determine transaction data to fill in the gap based on a set of substitute data from among a group of substitute data sets; and
adopt the set of substitute data into the gap;
wherein the transaction gap module includes
an equation Fh=g, wherein Fh=g includes a plurality of solutions, for h, wherein a solution from the plurality of solutions that is selected to fill in the gap is the solution for h, such that ∥h∥ is minimized solving for h=F + g, wherein a pseudoinverse of F includes F + , wherein vectors h and g include:
h
=
(
r
1
⋮
r
n
c
1
⋮
c
m
)
,
g
=
(
∑
j
∈
P
1
y
1
j
⋮
∑
j
∈
P
n
y
nj
∑
i
∈
Q
1
y
i
1
⋮
∑
i
∈
Q
m
y
im
)
,
respectively
,
wherein
F
=
(
∑
j
ɛ
1
j
0
⋯
0
ɛ
11
ɛ
12
⋯
ɛ
1
m
0
∑
j
ɛ
2
j
⋯
0
ɛ
21
ɛ
22
⋯
ɛ
2
m
⋮
⋮
⋰
⋮
⋮
⋮
⋰
⋮
0
0
⋯
∑
j
ɛ
nj
ɛ
n
1
ɛ
n
2
⋯
ɛ
n
m
ɛ
11
ɛ
21
⋯
ɛ
n
1
∑
i
ɛ
i
1
0
⋯
0
ɛ
12
ɛ
22
⋯
ɛ
n
2
0
∑
i
ɛ
i
2
⋯
0
⋮
⋮
⋰
⋮
⋮
⋮
⋰
⋮
ɛ
1
m
ɛ
2
m
⋯
ɛ
n
m
0
0
⋯
∑
i
ɛ
im
)
,
wherein a matrix of column differences, c ij , includes c ij =w ij+1 −w ij whenever both w ij+1 and w ij are not null, and c ij =null, otherwise, wherein a matrix of row differences, r ij , includes r ij =w i+1j −w ij whenever both w i+1j and w ij are not null, and r ij =null, otherwise, wherein (i, j) refers to a j th day of an i th week, for n weeks with m days in each week, wherein x ij includes valid numerical data, and if data is not valid on (i, j), x ij =null, wherein v ij includes v ij =x ij , unless x ij =0, in which case, v ij =null, wherein w ij includes w ij =ln(v ij ) whenever v ij is not null, and w ij =null whenever v ij =null,
wherein in the matrix F the symbol ε ij , where ε ij =1 when y ij is not null, and ε ij =0 when y ij is null, wherein
y
ij
=
{
x
ij
-
μ
;
(
i
,
j
)
∈
S
null
;
(
i
,
j
)
∉
S
,
wherein x ij denotes entries of a logarithm matrix, X, wherein a set, S=
{
(
i
,
j
)
|
x
ij
≠
null
}
,
wherein
μ
=
1
o
(
S
)
∑
(
i
,
j
)
∈
S
x
ij
.
9. A system of claim 8 wherein the gap includes at least one of a data error and a data omission, and the transaction data comprises data regarding frequency of transactions during periods of time.
10. A system comprising:
means for receiving transaction data;
means for determining a gap in the transaction data;
means for determining a dominant pattern in the transaction data;
means for identifying a region within the dominant pattern that corresponds to the gap in the transaction data; and
means for adopting data associated with the corresponding region into the gap to minimize impact on the dominant pattern and wherein the means for determining a gap incorporates an algorithm that includes a formula for output z ij =Round(exp(y ij )), wherein the output z ij corresponds to filling in the gap, wherein (i, j) refers to a j th day of an i th week, for n weeks with m days in each week, wherein y ij =w ij whenever w ij is not null and otherwise, y ij =K +u ij , wherein K includes an average of w ij −u ij over each (i, j) entry where w ij is not null,
wherein C j+1 =C j+c *j , where C 1 =0, wherein R i+1 =R i +r i* , where R 1 =0, wherein u ij =R i +C j ,
wherein a matrix of column differences, c ij , includes c ij =w ij+1 −w ij whenever both w ij+1 and w ij are not null, and c ij =null, otherwise,
wherein a matrix of row differences, r ij , includes r ij =w i+1j −w ij whenever both w i+1j and w ij are not null, and r ij =null, otherwise,
wherein a j th column of c ij includes at least one non-null entry, and c *j includes an average of each non-null entry in the j th column of c ij , otherwise, c *j =0,
wherein an i th row of r ij includes at least one non-null entry, and r i* includes an average of each non-null entry in the i th row of r ij , otherwise, r i* =0, wherein x ij includes valid numerical data, and if the data is not valid on (i, j), x ij =null, wherein v ij includes v ij =x ij , unless x ij =0,in which case, v ij =null, wherein includes w ij =ln(v ij ) whenever v ij is not null, and w ij =null whenever v ij =null.