Systems and methods for generating composite images of long documents using mobile video data

1. A computer program product comprising a non-transitory computer readable medium having stored thereon instructions executable by a mobile device, the instructions being configured to cause the mobile device, upon execution thereof, to:

initiate a capture operation using an image capture component of the mobile device, the capture operation comprising;

capturing video data; and

estimating a plurality of motion vectors corresponding to motion of the image capture component during the capture operation;

detect a document depicted in the video data;

track a position of the detected document throughout the video data;

select a plurality of images using the image capture component of the mobile device, wherein the selection is based at least in part on:

the tracked position of the detected document; and

the estimated motion vectors; and

generate a composite image based on at least some of the selected plurality of images.

2. The computer program product as recited in claim 1 , wherein the document is a long document.

3. The computer program product as recited in claim 1 , wherein the tracking is based exclusively on the estimated plurality of motion vectors.

4. The computer program product as recited in claim 1 , the instructions configured to cause the mobile device to detect the document further comprising instructions configured to cause the mobile device to identify at least one edge of the document depicted in the captured video data.

5. The computer program product as recited in claim 1 , wherein each of the selected plurality of images depicts a portion of the document, and wherein the composite image depicts an entirety of the document.

6. The computer program product as recited in claim 1 , wherein the composite image is characterized by at least one of:

an image resolution greater than an image resolution of any of the selected plurality of images; and

an image size greater than an image size of any of the selected plurality of images.

7. The computer program product as recited in claim 1 , further comprising instructions configured to cause the mobile device to downsample the video data, and

wherein the instructions configured to cause the mobile device to detect the document, track the position of the document, and select the plurality of images is configured to perform the detecting, the tracking, and the selecting using the downsampled video data.

8. The computer program product as recited in claim 1 , further comprising instructions configured to cause the mobile device to:

determine at least one motion displacement based on some or all of the estimated plurality of motion vectors, each motion displacement corresponding to the image capture component during the capture operation; and

either terminate or pause the capture operation in response to determining one of the motion displacement(s) is characterized by a value exceeding a predefined motion displacement threshold; and

either initiate a new capture operation in response to terminating the capture operation; or

resume the capture operation in response to pausing the capture operation.

9. The computer program product as recited in claim 8 , wherein the predefined motion displacement threshold has a value in a range from about 5 pixels to about 25 pixels.

10. The computer program product as recited in claim 1 , wherein each selected image depicts a portion of the document, and

wherein the composite image depicts only portion(s) of the document that correspond to a financial transaction memorialized by the document.

11. The computer program product as recited in claim 10 , further comprising instructions configured to cause the mobile device to:

identify, based on the composite image, one or more portions of the document depicting textual information;

classify each identified portion of the document based on the textual information depicted therein; determine whether each classified portion is relevant to the financial transaction or irrelevant to the financial transaction, the determining being based on the portion classification; and

remove each portion determined to be irrelevant to the financial transaction from the composite image.

12. The computer program product as recited in claim 11 , further comprising instructions configured to cause the mobile device to:

align the portions determined to be relevant to the financial transaction; and

generate a second composite image, wherein the second composite image is characterized by:

approximately a same image size as an image size of the composite image;

approximately a same image resolution as an image resolution of the composite image;

excluding textual information irrelevant to the financial transaction; and

including textual information relevant to the financial transaction, wherein a plurality of characters comprising the textual information relevant to the financial transaction are aligned.

13. The computer program product as recited in claim 1 , the instructions configured to cause the mobile device to select the plurality of images further comprising instructions configured to cause the mobile device to define a plurality of frame pairs,

wherein each frame pair consists of a reference frame and a test frame, and

wherein each reference frame and each test frame is selected from the video data.

14. The computer program product as recited in claim 13 , the instructions configured to cause the mobile device to generate the composite image further comprising instructions configured to cause the mobile device to:

detect a skew angle in one or more of the reference frame and the test frame of at least one of the frame pairs, the skew angle corresponding to the document and having a magnitude of >0.0 degrees; and

correct the skew angle in at least one of the reference frame and the test frame,

wherein the document depicted in the composite image is characterized by a skew angle of approximately 0.0 degrees.

15. The computer program product as recited in claim 13 , the instructions configured to cause the mobile device to select the plurality of images further comprising instructions configured to cause the mobile device to:

determine an amount of overlap between the reference frame and the test frame of each frame pair; and

select an image corresponding to each frame pair for which the amount of overlap between the reference frame and the test frame is greater than a predetermined overlap threshold.

16. The computer program product as recited in claim 15 , wherein the amount of overlap corresponds to the document,

wherein the amount of overlap does not correspond to a background depicted in the reference frame, and

wherein the amount of overlap does not correspond to a background depicted in the test frame.

17. The computer program product as recited in claim 15 , wherein the predetermined overlap threshold corresponds to a distance of at least 40% of a length of the reference frame.

18. The computer program product as recited in claim 13 , the instructions configured to cause the mobile device to generate the composite image further comprising instructions configured to cause the mobile device to:

detect textual information in each of the reference frame and the test frame of at least one frame pair, the textual information being depicted in the document.

19. The computer program product as recited in claim 18 , the instructions configured to cause the mobile device to detect textual information in each of the reference frame and the test frame of at least one frame pair further comprising instructions configured to cause the mobile device to:

define, in the reference frame, at least one rectangular portion of the document depicting some or ail of the textual information;

define, in the test frame, at least one corresponding rectangular portion of the document depicting some or all of the textual information; and

align the document depicted in the test frame with the document depicted in the reference frame, the alignment being based on:

the textual information depicted in at least one of the rectangular portion(s); and

the textual information depicted in at least one of the corresponding rectangular portion(s).

20. The computer program product as recited in claim 19 , wherein the textual information comprises at least one of:

an identity of one or more characters represented in the rectangular portion;

an identity of one or more characters represented in the corresponding rectangular portion;

a sequence of characters represented in the rectangular portion;

a sequence of characters represented in the corresponding rectangular portion;

a position of one or more characters represented in the rectangular portion;

a position of one or more characters represented in the corresponding rectangular portion;

an absolute size of one or more characters represented in the rectangular portion;

an absolute size of one or more characters represented in the corresponding rectangular portion

a size of one or more characters represented in the rectangular portion relative to a size of one or more characters represented in the corresponding rectangular portion;

a size of one or more characters represented in the corresponding rectangular portion relative to a size of one or more characters represented in the rectangular portion;

a color of one or more characters represented in the rectangular portion;

a color of one or more characters represented in the corresponding rectangular portion;

a shape of one or more characters represented in the rectangular portion; and

a shape of one or more characters represented in the corresponding rectangular portion.

21. The computer program product as recited in claim 19 , the instructions configured to cause the mobile device to align the document depicted in the test frame with the document depicted in the reference frame further comprising instructions configured to cause the mobile device to perform optical character recognition (OCR) on at least the rectangular portion and the corresponding rectangular portion.