Granted Patent B2

US 11,657,631 · App. 17/243,467 · Granted May 23, 2023

Inventors: Christos Sagonas (London, GB); Karolina Dabkowska (London, GB); Zhiyuan Shi (London, GB); Edward Fieri Soler (London, GB); Mohan Mahadevan (London, GB); Iona Grace Vincent (London, GB); Luca Peric (London, GB); Alessandro Lenzi (London, GB); Alvaro Fernando Lara (London, GB); James Stonehill (London, GB)

Assignee: Onfido Ltd.

G06T7/30G06K9/6256G06K9/6267G06T3/40G06T7/11G06T7/136G06T7/70G06V10/22G06V30/414G06T2207/20132G06T2207/30176G06V30/10

Abstract

Claims (47)

1. A computer-implemented method for extracting data from a document comprising:

acquiring an input image comprising a document portion, the document portion being of a document of a first document type;

performing image segmentation on the input image to form a binary input image that distinguishes the document portion from the remaining portion of the input image;

estimating a first image transform to align the binary input image to a binary template image;

using the first image transform on the input image to form an intermediate image;

estimating a second image transform to align the intermediate image to a template image, the template image comprising a template document portion, the template document portion being of a different document of the first document type;

using the second image transform on the intermediate image to form an output image; and

extracting a field from the output image using a predetermined field of the template image.

2. The method of claim 1 , wherein estimating the first image transform, using the first image transform, estimating the second image transform, and using the second image transform do not use a deep learning-based model.

3. The method of claim 1 , wherein the template document portion has predetermined coordinates in the template image, the method further comprising:

after using the first image transform and prior to estimating the second image transform, cropping the intermediate image using the predetermined coordinates; and

cropping the template image using the predetermined coordinates.

4. The method of claim 1 , further comprising:

after using the first image transform and prior to estimating the second image transform, reducing the size of the intermediate image and the template image using a resize operator; and

after estimating the second image transform and prior to using the second image transform, applying a transverse of the resize operator to the second image transform.

5. The method of claim 1 , wherein the binary input image is a binary segmentation mask of the same size as the input image that represents the document portion with a pixel value of 255 and the remaining portion with a pixel value of 0, and the binary template image is a binary segmentation mask of the same size as the template image that represents the template document portion with a pixel value of 255 and the remaining portion with a pixel value of 0.

6. The method of claim 1 , wherein the predetermined field has predetermined coordinates within the template image, and extracting the field comprises using the predetermined coordinates.

7. The method of claim 1 , wherein the field is a text field, the method further comprising cropping the text field so that the text field fits the text contained therein and performing optical character recognition on the text field to extract text from the document.

8. The method of claim 7 , wherein the optical character recognition (OCR) is performed using a deep learning-based OCR model, the method further comprising generating training images to train the deep learning-based OCR model.

9. The method of claim 1 , wherein the first image transform comprises a plurality of first image transforms, each of the plurality of first image transforms configured to align a different sub-frame of the binary input image to the binary template image.

10. The method of claim 1 , wherein the second image transform comprises a plurality of second image transforms, each of the plurality of second image transforms configured to align a different sub-frame of the intermediate input image to the template image.

11. The method of claim 1 , further comprising, prior to performing image segmentation, pre-processing the input image by determining document location within the input image and estimating the pose of the document within the input image.

12. The method of claim 1 , further comprising, prior to performing image segmentation, classifying the document of the input image as the first document type, and retrieving the template image and the binary template image of the first document type from a database.

13. The method of claim 1 , further comprising:

acquiring a second input image comprising a document portion, the document portion being of a document of a second document type;

performing image segmentation on the second input image to form a second binary input image that distinguishes the document portion from the remaining portion of the second input image;

estimating a third image transform to align the second binary input image to a second binary template image,

using the third image transform on the second input image to form a second intermediate image;

estimating a fourth image transform to align the second intermediate image to a second template image, the second template image comprising a template document portion, the template document portion being of a different document of the second document type;

using the second image transform on the second intermediate image to form a second output image; and

extracting a field from the second output image using a predetermined field of the second template image.

14. A non-transitory computer-readable medium comprising a plurality of instructions that, when executed by a processor, cause the processor to:

acquire an input image comprising a document portion, the document portion being of a document of a first document type;

perform image segmentation on the input image to form a binary input image that distinguishes the document portion from the remaining portion of the input image;

estimate a first image transform to align the binary input image to a binary template image;

use the first image transform on the input image to form an intermediate image;

estimate a second image transform to align the intermediate image to a template image, the template image comprising a template document portion, the template document portion being of a different document of the first document type;

use the second image transform on the intermediate image to form an output image; and

extract a field from the output image using a predetermined field of the template image.

15. A server comprising a processor configured to execute executable code stored in a computer-readable memory, wherein the executable code comprises instructions that cause the processor to:

acquire an input image comprising a document portion, the document portion being of a document of a first document type;

perform image segmentation on the input image to form a binary input image that distinguishes the document portion from the remaining portion of the input image;

estimate a first image transform to align the binary input image to a binary template image;

use the first image transform on the input image to form an intermediate image;

estimate a second image transform to align the intermediate image to a template image, the template image comprising a template document portion, the template document portion being of a different document of the first document type;

use the second image transform on the intermediate image to form an output image; and

extract a field from the output image using a predetermined field of the template image.

Assignments (5)

Priority Claims (1)

Continuity (1)

Cited By (12)

US 12,271,929 US 12,321,965 US 12,322,259 US 12,380,420 US 12,462,635 US 12,475,756 US 12,525,000 US 12,536,643 US 12,597,004 US 12,608,687 US 12,646,041 US 12,676,045