Granted Patent B2

US 12,424,013 · App. 17/985,070 · Granted Sep 23, 2025

Inventors: Michael Benjamin Brodie (Highland, UT); Gopalkrishna Balkrishna Veni (Lehi, UT); Jack Reese (Lindon, UT); Azadeh Moghtaderi (Kentfield, CA); Randon Morford (Saratoga Springs, UT)

Assignee: Ancestry.com Operations Inc.

G06V30/414G06T5/50G06V10/267G06V30/15G06T2207/20081G06T2207/20084

Abstract

Claims (71)

1. A computer-implemented method, comprising:

receiving, by a genealogy server, an image that is digitalized from a physical record, the image associated with a genealogy record or an individual profile of the genealogy server;

identifying a sub-region of the image as a target region for image enhancement;

classifying that the sub-region includes a type of image component;

enhancing the sub-region based on the classified type of the image component to generate an enhanced sub-region, enhancing the sub-region comprising restoring or colorizing the image component, wherein enhancing the sub-region is performed at least partially by a machine learning model and the machine learning model comprises a generative adversarial network that is trained using a plurality of image records stored in the genealogy server and faux-real images generated by randomly oversaturating real images; and

merging the enhanced sub-region with one or more other sub-regions or an original version of the image.

2. The computer-implemented method of claim 1 , wherein the type of image component is selected from candidate types of image components, the candidate types comprise a text component, a single image component, a multi-image component, and a face component.

3. The computer-implemented method of claim 1 , wherein the image enhancement comprises a combination of image enhancing techniques that are selectable by a user via a graphical user interface.

4. The computer-implemented method of claim 1 , wherein enhancing the sub-region based on the classified type of the image component comprises:

selecting a set of image processing techniques according to the classified type, wherein the set of image processing techniques is predetermined for the classified type; and

applying the set of image processing techniques to the sub-region.

5. The computer-implemented method of claim 4 , wherein selecting the set of image processing techniques is further based on a user's request on the image enhancement.

6. The computer-implemented method of claim 1 , wherein the machine learning model comprises a generative adversarial network that is trained using faux-real images generated by randomly oversaturating a set of real images by a percentage within a predetermined range.

7. The computer-implemented method of claim 1 , further comprising:

segmenting a text component from an image component of the image; and

performing text restoration separately from image restoration and/or colorization.

8. The computer-implemented method of claim 1 , wherein enhancing the sub-region comprises restoring the image component, and restoring the image component comprises:

determining that a size of the image exceeds a predetermined size threshold;

adjusting the size of the image;

performing image restoration on the image component;

merging restored image component into the original version of the image that has the size adjusted; and

restoring an original image size and aspect ratio.

9. The computer-implemented method of claim 1 , wherein enhancing the sub-region comprises performing a facial enhancement, and performing the facial enhancement comprises:

detecting a face is present in the sub-region;

expanding the face;

selecting an image-processing machine learning model that is trained specifically for enhancing faces; and

enhancing an expanded face using the image-processing machine learning model.

10. The computer-implemented method of claim 1 , wherein enhancing the sub-region comprising colorizing the image component, and colorizing the image component comprises:

identify a color scheme of the image component; and

colorizing the image component based on the color scheme.

11. The computer-implemented method of claim 1 , further comprising:

determining that an enhanced image component was cropped from a larger image; and

merging the enhanced image component into the larger image.

12. A system comprising:

one or more processors; and

memory configured to store code comprising instructions, the instructions, when executed by the one or more processors, cause the one or more processors to perform steps comprising:

receiving an image that is digitalized from a physical record, the image associated with a genealogy record or an individual profile of a genealogy server;

identifying a sub-region of the image as a target region for image enhancement;

classifying that the sub-region includes a type of image component;

enhancing the sub-region based on the classified type of the image component to generate an enhanced sub-region, enhancing the sub-region comprising restoring or colorizing the image component, wherein enhancing the sub-region is performed at least partially by a machine learning model and the machine learning model is trained using a plurality of image records stored in the genealogy server, and wherein enhancing the sub-region comprises restoring the image component by:

adjusting a size of the image in response to determining that the size of the image exceeds a predetermined size threshold;

generating a modified image by merging a restored image component into the image that has the size adjusted; and

restoring the modified image to an original image size and an original image aspect ratio; and

merging the enhanced sub-region with one or more other sub-regions or an original version of the image.

13. The system of claim 12 , wherein the type of image component is selected from candidate types of image components, the candidate types comprise a text component, a single image component, a multi-image component, and a face component.

14. The system of claim 12 , wherein the image enhancement comprises a combination of image enhancing techniques that are selectable by a user via a graphical user interface.

15. The system of claim 12 , wherein enhancing the sub-region based on the classified type of the image component comprises:

selecting a set of image processing techniques according to the classified type, wherein the set of image processing techniques is predetermined for the classified type; and

applying the set of image processing techniques to the sub-region.

16. The system of claim 15 , wherein selecting the set of image processing techniques is further based on a user's request on the image enhancement.

17. The system of claim 12 , wherein the machine learning model comprises a generative adversarial network that is trained using faux-real images generated by randomly oversaturating real images.

18. The system of claim 12 , further comprising:

segmenting a text component from an image component of the image; and

performing text restoration separately from image restoration and/or colorization.

19. The system of claim 12 , wherein enhancing the sub-region comprises performing a facial enhancement, and the facial enhancement comprises:

detecting a face is present in the sub-region;

expanding the face;

selecting an image-processing machine learning model that is trained specifically for enhancing faces; and

enhancing an expanded face using the image-processing machine learning model.

20. A system comprising:

a graphical user interface configured to:

provide a user an interface to upload an image that is digitalized from a physical record; and

receive one or more options from the user on image enhancement of the image; and

a genealogy server in communication with the graphical user interface, the genealogy server comprising one or more processors and memory, the memory configured to store code comprising instructions, the instructions, when executed by the one or more processors, cause the one or more processors to perform steps comprising:

receiving the image, the image to be associated with a genealogy record or an individual profile of the genealogy server;

identifying a sub-region of the image as a target region for image enhancement;

classifying that the sub-region includes a type of image component;

enhancing the sub-region based on the classified type of the image component to generate an enhanced sub-region, enhancing the sub-region comprising restoring or colorizing the image component, wherein enhancing the sub-region is performed at least partially by a machine learning model and the machine learning model is trained using a plurality of image records stored in a genealogy server, and wherein enhancing the sub-region comprises performing a facial enhancement by:

expanding a face detected in the sub-region; and

enhancing the expanded face using an image-processing machine learning model that is trained specifically for enhancing faces; and

merging the enhanced sub-region with one or more other sub-regions or an original version of the image.

Assignments (1)

Continuity (3)

References Cited (81)