Enhancing documents portrayed in digital images
View Patent ↗The present disclosure is directed toward systems and methods that efficiently and effectively generate an enhanced document image of a displayed document in an image frame captured from a live image feed. For example, systems and methods described herein apply a document enhancement process to a displayed document in an image frame that result in an enhanced document image that is cropped, rectified, un-shadowed, and with dark text against a mostly white background. Additionally, systems and method described herein determine whether a stored digital content item includes a displayed document. In response to determining that a stored digital content item does include a displayed document, systems and methods described herein generate an enhanced document image of a displayed document included in the stored digital content item.
1. A system comprising:
at least one processor; and
a non-transitory computer-readable medium storing instructions thereon that, when executed by the at least one processor, cause the system to:
receive, at a content management system and from a client device, a content item;
determine, by the content management system, the content item comprises an image of a displayed document by analyzing the content item using a trained image classifier;
provide, for display on the client device, a graphical user interface comprising a listing of content items, the listing comprising the content item;
provide, for display on the client device, a selectable graphical element based on determining that the content item comprises the image of the displayed document; and
based on receiving an indication of a user selection of the selectable graphical element, generate an enhanced document image of the displayed document, the enhanced document image comprising one or more visual alterations to the displayed document.
2. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to convert the enhanced document image of the displayed document to a document file format.
3. The system as recited in claim 2 , further comprising instructions that, when executed by the at least one processor, cause the system to replace the content item with the enhanced document image in the document file format within the content management system.
4. The system as recited in claim 1 , wherein generating the enhanced document image for the displayed document within the content item comprises at least one of:
cropping the image of the displayed document; or
altering the image of the displayed document from a color version to a grayscale version.
5. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
associate metadata with the content item that designates the content item as comprising the displayed document based on determining that the content item comprises the image of the displayed document; and
wherein providing the selectable graphical element is based on the metadata associated with the content item.
6. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
provide the enhanced document image in a document file format to the client device; and
store the enhanced document image in the document file format on the content management system in an account associated with the client device.
7. The system as recited in claim 1 ,
the content item using the trained image classifier comprises analyzing the content item using a convolutional neural network.
8. A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause a computer device to:
determine, by a content management system, a content item comprises an image of a displayed document by analyzing the content item using a trained image classifier;
provide, for display on a client device, a graphical user interface comprising a listing of content items, the listing comprising the content item;
provide, for display on the client device, a selectable graphical element based on determining that the content item comprises the image of the displayed document; and
based on receiving an indication of a user selection of the selectable graphical element, generate an enhanced document image of the displayed document, the enhanced document image comprising one or more visual alterations to the displayed document.
9. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to convert the enhanced document image of the displayed document to a document file format.
10. The non-transitory computer-readable medium recited in claim 9 , further comprising instructions that, when executed by the at least one processor, cause the computer device to replace the content item with the enhanced document image in the document file format within the content management system.
11. The non-transitory computer-readable medium recited in claim 8 , wherein generating the enhanced document image for the displayed document within the content item comprises at least one of:
cropping the image of the displayed document to remove portions of the image that are not part of the displayed document; or
altering a color scheme of the image of the displayed document.
12. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to:
provide the enhanced document image in a document file format to the client device; and
store the enhanced document image in the document file format on the content management system in an account associated with the client device.
13. The non-transitory computer-readable medium recited in claim 8 , wherein analyzing the content item using the trained image classifier comprises analyzing the content item using a convolutional neural network.
14. A method comprising:
determining, by a content management system, a content item comprises an image of a displayed document by analyzing the content item using a trained image classifier;
providing, for display on a client device, a graphical user interface comprising a listing of content items, the listing comprising the content item;
providing, for display on the client device, a selectable graphical element based on determining that the content item comprises the image of the displayed document; and
based on receiving an indication of a user selection of the selectable graphical element, generating an enhanced document image of the displayed document, the enhanced document image comprising one or more visual alterations to the displayed document.
15. The method as recited in claim 14 , further comprising receiving, from the client device, the content item, wherein the content item is a captured image from a camera on the client device.
16. The method as recited in claim 14 , further comprising converting the enhanced document image of the displayed document to a document file format.
17. The method as recited in claim 16 , wherein the document file format is a PDF file format.
18. The method as recited in claim 14 , wherein generating the enhanced document image for the displayed document within the content item comprises at least one of:
cropping the image of the displayed document; or
altering the image of the displayed document from a color version to a grayscale version.
19. The method as recited in claim 14 , further comprising:
providing the enhanced document image in a document file format to the client device; and
storing the enhanced document image in the document file format on the content management system in an account associated with the client device.
20. The method as recited in claim 14 , wherein analyzing the content item using the trained image classifier comprises analyzing the content item using a convolutional neural network.