IP Library Granted Patent US 12,039,252
Granted Patent B2
US 12,039,252 · App. 18/295,049 · Granted Jul 16, 2024

Enhancing documents portrayed in digital images

Inventors: Nils Peter Welinder (San Francisco, CA); Peter N Belhumeur (San Francisco, CA); Ying Xiong (Sunnyvale, CA); Jongmin Baek (San Jose, CA); Simon Kozlov (San Francisco, CA); Thomas Berg (San Francisco, CA); David J Kriegman (San Diego, CA)
Assignee: Dropbox, Inc.
G06F40/123G06F16/93G06F40/106G06F40/166G06F40/197G06T5/90G06T7/194G06T11/001G06V10/10G06V10/44G06V20/46G06V30/40G06V30/413G06V30/414G06F3/04842G06F3/126G06F40/103G06F40/169G06T5/70G06T11/60G06T2207/20056G06T2207/20081G06T2207/20084G06T2207/30176G06T2210/22G06V10/242G06V10/30G06V30/418
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,039,252
App. No.
18/295,049
Granted
Jul 16, 2024
Kind
B2
Abstract

The present disclosure is directed toward systems and methods that efficiently and effectively generate an enhanced document image of a displayed document in an image frame captured from a live image feed. For example, systems and methods described herein apply a document enhancement process to a displayed document in an image frame that result in an enhanced document image that is cropped, rectified, un-shadowed, and with dark text against a mostly white background. Additionally, systems and method described herein determine whether a stored digital content item includes a displayed document. In response to determining that a stored digital content item does include a displayed document, systems and methods described herein generate an enhanced document image of a displayed document included in the stored digital content item.

Claims (55)

1. A system comprising:

at least one processor; and

a non-transitory computer-readable medium storing instructions thereon that, when executed by the at least one processor, cause the system to:

provide, within a graphical user interface comprising a live camera feed, a live document boundary indicator associated with a visual representation of a physical document within the live camera feed;

capture, from the live camera feed, an image frame comprising the visual representation of the physical document;

generate an adjustable boundary indicator to display in association with the visual representation of the physical document in the image frame;

modify the adjustable boundary indicator based on receiving an indication of a user interaction with the adjustable boundary indicator; and

generate an enhanced document image comprising a modified visual representation of the physical document based on converting the visual representation to grayscale and based on the modified adjustable boundary indicator.

2. The system as recited in claim 1 , wherein modifying the adjustable boundary indicator comprises modifying a size of the adjustable boundary indicator.

3. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

provide a selectable black and white graphical element for display in association with the visual representation of the physical document in the image frame; and

wherein generating the enhanced document image comprises generating a black and white document image based on a selection of the selectable black and white graphical element.

4. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

provide a selectable original graphical element for display in association with the visual representation of the physical document in the image frame; and

wherein generating the enhanced document image comprises generating an original color document image based on a selection of the original graphical element.

5. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

provide a contrast control graphical element for display in association with the visual representation of the physical document in the image frame; and

wherein generating the enhanced document image comprises modifying a contrast attribute based on detecting a user interaction with the contrast control graphical element.

6. The system as recited in claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to convert the enhanced document image into a document file.

7. The system as recited in claim 6 , further comprising instructions that, when executed by the at least one processor, cause the system to:

provide a storage configuration graphical user interface after converting the enhanced document image into a document file; and

in response to receiving one or more user interactions with the storage configuration graphical user interface, storing the document file in a user-specified location.

8. A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause a computer device to:

provide, within a graphical user interface comprising a live camera feed, a live document boundary indicator associated with a visual representation of a physical document within the live camera feed;

capture, from the live camera feed, an image frame comprising the visual representation of the physical document;

generate an adjustable boundary indicator to display in association with the visual representation of the physical document in the image frame;

modify the adjustable boundary indicator based on receiving an indication of a user interaction with the adjustable boundary indicator;

generate an enhanced document image comprising a modified visual representation of the physical document based on converting the visual representation to grayscale and based on the modified adjustable boundary indicator; and

convert the enhanced document image into a document file.

9. The non-transitory computer-readable medium recited in claim 8 , wherein modifying the adjustable boundary indicator comprises modifying a size of the adjustable boundary indicator to more correctly align with an edge of the visual representation of the physical document.

10. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to:

provide a selectable black and white graphical element for display in association with the visual representation of the physical document in the image frame; and

wherein generating the enhanced document image comprises generating a black and white version of the enhanced document image based on a selection of the selectable black and white graphical element.

11. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to:

provide a selectable original graphical element for display in association with the visual representation of the physical document in the image frame; and

wherein generating the enhanced document image comprises generating an original color version of the enhanced document image based on a selection of the original graphical element.

12. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to:

provide a contrast control graphical element for display in association with the visual representation of the physical document in the image frame; and

wherein generating the enhanced document image comprises modifying a contrast attribute based on detecting a user interaction with the contrast control graphical element.

13. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to:

provide a storage configuration graphical user interface after converting the enhanced document image into a document file; and

in response to receiving one or more user interactions with the storage configuration graphical user interface, storing the document file in either a local storage or a remote storage.

14. The non-transitory computer-readable medium recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer device to upload the document file to an online content management system.

15. A computer-implemented method comprising:

capturing, from a live camera feed, an image frame comprising a visual representation of a physical document;

generating an adjustable boundary indicator to display in association with the visual representation of the physical document in the image frame;

modifying the adjustable boundary indicator based on receiving an indication of a user interaction with the adjustable boundary indicator; and

generating an enhanced document image comprising a modified visual representation of the physical document based on converting the visual representation to grayscale and based on the modified adjustable boundary indicator.

16. The computer-implemented method as recited in claim 15 , further comprising providing a preview of the enhanced document image for display based on the modified adjustable boundary indicator.

17. The computer-implemented method as recited in claim 16 , further comprising providing color editing options within a graphical user interface, the color editing options allowing a user to provide input to edit a color attribute of the enhanced document image.

18. The computer-implemented method as recited in claim 17 , where the color editing options comprise at least one of a black and white option, an original color option, or a grayscale option.

19. The computer-implemented method as recited in claim 15 , further comprising converting the enhanced document image to a document file.

20. The computer-implemented method as recited in claim 19 , further comprising:

providing a storage configuration graphical user interface after converting the enhanced document image into a document file; and

in response to receiving one or more user interactions with the storage configuration graphical user interface, storing the document file in a user-specified location.

Assignments (1)
SECURITY INTEREST Recorded Dec 12, 2024
From: DROPBOX, INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 069604/0611 →