IP Library Granted Patent US 7,868,922
Granted Patent B2
US 7,868,922 · App. 11/573,713 · Granted Jan 11, 2011

Foreground/background segmentation in digital images

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,868,922
App. No.
11/573,713
Granted
Jan 11, 2011
Kind
B2
Abstract

An implementation efficient method of distinguishing between foreground and background regions of a digital image of a scene includes capturing two images of nominally the same scene and storing the captured images in DCT-coded format. The first image is taken with the foreground more in focus than the background and the second image is taken with the background more in focus than the foreground. Regions of the first image are assigned as foreground or background according to whether the sum of selected higher order DCT coefficients decreases or increases for the equivalent regions of the second image.

Claims (40)

1. A method of distinguishing between foreground and background regions of a digital image of a scene, the method comprising: capturing first and second images of nominally the same scene, the first image being taken with the foreground more in focus than the background and the second image being taken with the background more in focus than the foreground; storing the captured images in DCT-coded format; assigning regions of the first image as foreground or background according to whether a difference between sums of selected higher order DCT coefficients for the first and second images is positive or negative; and rendering in accordance with the assigning one or more processed images based on the first image or the second image, or both, at a digital rendering device, display or printer, or combinations thereof.

2. The method of claim 1 , wherein regions are assigned as foreground when the difference is negative and as background when the difference is positive.

3. The method of claim 1 , wherein the assigning further comprises: (i) calculating the sum of selected high order coefficients for a plurality of DCT blocks of the first image, (ii) calculating the sum of the same high order coefficients for a plurality of DCT blocks of the second image, (iii) comparing the sum calculated in step (i) for each DCT block of the first image with the sum calculated in step (ii) for the DCT block of the equivalent region of the second image, (iv) if the difference between the sums calculated for the first and second images is negative, assigning the given block as corresponding to a foreground region of the image, and (v) if the difference between the sums calculated for the first and second images is positive, assigning that block as corresponding to a background region of the image.

4. The method of claim 3 , further comprising aligning said first and second images.

5. The method of claim 3 , further comprising creating a digital map from said assigning said blocks as corresponding to background or foreground.

6. The method of claim 3 , wherein the first and second images have different pixel resolutions.

7. The method of claim 6 , further comprising making the image resolution of said first and second images equal.

8. The method of claim 6 , wherein the first and second images are captured by a digital camera.

9. The method of claim 8 , wherein the first image is a relatively high resolution image, and wherein the second image is a relatively low resolution pre- or post-view version of the first image.

10. The method of claim 8 , wherein said calculating and comparing is performed as a post processing stage in an external device to said digital camera.

11. The method of claim 8 , wherein said calculating and comparing is performed as a post processing stage in said digital camera.

12. The method of claim 3 , wherein the first and second images have the same pixel resolution, and wherein the first and second images are captured by a digital camera and are relatively low resolution pre- and/or post-view versions of a higher resolution image of said scene also captured by the camera.

13. The method of claim 3 , wherein the selected high order coefficients are the top 25% of coefficients.

14. An image acquisition apparatus, comprising: including an imaging optical system; and a digital image detector; a processor; and a non-transitory, processor-readable storage medium having stored therein program instructions for programming the processor to perform a method of distinguishing between foreground and background regions of a digital image of a scene, wherein the program instructions stored in the storage medium are configured to program the apparatus to capture first and second images of nominally the same scene, the first image being taken with the foreground more in focus than the background and the second image being taken with the background more in focus than the foreground; and to store the captured images in DCT-coded format; and to assign one or more regions of one or both of the first and second images as foreground or background according to whether a difference between sums of selected higher order DCT coefficients for the first and second images is positive or negative.

15. The apparatus of claim 14 , wherein the program instructions stored in the storage medium are configured to program the apparatus to create a digital map from said assigning said blocks as corresponding to background or foreground.

16. The apparatus of claim 14 , wherein the program instructions stored in the storage medium are configured to program the apparatus to make the image resolution of said first and second images equal.

17. The apparatus of claim 14 , wherein the program instructions stored in the storage medium are configured to program the apparatus to capture first and second images of nominally the same scene and to store the captured images in DCT-coded format, the first image being taken with the foreground more in focus than the background and the second image being taken with the background more in focus than the foreground, and to assign regions of the first image as foreground or background according to whether a difference between sums of selected higher order DCT coefficients for the first and second images is positive or negative.

18. The apparatus of claim 14 , wherein regions are assigned as foreground when the difference is negative and as background when the difference is positive.

19. The apparatus of claim 14 , wherein the processor-readable code configured to program the apparatus to assign is configured to program the apparatus to (i) calculate the sum of selected high order coefficients for a plurality of DCT blocks of the first image, and to (ii) calculating the sum of the same high order coefficients for a plurality of DCT blocks of the second image, and to (iii) compare the sum calculated in (i) for each DCT block of the first image with the sum calculated in (ii) for the DCT block of the equivalent region of the second image, (iv) if the difference between the sums calculated for the first and second images is negative, then the processor-readable code is configured to program the apparatus to assign the given block as corresponding to a foreground region of the image, and (v) if the difference between the sums calculated for the first and second images is positive, then the processor-readable code is configured to program the apparatus to assign that block as corresponding to a background region of the image.

20. The apparatus of claim 19 , wherein the program instructions stored in the storage medium are configured to program the apparatus to align said first and second images.

21. The apparatus of claim 19 , wherein the first and second images have different pixel resolutions.

22. The apparatus of claim 19 , wherein the first and second images are captured by a digital camera.

23. The apparatus of claim 22 , wherein the first image is a relatively high resolution image, and wherein the second image is a relatively low resolution pre- or post-view version of the first image.

24. The apparatus of claim 22 , wherein said calculating and comparing is performed as a post processing stage in an external device to said digital camera.

25. The apparatus of claim 22 , wherein said calculating and comparing is performed as a post processing stage in said digital camera.

26. The apparatus of claim 14 , wherein the first and second images have the same pixel resolution, and wherein the first and second images are captured by a digital camera and are relatively low resolution pre- and/or post-view versions of a higher resolution image of said scene also captured by the camera.

27. The apparatus of claim 14 , wherein the selected high order coefficients are the top 25% of coefficients.

28. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of distinguishing between foreground and background regions of a digital image of a scene, wherein the processor-readable code is configured to program the one or more processors to capture first and second images of nominally the same scene, the first image being taken with the foreground more in focus than the background and the second image being taken with the background more in focus than the foreground; and to store the captured images in DCT-coded format; and to assign regions of one or both of the first and second image as foreground or background according to whether a difference between one or more selected higher order DCT coefficients for equivalent regions of the first and second image is positive or negative.

29. The one or more storage devices of claim 28 , wherein the processor-readable code is configured to program the one or more processors to create a digital map from said assigning said blocks as corresponding to background or foreground.

30. The one or more storage devices of claim 28 , wherein the processor-readable code is configured to program the one or more processors to align the first and second images.

31. The one or more storage devices of claim 28 , wherein regions are assigned as foreground when the difference is negative and as background when the difference is positive.

32. The one or more storage devices of claim 28 , wherein the processor-readable code configured to program the one or more processors to assign is configured to program the one or more processors to (i) calculate the sum of selected high order coefficients for a plurality of DCT blocks of the first image, and to (ii) calculating the sum of the same high order coefficients for a plurality of DCT blocks of the second image, and to (iii) compare the sum calculated in (i) for each DCT block of the first image with the sum calculated in (ii) for the DCT block of the equivalent region of the second image, (iv) if the difference between the sums calculated for the first and second images is negative, assigning the given block as corresponding to a foreground region of the image, and (v) if the difference between the sums calculated for the first and second images is positive, assigning that block as corresponding to a background region of the image.

33. The one or more storage devices of claim 32 , wherein the first and second images have different pixel resolutions.

34. The one or more storage devices of claim 33 , wherein the processor-readable code is configure to program the one or more processors to make the image resolution of said first and second images equal.

35. The one or more storage devices of claim 32 , wherein the first and second images are captured by a digital camera.

36. The one or more storage devices of claim 35 , wherein the first image is a relatively high resolution image, and wherein the second image is a relatively low resolution pre- or post-view version of the first image.

37. The one or more storage devices of claim 35 , wherein the processor-readable code is configured to program the one or more processors to calculate and compare as a post processing stage in an external device to said digital camera.

38. The one or more storage devices of claim 35 , wherein the processor-readable code is configured to program the one or more processors to calculate and compare as a post processing stage in said digital camera.

39. The one or more storage devices of claim 32 , wherein the first and second images have the same pixel resolution, and wherein the first and second images are captured by a digital camera and are relatively low resolution pre- and/or post-view versions of a higher resolution image of said scene also captured by the camera.

40. The one or more storage devices of claim 32 , wherein the selected high order coefficients comprise the top 25% of coefficients.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2025
From: FOTONATION LIMITED
To: ADEIA IMAGING LLC
Reel/Frame 073635/0320 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
CHANGE OF NAME Recorded Dec 3, 2014
From: DIGITALOPTICS CORPORATION EUROPE LIMITED
To: FOTONATION LIMITED
Reel/Frame 034524/0330 →
CHANGE OF NAME Recorded May 17, 2012
From: TESSERA TECHNOLOGIES IRELAND LIMITED
To: DIGITALOPTICS CORPORATION EUROPE LIMITED
Reel/Frame 028221/0644 →