Method and apparatus for performing a clean background subtraction
View Patent ↗A background subtraction apparatus of the present invention includes a key point locator for locating key points on a known object type, a boundary point locator for locating boundary points of the known object that make up the edges of the known object, and an edge processor for processing the edges to provide a clean-edged extraction of the known object from a background image. Preferably, the key point locator includes an alignment detector for detecting alignment of an image of the known object type with a skeleton image. Still more preferably, the skeleton image is an exoskeleton image and the known object type is a human being.
1. A method for subtracting a background comprising:
locating key points on an image of a known object type;
determining line segments between the key points;
locating boundary points of the image that comprise edges of the known object by using the line segments;
eliminating pixel gaps between the boundary points;
connecting the boundary points to obtain a closed loop of pixels; processing the edges to provide a clean-edged extraction of the image from a background image, including using an alpha image to determine a key point, which is selected based on a similarity score for a region in the alpha image; and
reducing noise from the edges of the known object, after determining the key points, using an edge strength, the alpha image, and the line segments by aligning a snake to the line segments;
wherein at least one key point is located based at least in part on a key point template comprising a plurality of pixels and specially configured for that key point; and
wherein the pixels in the specially configured key point template that make up a shape associated with the key point have a first value and the other pixels in the specially configured key point template have a second value.
2. A method for subtracting a background as recited in claim 1 further including aligning the image with a skeleton image prior to locating the key points.
3. A method for subtracting a background as recited in claim 1 wherein locating boundary points includes scanning the image in directions dependent upon the orientation of the line segments.
4. A method for subtracting a background as recited in claim 1 wherein the key points include primary key points and secondary key points located based on the primary key points.
5. A method for subtracting a background as recited in claim 1 wherein locating the key points includes using a general search area.
6. A method for subtracting a background as recited in claim 1 wherein locating the key points includes using a general search area and a Summed Area Table.
7. A method for subtracting a background as recited in claim 1 wherein locating the key points includes:
locating primary key points;
determining whether the primary key points are in acceptable locations;
repeating locating primary key points if the primary key points are determined not to be in acceptable locations;
locating secondary key points based on the primary points;
determining whether the secondary points are in acceptable locations;
repeating locating secondary key points if the secondary key points are determined not to be in acceptable locations.
8. A method for subtracting a background as recited in claim 1 further including aligning the image of the known object type with a skeleton image prior to locating the key points, wherein the skeleton image is an outline image.
9. A method for subtracting a background as recited in claim 1 further including:
aligning the image of the known object type with a skeleton image prior to locating the key points;
scanning the image in directions dependent upon the orientation of the line segments, beginning at an offset away from the skeleton image.
10. A method for subtracting a background as recited in claim 1 further including:
aligning the image with a skeleton image prior to locating the key points;
scanning the image in directions dependent upon the orientation of the line segments, beginning at an offset away from the skeleton image;
wherein the offset is determined by a bounding box whose corners are set by the key points.
11. A method for subtracting a background as recited in claim 1 further including:
aligning the image with a skeleton image prior to locating the key points;
scanning the image in directions dependent upon the orientation of the line segments, beginning at an offset away from the skeleton image;
wherein the offset is determined by a bounding box whose corners are set by the key points plus a pad of pixels.
12. A method for subtracting a background as recited in claim 1 further including aligning the image with a skeleton image prior to locating the key points, wherein the skeleton image is an outline image and the known object type is a human being.
13. A method for subtracting a background as recited in claim 1 further including aligning the image with a skeleton image prior to locating the key points, wherein:
the skeleton image is an outline image;
the known object type is a human being; and
the key points include primary key points at extremities of the image and secondary key points near a torso of the image.
14. A method for subtracting a background as recited in claim 1 wherein processing the edges includes filtering the edges.
15. A background subtraction apparatus comprising:
an image acquisition device for acquiring an image of a known object type; and
a processor configured to:
locate key points on the image;
determine line segments between the key points;
locate boundary points of the image that comprise edges of the known object by using the line segments;
eliminate pixel gaps between the boundary points;
connect the boundary points to obtain a closed loop of pixels;
process the edges to provide a clean-edged extraction of the image from a background image including using an alpha image to determine a key point, which is selected based on a similarity score for a region in the alpha image; and
reduce noise from the edges of the known object, after determining the key points, using an edge strength, the alpha image, and the line segments by aligning a snake to the line segments;
wherein at least one key point is located based at least in part on a key point template comprising a plurality of pixels and specially configured for that key point; and
wherein the pixels in the specially configured key point template that make up a shape associated with the key point have a first value and the other pixels in the specially configured key point template have a second value.
16. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to align the image with a skeleton image.
17. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to scan the image in directions dependent upon the orientation of the line segments.
18. A background subtraction apparatus as recited in claim 15 wherein the key points include primary key points and secondary key points located based on the primary key points.
19. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to locate the key points using a general search area.
20. A background subtraction apparatus as recited in claim 15 the processor is further configured to locate the key points using a general search area and a Summed Area Table.
21. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to:
locate primary key points;
repeat locating primary key points if the primary key points are determined not to be in acceptable locations;
locate secondary key points based on the primary key points;
determine whether the secondary key points are in acceptable locations;
repeat locating secondary key points if the secondary key points are determined not to be in acceptable locations.
22. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to align the image of the known object type with a skeleton image, wherein the skeleton image is an outline image.
23. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to:
align the image of the known object type with a skeleton image prior to locating the key points;
scan the image in directions dependent upon the orientation of the line segments, beginning at an offset away from the skeleton image.
24. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to:
align the image with a skeleton image prior to locating the key points;
scan the image in directions dependent upon the orientation of the line segments, beginning at an offset away from the skeleton image;
wherein the offset is determined by a bounding box whose corners are set by the key points.
25. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to:
align the image with a skeleton image prior to locating the key points;
scan the image in directions dependent upon the orientation of the line segments, beginning at an offset away from the skeleton image;
wherein the offset is determined by a bounding box whose corners are set by the key points plus a pad of pixels.
26. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to align the image with a skeleton image, wherein the skeleton image is an outline image and the known object type is a human being.
27. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to align the image with a skeleton image, wherein:
the skeleton image is an outline image;
the known object type is a human being; and
the key points include primary key points at extremities of the image and secondary key points near a torso of the image.
28. A background subtraction apparatus as recited in claim 15 wherein the processor is further configured to filter the edges.