Robust cropping of license plate images
View Patent ↗A method, system, and computer-usable tangible storage device for robustly cropping and accurately recognizing license plates to account for noise sources and interfering artifacts are disclosed. License plate images and sub-images can be tightly cropped utilizing an image-based classifier and gradient-based cropping. An image-based classifier can identify the location of valid characters within the image. Because of a number of noise sources, such as, for example, residual plate rotation and shear in the characters within the image, the image-based classifier performs a “rough” identification of the image boundaries. An additional processing step utilizing gradient-based cropping is performed to fine-tune the license plate image boundaries. Gradient-based cropping eliminates unwanted border artifacts that could substantially impact the segmentation and license plate character recognition results.
1. A method for cropping of a license plate image, said method comprising:
identifying a first plurality of details of said license plate image utilizing an image-based classifier that serves as a valid character detector to indicate a subset of said license plate image where character images likely reside wherein said first plurality of details comprise at least one of: a character, a valid license plate character, a symbol, a logo, a license plate border, a license plate border with poor contrast, and extraneous edge information outside of said license plate;
resizing said subset of said license plate image where character images likely reside to a standard vertical dimension associated with a classifier template;
identifying candidate character locations of said license plate image and determining if said candidate character locations exceed a minimum number of characters;
performing a first cropping of said license plate image including removing tilt from and vertically cropping said license plate image;
identifying a second detail of said license plate image utilizing a gradient-based classifier on said candidate character locations and performing a second cropping of said first cropping of said license plate image based on detected character locations, further comprising a horizontal cropping via projective segmentation of said license plate image based on said detected character locations, if said candidate character locations exceed said minimum number of characters; and
utilizing said identified first plurality of details and said identified second detail to recognize and identify a license plate in said license plate image;
further comprising identifying said first plurality of details and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for the following: varying distances between characters on said license plate, a license plate border, a symbol, and logos on said license plate.
2. The method of claim 1 wherein said plurality of details comprise a at least one noise source, said at least one noise source comprising at least one of: plate rotation, shear in a character within said license plate image, a shadow, non-uniform illumination, tilt, shear, projective distortion, a license plate frame, a license plate sticker, a license plate frame that covers or partially covers a character on said license plate, a license plate sticker that covers or partially covers a character on said license plate, partial occlusion of a character, a trailer hitch ball, and salt and pepper noise.
3. The method of claim 1 wherein identifying said second detail of said license plate image utilizing said gradient-based classifier further comprises fine-tuning said second detail for robustness for a license plate frame, a license plate border artifact, and character shear to improve segmentation and character recognition results.
4. The method of claim 3 wherein said second detail comprises a plurality of details comprising at least one of: a character, a valid license plate character, a symbol, a license plate special symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.
5. The method of claim 1 further comprising:
cropping said license plate image with said image-based classifier and said gradient-based classifier to create a license plate sub-image, wherein cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;
at least one of removing a logo from said license plate sub-image, removing a symbol from said license plate sub-image, and flagging an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;
identifying a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;
capturing said license plate image; and
localizing said license plate image.
6. The method of claim 1 further comprising:
extracting a bounding box license plate sub-image to reduce noise present in said license plate image;
extracting a license plate character from said bounding box license plate sub-image via the following: tilt removal, vertical cropping, and horizontal cropping; and
segmenting said license plate character in said bounding box license plate sub-image to recognize said license plate character and identify said license plate.
7. A system for cropping of a license plate image, comprising:
a processor that communicates electronically with a memory; and
a non-transitory computer-usable medium embodying computer program code stored in said memory, said non-transitory computer-usable medium capable of communicating with said processor, said computer program code comprising instructions executable by said processor and configured for:
identifying a first plurality of details of said license plate image utilizing an image-based classifier that serves as a valid character detector to indicate a subset of said license plate image where character images likely reside wherein said first plurality of details comprise at least one of:
a character, a valid license plate character, a symbol, a special license plate symbol, a logo, a license plate border, a license plate border with poor contrast, and extraneous edge information outside of said license plate;
resizing said subset of said license plate image where character images likely reside to a standard vertical dimension associated with a classifier template;
identifying candidate character locations of said license plate image and determining if said candidate character locations exceed a minimum number of characters;
performing a first cropping of said license plate image including removing tilt from and vertically cropping said license plate image;
identifying a second detail of said license plate image utilizing a gradient-based classifier on said candidate character locations and performing a second cropping of said first cropping of said license plate image based on detected character locations, further comprising a horizontal cropping via projective segmentation of said license plate image based on said detected character locations, if said candidate character locations exceed said minimum number of characters; and
utilizing said identified first plurality of details and said identified second detail to recognize and identify a license plate in said license plate image;
further comprising identifying said first plurality of details and said second detail utilizing a Sparse Network of Winnows classifier, wherein said Sparse Network of Winnows classifier accounts for the following: varying distances between characters on said license plate, a license plate border, a symbol, and logos on said license plate.
8. The system of claim 7 wherein said instructions are further configured to fine-tune said second detail for robustness for a license plate frame, a license plate border artifact, and character shear to improve segmentation and character recognition results, wherein said second detail comprises a plurality of details comprising at least one of: a character, a valid license plate character, a symbol, a special license plate symbol, a logo, a license plate border, a license plate border with poor contrast, no detectable plate border, and extraneous edge information outside of said license plate.
9. The system of claim 7 wherein said instructions are further configured for:
cropping said license plate image to create a license plate sub-image utilizing said image-based classifier and said gradient-based classifier, wherein cropping said license plate image comprises horizontally cropping said license plate image to account for a noise source and an interfering artifact in said license plate image;
removing a logo from said license plate sub-image, program instructions to remove a symbol from said license plate sub-image, and program instructions to flag an artifact from said license plate sub-image before submitting said license plate sub-image for character recognition;
identifying a license plate edge artifact utilizing said gradient-based classifier on said license plate sub-image to determine a license plate border artifact for removal without incorrectly discarding a valid license plate character;
capturing said license plate image; and
localizing said license plate image.
10. The system of claim 7 wherein said instructions are further configured for:
extracting a bounding box license plate sub-image to reduce noise present in said license plate image;
extracting a license plate character from said bounding box license plate sub-image via at least one of: tilt removal, vertical cropping, and horizontal cropping; and
segmenting said license plate character in said bounding box license plate sub-image to recognize said license plate character and identify said license plate.