IP Library Granted Patent US 10,515,285
Granted Patent B2
US 10,515,285 · App. 15/616,823 · Granted Dec 24, 2019

Method and apparatus for blocking information from an image

Inventors: Anthony Russell Wilbert (Denver, CO); Hans Brandon Wach (Longmont, CO); David Ching-Chien Chung (San Francisco, CA)
Assignee: Blinker, Inc.
G06K9/325G06K9/36G06T11/60H04W12/02G06K2009/363G06K2209/15G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,515,285
App. No.
15/616,823
Granted
Dec 24, 2019
Kind
B2
Abstract

Some aspects of the invention relate to a mobile apparatus including an image sensor configured to convert an optical image into an electrical signal. The optical image includes an image of a vehicle license plate. The mobile apparatus includes a license plate detector configured to process the electrical signal to recover information from the vehicle license plate image.

Claims (89)

1. A mobile apparatus, comprising:

an image sensor configured to capture an optical image of a vehicle license plate;

a license plate detector configured to:

identify and crop, by a processor, one or more object images from the captured optical image, each of said one or more cropped object images comprising a candidate vehicle license plate image having a plurality of alphanumeric characters,

score, by the processor, each of said cropped one or more object images based on a probability that the respective cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by:

providing a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images,

fitting a line to a center of the respective rectangular fittings of each pair of alphanumeric characters,

for each fitted line, summing a distance of all other respective rectangular fittings to the respective fitted line,

selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate,

computing a score for the candidate vehicle license plate having the selected text baseline based at least on the rectangular fitting around each of the alphanumeric characters, relative positions of each of the alphanumeric characters relative to each other within the respective cropped object image and a calculated slope of the fitted line for the selected pair of alphanumeric characters, and

based on the computed score, block or replace information from the candidate vehicle license plate image when the optical image including the candidate vehicle license plate is displayed on a display.

2. The mobile apparatus of claim 1 , wherein the image sensor is configured to convert the optical image into an electric signal.

3. The mobile apparatus of claim 2 , wherein the license plate detector is further configured to recover the information from a portion of the electrical signal corresponding to a selected one of said identified one or more of the object images.

4. The mobile apparatus of claim 2 , wherein the license plate detector is further configured to process the electrical signal to dewarp at least one of said one or more of the object images.

5. The mobile apparatus of claim 2 , further comprising the display and a rendering module configured to render the optical image to the display and overlay a detection indicator on each of said identified one or more of the object images in the optical image.

6. The mobile apparatus of claim 2 , further comprising an image filter configured to:

apply a set of filter parameters to the electrical signal; and dynamically adjust the parameters based on at least one of color temperature, ambient light, object image location, and the location of the apparatus.

7. A mobile apparatus, comprising:

an image sensor configured to capture an optical image comprising a plurality of object images, wherein one of the plurality of object images comprises an image of a vehicle license plate;

a license plate detector configured to:

identify and crop, by a processor, at least one of the object images from the captured optical image, with the at least one cropped object image comprising a candidate vehicle license plate image having a plurality of alphanumeric characters,

score, by the processor, each of said cropped one or more object images based on a probability that the respective cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by:

providing a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images,

fitting a line to a center of the respective rectangular fittings of each pair of alphanumeric characters,

for each fitted line, summing a distance of all other respective rectangular fittings to the respective fitted line,

selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate,

computing a score for the candidate vehicle license plate having the selected text baseline based at least on the rectangular fitting around each of the alphanumeric characters, relative positions of each of the alphanumeric characters relative to each other within the respective cropped object image and a calculated slope of the fitted line for the selected pair of alphanumeric characters, and

based on the computed score, block or replace information from the candidate vehicle license plate image when the image including the candidate vehicle license plate is displayed on a display.

8. The mobile apparatus of claim 7 , wherein the image sensor is configured to convert the optical image into an electrical signal, and wherein the license plate detector is further configured to process the electrical signal to dewarp said one of the object images prior to recovering the information.

9. The mobile apparatus of claim 8 , wherein the license plate detector is further configured to process the electrical signal to score each of the plurality of object images based on a probability that the each of the plurality of object images comprises the vehicle license plate image.

10. The mobile apparatus of claim 9 , wherein the license plate detector is further configured to select said one of the plurality of object images based on the selected object image's score.

11. The mobile apparatus of claim 7 , further comprising the display and a rendering module configured to render the optical image to the display and overlay a detection indicator on each of the plurality of object images.

12. The mobile apparatus of claim 11 , wherein the rendering module is further configured to overlay a first detection indicator over said one of the plurality of object images and a second detection indicator over the other object images of said plurality of object images.

13. The mobile apparatus of claim 12 , wherein the rendering module is further configured to cause the first detection indicator to appear visually different from the second detection indicator on the display.

14. A mobile apparatus, comprising:

an image sensor configured to capture an optical image of a vehicle license plate;

a display;

a rendering module configured to render the optical image to the display;

and

a license plate detector configured to:

identify and crop, by a processor, one or more object images from the captured optical image, each of said one or more cropped object images comprising a candidate vehicle license plate image having a plurality of alphanumeric characters,

score, by the processor, each of said cropped one or more object images based on a probability that the respective cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by:

providing a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images,

fitting a line to a center of the respective rectangular fittings of each pair of alphanumeric characters,

for each fitted line, summing a distance of all other respective rectangular fittings to the respective fitted line,

selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate,

computing a score for the candidate vehicle license plate having the selected text baseline based at least on the rectangular fitting around each of the alphanumeric characters, relative positions of each of the alphanumeric characters relative to each other within the respective cropped object image and a calculated slope of the fitted line for the selected pair of alphanumeric characters, and

based on the computed score, block or replace information from the candidate vehicle license plate image when the optical image including the candidate vehicle license plate is displayed on the display,

wherein the rendering module is further configured to overlay a detection indicator on the displayed optical image for positioning the apparatus, and

wherein the rendering module is further configured to provide an alert to the display when the license plate detector fails to recover the vehicle license plate information.

15. The mobile apparatus of claim 14 , wherein the optical image is converted into an electrical signal and comprises a plurality of object images, and wherein the license plate detector is further configured to process the electrical signal to identify one or more of the plurality of object images from the image, each of said one or more of the plurality of object images comprising a candidate vehicle license plate image.

16. The mobile apparatus of claim 15 , wherein the license plate detector is further configured to process the electrical signal to dewarp at least one of said one or more of the plurality of object images.

17. A computer program product for a mobile apparatus having an image sensor configured to capture an optical image including a plurality of object images, wherein one of the object images comprises an image of a vehicle license plate, the computer program product comprising:

a non-transitory machine readable medium comprising code to:

select, by a processor, said one of the object images; and

identify and crop, by the processor, one or more object images from the captured optical image, each of said one or more cropped images comprising a candidate vehicle license plate image having a plurality of alphanumeric characters,

score, by the processor, each of said cropped one or more object images based on a probability that the respective cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by:

providing a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images,

fitting a line to a center of the respective rectangular fittings of each pair of alphanumeric characters,

for each fitted line, summing a distance of all other respective rectangular fittings to the respective fitted line,

selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate,

computing a score for the candidate vehicle license plate having the selected text baseline based at least on the rectangular fitting around each of the alphanumeric characters, relative positions of each of the alphanumeric characters relative to each other within the respective cropped object image and a calculated slope of the fitted line for the selected pair of alphanumeric characters, and

based on the computed score, block or replace information from the candidate vehicle license plate image when the image including the candidate vehicle license plate is displayed on a display.

18. The computer program product of claim 17 , wherein the optical image is converted into an electrical signal and the code to process the electrical signal is configured to process the electrical signal to identify one or more of the object images from the captured optical image, each of said one or more of the object images comprising the candidate vehicle license plate image.

19. The computer program product of claim 18 , wherein the code to process the electrical signal is further configured to select said one of the object images from said identified one or more of the object images based on the scores.

20. The computer program product of claim 19 , wherein the code to process the electrical signal is further configured to recover the information from a portion of the electrical signal corresponding to the selected one of said identified one or more of the object images.

21. The computer program product of claim 18 , wherein the code for processing the electrical signal is further configured to dewarp at least one of said one or more of the object images.

22. A mobile apparatus, comprising:

an image sensor configured to capture an optical image of a vehicle license plate;

a timing circuit configured to sample an electrical signal corresponding to the captured optical image at a frame rate;

a license plate detector configured to:

identify and crop, by a processor, one or more object images from the captured optical image, each of said one or more cropped object images comprising a candidate vehicle license plate image having a plurality of alphanumeric characters,

score, by the processor, each of said cropped one or more object images based on a probability that the respective cropped object image comprises the vehicle license plate image, wherein each of the cropped one or more object images is scored by:

providing a rectangular fitting around each character of the plurality of alphanumeric characters in the one or more object images,

fitting a line to a center of the respective rectangular fittings of each pair of alphanumeric characters,

for each fitted line, summing a distance of all other respective rectangular fittings to the respective fitted line,

selecting the respective pair of alphanumeric characters corresponding to the respective fitted line with a smallest summed distance of all of the fitted lines as a text baseline for the candidate vehicle license plate,

computing a score for the candidate vehicle license plate having the selected text baseline based at least on the rectangular fitting around each of the alphanumeric characters, relative positions of each of the alphanumeric characters relative to each other within the respective cropped object image and a calculated slope of the fitted line for the selected pair of alphanumeric characters, and

based on the computed score, block or replace information from the candidate vehicle license plate image when the optical image including the candidate vehicle license plate is displayed on a display.

23. The apparatus of claim 22 , wherein the sampled electrical signal comprises a plurality of samples, and wherein the license plate detector is further configured to identify one or more object images from at least one of the plurality of samples.

24. The apparatus of claim 23 , wherein at least one of said one or more of the object images comprising a candidate vehicle license plate image.

25. The apparatus of claim 24 , wherein the license plate detector is further configured to process said at least one of the plurality of samples to score each of said one or more of the object images based on a probability that the object image comprises the vehicle license plate image.

26. The apparatus of claim 25 , wherein the license plate detector is further configured to process at least first and second samples of the plurality of samples to score each of said one or more of the object images from each of said at least first and second samples.

27. The apparatus of claim 26 , wherein each of said one or more object images detected in the first sample is associated with location information, and wherein the license plate detector is further configured to use the location information to estimate the location of said one or more of the objects in the second sample.

28. The apparatus of claim 25 , wherein the license plate detector is further configured to stop processing the sampled electrical signal when said score is above a threshold value.

29. The apparatus of claim 23 , further comprising

the display, wherein each sample represents a frame of a video image presented to the display; and

a rendering module configured to render each frame of the video image to the display and overlay a detection indicator on each of said identified one or more of the object images.

30. The apparatus of claim 29 , wherein the overlay provides a visual indication of the object image associated with the computed score that is above a threshold value.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded May 10, 2021
From: GMIT LENDING COMPANY, LLC
To: BLINKER, INC.
Reel/Frame 056183/0119 →
SECURITY INTEREST Recorded Feb 18, 2019
From: BLINKER INC.
To: GMIT LENDING COMPANY, LLC
Reel/Frame 048361/0613 →
SECURITY INTEREST Recorded Feb 18, 2019
From: BLINKER INC.
To: GMIT LENDING COMPANY, LLC
Reel/Frame 048361/0991 →
SECURITY INTEREST Recorded Jan 23, 2018
From: BLINKER INC.,
To: GMIT LENDING COMPANY, LLC,
Reel/Frame 045117/0662 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2017
From: WILBERT, ANTHONY RUSSELL; WACH, HANS BRANDON; CHUNG, DAVID CHING-CHIEN
To: BLINKER INC.
Reel/Frame 042830/0732 →
Continuity (4)
Continuation In Part 14613323 · Feb 3, 2015
Continuation In Part 14455841 · Aug 8, 2014
Continuation In Part 14318397 · Jun 27, 2014
Related Publication 20170270384A1 · Sep 21, 2017