IP Library Granted Patent US 10,621,464
Granted Patent B1
US 10,621,464 · App. 16/429,440 · Granted Apr 14, 2020

Block based non-maximum suppression

Inventors: Elliot N. Linzer (Bergenfield, NY); Guy Rapaport (Sunnyvale, CA); Leslie D. Kohn (Saratoga, CA); Yu Wang (San Jose, CA)
Assignee: Ambarella International LP
G06K9/4604G06K9/4642G06K9/6202
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,621,464
App. No.
16/429,440
Granted
Apr 14, 2020
Kind
B1
Abstract

An apparatus includes an interface and a processor. The interface may be configured to receive an array of scores. The processor may be configured to (i) parse the array of scores into a data flow including one or more operators, (ii) schedule the operators in one or more data paths, (iii) divide the array of scores into blocks of two or more scores by processing the array of scores using the one or more data paths, (iv) compute per-block information using the one or more data paths, and (v) compute local maxima position information for the array of scores by performing block based non-maximum suppression based on the per-block information using the one or more data paths. The data paths are generally implemented with a plurality of hardware circuits.

Claims (53)

1. An apparatus comprising:

an interface configured to receive an array of scores; and

a processor configured to (i) parse said array of scores into a data flow including one or more operators, (ii) schedule said operators in one or more data paths, (iii) divide said array of scores into blocks of two or more scores by processing said array of scores using said one or more data paths, (iv) compute per-block information using said one or more data paths, and (v) compute local maxima position information for said array of scores by performing block based non-maximum suppression based on said per-block information using said one or more data paths, wherein said data paths are implemented with a plurality of hardware circuits.

2. The apparatus according to claim 1 , further comprising a common memory configured as (i) one or more input data structures that store said array of scores and (ii) one or more output data structures that receive one or more of said per-block information or said local maxima position information from said processor.

3. The apparatus according to claim 1 , wherein:

said array of scores corresponds to a portion of pixels of an image containing one or more objects or features; and

said scores represent a likelihood that a corresponding pixel is a reference point for a region of a predetermined size containing a particular object or feature.

4. The apparatus according to claim 3 , wherein said region is rectangular in shape and said reference point is an upper-left corner of said region.

5. The apparatus according to claim 1 , wherein said processor is part of a computer vision system.

6. The apparatus according to claim 1 , wherein said blocks are multi-dimensional.

7. The apparatus according to claim 1 , wherein said per-block information comprises one or more of:

a maximum score for each of said blocks; and

a location within each of said blocks containing said maximum score.

8. The apparatus according to claim 1 , wherein said processor is further configured to store said per block information instead of said scores.

9. The apparatus according to claim 1 , wherein said per block information includes at least one of (i) a maximum score for at least one block or (ii) a location of a maximum score for at least one block.

10. The apparatus according to claim 1 , wherein:

said data paths are implemented with a plurality of hardware engines; and

said hardware engines operate in parallel to each other.

11. A method of improving object or feature detection using a processor circuit to apply a block based non-maximum suppression technique comprising the steps of:

dividing an array of scores into blocks of two or more scores; and

computing per-block information using one or more data paths of said processor circuit, wherein said data paths are implemented with a plurality of hardware circuits.

12. The method according to claim 11 , wherein said block based non-maximum suppression technique suppresses scores that are below a determined local maximum score based on a predefined suppression region.

13. The method according to claim 11 , wherein said blocks are multi-dimensional.

14. The method according to claim 11 , wherein computing said per-block information comprises one or more of:

identifying a maximum score for each of said blocks; and

identifying a location within each of said blocks containing said maximum score.

15. The method according to claim 14 , wherein said block based non-maximum suppression technique provides an approximate non-maximum suppression comprising:

using block-based calculations to determine whether a center score of a predefined suppression region is the maximum within the block containing the center score based on said per block information identifying the location within each of said blocks containing said maximum score;

using block-based calculations to determine whether the block that contains the center score has a highest maximum of all blocks that are either fully within the predefined suppression region or have a predefined number of scores within the predefined suppression region based on per block information indicating the maximum score of each block; and

designating the center score as a local maximum when both of the above determinations are true.

16. The method according to claim 14 , wherein said block based non-maximum suppression technique provides an approximate non-maximum suppression comprising:

using block-based calculations to determine whether a center score of a predefined suppression region is the maximum within the block containing the center score based on said per block information identifying the location within each of said blocks containing said maximum score;

using block-based calculations to determine whether the block that contains the center score has a highest maximum of all blocks that are either fully or partially within the predefined suppression region based on per block information indicating the maximum score of each block; and

designating the center score as a local maximum when both of the above determinations are true.

17. The method according to claim 14 , wherein said block based non-maximum suppression technique provides an approximate non-maximum suppression comprising:

using block-based calculations to determine whether a center score of a predefined suppression region is the maximum within the block containing the center score based on said per block information identifying the location within each of said blocks containing said maximum score;

using block-based calculations to determine whether the block that contains the center score has a highest maximum of all blocks fully within the predefined suppression region based on per block information indicating the maximum score of each block fully within the predefined suppression region;

using block-based calculations to determine whether any block partially within the predefined suppression region either (i) has per block information indicating the maximum score of the block is lower than the block containing the center score or (ii) has per block information identifying the location within the block containing a largest score is not within the predefined suppression region; and

designating the center score as a local maximum when all of the above determinations are true.

18. The method according to claim 14 , wherein said block based non-maximum suppression technique provides an approximate non-maximum suppression comprising:

(A) using block-based calculations to determine whether a center score of a predefined suppression region is the maximum within the block containing the center score based on said per block information identifying the location within each of said blocks containing said maximum score;

(B) using block-based calculations to determine whether the block that contains the center score has a highest maximum of all blocks that are fully within the predefined suppression region based on per block information indicating the maximum score of each block;

(C) for each block that is only partially within the predefined suppression region,

if a number of scores in a respective block that are in the predefined suppression region is greater than a predefined threshold, using block-based calculations to determine whether the block that contains the center score has a higher maximum than the respective block based on per block information indicating the maximum score of each block, or

if the number of scores in the respective block that are in the predefined suppression region is not greater than the predefined threshold, using block-based calculations to determine whether the respective block either (a) has per block information indicating the maximum score of the respective block is lower than the block containing the center score or (b) has per block information identifying the location within the respective block containing a largest score is not within the predefined suppression region; and

(D) designating the center score as a local maximum when all of the above determinations are true.

19. The method according to claim 14 , wherein said block based non-maximum suppression technique provides an exact non-maximum suppression comprising:

using block-based calculations to determine whether a center score of a predefined suppression region is the maximum within the block containing the center score based on said per block information identifying the location within each of said blocks containing said maximum score;

using block-based calculations to determine whether the block that contains the center score has a highest maximum of all blocks fully within the predefined suppression region;

using score-based calculations to determine whether the center score is larger than all individual scores that are both in the predefined suppression region and also within blocks that are only partially within the predefined suppression region; and

designating the center score as a local maximum when all of the above determinations are true.

20. The method according to claim 11 , further comprising:

storing said per block information instead of said scores.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2020
From: AMBARELLA, INC.
To: AMBARELLA INTERNATIONAL LP
Reel/Frame 051692/0711 →
Continuity (2)
Continuation 15605411 · May 27, 2017
Provisional Application 62500060 · May 2, 2017