IP Library › Granted Patent US 11,461,919
Granted Patent B2
US 11,461,919 · App. 16/844,483 · Granted Oct 4, 2022

Cascaded neural network

Inventors: Lior Wolf (Herzliya, IL); Assaf Mushinsky (Tel Aviv, IL)
Assignee: Ramot at Tel Aviv University Ltd.
G06T7/70G06K9/6256G06K9/6267G06N3/04G06N3/08G06T3/4046G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,461,919
App. No.
16/844,483
Granted
Oct 4, 2022
Kind
B2
Abstract

A neural network system for detecting at least one object in at least one image, the system includes a plurality of object detectors. Each object detector receives respective image information thereto. Each object detector includes a respective neural network. Each the neural network including a plurality of layers. Layers in different object detectors are common layers when the layers receive the same input thereto and produce the same output therefrom. Common layers are computed only once during object detection for all the different object detectors.

Claims (28)

1. A neural network system for detecting at least one object in at least one image, the system comprising a plurality of object detectors, each object detector is configured to receive image information, each said object detector includes a respective neural network, each said respective neural network includes a plurality of layers, wherein said layers in different said object detectors are common layers when said layers receive the same input and produce the same output, said neural network system is configured to compute once for different said object detectors.

2. The neural network system according to claim 1 , further including a plurality of down-samplers each respectively associated with a down-sampling ratio, said down-samplers are configured to produce scaled versions of said at least one image, each scaled version being respectively associated with a down-sampling ratio.

3. The neural network system according to claim 2 , wherein said down-samplers and said object detectors respectively associated with a same image window size with respect to said at least one image, define a scale detector, each said scale detector is respectively associated with a scaled version of said at least one image.

4. The neural network system according to claim 1 further including an object classifier, coupled with said respective neural network, said object classifier is configured to classify objects in said at least one image according to results from said respective neural network.

5. The neural network system according to claim 4 , wherein said object classifier is configured to convolve at least one classification filter with a features map respectively provided by said respective neural network.

6. The neural network system according to claim 5 , wherein said respective neural network is configured to produce a features map that includes a plurality of features, where each entry in said features map represents the feature's intensities within an image window associated with said entry and with an image window size.

7. The neural network system according to claim 6 , where said object classifier is configured to produce information relating to a probability that said object is located in each said image window associated with said features map.

8. The neural network system according to claim 7 , wherein each said object classifier is configured to determine a classification vector that includes image window correction factors for each image window associated said features map, said image window correction factors include corrections to: width and height of each said image window, location of each image window, and orientation of each said image window.

9. The neural network system according to claim 3 , wherein said neural network system involves training of a single said training scale detector when scale detectors exhibit the same configuration of said object detectors, and when said respective neural networks in said object detectors exhibit group of layers with identical characteristics.

10. The neural network system according to claim 9 , wherein prior to said training of said training scale detector, said neural network system increases a number of training samples in a training set beyond an initial number of said training samples by:

for each object-key point, determining a feature location within its associated training sample bounding box;

for object key-point type, determining a feature reference location according to an average location of a plurality of object key-points of the same type, said average determined according to said feature locations of said object key-points of all of said objects in an initial training set;

registering all of said training samples in said initial training set with said feature reference locations;

aligning each of said objects in said initial training set with said feature reference locations of said object key points to create an aligned training set; and

randomly perturbing each of said objects in said aligned training set.

11. A neural network method comprising: detecting objects in an image by employing a neural network, said neural network including a plurality of object detectors, each object detector receiving image information, each object detector includes a respective neural network, said respective neural network includes a plurality of layers, wherein said layers in different said object detectors are common layers when said layers receive the same input thereto and produce the same output, so as to compute said common layers for different said object detectors.

12. The neural network method according to claim 11 , including, prior to said detecting, down-sampling of said image according to a plurality of down-sampling ratios, to produce a plurality of down-sampled images, each down-sampled image is associated with a down-sampling ratio.

13. The neural network method according to claim 12 , including, prior to said down-sampling:

producing augmented training samples from an initial training set; and

training respective neural networks of said object detectors to have said common layers.

14. The neural network method according to claim 13 , wherein said training involves averaging weights and parameters of all groups of said layers that exhibit identical characteristics in said object detectors.

15. The neural network method according to claim 13 , wherein said training involves training of a single training scale detector by employing said augmented training samples, and deploying duplicates of said single training scale detector, each duplicate being associated with a scaled version of said image, said duplicates of said single training scale detector define said neural network system.

16. The neural network method according to claim 13 , wherein said producing augmented training samples includes:

for each object-key point, determining a feature location its associated training sample bounding box;

for object key-point type, determining a feature reference location according to an average location of a plurality of object key-points of the same type, said average determined according to said feature locations of said object key-points of all of said objects in an initial training set;

registering all of said training samples in said initial training set with said feature reference locations;

aligning each of said objects in said initial set with said feature reference locations of said object key points to create an aligned training set; and

randomly perturbing each of said objects in said aligned training set.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2020
From: WOLF, LIOR; MUSHINSKY, ASSAF
To: RAMOT AT TEL AVIV UNIVERSITY LTD.
Reel/Frame 052501/0930 →
Priority Claims (1)
GB 1614009 · Aug 16, 2016 · national
Continuity (7)
Continuation In Part 15909350 · Mar 1, 2018
Continuation PCTIL2017050461 · Apr 20, 2017
Provisional Application 62486997 · Apr 19, 2017
Provisional Application 62325553 · Apr 21, 2016
Provisional Application 62325551 · Apr 21, 2016
Provisional Application 62325562 · Apr 21, 2016
Related Publication 20200279393A1 · Sep 3, 2020
Cited By (1)
US 12,254,659