IP Library Granted Patent US 10,635,934
Granted Patent B2
US 10,635,934 · App. 16/132,518 · Granted Apr 28, 2020

Method and system for image content recognition

Inventors: Lior Wolf (Herzlia, IL); Noam Mor (Tel-Aviv, IL)
Assignee: Ramot at Tel-Aviv University Ltd.
G06K9/6217G06K9/00442G06K9/03G06K9/42G06K9/6277G06K9/6878G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,635,934
App. No.
16/132,518
Granted
Apr 28, 2020
Kind
B2
Abstract

A method of recognizing image content, comprises applying to the image a neural network which comprises an input layer for receiving the image, a plurality of hidden layers for processing the image, and an output layer for generating output pertaining to an estimated image content based on outputs of the hidden layers. The method further comprises applying to an output of at least one of the hidden layers a neural network branch, which is independent of the neural network and which has an output layer for generating output pertaining to an estimated error level of the estimate. A combined output indicative of the estimated image content and the estimated error level is generated.

Claims (34)

1. A method of recognizing image content, comprising:

applying a neural network to the image, said neural network comprising an input layer for receiving the image, a plurality of hidden layers for processing the image, and an output layer for generating output pertaining to an estimated image content based on outputs of said hidden layers;

applying a neural network branch to an output of at least one of said hidden layers, said neural network branch being independent of said neural network and having an output layer for generating output pertaining to an estimated error level of said estimate; and

generating a combined output indicative of the estimated image content and the estimated error level.

2. The method according to claim 1 , wherein said neural network branch comprises at least one recurrent layer generating a plurality of output values.

3. The method of claim 2 , wherein said at least one recurrent neural layer is a Long Short Term Memory (LSTM) layer.

4. The method of claim 3 , wherein said LSTM layer is a bi-directional layer.

5. The method according to claim 2 , further comprising summing or averaging said plurality of output values or projections thereof, thereby providing said estimated error level.

6. The method according to claim 1 , wherein said neural network comprises a convolutional neural subnetwork generating a set of feature vectors, each corresponding to an application of said convolutional subnetwork different window within the image.

7. The method of claim 6 , wherein said neural network comprises a recurrent neural subnetwork receiving said set of feature vectors from said convolutional neural subnetwork, and wherein said output of said at least one of said hidden layers is an output of at least one layer of said recurrent neural subnetwork.

8. The method of claim 7 , wherein said recurrent neural subnetwork comprises Long Short Term Memory (LSTM) layers.

9. The method of claim 8 , wherein at least one of said LSTM layers is a bi-directional layer.

10. The method according to claim 7 , wherein said recurrent neural subnetwork generates a sequence of vectors, and the method comprises projecting a dimension of each vector of said sequence onto a predetermined dimension corresponding to a length of an alphabet, to provide a sequence of probability distributions over said alphabet, each probability distribution corresponding to a region within the image.

11. The method according to claim 10 , wherein said neural network comprises a Connectionist temporal classification (CTC) layer receiving said sequence of probability distributions and generating said estimate.

12. The method according to claim 1 , further comprising transforming a height of the image to a predetermined height, prior to said application of said neural network.

13. The method according to claim 1 , wherein the image is an image of handwritten symbols.

14. The method according to claim 1 , wherein the image is an image of machine printed symbols.

15. The method according to claim 1 , wherein the image comprises at least one region containing handwritten symbols and at least one region containing machine printed symbols.

16. The method according to claim 1 , wherein the image is an image of a bank check.

17. The method according to claim 16 , wherein said bank check comprises a plurality of bank check fields, and the method is repeated separately for at least two of said bank check fields.

18. A computer software product, comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive an input image and to execute the method according to claim 1 .

19. A system for recognizing image content, comprising:

an imaging system for imaging an object to provide an image thereof; and

an image processor configured for:

receiving the image;

applying a neural network to the image, said neural network comprising an input layer for receiving the image, a plurality of hidden layers for processing the image, and an output layer for generating output pertaining to an estimated image content based on outputs of said hidden layers;

applying a neural network branch to an output of at least one of said hidden layers, said neural network branch being independent of said neural network and having an output layer for generating output pertaining to an estimated error level of said estimate; and

generating a combined output indicative of the estimated image content and the estimated error level.

20. A system for recognizing image content, comprising:

a network interface for receiving an image over a communication network; and

an image processor configured for:

applying a neural network to the image, said neural network comprising an input layer for receiving the image, a plurality of hidden layers for processing the image, and an output layer for generating output pertaining to an estimated image content based on outputs of said hidden layers;

applying a neural network branch to an output of at least one of said hidden layers, said neural network branch being independent of said neural network and having an output layer for generating output pertaining to an estimated error level of said estimate; and

generating a combined output indicative of the estimated image content and the estimated error level.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2019
From: WOLF, LIOR; MOR, NOAM
To: RAMOT AT TEL-AVIV UNIVERSITY LTD.
Reel/Frame 047910/0452 →
Continuity (2)
Provisional Application 62560275 · Sep 19, 2017
Related Publication 20190095753A1 · Mar 28, 2019
Cited By (1)
US 12,205,388