IP Library Granted Patent US 7,646,887
Granted Patent B2
US 7,646,887 · App. 11/324,957 · Granted Jan 12, 2010

Optical flow for object recognition

Assignee: Evolution Robotics Retail, Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,646,887
App. No.
11/324,957
Granted
Jan 12, 2010
Kind
B2
Abstract

Disclosed is a system and method for using optical flow to detect objects moving past a camera and to select images of the moving objects. A shopping cart, for example, may be detected by subdividing an image into a plurality of image blocks; comparing the blocks to a preceding image to determine the motion of the portion of the object pictured; associating the most common motion with the shopping cart. The motion of the cart may also be integrated over time for purposes of tracking cart motion and selecting a subset of the captured images for object recognition processing. Detection of the cart and image selection improves computational efficiency an increase merchandise throughput.

Claims (53)

1. A method for determining optical flow of a region of interest, the method comprising the steps of:

capturing with an imager a plurality of images of one or more objects on a moveable structure, the plurality of images comprising a first image and a second image;

partitioning the second image into a plurality of data blocks;

comparing a set of one or more of said blocks with at least a portion of the first image:

determining the displacement between the first image and block for each of the blocks in which a match is detected;

identifying the non-zero displacement common to the most blocks; and

associating the most common displacement with the moveable structure and one or more objects.

2. The method of claim 1 , wherein the movable structure is a shopping cart.

3. The method of claim 2 , wherein one or more of the steps are executed by a processor operatively coupled to a checkout terminal.

4. The method of claim 2 , wherein the step of comparing comprises the step correlating the set of blocks with an associated portion of the first image.

5. The method of claim 4 , wherein the associated portion of the first image is a search field.

6. The method of claim 5 , wherein the search field has a width sized to capture substantially all horizontal motion of the cart.

7. The method of claim 6 , wherein the search field has a height sized to exclude substantially all human motion.

8. The method of claim 1 , wherein the plurality of images are captured by a video camera.

9. The method of claim 8 , wherein the video camera is directed to a cart bottom of basket.

10. The method claim 1 , wherein the second image is down-sampled prior to the partitioning step.

11. The method of claim 1 , further comprising the step of determining a measure of texture in each block after the step of partitioning.

12. The method of claim 11 , wherein the set of blocks consists of those blocks for which the measure of texture exceeds a first threshold.

13. The method claim 1 , wherein the method further comprises the step of generating a histogram for binning the displacement of each of the set of blocks.

14. The method of claim 1 , further comprising the step of identifying the one or more objects from the blocks associated with the most common displacement.

15. The method of claim 14 , wherein the step of identifying the one or more objects comprises:

extracting one or more visual features from the blocks associated with the most common displacement; and

comparing the one or more visual features to a database of known objects.

16. The method of claim 15 , wherein the visual features are scale-invariant feature transformation (SIFT) features.

17. A system for measuring optical flow of a region of interest, the system comprising:

a visual sensor for capturing a plurality of images of one or more objects on a moveable structure, the plurality of images comprising a first image and a second image:

a processor adapted to:

a) partition the second image into a plurality of blocks;

b) compare a set of one or more of said blocks with at least a portion of the first image;

c) determine a displacement between the first image and block for each of the blocks in which a match is detected;

d) identify the non-zero displacement common to a set of blocks; and

e) identifying one or more of the objects from the blocks associated with the most common displacement.

18. The system of claim 17 , wherein the identified non-zero displacement is common to the greatest number of blocks of the second image.

19. A method of selecting Images of Interest, the method comprising:

capturing with an imager a plurality of images of one or more objects on a moveable structure;

partitioning each of the plurality of images into a plurality of blocks;

generating a displacement measurement for one or more of the blocks based on at least one preceding image;

identifying, for each image, the displacement measurement common to the most blocks;

adding the identified displacement measurement to a cumulative displacement measurement; and

selecting each image for which the cumulative displacement measurement exceeds a predetermined value with respect to a previously selected image.

20. The method of claim 19 , wherein the predetermined value corresponds to approximately one third a field of view of a camera for capturing the plurality of images.

21. The method of claim 19 , further comprising the step of identifying the one or more objects from the selected images.

22. The method of claim 21 , wherein the step of identifying the one or more objects from the identified images is based on scale-Invariant feature transformation (SIFT) features.

23. The method of claim 19 , further comprising the step of down-sample each of the plurality of images prior to the partitioning step.

24. The method of claim 23 , further comprising the step of identifying the one or more objects from the captured images associated with the selected images.

25. A method of selecting images of interest, the method comprising:

a visual sensor for capturing a plurality of images of one or more objects on a moveable structure;

a processor adapted to:

a) partition each of the plurality of images into a plurality of blocks;

b) generate a displacement measurement for one or more of the blocks based on at least one preceding image;

c) identify, for each image, the displacement measurement common to the most blocks;

d) add the identified displacement measurement to a cumulative displacement measurement; and

e) select each image for which the cumulative displacement measurement exceeds a predetermined value.

Assignments (3)
MERGER Recorded Apr 21, 2017
From: EVOLUTION ROBOTICS RETAIL, INC.
To: DATALOGIC ADC, INC.
Reel/Frame 042120/0669 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2010
From: GONCALVES, LUIS; GREMMER, ISAAC
To: EVOLUTION ROBOTICS RETAIL, INC.
Reel/Frame 024594/0709 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2009
From: GONCALVES, LUIS; GREMMER, ISAAC
To: EVOLUTION ROBOTICS RETAIL, INC.
Reel/Frame 023590/0459 →
Continuity (2)
Provisional Application 6064142700 · Jan 4, 2005
Related Publication 20060147087A1 · Jul 6, 2006