IP Library Granted Patent US 10,783,379
Granted Patent B2
US 10,783,379 · App. 16/109,533 · Granted Sep 22, 2020

Method for new package detection

Inventors: Marios Savvides (Wexford, PA); An Pang Lin (Mountain View, CA); Shreyas Venugopalan (Pittsburgh, PA); Ajmal Thanikkal (Pittsburgh, PA); Karanhaar Singh (West Windsor, NJ); Gavriel Adler (Brookline, MA); Kyle Neblett (Brookline, MA); Christopher Palmer (Pittsburgh, PA)
Assignee: BOSSA NOVA ROBOTICS IP, INC.
G06K9/00771G06F16/784G06K9/00362G06K9/00369G06K9/00979
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,783,379
App. No.
16/109,533
Granted
Sep 22, 2020
Kind
B2
Abstract

A method for processing arrival or removal of packages within the field of view of a video camera includes providing a database for recording packages placed in the field of view. Based on real-time analysis of successive image frames in the camera, a human person's entry and exit from the field of view of the camera is also detected. Delivery or removal of objects is recorded in the database. In one embodiment, the method also determines whether or not a newly arrived package is placed alongside or on top of an existing package.

Claims (40)

1. A method for processing arrival or removal of an object within the field of view of a video camera, comprising:

providing a database for recording objects residing in the field of view;

based on real-time analysis of successive image frames in the camera, detecting a human person's entry into the field of view of the camera;

saving a first set of image frames, corresponding image frames captured by the video camera immediately prior to the detection of the human person's entry into the field of view of the camera;

based on real-time analysis of successive image frames in the camera, detecting a human person's exit from the field of view of the camera;

saving a second set of image frames, corresponding to image frames captured by the video camera immediately after the detection of the human person's exit from the field of view of the camera;

comparing the first set of image frames with the second set of image frames to identify one or more objects that appear or disappear from the field of view based on difference between the first and the set of image frames;

determining and eliminating objects identified by comparing the first and second sets of image frames as artifacts;

determining from the objects remaining and the objects residing in the database: (a) whether or not one of the objects residing in the database has been removed; and (b) whether or not an object has newly arrived; and

entering any newly arrived objects into the database and removing any objects residing in the database that have been determined to have been removed.

2. The method of claim 1 , wherein the objects comprise packages that are delivered or collected.

3. The method of claim 1 , wherein the artifacts correspond to change in lighting conditions.

4. The method of claim 1 , wherein the artifacts correspond to change in lighting conditions due to the opening or shutting of a door.

5. The method of claim 1 , further comprising determining whether or not the newly arrived object is placed alongside or on top of an existing object residing in the database.

6. The method of claim 1 , wherein comparing the first and second sets of image frames comprises computing a block-based discrete cosine transform on each of the first set of image frames and the second set of image frames to determine therefrom a change in spatial frequencies.

7. The method of claim 1 , wherein comparing the first and second sets of image frames comprises computing differences in light intensity between corresponding pixels in the first set of image frames and the second set of image frames.

8. The method of claim 7 , further comprising computing accumulated differences for predetermined corresponding regions in the computed differences in light intensity.

9. The method of claim 1 , wherein determining from the objects remaining and the objects residing in the database comprises computing one or more intersections between each object remaining and each object residing in the database.

10. The method of claim 9 , wherein the computed intersections comprise an intersection in shape between objects.

11. The method of claim 9 , wherein the computed intersections comprise an intersection in location between objects.

12. The method of claim 1 , wherein the determining from the objects remaining and the objects residing in the database comprises comparing color histograms.

13. A method for package detection, comprising:

providing a database for recording packages in a field of view of a camera;

based on real-time analysis of successive image frames captured by the camera, detecting a human person's entry into the field of view and subsequently detecting the human person's exit from the field of view;

after the human person's exit, comparing a first set of image frames captured by the camera prior to detecting the human person's entry to a second set of image frames captured subsequent to detecting the human persons exit to identify whether packages have been removed or added to the field of view by computing one or more intersections between each package remaining and each package residing in the database;

determining and eliminating identified packages by comparing the first set of image frames and the second set of image frames as artifacts; and

based on the remaining packages and packages recorded in the database, entering any added packages into the database and removing packages residing in the database that have been determined to have been removed.

14. The method of claim 13 , wherein the computed intersections comprise an intersection in shape between objects.

15. The method of claim 13 , wherein the computed intersections comprise an intersection in location between objects.

16. The method of claim 13 , further comprising determining whether or not the newly arrived object is placed alongside or on top of an existing object residing in the database.

17. The method of claim 13 , wherein comparing a first set of image frames and a second set of image frames comprises computing a block-based discrete cosine transform on each of the first set of image frames and the second set of image frames to determine therefrom a change in spatial frequencies.

18. The method of claim 13 , further comprising use of a change detection module to define a bounding box around an object and determine whether it is a package.

19. The method of claim 13 , further comprising a change detection module and a bounding box filtering module to eliminate bounding boxes unlikely to surround packages.

20. A system for processing arrival or removal of an object within the field of view of a video camera, comprising:

a database module for recording objects residing in the field of view;

a detection module to determine a human person's entry into the field of view of the camera based on real-time analysis of successive image frames in the camera, with the detection module saving a first set of image frames corresponding to image frames captured by the video camera immediately prior to the detection of the human person's entry into the field of view of the camera and saving a second set of image frames corresponding to image frames captured by the video camera immediately after the detection of the human person's exit from the field of view of the camera;

a comparison module to compare the first set of image frames with the second set of image frames to identify one or more objects that appear or disappear from the field of view based on differences between the first and the set of image frames;

an artifact elimination module to determine and eliminate objects identified by comparing the first and second sets of image frames as artifacts; and

wherein the database module records (a) whether or not one of the objects residing in the database has been removed; and (b) whether or not an object is newly arrived; and

c) enters any newly arrived objects into the database and removes any objects residing in the database that have been determined to have been removed.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2023
From: BOSSA, LLC
To: HANSHOW AMERICA, INC.
Reel/Frame 065617/0320 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2023
From: BOSSA NOVA ROBOTICS IP, INC.
To: BOSSA, LLC
Reel/Frame 065617/0298 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2023
From: HANSHOW AMERICA, INC.
To: SHANGHAI HANSHI INFORMATION TECHNOLOGY CO., LTD.
Reel/Frame 065617/0349 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2019
From: SAVVIDES, MARIOS; LIN, AN PANG; VENUGOPALAN, SHREYAS; THANIKKAL, AJMAL; SINGH, KARANHAAR; ADLER, GAVRIEL; NEBLETT, KYLE; PALMER, CHRISTOPHER
To: BOSSA NOVA ROBOTICS IP, INC.
Reel/Frame 049907/0819 →
Continuity (2)
Provisional Application 62549368 · Aug 23, 2017
Related Publication 20190065861A1 · Feb 28, 2019
Cited By (1)
US 12,633,143