IP Library Granted Patent US 11,276,213
Granted Patent B2
US 11,276,213 · App. 16/799,319 · Granted Mar 15, 2022

Neural network based detection of items of interest and intelligent generation of visualizations thereof

Inventors: Ian Cinnamon (Sherman Oaks, CA); Bruno Brasil Ferrari Faviero (Coconut Creek, FL); Simanta Gautam (Charlottesville, VA)
Assignee: Rapiscan Laboratories, Inc.
G06T11/60G06K9/2054G06K9/3241G06K9/46G06K9/627G06K9/6267G06Q50/26G06Q50/265G06T1/0007G06T7/001G06T7/0004G06T7/62G06T7/73G06K2209/09G06T2207/20021G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,276,213
App. No.
16/799,319
Granted
Mar 15, 2022
Kind
B2
Abstract

In example embodiments, a computing system is capable of (a) receiving image data that represents a scene that was scanned by a detection device of a security screening system, (b) based at least on the image data, at least one neural network, and a confidence threshold defined for the security screening system, determining that the image data includes an identified item that has been deemed to be of interest, (c) based at least on determining that the image data includes the identified item that has been deemed to be of interest and one or more security parameters for the security screening system, determining that the identified item is deemed to be a security interest for the security screening system, and (d) based at least on determining that the identified item is deemed to be a security interest for the security screening system, presenting a visualization corresponding to the identified item.

Claims (53)

1. A computing system comprising

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is capable of:

receiving, from a detection device of a security screening system, image data that represents a scene that was scanned by the detection device;

based at least on (i) the image data, (ii) at least one neural network trained for identification of items deemed to be of interest, and (iii) a confidence threshold defined for the security screening system that is utilized by the at least one neural network, determining that the image data includes at least one identified item that has been deemed to be of interest;

based at least on (i) determining that the image data includes the at least one identified item that has been deemed to be of interest and (ii) one or more security parameters for the security screening system, determining that the at least one identified item is deemed to be a security interest for the security screening system; and

based at least on determining that the at least one identified item is deemed to be a security interest for the security screening system, generating, for output at a display associated with the security screening system, a visualization corresponding to the at least one identified item.

2. The computing system of claim 1 , wherein the at least one neural network comprises at least one deep neural network comprising a plurality of layers.

3. The computing system of claim 1 , wherein the scene comprises a three-dimensional space where one or more of storage receptacles or humans are scanned.

4. The computing system of claim 1 , wherein the image data that represents the scene further includes one or more additional items that are not deemed to be of interest, and wherein generating the visualization corresponding to the at least one identified item comprises:

generating a visualization that (i) includes a visualization for the at least one identified item and (ii) does not include a visualization for any of the one or more additional items that are not deemed to be of interest.

5. The computing system of claim 1 , wherein the image data that represents the scene further includes one or more additional items that are (i) deemed to be of interest and (ii) not deemed to be a security interest for the security screening system, and wherein generating the visualization corresponding to the at least one identified item comprises:

generating a visualization that (i) includes a visualization for the at least one identified item and (ii) does not include a visualization for any of the one or more additional items that are (a) deemed to be of interest and (b) not deemed to be a security interest for the security screening system.

6. The computing system of claim 1 , wherein the image data that represents the scene that was scanned by the detection device comprises first image data that represents a first scan of the scene by the detection device, wherein the at least one identified item comprises at least one first identified item, and further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is capable of:

receiving, from the detection device of the security screening system, second image data that represents a second scan of the scene by the detection device;

based at least on (i) the second image data, (ii) the at least one neural network trained for identification of items deemed to be of interest, and (iii) the confidence threshold defined for the security screening system that is utilized by the at least one neural network, determining that the second image data includes at least one second identified item that has been deemed to be of interest;

based at least on (i) determining that the image data includes the at least one second identified item that has been deemed to be of interest and (ii) one or more security parameters for the security screening system, determining that the at least one second identified item is not deemed to be a security interest for the security screening system; and

based at least on determining that the at least one second identified item is not deemed to be a security interest for the security screening system, declining to generate a visualization corresponding to the at least one second identified item.

7. The computing system of claim 6 , wherein declining to generate the visualization corresponding to the at least one second identified item comprises generating, for output at the display associated with the security screening system, a visualization that obscures at least a region of the second image data corresponding to the at least one second identified item.

8. The computing system of claim 6 , wherein declining to generate the visualization corresponding to the at least one second identified item comprises causing the display associated with the security screening system to output an empty visualization for the second scan of the scene by the detection device.

9. A tangible, non-transitory computer-readable medium comprising program instructions that are executable by at least one processor such that a computing system is capable of:

receiving, from a detection device of a security screening system, image data that represents a scene that was scanned by the detection device;

based at least on (i) the image data, (ii) at least one neural network trained for identification of items deemed to be of interest, and (iii) a confidence threshold defined for the security screening system that is utilized by the at least one neural network, determining that the image data includes at least one identified item that has been deemed to be of interest;

based at least on (i) determining that the image data includes the at least one identified item that has been deemed to be of interest and (ii) one or more security parameters for the security screening system, determining that the at least one identified item is deemed to be a security interest for the security screening system; and

based at least on determining that the at least one identified item is deemed to be a security interest for the security screening system, generating, for output at a display associated with the security screening system, a visualization corresponding to the at least one identified item.

10. The tangible, non-transitory computer-readable medium of claim 9 , wherein the at least one neural network comprises at least one deep neural network comprising a plurality of layers.

11. The tangible, non-transitory computer-readable medium of claim 9 , wherein the scene comprises a three-dimensional space where one or more of storage receptacles or humans are scanned.

12. The tangible, non-transitory computer-readable medium of claim 9 , wherein the image data that represents the scene further includes one or more additional items that are not deemed to be of interest, and wherein generating the visualization corresponding to the at least one identified item comprises:

generating a visualization that (i) includes a visualization for the at least one identified item and (ii) does not include a visualization for any of the one or more additional items that are not deemed to be of interest.

13. The tangible, non-transitory computer-readable medium of claim 9 , wherein the image data that represents the scene further includes one or more additional items that are (i) deemed to be of interest and (ii) not deemed to be a security interest for the security screening system, and wherein generating the visualization corresponding to the at least one identified item comprises:

generating a visualization that (i) includes a visualization for the at least one identified item and (ii) does not include a visualization for any of the one or more additional items that are (i) deemed to be of interest and (ii) not deemed to be a security interest for the security screening system.

14. The tangible, non-transitory computer-readable medium of claim 9 , wherein the image data that represents the scene that was scanned by the detection device comprises first image data that represents a first scan of the scene by the detection device, wherein the at least one identified item comprises at least one first identified item, and further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is capable of:

receiving, from the detection device of the security screening system, second image data that represents a second scan of the scene by the detection device;

based at least on (i) the second image data, (ii) the at least one neural network trained for identification of items deemed to be of interest, and (iii) the confidence threshold defined for the security screening system that is utilized by the at least one neural network, determining that the second image data includes at least one second identified item that has been deemed to be of interest;

based at least on (i) determining that the image data includes the at least one second identified item that has been deemed to be of interest and (ii) one or more security parameters for the security screening system, determining that the at least one second identified item is not deemed to be a security interest for the security screening system; and

based at least on determining that the at least one second identified item is not deemed to be a security interest for the security screening system, declining to generate a visualization corresponding to the at least one second identified item.

15. The tangible, non-transitory computer-readable medium of claim 14 , wherein declining to generate the visualization corresponding to the at least one second identified item comprises generating, for output at the display associated with the security screening system, a visualization that obscures at least a region of the second image data corresponding to the at least one second identified item.

16. The tangible, non-transitory computer-readable medium of claim 14 , wherein declining to generate the visualization corresponding to the at least one second identified item comprises causing the display associated with the security screening system to output an empty visualization for the second scan of the scene by the detection device.

17. A computer-implemented method comprising:

receiving, from a detection device of a security screening system, image data that represents a scene that was scanned by the detection device;

based at least on (i) the image data, (ii) at least one neural network trained for identification of items deemed to be of interest, and (iii) a confidence threshold defined for the security screening system that is utilized by the at least one neural network, determining that the image data includes at least one identified item that has been deemed to be of interest;

based at least on (i) determining that the image data includes the at least one identified item that has been deemed to be of interest and (ii) one or more security parameters for the security screening system, determining that the at least one identified item is deemed to be a security interest for the security screening system; and

based at least on determining that the at least one identified item is deemed to be a security interest for the security screening system, generating, for output at a display associated with the security screening system, a visualization corresponding to the at least one identified item.

18. The computer-implemented method of claim 17 , wherein the image data that represents the scene further includes one or more additional items that are not deemed to be of interest, and wherein generating the visualization corresponding to the at least one identified item comprises:

generating a visualization that (i) includes a visualization for the at least one identified item and (ii) does not include a visualization for any of the one or more additional items that are not deemed to be of interest.

19. The computer-implemented method of claim 17 , wherein the image data that represents the scene further includes one or more additional items that are (i) deemed to be of interest and (ii) not deemed to be a security interest for the security screening system, and wherein generating the visualization corresponding to the at least one identified item comprises:

generating a visualization that (i) includes a visualization for the at least one identified item and (ii) does not include a visualization for any of the one or more additional items that are (a) deemed to be of interest and (b) not deemed to be a security interest for the security screening system.

20. The computer-implemented method of claim 17 , wherein the image data that represents the scene that was scanned by the detection device comprises first image data that represents a first scan of the scene by the detection device, wherein the at least one identified item comprises at least one first identified item, and wherein the method further comprises:

receiving, from the detection device of the security screening system, second image data that represents a second scan of the scene by the detection device;

based at least on (i) the second image data, (ii) the at least one neural network trained for identification of items deemed to be of interest, and (iii) the confidence threshold defined for the security screening system that is utilized by the at least one neural network, determining that the second image data includes at least one second identified item that has been deemed to be of interest;

based at least on (i) determining that the image data includes the at least one second identified item that has been deemed to be of interest and (ii) one or more security parameters for the security screening system, determining that the at least one second identified item is not deemed to be a security interest for the security screening system; and

based at least on determining that the at least one second identified item is not deemed to be a security interest for the security screening system, declining to generate a visualization corresponding to the at least one second identified item.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2020
From: SYNAPSE TECHNOLOGY CORPORATION
To: RAPISCAN LABORATORIES, INC.
Reel/Frame 052322/0078 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 25, 2020
From: CINNAMON, IAN; FAVIERO, BRUNO BRASIL FERRARI; GAUTAM, SIMANTA
To: SYNAPSE TECHNOLOGY CORPORATION
Reel/Frame 051916/0809 →
Continuity (4)
Continuation 15714932 · Sep 25, 2017
Provisional Application 62532821 · Jul 14, 2017
Provisional Application 62532865 · Jul 14, 2017
Related Publication 20200193666A1 · Jun 18, 2020
Cited By (1)
US 12,385,854