IP Library › Granted Patent US 12,641,315
Granted Patent B2
US 12,641,315 · App. 19/092,973 · Granted May 26, 2026

Image processing system, image processing method, and program

Inventor: Yusuke Takahashi (Tokyo, JP)
Assignee: NEC CORPORATION
H04N21/47G06V40/103H04N7/181H04N21/4316G06T2207/10016G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,641,315
App. No.
19/092,973
Granted
May 26, 2026
Kind
B2
Abstract

The present invention provides an image processing system, an image processing method, and a program capable of suppressing confusion that may arise in a case where a target person is identified when tracking a person. An image processing system includes: a camera control unit which accepts input video images captured by a plurality of video cameras; a tracked person registering unit capable of registering one or more persons appearing in the video images inputted from the camera control unit; and a display screen generating unit which displays switchable windows of the video images inputted from the video cameras for each person registered by the tracked person registering unit.

Claims (71)

1 . A video surveillance method, comprising:

receiving a first video frame captured by a first camera and a second video frame captured by a second camera, the second camera being different from the first camera;

receiving detection information of a first person in the first video frame;

outputting the first video frame including the detected first person;

controlling to output first notification information, in a case where a second person is detected from the second video frame while the first video frame including the detected first person is being displayed on a display, wherein the second video frame is captured by the second camera that is different from the first camera and the first notification information includes a detection result of the second person;

controlling to output screen information including a first plurality of images and first time information, the first time information being related to detection of the first person;

receiving a user input for selecting at least one camera from among a plurality of cameras; and

based on the user input, controlling to output the screen information including a second plurality of images, the second plurality of images being obtained from video captured by the at least one selected camera, the first plurality of images having a difference from the second plurality of images.

2 . The video surveillance method according to claim 1 , further comprising:

notifying a user with the first notification information, in a case where the second person is detected in the second video frame while the first video frame including the detected first person is being displayed on the display, the first notification information including the detection result of the second person.

3 . The video surveillance method according to claim 1 , further comprising:

controlling to output the screen information including the first plurality of images from video frames, detection time information related to the video frames, and the first time information related to the detection of the first person.

4 . The video surveillance method according to claim 1 , wherein the first notification information includes at least one of an image and a text message.

5 . The video surveillance method according to claim 4 , wherein the image includes at least one of the second person and an area where the second person is detected.

6 . The video surveillance method according to claim 1 , further comprising:

receiving a third video frame captured by a third camera;

receiving second time information related to detection of a third person in the third video frame; and

controlling to output second notification information on the display or sound information, in a case where the second time information satisfies a predetermined time.

7 . The video surveillance method according to claim 6 , wherein the predetermined time is set in advance.

8 . The video surveillance method according to claim 1 , further comprising:

detecting the first person in the first video frame; and

tracking the detected first person.

9 . A video surveillance system, comprising:

at least one memory storing instructions; and

at least one processor configured to:

receive a first video frame captured by a first camera and a second video frame captured by a second camera, the second camera being different from the first camera;

receive detection information of a first person in the first video frame;

output the first video frame including the detected first person;

control to output first notification information, in a case where a second person is detected from the second video frame while the first video frame including the detected first person is being displayed on a display, wherein the second video frame is captured by the second camera that is different from the first camera and the first notification information includes a detection result of the second person;

control to output screen information including a first plurality of images and first time information, the first time information being related to detection of the first person;

control to receive a user input for selecting at least one camera from among a plurality of cameras; and

control to output, based on the user input, the screen information including a second plurality of images, the second plurality of images being obtained from video captured by the at least one selected camera, the first plurality of images having a difference from the second plurality of images.

10 . The video surveillance system according to claim 9 , wherein the at least one processor is further configured to:

notify a user with the first notification information, in a case where the second person is detected in the second video frame while the first video frame including the detected first person is being displayed on the display, the first notification information including the detection result of the second person.

11 . The video surveillance system according to claim 9 , wherein the at least one processor is further configured to:

control to output the screen information including the first plurality of images from video frames, detection time information related to the video frames, and the first time information related to the detection of the first person.

12 . The video surveillance system according to claim 9 , wherein the first notification information includes at least one of an image and a text message.

13 . The video surveillance system according to claim 12 , wherein the image includes at least one of the second person and an area where the second person is detected.

14 . The video surveillance system according to claim 9 , wherein the at least one processor is further configured to:

receive a third video frame captured by a third camera;

receive second time information related to detection of a third person in the third video frame; and

control to output second notification information on the display or sound information, in a case where the second time information satisfies a predetermined time.

15 . The video surveillance system according to claim 14 , wherein the predetermined time is set in advance.

16 . The video surveillance system according to claim 9 , wherein the at least one processor is further configured to:

detect the first person in the first video frame; and

track the detected first person.

17 . The video surveillance method according to claim 1 , wherein the first notification information includes a text message notifying the second person was detected.

18 . The video surveillance method according to claim 1 , wherein the screen information includes detection results of the first person, the detection results arranged in chronological order.

19 . The video surveillance method according to claim 1 , further comprising:

controlling to display the first video frame including the detected first person on the display;

controlling to display the first notification information on the display; and

in a case of receiving an input for the first notification information after displaying the first notification information, controlling to change a video frame displayed on the display from the first video frame including the detected first person to the second video frame including the detected second person.

20 . A video surveillance method, comprising:

obtaining a plurality of first video images captured by a first video camera;

determining that at least one of the plurality of first video images includes a first image of a first person;

selecting a first thumbnail image from the plurality of first video images that includes the first image of the first person, the first thumbnail image being a still image;

generating a first notification, wherein the first notification provides time information related to detection of the first person and includes the first thumbnail image;

controlling to display in a first window the first notification;

obtaining a plurality of second video images captured by a second video camera;

determining that at least one of the plurality of second video images includes a second image of a second person;

selecting a second thumbnail image from the plurality of second video images that includes the second image of the second person, the second thumbnail image being a still image;

generating a second notification, wherein the second notification provides time information related to detection of the second person and includes the second thumbnail image;

controlling to display in the first window the second notification;

controlling to display the first notification and the second notification on a display;

detecting a selection of the first notification on the display by a user; and

after detecting the selection of the first notification on the display by the user, controlling to display a second window, wherein the second window is configured to display the plurality of first video images including the first person.

21 . The video surveillance method according to claim 20 , further comprising displaying a third window that displays real-time video image captured by the first video camera and a fourth window that displays real-time video image captured by the second video camera.

22 . The video surveillance method according to claim 20 , wherein the first notification further includes text information indicating a name corresponding to the first video camera, and the second notification further includes text information indicating a name corresponding to the second video camera.

23 . The video surveillance method according to claim 20 , further comprising:

tracking the first person by the first video camera; and

tracking the second person by the second video camera.

Priority Claims (1)
JP 2012-170406 · Jul 31, 2012 · national
Continuity (5)
Continuation 17563261 · Dec 28, 2021
Continuation 16925183 · Jul 9, 2020
Continuation 16378081 · Apr 8, 2019
Continuation 14416716
Related Publication 20250227335A1 · Jul 10, 2025
References Cited (100)
US 6061055A · Marks · 2000 [cited by applicant]
US 6271805B1 · Yonezawa · 2001 [cited by applicant]
US 7015954B1 · Foote et al. · 2006 [cited by applicant]
US 7058204B2 · Hildreth et al. · 2006 [cited by applicant]
US 7127083B2 · Han et al. · 2006 [cited by applicant]
US 7196722B2 · White et al. · 2007 [cited by applicant]
US 7286157B2 · Buehler · 2007 [cited by applicant]
US 7304662B1 · Sullivan et al. · 2007 [cited by applicant]
US 7613324B2 · Venetianer et al. · 2009 [cited by applicant]
US 7643653B2 · Garoutte · 2010 [cited by applicant]
US 7746380B2 · Maruya et al. · 2010 [cited by applicant]
US 7777783B1 · Chin et al. · 2010 [cited by applicant]
US 7843491B2 · Vallone et al. · 2010 [cited by applicant]
US 7881505B2 · Schneiderman et al. · 2011 [cited by applicant]
US 8174572B2 · Buehler et al. · 2012 [cited by applicant]
US 8179441B2 · Yen et al. · 2012 [cited by applicant]
US 8274564B2 · Girgensohn et al. · 2012 [cited by applicant]
US 8390684B2 · Piran et al. · 2013 [cited by applicant]
US 8437508B2 · Akisada et al. · 2013 [cited by applicant]
US 8655020B2 · Saptharishi · 2014 [cited by examiner]
US 10424342B2 · Brown et al. · 2019 [cited by applicant]
US 20030025599A1 · Monroe · 2003 [cited by applicant]
US 20050122397A1 · Henson et al. · 2005 [cited by applicant]
US 20050152579A1 · Park et al. · 2005 [cited by applicant]
US 20050265582A1 · Buehler et al. · 2005 [cited by applicant]
US 20060221184A1 · Vallone et al. · 2006 [cited by applicant]
US 20060225114A1 · Walker et al. · 2006 [cited by applicant]
US 20070182818A1 · Buehler · 2007 [cited by applicant]
US 20070206834A1 · Shinkai et al. · 2007 [cited by applicant]
US 20080018738A1 · Lipton et al. · 2008 [cited by applicant]
US 20080144890A1 · Ogawa · 2008 [cited by applicant]
US 20080292140A1 · Morris et al. · 2008 [cited by applicant]
US 20080303901A1 · Variyath et al. · 2008 [cited by applicant]
US 20080304706A1 · Akisada et al. · 2008 [cited by applicant]
US 20090079831A1 · Sethuraman et al. · 2009 [cited by applicant]
US 20090153654A1 · Enge et al. · 2009 [cited by applicant]
US 20090183177A1 · Brown et al. · 2009 [cited by applicant]
US 20090225163A1 · Chathukutty et al. · 2009 [cited by applicant]
US 20090251537A1 · Keidar et al. · 2009 [cited by applicant]
US 20090288011A1 · Piran · 2009 [cited by examiner]
US 20090295918A1 · Horovitz et al. · 2009 [cited by applicant]
US 20100002082A1 · Buehler et al. · 2010 [cited by applicant]
US 20100157050A1 · Drive et al. · 2010 [cited by applicant]
US 20100183227A1 · Park · 2010 [cited by applicant]
US 20100201815A1 · Anderson et al. · 2010 [cited by applicant]
US 20110007139A1 · Brunetti · 2011 [cited by examiner]
US 20110019003A1 · Asa et al. · 2011 [cited by applicant]
US 20110169867A1 · Kniffen · 2011 [cited by examiner]
US 20110170749A1 · Schneiderman et al. · 2011 [cited by applicant]
US 20110228092A1 · Park et al. · 2011 [cited by applicant]
US 20110317010A1 · Dhanapal et al. · 2011 [cited by applicant]
US 20120007975A1 · Lyons et al. · 2012 [cited by applicant]
US 20120019644A1 · Hamabashiri et al. · 2012 [cited by applicant]
US 20120026308A1 · Johnson et al. · 2012 [cited by applicant]
US 20120062732A1 · Marman et al. · 2012 [cited by applicant]
US 20120113264A1 · Moshrefi et al. · 2012 [cited by applicant]
US 20120120241A1 · Lewis et al. · 2012 [cited by applicant]
US 20120188370A1 · Bordonaro · 2012 [cited by applicant]
US 20120194336A1 · Thiruvengada · 2012 [cited by examiner]
US 20130050502A1 · Saito et al. · 2013 [cited by applicant]
US 20130091432A1 · Shet et al. · 2013 [cited by applicant]
US 20130208124A1 · Boghossian et al. · 2013 [cited by applicant]
US 20130329958A1 · Oami et al. · 2013 [cited by applicant]
US 20150103178A1 · Itoh et al. · 2015 [cited by applicant]
CN 1574961 · 2005 [cited by applicant]
CN 101321271 · 2008 [cited by applicant]
CN 102170560 · 2011 [cited by applicant]
EP 1150510A2 · 2001 [cited by applicant]
EP 1489847A3 · 2011 [cited by applicant]
EP 2458553A1 · 2012 [cited by applicant]
EP 2596630A1 · 2013 [cited by applicant]
GB 2482127A · 2012 [cited by applicant]
JP 8221592 · 1996 [cited by applicant]
JP 2001236514 · 2001 [cited by applicant]
JP 2006093955A · 2006 [cited by applicant]
JP 2008219570 · 2008 [cited by applicant]
JP 201049349 · 2010 [cited by applicant]
WO 2009038994A1 · 2009 [cited by applicant]
WO 2009079809A1 · 2009 [cited by applicant]
WO 2012131816A1 · 2012 [cited by applicant]
Haering et al., “Automatic Visual Analysis for Transportation Security”, 2007, IEEE, pp. 13-18. (Year: 2007). [cited by examiner]
International Search Report mailed Jul. 30, 2013 in corresponding PCT International Application. [cited by applicant]
Written Opinion and Search Report issued by the Singapore Patent Office on Aug. 5, 2016, in counterpart Singapore Patent Application No. 11201500693Q. [cited by applicant]
Notification of the First Office Action issued on Jun. 20, 2017, from the State Intellectual Property office of the People's Republic of China is counterpart Chinese Paten Application No. 201380040754.3. [cited by applicant]
Office Action, dated Mar. 18, 2020, issued by the United States Patent Office in counterpart U.S. Appl. No. 14/416,716. [cited by applicant]
U.S. Office Action dated Jan. 23, 2020, issued in co-pending U.S. Appl. No. 16/286,430. [cited by applicant]
Non-final U.S. Office Action dated Oct. 21,2019, issued on co-pending U.S. Appl. No. 16/286,449. [cited by applicant]
U.S. Appl. No. 17/563,261, filed Dec. 28, 2021. [cited by applicant]
U.S. Appl. No. 16/925,183, filed Jul. 9, 2020. [cited by applicant]
U.S. Appl. No. 16/378,081, filed Apr. 8, 2019. [cited by applicant]
U.S. Appl. No. 14/416,716, filed Jan. 23, 2015. [cited by applicant]
SangKyu Kang et al., “Multiple-camera tracking system with seamless object handover”, Proc. SPIE, vol. 4303, pp. 27-34, 2001. [cited by applicant]
Fatih Porikli, “Intercamera color calibration by correlation model function”, IEEE, 2003. [cited by applicant]
Omar Javed et al., “Appearance modeling for tracking in multiple non-overlapping cameras”, IEEE, 2005. [cited by applicant]
Kuan-Wen Chen et al., “An adaptive learning method for target tracking across multiple cameras”, IEEE, 2008. [cited by applicant]
Bryan Prosser et al., “Person re-identification by support vector ranking”, BMVC 2010, doi:10.5244/c.24.21. [cited by applicant]
Jae-Ho Lee, “Automatic Video Management System Using Face Recognition and MPEG-7 Visual Descriptors”, ETRI Journal, vol. 27, No. 6, Dec. 2005, pp. 806-809. [cited by applicant]
GeoVision, GV Series (CMS/Video Analytics), Sep. 30, 2011. [cited by applicant]
Avigilon, Avigilon Control Center with High Definition Stream Management (HDSM). [cited by applicant]
Axis Introduces HDTV Image Quality and Precise Iris Control in New Fixed Day and Night Network Camera Series; AXIS P13 Network Cameras Ideal for Securing Government Buildings, Retail Environments, Airports, Public Trans… [cited by applicant]