IP Library › Granted Patent US 12,506,970
Granted Patent B2
US 12,506,970 · App. 18/788,316 · Granted Dec 23, 2025

Image processing device, image processing method, and non-transitory storage medium

Inventors: Jianquan Liu (Tokyo, JP); Junnan Li (Tokyo, JP)
Assignee: NEC CORPORATION
H04N23/698H04N23/611
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,506,970
App. No.
18/788,316
Granted
Dec 23, 2025
Kind
B2
Abstract

The present invention provides an image processing apparatus ( 10 ) including a detection unit ( 12 ) that detects a plurality of predetermined points of a body of each of a plurality of persons from an image in an image circle of a fisheye image, a gravity direction determination unit ( 13 ) that determines a gravity direction in a position of each of the plurality of persons from the plurality of predetermined points, a reference point decision unit ( 14 ) that decides a reference point, based on the gravity direction in the position of each of the plurality of persons, a complementary circular image generation unit ( 16 ) that generates a complementary circular image that is a circular image acquired by adding a complementary image to the image in the image circle of the fisheye image, and that has, as a center, the reference point different from a center of the image in the image circle, and an expansion unit ( 17 ) that panoramically expands the complementary circular image, based on the reference point, and generates a panoramic image.

Claims (16)

1 . An image processing apparatus comprising;

at least one memory storing instructions and

at least one processor configured to execute the instructions to:

receive a fisheye image;

determine a reference point regarding the fisheye image based on information related to a person shown in the fisheye image; and

generate a panoramic image based on the reference point regarding the fisheye image.

2 . The image processing apparatus according to claim 1 , wherein the information related to a person shown in the fisheye image includes a gravity direction in a position of the person.

3 . The image processing apparatus according to claim 1 , wherein the at least one processor further configured to execute the instructions to generate a complementary circular image,0 wherein the instruction to generate the panoramic image uses the complementary circular image for generating the panoramic image.

4 . The image processing apparatus according to claim 3 , wherein the complementary circular image is used in a case where the reference point dose not coincide with a center of an image regarding the fisheye image.

5 . An image processing method executed by a computer, the method comprising;

receiving a fisheye image;

determining a reference point regarding the fisheye image based on information related to a person shown in the fisheye image; and

generating a panoramic image based on the reference point regarding the fisheye image.

6 . The image processing method according to claim 5 , wherein the information related to a person shown in the fisheye image includes a gravity direction in a position of the person.

7 . The image processing method according to claim 5 , wherein the computer generates a complementary circular image, wherein the computer uses the complementary circular image for generating the panoramic image.

8 . The image processing method according to claim 7 , wherein the complementary circular image is used in a case where the reference point dose not coincide with a center of an image regarding the fisheye image.

Continuity (2)
Continuation 17617420
Related Publication 20240388804A1 · Nov 21, 2024
References Cited (84)
US 5990941A · Jackson · 1999 [cited by applicant]
US 6687387B1 · Molnar · 2004 [cited by applicant]
US 9479732B1 · Saleh · 2016 [cited by examiner]
US 10699389B2 · Van der Auwera · 2020 [cited by examiner]
US 20010040624A1 · Kobayashi · 2001 [cited by applicant]
US 20020118890A1 · Rondinelli · 2002 [cited by applicant]
US 20060215930A1 · Terui · 2006 [cited by applicant]
US 20080068601A1 · Thayer · 2008 [cited by applicant]
US 20090160940A1 · Imamura · 2009 [cited by applicant]
US 20090303351A1 · Ogawa · 2009 [cited by applicant]
US 20100001997A1 · Kajikawa · 2010 [cited by examiner]
US 20100053325A1 · Inagaki · 2010 [cited by examiner]
US 20110018964A1 · Krishnan · 2011 [cited by applicant]
US 20120093365A1 · Aragane · 2012 [cited by applicant]
US 20120154643A1 · Okuyama · 2012 [cited by examiner]
US 20120257064A1 · Kim · 2012 [cited by examiner]
US 20120274782A1 · Kitaguchi · 2012 [cited by applicant]
US 20130314444A1 · Iwamoto · 2013 [cited by applicant]
US 20140043477A1 · Luczak · 2014 [cited by examiner]
US 20140184646A1 · Liu · 2014 [cited by examiner]
US 20140314336A1 · Yagi · 2014 [cited by applicant]
US 20150116452A1 · Koga · 2015 [cited by examiner]
US 20150312498A1 · Kawano · 2015 [cited by applicant]
US 20160028951A1 · Mayuzumi · 2016 [cited by examiner]
US 20160080699A1 · Scholl · 2016 [cited by applicant]
US 20160119551A1 · Brown · 2016 [cited by examiner]
US 20160234438A1 · Satoh · 2016 [cited by applicant]
US 20160286123A1 · Hwang · 2016 [cited by examiner]
US 20160360104A1 · Zhang · 2016 [cited by applicant]
US 20170011539A1 · Oshima · 2017 [cited by applicant]
US 20170019595A1 · Chen · 2017 [cited by examiner]
US 20170039457A1 · Yu · 2017 [cited by examiner]
US 20170076477A1 · Nakakusu · 2017 [cited by applicant]
US 20170243384A1 · Huang · 2017 [cited by applicant]
US 20170287107A1 · Forutanpour · 2017 [cited by applicant]
US 20170287200A1 · Forutanpour · 2017 [cited by applicant]
US 20170324943A1 · Wu · 2017 [cited by applicant]
US 20170330337A1 · Mizutani · 2017 [cited by applicant]
US 20180012335A1 · Adsumilli · 2018 [cited by applicant]
US 20180027181A1 · Roulet · 2018 [cited by applicant]
US 20180061000A1 · Kawaguchi · 2018 [cited by applicant]
US 20180144488A1 · Kim · 2018 [cited by examiner]
US 20180146136A1 · Yamamoto · 2018 [cited by applicant]
US 20180204311A1 · Kishine · 2018 [cited by examiner]
US 20180204387A1 · Utsugi · 2018 [cited by applicant]
US 20180255307A1 · Douady-Pleven · 2018 [cited by applicant]
US 20180268528A1 · Matsushita · 2018 [cited by applicant]
US 20180286026A1 · Fan · 2018 [cited by applicant]
US 20180343472A1 · Wang · 2018 [cited by applicant]
US 20180357804A1 · Forutanpour · 2018 [cited by applicant]
US 20190014260A1 · Mu · 2019 [cited by applicant]
US 20190014269A1 · Wang · 2019 [cited by applicant]
US 20190014350A1 · Wang · 2019 [cited by applicant]
US 20190020802A1 · Melkote Krishnaprasad · 2019 [cited by applicant]
US 20190098274A1 · Douady-Pleven · 2019 [cited by applicant]
US 20190124260A1 · Lin · 2019 [cited by examiner]
US 20190166299A1 · Kitagawa · 2019 [cited by examiner]
US 20190197671A1 · Jeong · 2019 [cited by examiner]
US 20190208141A1 · Oyama · 2019 [cited by applicant]
US 20190208142A1 · Kitaya · 2019 [cited by applicant]
US 20190236805A1 · Mou · 2019 [cited by applicant]
US 20190287213A1 · Xu · 2019 [cited by examiner]
US 20190306419A1 · Kajiwara · 2019 [cited by applicant]
US 20190313075A1 · Moss · 2019 [cited by applicant]
US 20190325552A1 · Umesawa · 2019 [cited by examiner]
US 20200126179A1 · Yan · 2020 [cited by applicant]
US 20200382757A1 · Oh · 2020 [cited by applicant]
US 20210006718A1 · Chen · 2021 [cited by examiner]
US 20210041780A1 · Takenaka · 2021 [cited by examiner]
US 20210072518A1 · Giordano · 2021 [cited by applicant]
US 20210073956A1 · Guerin · 2021 [cited by applicant]
US 20210084281A1 · Douady · 2021 [cited by applicant]
US 20220198605A1 · Pitts · 2022 [cited by applicant]
JP 2011182176A · 2011 [cited by applicant]
JP 2012141972A · 2012 [cited by applicant]
JP 2013127739A · 2013 [cited by applicant]
JP 2016025516A · 2016 [cited by applicant]
JP 2018524678A · 2018 [cited by applicant]
Zhu, Ling & Liu, Yu & Lai, ShiMing & Jing, Li. (2016). Expanding a fish-eye panoramic image through perspective transformation. (Year: 2016). [cited by examiner]
O. Krams and N. Kiryati, “People detection in top-view fisheye imaging,” 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy, 2017, pp. 1-6 (Year: 2017). [cited by examiner]
X. Shouzhang and W. Fengwen, “Generation of Panoramic View from 360 Degree Fisheye Images Based on Angular Fisheye Projection,” 2011 10th International Symposium on Distributed Computing and Applications to Business, En… [cited by examiner]
International Search Report for PCT Application No. PCT/JP2019/023509, mailed on Aug. 13, 2019. [cited by applicant]
Kensho Hara et al., “Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?”, [online], Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 6546-6555), [searched on May 28… [cited by applicant]
Japanese Office Action for JP Application No. 2021-525512 mailed on Jan. 31, 2023 with English Translation. [cited by applicant]