IP Library › Granted Patent US 12,367,605
Granted Patent B2
US 12,367,605 · App. 17/897,340 · Granted Jul 22, 2025

Image processing apparatus, control method thereof, and image capturing apparatus

Inventors: Tomoyuki Tenkawa (Chiba, JP); Kotaro Yano (Tokyo, JP); Kenshi Saito (Kanagawa, JP)
Assignee: CANON KABUSHIKI KAISHA
G06T7/70G06T7/20G06V10/25G06V10/761G06V10/771G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,367,605
App. No.
17/897,340
Granted
Jul 22, 2025
Kind
B2
Abstract

An image processing apparatus comprises: an image input unit configured to input an image; a detection unit configured to detect an object from the image; an accepting unit configured to accept an input of a locus to the image, a selection unit configured to select, based on a locus region decided by the locus, at least two objects included in a plurality of objects detected by the detection unit; and an integration unit configured to generate an integration region that integrates at least two regions in the image corresponding to the at least two objects selected by the selection unit and set the integration region as a region of interest in the image.

Claims (63)

1. An image processing apparatus comprising:

at least one processor; and

at least one memory having stored thereon instructions which, when executed by the at least one processor, cause the image processing apparatus at least to:

input an image;

detect an object from the image;

based on a selection instruction from a user, select one detection frame of an object, among a plurality of objects detected, and set the detection frame as an initial region of interest in the image;

accept an input of a locus to the image;

select, based on a locus region decided by the locus, at least two objects included in the plurality of objects;

generate an integration region that integrates at least two regions in the image corresponding to the at least two objects selected; and

if the integration region is generated, update a region of interest in the image from the initial region of interest to the integration region.

2. The apparatus according to claim 1 , wherein the instructions further cause the image processing apparatus to display the image on a touch display and accept the input of the locus to the image by a touch operation to the touch display.

3. An image processing apparatus comprising:

at least one processor; and

at least one memory having stored thereon instructions which, when executed by the at least one processor, cause the image processing apparatus at least to:

input an image;

detect an object from the image;

select one detection frame of an object, among a plurality of objects detected, whose distance from a center of the image is not more than a threshold set in advance and whose likelihood is maximum, and set the detection frame as an initial region of interest in the image;

accept an input of a locus to the image;

select, based on a locus region decided by the locus, at least two objects included in the plurality of objects;

generate an integration region that integrates at least two regions in the image corresponding to the at least two objects selected; and

if the integration region is generated, update a region of interest in the image from the initial region of interest to the integration region.

4. An image processing apparatus comprising:

at least one processor; and

at least one memory having stored thereon instructions which, when executed by the at least one processor, cause the image processing apparatus at least to:

input an image,

detect an object from the image,

accept an input of a locus to the image,

decide a plurality of combinations including at least two objects each including an overlap portion with the locus region of a plurality of objects detected,

select, among the plurality of combinations, one combination in which a ratio of an area of an intersection between the integration region and the locus region to an area of a union of the locus region and the integration region that integrates at least two regions corresponding to the at least two objects included in the combination is maximum,

select at least two objects included in the one selected combination,

generate an integration region that integrates at least two regions in the image corresponding to the at least two objects selected, and

set the integration region as a region of interest in the image.

5. The apparatus according to claim 4 , wherein the instructions further cause the image processing apparatus to exclude, from the plurality of combinations, at least one combination including an object for which a difference from an average value of distance information corresponding to at least two objects included in the combination is not less than a predetermined threshold.

6. The apparatus according to claim 1 , wherein the locus region is a rectangular region including the locus.

7. The apparatus according to claim 1 , wherein the locus region is a region having an arbitrary shape surrounded by the locus.

8. An image capturing apparatus comprising:

a camera that performs image capturing to generate a moving image;

at least one processor; and

at least one memory having stored thereon instructions which, when executed by the at least one processor, cause the image processing apparatus at least to:

input an image included in the moving image;

detect an object from the image;

based on a selection instruction from a user, select one detection frame of an object, among a plurality of objects detected, and set the detection frame as an initial region of interest in the image;

accept an input of a locus to the image;

select, based on a locus region decided by the locus, at least two objects included in the plurality of objects;

generate an integration region that integrates at least two regions in the image corresponding to the at least two objects selected;

if the integration region is generated, update a region of interest in the image from the initial region of interest to the integration region; and

track an object included in the moving image in accordance with the region of interest.

9. A control method of an image processing apparatus for setting a region of interest in an image, comprising:

inputting the image;

detecting an object from the image;

based on a selection instruction from a user, selecting one detection frame of an object, among a plurality of objects detected, and setting the detection frame as an initial region of interest in the image;

accepting an input of a locus to the image;

selecting, based on a locus region decided by the locus, at least two objects included in the plurality of objects;

generating an integration region that integrates at least two regions in the image corresponding to the at least two objects selected; and

if the integration region is generated, updating a region of interest in the image from the initial region of interest to the integration region.

10. A non-transitory computer-readable recording medium storing a program for causing a computer to execute a control method of an image processing apparatus for setting a region of interest in an image, the control method comprising:

inputting the image;

detecting an object from the image;

based on a selection instruction from a user, selecting one detection frame of an object, among a plurality of objects detected, and setting the detection frame as an initial region of interest in the image;

accepting an input of a locus to the image;

selecting, based on a locus region decided by the locus, at least two objects included in the plurality of objects;

generating an integration region that integrates at least two regions in the image corresponding to the at least two objects selected; and

if the integration region is generated, updating a region of interest in the image from the initial region of interest to the integration region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2022
From: TENKAWA, TOMOYUKI; YANO, KOTARO; SAITO, KENSHI
To: CANON KABUSHIKI KAISHA
Reel/Frame 062191/0848 →
Priority Claims (1)
JP 2021-142699 · Sep 1, 2021 · national
Continuity (1)
Related Publication 20230065506A1 · Mar 2, 2023
References Cited (19)
US 9691155B2 · Tojo et al. · 2017 [cited by applicant]
US 9773322B2 · Uchiyama et al. · 2017 [cited by applicant]
US 10140726B2 · Nishino et al. · 2018 [cited by applicant]
US 10346465B2 · Gao · 2019 [cited by examiner]
US 10438361B2 · Tojo et al. · 2019 [cited by applicant]
US 11095814B2 · Makita et al. · 2021 [cited by applicant]
US 20200265495A1 · Cho · 2020 [cited by examiner]
US 20210365707A1 · Mao · 2021 [cited by examiner]
CN 112714253A · 2021 [cited by applicant]
EP 2631778A2 · 2013 [cited by applicant]
JP 4849163B2 · 2012 [cited by applicant]
JP 6397454B2 · 2018 [cited by applicant]
JP 2019126091A · 2019 [cited by applicant]
JP 2020021250A · 2020 [cited by applicant]
WO 2021145071A1 · 2021 [cited by applicant]
Ross Girshick et al., “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation” IEEE Conference on Computer Vision and Pattern Recognition (Oct. 2014) pp. 1-21. [cited by applicant]
Wei Liu et al., “SSD: Single Shot MultiBox Detector” Computer Vision-ECCV (Dec. 2016) pp. 1-17. [cited by applicant]
Joseph Redmon et al., “You Only Look Once: Unified, Real-Time Object Detection” IEEE Conference on Computer Vision and Pattern Recognition (May 2016) pp. 779-788. [cited by applicant]
Notice of Reasons for Refusal issued by the Japanese Patent Office on May 19, 2025 in corresponding JP Patent Application No. 2021-142699, with English translation. [cited by applicant]