IP Library Granted Patent US 11,741,632
Granted Patent B2
US 11,741,632 · App. 17/376,339 · Granted Aug 29, 2023

System, information processing method, method of manufacturing product, and recording medium with images of object that moves relative to cameras being captured at predetermined intervals and having different image capture times

Inventors: Hiroto Mizohana (Tokyo, JP); Keita Dan (Tokyo, JP); Kenkichi Yamamoto (Tokyo, JP); Kei Watanabe (Tokyo, JP)
Assignee: Canon Kabushiki Kaisha
G06T7/80G06T1/0014G06T7/0004G06T7/292H04N17/002H04N23/695H04N23/90G06T2207/30164
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,741,632
App. No.
17/376,339
Granted
Aug 29, 2023
Kind
B2
Abstract

A system includes an information processing device. The information processing device is configured to acquire an image group from each of a plurality of cameras. The image group includes a plurality of images of an object that moves relative to each of the plurality of cameras. The plurality of images are captured by a corresponding camera at predetermined intervals and having different image capture times. The information processing device is configured to calculate an amount of movement of the object relative to each of the plurality of cameras, based on the image group acquired from each of the plurality of cameras. The information processing device is configured to detect a lag between image capture timings of the plurality of cameras, by using the amount of movement of the object relative to each of the plurality of cameras.

Claims (54)

1. A system comprising an information processing device, the information processing device being configured to:

acquire a first image group from a first camera of a plurality of cameras, the first image group including at least three first images of an object that moves relative to the plurality of cameras, the at least three first images being captured by the first camera at predetermined intervals and having different image capture times;

acquire a second image group from a second camera of the plurality of cameras, the second image group including at least two second images of the object that moves relative to the plurality of cameras, the at least two second images being captured by the second camera at the predetermined intervals and having different image capture times;

calculate two first amounts of movement of the object between the at least three first images;

calculate a second amount of movement of the object between the at least two second images; and

detect a lag between image capture timings of the first camera and the second camera, by using a relationship between the two first amounts of movement, the second amount of movement, and the predetermined intervals.

2. A system comprising an information processing device, the information processing device being configured to:

acquire an image group from each of a plurality of cameras, the image group including a plurality of images of an object that moves relative to each of the plurality of cameras, the plurality of images being captured by a corresponding camera at predetermined intervals and having different image capture times;

calculate an amount of movement of the object relative to each of the plurality of cameras, based on the image group acquired from each of the plurality of cameras;

detect a lag between image capture timings of the plurality of cameras, by using the amount of movement of the object relative to each of the plurality of cameras;

reset an image capture timing of at least one of the plurality of cameras if the lag exceeds a predetermined value;

acquire a new image group from each of the plurality of cameras, the new image group including a plurality of images of the object that moves relative to each of the plurality of cameras, the plurality of images being captured by a corresponding camera at the predetermined intervals and having different image capture times;

calculate a new amount of movement of the object relative to each of the plurality of cameras, based on the new image group acquired from each of the plurality of cameras; and

detect a new lag between image capture timings of the plurality of cameras, by using the new amount of movement of the object relative to each of the plurality of cameras, for causing the new lag to become equal to or smaller than the predetermined value.

3. The system according to claim 2 , wherein the information processing device is configured to calculate the amount of movement of the object relative to each of the plurality of cameras, by using at least three images of the plurality of images of the image group acquired from each of the plurality of cameras.

4. A system comprising:

an information processing device; and

a measurement portion configured to measure a position and a posture of an object with a corresponding time,

wherein the information processing device is configured to

(1) acquire a first image group from a first camera of a plurality of cameras, the first image group including at least two first images of the object that moves relative to the plurality of cameras, the at least two first images being captured by the first camera at predetermined intervals and having different image capture times,

(2) acquire a second image group from a second camera of the plurality of cameras, the second image group including at least two second images of the object that moves relative to the plurality of cameras, the at least two second images being captured by the second camera at the predetermined intervals and having different image capture times,

(3) calculate a first amount of movement between the at least two first images,

(4) calculate a second amount of movement between the at least two second images, and

(5) detect a lag between image capture timings of the first camera and the second camera, by using a relationship between the first amount of movement, the second amount of movement, and a measurement result obtained by the measurement portion.

5. The system according to claim 2 , further comprising a driving device configured to support and move one of the object and the plurality of cameras.

6. The system according to claim 5 , wherein the driving device is configured to move one of the object and the plurality of cameras nonlinearly with respect to another when the plurality of cameras capture images of the object.

7. The system according to claim 5 , wherein the driving device comprises a robot arm and an end effector attached to the robot arm.

8. The system according to claim 2 , wherein the information processing device is configured to control at least one of the plurality of cameras such that the lag between the image capture timings of the plurality of cameras is decreased.

9. The system according to claim 2 , further comprising a communication interface via which the plurality of cameras and the information processing device communicate with each other.

10. The system according to claim 2 , further comprising the plurality of cameras.

11. An information processing method performed by a processing portion that performs information processing, the method comprising:

acquiring, by the processing portion, an image group from each of a plurality of cameras, the image group including a plurality of images of an object that moves relative to each of the plurality of cameras, the plurality of images being captured by a corresponding camera at predetermined intervals and having different image capture times;

calculating, by the processing portion, an amount of movement of the object relative to each of the plurality of cameras, based on the image group acquired from each of the plurality of cameras;

detecting, by the processing portion, a lag between image capture timings of the plurality of cameras, by using the amount of movement of the object relative to each of the plurality of cameras;

resetting, by the processing portion, an image capture timing of at least one of the plurality of cameras if the lag exceeds a predetermined value;

acquiring, by the processing portion, a new image group from each of the plurality of cameras, the new image group including a plurality of images of the object that moves relative to each of the plurality of cameras, the plurality of images being captured by a corresponding camera at the predetermined intervals and having different image capture times;

calculating, by the processing portion, a new amount of movement of the object relative to each of the plurality of cameras, based on the new image group acquired from each of the plurality of cameras; and

detecting, by the processing portion, a new lag between image capture timings of the plurality of cameras, by using the new amount of movement of the object relative to each of the plurality of cameras, for causing the new lag to become equal to or smaller than the predetermined value.

12. The information processing method according to claim 11 , wherein the processing portion is configured to reset image capture timings of at least one of the plurality of cameras such that a lag between image capture timings of the plurality of cameras is decreased.

13. The information processing method according to claim 11 , wherein the processing portion is configured to determine the amount of movement of the object relative to each of the plurality of cameras, by using at least three images of the plurality of images of the image group acquired from each of the plurality of cameras.

14. The information processing method according to claim 11 , further comprising:

measuring a position and a posture of the object relative to at least one of the plurality of cameras; and

associating the position and the posture of the object with a corresponding time,

wherein the calculating the amount of movement of the object relative to each of the plurality of cameras calculates the amount of movement of the object relative to each of the plurality of cameras by using at least two images of the plurality of images of the image group acquired from each of the plurality of cameras, and

wherein the detecting the lag between image capture timings of the plurality of cameras detects the lag between image capture timings of the plurality of cameras by using the amount of movement of the object relative to each of the plurality of cameras and a result of the measurement.

15. The information processing method according to claim 11 , further comprising:

acquiring the plurality of images from each of the plurality of cameras by causing the camera to capture images of the object at the predetermined intervals, wherein the object moves nonlinearly relative to the plurality of cameras.

16. A method of manufacturing a product, wherein the method uses a system including (a) an information processing device and (b) a driving device configured to support and move one of an object and a plurality of cameras, the driving device including (a) a robot arm and (b) an end effector attached to the robot arm,

wherein the information processing device is configured to

(1) acquire an image group from each of the plurality of cameras, the image group including a plurality of images of an object that moves relative to each of the plurality of cameras, the plurality of images being captured by a corresponding camera at predetermined intervals and having different image capture times,

(2) calculate an amount of movement of the object relative to each of the plurality of cameras, based on the image group acquired from each of the plurality of cameras, and

(3) detect a lag between image capture timings of the plurality of cameras, by using the amount of movement of the object relative to each of the plurality of cameras, and

wherein if the lag detected is equal to or smaller than a predetermined value, the method (a) causes the plurality of cameras to capture images of a workpiece, (b) performs three-dimensional measurement on the workpiece, and (c) manufactures a product by controlling the workpiece by using the robot arm and the end effector.

17. A non-transitory computer-readable recording medium storing a program that causes a computer including the processing portion to perform the method according to claim 11 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2021
From: MIZOHANA, HIROTO; DAN, KEITA; YAMAMOTO, KENKICHI; WATANABE, KEI
To: CANON KABUSHIKI KAISHA
Reel/Frame 057154/0307 →
Priority Claims (1)
JP 2020-124856 · Jul 22, 2020 · national
Continuity (1)
Related Publication 20220028117A1 · Jan 27, 2022
Cited By (1)
US 12,409,004