IP Library Granted Patent US 9,723,225
Granted Patent B2
US 9,723,225 · App. 14/361,477 · Granted Aug 1, 2017

Moving-image processing device, moving-image processing method, and information recording medium

Inventors: Rikio Onai (Chofu, JP); Makoto Okabe (Chofu, JP); Tatsuya Kurihara (Chofu, JP); Masahiro Sanjo (Shinagawa-ku, JP); Hiromi Hirano (Shinagawa-ku, JP)
Assignee: Rakuten, Inc.
H04N5/265G06T7/50G06T7/73G06T13/80H04N5/272H04N13/004G06T2207/10016G06T2207/10028G06T2207/20221G06T2210/21
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,723,225
App. No.
14/361,477
Granted
Aug 1, 2017
Kind
B2
Abstract

A moving-image processing device ( 101 ) synthesizes a first moving image and a second moving image. In the first moving image, a first object is drawn, and accompanied by the anterior-surface depth of the first object. In the second moving image, a second object is drawn, and accompanied by the anterior-surface depth of the second object. A rear-surface-depth acquirer ( 102 ) acquires the rear-surface depth of the first object and the rear-surface depth of the second object. A clash determiner ( 103 ) functioning as an interference determiner refers to the anterior-surface depth and rear-surface depth of the first object to obtain an occupancy space which the first object can occupy, and refers to the anterior-surface depth and the rear-surface depth of the second object to determine whether the occupancy space and the second object drawn in certain frames of the second moving image satisfy interference conditions. A corrector ( 106 ) functioning as a range setter sets, on the basis of a determination result from the interference determiner, the possible range in which the second object can be positioned without interfering with the occupancy space.

Claims (35)

1. A moving-image processing device for synthesizing a first moving image and a second moving image, wherein a first object is depicted in the first moving image and is accompanied by anterior-surface depth information of the first object and a second object is depicted in the second moving image and is accompanied by anterior-surface depth information of the second object, the moving-image processing device comprising:

at least one non-transitory memory operable to store program code;

at least one processor operable to read said program code and operate as instructed by said program code, said program code including:

acquiring code configured to cause said at least one processor to acquire rear-surface depth information of the first object, and rear-surface depth information of the second object;

interference determining code configured to cause said at least one processor to find an occupancy zone that can be occupied by the first object depicted in the first moving image, referencing the anterior-surface depth information and the rear-surface depth information of the first object, and to determine whether or not the occupancy zone and the second object depicted in a given frame of the second moving image satisfy an interference condition, by referencing the anterior-surface depth information and the rear-surface depth information of the second object; and

range setting code configured to cause said at least one processor to set a possible range where the second object can be positioned without interfering with the occupancy zone, from the determination result.

2. The moving-image processing device according to claim 1 , wherein the acquiring code causes said at least one processor to acquire first depth length associated in advance with an attribute of the first object, from a database, and acquires second depth length associated in advance with an attribute of the second object, from the database, and acquires rear-surface depth information of the first object from the anterior-surface depth information of the first object and the first depth length, and acquires rear-surface depth information of the second object from the anterior-surface depth information of the second object and the second depth length.

3. The moving-image processing device according to claim 1 , wherein the range setting code causes said at least one processor to acquire a positioning condition for the second object associated in advance with an attribute of the second object, from a database, and sets the possible range so that the positioning condition is satisfied.

4. The moving-image processing device according to claim 1 , wherein the range setting code causes said at least one processor to set the possible range so that the occupancy zone and the second object do not satisfy the interference condition, for all frames of the second moving image.

5. The moving-image processing device according to claim 1 , wherein:

in one among the first moving image and the second moving image, a non-deforming transformation indicating translation within a three-dimensional space is applied; and

the possible range is expressed by a range of values that can be taken by a

transformation parameter indicating an amount of translation relating to the non-deforming transformation.

6. The moving-image processing device according to claim 5 , wherein:

the non-deforming transformation is a translation with shift amounts in a horizontal direction and a vertical direction as the transformation parameter;

the one among the first moving image and the second moving image is the second moving image; and

a possible range of shift amounts in the horizontal direction and the vertical direction set by the range setting code, and a given frame of the first moving image, are displayed on a screen so as to prompt a user to correct the shift amounts in the horizontal direction and the vertical direction.

7. The moving-image processing device according to claim 5 , wherein:

the non-deforming transformation is a translation with shift amounts in a horizontal direction, a vertical direction and a depth direction as the transformation parameter;

the one among the first moving image and the second moving image is the second moving image; and

a possible range of shift amounts in the horizontal direction and the vertical direction set with respect to a shift amount in a current depth direction by the range setting code, and a given frame of the first moving image, are displayed on a screen to prompt a user to correct the shift amounts in the horizontal direction, the vertical direction and the depth direction.

8. The moving-image processing device according to claim 5 , wherein said program code further includes:

correcting code configured to cause said at least one processor to correct the one of the moving images by selecting any of the transformation parameters from the possible range and applying the non-deforming transformation; and

moving image generating code configured to cause said at least one processor to generate a third moving image synthesizing the first moving image and the second moving image.

9. The moving-image processing device according to claim 8 , wherein said program code further includes:

initial value setting code configured to cause said at least one processor to set an initial value of the transformation parameters;

wherein the correcting code causes said at least one processor to select transformation parameters closest to the set initial value, from the possible range, if the set initial value is not contained in the possible range.

10. A moving-image processing method, executed by a moving-image processing device including at least one processor, for synthesizing a first moving image and a second moving image, wherein a first object is depicted in the first moving image and is accompanied by anterior-surface depth information of the first object and a second object is depicted in the second moving image and is accompanied by anterior-surface depth information of the second object, the moving-image processing method comprising:

acquiring, using said at least one processor, rear-surface depth information of the first object, and rear-surface depth information of the second object;

finding, using said at least one processor, an occupancy zone that can be occupied by the first object depicted in the first moving image, referencing, using said at least one processor, the anterior-surface depth information and the rear-surface depth information of the first object, and determining, using said at least one processor, whether or not the occupancy zone and the second object depicted in a given frame of the second moving image satisfy an interference condition, by referencing the anterior-surface depth information and the rear-surface depth information of the second object; and

setting, using said at least one processor, a possible range where the second object can be positioned without interfering with the occupancy zone, from the determination result.

11. A non-transitory computer-readable information recording medium on which is recorded a program for synthesizing a first moving image and a second moving image, wherein a first object is depicted in the first moving image and is accompanied by anterior-surface depth information of the first object and a second object is depicted in the second moving image and is accompanied by anterior-surface depth information of the second object, the program causing a computer to:

acquire rear-surface depth information of the first object, and rear-surface depth information of the second object;

find an occupancy zone that can be occupied by the first object depicted in the first moving image, referencing the anterior-surface depth information and the rear-surface depth information of the first object, and determining whether or not the occupancy zone and the second object depicted in a given frame of the second moving image satisfy an interference condition, by referencing the anterior-surface depth information and the rear-surface depth information of the second object; and

set a possible range where the second object can be positioned without interfering with the occupancy zone, from the determination result from the interference determiner.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE PATENT NUMBERS 10342096;10671117; 10716375; 10716376;10795407;10795408; AND 10827591 PREVIOUSLY RECORDED AT REEL: 58314 FRAME: 657. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Feb 29, 2024
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 068066/0103 →
CHANGE OF NAME Recorded Dec 6, 2021
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 058314/0657 →
CHANGE OF ADDRESS Recorded Feb 3, 2016
From: RAKUTEN, INC.
To: RAKUTEN, INC.
Reel/Frame 037690/0315 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 29, 2014
From: ONAI, RIKIO; OKABE, MAKOTO; KURIHARA, TATSUYA; SANJO, MASAHIRO; HIRANO, HIROMI
To: RAKUTEN, INC.
Reel/Frame 032990/0326 →
Priority Claims (3)
JP 2012-161924 · Jul 20, 2012 · national
WO PCT/JP2012/072988 · Sep 7, 2012 · international
WO PCT/JP2012/072989 · Sep 7, 2012 · international
Continuity (1)
Related Publication 20140340477A1 · Nov 20, 2014