IP Library › Granted Patent US 10,607,405
Granted Patent B2
US 10,607,405 · App. 15/749,781 · Granted Mar 31, 2020

3D model generating system, 3D model generating method, and program

Inventors: Tony Tung (Tokyo, JP); Jiu Xu (Tokyo, JP); Bjorn Stenger (Tokyo, JP)
Assignee: RAKUTEN, INC.
G06T17/00G06T7/536G06T7/62G06T7/75G06T19/20G06T2210/04G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,607,405
App. No.
15/749,781
Granted
Mar 31, 2020
Kind
B2
Abstract

Processing load on a computer can be reduced while saving the labor of photographing an object to estimate a size of the object. Photographed image acquiring means of a 3D model generating system acquires a photographed image in which a first object and a second object are disposed in a real space. Virtual image generating means generates a virtual image in which a first model representing the first object that is a standard size and a second model representing the second object that is not the standard size are disposed in a virtual space. Parameter determining means determines a size parameter indicating a size of the second object based on a result of comparing the photographed image with the virtual image.

Claims (57)

1. A 3D model generating system comprising at least one processor configured to:

acquire a photographed image in which a first object and a second object, each located in a real space, are captured;

generate a virtual image in which a first model representing the first object and a second model representing the second object are positioned in a virtual space, the first object being a standard size, the second object not being the standard size;

wherein the first model is a standard shape of the first object, and

wherein the second model is a shape of the second object, the shape being detected based on the photographed image;

acquire an orientation map of the photographed image and an orientation map of the virtual image;

count a number of pixels having matched directions of the orientation maps between the photographed image and the virtual image; and

determine a size parameter indicating a size of the second object based on the counted number of pixels.

2. The 3D model generating system according to claim 1 , wherein

the second object is partially hidden by the first object in the photographed image,

the second model is partially hidden by the first model in the virtual image, and

the at least one processor determines the size parameter based on the counted number of pixels of the second object other than the part hidden by the first object and the counted number of pixels of the second model other than the part hidden by the first model.

3. The 3D model generating system according to claim 1 , wherein the at least one processor positions, in the virtual space, the first model oriented in a direction that is determined based on the photographed image.

4. The 3D model generating system according to claim 1 , wherein

the at least one processor determines a position parameter indicating a position of the first model based on a result of comparing the photographed image with the virtual image.

5. The 3D model generating system according to claim 4 , wherein

the at least one processor determines a range in which the position parameter can be changed based on the first object captured in the photographed image.

6. The 3D model generating system according to claim 1 , wherein

the at least one processor determines an orientation parameter indicating an orientation of the first model based on a result of comparing the photographed image with the virtual image.

7. The 3D model generating system according to claim 1 , wherein

the at least one processor determines a position parameter indicating the position of the first model based on the relative relationship between the first model and the second model in the virtual space.

8. The 3D model generating system according to claim 1 , wherein

the at least one processor determines an orientation parameter indicating the orientation of the first model based on the relative relationship between the first model and the second model in the virtual space.

9. The 3D model generating system according to claim 1 , wherein

a plurality of first objects are captured in the photographed image,

a plurality of first models respectively corresponding to the first objects are positioned in the virtual space, and

the at least one processor determines the size parameter of the second objects based on the counted number of pixels.

10. The 3D model generating system according to claim 1 , wherein

a plurality of first objects are captured in the photographed image,

a plurality of first models respectively corresponding to the first objects are positioned in the virtual space, and

the at least one processor determines a position parameter indicating respective positions of the first models based on a relative relationship between the first models in the virtual space.

11. The 3D model generating system according to claim 1 , wherein

a plurality of first objects are captured in the photographed image,

a plurality of first models respectively corresponding to the first objects are positioned in the virtual space, and

the at least one processor determines an orientation parameter indicating respective orientations of the first models based on the relative relationship between the first models in the virtual space.

12. The 3D model generating system according to claim 1 , wherein

the first model has a three-dimensional shape with a depth.

13. The 3D model generating system according to claim 1 , wherein

the first object is furniture located in a room,

the second object is a wall of the room, and

the at least one processor determines the size parameter of the second model representing the wall.

14. A 3D model generating method comprising:

acquiring a photographed image in which a first object and a second object, each located in a real space, are captured;

generating a virtual image in which a first model representing the first object and a second model representing the second object are positioned in a virtual space, the first object being a standard size, the second object not being the standard size;

wherein the first model is a standard shape of the first object, and

wherein the second model is a shape of the second object, the shape being detected based on the photographed image;

acquire an orientation map of the photographed image and an orientation map of the virtual image;

count a number of pixels having matched directions of the orientation maps between the photographed image and the virtual image; and

determining a size parameter indicating a size of the second object based on the counted number of pixels.

15. A non-transitory computer-readable information storage medium for storing a program for causing a computer to:

acquire a photographed image in which a first object and a second object, each located in a real space, are captured;

generate a virtual image in which a first model representing the first object and a second model representing the second object are positioned in a virtual space, the first object being a standard size, the second object not being the standard size;

wherein the first model is a standard shape of the first object, and

wherein the second model is a shape of the second object, the shape being detected based on the photographed image;

acquire an orientation map of the photographed image and an orientation map of the virtual image;

count a number of pixels having matched directions of the orientation maps between the photographed image and the virtual image; and

determine a size parameter indicating a size of the second object based on the counted number of pixels.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE PATENT NUMBERS 10342096;10671117; 10716375; 10716376;10795407;10795408; AND 10827591 PREVIOUSLY RECORDED AT REEL: 58314 FRAME: 657. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Feb 29, 2024
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 068066/0103 →
CHANGE OF NAME Recorded Dec 6, 2021
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 058314/0657 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2018
From: TUNG, TONY; XU, JIU; STENGER, BJORN
To: RAKUTEN, INC.
Reel/Frame 044826/0276 →
Continuity (1)
Related Publication 20190005715A1 · Jan 3, 2019
Cited By (1)
US 12,316,689