IP Library Granted Patent US 9,646,408
Granted Patent B2
US 9,646,408 · App. 14/270,427 · Granted May 9, 2017

Methods and systems for generating a three dimensional representation of a subject

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,646,408
App. No.
14/270,427
Granted
May 9, 2017
Kind
B2
Abstract

According to one embodiment, a method of generating a three dimensional representation of a subject from a depth image, comprises comparing a depth image of the subject with a plurality of representative images, wherein each representative image is associated with a respective parametric model of a subject; identifying a representative image of the plurality of representative images as a closest representative image to the depth image of the subject; selecting the parametric model associated with the closest representative image to the depth image; and generating a three dimensional representation of the subject by fitting the selected parametric model to the depth image of the subject.

Claims (30)

1. A method of generating a three dimensional representation of the shape of a subject from a depth image, the method comprising

comparing a depth image of the subject with a plurality of representative images, wherein each representative image is associated with a respective parametric model for the shape and pose of a subject, each respective parametric model including numerical variables and the representative images being representations of a mean value of said variables for a cluster of possible shapes and poses of the subject;

identifying a representative image of the plurality of representative images as a closest representative image to the depth image of the subject;

selecting the parametric model for the shape and pose of the subject associated with the closest representative image to the depth image; and

generating a three dimensional representation of the subject by fitting the selected parametric model to the depth image of the subject.

2. The method according to claim 1 , wherein fitting the selected parametric model to the depth image of the subject comprises minimising an energy function, wherein the energy function comprises a distance term which depends on the distance between points on the depth image of the subject and corresponding points on the selected parametric model.

3. The method according to claim 2 , wherein the distance term gives a preference to points on the model being inside a volume defined by points on the depth image.

4. The method according to claim 2 , wherein the distance term comprises a gravity function dependant on the normal to a surface of the selected model and a gravity direction.

5. The method according to claim 1 , wherein each representative image is a depth image calculated from the respective parametric model.

6. The method according to claim 1 , wherein each representative image is a weighted combination of a depth image and an alpha image calculated from the respective parametric model.

7. The method according to claim 1 , further comprising normalising the orientation of the subject and/or the depth of the subject in the depth image before comparing the depth image of the subject with the plurality of representative images.

8. The method according to claim 1 , further comprising reducing the resolution of the depth image before comparing the depth image of the subject with the plurality of representative images.

9. The method according to claim 1 , further comprising calculating a size indicator and/or a measurement of the subject from the three dimensional representation.

10. A non-transitory computer readable medium carrying processor executable instructions which when executed on a processor cause the processor to carry out the method according to claim 1 .

11. A system for generating a three dimensional representation of the shape of a subject from a depth image, the system comprising

storage for a plurality of parametric models for the shape and pose of the subject and a plurality of representative images, each representative image being associated with a respective parametric model, each respective parametric model including numerical variables and the representative images being representations of a mean value of said variables for a cluster of possible shapes and poses of the subject;

a selection module operable to

compare the depth image of the subject with each of the plurality of representative images;

identify a representative image of the plurality of representative images as a closest representative image to the depth image of the subject; and

select the parametric model for the shape and pose of the subject associated with the closest representative image to the depth image; and

a fitting module operable to generate a three dimensional representation of the subject by fitting the selected parametric model to the depth image of the subject.

12. The system according to claim 11 , wherein the fitting module is operable to fit the selected parametric model to the depth image of the subject by minimising an energy function, wherein the energy function comprises a distance term which depends on the distance between points on the depth image of the subject and corresponding points on the selected parametric model.

13. The system according to claim 12 , wherein the distance term gives a preference to points on the model being inside a volume defined by points on the depth image.

14. The system according to claim 12 , wherein the distance term comprises a gravity function dependant on the normal to a surface of the selected model and a gravity direction.

15. The system according to claim 11 , wherein each representative image is a depth image calculated from the respective parametric model.

16. The system according to claim 11 , wherein each representative image is a weighted combination of a depth image and an alpha image calculated from the respective parametric model.

17. The system according to claim 11 , wherein each representative image as a resolution of less than 100×100 pixels.

18. The system according to claim 11 , wherein the selection module is operable to normalise the orientation of the subject and/or the depth of the subject in the depth image before comparing the depth image of the subject with the plurality of representative images.

19. The system according to claim 11 , being further operable to calculate a size indicator and/or a measurement of the subject from the three dimensional representation.

20. The system according to claim 11 , further comprising a depth image capture device configured to capture the depth image of the subject.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY'S ADDRESS PREVIOUSLY RECORDED ON REEL 048547 FRAME 0187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST. Recorded May 6, 2020
From: KABUSHIKI KAISHA TOSHIBA
To: TOSHIBA DIGITAL SOLUTIONS CORPORATION
Reel/Frame 052595/0307 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ADD SECOND RECEIVING PARTY PREVIOUSLY RECORDED AT REEL: 48547 FRAME: 187. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 13, 2019
From: KABUSHIKI KAISHA TOSHIBA
To: KABUSHIKI KAISHA TOSHIBA; TOSHIBA DIGITAL SOLUTIONS CORPORATION
Reel/Frame 050041/0054 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 8, 2019
From: KABUSHIKI KAISHA TOSHIBA
To: TOSHIBA DIGITAL SOLUTIONS CORPORATION
Reel/Frame 048547/0187 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 18, 2014
From: JOHNSON, SAM; PERBET, FRANK; STENGER, BJORN; PHAM, MINH-TRI; WOODFORD, OLIVER; GHERARDI, RICCARDO; CIPOLLA, ROBERTO
To: KABUSHIKI KAISHA TOSHIBA
Reel/Frame 033124/0001 →