IP Library Granted Patent US 11,288,870
Granted Patent B2
US 11,288,870 · App. 16/643,070 · Granted Mar 29, 2022

Methods for guiding a user when performing a three dimensional scan and related mobile devices and computer program products

Inventors: Mattias Walls (Häljarp, SE); Francesco Michielin (Stuttgart, DE); Henrick Tuszynski (Lomma, SE); Johannes Elg (Helsingborg, SE); Fredrik Olofsson (Lund, SE); Lars Novak (Bjärred, SE)
Assignee: SONY GROUP CORPORATION
G06T19/006G06T17/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,288,870
App. No.
16/643,070
Granted
Mar 29, 2022
Kind
B2
Abstract

A method of guiding a user when performing a three-dimensional scan of an object includes determining two-dimensional feature points from two-dimensional captured images of the object, determining three-dimensional coordinates for the two-dimensional feature points, determining a cut plane based on the three-dimensional coordinates that divides the object into a target portion and a cut-away portion, and displaying the target portion of the object on a display using a first indicium and the cut-away portion of the object on the display using a second indicium that is different from the first indicium.

Claims (70)

1. A method of guiding a user when performing a three-dimensional scan of an object, comprising:

determining two-dimensional feature points from two-dimensional captured images of the object;

determining three-dimensional coordinates for the two-dimensional feature points;

determining a cut plane based on the three-dimensional coordinates that divides the object into a target portion and a cut-away portion; and

displaying the target portion of the object on a display using a first indicium and the cut-away portion of the object on the display using a second indicium that is different from the first indicium,

wherein the first indicium and the second indicium are audible or touch indicia.

2. The method of claim 1 , wherein determining the three-dimensional coordinates for the two-dimensional feature points comprises:

determining the three-dimensional coordinates for the two-dimensional feature points using a Simultaneous Localization and Mapping (SLAM) method.

3. The method of claim 1 , further comprising:

receiving input from the user selecting use of the cut-plane for the three-dimensional scan.

4. The method of claim 1 , wherein the object is a person and the target portion is a face of the person.

5. The method of claim 1 , wherein the object is a person and the target portion is an entire head of the person.

6. The method of claim 1 , wherein the object comprises a subject and a support structure, the subject being supported by the support structure.

7. The method of claim 1 , wherein the first indicium is a first color and the second indicium is a second color that is different from the first color.

8. A method of guiding a user when performing a three-dimensional scan of an object, comprising:

determining two-dimensional feature points from two-dimensional captured images of the object;

associating a plurality of indicia with a plurality of two-dimensional feature point quality levels;

evaluating a quality of the two-dimensional feature points that were determined from the two-dimensional captured images of the object; and

displaying the object on the display using one or more of the plurality of indicia associated with the plurality of two-dimensional feature point quality levels based on the quality of the two-dimensional feature points that was evaluated.

9. The method of claim 8 , further comprising:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining a precision of the three-dimensional coordinates based on a statistical uncertainty measurement between previously determined ones of the three-dimensional coordinates and subsequently determined ones of the three-dimensional coordinates.

10. The method of claim 9 , wherein the statistical uncertainty measurement is a standard deviation determination.

11. The method of claim 8 , further comprising:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining how much the three-dimensional coordinates have changed between previously determined ones of the three-dimensional coordinates and subsequently determined ones of the three-dimensional coordinates.

12. The method of claim 8 , further comprising:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining when the object moves during performing of the three-dimensional scan.

13. The method of claim 11 , further comprising:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining a density of polygons formed from the three-dimensional coordinates in areas corresponding to at least a portion of the object.

14. The method of claim 11 , further comprising:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining pose information for a camera used to capture the two-dimensional images of the object.

15. A mobile device including a user interface for guiding a user when performing a three-dimensional scan of an object, comprising:

a display;

a processor; and

a computer readable storage medium comprising computer readable program code that when executed by the processor causes the processor to perform operations comprising:

determining two-dimensional feature points from two-dimensional captured images of the object;

associating a plurality of indicia with a plurality of two-dimensional feature point quality levels;

evaluating a quality of the two-dimensional feature points that were determined from the two-dimensional captured images of the object; and

displaying the object on the display using one or more of the plurality of indicia associated with the plurality of two-dimensional feature point quality levels based on the quality of the two-dimensional feature points that was evaluated.

16. The mobile device of claim 15 , wherein the operations further comprise:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining a precision of the three-dimensional coordinates based on a statistical uncertainty measurement between previously determined ones of the three-dimensional coordinates and subsequently determined ones of the three-dimensional coordinates.

17. The mobile device of claim 15 , wherein the operations further comprise:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining how much the three-dimensional coordinates have changed between previously determined ones of the three-dimensional coordinates and subsequently determined ones of the three-dimensional coordinates.

18. The mobile device of claim 15 , wherein the operations further comprise:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining when the object moves during performing of the three-dimensional scan.

19. The mobile device of claim 15 , wherein the operations further comprise:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining a density of polygons formed from the three-dimensional coordinates in areas corresponding to at least a portion of the object.

20. The mobile device of claim 15 , further comprising:

a camera;

wherein the operations further comprise:

determining three-dimensional coordinates for the two-dimensional feature points;

wherein evaluating the quality of the two-dimensional feature points comprises:

determining pose information for a camera used to capture the two-dimensional images of the object.

Assignments (3)
CHANGE OF NAME Recorded Nov 17, 2021
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 058163/0305 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2020
From: WALLS, MATTIAS; MICHIELIN, FRANCESCO; TUSZYNSKI, HENRIK; ELG, JOHANNES; OLOFSSON, FREDRIK; NOVAK, LARS
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 052400/0476 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2020
From: SONY MOBILE COMMUNICATIONS INC.
To: SONY CORPORATION
Reel/Frame 052403/0799 →
Continuity (1)
Related Publication 20200334909A1 · Oct 22, 2020