IP Library › Granted Patent US 11,741,624
Granted Patent B2
US 11,741,624 · App. 17/076,507 · Granted Aug 29, 2023

Method and system for determining spatial coordinates of a 3D reconstruction of at least part of a real object at absolute spatial scale

Inventors: Sebastian Knorr (Mountain View, CA); Daniel Kurz (Sunnyvale, CA)
Assignee: Apple Inc.
G06T7/579G06T7/251G06T7/277G06T2207/10016G06T2207/10028G06T2207/10048G06T2207/20076G06T2207/20092G06T2207/30201G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,741,624
App. No.
17/076,507
Granted
Aug 29, 2023
Kind
B2
Abstract

Determining spatial coordinates of a 3D reconstruction includes obtaining, from a first camera system, a first image comprising a first real object, obtaining, from a second camera system, a second image comprising a second real object associated with known geometric properties, wherein the first camera system and the second camera system have a known spatial relationship, and determining a scale of the face based on the second image and the known geometric properties of the at least part of the second real object. Determining the spatial coordinates of the 3D reconstruction also includes determining a pose of the first and second camera systems, and determining the spatial coordinates based on the pose of the first camera system and the scale of the at least part of the second real object.

Claims (39)

1. A method of determining spatial coordinates of a 3D reconstruction of at least part of a first real object comprising:

obtaining, from a first camera system, a first image comprising at least part of a first real object;

obtaining, from a second camera system, a second image comprising at least part of a second real object associated with known geometric properties, wherein the first camera system and the second camera system have a known spatial relationship;

determining a scale of at least part of the second real object based on the second image and the known geometric properties of the at least part of the second real object; and

determining, based on the known spatial relationship and the scale of the at least part of the second real object, one or more coordinates of a geometrical model of the first real object at absolute scale.

2. The method of claim 1 , wherein the second real object comprises at least part of a human face.

3. The method of claim 2 , wherein the scale of at least part of the human face is determined based on a generic face model.

4. The method of claim 2 , wherein the human face belongs to a particular user, and wherein the scale of the at least part of the human face is determined based on a user-specific face model.

5. The method of claim 1 , wherein the first camera system and the second camera system are comprised in an electronic device.

6. The method of claim 1 , wherein a frustum of the first camera system and a frustum of the second camera system do not overlap.

7. The method of claim 1 , further comprising:

presenting a virtual object at a scale consistent with the first real object according to the one or more coordinates of a geometrical model of the first real object at absolute scale.

8. A non-transitory computer readable medium comprising computer readable code for determining spatial coordinates of a 3D reconstruction of at least part of a first real object, the computer readable code executable by a processor to:

obtain, from a first camera system, a first image comprising at least part of a first real object;

obtain, from a second camera system, a second image comprising at least part of a second real object associated with known geometric properties, wherein the first camera system and the second camera system have a known spatial relationship;

determine a scale of at least part of the second real object based on the second image and the known geometric properties of the at least part of the second real object; and

determine, based on the known spatial relationship and the scale of the at least part of the second real object, one or more coordinates of a geometrical model of the first real object at absolute scale.

9. The non-transitory computer readable medium of claim 8 , wherein the second real object comprises at least part of a human face.

10. The non-transitory computer readable medium of claim 9 , wherein the scale of at least part of the human face is determined based on a generic face model.

11. The non-transitory computer readable medium of claim 9 , wherein the human face belongs to a particular user, and wherein the scale of the at least part of the human face is determined based on a user-specific face model.

12. The non-transitory computer readable medium of claim 8 , wherein the first camera system and the second camera system are comprised in an electronic device.

13. The non-transitory computer readable medium of claim 8 , wherein a frustum of the first camera system and a frustum of the second camera system do not overlap.

14. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to:

present a virtual object at a scale consistent with the first real object according to the one or more coordinates of a geometrical model of the first real object at absolute scale.

15. A system for determining spatial coordinates of a 3D reconstruction of at least part of a first real object, comprising:

a first camera system;

a second camera system having a known spatial relationship with the first camera system;

one or more processors; and

a computer readable medium comprising computer readable code executable by the one or more processors to:

obtain, from the first camera system, a first image comprising at least part of a first real object;

obtain, from the second camera system, a second image comprising at least part of a second real object associated with known geometric properties;

determine a scale of at least part of the second real object based on the second image and the known geometric properties of the at least part of the second real object; and

determine, based on the known spatial relationship and the scale of the at least part of the second real object, one or more coordinates of a geometrical model of the first real object at absolute scale.

16. The system of claim 15 , wherein the second real object comprises at least part of a human face.

17. The system of claim 16 , wherein the scale of at least part of the human face is determined based on a generic face model.

18. The system of claim 16 , wherein the human face belongs to a particular user, and wherein the scale of the at least part of the human face is determined based on a user-specific face model.

19. The system of claim 15 , wherein a frustum of the first camera system and a frustum of the second camera system do not overlap.

20. The system of claim 15 , further comprising computer readable code to:

present a virtual object at a scale consistent with the first real object according to the one or more coordinates of a geometrical model of the first real object at absolute scale.

Continuity (3)
Continuation 16546833 · Aug 21, 2019
Continuation 15525310
Related Publication 20210110560A1 · Apr 15, 2021