IP Library Granted Patent US 11,879,750
Granted Patent B2
US 11,879,750 · App. 16/879,000 · Granted Jan 23, 2024

Distance estimation using multi-camera device

Inventors: Eyal Zak (Megiddo, IL); Ozi Egri (Yassur, IL)
Assignee: SNAP INC.
G01C3/085G01C3/08G03B35/02G03B35/10G06T7/593G06V10/147G06V10/17H04N23/45H04N23/51G06T2200/24G06T2207/10016G06T2207/20221G06T2207/30041G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,879,750
App. No.
16/879,000
Granted
Jan 23, 2024
Kind
B2
Abstract

A method and apparatus for measuring a distance to an object, be a device ( 100 ) having at least two cameras ( 104, 106 ), is described. One or more first images including the object are acquired by a first camera of the device and one or more first reference images are acquired by a second camera of the device, while the device is in a first position. One or more second images including the object and one or more second reference images are acquired by cameras of the device, while the device is in a second position, different from the first position. Based on the first and second reference images, information on the displacement of at least one camera of the device between the first and second position are determined. The distance from the device to the object is calculated based on the first and second images including e object and the determined information on the displacement of the at least one camera.

Claims (61)

1. A method of measuring a distance to an object, by a device having at least two cameras, comprising:

acquiring, by a first camera of the device while the device is in a first position and a first side of a housing of the device is facing the object, one or more first images that include the object, the first camera being located on the first side of the housing;

acquiring, by a second camera of the device while the device is in the first position, one or more first reference images, the second camera being located on a second side of the housing opposite the first side of the housing, a screen being located on the second side of the housing;

after the acquiring of the one or more first images and the one or more first reference images, displaying on the screen:

an image that is rotated in order to cause a user to rotate the device; and

an indication of an extent of matching between the one or more first reference images and a current image acquired by the second camera, the indication comprising a numerical score that indicates the extent of matching;

acquiring, by the first camera of the device while the device is in a second position, one or more second images including the object;

acquiring, by the second camera of the device while the device is in the second position, one or more second reference images;

determining an extent of relative rotation between the one or more first and second reference images based on a comparison of the one or more first reference images to the one or more second reference images;

determining, based on the extent of relative rotation, information on a displacement of at least one camera of the device between the first and second positions; and

calculating the distance from the device to the object, based on the one or more first and second images including the object and the determined information on the displacement of the at least one camera.

2. The method of claim 1 , wherein acquiring the one or more first images that include the object comprises acquiring a plurality of images while the device is in the first position and generating a single image that includes the object from the plurality of acquired images.

3. The method of claim 2 , wherein generating the single image that includes the object from the plurality of acquired images comprises selecting one of the plurality of acquired images.

4. The method of claim 2 , wherein generating the single image that includes the object from the plurality of acquired images comprises generating a combination of some or all of the plurality of acquired images.

5. The method of claim 2 , wherein generating the single image that includes the object from the plurality of acquired images comprises analyzing quality of the plurality of acquired images and discarding images of a quality below a given threshold.

6. The method of claim 1 , wherein the calculation is based on a fixed measurement comprising a size of a body organ and wherein the body organ is included in the one or more first and second reference images.

7. The method of claim 6 , wherein the body organ comprises a cornea.

8. The method of claim 1 , wherein the calculation is based on a fixed measurement comprising a distance between the first and second cameras of the at least two cameras.

9. The method of claim 1 , wherein the first and second cameras of the at least two cameras are on opposite sides of the device.

10. The method of claim 1 , wherein the one or more first images that includes the object and the one or more first reference images are acquired in pairs of reference and object images, the reference and object images of each pair being acquired concurrently.

11. A device, comprising:

a housing;

a screen;

a first camera included in the housing, the first camera being located on a first side of the housing;

a second camera included in the housing, the second camera being located on a second side of the housing opposite the first side of the housing, the screen being located on the second side of the housing; and

a processor, included in the housing, configured to perform operations comprising:

receiving one or more first images including an object, acquired by the first camera while the device is in a first position and the first side of the housing is facing the object;

receiving one or more first reference images acquired by the second camera while the device is in the first position;

after the receiving of the one or more first images and the one or more first reference images, displaying on the screen:

an image that is rotated in order to cause a user to rotate the device; and

an indication of an extent of matching between the one or more first reference images and a current image acquired by the second camera, the indication comprising a numerical score that indicates the extent of matching;

receiving one or more second images including the object acquired by the first camera while the device is in a second position;

receiving one or more second reference images acquired by the second camera while the device is in the second position;

determining an extent of relative rotation between the one or more first and second reference images based on a comparison of the one or more first reference images to the one or more second reference images;

determining, based on the extent of relative rotation, information on a displacement of at least one camera of the device between the first and second positions; and

calculating a distance from the device to the object, based on the one or more first and second images including the object and the determined information on the displacement of the at least one camera.

12. The device of claim 11 , wherein the first and second cameras are on opposite sides of the housing.

13. The device of claim 11 , wherein the operations further comprise:

controlling the first and second cameras to acquire the one or more first images and the one or more first reference images in pairs of concurrently acquired images and respective reference images.

14. The device of claim 11 , wherein the operations further comprise:

analyzing an input stream provided by the second camera relative to the one or more first reference images, after receiving the one or more first images and the one or more first reference images, in order to determine when the housing is in a position suitable for acquiring the one or more second reference images and the one or more second images.

15. The device of claim 11 , comprising the screen and wherein the operations further comprise:

displaying on the screen, after receiving the one or more first images and the one or more first reference images, a stream of images acquired by the second camera, in order to allow the user to position the housing for acquiring the one or more second images and the one or more second reference images.

16. A computer program product for distance estimation, comprising:

a computer readable non-transitory storage medium having computer readable program code embodied therewith, which when executed by a processor performs operations comprising:

receiving one or more first images including an object acquired by a first camera of a device while the device is in a first position and a first side of a housing of the device is facing the object, the first camera being located on the first side of the housing;

receiving one or more first reference images acquired by a second camera of the device while the device is in the first position, the second camera being located on a second side of the housing opposite the first side of the housing, a screen being located on the second side of the housing;

after the receiving of the one or more first images and the one or more first reference images, displaying on the screen:

an image that is rotated in order to cause a user to rotate the device; and

an indication of an extent of matching between the one or more first reference images and a current image acquired by the second camera, the indication comprising a numerical score that indicates the extent of matching;

receiving one or more second images including the object acquired by the first camera while the device is in a second position;

receiving one or more second reference images acquired by the second camera while the device is in the second position;

determining an extent of relative rotation between the one or more first and second reference images based on a comparison of the one or more first reference images to the one or more second reference images;

determining, based on the extent of relative rotation, information on a displacement of at least one camera of the device between the first and second positions; and

calculating a distance from the device to the object, based on the one or more first and second images including the object and the determined information on the displacement of the at least one camera.

17. The computer program product of claim 16 , wherein the operations further comprise:

controlling the first and second cameras to acquire the one or more first images and the one or more first reference images in pairs of concurrently acquired images and respective reference images.

18. The computer program product of claim 16 , wherein the operations further comprise:

analyzing an input stream provided by the second camera relative to the one or more first reference images, after receiving the one or more first images and the one or more first reference images, in order to determine when the device is in a position suitable for acquiring the one or more second reference images and the one or more second images.

19. The computer program product of claim 16 , wherein the operations further comprise:

displaying on the screen of the device, after receiving the one or more first images and the one or more first reference images, a stream of images acquired by the second camera, in order to allow the user to position the device for acquiring the one or more second images and the one or more second reference images.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 25, 2023
From: ZAK, EYAL; EGRI, OZI
To: CIMAGINE MEDIA LTD.
Reel/Frame 064706/0195 →
CHANGE OF NAME Recorded Aug 25, 2023
From: CIMAGINE MEDIA LTD
To: C3D AUGMENTED REALITY SOLUTIONS LTD
Reel/Frame 064706/0383 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 25, 2023
From: SNAP GROUP LIMITED (SNAPCHAT LIMITED)
To: SNAP INC.
Reel/Frame 064706/0526 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 25, 2023
From: CIMAGINE MEDIA LTD.
To: SNAPCHAT LIMITED
Reel/Frame 064718/0736 →
Continuity (6)
Continuation 16591292 · Oct 2, 2019
Continuation 16219519 · Dec 13, 2018
Continuation 14783119
Provisional Application 61809447 · Apr 8, 2013
Provisional Application 61809464 · Apr 8, 2013
Related Publication 20200293805A1 · Sep 17, 2020
Cited By (1)
US 12,209,883