Granted Patent B2

US 12,198,376 · App. 18/144,171 · Granted Jan 14, 2025

Inventors: Piers Cowburn (London, GB); David Li (London, GB); Isac Andreas Müller Sandvik (London, GB); Qi Pan (London, GB); Matan Zohar (Rishon LeZion, IL)

Assignee: Snap Inc.

G06T7/74G06F3/011G06T19/006G06T2207/30204G06T2219/004G06T2219/024

Abstract

Claims (64)

1. A method comprising:

initializing a shared augmented reality (AR) session by a first device and by a second device;

detecting a marker using a camera of the second device;

capturing a data stream of images of the marker using the camera;

determining a transformation between the first device and the second device using an image of the marker from the data stream of images of the marker, wherein determining the transformation comprises:

receiving, by the second device, a stream of first device transformations from the first device,

determining, by the second device, an alignment of the stream of first device transformations with the data stream of images of the marker, and

selecting, based on the alignment, the image of the marker from the data stream of images of the marker and a corresponding first device transformation from the stream of first device transformations;

determining a common coordinate frame using the transformation;

generating the shared AR session using the common coordinate frame; and

causing the shared AR session to be displayed by the first device and by the second device.

2. The method of claim 1 , wherein initializing the shared AR session by the first device and by the second device comprises:

determining, by the first device, a first device transformation, wherein the first device transformation is based on a first device current pose and a first device origin, wherein the first device origin is an origin of a coordinate frame tracked by the first device;

determining, by the second device, a second device transformation, wherein the second device transformation is based on a second device current pose and a second device origin, wherein the second device origin is an origin of a coordinate frame tracked by the second device.

3. The method of claim 2 , wherein determining the transformation between the first device and the second device using the image of the marker further comprises:

determining, by the second device, a second offset based on the first device transformation, wherein the second offset is an offset of the second device current pose and the first device origin.

4. The method of claim 3 , wherein determining the transformation between the first device and the second device using the image of the marker further comprises:

determining the transformation using a position of the marker in the image, a scale of the marker in the image, or a rotation of the marker in the image.

5. The method of claim 4 , further comprising:

performing, by the second device, a gravity check, wherein performing the gravity check includes:

determining whether a direction of gravity based on the first device transformation corresponds to a direction of gravity based on the transformation between the first device and the second device.

6. The method of claim 1 , wherein the marker includes a session identifier.

7. The method of claim 6 , further comprising:

extracting, by the second device, the session identifier from the image of the marker.

8. The method of claim 7 , further comprising:

accessing by the first device a communication session using the session identifier included in the marker; and

accessing, by the second device, the communication session using the session identifier that is extracted from the image of the marker.

9. The method of claim 1 , wherein the marker is a Quick Response (QR) code, a natural feature marker, or a user interface of an application displayed on a display of the first device, or the first device.

10. The method of claim 1 , wherein the marker is a rotating marker and the selecting of the image of the marker from the data stream of images of the marker is based on a rotation speed of the marker.

11. A system comprising:

a processor; and

a memory component having instructions stored thereon, when executed by the processor, causes the processor to perform operations comprising:

detecting a marker using a camera of a second device;

capturing a data stream of images of the marker using the camera;

determining a transformation between a first device and the second device using an image of the marker from the data stream of images of the marker, wherein determining the transformation comprises:

receiving from the first device, a stream of first device transformations,

determining an alignment of the stream of first device transformations with the data stream of images of the marker, and

selecting, based on the alignment, the image of the marker from the data stream of images of the marker and a corresponding first device transformation from the stream of first device transformations;

determining a common coordinate frame using the transformation;

generating a shared AR session using the common coordinate frame; and

causing the shared AR session to be displayed by the first device and by the second device.

12. The system of claim 11 , wherein the marker includes a session identifier.

13. The system of claim 12 , wherein the operations further comprising:

extracting the session identifier from the image of the marker.

14. The system of claim 13 , wherein the operations further comprising:

providing access to the first device to a communication session based on the session identifier included in the marker; and

providing access to the second device to the communication session based on the session identifier that is extracted from the image of the marker.

15. The system of claim 11 , wherein the marker is a Quick Response (QR) code, a natural feature marker, a user interface of an application displayed on a display of the first device, or the first device.

16. The system of claim 11 ,

wherein each first device transformation in the stream is based on a first device pose and a first device origin, wherein the first device origin is an origin of a coordinate frame tracked by the first device.

17. The system of claim 16 , wherein determining the transformation between the first device and the second device using the image of the marker further comprises:

determining a second offset based on a first device transformation, wherein the second offset is an offset of a second device current pose and the first device origin.

18. The system of claim 17 , wherein determining the transformation between the first device and the second device using the image of the marker further comprises:

determining the transformation using a position of the marker in the image, a scale of the marker in the image, or a rotation of the marker in the image.

19. A non-transitory computer-readable storage medium having stored thereon, instructions when executed by a processor, causes the processor to perform operations comprising:

detecting a marker using a camera of a second device;

capturing a data stream of images of the marker using the camera;

determining a transformation between a first device and the second device using an image of the marker from the data stream of images of the marker, wherein determining the transformation comprises:

receiving from the first device, a stream of first device transformations,

determining an alignment of the stream of first device transformations with the data stream of images of the marker, and

selecting, based on the alignment, the image of the marker from the data stream of images of the marker and a corresponding first device transformation from the stream of first device transformations;

determining a common coordinate frame using the transformation;

generating a shared AR session using the common coordinate frame; and

causing the shared AR session to be displayed by the first device and by the second device.

Assignments (1)

Continuity (2)

References Cited (34)