Granted Patent B2

US 11,210,850 · App. 16/696,600 · Granted Dec 28, 2021

Inventors: Kyle Goodrich (Venice, CA); Samuel Edward Hare (Los Angeles, CA); Maxim Maximov Lazarov (Culver City, CA); Tony Mathew (Los Angeles, CA); Andrew James McPhee (Culver City, CA); Daniel Moreno (Los Angeles, CA); Wentao Shang (Los Angeles, CA)

Assignee: Snap Inc.

G06T17/20G06K9/00496G06T3/20G06T3/40G06T7/20G06T7/251G06T11/60G06T13/20G06T15/00G06T15/04G06T19/006G06T19/20G06T2219/2004G06T2219/2012G06T2219/2016

Abstract

Claims (62)

1. A system comprising:

at least one hardware processor;

a memory storing instructions which, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:

causing display, on a display device of a client device, of a three-dimensional (3D) caption editing interface, the 3D caption editing interface including a keyboard displayed in conjunction with a view of a camera feed of the client device;

receiving a first input comprising one or more text characters entered via the keyboard;

in response to receiving the first input, causing display of a two-dimensional (2D) representation of the one or more text characters within the editing interface, the 2D representation of the one or more text characters being overlaid on the view of the camera feed;

in response to detecting a second input, causing display of a preview interface, the causing the display of the preview interface comprising rendering a 3D caption based on the one or more text characters at a position in a first 3D space captured within the camera feed;

generating a message that includes the 3D caption rendered at the position in the first 3D space captured within the camera feed;

detecting movement of the client device that causes a second 3D space to be captured in the camera feed; and

animating the 3D caption moving from the first 3D space to the second 3D space during the movement of the client device.

2. The system of claim 1 , wherein the causing the display of the preview interface further comprises:

detecting a reference surface in the first 3D space captured within the camera feed; and

orienting the 3D caption at the position in the first 3D space based on the detected reference surface.

3. The system of claim 2 , wherein orienting the 3D caption at the position in the 3D space comprises:

assigning the 3D caption to the position in the first 3D space based on the detected reference surface; and

identifying tracking indicia operable to track the 3D caption in the first 3D space.

4. The system of claim 3 , wherein the operations further comprise:

tracking, by a first tracking subsystem from among a set of tracking subsystems, the 3D caption at the position in the first 3D space using the tracking indicia;

detecting an interruption of the tracking indicia; and

in response to detecting the interruption of the tracking indicia, tracking the 3D caption at the position in the first 3D space via a second tracking subsystem from among the set of tracking subsystems.

5. The system of claim 1 ; wherein the operations further comprise:

receiving a third input indicative of an edit to the 3D caption; and

updating the 3D caption based on the edit.

6. The system of claim 5 , wherein the edit to the 3D caption comprises one or more of: an additional text character, a deletion of one or more text characters corresponding to the first input, a scale change, an orientation change, a placement change, a font change, or a color change.

7. The system of claim 1 , wherein the second input comprises a change of orientation of the client device.

8. The system of claim 1 , wherein the operations further comprise:

receiving a third input to activate a 3D caption lens, wherein the causing of the display of the 3D caption editing interface is in response to the third input.

9. A method comprising:

causing display, on a display device of a client device, of a three-dimensional (3D) caption editing interface, the 3D caption editing interface including a keyboard displayed in conjunction with a view of a camera feed of the client device;

receiving a first input comprising one or more text characters entered via the keyboard;

in response to receiving the first input, causing display of a two-dimensional (2D) representation of the one or more text characters within the editing interface, the 2D representation of the one or more text characters being overlaid on the view of the camera feed;

in response to detecting a second input, causing display of a preview interface, the causing the display of the preview interface comprising rendering a 3D caption based on the one or more text characters at a position in a first 3D space captured within the camera feed; and

generating, using one or more processors of a machine, a message that includes the 3D caption rendered at the position in the first 3D space captured within the camera feed detecting movement of the client device that causes a second 3D space to be captured in the camera feed; and animating the 3D caption moving from the first 3D space to the second 3D space during the movement of the client device.

10. The method of claim of claim 9 , wherein the causing the display of the preview interface further comprises:

detecting a reference surface in the first 3D space captured within the camera feed; and

orienting the 3D caption at the position in the first 3D space based on the detected reference surface.

11. The method of claim 10 , wherein orienting the 3D caption at the position in the 3D space comprises:

assigning the 3D caption to the position in first 3D space based on the detected reference surface; and

identifying tracking indicia operable to track the 3D caption in the first 3D space.

12. The method of claim 11 , wherein the operations further comprise:

tracking, by a first tracking subsystem from among a set of tracking subsystems, the 3D caption at the position in the first 3D space using the tracking indicia;

detecting an interruption of the tracking indicia; and

in response to detecting the interruption of the tracking indicia, tracking the 3D caption at the position in the first 3D space via a second tracking subsystem from among the set of tracking subsystems.

13. The method of claim 9 , further comprising:

receiving a third input indicative of an edit to the 3D caption; and

updating the 3D caption based on the edit.

14. The method of claim 13 , wherein the edit to the 3D caption comprises one or more of:

an additional text character, a deletion of one or more text characters corresponding to the first input, a scale change, an orientation change, a placement change, a font change, or a color change.

15. The method of claim 9 , wherein the second input comprises a change of orientation of the client device.

16. The method of claim 9 , further comprising:

receiving a third input to activate a 3D caption lens, wherein the causing of the display of the 3D caption editing interface is in response to the third input.

17. A machine-readable medium storing instructions which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

causing display, on a display device of a client device, of a three-dimensional (3D) caption editing interface, the 3D caption editing interface including a keyboard displayed in conjunction with a view of a camera feed of the client device;

receiving a first input comprising one or more text characters entered via the keyboard;

in response to receiving the first input, causing display of a two-dimensional (2D) representation of the one or more text characters within the editing interface, the 2D representation of the one or more text characters being overlaid on the view of the camera feed;

in response to detecting a second input, causing display of a preview interface, the causing the display of the preview interface comprising rendering a 3D caption based on the one or more text characters at a position in a first 3D space captured within the camera feed; and

generating a message that includes the 3D caption rendered at the position in the first 3D space captured within the camera feed detecting movement of the client device that causes a second 3D space to be captured in the camera feed; and animating the 3D caption moving from the first 3D space to the second 3D space during the movement of the client device.

18. The machine-readable medium of claim 17 , wherein the causing the display of the preview interface further comprises:

detecting a reference surface in the first 3D space captured within the camera feed;

assigning the 3D caption to the position in the first 3D space based on the detected reference surface;

identifying tracking indicia operable to track the 3D caption in the first 3D space; and

tracking, by a first tracking subsystem from among a set of tracking subsystems, the first 3D caption at the position in the 3D space using the tracking indicia.

Assignments (1)

Continuity (3)

Cited By (12)

US 1,109,163 US 12,211,159 US 12,217,374 US 12,315,495 US 12,347,045 US 12,361,652 US 12,361,934 US 12,387,436 US 12,443,325 US 12,444,138 US 12,488,548 US 12,541,929