Rendering 3D captions within real-world environments
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and method for rendering three-dimensional captions (3D) in real-world environments depicted in image content. An editing interface is displayed on a client device. The editing interface includes an input component displayed with a view of a camera feed. A first input comprising one or more text characters is received. In response to receiving the first input, a two-dimensional (2D) representation of the one or more text characters is displayed. In response to detecting a second input, a preview interface is displayed. Within the preview interface, a 3D caption based on the one or more text characters is rendered at a position in a 3D space captured within the camera feed. A message is generated that includes the 3D caption rendered at the position in the 3D space captured within the camera feed.
1. A system comprising:
at least one hardware processor; and
at least one memory storing instructions which, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
causing display, on a display device of a client device, of an interactive interface comprising a display of a user-specified three-dimensional (3D) caption at a position in a first 3D space captured within a camera feed of the client device;
detecting movement of the client device that causes a second 3D space to be captured in the camera feed; and
animating the 3D caption moving from the first 3D space to the second 3D space during the movement of the client device.
2. The system of claim 1 , wherein:
the interactive interface is a first interactive interface; and
the operations further comprise:
causing display, on a display device of a client device, of a second interactive interface comprising a keyboard displayed in conjunction with a view of the camera feed of the client device; and
receiving input comprising one or more text characters entered via the keyboard, the one or more text characters corresponding to the user-specified 3D caption.
3. The system of claim 2 , wherein the operations further comprise: causing display of a two-dimensional (2D) representation of the one or more text characters within the second interactive interface in response to receiving the first input, the 2D representation of the one or more text characters being overlaid on the view of the camera feed.
4. The system of claim 1 , wherein the operations further comprise:
receiving input indicative of an edit to the 3D caption; and
updating the display of the 3D caption based on the edit.
5. The system of claim 4 , wherein the edit to the 3D caption comprises one or more of: an additional text character, a deletion of one or more text characters, a scale change, an orientation change, a placement change, a font change, or a color change.
6. The system of claim 1 , wherein the operations further comprise:
generating a message that includes the 3D caption rendered at the position in the first 3D space captured within the camera feed.
7. The system of claim 1 , wherein the causing the display of the interactive interface further comprises:
detecting a reference surface in the first 3D space captured within the camera feed; and
orienting the 3D caption at the position in the first 3D space based on the detected reference surface.
8. The system of claim 7 , wherein orienting the 3D caption at the position in the 3D space comprises:
assigning the 3D caption to the position in the first 3D space based on the detected reference surface; and
identifying tracking indicia operable to track the 3D caption in the first 3D space.
9. The system of claim 8 , wherein the operations further comprise:
tracking, by a first tracking subsystem from among a set of tracking subsystems, the 3D caption at the position in the first 3D space using the tracking indicia;
detecting an interruption of the tracking indicia; and
in response to detecting the interruption of the tracking indicia, tracking the 3D caption at the position in the first 3D space via a second tracking subsystem from among the set of tracking subsystems.
10. A method comprising:
causing display, on a display device of a client device, of an interactive interface comprising a display of a user-specified three-dimensional (3D) caption at a position in a first 3D space captured within a camera feed of the client device;
detecting, one or more hardware processors, movement of the client device that causes a second 3D space to be captured in the camera feed; and
animating the 3D caption moving from the first 3D space to the second 3D space during the movement of the client device.
11. The method of claim 10 , wherein:
the interactive interface is a first interactive interface; and
the method further comprises:
causing display, on a display device of a client device, of a second interactive interface comprising a keyboard displayed in conjunction with a view of the camera feed of the client device; and
receiving input comprising one or more text characters entered via the keyboard, the one or more text characters corresponding to the user-specified 3D caption.
12. The method of claim 11 , further comprising: causing display of a two-dimensional (2D) representation of the 3D caption within the second interface, the 2D representation of 3D caption being overlaid on the view of the camera feed.
13. The method of claim 10 , further comprising:
receiving input indicative of an edit to the 3D caption; and
updating the display of the 3D caption based on the edit.
14. The method of claim 13 , wherein the edit to the 3D caption comprises one or more of: an additional text character, a deletion of one or more text characters, a scale change, an orientation change, a placement change, a font change, or a color change.
15. The method of claim 10 , further comprise:
generating a message that includes the 3D caption rendered at the position in the first 3D space captured within the camera feed.
16. The method of claim 10 , wherein the causing the display of the interactive interface further comprises:
detecting a reference surface in the first 3D space captured within the camera feed; and
orienting the 3D caption at the position in the first 3D space based on the detected reference surface.
17. The method of claim 16 , wherein orienting the 3D caption at the position in the 3D space comprises:
assigning the 3D caption to the position in the first 3D space based on the detected reference surface; and
identifying tracking indicia operable to track the 3D caption in the first 3D space.
18. The method of claim 17 , further comprising:
tracking, by a first tracking subsystem from among a set of tracking subsystems, the 3D caption at the position in the first 3D space using the tracking indicia;
detecting an interruption of the tracking indicia; and
in response to detecting the interruption of the tracking indicia, tracking the 3D caption at the position in the first 3D space via a second tracking subsystem from among the set of tracking subsystems.
19. A machine-readable medium storing instructions which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
causing display, on a display device of a client device, of an interactive interface comprising a display of a user-specified three-dimensional (3D) caption at a position in a first 3D space captured within a camera feed of the client device;
detecting movement of the client device that causes a second 3D space to be captured in the camera feed; and
animating the 3D caption moving from the first 3D space to the second 3D space during the movement of the client device.
20. The machine-readable medium of claim 19 , wherein:
the interactive interface is a first interactive interface;
the operations further comprise causing display of a second interactive interface that includes a display of a two-dimensional (2D) representation of the 3D caption in conjunction with an element that is operable to edit the 3D caption.