IP Library Granted Patent US 12,299,825
Granted Patent B2
US 12,299,825 · App. 17/888,601 · Granted May 13, 2025

Handcrafted augmented reality effort evidence

Inventors: Tianying Chen (Pittsburgh, PA); Timothy Chong (Kirkland, WA); Sven Kratz (Mercer Island, WA); Fannie Liu (New York, NY); Andrés Monroy-Hernández (Seattle, WA); Olivia Seow (Cambridge, MA); Yu Jiang Tham (Seattle, WA); Rajan Vaish (Beverly Hills, CA); Lei Zhang (Ann Arbor, MI)
Assignee: Snap Inc.
G06T19/006G06T19/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,825
App. No.
17/888,601
Granted
May 13, 2025
Kind
B2
Abstract

Augmented reality (AR) systems, devices, media, and methods are described for capturing and presenting effort put into generating a handcrafted AR experience. AR object generation data is captured during the generation of a handcrafted AR object. The AR object generation data is then processed to generate proof of effort data for inclusion with the handcrafted AR object. Examples of proof of effort include a time lapse view of the steps taken during generation of the AR object and statistics such as total time spent, number of images or songs considered for selection, number of actions implemented, etc.

Claims (97)

1. An augmented reality (AR) device for generating a handcrafted AR object including proof of effort, the AR device comprising:

a camera system configured to capture images of a scene within a field of view;

a display system configured to present AR overlays on a display, the display having a viewing area corresponding to the camera system field of view;

a user input system configured to receive input from a user; and

a processor coupled to the camera system, the display system, and the user input system, the processor configured to:

initiate handcrafted AR object generation;

begin capture of AR object generation data;

generate handcrafted AR overlays including customizations associated with a customizable AR primary object responsive to customization commands received via the user input system after the begin capture of the AR object generation data;

present, via the display system, the handcrafted AR overlays;

selectively save the handcrafted AR overlays including the customizations associated with the customizable AR primary object as the handcrafted AR object;

terminate the handcrafted AR object generation;

end capture of the AR object generation data;

process the captured AR object generation data to generate proof of effort data, wherein the proof of effort data includes a total time from the begin capture of the AR object generation data to the end capture of the AR object generation data;

assemble a handcrafted AR file including the handcrafted AR object and the generated proof of effort data; and

transmit the handcrafted AR file.

2. The AR device of claim 1 , wherein to assemble the handcrafter AR file the processor is further configured to:

incorporate the proof of effort data into the handcrafted AR object.

3. The AR device of claim 1 , wherein the generated proof of effort data is at least one of (i) representative images presented on the display or (ii) customization commands received from the beginning to the end of the capture of the AR object generation data.

4. The AR device of claim 3 , wherein the processor is further configured to:

identify representative images including potentially sensitive content; and

blur the identified representative images.

5. The AR device of claim 1 , wherein the generated proof of effort data is a link to at least one of (i) representative images presented on the display or (ii) customization commands received from the beginning to the end of the capture of the AR object generation data.

6. The AR device of claim 1 , wherein the customizable AR primary object includes an image receiving area and wherein to receive the customization commands the processor is configured to:

present a number of images on the display;

receive an image selection for the image receiving area from the number of images; and

obtain the image responsive to the image selection;

wherein to generate the handcrafted AR overlays the processor is configured to apply the obtained image to the image receiving area; and

wherein the proof of effort data includes the number of images presented prior to receiving an image selection.

7. The AR device of claim 1 , wherein the customization commands include customization actions and wherein the processor is further configured to:

record the customization actions; and

generate statistics regarding the recorded customization actions;

wherein the proof of effort data includes the generated statistics.

8. The AR device of claim 1 , wherein the processor is further configured to:

receive an audio personalization selection;

present audio options;

receive an audio selection based on the audio options; and

generate a soundtrack responsive to the audio selections;

wherein to generate the handcrafted AR file the processor further includes the soundtrack; and

wherein the proof of effort data includes the number of audio options presented prior to receiving the audio selection.

9. The handcrafted AR system including the AR device of claim 1 , the handcrafted AR system further comprising:

another camera system configured to capture images of another scene within a field of view of the other camera system;

another display system configured to present AR overlays on a display of the other display system, the display of the other display system having a viewing area corresponding to the other camera system field of view;

another processor coupled to the other camera system and the other display system, the processor configured to:

receive the handcrafted AR file including the handcrafted AR object and the generated proof of effort data;

generate handcrafted AR overlays from the handcrafted AR file including the customizations associated with the customizable AR primary object; and

present, via the other display system, the handcrafted AR overlays and the proof of effort data.

10. The handcrafted AR system of claim 9 , wherein the handcrafted AR overlays include a selectable feature and wherein the proof of effort data is presented in response to selection of the selectable feature.

11. A method for generating a handcrafted augmented reality (AR) experience, the method comprising:

initiating handcrafted AR object generation;

begin capturing of AR object generation data;

generating handcrafted AR overlays including customizations associated with a customizable AR primary object responsive to customization commands received via a user input system after the begin capturing of the AR object generation data;

presenting, via a display system, the handcrafted AR overlays;

selectively saving the handcrafted AR overlays including the customizations associated with the customizable AR primary object as the handcrafted AR object;

terminating the handcrafted AR object generation;

end capturing of the AR object generation data;

processing the captured AR object generation data to generate proof of effort data, wherein the proof of effort data includes a total time from the begin capturing of the AR object generation data to the end capturing of the AR object generation data;

assembling a handcrafted AR file including the handcrafted AR object and the generated proof of effort data; and

transmitting the handcrafted AR file.

12. The method of claim 11 , wherein assembling the handcrafted AR file comprises:

incorporating the proof of effort data into the handcrafted AR object.

13. The method of claim 11 , wherein the generated proof of effort data is at least one of (i) representative images presented on the display, (ii) customization commands, or (iii) a link to the representative images or customization commands received from the beginning to the end of the capture of the AR object generation data.

14. The method of claim 13 , further comprising:

identifying representative images including potentially sensitive content; and

blurring the identified representative images.

15. The method of claim 11 , wherein the customizable AR primary object includes an image receiving area and wherein receiving the customization commands comprises:

presenting a number of images on a display of the display system;

receiving an image selection for the image receiving area from the number of images; and

obtaining the image responsive to the image selection;

wherein generating the handcrafted AR overlays comprises applying the obtained image to the image receiving area; and

wherein the proof of effort data includes the number of images presented prior to receiving an image selection.

16. The method of claim 11 , wherein the customization commands include customization actions and wherein the method further comprises:

recording the customization actions; and

generating statistics regarding the recorded customization actions;

wherein the proof of effort data includes the generated statistics.

17. The method of claim 11 , further comprising:

receiving an audio personalization selection;

presenting audio options;

receiving an audio selection based on the audio options; and

generating a soundtrack responsive to the audio selections;

wherein the handcrafted AR file includes the soundtrack; and

wherein the proof of effort data includes the number of audio options presented prior to receiving the audio selection.

18. The method of claim 11 , further comprising:

receiving another handcrafted AR file including another handcrafted AR object and corresponding generated proof of effort data;

generating handcrafted AR overlays from the other handcrafted AR file including corresponding customizations associated with another customizable AR primary object; and

presenting the handcrafted AR overlays and the corresponding proof of effort data.

19. The method of claim 18 , wherein the handcrafted AR overlays include a selectable feature and wherein the corresponding proof of effort data is presented in response to selection of the selectable feature.

20. A non-transitory computer-readable medium storing program code including instructions that, when executed, are operative to cause an electronic processor to perform the steps of:

initiating handcrafted AR object generation;

begin capturing of AR object generation data;

generating handcrafted AR overlays including customizations associated with a customizable AR primary object responsive to customization commands received via a user input system after the begin capturing of the AR object generation data;

presenting, via a display system, the handcrafted AR overlays;

selectively saving the handcrafted AR overlays including the customizations associated with the customizable AR primary object as the handcrafted AR object;

terminating the handcrafted AR object generation;

end capturing of the AR object generation data;

processing the captured AR object generation data to generate proof of effort data, wherein the proof of effort data includes a total time from the begin capturing of the AR object generation data to the end capturing of the AR object generation data;

assembling a handcrafted AR file including the handcrafted AR object and the generated proof of effort data; and

transmitting the handcrafted AR file.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2024
From: CHEN, TIANYING; CHONG, TIMOTHY; KRATZ, SVEN; LIU, FANNIE; MONROY-HERNÁNDEZ, ANDRÉS; SEOW, OLIVIA; THAM, YU JIANG; VAISH, RAJAN; ZHANG, LEI
To: SNAP INC.
Reel/Frame 069400/0335 →
Continuity (2)
Provisional Application 63239902 · Sep 1, 2021
Related Publication 20230066318A1 · Mar 2, 2023
References Cited (44)
US 10139899B1 · Niemeyer et al. · 2018 [cited by applicant]
US 10306315B2 · Bailey · 2019 [cited by examiner]
US 11003308B1 · Dryer · 2021 [cited by examiner]
US 20110310227A1 · Konertz et al. · 2011 [cited by applicant]
US 20120023534A1 · Dasilva et al. · 2012 [cited by applicant]
US 20120249443A1 · Anderson et al. · 2012 [cited by applicant]
US 20120249591A1 · Maciocci et al. · 2012 [cited by applicant]
US 20120293407A1 · Lee · 2012 [cited by applicant]
US 20130044128A1 · Liu et al. · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130293584A1 · Anderson · 2013 [cited by applicant]
US 20180129276A1 · Nguyen et al. · 2018 [cited by applicant]
US 20180157398A1 · Kaehler et al. · 2018 [cited by applicant]
US 20180322706A1 · Drouin et al. · 2018 [cited by applicant]
US 20190347854A1 · Karlov et al. · 2019 [cited by applicant]
US 20200117268A1 · Kritzler et al. · 2020 [cited by applicant]
US 20200160607A1 · Kjallstrom et al. · 2020 [cited by applicant]
US 20200342683A1 · Panec et al. · 2020 [cited by applicant]
US 20200357184A1 · Paul et al. · 2020 [cited by applicant]
US 20200363929A1 · Srinivasan et al. · 2020 [cited by applicant]
US 20200387289A1 · Dunn et al. · 2020 [cited by applicant]
US 20210132781A1 · Daniels et al. · 2021 [cited by applicant]
US 20210178278A1 · Qiu et al. · 2021 [cited by applicant]
US 20210082084A1 · Baldus · 2021 [cited by applicant]
US 20210405761A1 · Canberk · 2021 [cited by applicant]
US 20220005254A1 · Gladkov et al. · 2022 [cited by applicant]
US 20220157026A1 · Bennett et al. · 2022 [cited by applicant]
US 20220301245A1 · Shuvi · 2022 [cited by applicant]
US 20220335697A1 · Harding et al. · 2022 [cited by applicant]
US 20220351475A1 · Ben-Dor et al. · 2022 [cited by applicant]
CN 103139394A · 2013 [cited by applicant]
CN 103974190A · 2014 [cited by applicant]
CN 109728918A · 2019 [cited by applicant]
CN 111450538A · 2020 [cited by applicant]
EP 3462294A1 · 2019 [cited by applicant]
WO 2018227098A1 · 2018 [cited by applicant]
Burgess Colin J. et al.: “Can genetic programming improve software effort estimation? A comparative evaluation”, Information and Software Technology, vol. 43, No. 14, Dec. 1, 2001 (Dec. 1, 2001), pp. 863-873. [cited by applicant]
Gullichsen, Frida: “Augmented audio—the importance of sound in AR”, Jun. 12, 2019 (Jun. 12, 2019), XP093003111, Retrieved from the Internet: URL:https://blog.arilyn.com/augmented-audio [retrieved on Nov. 29, 2022], 8 pa… [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/040305, dated Dec. 19, 2022 (Dec. 19, 2022)—14 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/040314, dated Nov. 28, 2022 (Nov. 28, 2022)—12 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/040398, dated Dec. 21, 2022 (Dec. 21, 2022)—12 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/040443, dated Nov. 25, 2022 (Nov. 25, 2022)—11 pages. [cited by applicant]
Langlotz, Tobias et al: “Sketching up the world: In situ authoring for mobile Augmented Reality”, Personal and Ubiquitous Computing, Springer Verlag, London, GB, vol. 16, No. 6, Aug. 1, 2012 (Aug. 1, 2012), pp. 623-630. [cited by applicant]
Wang Ming-Jen et al.: “An Easy to Use Augmented Reality Authoring Tool for Use in Examination Purpose” in: “International Conference on Intelligent Information Processing”, Jan. 1, 2010 (Jan. 1, 2010), Springer New York… [cited by applicant]