IP Library › Granted Patent US 12,482,201
Granted Patent B2
US 12,482,201 · App. 18/617,101 · Granted Nov 25, 2025

Augmented reality content experience sharing using digital multimedia files

Inventor: Yen-Lin Chen (San Jose, CA)
Assignee: GOOGLE LLC
G06T19/006G06F16/483G06F16/487G06T17/20G06V20/20G06V40/171
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,482,201
App. No.
18/617,101
Granted
Nov 25, 2025
Kind
B2
Abstract

A computer-implemented method includes capturing visual data of an environment using an image sensor of an electronic device and non-visual data of the environment using one or more non-image sensors of the electronic device. Feature descriptors of one or more objects in the environment are generated using the visual data of the environment and the non-visual data of the environment. A map of the environment is generated using the feature descriptors of the one or more objects. One or more virtual objects are anchored to at least one of the objects using the map. The visual data, the non-visual data, and the map are combined in a digital multimedia container file. The digital multimedia container file is stored on the electronic device or on another electronic device connected to the electronic device.

Claims (79)

1 . A computer-implemented method, comprising:

recording an augmented reality session of a scene using an electronic device, the augmented reality session including one or more virtual objects displayed in the scene;

storing the augmented reality session in a file in on the electronic device;

retrieving the augmented reality session from the file on the electronic device;

playing the augmented reality session on a display of the electronic device in a new scene including displaying the one or more virtual objects in the new scene; and

editing the augmented reality session of the new scene including adding one or more new virtual objects in the new scene.

2 . The computer-implemented method of claim 1 , wherein:

recording the augmented reality session includes generating a map having anchor information for the one or more virtual objects; and

storing the augmented reality session includes storing the map having the anchor information in the file on the electronic device.

3 . The computer-implemented method of claim 2 , wherein:

the electronic device includes an image sensor;

recording the augmented reality session includes recording the augmented reality session using the image sensor to capture the scene and create the map of the anchor information for the one or more virtual objects; and

playing the augmented reality session includes playing the augmented reality session at a same time the image sensor views the new scene including using the map of the anchor information to display the one or more virtual objects in the new scene.

4 . The computer-implemented method of claim 1 , wherein the electronic device is a first electronic device and further comprising:

transferring the file to a second electronic device;

retrieving the augmented reality session from the file on the second electronic device; and

playing the augmented reality session on a display of the second electronic device in the new scene including displaying the one or more virtual objects in the new scene.

5 . The computer-implemented method of claim 1 , wherein:

recording the augmented reality session of the scene includes recording the augmented reality session of the scene having an object, the augmented reality session including the one or more virtual objects associated with the object inserted in the scene; and

playing the augmented reality session includes playing the augmented reality session on the display of the electronic device in the new scene having a new object similar to the object including the one or more virtual objects associated with the new object in the new scene.

6 . The computer-implemented method of claim 5 , further comprising:

editing the augmented reality session of the new scene including adding one or more new virtual objects associated with the new object in the new scene.

7 . The computer-implemented method of claim 1 , wherein storing the augmented reality session in the file includes storing the augmented reality session in a QuickTime format.

8 . A computer program product, the computer program product tangibly embodied on a non-transitory computer-readable medium and including executable code that, when executed by an electronic device, is configured to cause the electronic device to:

record an augmented reality session of a scene using an electronic device, the augmented reality session including one or more virtual objects displayed in the scene;

store the augmented reality session in a file in on the electronic device;

retrieve the augmented reality session from the file on the electronic device;

play the augmented reality session on a display of the electronic device in a new scene including displaying the one or more virtual objects in the new scene; and

edit the augmented reality session of the new scene including adding one or more new virtual objects in the new scene.

9 . The computer program product of claim 8 , wherein:

recording the augmented reality session includes generating a map having anchor information for the one or more virtual objects; and

storing the augmented reality session includes storing the map having the anchor information in the file on the electronic device.

10 . The computer program product of claim 9 , wherein:

the electronic device includes an image sensor;

recording the augmented reality session includes recording the augmented reality session using the image sensor to capture the scene and create the map of the anchor information for the one or more virtual objects; and

playing the augmented reality session includes playing the augmented reality session at a same time the image sensor views the new scene including using the map of the anchor information to display the one or more virtual objects in the new scene.

11 . The computer program product of claim 8 , wherein the electronic device is a first electronic device and further comprising executable code that, when executed by the electronic device, is configured to cause the electronic device to:

transfer the file to a second electronic device;

retrieve the augmented reality session from the file on the second electronic device; and

play the augmented reality session on a display of the second electronic device in the new scene including displaying the one or more virtual objects in the new scene.

12 . The computer program product of claim 8 , wherein:

recording the augmented reality session of the scene includes recording the augmented reality session of the scene having an object, the augmented reality session including the one or more virtual objects associated with the object inserted in the scene; and

playing the augmented reality session includes playing the augmented reality session on the display of the electronic device in the new scene having a new object similar to the object including the one or more virtual objects associated with the new object in the new scene.

13 . The computer program product of claim 12 , further comprising executable code that, when executed by the electronic device, is configured to cause the electronic device to:

edit the augmented reality session of the new scene including adding one or more new virtual objects associated with the new object in the new scene.

14 . The computer program product of claim 8 , wherein storing the augmented reality session in the file includes storing the augmented reality session in a QuickTime format.

15 . An electronic device, comprising:

a memory having instructions; and

a processor configured by the instructions to:

record an augmented reality session of a scene using the electronic device, the augmented reality session including one or more virtual objects displayed in the scene;

store the augmented reality session in a file in on the electronic device;

retrieve the augmented reality session from the file on the electronic device;

play the augmented reality session on a display of the electronic device in a new scene including displaying the one or more virtual objects in the new scene; and

edit the augmented reality session of the new scene including adding one or more new virtual objects in the new scene.

16 . The electronic device of claim 15 , wherein:

recording the augmented reality session includes generating a map having anchor information for the one or more virtual objects; and

storing the augmented reality session includes storing the map having the anchor information in the file on the electronic device.

17 . The electronic device of claim 16 , wherein:

the electronic device includes an image sensor;

recording the augmented reality session includes recording the augmented reality session using the image sensor to capture the scene and create the map of the anchor information for the one or more virtual objects; and

playing the augmented reality session includes playing the augmented reality session at a same time the image sensor views the new scene including using the map of the anchor information to display the one or more virtual objects in the new scene.

18 . The electronic device of claim 15 , wherein the electronic device is a first electronic device and comprising further instructions that cause the processor to:

transfer the file to a second electronic device;

retrieve the augmented reality session from the file on the second electronic device; and

play the augmented reality session on a display of the second electronic device in the new scene including displaying the one or more virtual objects in the new scene.

19 . The electronic device of claim 15 , wherein:

recording the augmented reality session of the scene includes recording the augmented reality session of the scene having an object, the augmented reality session including the one or more virtual objects associated with the object inserted in the scene; and

playing the augmented reality session includes playing the augmented reality session on the display of the electronic device in the new scene having a new object similar to the object including the one or more virtual objects associated with the new object in the new scene.

20 . The electronic device of claim 19 , comprising further instructions that cause the processor to:

edit the augmented reality session of the new scene including adding one or more new virtual objects associated with the new object in the new scene.

21 . The electronic device of claim 15 , wherein storing the augmented reality session in the file includes storing the augmented reality session in a QuickTime format.

22 . The electronic device of claim 15 , wherein the electronic device is an augmented reality wearable device.

23 . A computer program product, the computer program product tangibly embodied on a non-transitory computer-readable medium and including executable code that, when executed by an electronic device, is configured to cause the electronic device to:

record an augmented reality session of a scene having an object using an electronic device, the augmented reality session including one or more virtual objects inserted in the scene and associated with the object;

store the augmented reality session in a file in on the electronic device;

retrieve the augmented reality session from the file on the electronic device; and

play the augmented reality session on a display of the electronic device in a new scene having a new object similar to the object including displaying the one or more virtual objects associated with the new object in the new scene.

24 . The computer program product of claim 23 , further comprising executable code that, when executed by the electronic device, is configured to cause the electronic device to:

edit the augmented reality session of the new scene including adding one or more new virtual objects associated with the new object in the new scene.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 9, 2024
From: CHEN, YEN-LIN
To: GOOGLE LLC
Reel/Frame 067356/0191 →
Continuity (3)
Continuation 18066094 · Dec 14, 2022
Continuation 17301596 · Apr 8, 2021
Related Publication 20250054245A1 · Feb 13, 2025
References Cited (30)
US 10802147B2 · Nerurkar et al. · 2020 [cited by applicant]
US 11222478B1 · Martin et al. · 2022 [cited by applicant]
US 11557100B2 · Chen · 2023 [cited by applicant]
US 20140178029A1 · Raheman et al. · 2014 [cited by applicant]
US 20160358383A1 · Gauglitz et al. · 2016 [cited by applicant]
US 20170336511A1 · Nerurkar et al. · 2017 [cited by applicant]
US 20180253900A1 · Finding et al. · 2018 [cited by applicant]
US 20190155465A1 · Maxwell · 2019 [cited by applicant]
US 20190340836A1 · Lynen et al. · 2019 [cited by applicant]
US 20190354761A1 · Arshad et al. · 2019 [cited by applicant]
US 20200074738A1 · Hare et al. · 2020 [cited by applicant]
US 20200106965A1 · Malia et al. · 2020 [cited by applicant]
US 20200342669A1 · Nattinger et al. · 2020 [cited by applicant]
US 20200342670A1 · Nattinger et al. · 2020 [cited by applicant]
WO 2022056079A1 · 2022 [cited by applicant]
U.S. Appl. No. 17/301,596, filed Apr. 8, 2021, Issued. [cited by applicant]
U.S. Appl. No. 18/066,094, filed Dec. 14, 2022, Allowed. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/071595, mailed Jul. 15, 2022, 15 pages. [cited by applicant]
“ARCore and Arkit, What is under the hood: SLAM (Part 2)”, RK's Rambling, retrieved on Apr. 8, 2021 from https://blog.rabimba.com/2018/10/arcore-and-arkit-SLAM.html, Oct. 30, 2018, 11 pages. [cited by applicant]
“ARCore and Arkit: What is under the hood: Anchors and World Mapping (Part 1)”, RK's Rambling, retrieved on Apr. 8, 2021 from https://blog.rabimba.com/2018/10/arcore-and-arkit-what-is-under-hood.html, Oct. 29, 2018, 11 … [cited by applicant]
“ARCore Overview”, Google Developers, retrieved on Apr. 8, 2021 from https://developers.google.com/ar/discover, Nov. 6, 2020, 2 pages. [cited by applicant]
“Cloud Anchors overview for Android”, Google Developers, retrieved on Apr. 8, 2021 from https://developers.google.com/ar/develop/java/cloud-anchors/overview-android#:˜: text=users%20can%20share.-,What%20is%20a%20Cloud%2… [cited by applicant]
“Creating a Multiuser AR Experience”, Apple Developer Documentation, retrieved on Apr. 8, 2021 from https://developer.apple.com/documentation/arkit/creating_a_multiuser_ar_experience, 6 pages. [cited by applicant]
“Fundamental Concepts”, Google Developers, ARCore, retrieved on Apr. 8, 2021 from https://developers.google.com/ar/discover/concepts, Jan. 8, 2021, 4 pages. [cited by applicant]
BATT, “Understanding Anchors in Augmented Reality Experiences”, Learning Solutions, https://learningsolutionsmag.com/articles/understanding-anchors-in-augmented-reality-experiences, Jan. 27, 2020, 12 pages. [cited by applicant]
Chen, et al., “Remote Creation and Editing of Augmented Reality Content Using Cloud Anchors”, Technical Disclosure Commons, https://www.tdcommons.org/dpubs_series/3882, Dec. 2020, 6 pages. [cited by applicant]
Fathi, “Creating Composite AR and Video Experiences with ARVideoKit and Agora”, retrieved on Jan. 14, 2021 from https://www.agora.io/en/blog/creating-composite-ar-and-video-experiences-witharvideokit-and-agora/, Sep. 17… [cited by applicant]
Google, “Share AR Experiences with Cloud Anchors”, retrieved on Aug. 31, 2020 from https://developers.google.com/ar/develop/java/cloud-anchors/overview-android, Feb. 28, 2019, 3 pages. [cited by applicant]
Jakl, “Basics of AR: SLAM—Simultaneous Localization and Mapping”, retrieved on Apr. 8, 2021 from https://www. andreasjakl.com/basics-of-ar-slam-simultaneous-localization-and-mapping/, Aug. 14, 2018, 11 pages. [cited by applicant]
Lee, “Design of an Augmented Reality-Enabled Multimedia File Format With Embedded Object Tracking Metadata”, International Journal of Multimedia and Ubiquitous Engineering, vol. 8, No. 4, Jul. 2013, 10 pages. [cited by applicant]