IP Library › Granted Patent US 12,266,068
Granted Patent B2
US 12,266,068 · App. 17/821,211 · Granted Apr 1, 2025

System and method for updating electronic content by selectively replacing virtual 3D objects

Inventors: Russ Maschmeyer (Berkeley, CA); Brennan Letkeman (Calgary, CA)
Assignee: Shopify Inc.
G06T19/20G06F3/04883G06Q30/0643G06T19/006G06F3/0482G06T2200/24G06T2210/04G06T2219/2004G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,266,068
App. No.
17/821,211
Granted
Apr 1, 2025
Kind
B2
Abstract

A system and method are provided for updating electronic content by selectively replacing virtual 3D objects being displayed in an electronic environment. The method includes in one example, presenting augmented reality (AR) content, the presented AR content showing a set of objects, at least some of the objects of the set of objects being rendered 3D objects; receiving an input identifying one of the objects of the set of objects; and responsive to the input, providing updated AR content, the updated AR content replacing at least one of the objects with a different rendered 3D object, wherein the replaced at least one of the objects is determined based on the identified one of the objects of the set of objects.

Claims (43)

1. A computer-implemented method comprising:

presenting augmented reality (AR) content in a scene, the presented AR content showing a set of objects in the scene, at least some of the objects of the set of objects being rendered 3D objects;

receiving a first input identifying a first object of the set of objects in the scene; and

responsive to receiving a second input, providing updated AR content in the scene based on the first input and the second input, the updated AR content keeping either the first object or at least one other object of the set of objects while replacing the other of the first object or the at least one other object of the set of objects with at least one different rendered 3D object, wherein the at least one different rendered 3D object is determined based on the identified first object of the set of objects in the scene.

2. The method of claim 1 , wherein the second input comprises a swipe gesture.

3. The method of claim 1 , wherein the first input is one of a lock input and an unlock input.

4. The method of claim 1 , wherein the at least one different rendered 3D object replaces the first object of the set of objects.

5. The method of claim 1 , wherein the at least one different rendered 3D object replaces the at least one other object of the set of objects.

6. The method of claim 1 , wherein the at least one different rendered 3D object comprises at least one new rendered 3D object.

7. The method of claim 1 , wherein a position of the at least one different rendered 3D object is offset from a particular location of the first object or the at least one other of the objects.

8. The method of claim 1 , wherein one or more of the at least one different rendered 3D object or at least one remaining rendered 3D object comprises a rendered indication of a selected real-world object.

9. The method of claim 8 , wherein the rendered indication of the selected real-world object is displayed as an overlay in the updated AR content.

10. The method of claim 1 , comprising:

prior to presenting the AR content showing the set of objects:

receiving an input in a first environment identifying an object from a set of objects rendered in the field of view of the user device;

generating new content in the first environment including: an updated object from the set; and at least one remaining object from the set; and

transitioning to a second environment comprising the AR content.

11. The method of claim 10 , wherein the first environment is a two-dimensional user interface displaying the set of objects.

12. The method of claim 1 , wherein the first input comprises a swipe gesture that identifies the first object of the set of objects.

13. The method of claim 1 , wherein the first input comprises identifying the first object of the set of objects based on a zoomed field of view in the AR content.

14. The method of claim 1 , comprising using the first input to determine a replacement object as the different rendered 3D object, for the new AR content.

15. The method of claim 14 , wherein the first input is used to determine at least one characteristic found in the replacement object.

16. The method of claim 1 , comprising:

repeating the method to identify a different one of the set of objects or identify an additional one of the set of objects; and

further updating the AR content based on the repeating.

17. The method of claim 1 , comprising:

receiving a third input confirming a desired set of objects in the updated AR content; and

executing an action associated with the desired set of objects.

18. The method of claim 17 , wherein the action comprises a retail checkout mechanism.

19. A system comprising:

a processor;

at least one communications module to communicate with an input module of a user device; and

at least one memory, the at least one memory comprising computer executable instructions that, when executed by the at least one processor, causes the system to:

present augmented reality (AR) content in a scene, the presented AR content showing a set of objects in the scene, at least some of the objects of the set of objects being rendered 3D objects;

receive a first input identifying a first object of the set of objects in the scene; and

responsive to receiving a second input, provide updated AR content in the scene based on the first input and the second input, the updated AR content keeping either the first object or at least one other object of the set of objects while replacing the other of the first object or the at least one other object of the set of objects with at least one different rendered 3D object, wherein the at least one different rendered 3D object is determined based on the identified first object of the set of objects in the scene.

20. The system of claim 19 , wherein the second input comprises a swipe gesture.

21. The system of claim 19 , wherein the at least one different rendered 3D object replaces the first object of the set of objects.

22. The system of claim 19 , wherein the at least one different rendered 3D object replaces the at least one other object of the set of objects.

23. A non-transitory computer readable medium comprising computer executable instructions that when executed by a processor, cause the processor to execute instructions for:

presenting augmented reality (AR) content in a scene, the presented AR content showing a set of objects in the scene, at least some of the objects of the set of objects being rendered 3D objects;

receiving a first input identifying a first object of the set of objects in the scene; and

responsive to receiving a second input, providing updated AR content in the scene based on the first input and the second input, the updated AR content keeping either the first object or at least one other object of the set of objects while replacing the other of the first object or the at least one other object of the set of objects with at least one different rendered 3D object, wherein the at least one different rendered 3D object is determined based on the identified first object of the set of objects in the scene.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2024
From: SHOPIFY (USA) INC.
To: SHOPIFY INC.
Reel/Frame 066149/0734 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2022
From: SHOPIFY (USA) INC.
To: SHOPIFY INC.
Reel/Frame 062109/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2022
From: MASCHMEYER, RUSS
To: SHOPIFY (USA) INC.
Reel/Frame 061076/0309 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2022
From: LETKEMAN, BRENNAN
To: SHOPIFY INC.
Reel/Frame 061076/0425 →
Continuity (2)
Provisional Application 63366352 · Jun 14, 2022
Related Publication 20230401802A1 · Dec 14, 2023
References Cited (13)
US 11003308B1 · Dryer · 2021 [cited by examiner]
US 11508166B2 · Kim · 2022 [cited by examiner]
US 20080071559A1 · Arrasvuori · 2008 [cited by examiner]
US 20120210255A1 · Ooi · 2012 [cited by examiner]
US 20190197785A1 · Tate-Gans · 2019 [cited by examiner]
US 20190228581A1 · Dascola · 2019 [cited by examiner]
US 20190370544A1 · Wright, Jr. · 2019 [cited by examiner]
US 20200117335A1 · Mani · 2020 [cited by examiner]
US 20220058882A1 · Paul · 2022 [cited by examiner]
US 20220076323A1 · Angeli · 2022 [cited by examiner]
US 20220092857A1 · Haapoja · 2022 [cited by examiner]
US 20220319058A1 · Skidmore · 2022 [cited by examiner]
US 20230177832A1 · Besecker · 2023 [cited by examiner]