IP Library › Granted Patent US 12,205,232
Granted Patent B2
US 12,205,232 · App. 18/615,683 · Granted Jan 21, 2025

Systems and methods for dynamic interaction with an augmented reality environment

Inventors: Alexander Mascarin (Kleinburg, CA); Stephan Peralta (Etobicoke, CA); Andrea Tuzi (Innisfil, CA); Matthew David Presta (Vaughan, CA); Michael James Presta (Nobleton, CA)
Assignee: EQPME INC.
G06T19/006G06T7/13
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,205,232
App. No.
18/615,683
Granted
Jan 21, 2025
Kind
B2
Abstract

Methods and systems are provided for dynamic interaction with an augmented reality environment. In some embodiments, the systems and methods are directed at dynamically interacting with machinery within the augmented reality environment via an augmented reality device. The method involves analyzing a site in which the machinery is to be operated to capture environment data; displaying on the augmented reality device an augmented reality representation of the machinery as an overlay of a portion of an augmented reality environment; receiving an interaction request for interacting with the augmented reality representation of the machinery; determining whether the interaction request can be completed in respect of the machinery within the site; and in response to determining that the interaction request can be completed, displaying the augmented reality representation of the machinery in accordance with the interaction request, otherwise, indicating that the interaction request cannot be completed within the site.

Claims (52)

1. A method for dynamically interacting with an augmented reality environment via an augmented reality device, the method comprising:

capturing, with the augmented reality device, environment data for estimating a site boundary of a site in which an object is to be engaged;

displaying an augmented reality representation of the object as an overlay of a portion of an augmented reality environment corresponding to the site;

receiving an interaction request via the augmented reality device for interacting with the augmented reality representation of the object;

continuously determining an available space within the augmented reality environment for displaying one or more advertisements based on the interaction request, comprising:

determining an operational area required for completing the interaction request in respect of the object; and

identifying the available space for displaying the one or more advertisements based on the determined operational area; and

displaying the augmented reality representation of the object in accordance with the interaction request and dynamically adapting the one or more advertisements based at least on the available space, comprising:

converting the one or more advertisements between an embedded advertisement format selectable to display the one or more advertisements and a content-based advertisement format in which a content of at least one advertisement of the one or more advertisements is shown on a display of the augmented reality device based on the available space.

2. The method of claim 1 , wherein continuously determining the available space within the augmented reality environment comprises determining whether to adapt the one or more advertisements in response to a subsequent interaction request.

3. The method of claim 2 , wherein the subsequent interaction request results in a reduced available space for the one or more advertisements and in response to detecting the reduced available space, automatically adapting the one or more advertisements based on a current available space resulting from the subsequent interaction request.

4. The method of claim 3 , wherein automatically adapting the one or more advertisements based on the current available space resulting from the subsequent interaction request comprises overlaying at least a portion of the one or more advertisements on the object.

5. The method of claim 3 , wherein automatically adapting the one or more advertisements based on the current available space resulting from the subsequent interaction request comprises replacing at least one advertisement of the one or more advertisements with an alternative advertisement.

6. The method of claim 3 , wherein automatically adapting the one or more advertisements based on the current available space resulting from the subsequent interaction request comprises replacing the one or more advertisements with an embedded advertisement selectable to display the one or more advertisements.

7. The method of claim 1 , wherein at least one advertisement of the one or more advertisements comprises a limited time offer.

8. The method of claim 1 , wherein at least one advertisement of the one or more advertisements comprises an advertisement in respect of the object.

9. The method of claim 1 , wherein at least one advertisement of the one or more advertisements comprises an advertisement for a component compatible with the object.

10. The method of claim 1 , wherein at least one advertisement of the one or more advertisements is engageable for conducting a transaction in respect of a content of the at least one advertisement.

11. A method for dynamically interacting with an augmented reality environment via an augmented reality device, the method comprising:

capturing, with the augmented reality device, environment data for estimating a site boundary of a site in which an object is to be engaged;

displaying an augmented reality representation of the object as an overlay of a portion of an augmented reality environment corresponding to the site;

receiving an interaction request via the augmented reality device for interacting with the augmented reality representation of the object;

continuously determining an available space within the augmented reality environment for displaying one or more advertisements based on the interaction request; and

displaying the augmented reality representation of the object in accordance with the interaction request and dynamically adapting the one or more advertisements based at least on the available space, comprising:

continuously monitoring the available space during the interaction request in respect of the object, and modifying the one or more advertisements on a display of the augmented reality device, wherein modifying the one or more advertisements on the display comprises dynamically varying a display of the one or more advertisements between one or more of an area displaying the augmented reality environment and an interface area.

12. The method of claim 11 , wherein modifying the one or more advertisements on the display comprises: in response to detecting insufficient space for displaying the one or more advertisements within an area displaying the augmented reality environment, displaying the one or more advertisements on an interface area of the display.

13. The method of claim 11 , wherein continuously determining the available space within the augmented reality environment comprises determining whether to adapt the one or more advertisements in response to a subsequent interaction request.

14. The method of claim 13 , wherein the subsequent interaction request results in a reduced available space for the one or more advertisements and in response to detecting the reduced available space, automatically adapting the one or more advertisements based on a current available space resulting from the subsequent interaction request.

15. The method of claim 14 , wherein automatically adapting the one or more advertisements based on the current available space resulting from the subsequent interaction request comprises overlaying at least a portion of the one or more advertisements on the object.

16. The method of claim 14 , wherein automatically adapting the one or more advertisements based on the current available space resulting from the subsequent interaction request comprises replacing at least one advertisement of the one or more advertisements with an alternative advertisement.

17. The method of claim 14 , wherein automatically adapting the one or more advertisements based on the current available space resulting from the subsequent interaction request comprises replacing the one or more advertisements with an embedded advertisement selectable to display the one or more advertisements.

18. A system for dynamically interacting with an augmented reality environment via an augmented reality device, the system comprising a processor operable to:

capture environment data for estimating a site boundary of a site in which an object is to be engaged;

display an augmented reality representation of the object as an overlay of a portion of an augmented reality environment corresponding to the site;

receive an interaction request via the augmented reality device for interacting with the augmented reality representation of the object;

continuously determine an available space within the augmented reality environment for displaying one or more advertisements based on the interaction request, comprising:

determining an operational area required for completing the interaction request in respect of the object; and

identifying the available space for displaying the one or more advertisements based on the determined operational area; and

display the augmented reality representation of the object in accordance with the interaction request and dynamically adapt the one or more advertisements based at least on the available space, comprising:

converting the one or more advertisements between an embedded advertisement format selectable to display the one or more advertisements and a content-based advertisement format in which a content of at least one advertisement of the one or more advertisements is shown on a display of the augmented reality device based on the available space.

19. The system of claim 18 , wherein the processor is operable to determine whether to adapt the one or more advertisements in response to a subsequent interaction request.

20. The system of claim 19 , wherein the subsequent interaction request results in a reduced available space for the one or more advertisements and in response to detecting the reduced available space, the processor is operable to automatically adapt the one or more advertisements based on a current available space resulting from the subsequent interaction request.

21. The system of claim 20 , wherein the processor is operable to overlay at least a portion of the one or more advertisements on the object.

22. The system of claim 20 , wherein the processor is operable to replace at least one advertisement of the one or more advertisements with an alternative advertisement.

23. The system of claim 20 , wherein the processor is operable to replace the one or more advertisements with an embedded advertisement selectable to display the one or more advertisements.

24. The system of claim 18 , wherein the processor is operable to continuously monitor the available space during the interaction request in respect of the object and modify the one or more advertisements on a display of the augmented reality device.

25. The system of claim 24 , wherein the processor is operable to dynamically vary a display of the one or more advertisements between one or more of an area displaying the augmented reality environment and an interface area.

26. The system of claim 24 , wherein the processor is operable to display the one or more advertisements on an interface area of the display.

27. The system of claim 18 , wherein at least one advertisement of the one or more advertisements comprises a limited time offer.

28. The system of claim 18 , wherein at least one advertisement of the one or more advertisements comprises an advertisement in respect of the object.

29. The system of claim 18 , wherein at least one advertisement of the one or more advertisements comprises an advertisement for a component compatible with the object.

30. The system of claim 18 , wherein at least one advertisement of the one or more advertisements is engageable for conducting a transaction in respect of a content of the at least one advertisement.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2024
From: MASCARIN, ALEXANDER; PERALTA, STEPHAN; TUZI, ANDREA; PRESTA, MATTHEW DAVID; PRESTA, MICHAEL JAMES
To: EQPME INC.
Reel/Frame 066915/0704 →
Continuity (5)
Continuation 18229386 · Aug 2, 2023
Continuation In Part 18316498 · May 12, 2023
Continuation 17984338 · Nov 10, 2022
Provisional Application 63394716 · Aug 3, 2022
Related Publication 20240233293A1 · Jul 11, 2024
References Cited (37)
US 5646844A · Gudat et al. · 1997 [cited by applicant]
US 9070216B2 · Golparvar-Fard et al. · 2015 [cited by applicant]
US 9222771B2 · Rosengaus et al. · 2015 [cited by applicant]
US 10580207B2 · Pejic et al. · 2020 [cited by applicant]
US 20080071559A1 · Arrasvuori · 2008 [cited by applicant]
US 20090061901A1 · Arrasvuori et al. · 2009 [cited by applicant]
US 20100185529A1 · Chestnut · 2010 [cited by applicant]
US 20130234850A1 · Lee · 2013 [cited by applicant]
US 20140025481A1 · Kang et al. · 2014 [cited by applicant]
US 20160049005A1 · Mullins et al. · 2016 [cited by applicant]
US 20160165170A1 · McRae · 2016 [cited by applicant]
US 20160267516A1 · Walz · 2016 [cited by examiner]
US 20170116644A1 · Huang et al. · 2017 [cited by applicant]
US 20180349946A1 · Nguyen · 2018 [cited by applicant]
US 20200258144A1 · Chaturvedi et al. · 2020 [cited by applicant]
US 20200402219A1 · Gilbert · 2020 [cited by applicant]
US 20210020141A1 · Yuasa · 2021 [cited by applicant]
CA 3105792A1 · 2021 [cited by applicant]
WO 2018060783A1 · 2018 [cited by applicant]
PIX4D, “Turn your mobile device into a professional 3D scanner”, https://www.pix4d.com/product/pix4dcatch, downloaded May 11, 2022. [cited by applicant]
Sitemark, “Software for solar construction”, https://www.sitemark.com/industries/construction/, downloaded May 11, 2022. [cited by applicant]
ThreeKit, “3D Product Configurator & Augmented Reality For Commerce”, https://www.threekit.com/, downloaded Jun. 28, 2022. [cited by applicant]
JLG, “Product Announcement”, May 2022. [cited by applicant]
Strange, “Now You Can Drive Elon Musk's Tesla Cybertruck in AR from the Comfort of Your Home via iOS & Android”, Dec. 9, 2019, <https://mobile-ar.reality.news/news/now-you-can-drive-elon-musks-tesla-cybertruck-ar-from-c… [cited by applicant]
Makarov, “12 Augmented Reality Trends of 2023: New Milestones in Immersive Technology”, Aug. 2, 2022, <https://mobidev.biz/blog/augmented-reality-trends-future-ar-technologies> retrieved on Jun. 19, 2023. [cited by applicant]
“On-device voice commands in AR Glasses—Vuzix”, <https://vivoka.com/project/vuzix/>, Vivoka, Oct. 2021, retrieved on Jun. 7, 2023. [cited by applicant]
Echo3D, “Top 5 Augmented Reality (AR) Remote Assistance Tools”, <https://medium.com/echo3d/top-5-augmented-reality-ar-remote-assistance-tools-76e830cddf99>, Medium, Jul. 22, 2021, retrieved on Jun. 7, 2023. [cited by applicant]
Virsabi, “Tesla model 3”, <https://virsabi.com/tesla/> (Jul. 15, 2019), retrieved on Jun. 7, 2023. [cited by applicant]
Zero Density, “Augmented Reality—Revolutionize Your Storytelling with Augmented Reality Solutions”, <https://www.zerodensity.tv/solutions/augmented-reality/> (Apr. 6, 2023), retrieved on Jun. 14, 2023. [cited by applicant]
Virsabi, “Metaverse in Business”, <https://virsabi.com/> (Apr. 20, 2017), retrieved on Jun. 14, 2023. [cited by applicant]
Trimble <https://sitevision.trimble.com/> (Oct. 24, 2018), retrieved on Jun. 14, 2023. [cited by applicant]
Mantooth, TJ, “IP Issues for Retail Businesses Advertising in Augmented Reality”, Jul. 26, 2022, <https://ipwatchdog.com/2022/07/26/ip-issues-retail-businesses-advertising-augmented-reality/id=150383/> (retrieved on Jul… [cited by applicant]
Youtube. “Tesla Cybertruck doesn't fit in the average garage!”, Dec. 6, 2019, https://youtu.be/PuZz6OFMW5A (retrieved on Jul. 31, 2023). [cited by applicant]
AR Cybertruck. LuvBug Media, Mar. 8, 2020. Google Play Store, <https://play.google.com/store/apps/details?id=com.luvbug.cybertruck&pli=1> (retrieved on on Jul. 31, 2023). [cited by applicant]
International Search Report and Written Opinion mailed Oct. 25, 2023 in PCT/CA2023/051032. [cited by applicant]
Mourtzis et al., “Real-Time Remote Maintenance Support Based on Augmented Reality (AR)”, 2020. (Year: 2020). [cited by applicant]
Zhao et al., “Voice Interaction for Augmented Reality Navigation Interfaces with Natural Language Understanding”, IEEE, 2020. (Year: 2020). [cited by applicant]