IP Library › Granted Patent US 12,236,545
Granted Patent B2
US 12,236,545 · App. 18/663,876 · Granted Feb 25, 2025

Systems and methods for dynamic interaction with an augmented reality environment

Inventors: Alexander Mascarin (Kleinburg, CA); Stephan Peralta (Etobicoke, CA); Andrea Tuzi (Innisfil, CA); Matthew David Presta (Vaughan, CA); Michael James Presta (Nobleton, CA)
Assignee: EQPME INC.
G06T19/006G06T7/13
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,236,545
App. No.
18/663,876
Granted
Feb 25, 2025
Kind
B2
Abstract

Methods and systems are provided for dynamic interaction with an augmented reality environment. In some embodiments, the systems and methods are directed at dynamically interacting with machinery within the augmented reality environment via an augmented reality device. The method involves analyzing a site in which the machinery is to be operated to capture environment data; displaying on the augmented reality device an augmented reality representation of the machinery as an overlay of a portion of an augmented reality environment; receiving an interaction request for interacting with the augmented reality representation of the machinery; determining whether the interaction request can be completed in respect of the machinery within the site; and in response to determining that the interaction request can be completed, displaying the augmented reality representation of the machinery in accordance with the interaction request, otherwise, indicating that the interaction request cannot be completed within the site.

Claims (51)

1. A method for interacting with an augmented reality environment, the method comprising:

determining a site boundary of a site for an object;

receiving an interaction request at an augmented reality device for interacting with an augmented reality representation of the object displayed relative to an augmented reality environment of the site;

determining whether the interaction request can be completed within the site boundary, comprising:

determining whether an operating range of the object for completing the interaction request is restricted by one or more of the site boundary and an obstacle within the site; and

in response to determining that the interaction request can be completed, proceeding with the interaction request in respect of the augmented reality representation of the object.

2. The method of claim 1 , wherein the interaction request comprises at least one engagement corresponding to a real-world engagement.

3. The method of claim 1 further comprises indicating that the interaction request cannot be completed as an operating range of the object is restricted by one or more of the site boundary and an obstacle within the site.

4. The method of claim 1 further comprises:

indicating that the object is unsuitable for the site when it is determined that the interaction request cannot be completed.

5. The method of claim 4 further comprises recommending an alternative object suitable for the site.

6. The method of claim 1 further comprises requesting an analysis of the site with the augmented reality device when the interaction request cannot be completed within the site.

7. The method of claim 1 further comprises displaying one or more alternative objects suitable for the site when the interaction request cannot be completed within the site.

8. The method of claim 7 , further comprises determining the one or more alternative objects based on one or more of a user preference and a historical user data.

9. The method of claim 1 further comprises receiving a transaction request for the object via the augmented reality device.

10. The method of claim 9 , wherein receiving the transaction request for the object comprises:

at least one or more of selecting and substituting compatible components for the object to be displayed within the site for a transaction involving a real-world version of the object.

11. The method of claim 1 , wherein the interaction request comprises an engagement with one or more components of the object.

12. The method of claim 1 further comprises:

determining an available space within the augmented reality environment for displaying one or more advertisement contents without interfering with the interaction request; and

displaying the one or more advertisement contents within the available space.

13. The method of claim 12 further comprises:

receiving a user engagement with an advertisement content of the one or more advertisement contents when initiating a transaction request related to one or more of the object and one or more components associated with the object.

14. A method for interacting with an augmented reality environment, the method comprising:

determining a site boundary of a site for an object;

receiving an interaction request at an augmented reality device for interacting with an augmented reality representation of the object displayed relative to an augmented reality environment of the site;

determining whether the interaction request can be completed within the site boundary; and

in response to determining that the interaction request can be completed, proceeding with the interaction request in respect of the augmented reality representation of the object, otherwise displaying on the augmented reality device at least one cause preventing completion of the interaction request when the interaction request cannot be completed within the site.

15. The method of claim 14 , wherein the interaction request comprises at least one engagement corresponding to a real-world engagement.

16. The method of claim 14 further comprises displaying one or more alternative objects suitable for the site when the interaction request cannot be completed within the site.

17. A system for interacting with an augmented reality environment, the system comprising a processor operable to:

determine a site boundary of a site for an object;

receive an interaction request at an augmented reality device for interacting with an augmented reality representation of the object displayed relative to an augmented reality environment of the site;

determine whether the interaction request can be completed within the site boundary, comprising:

determining whether an operating range of the object for completing the interaction request is restricted by one or more of the site boundary and an obstacle within the site; and

in response to determining that the interaction request can be completed, proceed with the interaction request in respect of the augmented reality representation of the object.

18. The system of claim 17 , wherein the interaction request comprises at least one engagement corresponding to a real-world engagement.

19. The system of claim 17 , wherein the processor is operable to indicate that the interaction request cannot be completed as an operating range of the object is restricted by one or more of the site boundary and an obstacle within the site.

20. The system of claim 17 , wherein the processor is operable to indicate that the object is unsuitable for the site when it is determined that the interaction request cannot be completed.

21. The system of claim 20 , wherein the processor is operable to recommend an alternative object suitable for the site.

22. The system of claim 17 , wherein the processor is operable to request an analysis of the site with the augmented reality device when the interaction request cannot be completed within the site.

23. The system of claim 17 , wherein the processor is operable to display on the augmented reality device at least one cause preventing completion of the interaction request when the interaction request cannot be completed within the site.

24. The system of claim 17 , wherein the processor is operable to display one or more alternative objects suitable for the site when the interaction request cannot be completed within the site.

25. The system of claim 24 , wherein the processor is operable to determine the one or more alternative objects based on one or more of a user preference and a historical user data.

26. The system of claim 17 , wherein the processor is operable to receive a transaction request for the object via the augmented reality device.

27. The system of claim 26 , wherein the processor is operable to: at least one or more of select and substitute compatible components for the object to be displayed within the site for a transaction involving a real-world version of the object.

28. The system of claim 17 , wherein the interaction request comprises an engagement with one or more components of the object.

29. The system of claim 17 , wherein the processor is operable to:

determine an available space within the augmented reality environment for displaying one or more advertisement contents without interfering with the interaction request; and

display the one or more advertisement contents within the available space.

30. The system of claim 29 , wherein the processor is operable to receive a user engagement with an advertisement content of the one or more advertisement contents when initiating a transaction request related to one or more of the object and one or more components associated with the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 14, 2024
From: MASCARIN, ALEXANDER; PERALTA, STEPHAN; TUZI, ANDREA; PRESTA, MATTHEW DAVID; PRESTA, MICHAEL JAMES
To: EQPME INC.
Reel/Frame 067409/0691 →
Continuity (5)
Continuation 18229386 · Aug 2, 2023
Continuation In Part 18316498 · May 12, 2023
Continuation 17984338 · Nov 10, 2022
Provisional Application 63394716 · Aug 3, 2022
Related Publication 20240296638A1 · Sep 5, 2024
References Cited (37)
US 5646844A · Gudat et al. · 1997 [cited by applicant]
US 9070216B2 · Golparvar-Fard et al. · 2015 [cited by applicant]
US 9222771B2 · Rosengaus et al. · 2015 [cited by applicant]
US 10580207B2 · Pejic et al. · 2020 [cited by applicant]
US 20080071559A1 · Arrasvuori · 2008 [cited by applicant]
US 20090061901A1 · Arrasvuori et al. · 2009 [cited by applicant]
US 20100185529A1 · Chestnut · 2010 [cited by applicant]
US 20130234850A1 · Lee · 2013 [cited by applicant]
US 20140025481A1 · Kang et al. · 2014 [cited by applicant]
US 20160049005A1 · Mullins et al. · 2016 [cited by applicant]
US 20160165170A1 · McRae · 2016 [cited by applicant]
US 20160267516A1 · Walz · 2016 [cited by examiner]
US 20170116644A1 · Huang et al. · 2017 [cited by applicant]
US 20180349946A1 · Nguyen · 2018 [cited by applicant]
US 20200258144A1 · Chaturvedi et al. · 2020 [cited by applicant]
US 20200402219A1 · Gilbert · 2020 [cited by applicant]
US 20210020141A1 · Yuasa · 2021 [cited by applicant]
CA 3105792A1 · 2021 [cited by applicant]
WO 2018060783A1 · 2018 [cited by applicant]
Pix4d, “Turn your mobile device into a professional 3D scanner”, https://www.pix4d.com/product/pix4dcatch, downloaded May 11, 2022. [cited by applicant]
Sitemark, “Software for solar construction”, https://www.sitemark.com/industries/construction/, downloaded May 11, 2022. [cited by applicant]
ThreeKit, “3D Product Configurator & Augmented Reality For Commerce”, https://www.threekit.com/, downloaded Jun. 28, 2022. [cited by applicant]
JLG, “Product Announcement”, May 2022. [cited by applicant]
Strange, “Now You Can Drive Elon Musk's Tesla Cybertruck in AR from the Comfort of Your Home via iOS & Android”, Dec. 9, 2019, <https://mobile-ar.reality.news/news/now-you-can-drive-elon-musks-tesla-cybertruck-ar-from-c… [cited by applicant]
Makarov, “12 Augmented Reality Trends of 2023: New Milestones in Immersive Technology”, Aug. 2, 2022, <https://mobidev.biz/blog/augmented-reality-trends-future-ar-technologies> retrieved on Jun. 19, 2023. [cited by applicant]
“On-device voice commands in AR Glasses—Vuzix”, <https://vivoka.com/project/vuzix/>, Vivoka, Oct. 2021, retrieved on Jun. 7, 2023. [cited by applicant]
Echo3D, “Top 5 Augmented Reality (AR) Remote Assistance Tools”, <https://medium.com/echo3d/top-5-augmented-reality-ar-remote-assistance-tools-76e830cddf99>, Medium, Jul. 22, 2021, retrieved on Jun. 7, 2023. [cited by applicant]
Virsabi, “Tesla model 3”, <https://virsabi.com/tesla/> (Jul. 15, 2019), retrieved on Jun. 7, 2023. [cited by applicant]
Zero Density, “Augmented Reality—Revolutionize Your Storytelling with Augmented Reality Solutions”, <https://www.zerodensity.tv/solutions/augmented-reality/> (Apr. 6, 2023), retrieved on Jun. 14, 2023. [cited by applicant]
Virsabi, “Metaverse in Business”, <https://virsabi.com/> (Apr. 20, 2017), retrieved on Jun. 14, 2023. [cited by applicant]
Trimble <https://sitevision.trimble.com/> (Oct. 24, 2018), retrieved on Jun. 14, 2023. [cited by applicant]
Mantooth, TJ, “IP Issues for Retail Businesses Advertising in Augmented Reality”, Jul. 26, 2022, <https://ipwatchdog.com/2022/07/26/ip-issues-retail-businesses-advertising-augmented-reality/id=150383/> (retrieved on Jul… [cited by applicant]
Youtube. “Tesla Cybertruck doesn't fit in the average garage!”, Dec. 6, 2019, https://youtu.be/PuZz6OFMW5A (retrieved on Jul. 31, 2023). [cited by applicant]
AR Cybertruck. LuvBug Media, Mar. 8, 2020. Google Play Store, <https://play.google.com/store/apps/details?id=com.luvbug.cybertruck&pli=1> (retrieved on on Jul. 31, 2023). [cited by applicant]
International Search Report and Written Opinion mailed Oct. 25, 2023 in PCT/CA2023/051032. [cited by applicant]
Mourtzis et al., “Real-Time Remote Maintenance Support Based on Augmented Reality (AR)”, 2020. (Year: 2020). [cited by applicant]
Zhao et al., “Voice Interaction for Augmented Reality Navigation Interfaces with Natural Language Understanding”, IEEE, 2020. (Year: 2020). [cited by applicant]