IP Library › Granted Patent US 12,283,009
Granted Patent B2
US 12,283,009 · App. 18/378,899 · Granted Apr 22, 2025

Augmented reality system

Inventors: Vladimir Parfenov (Hopkinton, MA); Kevin Elliott Jordan (Cape Elizabeth, ME); Steven Thomas Dertien (Holliston, MA); Moshe Jacob Baum (Carlisle, MA); Andre Gosselin (Montreal, CA); Stephen Prideaux-Ghee (Bristol, GB); James E. Heppelman (Needham, MA)
Assignee: PTC Inc.
G06T19/006G06F3/0482G06F9/453G06F40/169G06T11/60G06T13/00G06T15/20G06T17/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,283,009
App. No.
18/378,899
Granted
Apr 22, 2025
Kind
B2
Abstract

An example augmented reality system includes: obtaining information about an instance of a device; recognizing the instance of the device based on the information; selecting a digital twin for the instance of the device, with the digital twin being unique to the instance of the device; and generating augmented reality content based on the digital twin and an actual graphic of the instance of the device.

Claims (60)

1. A method performed by one or more processing devices, the method comprising:

obtaining a digital graphic of a device;

reconfiguring the digital graphic of the device by incorporating one or more augmented reality (AR) components into the digital graphic to produce a digital graphic of a reconfigured device;

generating a digital twin for the reconfigured device, the digital twin being unique to the reconfigured device, where the digital twin is generated based on a digital twin for the device and one or more digital twins for one or more of the AR components, respectively;

simulating operation of the reconfigured device using the digital twin for the reconfigured device; and

generating an output for display showing the operation of the reconfigured device.

2. The method of claim 1 , wherein reconfiguring the digital graphic of the device comprises at least one of: adjusting a position of a component of the device in the digital graphic of the device, changing a geometry of the device in the digital graphic of the device, or changing a kinematic characteristic of the device in the digital graphic of the device.

3. The method of claim 1 , further comprising:

receiving specified conditions associated with a simulation of operation of the reconfigured device;

wherein simulating operation of the reconfigured device is performed for the specified conditions.

4. The method of claim 1 , further comprising:

receiving information for one or more stimuli;

wherein simulating operation of the reconfigured device is performed in a presence of the one or more stimuli.

5. The method of claim 4 , further comprising:

recording a response of the reconfigured device to the one or more stimuli in the digital twin for the reconfigured device.

6. The method of claim 4 , further comprising:

reporting, in real-time, a response of the reconfigured device to the one or more stimuli.

7. The method of claim 1 , further comprising:

storing information about manipulations performed to reconfigure the digital graphic of the device; and

reporting the manipulations to a third-party system configured to generate a blueprint or model for building the reconfigured device.

8. One or more non-transitory machine-readable storage media storing instructions that are executable by one or more processing devices to perform operations comprising:

obtaining a digital graphic of a device;

reconfiguring the digital graphic of the device by incorporating one or more augmented reality (AR) components into the digital graphic to produce a digital graphic of a reconfigured device;

generating a digital twin for the reconfigured device, the digital twin being unique to the reconfigured device, where the digital twin is generated based on a digital twin for the device and one or more digital twins for one or more of the AR components, respectively;

simulating operation of the reconfigured device using the digital twin for the reconfigured device; and

generating an output for display showing the operation of the reconfigured device.

9. The one or more non-transitory machine-readable storage media of claim 8 , wherein reconfiguring the digital graphic of the device comprises at least one of:

adjusting a position of a component of the device in the digital graphic of the device, changing a geometry of the device in the digital graphic of the device, or changing a kinematic characteristic of the device in the digital graphic of the device.

10. The one or more non-transitory machine-readable storage media of claim 8 , wherein the operations comprise:

receiving specified conditions associated with a simulation of operation of the reconfigured device;

wherein simulating operation of the reconfigured device is performed for the specified conditions.

11. The one or more non-transitory machine-readable storage media of claim 8 , wherein the operations comprise:

receiving information for one or more stimuli;

wherein simulating operation of the reconfigured device is performed in a presence of the one or more stimuli.

12. The one or more non-transitory machine-readable storage media of claim 11 , wherein the operations comprise:

recording a response of the reconfigured device to the one or more stimuli in the digital twin for the reconfigured device.

13. The one or more non-transitory machine-readable storage media of claim 11 , wherein the operations comprise:

reporting, in real-time, a response of the reconfigured device to the one or more stimuli.

14. The one or more non-transitory machine-readable storage media of claim 8 , wherein the operations comprise:

storing information about manipulations performed to reconfigure the digital graphic of the device; and

reporting the manipulations to a third-party system configured to generate a blueprint or model for building the reconfigured device.

15. A system comprising:

one or more non-transitory machine-readable storage media storing instructions that are executable; and

one or more processing devices to execute the instructions to perform operations comprising:

obtaining a digital graphic of a device;

reconfiguring the digital graphic of the device by incorporating one or more augmented reality (AR) components into the digital graphic to produce a digital graphic of a reconfigured device;

generating a digital twin for the reconfigured device, the digital twin being unique to the reconfigured device, where the digital twin is generated based on a digital twin for the device and one or more digital twins for one or more of the AR components, respectively;

simulating operation of the reconfigured device using the digital twin for the reconfigured device; and

generating an output for display showing the operation of the reconfigured device.

16. The system of claim 15 , wherein reconfiguring the digital graphic of the device comprises at least one of: adjusting a position of a component of the device in the digital graphic of the device, changing a geometry of the device in the digital graphic of the device, or changing a kinematic characteristic of the device in the digital graphic of the device.

17. The system of claim 15 , wherein the operations comprise:

receiving specified conditions associated with a simulation of operation of the reconfigured device;

wherein simulating operation of the reconfigured device is performed for the specified conditions.

18. The system of claim 15 , wherein the operations comprise:

receiving information for one or more stimuli;

wherein simulating operation of the reconfigured device is performed in a presence of the one or more stimuli.

19. The system of claim 18 , wherein the operations comprise:

recording a response of the reconfigured device to the one or more stimuli in the digital twin for the reconfigured device.

20. The system of claim 18 , wherein the operations comprise:

reporting, in real-time, a response of the reconfigured device to the one or more stimuli.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 10, 2024
From: PARFENOV, VLADIMIR; JORDAN, KEVIN ELLIOTT; DERTIEN, STEVEN THOMAS; BAUM, MOSHE JACOB; GOSSELIN, ANDRE; PRIDEAUX-GHEE, STEPHEN; HEPPELMAN, JAMES E.
To: PTC INC.
Reel/Frame 067065/0256 →
Continuity (6)
Continuation 17944796 · Sep 14, 2022
Continuation 17146589 · Jan 12, 2021
Continuation 16534756 · Aug 7, 2019
Continuation 15146398 · May 4, 2016
Provisional Application 62157234 · May 5, 2015
Related Publication 20240037875A1 · Feb 1, 2024
References Cited (79)
US 9361716B1 · Erikson · 2016 [cited by applicant]
US 9754413B1 · Gray · 2017 [cited by applicant]
US 10121171B1 · Chang et al. · 2018 [cited by applicant]
US 10305734B2 · Dost et al. · 2019 [cited by applicant]
US 10431005B2 · Parfenov et al. · 2019 [cited by applicant]
US 10572716B2 · Prideaux-Ghee et al. · 2020 [cited by applicant]
US 10755480B2 · Prideaux-Ghee et al. · 2020 [cited by applicant]
US 10922893B2 · Parfenov et al. · 2021 [cited by applicant]
US 11030808B2 · Prideaux-Ghee et al. · 2021 [cited by applicant]
US 11188739B2 · Prideaux-Ghee et al. · 2021 [cited by applicant]
US 11461981B2 · Parfenov et al. · 2022 [cited by applicant]
US 11810260B2 · Parfenov et al. · 2023 [cited by applicant]
US 20040100556A1 · Stromme · 2004 [cited by applicant]
US 20110058187A1 · Smith et al. · 2011 [cited by applicant]
US 20120324213A1 · Ching et al. · 2012 [cited by applicant]
US 20120324313A1 · Ho et al. · 2012 [cited by applicant]
US 20130201185A1 · Kochi · 2013 [cited by applicant]
US 20140186002A1 · Hanaya et al. · 2014 [cited by applicant]
US 20140310595A1 · Acharya et al. · 2014 [cited by applicant]
US 20140333667A1 · Jung et al. · 2014 [cited by applicant]
US 20150029222A1 · Hofmann · 2015 [cited by applicant]
US 20150052479A1 · Ooi et al. · 2015 [cited by applicant]
US 20150074578A1 · Liang et al. · 2015 [cited by applicant]
US 20150170256A1 · Pettyjohn et al. · 2015 [cited by applicant]
US 20150332512A1 · Siddiqui et al. · 2015 [cited by applicant]
US 20160005211A1 · Sarkis et al. · 2016 [cited by applicant]
US 20160291922A1 · Montgomerie et al. · 2016 [cited by applicant]
US 20160328883A1 · Parfenov et al. · 2016 [cited by applicant]
US 20170068314A1 · Fu et al. · 2017 [cited by applicant]
US 20170344199A1 · Taguchi et al. · 2017 [cited by applicant]
US 20180025544A1 · Schoeller et al. · 2018 [cited by applicant]
US 20180158250A1 · Yamamoto et al. · 2018 [cited by applicant]
US 20180336729A1 · Prideaux-Ghee et al. · 2018 [cited by applicant]
US 20180341808A1 · Holzer et al. · 2018 [cited by applicant]
US 20180347988A1 · Marusco et al. · 2018 [cited by applicant]
US 20190120619A1 · Miura · 2019 [cited by applicant]
US 20190122027A1 · Prideaux-Ghee et al. · 2019 [cited by applicant]
US 20190122435A1 · Prideaux-Ghee et al. · 2019 [cited by applicant]
US 20220164502A1 · Pei · 2022 [cited by examiner]
WO 2016064435A1 · 2016 [cited by applicant]
WO 20160064435 · 2016 [cited by applicant]
WO 2016179248A1 · 2016 [cited by applicant]
WO 20160179248 · 2016 [cited by applicant]
WO 2018213702A1 · 2018 [cited by applicant]
WO 20180213702 · 2018 [cited by applicant]
Action and Response History in U.S. Appl. No. 17/944,796 (now U.S. Pat. No. 11,810,260). [cited by applicant]
Nicolai et al., “The Web of Augmented Physical Objects,” Proc. ICBM, 2005, pp. 340-346. [cited by applicant]
Ong, S.K. et al., “Augmented Reality Aided Assembly Design and Planning,” Annals of the CIRP, 56(1):49-52 (2007). [cited by applicant]
International Search Report for PCT/US2016/030720, 6 pages (Sep. 4, 2016). [cited by applicant]
Written Opinion for PCT/US2016/030720, 12 pages (Sep. 4, 2016). [cited by applicant]
Invitation to Pay Additional Fees and, Where Applicable, Protest Fee for PCT/US2016/030730, 7 pages (Jul. 11, 2016). [cited by applicant]
International Preliminary Report on Patentability for PCT/US2016/030720 (Augmented Reality System, filed May 4, 2016, issued by IB/WIPO, 15 pages (Nov. 16, 2017). [cited by applicant]
International Search Report for PCT/US2018/033385 (Augmented Reality System filed May 18, 2018), issued by ISA/EP, 4 pages (Jul. 25, 2018). [cited by applicant]
Written Opinion for PCT/US2018/033385 (Augmented Reality System filed May 18, 2018), issued by ISA/EP, 7 pages (Jul. 25, 2018). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,329, 34 pages (Sep. 7, 2018). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,341, 35 pages (Sep. 7, 2018). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,316, 16 pages (May 23, 2019). [cited by applicant]
Office Action for U.S. Appl. No. 15/146,398, 24 pages (Oct. 4, 2018). [cited by applicant]
Final Office Action for U.S. Appl. No. 15/146,398, 24 pages (Mar. 21, 2018). [cited by applicant]
Office Action for U.S. Appl. No. 15/146,398, 23 pages (Aug. 25, 2017). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,341, 22 pages (Sep. 19, 2019). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,329, 25 pages (May 1, 2019). [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/747,812, issued Apr. 19, 2021 (7 pages). [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/789,329, issued Feb. 5, 2021 (10 pages). [cited by applicant]
File History of U.S. Appl. No. 15/146,398, 195 pages. [cited by applicant]
International Preliminary Report on Patentability for PCT/US2016/030720 (Augmented Reality System, filed May 4, 2016), issued by IB/WIPO, 13 pages (Nov. 16, 2017). [cited by applicant]
International Search Report and Written Opinion for PCT/US2016/030720, 17 pages (Sep. 4, 2016). [cited by applicant]
International Search Report and Written Opinion for PCT/US2018/033385 (Augmented Reality System filed May 18, 2018), issued by ISA/EP, 3 pages (dated Jul. 25, 2018). [cited by applicant]
Nicolai et al., The web of augmented physical objects. Proceedings of the ICBM. 2005. 340-6. [cited by applicant]
Office Action for U.S. Appl. No. 15/146,398 issued Aug. 25, 2017 (19 pages). [cited by applicant]
Ong et al., Augmented reality aided assembly design and planning. Annals CIRP. 2007; 56(1): 49-52. [cited by applicant]
Final Office Action for U.S. Appl. No. 15/146,398 issued Mar. 21, 2018 (23 pages). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,329, dated Sep. 7, 2018. (31 pages). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,341, dated Sep. 7, 2018. (32 pages). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,316, dated May 23, 2019 (9 pages). [cited by applicant]
Office Action for U.S. Appl. No. 15/146,398, dated Oct. 4, 2018 (23 pages). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,341, dated Sep. 19, 2019 (20 pages). [cited by applicant]
Office Action for U.S. Appl. No. 15/789,329, dated May 1, 2019 (24 pages). [cited by applicant]
Final Office Action for U.S. Appl. No. 15/789,341, dated Mar. 8, 2019 (19 pages). [cited by applicant]