IP Library › Granted Patent US 12,347,056
Granted Patent B2
US 12,347,056 · App. 18/138,523 · Granted Jul 1, 2025

Smart placement of information based on dynamic environments

Inventors: Erik Alexander Hill (Woodinville, WA); Andrew Jackson Klein (Vancouver, CA); Benjamin James Andrews (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
G06T19/20G06T2210/61G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,347,056
App. No.
18/138,523
Granted
Jul 1, 2025
Kind
B2
Abstract

Techniques for intelligently repositioning a hologram for an object in a scene based on a past condition of the object and/or a planned future condition of the object are disclosed. A hologram is identified for an object in a scene. Identifying the hologram includes identifying a current location of the hologram within the scene. Time-based data associated with the object is accessed. The time-based data includes at least one of a past condition of the object or a planned future condition of the object. A new location for the hologram is selected within the scene. This selection is based on the time-based data. The hologram is then repositioned to the new location.

Claims (35)

1. A computer system that repositions a hologram for an object in a scene during a guided scenario, said computer system comprising:

a processor system; and

a memory system comprising instructions that are executable by the processor system to cause the computer system to:

identify the hologram for the object in the scene, identifying the hologram includes identifying a current location of the hologram within the scene, the hologram providing information associated with the object during the guided scenario;

access time-based data associated with the object, the time-based data includes at least one of: (i) a planned future condition of the object based on the guided scenario or (ii) a predicted future condition of the object based on an action of a user during the guided scenario, the action being directed either to the object or to a different object in the scene, the planned future condition and the predicted future condition identifying a future location of the object;

select a new location for the hologram within the scene using the time-based data; and

cause the hologram to be repositioned to the new location.

2. The computer system of claim 1 , wherein the new location is different than the future location of the object such that the new location is selected to avoid subsequently obfuscating the object when the object is subsequently located at the future location.

3. The computer system of claim 1 , wherein the time-based data includes a past condition of the object, wherein the past condition of the object includes a past usage of the object within a context of the scene, where the context of the scene is past usage and planned future usage of the object.

4. The computer system of claim 1 , wherein the planned future condition of the object includes a planned future usage of the object within a context of the scene, where the context of the scene includes a past usage and a planned future usage of the object.

5. The computer system of claim 1 , wherein selecting the new location for the hologram within the scene takes into account a plurality of criteria comprising one or more of: a specific placement of the hologram as specified by a content author, a placement range of the hologram as specified by the content author, a state of the object, a state of the different object in the scene, or a layout of the scene.

6. The computer system of claim 1 , wherein the instructions are executable by the processor system to cause the computer system to monitor objects in the scene and to maintain a historical log that tracks how those objects have been used, and wherein accessing the time-based data associated with the object comprises accessing the historical log.

7. The computer system of claim 1 , wherein the instructions are executable by the processor system to cause the computer system to monitor a duration that the object was used and/or a duration that the object was not used, and wherein accessing the time-based data associated with the object comprises accessing the duration.

8. The computer system of claim 1 , wherein the instructions are executable by the processor system to cause the computer system to monitor where the object is located or has previously been located, and wherein accessing the time-based data associated with the object comprises accessing the monitored location.

9. The computer system of claim 1 , wherein the instructions are executable by the processor system to cause the computer system to monitor what other objects the object interacted with, and wherein accessing the time-based data associated with the object comprises accessing the monitored other objects.

10. The computer system of claim 1 , wherein the instructions are executable by the processor system to cause the computer system to determine future usage of the object based on a procedure guidebook.

11. A method for repositioning a hologram for an object in a scene during a guided scenario provided by an extended reality (ER) device, said method being implemented by the ER device and comprising:

identifying the hologram for the object in the scene, identifying the hologram includes identifying a current location of the hologram within the scene, the hologram providing information associated with the object during the guided scenario;

accessing time-based data associated with the object, the time-based data includes at least one of: (i) a planned future condition of the object based on the guided scenario or (ii) a predicted future condition of the object based on an action of a user during the guided scenario, the action being directed either to the object or to a different object in the scene, the planned future condition and the predicted future condition identifying a future location of the object;

selecting a new location for the hologram within the scene based on the time-based data; and

causing the hologram to be repositioned to the new location.

12. The method of claim 11 , wherein the new location is one at which the hologram will not obfuscate a second object in the scene, where the second object is one that has a different planned usage within a context of the scene.

13. The method of claim 11 , wherein causing the hologram to be repositioned to the new location includes accessing a spatial map of the scene and relying on the spatial map to reposition the hologram.

14. The method of claim 11 , wherein the new location of the hologram is within a threshold distance relative to the object.

15. The method of claim 11 , wherein selecting the new location for the hologram within the scene is further based on an area that is identified as being one to avoid for placement of holograms.

16. A method for repositioning a hologram for an object in a scene during a guided scenario provided by an extended reality (ER) device, said method being implemented by the ER device and comprising:

identifying the hologram for the object in the scene, wherein identifying the hologram includes identifying a current location of the hologram within the scene, and wherein the hologram provides information associated with the object during the guided scenario;

accessing time-based data associated with the object, wherein the time-based data includes at least one of: (i) a planned future condition of the object based on the guided scenario or (ii) a predicted future condition of the object based on an action of a user during the guided scenario, the action being directed either to the object or to a different object in the scene, the planned future condition and the predicted future condition identifying a future location of the object;

accessing a set of placement governing criteria, wherein the set of placement governing criteria includes one or more of: (i) a specific placement of the hologram as specified by a content author, (ii) a placement range of the hologram as specified by the content author, (iii) a state of the object, (iv) a state of a different object in the scene, or (v) a layout of the scene;

selecting a new location for the hologram within the scene based on (i) the time-based data and (ii) the set of placement governing criteria; and

causing the hologram to be repositioned to the new location.

17. The method of claim 16 , wherein the time-based data includes a past condition of the object.

18. The method of claim 16 , wherein the set of placement governing criteria includes all of: (i) the specific placement of the hologram as specified by the content author, (ii) the placement range of the hologram as specified by the content author, (iii) the state of the object, (iv) the state of the different object in the scene, and (v) the layout of the scene.

19. The method of claim 16 , wherein the set of placement governing criteria includes the layout of the scene, wherein the layout of the scene includes an identification of an obstruction in the scene, and wherein the new location of the hologram is at least a threshold distance away from the obstruction.

20. The method of claim 16 , wherein the new location of the hologram is selected so as to avoid a dynamic area.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2023
From: HILL, ERIK ALEXANDER; KLEIN, ANDREW JACKSON; ANDREWS, BENJAMIN JAMES
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 063450/0231 →
Continuity (1)
Related Publication 20240355077A1 · Oct 24, 2024
References Cited (17)
US 10445942B1 · Pekelny · 2019 [cited by examiner]
US 10881353B2 · Shoudy · 2021 [cited by examiner]
US 10950056B2 · Park et al. · 2021 [cited by applicant]
US 11145126B1 · Bramwell et al. · 2021 [cited by applicant]
US 20110128555A1 · Rotschild · 2011 [cited by examiner]
US 20180299828A1 · Gasking · 2018 [cited by examiner]
US 20190096084A1 · Mayer · 2019 [cited by applicant]
US 20190282324A1 · Freeman · 2019 [cited by applicant]
US 20190380792A1 · Poltaretskyi · 2019 [cited by examiner]
KR 101623041B1 · 2016 [cited by applicant]
International Search Report and Written Opinion received for PCT Application No. PCT/US2024/024833, Sep. 2, 2024, 16 pages. [cited by applicant]
“Vuforia Studio Augmented Reality”, Retrieved from: https://web.archive.org/web/20221029100528/https://www.ptc.com/en/products/vuforia/vuforia-studio, Oct. 29, 2022, 6 Pages. [cited by applicant]
Chacko, Phil, “Unity Mars—a first-of-its-kind solution for intelligent AR”, Retrieved from: https://blog.unity.com/technology/introducing-unity-mars-a-first-of-its-kind-solution-for-intelligent-ar, Jun. 8, 2020, 11 Page… [cited by applicant]
Dixit, et al., “Optimal Information Placement in an Interactive 3D Environment”, In Proceedings of the 2007 ACM SIGGRAPH symposium on Video games, Aug. 4, 2007, pp. 141-148. [cited by applicant]
Gorajia, et al., “The future of manufacturing—Designing a robust industrial augmented reality solution”, Retrieved from: https://www.readkong.com/page/the-future-of-manufacturing-designing-a-robust-industrial-6637160, A… [cited by applicant]
Neb, et al., “Generation of AR-enhanced Assembly Instructions based on Assembly Features”, In proceedings of 51st CIRP Conference on Manufacturing Systems, Jan. 1, 2018, pp. 1118-1123. [cited by applicant]
Vice, et al., “Advanced deployment guides for Microsoft 365 and Office 365 products”, Retrieved from: https://learn.microsoft.com/en-us/microsoft-365/enterprise/setup-guides-for-microsoft-365?view=o365-worldwide, Feb. 2… [cited by applicant]