IP Library › Granted Patent US 12,536,740
Granted Patent B2
US 12,536,740 · App. 18/458,924 · Granted Jan 27, 2026

Providing awareness of privacy-related activities in virtual and real-world environments

Inventors: Youngwook Do (Atlanta, GA); Fraser Anderson (Toronto, CA); Frederik Brudy (Toronto, CA); George William Fitzmaurice (Toronto, CA)
Assignee: AUTODESK, INC.
G06T17/00G02B27/0172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,536,740
App. No.
18/458,924
Granted
Jan 27, 2026
Kind
B2
Abstract

One embodiment of the present invention sets forth a technique for providing awareness of privacy-related activities. The technique includes determining a privacy level associated with a user of an extended reality environment. The technique also includes presenting, using an internal display of a headset, one or more internal indicators identifying a location of a bystander located in a real-world environment, wherein a level of detail of each internal indicator is based on the privacy level associated with the user. The technique further includes presenting, using an external display, one or more external indicators that include a monitoring indicator representing being captured by the headset and presented to the user via the headset, and further include a user activity indicator representing one or more activities of the user, wherein a level of detail of the user activity indicator is based on the privacy level associated with the user.

Claims (31)

1 . A computer-implemented method for providing awareness of privacy-related activities, the method comprising:

determining a privacy level associated with a user of an extended reality environment;

presenting, using an internal display of a headset, one or more internal indicators identifying a location of a bystander located in a real-world environment, wherein a level of detail of each internal indicator is based on the privacy level associated with the user; and

presenting, using an external display, one or more external indicators that include a monitoring indicator depicting at least a portion of information presented on the internal display of the headset.

2 . The computer-implemented method of claim 1 , further comprising displaying, using the internal display of the headset, a privacy level indicator that is based on the privacy level associated with the user.

3 . The computer-implemented method of claim 2 , further comprising displaying the privacy level indicator in the real-world environment using a light emitting diode (LED), wherein the privacy level indicator has a color that corresponds to the privacy level.

4 . The computer-implemented method of claim 1 , wherein the privacy level is determined based on user input.

5 . The computer-implemented method of claim 1 , wherein the privacy level is determined based on a task being performed by the user in the extended reality environment.

6 . The computer-implemented method of claim 1 , wherein the one or more internal indicators are determined based on the privacy level.

7 . The computer-implemented method of claim 6 , wherein the one or more internal indicators are determined based on the privacy level using a user-specified indicator configuration that associates the privacy level with the one or more internal indicators.

8 . The computer-implemented method of claim 1 , wherein the one or more internal indicators comprise a side location indicator, wherein a location of the side location indicator on the internal display indicates a location of the bystander relative to the user.

9 . The computer-implemented method of claim 8 , wherein the side location indicator is displayed in response to determining that the privacy level associated with the user is a low privacy level.

10 . The computer-implemented method of claim 1 , wherein the one or more internal indicators comprise an overhead location indicator displaying a location of the bystander relative to the user in a two-dimensional overhead view.

11 . The computer-implemented method of claim 10 , wherein the overhead location indicator is displayed in response to determining that the privacy level associated with the user is a medium privacy level.

12 . The computer-implemented method of claim 1 , wherein the one or more internal indicators comprise a view of the bystander in the real-world environment, and wherein the view is displayed at a location on the internal display, wherein the location on the internal display corresponds to a location of the bystander in a field of view of the user.

13 . The computer-implemented method of claim 12 , wherein the view of the bystander is displayed in response to determining that the privacy level associated with the user is a high privacy level.

14 . The computer-implemented method of claim 1 , wherein the monitoring indicator includes a representation of the one or internal indicators that are being presented to the user.

15 . The computer-implemented method of claim 1 , wherein the one or more external indicators further include a user activity indicator representing one or more activities of the user, wherein a level of detail of the user activity indicator is based on the privacy level associated with the user.

16 . The computer-implemented method of claim 15 , wherein the user activity indicator includes a description of an activity in which the user is engaged.

17 . The computer-implemented method of claim 16 , wherein the description of the activity in which the user is engaged is displayed in response to determining that the privacy level associated with the user is a low privacy level.

18 . The computer-implemented method of claim 15 , wherein the user activity indicator includes a type of an activity in which the user is engaged, and the type of the activity is displayed in response to determining that the privacy level associated with the user is a medium privacy level.

19 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

determining a privacy level associated with a user of an extended reality environment;

presenting, using an internal display of a headset, one or more internal indicators identifying a location of a bystander located in a real-world environment, wherein a level of detail of each internal indicator is based on the privacy level associated with the user; and

presenting, using an external display, one or more external indicators that include a monitoring indicator depicting at least a portion of information presented on the internal display of the headset.

20 . A system, comprising:

one or more memories that store instructions, and

one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of:

determining a privacy level associated with a user of an extended reality environment;

presenting, using an internal display of a headset, one or more internal indicators identifying a location of a bystander located in a real-world environment, wherein a level of detail of each internal indicator is based on the privacy level associated with the user; and

presenting, using an external display, one or more external indicators that include a monitoring indicator depicting at least a portion of information presented on the internal display of the headset.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2023
From: DO, YOUNGWOOK; ANDERSON, FRASER; FITZMAURICE, GEORGE WILLIAM; BRUDY, FREDERIK
To: AUTODESK, INC.
Reel/Frame 064765/0773 →
Continuity (2)
Provisional Application 63493285 · Mar 30, 2023
Related Publication 20240331281A1 · Oct 3, 2024
References Cited (60)
US 11302027B2 · Keen · 2022 [cited by examiner]
US 11783552B2 · Ulbricht · 2023 [cited by examiner]
US 20090019553A1 · Narayanaswami · 2009 [cited by examiner]
US 20190320138A1 · Kaufthal · 2019 [cited by examiner]
US 20210286502A1 · Lemay · 2021 [cited by examiner]
US 20230237192A1 · Kahan · 2023 [cited by examiner]
Extended European Search Report for Application No. 24167105.6 dated Aug. 7, 2024. [cited by applicant]
Gottsacker et al., “Exploring Cues and Signaling to Improve Cross-Reality Interruptions”, IEEE International Symposium on Mixed and Augmented Reality Adjunct, DOI 10. I 109/ISMAR-Adjunct57072.2022.00179, Oct. 17, 2022, … [cited by applicant]
Ahmad et al., “Tangible Privacy: Towards User-Centric Sensor Designs for Bystander Privacy”, Association for Computing Machinery, https://doi.org/10.1145/3415187, vol. 4, No. CSCW2, Article 116, Oct. 2020, pp. 116:1-116… [cited by applicant]
Alharbi et al., “To Mask or Not to Mask? Balancing Privacy with Visual Confirmation Utility in Activity-Oriented Wearable Cameras”, https://doi.org/10.1145/3351230, vol. 3, No. 3, Article 72, Sep. 2019, pp. 72:1-72:29. [cited by applicant]
Balebako et al., “The Impact of Timing on the Salience of Smartphone App Privacy Notices”, SPSM'15, DOI: http://dx.doi.org/10.1145/2808117.2808119, Oct. 12, 2015, pp. 63-74. [cited by applicant]
Baudisch et al., “Halo: a Technique for Visualizing Off-Screen Locations”, Interaction Techniques for Constrained Displays, vol. 5, Issue 1, Apr. 5-10, 2003, pp. 481-488. [cited by applicant]
Cranor, Lorrie Faith, “Necessary But Not Sufficient: Standardized Mechanisms for Privacy Notice and Choice”, J. On Telecomm. & High Tech. L, vol. 10, 2012, pp. 273-308. [cited by applicant]
Denning et al., “In Situ with Bystanders of Augmented Reality Glasses: Perspectives on Recording and Privacy-Mediating Technologies”, http://dx.doi.org/10.1145/2556288.2557352, 2014, 10 pages. [cited by applicant]
Do et al., “Smart Webcam Cover: Exploring the Design of an Intelligent Webcam Cover to Improve Usability and Trust”, Association for Computing Machinery, https://doi.org/10.1145/3494983, vol. 5, No. 4, Article 154, Dec.… [cited by applicant]
Dourish et al., “Security In the Wild: User Strategies For Managing Security As An Every day, Practical Problem”, DOI 10.1007/s00779-004-0308-5, vol. 8, Jul. 6, 2004, pp. 391-401. [cited by applicant]
Ens et al., “Candid Interaction: Revealing Hidden Mobile and Wearable Computing Activities”, DOI: http://dx.doi.org/10.1145/2807442.2807449, UIST '15, Nov. 8-11, 2015, pp. 467-476. [cited by applicant]
Gaw et al., “Secrecy, Flagging, and Paranoia: Adoption Criteria in Encrypted E-Mail”, Apr. 22-27, 2006, pp. 591-600. [cited by applicant]
George et al., “Should I Interrupt or Not? Understanding Interruptions in Head-Mounted Display Settings”, DOI: https://doi:org/10:1145/3322276:3322363, Jun. 23-28, 2019, 14 pages. [cited by applicant]
George et al., “Seamless, Bi-directional Transitions along the Reality-Virtuality Continuum: A Conceptualization and Prototype Exploration”, IEEE International Symposium on Mixed and Augmented Reality, DOI 10.1109/ISMAR… [cited by applicant]
Ghosh et al., “NotifiVR: Exploring Interruptions and Notifications in Virtual Reality”, IEEE Transactions on Visualization and Computer Graphics, DOI: 10.1109/TVCG.2018.2793698, vol. 24, No. 4, Apr. 2018, pp. 1447-1456. [cited by applicant]
Gottsacker et al., “Diegetic Representations for Seamless Cross-Reality Interruptions”, 2021, 10 pages. [cited by applicant]
Grubert et al., “The Office of the Future: Virtual, Portable and Global”, 2018, 10 pages. [cited by applicant]
Hakkila et al., “Design Probes Study on User Perceptions of a Smart Glasses Concept”, The 14th International Conference on Mobile and Ubiquitous Multimedia, DOI: http://dx.doi.org/10.1145/2836041.2836064, Nov. 30-Dec. 2… [cited by applicant]
Koelle et al., “Beyond LED Status Lights—Design Requirements of Privacy Notices for Body-worn Cameras”, Shapechanging Textiles & Interactive Materials, DOI: https://doi.org/10.1145/3173225.3173234, Mar. 18-21, 2018, pp.… [cited by applicant]
Kudo et al., “Towards Balancing VR Immersion and Bystander Awareness”, In Proceedings of the ACM on Human-Computer Interaction, https://doi.org/10.1145/3486950, vol. 5, Article 484, Nov. 2021, pp. 484:1-484:22. [cited by applicant]
Langheinrich, Marc, “Privacy by Design—Principles of Privacy-Aware Ubiquitous Systems”, LNCS, 2001, pp. 273-291. [cited by applicant]
Ledo et al., “Evaluation Strategies for HCI Toolkit Research”, https://doi.org/10.1145/3173574.3173610, Apr. 21-26, 2018, pp. 1-17. [cited by applicant]
Mai et al., “Feeling Alone in Public—Investigating the Influence of Spatial Layout on Users' VR Experience”, DOI: https://doi:org/10:1145/3240167:3240200, Sep. 29-Oct. 3, 2018, pp. 1-13. [cited by applicant]
Mathis et al., “Prototyping Usable Privacy and Security Systems: Insights from Experts”, International Journal of Human-Computer Interaction, DOI: 10.1080/10447318.2021.1949134, Aug. 5, 2021, pp. 1-24. [cited by applicant]
Mathis et al., “RubikAuth: Fast and Secure Authentication in Virtual Reality”, DOI: https://doi.org/10.1145/3334480.3382827, Apr. 25-30, 2020, pp. 1-9. [cited by applicant]
Mcgill et al., “A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays”, http://dx.doi.org/10.1145/2702123.2702382, Apr. 18-23, 2015, pp. 2143-2152. [cited by applicant]
Medeiros et al., “Promoting Reality Awareness in Virtual Reality through Proxemics”, 2021, 10 pages. [cited by applicant]
Meta, “Meta Quest 2: Immersive All-In-One VR Headset”, Meta Store, Retrieved from https://store.facebook.com/ca/quest/products/quest-2/, on Sep. 4, 2023, 7 pages. [cited by applicant]
Emami-Naeini et al., “Privacy Expectations and Preferences in an IoT World”, Thirteenth Symposium on Usable Privacy and Security, USENIX Association, Jul. 12-14, 2017, pp. 399-412. [cited by applicant]
Ofek et al., “Towards a Practical Virtual Office for Mobile Knowledge Workers”, arXiv:2009.02947, Sep. 7, 2020, 7 pages. [cited by applicant]
O'Hagan et al., “Exploring Attitudes Towards Increasing User Awareness of Reality From Within Virtual Reality”, ACM International Conference on Interactive Media Experiences, https://doi.org/10.1145/3505284.3529971, Jun… [cited by applicant]
O'Hagan et al., “Surveying Consumer Understanding & Sentiment Of VR”, In International Workshop on Immersive Mixed and Virtual Environment Systems, https://doi.org/10.1145/3458307.3460965, Sep. 28-Oct. 1, 2021, 8 pages. [cited by applicant]
O'Hagan et al., “Reality Aware VR Headsets”, Session 1: Wearable Pervasive Displays, Association for Computing Machinery, https://doi.org/10.1145/3393712.3395334, 2020, 9 pages. [cited by applicant]
O'Hagan et al., “Safety, Power Imbalances, Ethics and Proxy Sex: Surveying In-The-Wild Interactions Between VR Users and Bystanders”, 2021, 10 pages. [cited by applicant]
Portnoff et al., “Somebody's Watching Me? Assessing the Effectiveness of Webcam Indicator Lights”, Social Media and Mobile Camera Privacy, http://dx.doi.org/10.1145/2702123.2702164, Apr. 18-23, 2015, pp. 1649-1658. [cited by applicant]
Redmon et al., “YOLOv3: An Incremental Improvement”, arXiv:1804.02767, Apr. 8, 2018, 6 pages. [cited by applicant]
“Ricoh Theta S”, Ltd Ricoh Company, Retrieved from https://theta360.com/en/about/theta/s.html, on Sep. 4, 2023, 14 pages. [cited by applicant]
Rietzler et al., “VaiR: Simulating 3D Airflows in Virtual Reality”, Interactions in Virtual Reality, DOI: http://dx.doi.org/10.1145/3025453.3026009, May 6-11, 2017, pp. 5669-5677. [cited by applicant]
Rossi et al., “Transparency By Design in Data-Informed Research: A Collection Of Information Design Patterns”, Computer Law & Security Review, vol. 37, https://doi.org/10.1016/j.clsr.2020.105402, 2020, pp. 1-22. [cited by applicant]
Schaub et al., “Designing Effective Privacy Notices and Controls”, Usable Security, IEEE Internet Computing, May/Jun. 2017, pp. 70-77. [cited by applicant]
Schaub et al., “A Design Space for Effective Privacy Notices”, Symposium on Usable Privacy and Security, USENIX Association, Jul. 22-24, 2015, pp. 1-17. [cited by applicant]
Schneider et al., “ReconViguRation: Reconfiguring Physical Keyboards in Virtual Reality”, arXiv:1907.08153, Jul. 18, 2019, pp. 1-12. [cited by applicant]
Schwind et al., “Virtual Reality on the Go? A Study on Social Acceptance of VR Glasses”, https://doi.org/10.1145/3236112.3236127, Sep. 3-6, 2018, pp. 111-118. [cited by applicant]
Simeone, Adalberto L., “The VR Motion Tracker: Visualising Movement of Non-Participants in Desktop Virtual Reality Experiences”, IEEE 2nd Workshop on Everyday Virtual Reality, 2016, 4 pages. [cited by applicant]
Starner, Thad, “The Challenges Of Wearable Computing: Part 2”, Jul.-Aug. 2001, pp. 54-67. [cited by applicant]
Richter, Felix, “AR & VR Adoption Is Still in Its Infancy”, Virtual and Augmented Reality, Jun. 2, 2023, 7 pages. [cited by applicant]
Steil et al., “PrivacEye: Privacy-Preserving Head-Mounted Eye Tracking Using Egocentric Scene Image and Eye Movement Features”, arXiv:1801.04457, Jun. 25-28, 2019, pp. 1-16. [cited by applicant]
Sunshine et al., “Crying Wolf: An Empirical Study of SSL Warning Effectiveness”, Aug. 10-14, 2009, 18th USENIX Security Symposium, 2009, pp. 1-18. [cited by applicant]
Willich et al., “You Invaded my Tracking Space! Using Augmented Virtuality for Spotting Passersby in Room-Scale Virtual Reality”, DOI: https://doi.org/10.1145/3322276.3322334, Jun. 23-28, 2019, pp. 1-10. [cited by applicant]
Whitten et al., “Why Johnny Can't Encrypt: A Usability Evaluation of PGP 5.0”, In Proceedings of the 8th USENIX Security Symposium, Aug. 1999, pp. 169-183. [cited by applicant]
Williamson et al., “PlaneVR: Social Acceptability of Virtual Reality for Aeroplane Passengers”, https://doi.org/10.1145/3290605.3300310, May 4-9, 2019, pp. 1-14. [cited by applicant]
Yang et al., “ShareSpace: Facilitating Shared Use of the Physical Space by both VR Head-Mounted Display and External Users”, Session 10: Navigation, Association of Computing Machinery, https://doi.org/10.1145/3242587.32… [cited by applicant]
Zhu et al., “BlinKey: A Two-Factor User Authentication Method for Virtual Reality Devices”, Association for Computing Machinery, https://doi.org/10.1145/3432217, vol. 4, No. 4, Article 164, Dec. 2020, pp. 164:1-164:29. [cited by applicant]
Rettinger et al., “Do You Notice Me? How Bystanders Affect the Cognitive Load in Virtual Reality”, In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces, DOI 10.1109/VR51125.2022.00025, 2022, pp. 77-82. [cited by applicant]