IP Library Granted Patent US 12,222,213
Granted Patent B2
US 12,222,213 · App. 17/490,043 · Granted Feb 11, 2025

Digital twin based binaural audio echolocation

Inventors: Partho Ghosh (Kolkata, IN); Vinod A. Valecha (Pune, IN); Aruna Ratna Kumari Peketi (Visakhapatnam, IN); Liliya Bojkova-Popova (Chelmsford, MA)
Assignee: International Business Machines Corporation
G01C21/3629G01C21/3644G06V20/58
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,222,213
App. No.
17/490,043
Granted
Feb 11, 2025
Kind
B2
Abstract

A processor may receive an information dataset. The information dataset may include object information and environment information. A processor may generate a digital twin of a physical environment. The digital twin of the physical environment may be based, at least in part, on the information dataset. A processor may simulate one or more factors on the digital twin of the physical environment. A processor may determine, responsive to simulating the one or more factors, one or more pathways to a destination. The destination may be in the physical environment.

Claims (69)

1. A method of managing navigation in a physical environment, the method comprising:

receiving, by a processor, an information dataset from a real-time data feed, wherein the information dataset includes object information, user information, and environment information;

generating a digital twin of a physical environment and a user, wherein the digital twin of the physical environment is based, at least in part, on the information dataset, and wherein the digital twin of the physical environment and the user are actively changing as the real-time data feed provides continuously updated data associated with the information dataset;

simulating one or more factors on the digital twin of the physical environment;

identifying a dynamic landmark from the digital twin, wherein the dynamic landmark is a particular parameter change associated with the physical environment;

determining an accuracy level of the information dataset based on observing the dynamic landmark in the physical environment and the digital twin;

predicting, responsive to simulating the one or more factors, one or more pathways to a destination, wherein the destination is in the physical environment;

identifying a user limitation from the user information and digital twin, wherein the user limitation reflects a physical condition suffered by the user;

identifying a particular pathway from the one or more pathways, based on the user limitation; and

generating one or more binaural auditory notations to the user associated with the particular pathway.

2. The method of claim 1 , wherein the information dataset is received from the real-time data feed associated with one or more data collection devices.

3. The method of claim 1 , wherein the one or more binaural auditory notations direct the user to a primary pathway of the one or more pathways.

4. The method of claim 1 , wherein predicting the one or more pathways to the destination, includes:

detecting a static obstacle in the environment from the environment information; and

identifying a primary pathway from the one or more pathways, wherein the primary pathway avoids the static obstacle.

5. The method of claim 1 , wherein predicting the one or more pathways to the destination, includes:

detecting the dynamic obstacle in the physical environment from the environment information;

receiving a real-time data feed from one or more data collection devices, wherein the real-time data feed includes environment information associated with the dynamic obstacle;

simulating the digital twin of the physical environment to predict a position of the dynamic obstacle; and

identifying a primary pathway from the one or more pathways, wherein the primary pathway avoids the dynamic obstacle.

6. The method of claim 1 , further comprising:

generating one or more updated binaural auditory notations, wherein the one or more updated binaural auditory notations are based, at least in part, on the digital twin.

7. The method of claim 1 , wherein the dynamic landmark includes a pressure landmark associated with one or more changes in pressure at one or more different location points in the physical environment.

8. A system for managing navigation in a physical environment, the system comprising:

a memory; and

a processor in communication with the memory, the processor being configured to perform operations comprising:

receiving an information dataset from a real-time data feed, wherein the information dataset includes object information, user information, and environment information;

generating a digital twin of a physical environment and a user, wherein the digital twin of the physical environment is based, at least in part, on the information dataset, and wherein the digital twin of the physical environment and the user are actively changing as the real-time data feed provides continuously updated data associated with the information dataset;

simulating one or more factors on the digital twin of the physical environment;

identifying a dynamic landmark from the digital twin, wherein the dynamic landmark is a particular parameter change associated with the physical environment;

determining an accuracy level of the information dataset based on observing the dynamic landmark in the physical environment and the digital twin;

predicting, responsive to simulating the one or more factors, one or more pathways to a destination, wherein the destination is in the physical environment;

identifying a user limitation from the user information and digital twin, wherein the user limitation reflects a physical condition suffered by the user;

identifying a particular pathway from the one or more pathways, based on the user limitation; and

generating one or more binaural auditory notations to the user associated with the particular pathway.

9. The system of claim 8 , wherein the information dataset is received from the real-time data feed associated with one or more data collection devices.

10. The system of claim 8 , wherein the one or more binaural auditory notations direct the user to a primary pathway of the one or more pathways.

11. The system of claim 8 , wherein predicting the one or more pathways to the destination, includes:

detecting a static obstacle in the environment from the environment information; and

identifying a primary pathway from the one or more pathways, wherein the primary pathway avoids the static obstacle.

12. The system of claim 8 , wherein predicting the one or more pathways to the destination, includes:

detecting the dynamic obstacle in the physical environment from the environment information;

receiving a real-time data feed from one or more data collection devices, wherein the real-time data feed includes environment information associated with the dynamic obstacle;

simulating the digital twin of the physical environment to predict a position of the dynamic obstacle; and

identifying a primary pathway from the one or more pathways, wherein the primary pathway avoids the dynamic obstacle.

13. The system of claim 8 , further comprising:

generating one or more updated binaural auditory notations, wherein the one or more updated binaural auditory notations are based, at least in part, on the digital twin.

14. The system of claim 8 , wherein the dynamic landmark includes a pressure landmark associated with one or more changes in pressure at one or more different location points in the physical environment.

15. A computer program product for managing navigation in a physical environment, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processors to perform a function, the function comprising:

receiving an information dataset from a real-time data feed, wherein the information dataset includes object information, user information, and environment information;

generating a digital twin of a physical environment and a user, wherein the digital twin of the physical environment is based, at least in part, on the information dataset, and wherein the digital twin of the physical environment and the user are actively changing as the real-time data feed provides continuously updated data associated with the information dataset;

simulating one or more factors on the digital twin of the physical environment;

identifying a dynamic landmark from the digital twin, wherein the dynamic landmark is a particular parameter change associated with the physical environment;

determining an accuracy level of the information dataset based on observing the dynamic landmark in the physical environment and the digital twin;

predicting, responsive to simulating the one or more factors, one or more pathways to a destination, wherein the destination is in the physical environment;

identifying a user limitation from the user information and digital twin, wherein the user limitation reflects a physical condition suffered by the user;

identifying a particular pathway from the one or more pathways, based on the user limitation; and

generating one or more binaural auditory notations to the user associated with the particular pathway.

16. The computer program product of claim 15 , wherein the information dataset is received from the real-time data feed associated with one or more data collection devices.

17. The computer program product of claim 15 , wherein the one or more binaural auditory notations direct the user to a primary pathway of the one or more pathways.

18. The computer program product of claim 15 , wherein predicting the one or more pathways to the destination, includes:

detecting a static obstacle in the environment from the environment information; and

identifying a primary pathway from the one or more pathways, wherein the primary pathway avoids the static obstacle.

19. The computer program product of claim 15 , wherein predicting the one or more pathways to the destination, includes:

detecting the dynamic obstacle in the physical environment from the environment information;

receiving a real-time data feed from one or more data collection devices, wherein the real-time data feed includes environment information associated with the dynamic obstacle;

simulating the digital twin of the physical environment to predict a position of the dynamic obstacle; and

identifying a primary pathway from the one or more pathways, wherein the primary pathway avoids the dynamic obstacle.

20. The computer program product of claim 15 , wherein the dynamic landmark includes a pressure landmark associated with one or more changes in pressure at one or more different location points in the physical environment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2021
From: GHOSH, PARTHO; VALECHA, VINOD A.; PEKETI, ARUNA RATNA KUMARI; BOJKOVA-POPOVA, LILIYA
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 057653/0613 →
Continuity (1)
Related Publication 20230099791A1 · Mar 30, 2023
References Cited (20)
US 6671226B1 · Finkel · 2003 [cited by applicant]
US 10278003B2 · Satongar · 2019 [cited by applicant]
US 20120136566A1 · Bamba · 2012 [cited by examiner]
US 20180073886A1 · Ilse · 2018 [cited by examiner]
US 20210148726A1 · Ostafew · 2021 [cited by examiner]
US 20210392456A1 · Lacoche · 2021 [cited by examiner]
EP 2700907B1 · 2019 [cited by applicant]
Anonymous, “Apparatus and method for locating objects for visually impair individuals in an indoor environment,” Mar. 11, 2019, IP.com, “HTTPS://priorart.ip.com/IPCOM/000257775” (Year: 2019). [cited by examiner]
Anonymous. “Apparatus and method for locating objects for visually impair individuals in an indoor environment.” Published Mar. 11, 2019. 4 pages. Published by IP.com. https://priorart.ip.com/IPCOM/000257775. [cited by applicant]
Anonymous. “System and method for utilising citizen sensing to provide route planning for people with disabilities.” Published Mar. 19, 2014. 4 pages. Published by IP.com. https://priorart.ip.com/IPCOM/000235672. [cited by applicant]
Fan, Shelly. “This AI Uses Echolocation to Follow Your Every Move.” Published Jun. 23, 2019. 6 pages. Published by Singularity Hub. https://singularityhub.com/2019/06/23/this-ai-uses-echolocation-to-follow-your-every-mo… [cited by applicant]
Geronazzo, et al., “Creating an Audio Story with Interactive Binaural Rendering in Virtual Reality.” Published Nov. 14, 2019. 15 pages. In Wireless Communications and Mobile Computing, vol. 2019, Article ID 1463204. Pub… [cited by applicant]
Goldstein, et al., “Acoustic analysis of the Sonic Guide.” Accepted for publication Apr. 10, 1981. 8 pages. Published by The Journal of the Acoustical Society of America. https://asa.scitation.org/doi/pdf/10.1121/1.3867… [cited by applicant]
Johansson, Mathias. “VR for Your Ears: Dynamic 3D Audio Is Coming Soon.” Published Jan. 24, 2019. 6 pages. Published by IEEE Spectrum. https://spectrum.ieee.org/consumer-electronics/audiovideo/vr-for-your-ears-dynamic-3… [cited by applicant]
Lalwani, Mona. “Surrounded by sound: how 3D audio hacks your brain.” Published Feb. 12, 2015. 8 pages. Published by The Verge. https://www.theverge.com/2015/2/12/8021733/3d-audio-3dio-binaural-immersive-vr-sound-times-s… [cited by applicant]
Marin, et al., “Indoor Localization Techniques Within a Home Monitoring Platform.” Published Sep. 3, 2020. 25 pages. Published by ARXIV. https://arxiv.org/pdf/2009.01654.pdf. [cited by applicant]
Mathur, Natasha. “Researchers introduce a deep learning method that converts mono audio recordings into 3D sounds using video scenes.” Published Dec. 28, 2018. 4 pages. Published by Packt. https://hub.packtpub.com/resea… [cited by applicant]
Mell, et al., “The NIST Definition of Cloud Computing,” Recommendations of the National Institute of Standards and Technology, U.S. Department of Commerce, Special Publication 800-145, Sep. 2011, 7 pgs. [cited by applicant]
Nandini, et al., “A novel path planning algorithm for visually impaired people.” Published Mar. 23, 2017. 7 pages. In Journal of King Saud University—Computer and Information Sciences. Published by Science Direct. https… [cited by applicant]
Yadav, et al., “Fusion of Object Recognition and Obstacle Detection approach for Assisting Visually Challenged Person.” Published in 2020. 4 pages. In 2020 43rd International Conference on Telecommunications and Signal … [cited by applicant]