IP Library › Granted Patent US 12,097,374
Granted Patent B2
US 12,097,374 · App. 17/027,957 · Granted Sep 24, 2024

Audio-visual interaction with implanted devices

Inventors: Subha Kiran Patnaikuni (Visakhapatnam, IN); Sarbajit K. Rakshit (Kolkata, IN)
Assignee: International Business Machines Corporation
A61N1/37217A61N1/08G06F3/011G06F3/167G06T15/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,097,374
App. No.
17/027,957
Granted
Sep 24, 2024
Kind
B2
Abstract

Aspects of the present invention disclose a method for audio-visual interaction of a user and implant device to resolve a voice command of the user. The method includes one or more processors determining a health-related issue of a user that corresponds to a voice command of the user. The method further includes identifying an implanted device of the user that relates to the health-related issue. The method further includes collecting sensor data of the implanted device, wherein the sensor data includes one or more conditions of the implanted device and operating environment of the implanted device in the body of the user. The method further includes generating an audio-visual response corresponding to the health-related issue of the user.

Claims (71)

1. A method for audio-visual interaction with implanted devices, the method comprising:

determining, by one or more processors, a health-related issue of a user based, at least in part, on historical medical data corresponding to the user and a voice command of the user;

identifying, by the one or more processors, an implanted device of the user that relates to the health-related issue of the user that was determined based at least in part on the historical medical data corresponding to the user and the voice command of the user;

collecting, by the one or more processors, sensor data of the implanted device, wherein the sensor data includes one or more conditions of the implanted device and operating environment of the implanted device in the body of the user; and

generating, by the one or more processors, an audio-visual response corresponding to the health-related issue of the user based at least in part on the sensor data that includes the one or more conditions of the implanted device and operating environment of the implanted device in the body of the user.

2. The method of claim 1 , further comprising:

generating, by the one or more processors, a three-dimensional augmented reality (AR) object corresponding to the implanted device based at least in part on the collected sensor data; and

generating, by the one or more processors, a visual representation of a condition of the operating environment of the implanted device in the body of the user based at least in part on the collected sensor data that includes the one or more conditions of the implanted device and operating environment of the implanted device in the body of the user.

3. The method of claim 1 , further comprising:

identifying, by the one or more processors, one or more objects within a sensor feed of an AR device;

determining, by the one or more processors, whether an activity associated with the one or more objects adversely affects the user based at least in part on the one or more conditions of the implanted device and the operating environment of the implanted device; and

generating, by the one or more processors, an overlay corresponding to each of the one or more objects, wherein the overlay indicates a recommendation of approval of one or more activities associated with the one or more objects.

4. The method of claim 1 , further comprising:

determining, by the one or more processors, whether a second device affects performance of the implanted device, wherein the second device is selected from a group consisting of: a second implanted device and external device.

5. The method of claim 1 , wherein identifying the implanted device of the user that relates to the health-related issue, further comprises:

correlating, by the one or more processors, one or more topics of the voice command with one or more conditions of the health-related issue of the user that was determined based at least in part on the historical medical data corresponding to the user and the voice command of the user; and

identifying, by the one or more processors, one or more implanted devices with a performance function that relates to the one or more topics of the voice command.

6. The method of claim 1 , wherein generating the audio-visual response corresponding to the health-related issue of the user, further comprises:

determining, by the one or more processors, a state of the implanted device based on the collected sensor data of the implanted device that relates to the voice command of the user;

determining, by the one or more processors, a condition of the body of the user in the location of the implanted device; and

generating, by the one or more processors, an audible response that includes information corresponding to the voice command of the user, wherein the information is based at least in part on the state of the implanted device and the condition of the body of the user in the location of the implanted device.

7. The method of claim 6 , further comprising:

transmitting, by the one or more processors, a three-dimensional augmented reality object corresponding to the implanted device and the audible response to the user through one or more computing devices, wherein at least one of the one or more computing devices includes an augmented reality enabled device.

8. A computer program product for audio-visual interaction with implanted devices, the computer product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to determine a health-related issue of a user based, at least in part, on historical medical data corresponding to the user and a voice command of the user;

program instructions to identify an implanted device of the user that relates to the health-related issue of the user that was determined based at least in part on the historical medical data corresponding to the user and the voice command of the user;

program instructions to collect sensor data of the implanted device, wherein the sensor data includes one or more conditions of the implanted device and operating environment of the implanted device in the body of the user; and

program instructions to generate an audio-visual response corresponding to the health-related issue of the user based at least in part on the sensor data that includes the one or more conditions of the implanted device and operating environment of the implanted device in the body of the user.

9. The computer program product of claim 8 , further comprising program instructions, stored on the one or more computer readable storage media, to:

generate a three-dimensional augmented reality (AR) object corresponding to the implanted device based at least in part on the collected sensor data; and

generate a visual representation of a condition of the operating environment of the implanted device in the body of the user based at least in part on the collected sensor data that includes the one or more conditions of the implanted device and operating environment of the implanted device in the body of the user.

10. The computer program product of claim 8 , further comprising program instructions, stored on the one or more computer readable storage media, to:

identify one or more objects within a sensor feed of an AR device;

determine whether an activity associated with the one or more objects adversely affects the user based at least in part on the one or more conditions of the implanted device and the operating environment of the implanted device; and

generate an overlay corresponding to each of the one or more objects, wherein the overlay indicates a recommendation of approval of one or more activities associated with the one or more objects.

11. The computer program product of claim 8 , further comprising program instructions, stored on the one or more computer readable storage media, to:

determine whether a second device affects performance of the implanted device, wherein the second device is selected from a group consisting of: a second implanted device and external device.

12. The computer program product of claim 8 , wherein program instructions to identify the implanted device of the user that relates to the health-related issue, further comprise program instructions to:

correlate one or more topics of the voice command with one or more conditions of the health-related issue of the user that was determined based at least in part on the historical medical data corresponding to the user and the voice command of the user; and

identify one or more implanted devices with a performance function that relates to the one or more topics of the voice command.

13. The computer program product of claim 8 , wherein program instructions to generate the audio-visual response corresponding to the health-related issue of the user, further comprise program instructions to:

determine a state of the implanted device based on the collected sensor data of the implanted device that relates to the voice command of the user;

determine a condition of the body of the user in the location of the implanted device; and

generate an audible response that includes information corresponding to the voice command of the user, wherein the information is based at least in part on the state of the implanted device and the condition of the body of the user in the location of the implanted device.

14. The computer program product of claim 13 , further comprising program instructions, stored on the one or more computer readable storage media, to:

transmit a three-dimensional augmented reality object corresponding to the implanted device and the audible response to the user through one or more computing devices, wherein at least one of the one or more computing devices includes an augmented reality enabled device.

15. A computer system for audio-visual interaction with implanted devices, the computer system comprising:

one or more computer processors;

one or more computer readable storage media; and

program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising:

program instructions to determine a health-related issue of a user based, at least in part, on historical medical data corresponding to the user and a voice command of the user;

program instructions to identify an implanted device of the user that relates to the health-related issue of the user that was determined based at least in part on the historical medical data corresponding to the user and the voice command of the user;

program instructions to collect sensor data of the implanted device, wherein the sensor data includes one or more conditions of the implanted device and operating environment of the implanted device in the body of the user; and

program instructions to generate an audio-visual response corresponding to the health-related issue of the user based at least in part on the sensor data that includes the one or more conditions of the implanted device and operating environment of the implanted device in the body of the user.

16. The computer system of claim 15 , further comprising program instructions, stored on the one or more computer readable storage media for execution by at least one of the one or more processors, to:

generate a three-dimensional augmented reality (AR) object corresponding to the implanted device based at least in part on the collected sensor data; and

generate a visual representation of a condition of the operating environment of the implanted device in the body of the user based at least in part on the collected sensor data that includes the one or more conditions of the implanted device and operating environment of the implanted device in the body of the user.

17. The computer system of claim 15 , further comprising program instructions, stored on the one or more computer readable storage media for execution by at least one of the one or more processors, to:

identify one or more objects within a sensor feed of an AR device;

determine whether an activity associated with the one or more objects adversely affects the user based at least in part on the one or more conditions of the implanted device and the operating environment of the implanted device; and

generate an overlay corresponding to each of the one or more objects, wherein the overlay indicates a recommendation of approval of one or more activities associated with the one or more objects.

18. The computer system of claim 15 , further comprising program instructions, stored on the one or more computer readable storage media for execution by at least one of the one or more processors, to:

determine whether a second device affects performance of the implanted device, wherein the second device is selected from a group consisting of: a second implanted device and external device.

19. The computer system of claim 15 , wherein identify the implanted device of the user that relates to the health-related issue, further comprise program instructions to:

correlate one or more topics of the voice command with one or more conditions of the health-related issue of the user that was determined based at least in part on the historical medical data corresponding to the user and the voice command of the user; and

identify one or more implanted devices with a performance function that relates to the one or more topics of the voice command.

20. The computer system of claim 15 , wherein generate the audio-visual response corresponding to the health-related issue of the user, further comprise program instructions to:

determine a state of the implanted device based on the collected sensor data of the implanted device that relates to the voice command of the user;

determine a condition of the body of the user in the location of the implanted device; and

generate an audible response that includes information corresponding to the voice command of the user, wherein the information is based at least in part on the state of the implanted device and the condition of the body of the user in the location of the implanted device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2020
From: PATNAIKUNI, SUBHA KIRAN; RAKSHIT, SARBAJIT K.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 053841/0828 →
Continuity (1)
Related Publication 20220088397A1 · Mar 24, 2022