Granted Patent B2

US 12,393,281 · App. 18/755,536 · Granted Aug 19, 2025

Inventors: Subburathinam Krishnan (Tamil Nadu, IN); Abhishek Arya (Telangana, IN); Prabhakaran Balasubramanian (Tamil Nadu, IN); Durga Prasad P. Khuttumolu (Telangana, IN)

Assignee: BANK OF AMERICA CORPORATION

G06F3/017G06F21/31G06T13/40

Abstract

Claims (71)

1. A system for multi-dimensional extrasensory deciphering of non-verbal communication of a user, the system comprising:

a virtual assistant device including a memory, one or more computing processing devices in communication with the memory and a plurality of sensors in communication with at least one of the one or more computing processing devices, wherein the plurality of sensors include at least one three-dimensional optical sensor configured for capturing, at least, optical signals that detect motion of the user, wherein the least one three-dimensional optical sensor are configured for detecting movements of an avatar;

an artificial intelligence (AI)-based engine stored in the memory, executable by at least one of the one or more computing processing devices, and including at least one first machine-learning (ML) model trained at least on user-specific data including historical resource exchange event data for purposes of determining non-verbal communication of the user, wherein the AI-based engine is configured to:

receive, at one or more of the first ML models, first inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor,

based on the first inputs, execute the one or more of the first ML models to determine a first transcript of the non-verbal communication of the user, and

communicate the first transcript of the non-verbal communication to a response system associated with the virtual assistant device, wherein the response system processes the non-verbal communication to determine a responsive communication to provide to the user; and

a floating image-generating and communication engine stored in the memory, executable by at least one of the one or more computing processing devices and configured to:

generate the avatar that is a three-dimensional floating image of a virtual assistant,

display the avatar in a three-dimensional-enabled display system, wherein the avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format, and

adjust, using a quality assurance sub-engine, the movements performed by the avatar based on the detected movements of the avatar in comparison to the responsive communication and the detected motion of the user.

2. The system of claim 1 , wherein the floating image-generating and communication engine is further configured to display the avatar in the three-dimensional-enabled display, wherein the avatar is configured to perform movements that replicate detected motion of the user.

3. The system of claim 1 , wherein the floating image-generating and communication engine is further configured to display the avatar in the three-dimensional-enabled display in at least one chosen from group consisting of (i) an active mode in which the avatar is displayed as a real-time response to capturing the optical signals associated with detected motion of the user and (ii) a passive mode in which the avatar is displayed in response to the virtual assistant device determining preconfigured communication that has been requested by the user.

4. The system of claim 1 , wherein the AI-based engine further comprises at least one second machine-learning (ML) model trained at least on user-specific data including historical resource exchange event emotional state data for purposes of determining an emotional state the user, and wherein the AI-based engine is configured to:

receive, at one or more of the second ML models, second inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor,

based on the second inputs, execute the one or more of the second ML models to determine an emotional state of the user, and

communicate the emotional state of the user to the response system, wherein the response system determines the responsive communication based further on the emotional state of the user.

5. The system of claim 1 , wherein the AI-based engine further comprises a user authentication sub-engine comprising at least one second machine-learning (ML) model trained at least on user-specific data including historical authentication credential input data for purposes of determining non-verbal authentication credential communication from the user, wherein the user authentication sub-engine is configured to:

receive, at one or more of the second ML models, second inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor,

based on the second inputs, execute the one or more of the second ML models to determine a second transcript of the non-verbal authentication credential communication of the user, and

communicate the second transcript of the non-verbal authentication credential communication to an authenticating entity, wherein the authenticating entity processes the non-verbal authentication credential communication to authenticate the user for conducting one or more resource exchange events.

6. The system of claim 1 , wherein the at least one three-dimensional optical sensor is further configured for detecting a preconfigured visual cue of the user, wherein detection of the preconfigured visual cue triggers further continual capturing of the optical signals.

7. A computer-implemented method for multi-dimensional extrasensory deciphering of non-verbal communication of a user, at least a portion of the method being executable by one or more computing device processors and comprising:

capturing optical signals from at least one three-dimensional optical sensor disposed within a virtual assistant device;

receiving, at one or more first ML models, first inputs derived from the optical signals captured from the at least one three-dimensional optical sensor, wherein the least one three-dimensional optical sensor are configured for detecting movements of an avatar;

based on the first inputs, executing the one or more first ML models to determine a first transcript of the non-verbal communication of the user;

communicating first transcript of the non-verbal communication to a response system associated with the virtual assistant device, wherein the response system processes the non-verbal communication to determine a responsive communication to provide to the user;

generating the avatar that is a three-dimensional floating image of a virtual assistant;

displaying the avatar in a three-dimensional-enabled display system, wherein the avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format; and

adjusting the movements performed by the avatar based on the detected movements of the avatar in comparison to the responsive communication and the detected motion of the user.

8. The computer-implemented method of claim 7 , further comprising:

generating at least one chosen from the group consisting of (i) a first avatar that is a three-dimensional floating image of the user and (ii) a second avatar that is a three-dimensional floating image of a virtual assistant; and

displaying the first avatar or second avatar in a three-dimensional-enabled display that is in communication with the virtual assistant device, wherein, upon display, the first avatar is configured to perform movements that replicate detected motion of the user and the second avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format.

9. The computer-implemented method of claim 8 , further comprising

detecting from the optical signals movements of at least one chosen from the group consisting of (i) the first avatar and (ii) the second avatar; and

adjusting the movements performed by at least one chosen from the group consisting of (i) the first avatar and (ii) the second avatar based on at least one chosen from the group consisting of (i) the detected movements of the first avatar in comparison to the detected motion of the user and (ii) the detected movements of the avatar in comparison to the responsive communication.

10. The computer-implemented method of claim 8 , wherein displaying the second avatar in the three-dimensional-enabled display further comprises displaying the second avatar in the three-dimensional-enabled display in at least one chosen from group consisting of (i) an active mode in which the avatar is displayed as a real-time response to capturing the optical signals associated with detected motion of the user and (ii) a passive mode in which the avatar is displayed in response to the virtual assistant device determining preconfigured communication that has been requested by the user.

11. The computer-implemented method of claim 7 , further comprising:

capturing signals from a plurality of sensors included within the virtual assistant device;

receiving, at one or more second ML models, second inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor;

based on the second inputs, executing the one or more second ML models to determine an emotional state of the user; and

communicating the emotional state of the user to the response system, wherein the response system determines the responsive communication based further on the emotional state of the user.

12. The computer-implemented method of claim 7 , further comprising:

capturing signals from a plurality of sensors included within the virtual assistant device;

receiving, at one or more second ML models, second inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor,

based on the second inputs, executing the one or more second ML models to determine a second transcript of a non-verbal authentication credential communication of the user; and

communicating the second transcript of a non-verbal authentication credential communication to an authenticating entity, wherein the authenticating entity processes the second transcript of a non-verbal authentication credential communication to authenticate the user for conducting one or more resource exchange events.

13. A computer program product comprising:

a non-transitory computer-readable medium comprising sets of codes for causing one or more computing devices to:

capture optical signals from at least one three-dimensional optical sensor disposed within a virtual assistant device, wherein the optical signals detect motion of a user;

receive, at one or more first ML models, first inputs derived from the optical signals captured from the at least one three-dimensional optical sensor, wherein the least one three-dimensional optical sensor are configured for detecting movements of an avatar;

based on the first inputs, (i) execute the one or more first ML models to determine a first transcript of the non-verbal communication of the user based on the detected motion;

communicate the first transcript of the non-verbal communication to a response system associated with the virtual assistant device, wherein the response system processes the non-verbal communication to determine a responsive communication to provide to the user;

generate the avatar that is a three-dimensional floating image of a virtual assistant;

display the avatar in a three-dimensional-enabled display system, wherein the avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format; and

adjust the movements performed by the avatar based on the detected movements of the avatar in comparison to the responsive communication and the detected motion of the user.

14. The computer program product of claim 13 , wherein the sets of codes further cause the one or more computing devices to:

generate at least one chosen from the group consisting of (i) a first avatar that is a three-dimensional floating image of the user and (ii) a second avatar that is a three-dimensional floating image of a virtual assistant; and

display the first avatar or second avatar in the three-dimensional-enabled display that is in communication with the virtual assistant device, wherein, upon display, the first avatar is configured to perform movements that replicate detected visual cues of the user and the second avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format.

15. The computer program product of claim 14 , wherein the sets of codes further cause the one or more computing devices to:

detect from the optical signals movements of at least one chosen from the group consisting of (i) the first avatar and (ii) the second avatar; and

adjust the movements performed by at least one chosen from the group consisting of (i) the first avatar and (ii) the second avatar based on at least one chosen from the group consisting of (i) the detected movements of the first avatar in comparison to the detected visual cues of the user and (ii) the detected movements of the avatar in comparison to the responsive communication.

16. The computer program product of claim 14 , wherein the sets of codes further cause the one or more computing devices to:

capture signals from a plurality of sensors included within the virtual assistant device;

receive, at one or more second ML models, second inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor;

based on the second inputs, execute the one or more second ML models to determine an emotional state of the user; and

communicate the emotional state of the user to the response system, wherein the response system determines the responsive communication based further on the emotional state of the user.

17. The computer program product of claim 14 , wherein the sets of codes further cause the one or more computing devices to:

capture signals from a plurality of sensors included within the virtual assistant device;

receive, at one or more second ML models, second inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor,

based on the second inputs, execute the one or more second ML models to determine a second transcript of a non-verbal authentication credential communication of the user; and

communicate the second transcript of the non-verbal authentication credential communication to an authenticating entity, wherein the authenticating entity processes the non-verbal authentication credential communication to authenticate the user for conducting one or more resource exchange events.

Assignments (1)

Continuity (2)

References Cited (26)