Granted Patent B2

US 10,217,375 · App. 15/377,795 · Granted Feb 26, 2019

Inventors: Willard H. Waldron (Charlotte, NC); Victoria L. Dravneek (Charlotte, NC); Joseph N. Johansen (Rock Hill, SC); Darryl C. Goebel (Cary, NC)

Assignee: Bank of America Corporation

G09B19/00G06F3/14G06T11/00G09B5/02H04L63/08H04L67/12G06F3/017G06F2221/2111

Abstract

Claims (118)

1. An augmented reality system comprising:

an augmented reality user device for a user comprising:

a display configured to overlay virtual objects onto tangible objects in a real scene in real-time;

a memory operable to store a plurality of scenarios linked with a virtual account for the user, wherein each scenario is linked with one or more challenges that affect the virtual account;

one or more processors operably coupled to the display and the memory, and configured to implement:

a virtual assessment engine configured to:

identify a user identifier for the user in response to authenticating the user;

receive an input from the user identifying a scenario from among the plurality of scenarios;

execute the scenario for the user, wherein executing the scenario identifies one or more challenges linked with the scenario;

identify one or more user actions;

determine whether the user actions are related to a challenge linked with the scenario;

generate user action information identifying the user actions and the scenario when the user actions are related to the scenario;

generate a token comprising:

 the user identifier, and

 the user action information;

send the token to a remote server;

receive virtual assessment data in response to sending the token to the remote server, wherein the virtual assessment data comprises:

 virtual account information for the user comprising a virtual account balance linked with the plurality of scenarios, and

 account information for the user; and

a virtual overlay engine configured to present the virtual account information and the account information as virtual objects overlaid with the real scene; and

the remote server comprising an augmented reality engine configured to:

identify the account information for the user based on the user identifier;

identify virtual account information for the user based on the user identifier;

modify the virtual account information based on the user action information;

generate the virtual assessment data comprising the modified virtual account information and the account information for the user; and

send the virtual assessment data to the augmented reality user device.

2. The system of claim 1 , wherein the augmented reality engine is configured to:

determine whether the user qualifies for a reward based on the modified virtual account information; and

modify the account information according to the reward when the user qualifies for the reward, and

wherein the virtual assessment data comprises the modified account information.

3. The system of claim 1 , wherein the augmented reality engine is configured to:

determine whether to modify the account information for the user based on the virtual account balance of the modified virtual account information; and

modify the account information for the user when the balance of the modified virtual account information meets a predetermined threshold value, and

wherein the virtual assessment data comprises the modified account information.

4. The system of claim 1 , wherein the augmented reality engine is configured to:

compare the virtual account information for the user to virtual account information for one or more other people;

compare the account information for the user to account information for one or more other people;

determine ranking information for the user based on the comparison of the virtual account information and the comparison of the account information; and

wherein the virtual assessment data comprises the ranking information.

5. The system of claim 1 , wherein the scenario comprises a task identifying a series of user actions; and

determining whether the user actions are related to the scenario comprises:

comparing the identified one or more user actions with the series of actions identified by the task; and

determining the user actions are related to the scenario when the identified one or more user actions match the series of actions identified by the task.

6. The system of claim 1 , wherein:

the augmented reality user device comprises a global position sensor configured to provide the geographic location of the user; and

the user action information comprises information identifying the geographic location of the user when performing the identified one or more user actions.

7. The system of claim 1 , wherein identifying the one or more user actions comprises:

performing object recognition to identify an object in the real scene; and

performing gesture recognition to identify a user action with the identified object.

8. The system of claim 1 , wherein the virtual overlay engine configured to present information related to the scenario as virtual objects overlaid with the real scene.

9. An augmented reality overlaying method comprising:

identifying, by a virtual assessment engine, a user identifier for a user in response to authenticating the user;

receiving, by the virtual assessment engine, an input from the user identifying a scenario from among a plurality of scenarios, wherein:

the plurality of scenarios is linked with a virtual account for the user; and

each scenario is linked with one or more challenges that affect the virtual account;

executing, by the virtual assessment engine, the scenario for the user, wherein executing the scenario identifies one or more challenges linked with the scenario;

identifying, by the virtual assessment engine, one or more user actions;

determining, by the virtual assessment engine, whether the user actions are related to a challenge linked with the scenario;

generating, by the virtual assessment engine, user action information identifying the user actions and the scenario when the user actions are related to the scenario;

generating, by the virtual assessment engine, a token comprising the user identifier and the user action information;

sending, by the virtual assessment engine, the token to a remote server;

identifying, by an augmented reality engine of the remote server, account information for the user based on the user identifier;

identifying, by the augmented reality engine, virtual account information for the user based on the user identifier, wherein the virtual account information comprises a virtual account balance linked with the plurality of scenarios;

modifying, by the augmented reality engine, the virtual account information based on the user action information;

generating, by the augmented reality engine, virtual assessment data comprising the modified virtual account information and the account information for the user; and

sending, by the augmented reality engine, the virtual assessment data to the augmented reality user device; and

presenting, by a virtual overlay engine, the virtual account information and the account information as virtual objects overlaid with the real scene.

10. The method of claim 9 , further comprising:

determining, by the augmented reality engine, whether the user qualifies for a reward based on the modified virtual account information; and

modifying, by the augmented reality engine, the account information according to the reward when the user qualifies for the reward, and

wherein the virtual assessment data comprises the modified account information.

11. The method of claim 9 , further comprising:

determining, by the augmented reality engine, whether to modify the account information for the user based on the virtual account balance of the modified virtual account information; and

modifying, by the augmented reality engine, the account information for the user when the balance of the modified virtual account information meets a predetermined threshold value, and

wherein the virtual assessment data comprises the modified account information.

12. The method of claim 9 , further comprising:

comparing, by the augmented reality engine, the virtual account information for the user to virtual account information for one or more other people;

comparing, by the augmented reality engine, the account information for the user to account information for one or more other people; and

determining, by the augmented reality engine, ranking information for the user based on the comparison of the virtual account information and the comparison of the account information; and

wherein the virtual assessment data comprises the ranking information.

13. The method of claim 9 , wherein the scenario comprises a task identifying a series of user actions; and

determining whether the user actions are related to the scenario comprises:

comparing the identified one or more user actions with the series of actions identified by the task; and

determining the user actions are related to the scenario when the identified one or more user actions match the series of actions identified by the task.

14. The method of claim 9 , wherein the user action information comprises information identifying a geographic location of the user when performing the identified one or more user actions.

15. The method of claim 9 , wherein identifying the one or more user actions comprises:

performing object recognition to identify an object in the real scene; and

performing gesture recognition to identify a user action with the identified object.

16. The method of claim 9 , further comprising presenting, by the virtual overlay engine, information related to the scenario as virtual objects overlaid with the real scene.

17. An augmented reality user device comprising:

a display configured to overlay virtual objects onto tangible objects in a real scene in real-time;

a memory operable to store a plurality of scenarios linked with a virtual account for a user, wherein each scenario is linked with one or more challenges that affect the virtual account;

one or more processors operably coupled to the display and the memory, and configured to implement:

a virtual assessment engine configured to:

identify a user identifier for the user in response to authenticating the user;

receive an input from the user identifying a scenario from among the plurality of scenarios;

execute the scenario for the user, wherein executing the scenario identifies one or more challenges linked with the scenario;

identify one or more user actions;

determine whether the user actions are related to a challenge linked with the scenario;

generate user action information identifying the user actions and the scenario when the user actions are related to the scenario;

generate a token comprising:

the user identifier, and

the user action information;

send the token to a remote server;

receive virtual assessment data in response to sending the token to the remote server, wherein the virtual assessment data comprises:

virtual account information for the user comprising a virtual account balance linked with the plurality of scenarios, and

account information for the user; and

a virtual overlay engine configured to present the virtual account information and the account information as virtual objects overlaid with the real scene.

18. The device of claim 17 , wherein the scenario comprises a task identifying a series of user actions; and

determining whether the user actions are related to the scenario comprises:

comparing the identified one or more user actions with the series of actions identified by the task; and

determining the user actions are related to the scenario when the identified one or more user actions match the series of actions identified by the task.

19. The device of claim 17 , wherein:

the augmented reality user device comprises a global position sensor configured to provide the geographic location of the user; and

the user action information comprises information identifying the geographic location of the user when performing the identified one or more user actions.

20. The device of claim 17 , wherein identifying the one or more user actions comprises:

performing object recognition to identify an object in the real scene; and

performing gesture recognition to identify a user action with the identified object.

Assignments (1)

Continuity (1)

Cited By (18)

US 12,211,015 US 12,211,095 US 12,229,737 US 12,236,700 US 12,260,381 US 12,260,658 US 12,260,700 US 12,400,257 US 12,505,414 US 12,511,692 US 12,530,666 US 12,572,936 US 12,573,224 US 12,579,832 US 12,608,690 US 12,626,526 US 12,632,899 US 12,639,680