IP Library Granted Patent US 11,069,141
Granted Patent B2
US 11,069,141 · App. 16/426,277 · Granted Jul 20, 2021

Dynamic configuration of an augmented reality overlay

Inventors: David Kelly Wurmfeld (Fairfax, VA); Kevin Osborn (Newton Highlands, MA)
Assignee: Capital One Services, LLC
G06T19/006G06K9/00208G06K9/00671G06K9/4671G06T7/579G07F19/20G06Q20/1085
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,141
App. No.
16/426,277
Granted
Jul 20, 2021
Kind
B2
Abstract

A device may detect, in a field of view of a camera, one or more components of an automated teller machine (ATM) device using a computer vision technique based on generating a three dimensional model of the one or more components. The device may identify the ATM device as a particular device or as a particular type of device based on the one or more components of the ATM device, or first information related to the ATM device. The device may identify a set of tasks to be performed with respect to the ATM device. The device may provide, for display via a display associated with the device, second information associated with the set of tasks as an augmented reality overlay. The device may perform an action related to the set of tasks, the ATM device, or the augmented reality overlay.

Claims (99)

1. A method, comprising:

detecting, by a device, one or more components of an automated teller machine (ATM) device;

identifying, by the device, the ATM device as a particular device or as a particular type of device after detecting the one or more components using a combination of computer vision techniques that include a monocular simultaneous localization and mapping (monoSLAM) technique and based on comparing a spatial model of the one or more components to spatial models associated with the particular device or the particular type of device,

wherein identifying the ATM device comprises:

determining that a particular combination of components is included in a set of components of the one or more components after detecting the one or more components,

identifying a closest matching spatial model, of the spatial models, to the spatial model of the one or more components, and

identifying remaining components individually to identify a modification to an expected set of components;

identifying, by the device, a set of tasks to be performed with respect to the ATM device based on identifying the ATM device as the particular device or as the particular type of device;

providing, by the device, information associated with the set of tasks as an augmented reality overlay; and

performing, by the device, an action after providing the augmented reality overlay for display,

wherein the action is related to at least one of:

the set of tasks,

the ATM device, or

the augmented reality overlay.

2. The method of claim 1 , further comprising:

monitoring a performance of the set of tasks by capturing one or more images; and

updating the augmented reality overlay based on monitoring the performance of the set of tasks.

3. The method of claim 1 , further comprising:

monitoring performance of the set of tasks by at least one of:

processing images to identify which components of the one or more components are being serviced,

identifying changes in a configuration of the one or more components, or

determining whether the one or more components have been replaced.

4. The method of claim 1 , where the one or more components include at least one of:

a cash drawer of the ATM device,

a cash recycler of the ATM device,

a printer of the ATM device,

a keypad of the ATM device,

a processor of the ATM device, or

a display of the ATM device.

5. The method of claim 1 , further comprising:

generating the spatial model of the one or more components using the combination of computer vision techniques, wherein the combination of computer vision techniques include a scale-invariant feature transform (SIFT) technique.

6. The method of claim 1 , further comprising:

processing the spatial model of the one or more components using a feature extraction technique.

7. The method of claim 1 , further comprising:

capturing a set of images of the set of tasks as the set of tasks are being performed; and

processing the set of images to determine that a task, of the set of tasks, has been completed after capturing the set of images.

8. A device, comprising:

a memory; and

one or more processors operatively coupled to the memory, the memory and the one or more processors configured to:

detect one or more components of an automated teller machine (ATM) device;

identify the ATM device as a particular device or as a particular type of device after detecting the one or more components using a combination of computer vision techniques that include a monocular simultaneous localization and mapping (monoSLAM) technique and based on comparing a spatial model of the one or more components to spatial models associated the particular device or the particular type of device,

wherein the one or more processors, when identifying the ATM device, are to:

determine that a particular combination of components is included in a set of components of the one or more components after detecting the one or more components,

identify a closest matching spatial model, of the spatial models, to the spatial model of the one or more components, and

identify remaining components individually to identify a modification to an expected set of components;

identify a set of tasks to be performed with respect to the ATM device based on identifying the ATM device as the particular device or as the particular type of device; and

provide information associated with the set of tasks as an augmented reality overlay.

9. The device of claim 8 , wherein the one or more processors are further configured to:

perform an action related to at least one of the set of tasks, the ATM device, or the augmented reality overlay after providing the augmented reality overlay for display.

10. The device of claim 9 , wherein the one or more processors are further configured to:

update the augmented reality overlay based on determining that a task, of the set of tasks, has been performed with respect to the ATM device.

11. The device of claim 9 , wherein the one or more processors are further configured to:

monitor performance of the set of tasks by at least one of:

processing images to identify which components of the one or more components are being serviced,

identifying changes in a configuration of the one or more components, or

determining whether the one or more components have been replaced.

12. The device of claim 8 , where the one or more components include at least one of:

a cash drawer of the ATM device,

a cash recycler of the ATM device,

a printer of the ATM device,

a keypad of the ATM device,

a processor of the ATM device, or

a display of the ATM device.

13. The device of claim 8 , wherein the one or more processors are further configured to:

generate the spatial model of the one or more components using the combination of computer vision techniques, wherein the combination of computer vision techniques include a scale-invariant feature transform (SIFT) technique.

14. The device of claim 8 , wherein the one or more processors are further configured to:

capture a set of images of the set of tasks as the set of tasks are being performed; and

process the set of images to determine that a task, of the set of tasks, has been completed after capturing the set of images.

15. A non-transitory computer-readable medium storing one or more instructions for wireless communication, the one or more instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to:

identify an automated teller machine (ATM) device as a particular device or as a particular type of device after detecting one or more components associated with the ATM device using a combination of computer vision techniques that include a monocular simultaneous localization and mapping (monoSLAM) technique and by comparing a spatial model of the one or more components to spatial models associated the particular device or the particular type of device,

wherein the one or more instructions, that cause the one or more processors to identify the ATM device, cause the one or more processors to:

determine that a particular combination of components is included in a set of components of the one or more components after detecting the one or more components,

identify a closest matching spatial model, of the spatial models, to the spatial model of the one or more components, and

identify remaining components individually to identify a modification to an expected set of components;

identify a set of tasks to be performed with respect to the ATM device based on identifying the ATM device as the particular device or as the particular type of device;

provide information associated with the set of tasks as an augmented reality overlay; and

perform an action after providing the augmented reality overlay for display,

wherein the action is related to at least one of:

the set of tasks,

the ATM device, or

the augmented reality overlay.

16. The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

monitor a performance of the set of tasks by capturing one or more images; and

update the augmented reality overlay based on monitoring the performance of the set of tasks.

17. The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

monitor performance of the set of tasks by at least one of:

processing images to identify which components of the one or more components are being serviced,

identifying changes in a configuration of the one or more components, or

determining whether the one or more components have been replaced.

18. The non-transitory computer-readable medium of claim 15 , where the one or more components include at least one of:

a cash drawer of the ATM device,

a cash recycler of the ATM device,

a printer of the ATM device,

a keypad of the ATM device,

a processor of the ATM device, or

a display of the ATM device.

19. The non-transitory computer-readable medium of claim 15 , wherein the combination of computer vision techniques include a scale-invariant feature transform (SIFT) technique.

20. The non-transitory computer-readable medium of claim 15 , wherein the set of tasks is associated with servicing the ATM device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 30, 2019
From: WURMFELD, DAVID KELLY; OSBORN, KEVIN
To: CAPITAL ONE SERVICES, LLC
Reel/Frame 049319/0835 →
Continuity (2)
Continuation 15905542 · Feb 26, 2018
Related Publication 20190279431A1 · Sep 12, 2019
Cited By (8)
US 12,262,201 US 12,536,643 US 12,597,004 US 12,608,687 US 12,646,041 US 12,676,045 US 12,725,187 US 12,725,200