IP Library › Granted Patent US 12,253,678
Granted Patent B2
US 12,253,678 · App. 18/437,301 · Granted Mar 18, 2025

Apparatus and methods to extract data with smart glasses

Inventors: Sandeep Verma (Gurugram, IN); Nandini Rathaur (Hyderabad, IN); Pavan Chayanam (Alamo, CA); Srinivas Dundigalla (Waxhaw, NC)
Assignee: Bank of America Corporation
G02B27/0172G06T11/60G06T19/006G06V30/18019G02B2027/0141G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,253,678
App. No.
18/437,301
Granted
Mar 18, 2025
Kind
B2
Abstract

Apparatus and methods are provided for extracting and displaying information and data from a stream of information on a pair of smart glasses. One or more content filters describing the desired information and data may be communicated to a content filtering module. The content filtering module may process the stream of information to search for the desired information and data. When the desired information is found within the stream of information, the content filtering module may highlight the desired information, while blurring all other information, within a display on the pair of smart glasses.

Claims (95)

1. An apparatus for extracting and displaying key information on a pair of smart glasses comprising:

the smart glasses comprising:

a display;

a camera;

an input/output (“I/O”) module;

a network interface card (“NIC”);

a non-transitory memory configured to store an operating system; and

a processor configured to operate at least:

the operating system,

the NIC; and

the I/O module; and

a backend computer comprising:

a backend input/output (“bI/O”) module;

a backend network interface card (“bNIC”);

a backend non-transitory memory configured to store at least:

a backend operating system; and

a content filtering module; and

a backend processor configured to operate at least:

the backend operating system;

the content filtering module;

the bNIC; and

the bI/O module;

wherein:

one or more content filters are transferred to the content filtering module by a mining module configured to automatically mine a user's emails and text messages with one or more artificial intelligence/machine learning (“AI/ML”) algorithms;

the camera captures a stream of information; and

the content filtering module:

converts the one or more content filters into one or more keys;

converts the stream of information into an array of images;

processes each image in the array of images to convert each image into text;

searches the text for the one or more keys;

when the module discovers one or more advertisements in one or more images in the array of images:

deletes the one or more images comprising the one or more advertisements; and

when the module discovers one of the one or more keys in the text:

displays a corresponding image from the array of images on the display;

surrounds the image with a box; and

displays the remaining images from the array of images on the display.

2. The apparatus of claim 1 wherein the backend computer is a server.

3. The apparatus of claim 1 wherein the backend computer is a mobile device located proximate to the user.

4. The apparatus of claim 1 wherein the backend computer is wired to the pair of smart glasses.

5. The apparatus of claim 1 wherein the backend computer communicates wirelessly with the pair of smart glasses.

6. The apparatus of claim 1 wherein the content filtering module converts non-text inputs from the user into text via natural language processing.

7. The apparatus of claim 1 wherein the content filtering module is configured to use one or more artificial intelligence/machine learning (“AI/ML”) algorithms.

8. The apparatus of claim 1 wherein the one or more content filters are communicated to the content filtering module by a user through the I/O module.

9. The apparatus of claim 1 wherein when the content filtering module displays the remaining images on the display, the module blurs the remaining images.

10. The apparatus of claim 1 wherein the stream of information is a livestream visible to the user through the pair of smart glasses.

11. The apparatus of claim 1 wherein the content filtering module provides one or more updates to the user after the user moves away from the livestream.

12. A method for extracting and displaying key information on a pair of smart glasses, the method comprising:

communicating one or more filter terms to a content filtering module, wherein the one or more filter terms:

are determined automatically by a mining module; and

comprise one or more advertisement terms;

capturing a stream of information with a camera located on the smart glasses;

converting, by the content filtering module, the stream of information into an array of images;

processing, by the content filtering module, the array of images;

extracting, by the content filtering module, all textual characters, numbers, and symbols from the processed array of images;

searching, by the content filtering module, the extracted textual characters, numbers, and symbols for a match between the extracted textual characters, numbers, and symbols and the one or more filter terms;

when the content filtering module finds the match:

displaying, on the smart glasses, a corresponding image from the array of images;

displaying a box surrounding the corresponding image;

when the match is to one or more of the advertisement terms, deleting the one or more images comprising the one or more advertisements; and

blurring all remaining images from the array of images and displaying the blurred images on the smart glasses.

13. The method of claim 12 wherein the content filtering module is located at a mobile device proximate to the smart glasses.

14. The method of claim 12 wherein the content filtering module is located at a server remote from the smart glasses.

15. The method of claim 12 wherein the wherein the stream of information is a livestream visible to a user wearing the smart glasses.

16. The method of claim 15 further comprising providing updates, from the content filtering module to the user, after the user moves away from the livestream.

17. The method of claim 12 wherein the content filtering module is configured to use one or more artificial intelligence/machine learning (“AI/ML”) algorithms.

18. An apparatus for extracting and displaying key information on a pair of smart glasses comprising:

the smart glasses comprising:

a display;

a camera;

an input/output (“I/O”) module;

a network interface card (“NIC”);

a non-transitory memory configured to store at least:

an operating system; and

a content filtering module; and

a processor configured to operate at least:

the operating system,

the NIC;

the content filtering module; and

the I/O module;

wherein:

one or more content filters determined automatically by a mining module are communicated to the content filtering module;

the camera captures a stream of information; and

the content filtering module:

converts the one or more content filters into one or more keys;

converts the stream of information into an array of images;

processes each image in the array of images to convert each image into text;

searches the text for the one or more keys;

when the module discovers one or more advertisement terms in one or more images in the array of images:

deletes the one or more images comprising the one or more advertisements; and

when the module discovers one of the one or more keys in the text:

creates a virtual view within the display;

cuts a key image corresponding to the discovered key; and

displays the key image within the virtual view; and

when the module finds one or more anomalies within the array of images, the module automatically highlights the one or more anomalies.

19. The apparatus of claim 18 wherein the content filtering module is configured to use one or more artificial intelligence/machine learning (“AI/ML”) algorithms.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2024
From: VERMA, SANDEEP; RATHAUR, NANDINI; CHAYANAM, PAVAN; DUNDIGALLA, SRINIVAS
To: BANK OF AMERICA CORPORATION
Reel/Frame 066423/0363 →
Continuity (2)
Continuation 17692283 · Mar 11, 2022
Related Publication 20240184115A1 · Jun 6, 2024
References Cited (32)
US 7872593B1 · Rauscher et al. · 2011 [cited by applicant]
US 8108408B2 · Kondo et al. · 2012 [cited by applicant]
US 9589149B2 · Livshits · 2017 [cited by examiner]
US 10741173B2 · Kline · 2020 [cited by examiner]
US 10996741B2 · Bastide et al. · 2021 [cited by applicant]
US 11126846B2 · Neumann et al. · 2021 [cited by applicant]
US 11194971B1 · Dobranic · 2021 [cited by applicant]
US 11195315B2 · Jones et al. · 2021 [cited by applicant]
US 11687216B2 · Zionpour et al. · 2023 [cited by applicant]
US 20130127980A1 · Haddick et al. · 2013 [cited by applicant]
US 20130278631A1 · Border et al. · 2013 [cited by applicant]
US 20150215253A1 · Vemuri · 2015 [cited by examiner]
US 20150261867A1 · Singal et al. · 2015 [cited by applicant]
US 20160301953A1 · Garcia Navarro et al. · 2016 [cited by applicant]
US 20170200296A1 · Jones et al. · 2017 [cited by applicant]
US 20170262045A1 · Rouvinez et al. · 2017 [cited by applicant]
US 20180018348A1 · Qin et al. · 2018 [cited by applicant]
US 20200125322A1 · Wilde · 2020 [cited by applicant]
US 20200160578A1 · Jones · 2020 [cited by examiner]
US 20200193081A1 · Salama · 2020 [cited by examiner]
US 20200218585A1 · Dotan-Cohen et al. · 2020 [cited by applicant]
US 20210004770A1 · Bansal et al. · 2021 [cited by applicant]
US 20210064871A1 · Sohn et al. · 2021 [cited by applicant]
US 20210097762A1 · Spivak et al. · 2021 [cited by applicant]
US 20210329338A1 · Khov et al. · 2021 [cited by applicant]
US 20210365502A1 · Hutchins · 2021 [cited by applicant]
US 20220027626A1 · Kharbanda · 2022 [cited by examiner]
US 20220375172A1 · Meisenholder et al. · 2022 [cited by applicant]
US 20230269255A1 · Bacastow et al. · 2023 [cited by applicant]
US 20230288704A1 · Verma et al. · 2023 [cited by applicant]
Ajanki et al., An augmented reality interface to contextual information, Dec. 16, 2010, Virtual Reality SI: Augmented realty, pp. 161-173 (Year: 2010). [cited by examiner]
Ajanki et al., An augmented reality interface to contextual information, 2011, Virtual reality, Springer pp. 161-173. [cited by examiner]