IP Library Granted Patent US 12694381
Granted Patent B2
US 12694381 · App. 18/828,798 · Granted Jul 28, 2026

Systems and methods for interactive video presentation of transactional information

Inventor: Dushyant Sharma (Waxhaw, NC)
Assignee: PAYMENTUS CORPORATION
G06Q20/102G06F40/221G06F40/284G06F40/30G06Q20/326
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12694381
App. No.
18/828,798
Granted
Jul 28, 2026
Kind
B2
Abstract

Methods, apparatuses, and computer program products are described for presenting an interactive audio-visual presentation of transaction documents. A method can include receiving a bill associated with a payor and payee, using a textual language processor or the like to identify content fields from the bill and assign markups and/or metadata to content fields, and using the content fields, markups, and/or metadata to generate an audio-visual presentation associated with the bill. This audio-visual presentation can be presented to the payor. The payee may then interact with the audio-visual presentation, for instance by verbal, visual, manual, or textual response. A verbal language processing engine, natural language processing engine, audio-visual language processing engine, or visual-manual language processing engine can be initiated to facilitate interpretation of the payee response and generate a further audio-visual presentation.

Claims (101)

1 . A computer-implemented method comprising:

receiving, from a user device associated with a user, a first request for information about a bill issued to the user by a billing entity,

wherein the first request for information comprises at least: information about the user, information about the billing entity, information about the bill issued to the user by the billing entity, and one or more first user questions;

in an instance in which the first request for information about the bill does not comprise a digital version of the bill, requesting the digital version of the bill from the user device or a billing entity device associated with the billing entity and, upon receiving the digital version of the bill from the user device or the billing entity device, storing the digital version of the bill in a database associated with the user;

in an instance in which the first request for information about the bill comprises the digital version of the bill, extracting the digital version of the bill from the first request for information and storing the digital version of the bill in the database associated with the user;

programmatically extracting a plurality of bill content elements from the digital version of the bill issued to the user by the billing entity and storing the plurality of bill content elements in the database associated with the user;

programmatically generating a plurality of markups associated with each of the plurality of bill content elements extracted from the digital version of the bill and storing the plurality of markups and a mapping of the plurality of markups to the plurality of bill content elements in the database associated with the user,

wherein at least one of the plurality of markups assigned to each of the plurality of bill content elements indicate a particular portion of the digital version of the bill from which the respective bill content element of the plurality of bill content elements was extracted;

programmatically generating at least one metadata item associated with each of the plurality of markups and storing the at least one metadata item associated with each of the plurality of markups in the database associated with the user,

wherein the at least one metadata item comprises at least one tag identifying a content type of the markup to which the at least one metadata item is assigned;

determining, based upon the one or more first user questions in the first request for information about the bill issued to the user by the billing entity and the at least one metadata item associated with each of the plurality of markups, one or more bill content elements from among the plurality of bill content elements extracted from the digital version of the bill that contain information responsive to the one or more first user questions;

generating, based at least upon the one or more bill content elements from among the plurality of bill content elements extracted from the digital version of the bill that contain information responsive to the one or more first user questions, a first audio-visual presentation in an audio-visual format,

wherein the first audio-visual presentation comprises the information from the one or more bill content elements, from among the plurality of bill content elements, that is responsive to the one or more first user questions;

providing, to the user device associated with the user, the first audio-visual presentation for presentation to the user via the user device;

receiving, from the user device, in response to a user interaction with the user device following the presentation of the first audio-visual presentation to the user, a second request for information about the bill issued to the user by the billing entity, the second request for information comprising one or more second user questions not answered by the first audio-visual presentation;

determining, based upon the one or more second user questions in the second request for information about the bill issued to the user by the billing entity and the at least one metadata item associated with each of the plurality of markups, that the one or more second user questions are unanswerable because the plurality of bill content elements extracted from the digital version of the bill do not contain information responsive to the one or more second user questions;

sending to the billing entity device associated with the billing entity, a missing information request for additional information to answer the one or more second user questions;

receiving, from the billing entity device, in response to the missing information request, one or more items of missing information and storing the one or more items of missing information in the database associated with the user;

programmatically generating a second audio-visual presentation comprising information indicative of the one or more items of missing information; and

providing, to the user device, the second audio-visual presentation for presentation to the user via the user device.

2 . The computer-implemented method of claim 1 , wherein the plurality of bill content elements comprise one or more of: a user identifier, a user account identifier, a billing entity identifier, a bill amount, a bill date, a bill memo, or a bill description.

3 . The computer-implemented method of claim 1 , wherein the generating the first audio-visual presentation further comprises compiling, according to a predefined or dynamically defined order or structure, relevant subject matter from the plurality of bill content elements extracted from the digital version of the bill.

4 . The computer-implemented method of claim 1 , wherein the providing the first audio-visual presentation to the user device further comprises providing, with the first audio-visual presentation, a request for the user device to present the first audio-visual presentation to the user.

5 . The computer-implemented method of claim 1 , further comprising:

in an instance in which the one or more second user questions are answerable based upon additional information extracted from the plurality of bill content elements, the plurality of markups associated with each of the plurality of bill content elements, and at least one metadata item associated with each of the plurality of markups, generating the second audio-visual presentation based upon the additional information extracted from the plurality of bill content elements, the plurality of markups, and/or the one or more metadata items.

6 . The computer-implemented method of claim 1 , further comprising:

determining, based upon the second request for information, that the user is requesting initiation of a payment between the user and the billing entity in response to the bill issued to the user by the billing entity;

determining, from the plurality of bill content elements extracted from the digital version of the bill, a bill amount associated with the bill; and

generating a payment command message indicative of a command to initiate the payment of the bill amount from a user account associated with the user to an account associated with the billing entity.

7 . The computer-implemented method of claim 6 , further comprising:

providing the payment command message to at least one of: the billing entity device, the user device, a payment processing device, or a bank device.

8 . An apparatus comprising:

at least one processor; and

at least one memory storing instructions thereon that, when executed by the at least one processor, cause the apparatus to perform at least:

receiving, from a user device associated with a user, a first request for information about a bill issued to the user by a billing entity,

wherein the first request for information comprises at least: information about the user, information about the billing entity, information about the bill issued to the user by the billing entity, and one or more first user questions;

in an instance in which the first request for information about the bill does not comprise a digital version of the bill, requesting the digital version of the bill from the user device or a billing entity device associated with the billing entity and, upon receiving the digital version of the bill from the user device or the billing entity device, storing the digital version of the bill in a database associated with the user;

in an instance in which the first request for information about the bill comprises the digital version of the bill, extracting the digital version of the bill from the first request for information and storing the digital version of the bill in the database associated with the user;

extracting a plurality of bill content elements from the digital version of the bill issued to the user by the billing entity and storing the plurality of bill content elements in the database associated with the user;

generating a plurality of markups associated with each of the plurality of bill content elements extracted from the digital version of the bill and storing the plurality of markups and a mapping of the plurality of markups to the plurality of bill content elements in the database associated with the user,

wherein at least one of the plurality of markups assigned to each of the plurality of bill content elements indicates a particular portion of the digital version of the bill from which the respective bill content element of the plurality of bill content elements was extracted;

generating at least one metadata item associated with each of the plurality of markups and storing the at least one metadata item associated with each of the plurality of markups in the database associated with the user,

wherein the at least one metadata item comprises at least one tag identifying a content type of the markup to which the at least one metadata item is assigned;

determining, based upon the one or more first user questions in the first request for information about the bill issued to the user by the billing entity and the at least one metadata item associated with each of the plurality of markups, one or more bill content elements from among the plurality of bill content elements extracted from the digital version of the bill that contain information responsive to the one or more first user questions;

generating, based at least upon the one or more bill content elements from among the plurality of bill content elements extracted from the digital version of the bill that contain information responsive to the one or more first user questions, a first audio-visual presentation in an audio-visual format,

wherein the first audio-visual presentation comprises the information from the one or more bill content elements, from among the plurality of bill content elements, that is responsive to the one or more first user questions;

providing, to the user device associated with the user, the first audio-visual presentation for presentation to the user via the user device;

receiving, from the user device, in response to a user interaction with the user device following the presentation of the first audio-visual presentation to the user via the user device, a second request for information about the bill issued to the user by the billing entity,

wherein the second request for information comprises one or more second user questions not answered by to the first audio-visual presentation;

determining, based upon the one or more second user questions in the second request for information about the bill issued to the user by the billing entity and the at least one metadata item associated with each of the plurality of markups, that the one or more second user questions are unanswerable because the plurality of bill content elements extracted from the digital version of the bill do not contain information responsive to the one or more second user questions;

sending, to the billing entity device associated with the billing entity, a missing information request for additional information to answer the one or more second user questions;

receiving, from the billing entity device, in response to the missing information request, one or more items of missing information and storing the one or more items of missing information in the database associated with the user;

generating a second audio-visual presentation comprising information indicative of the one or more items of missing information; and

providing, to the user device, the second audio-visual presentation for presentation to the user via the user device.

9 . The apparatus of claim 8 , wherein the plurality of bill content elements comprise one or more of: a user identifier, a user account identifier, a billing entity identifier, a bill amount, a bill date, a bill memo, or a bill description, and wherein the first audio-visual presentation associated with the digital version of the bill is generated by compiling, according to a predefined or dynamically defined order or structure, relevant subject matter from the plurality of bill content elements extracted from the digital version of the bill.

10 . The apparatus of claim 8 , wherein the instructions stored on the at least one memory, when executed by the at least one processor, further cause the apparatus to perform at least:

in an instance in which the one or more second user questions are answerable based upon the plurality of bill content elements, the plurality of markups associated with each of the plurality of bill content elements, and the at least one metadata item associated with each of the plurality of markups, generating the second audio-visual presentation based at least upon additional information extracted from the plurality of bill content elements.

11 . The apparatus of claim 8 , wherein the instructions stored on the at least one memory, when executed by the at least one processor, further cause the apparatus to perform at least:

determining, based upon the second user request received in response to the presentation of the first audio-visual presentation to the user via the user device, that the user is requesting initiation of a payment between the user and the billing entity in response to the bill issued to the user by the billing entity;

determining, from the plurality of bill content elements extracted from the digital version of the bill, a bill amount associated with the bill; and

generating a payment command message indicative of a command to initiate the payment of the bill amount from a user account associated with the user to an account associated with the billing entity.

12 . The apparatus of claim 11 , wherein the instructions stored on the at least one memory, when executed by the at least one processor, further cause the apparatus to perform at least:

providing the payment command message to at least one of: the billing entity device, the user device, a payment processing device, or a bank device.

13 . The apparatus of claim 8 , wherein the at least one processor comprises at least one of:

a textual language processor configured to process and analyze textual language;

an audio-visual language processor configured to process, linguistically analyze, and generate verbal language, aural language, data indicative of recorded verbal language, data indicative of recorded aural language, visual textual information, visual numerical information, video, and images;

a verbal language processor configured to process and linguistically analyze verbal language, aural language, data indicative of recorded verbal language, and data indicative of recorded aural language; or

a natural language processor configured to process and linguistically analyze verbal language, aural language, data indicative of recorded verbal language, and data indicative of recorded aural language.

14 . The apparatus of claim 13 , wherein the at least one processor is configured to perform at least one of: optical character recognition, speech recognition, speech segmentation, text-to-speech processing, word segmentation, tokenization, knowledge extraction, neural network analysis, machine learning analysis, probabilistic analysis and decision making, lemmatization, morphological segmentation, part-of-speech tagging, stemming, statistical analysis and semantic extrapolation, grammar induction, sentence breaking, sentence boundary disambiguation, parsing, stochastic grammar analysis, lexical semantic analysis, distribution semantic analysis, named entry recognition, sentiment analysis, terminology extraction, word sense disambiguation, relationship extraction, semantic parsing, semantic role labelling, coreference resolution, discourse analysis, implicit semantic role labelling, textual entailment recognition, topic segmentation and recognition, automatic summarization, machine translation, natural language understanding, natural language generation, or question answering.

15 . The apparatus of claim 13 , wherein the instructions stored on the at least one memory, when executed by the at least one processor, further cause the apparatus to perform at least:

in response to receiving the first user request from the user device, initiating at least one of the verbal language processor, the natural language processor, or the audio-visual language processor.

16 . A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by a computing device, cause the computing device to perform at least the following:

receiving, from a user device associated with a user, a first request for information about a bill issued to the user by a billing entity,

wherein the first request for information comprises at least: information about the user, information about the billing entity, information about the bill issued to the user by the billing entity, and one or more first user questions;

in an instance in which the first request for information about the bill does not comprise a digital version of the bill, requesting the digital version of the bill from the user device or a billing entity device associated with the billing entity and, upon receiving the digital version of the bill from the user device or the billing entity device, storing the digital version of the bill in a database associated with the user;

in an instance in which the first request for information about the bill comprises the digital version of the bill, extracting the digital version of the bill from the first request for information and storing the digital version of the bill in the database associated with the user;

programmatically extracting a plurality of bill content elements from the digital version of the bill issued to the user by the billing entity and storing the plurality of bill content elements in the database associated with the user;

programmatically generating a plurality of markups associated with each of the plurality of bill content elements extracted from the digital version of the bill and storing the plurality of markups and a mapping of the plurality of markups to the plurality of bill content elements in the database associated with the user,

wherein at least one of the plurality of markups assigned to each of the plurality of bill content elements indicate a particular portion of the digital version of the bill from which the respective bill content element of the plurality of bill content elements was extracted;

programmatically generating at least one metadata item associated with each of the plurality of markups and storing the at least one metadata item associated with each of the plurality of markups in the database associated with the user,

wherein the at least one metadata item comprises at least one tag identifying a content type of the respective-markup to which the at least one metadata item is assigned;

determining, based upon the one or more first user questions in the first request for information about the bill issued to the user by the billing entity and the at least one metadata item associated with each of the plurality of markups, one or more bill content elements from among the plurality of bill content elements extracted from the digital version of the bill that contain information responsive to the one or more first user questions;

generating, based at least upon the one or more bill content elements from among the plurality of bill content elements extracted from the digital version of the bill that contain information responsive to the one or more first user questions, a first audio-visual presentation presenting in an audio-visual format,

wherein the first audio-visual presentation comprises the information from the one or more bill content elements, from among the plurality of bill content elements, that is responsive to the one or more first user questions; digital bill in an audio visual format;

providing, to the user device associated with the user, the first audio-visual presentation-associated with the digital bill for presentation of the first audio visual presentation to the user via the user device;

receiving, from the user device, a user request in response to a user interaction with the user device following the presentation of the first audio-visual presentation to the user-associated with the digital bill, a second request for information about the bill issued to the user by the billing entity, the second request for information comprising one or more second the user request comprising a user questions not answered by related to the first audio-visual presentation associated with the digital bill;

determining, based upon the one or more second user questions in the second request for information about the bill issued to the user by the billing entity and the at least one metadata item associated with each of the plurality of markups, that the one or more second user questions are unanswerable because the plurality of bill content elements extracted from the digital version of the bill do not contain information responsive to the one or more second user questions;

sending, to the billing entity device associated with the billing entity, a missing information request for additional information to answer the one or more second user questions;

receiving, from the billing entity device, in response to the missing information request, one or more items of missing information and storing the one or more items of missing information in the database associated with the user;

programmatically generating a second audio-visual presentation comprising information indicative of the one or more items of missing information; and

providing, to the user device, the second audio-visual presentation for presentation to the user via the user device.

17 . The non-transitory computer-readable-computer readable storage medium of claim 16 , wherein the instructions stored thereon, when executed by the computing device, further cause the computing device to perform:

in an instance in which the one or more second user questions are answerable based upon the plurality of bill content elements, the plurality of markups associated with each of the plurality of bill control elements, and the at least one metadata item associated with each of the plurality of markups, generating the second audio-visual presentation comprising the one or more second answers to the one or more second user questions based upon additional information extracted from the plurality of bill content elements.

18 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions stored thereon, when executed by the computing device, further cause the computing device to perform:

determining, based upon the second user request, that the user is requesting initiation of a payment between the user and the billing entity in response to the bill issued to the user by the billing entity;

determining, from the plurality of bill content elements extracted from the digital version of the bill, a bill amount associated with the digital bill; and

generating a payment command message indicative of a command to initiate the payment of the bill amount from a user account associated with the user to an account associated with the billing entity.

19 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions stored thereon, when executed by the computing device, further cause the computing device to perform:

providing the payment command message to at least one of: the billing entity device, the user device, a payment processing device, or a bank device.

20 . The non-transitory computer-readable storage medium of claim 19 , wherein the instructions stored thereon, when executed by the at least one processor, further cause the computing device to perform:

in response to receiving, from the user device, the first request for information, initiating at least one of the verbal language processor, the natural language processor, or the audio-visual language processor.