Granted Patent B2

US 12,639,946 · App. 18/193,216 · Granted May 26, 2026

Inventor: Alejandro Paiuk (West Hartford, CT)

Assignee: Zoom Communications, Inc.

G06V20/46G06V20/63G06V30/416

Abstract

Claims (78)

1 . A method comprising:

receiving a video stream captured by a camera device of a first participant of a video conference;

performing object detection on the video stream during the video conference to determine an object type of a physical object presented by the first participant to the camera device;

determining a workflow related to the object type based on the determined object type,

wherein the workflow is selected from a plurality of different workflows based on the determined object type, and

wherein a first object type corresponds to a first workflow and a second, different object type corresponds to a second, different workflow; and

outputting data for executing the workflow at a device of a second participant of the video conference.

2 . The method of claim 1 , wherein the first participant is a user of a contact center, and the second participant is an agent of the contact center.

3 . The method of claim 1 , further comprising:

determining that the type of object is a payment card; and

determining that the workflow related to the type of object is a payment workflow.

4 . The method of claim 1 , further comprising:

determining that the object presented by the first participant displays sensitive information; and

masking the sensitive information in the video stream to generate a redacted video stream for display to the second participant.

5 . The method of claim 1 , further comprising:

determining that the object presented by the first participant displays sensitive information;

extracting the sensitive information from the video stream for use in the workflow;

masking the sensitive information in the video stream to generate a redacted video stream for display to the second participant; and

sending the sensitive information to the device of the second participant.

6 . The method of claim 1 , further comprising:

determining that the object includes text information;

performing optical character recognition to extract the text information from the object; and

sending the text information to the device of the second participant.

7 . The method of claim 1 , further comprising:

extracting information from the object; and

unlocking a physical object in a vicinity of the first participant based on contents of the extracted information.

8 . An apparatus, comprising:

a network communication interface;

a memory; and

a processor configured to execute instructions stored in the memory to:

receive a video stream captured by a camera device of a first participant of a video conference;

perform object detection on the video stream during the video conference to determine an object type of a physical object presented by the first participant to the camera device;

determine a workflow related to the object type based on the determined object type,

wherein the workflow is selected from a plurality of different workflows based on the determined object type, and

wherein a first object type corresponds to a first workflow and a second, different object type corresponds to a second, different workflow; and

output data for executing the workflow at a device of a second participant of the video conference.

9 . The apparatus of claim 8 , wherein the type of object is a payment card and to determine the workflow related to the object type comprises:

determining a payment workflow based on the payment card type of object.

10 . The apparatus of claim 8 , wherein the workflow is a customer service workflow executed at the device of the second participant.

11 . The apparatus of claim 8 , wherein the processor is configured to execute instructions stored in the memory to:

determine that the object type is associated with sensitive information; and

mask the sensitive information in the video stream to generate a redacted video stream for display to the second participant.

12 . The apparatus of claim 8 , wherein the processor is configured to execute instructions stored in the memory to:

determine that the object type is associated with sensitive information;

extract the sensitive information from the video stream for use in the workflow;

mask the sensitive information in the video stream to generate a redacted video stream for display to the second participant; and

send the sensitive information to the device of the second participant.

13 . The apparatus of claim 8 , wherein the processor is configured to execute instructions stored in the memory to:

determine that the object type is associated with text information;

perform optical character recognition to extract the text information from the object; and

send the text information to the device of the second participant.

14 . The apparatus of claim 8 , wherein the processor is configured to execute instructions stored in the memory to:

extract unlocking information from the object; and

send a signal to unlock a physical object in a vicinity of the first participant based on the unlocking information.

15 . A non-transitory computer-readable medium storing instructions operable to cause one or more processors to perform operations comprising:

receiving a video stream captured by a camera device of a first participant of a video conference;

performing object detection on the video stream during the video conference to determine an object type of a physical object presented by the first participant to the camera device;

determining a workflow related to the object type based on the determined object type,

wherein the workflow is selected from a plurality of different workflows based on the determined object type, and

wherein a first object type corresponds to a first workflow and a second, different object type corresponds to a second, different workflow; and

outputting data for executing the workflow at a device of a second participant of the video conference.

16 . The non-transitory computer-readable medium of claim 15 , wherein the first participant is a user of a contact center, and the second participant is a local agent of the contact center.

17 . The non-transitory computer-readable medium of claim 15 , wherein the instructions are further operable to cause the one or more processors to perform operations comprising:

determining that the type of object is a payment card displaying a payment card number;

performing optical character recognition to read the payment card number; and

sending the payment card number to the device of the second participant.

18 . The non-transitory computer-readable medium of claim 15 , wherein the instructions are further operable to cause the one or more processors to perform operations comprising:

determining that the type of object is a payment card displaying a payment card number;

performing optical character recognition to read the payment card number;

masking the payment card number in the video stream to generate a redacted video stream; and

sending the redacted video stream to the device of the second participant for display and including the payment card number in the data output for executing a workflow at the device of the second participant.

19 . The non-transitory computer-readable medium of claim 15 , wherein the instructions are further operable to cause the one or more processors to perform operations comprising:

determining that the object presented by the first participant displays sensitive information;

extracting the sensitive information from the video stream for use in the workflow; and

masking the sensitive information in the video stream to generate a redacted video stream for display to the second participant.

20 . The non-transitory computer-readable medium of claim 15 , wherein the instructions are further operable to cause the one or more processors to perform operations comprising:

determining that the object type is associated with a locked physical object in a vicinity of the first participant; and

unlocking the physical object based on the object.

Assignments (2)

Continuity (1)

References Cited (61)