IP Library › Granted Patent US 11,194,995
Granted Patent B1
US 11,194,995 · App. 16/452,323 · Granted Dec 7, 2021

Video composition management system

Inventor: Francisco Antonio Lunalvo Profida Ferreira (Everett, WA)
Assignee: Amazon Technologies, Inc.
G06K9/00255G06K9/00234G06K9/00275G06T7/73H04N7/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,194,995
App. No.
16/452,323
Granted
Dec 7, 2021
Kind
B1
Abstract

Techniques for face tracking in a telemedicine environment. A method of face tracking in a telemedicine environment may include streaming video data to a user device from an agent device via an application on the agent device, the video data captured by a camera connected to the agent device through a first connector, analyzing a frame of the video data by an extension of the application to determine a position of a provider's face in the video data, generating, by the extension, a movement instruction based at least on the position of the provider's face in the video data, sending, by the extension, the movement instruction to a base application in communication with a motorized camera base coupled to the agent device through a second connector, wherein the motorized camera base executes the movement instruction causing a change in an orientation of the camera.

Claims (79)

1. A computer-implemented method comprising:

sending a request to log in to a videoconference web application through a web browser executing on an agent device, the web browser including a face tracking extension that is activated by the videoconference web application;

establishing an audio connection between the agent device and a user device via a call routing service of a provider network, wherein the call routing service provides a contact identifier associated with the user device to the agent device;

establishing a second connection between the agent device and the user device via a peer-to-peer (P2P) contact service using the contact identifier to obtain endpoint data for a P2P media server and a P2P signalizing server;

streaming video data to the user device via the videoconference web application over the second connection, the video data captured by a camera connected to the agent device through a first connector;

intercepting a frame of the video data by a face tracking browser extension installed in the web browser;

identifying a position of a provider's face in the video data;

determining a distance in pixels from the position of the provider's face to a center of the frame of the video data;

generating a step instruction based at least on the distance; and

sending the step instruction to a base application in communication with a motorized camera base coupled to the agent device through a second connector, wherein the motorized camera base executes the step instruction causing a change in an orientation of the camera.

2. The computer-implemented method of claim 1 , wherein generating a step instruction based at least on the distance, further comprises:

converting the distance in pixels to a number of steps; and

generating the step instruction based at least on the number of steps and a step rate.

3. The computer-implemented method of claim 2 , further comprising:

determining an interval of time has elapsed since the frame of the video data was intercepted, the interval of time based at least on the step rate;

intercepting a second frame of the video data by the face tracking browser extension;

analyzing the second frame of the video data to identify a second position of the provider's face in the video data;

generating a second step instruction based at least on the second position of the provider's face; and

sending the second step instruction to the base application.

4. A computer-implemented method comprising:

establishing a first connection between an agent device and a user device via a call routing service of a provider network, wherein the call routing service provides a contact identifier associated with the user device to the agent device;

establishing a second connection between the agent device and the user device via a peer-to-peer (P2P) contact service using the contact identifier;

streaming video data to the user device from the agent device via an application on the agent device over the second connection, the video data captured by a camera connected to the agent device through a first connector;

analyzing a frame of the video data by an extension of the application to determine a position of a provider's face in the video data;

generating, by the extension, a movement instruction based at least on the position of the provider's face in the video data; and

sending, by the extension, the movement instruction to a base application in communication with a motorized camera base coupled to the agent device through a second connector, wherein the motorized camera base executes the movement instruction causing a change in an orientation of the camera.

5. The computer-implemented method of claim 4 , further comprising:

storing, by the base application, the movement instruction in a buffer;

detecting, by the base application, the motorized camera base via the second connector; and

sending, by the base application, the movement instruction to the motorized camera base from the buffer via the second connector.

6. The computer-implemented method of claim 4 , further comprising:

sending a request to log in to the application through a web browser executing on the agent device, wherein the application is a videoconference web application and wherein the extension is a face tracking browser extension.

7. The computer-implemented method of claim 4 , further comprising:

determining an interval of time has elapsed since the frame of the video data was analyzed;

analyzing a second frame of the video data by the extension to identify a second position of the provider's face in the video data;

generating a second movement instruction based at least on the second position of the provider's face; and

sending the second movement instruction to the base application.

8. The computer-implemented method of claim 4 , wherein the first connector and the second connector are universal serial bus (USB) connectors.

9. The computer-implemented method of claim 4 , wherein the camera is coupled to the motorized camera base using a universal camera mount, the universal camera mount comprising at least one of a friction mount or a threaded mount.

10. The computer-implemented method of claim 6 , wherein the face tracking browser extension is activated by the videoconference web application following log in.

11. The computer-implemented method of claim 6 , wherein analyzing a frame of the video data by an extension of the application to determine a position of a provider's face in the video data, further comprises:

determining a distance in pixels from the position of the provider's face to a center of the frame of the video data;

converting the distance in pixels to a number of steps; and

generating the movement instruction based at least on the number of steps.

12. The computer-implemented method of claim 11 , wherein sending, by the extension, the movement instruction to a base application in communication with a motorized camera base coupled to the agent device through a second connector, wherein the motorized camera base executes the movement instruction causing a change in an orientation of the camera, further comprises:

sending a secure socket message from the extension to the base application.

13. The computer-implemented method of claim 12 , wherein sending a secure socket message from the extension to the base application further comprising:

starting a first secure connection between the face tracking browser extension and the base application;

sending the secure socket message to the base application in the first secure connection;

starting a second secure connection between the base application and the motorized camera base; and

sending the movement instruction to the motorized camera base in the second secure connection.

14. A system comprising:

a camera;

a motorized camera base coupled to the camera;

an agent device connected to the camera by a first connector and connected to the motorized camera base by a second connector, the agent device including instructions that upon execution cause the agent device to:

establish a first connection between an agent device and a user device via a call routing service of a provider network, wherein the call routing service provides a contact identifier associated with the user device to the agent device;

establish a second connection between the agent device and the user device via a peer-to-peer (P2P) contact service using the contact identifier;

stream video data to the user device from the agent device via an application on the agent device over the second connection, the video data captured by a camera connected to the agent device through a first connector;

analyze a frame of the video data by an extension of the application to determine a position of a provider's face in the video data;

generate, by the extension, a movement instruction based at least on the position of the provider's face in the video data; and

send, by the extension, the movement instruction to a base application in communication with a motorized camera base coupled to the agent device through a second connector, wherein the motorized camera base executes the movement instruction causing a change in an orientation of the camera.

15. The system of claim 14 , wherein the instructions, when executed further cause the agent device to:

store, by the base application, the movement instruction in a buffer;

detect, by the base application, the motorized camera base via the second connector; and

send, by the base application, the movement instruction to the motorized camera base from the buffer via the second connector.

16. The system of claim 14 , wherein the instructions, when executed further cause the agent device to:

send a request to log in to the application through a web browser executing on the agent device, wherein the application is a videoconference web application and wherein the extension is a face tracking browser extension.

17. The system of claim 16 , wherein the face tracking browser extension is activated by the videoconference web application following log in.

18. The system of claim 16 , wherein the instructions to analyze a frame of the video data by an extension of the application to determine a position of a provider's face in the video data, when executed, further causes the agent device to:

determine a distance in pixels from the position of the provider's face to a center of the frame of the video data;

convert the distance in pixels to a number of steps; and

generate the movement instruction based at least on the number of steps.

19. The system of claim 18 , wherein the instructions to send, by the extension, the movement instruction to a base application in communication with a motorized camera base coupled to the agent device through a second connector, wherein the motorized camera base executes the movement instruction causing a change in an orientation of the camera, when executed, further causes the agent device to:

send a secure socket message from the extension to the base application.

20. The system of claim 19 , wherein the instructions, when executed, further cause the agent device to:

determine an interval of time has elapsed since the frame of the video data was analyzed;

analyze a second frame of the video data by the extension to identify a second position of the provider's face in the video data;

generate a second movement instruction based at least on the second position of the provider's face; and

send the second movement instruction to the base application.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 26, 2019
From: PORFIDA FERREIRA, FRANCISCO ANTONIO LUNALVO
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 049590/0946 →
Cited By (3)
US 12,211,153 US 12,483,792 US 12,705,737