IP Library Granted Patent US 11,069,019
Granted Patent B2
US 11,069,019 · App. 15/587,341 · Granted Jul 20, 2021

Multi-threaded asynchronous frame processing

Inventors: Andrew Lankes Street (Midlothian, VA); Jordan Christopher Walke (Saratoga, CA); Ashwin Ravindra Bharambe (Sunnyvale, CA); Lin He (Los Altos, CA)
Assignee: Facebook, Inc.
G06T1/20G06F9/3009G06F9/3885G06F9/4881
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,019
App. No.
15/587,341
Granted
Jul 20, 2021
Kind
B2
Abstract

Parallel processing of frames by a first and second thread is disclosed. In particular, the systems and methods can process multiple frames in a pipelines manner by a JavaScript thread and a UI thread. The system first executes a first frame by a JavaScript thread during a first time interval. The system then executes the first frame by a UI thread during a second time interval. While the UI thread is executing the first frame, the JavaScript thread executes a second thread in parallel during the second time internal. As a result, latency is added to UI processing while maintaining the same throughput, to add parallelism. The JavaScript and UI threads can be handled by different processors (e.g., using a multi-core processor), thereby adding further parallelism and increasing the efficiency of processing an event.

Claims (34)

1. A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations for parallel processing of a first thread and a second thread, the operations comprising:

processing a first frame on the first thread on a first processor over a first time interval, wherein the first thread is configured for event processing;

processing the first frame on the second thread on a second processor over a second time interval overlapping with the first time interval such that the second thread processes the first frame in parallel with the first thread during an overlapping portion of the first and second time intervals, wherein the second thread is configured to interface with an operating system, and wherein the first thread is configured to be implemented in parallel to the second thread over the second time interval such that the first thread processes a second frame after processing the first frame; and

rendering, by the second thread, the first frame after processing the first frame is complete.

2. The computer-readable storage medium of claim 1 wherein the first thread is a JavaScript thread.

3. The computer-readable storage medium of claim 1 wherein the second thread is a User Interface thread.

4. The computer-readable storage medium of claim 1 wherein the first time interval and the second time interval are offset by a length of time.

5. The computer-readable storage medium of claim 4 wherein the length of time is time taken to process the first frame or the second frame.

6. The computer-readable storage medium of claim 5 wherein the time taken to process the first frame or the second frame is 16 milliseconds.

7. The computer-readable storage medium of claim 1 wherein the first frame is rendered to a graphical user interface coupled to the first processor or the second processor.

8. A method performed by a computing system having memory and a processor for parallel processing of a first thread and a second thread, comprising:

processing a first frame on the first thread on the processor over a first time interval, wherein the first thread is configured for event processing;

processing the first frame on the second thread on the processor over a second time interval overlapping with the first time interval such that the second thread processes the first frame in parallel with the first thread during an overlapping portion of the first and second time intervals, wherein the second thread is configured to interface with an operating system, and wherein the first thread is configured to be implemented in parallel to the second thread over the second time interval such that the first thread processes a second frame after processing the first frame; and

rendering, by the second thread, the first frame after processing the first frame is complete.

9. The method of claim 8 wherein the first thread is a JavaScript thread.

10. The method of claim 8 wherein the second thread is a User Interface thread.

11. The method of claim 8 wherein the first time interval and the second time interval are offset by a length of time.

12. The method of claim 11 wherein the length of time is time taken to process the first frame or the second frame.

13. The method of claim 12 wherein the time taken to process the first frame or the second frame is 16 milliseconds.

14. The method of claim 8 wherein the first frame is rendered to a graphical user interface coupled to the processor.

15. A system for processing a first thread and a second thread in parallel, comprising:

at least one hardware processor coupled to a network; and

a memory configured to store processor-executable components, wherein the processor-executable components further comprises:

a component configured to process a first frame on the first thread on the processor over a first time interval, wherein the first thread is configured for event processing; and

a component configured to process the second thread on the processor over a second time interval overlapping with the first time interval, wherein:

the second thread is configured to process the first frame in parallel with the first thread during an overlapping portion of the first and second time intervals,

the first thread is configured to be implemented in parallel to the second thread over the second time interval such that the first thread processes a second frame after processing the first frame, and

the second thread is configured to render the first frame after processing the first frame is complete.

16. The system of claim 15 wherein the first thread is a JavaScript thread.

17. The system of claim 15 wherein the second thread is a User Interface thread.

18. The system of claim 15 wherein the first time interval and the second time interval are offset by a length of time.

19. The system of claim 18 wherein the length of time is time taken to process the first frame or the second frame.

20. The system of claim 15 further comprising:

a graphical user interface to render the first frame or the second frame.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2022
From: WALKE, JORDAN
To: FACEBOOK, INC.
Reel/Frame 059019/0359 →
CHANGE OF NAME Recorded Jan 27, 2022
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058871/0336 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2021
From: STREET, ANDREW LANKES; BHARAMBE, ASHWIN RAVINDRA; HE, LIN
To: FACEBOOK, INC.
Reel/Frame 056568/0560 →