IP Library › Granted Patent US 11,443,402
Granted Patent B2
US 11,443,402 · App. 16/609,084 · Granted Sep 13, 2022

Synchronized data chaining using on-chip cache

Inventors: Benjamin Dodge (San Jose, CA); Jason Rupert Redgrave (Mountain View, CA); Xiaoyu Ma (Mountain View, CA)
Assignee: Google LLC
G06T1/20G06F13/28G06F15/7825G06T1/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,443,402
App. No.
16/609,084
Granted
Sep 13, 2022
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating, by an image sensor of a computing device, frame data comprising sub-frames of image pixel data. A first resource of the system-on-chip provides the frame data to a second resource of the system-on-chip. The frame data is provided to the second resource using a first data path included in the system-on-chip. The first resource provides a token to the second resource using a second data path included in the system-on-chip. A processor of the system-on-chip, uses the token to synchronize production of sub-frames of image pixel data provided by the first resource to the second resource and to synchronize consumption of the sub-frames of image pixel data received by the second resource from the elastic memory buffer.

Claims (68)

1. A computer-implemented method performed using a system-on-chip, the method comprising:

generating, by an image sensor of a computing device, frame data comprising sub-frames of image pixel data;

providing, by a first resource of the system-on-chip, the frame data to at least one second resource of the system-on-chip, wherein the frame data is provided to the at least one second resource using an elastic memory buffer and a first data path included in the system-on-chip;

providing, by the first resource of the system-on-chip, a token to the at least one second resource of the system-on-chip, wherein the token is provided to the at least one second resource using a second data path included in the system-on-chip; and

using, by a processor of the system-on-chip, at least the token provided using the second data path to:

i) synchronize production of sub-frames of image pixel data provided by the first resource to the at least one second resource; and

ii) synchronize consumption of the sub-frames of image pixel data received by the at least one second resource from the elastic memory buffer.

2. The method of claim 1 , further comprising:

providing, by the at least one second resource of the system-on-chip and using the second data path, a return token to the first resource, wherein the return token corresponds to the token provided by the first resource, and the return token is used along with the token to synchronize production of the sub-frames of image pixel data.

3. The method of claim 2 , wherein the first resource is a producer resource that uses a producer direct memory access unit to:

i) provide the frame data using the first data path;

ii) provide the token using the second data path; and

iii) receive the return token using the second data path.

4. The method of claim 3 , wherein the at least one second resource is a consumer resource that uses a consumer direct memory access unit to:

i) consume the frame data provided using the first path;

ii) receive the token provided using the second path; and

iii) provide the return token using the second data path.

5. The method of claim 4 , wherein:

i) the processor allocates a variable amount of memory included in the elastic memory buffer, the memory being used for transfers of sub-frame data between multiple respective producer-consumer pairs; and

ii) the processor adjusts an initial amount of allocated memory based on a quantity of producer-consumer pairs that actively transfer sub-frame data, and wherein the sub-frame data has a data size that is less than a threshold data size.

6. The method of any one of claims 2 to 5 , wherein the first resource provides the frame data to the at least one second resource via the first data path based on a sub-frame data consumption rate of the at least one second resource, wherein the sub-frame data consumption rate is computed by the processor at least by using the return token.

7. The method of any one of claims 2 to 6 , further comprising:

i) determining, by the processor and using the return token, a production rate by which the first resource is required to produce frame data to synchronize production of the sub-frames of image pixel data; and

ii) providing, by the first resource, the frame data based on the determined production rate.

8. The method of claim 7 , further comprising:

i) determining, by the processor and using the token, a consumption rate by which the at least one second resource is required to consume sub-frames of data to synchronize consumption of the sub-frames of image pixel data; and

ii) consuming, by the at least one second resource, the sub-frames of data based on the determined consumption rate.

9. The method of any one of the preceding claims, wherein the first path and the second path are part of the same data communications network included in the system-on-chip.

10. The method of any one of the preceding claims, wherein the second path is formed at the system-on-chip using one of multiple virtual channels included in the system-on-chip.

11. A system-on-chip comprising:

one or more processing devices; and

one or more non-transitory machine-readable storage devices storing instructions that are executable by the one or more processing devices to cause performance of operations comprising:

generating, by an image sensor of a computing device, frame data comprising sub-frames of image pixel data;

providing, by a first resource of the system-on-chip, the frame data to at least one second resource of the system-on-chip, wherein the frame data is provided to the at least one second resource using an elastic memory buffer and a first data path included in the system-on-chip;

providing, by the first resource of the system-on-chip, a token to the at least one second resource of the system-on-chip, wherein the token is provided to the at least one second resource using a second data path included in the system-on-chip; and

using, by a processor of the system-on-chip, at least the token provided using the second data path to:

i) synchronize production of sub-frames of image pixel data provided by the first resource to the at least one second resource; and

ii) synchronize consumption of the sub-frames of image pixel data received by the at least one second resource from the elastic memory buffer.

12. The system-on-chip of claim 11 , further comprising:

providing, by the at least one second resource of the system-on-chip and using the second data path, a return token to the first resource, wherein the return token corresponds to the token provided by the first resource, and the return token is used along with the token to synchronize production of the sub-frames of image pixel data.

13. The system-on-chip of claim 12 , wherein the first resource is a producer resource that uses a producer direct memory access unit to:

i) provide the frame data using the first data path;

ii) provide the token using the second data path; and

iii) receive the return token using the second data path.

14. The system-on-chip of claim 13 , wherein the at least one second resource is a consumer resource that uses a consumer direct memory access unit to:

i) consume the frame data provided using the first path;

ii) receive the token provided using the second path; and

iii) provide the return token using the second data path.

15. The system-on-chip of claim 14 , wherein:

i) the processor allocates a variable amount of memory included in the elastic memory buffer, the memory being used for transfers of sub-frame data between multiple respective producer-consumer pairs; and

ii) the processor adjusts an initial amount of allocated memory based on a quantity of producer-consumer pairs that actively transfer sub-frame data, and wherein the sub-frame data has a data size that is less than a threshold data size.

16. The system-on-chip of any one of claims 12 to 15 , wherein the first resource provides the frame data to the at least one second resource via the first data path based on a sub-frame data consumption rate of the at least one second resource, wherein the sub-frame data consumption rate is computed by the processor at least by using the return token.

17. The system-on-chip of any one of claims 12 to 16 , further comprising:

i) determining, by the processor and using the return token, a production rate by which the first resource is required to produce frame data to synchronize production of the sub-frames of image pixel data; and

ii) providing, by the first resource, the frame data based on the determined production rate.

18. The system-on-chip of claim 17 , further comprising:

i) determining, by the processor and using the token, a consumption rate by which the at least one second resource is required to consume sub-frames of data to synchronize consumption of the sub-frames of image pixel data; and

ii) consuming, by the at least one second resource, the sub-frames of data based on the determined consumption rate.

19. The system-on-chip of any one of claims 11 to 18 , wherein:

the first path and the second path are part of the same data communications network included in the system-on-chip; and

the second path is formed at the system-on-chip using one of multiple virtual channels included in the system-on-chip.

20. One or more non-transitory machine-readable storage devices storing instructions that are executable by one or more processing devices to cause performance of operations comprising:

generating, by an image sensor of a computing device, frame data comprising sub-frames of image pixel data;

providing, by a first resource of the system-on-chip, the frame data to at least one second resource of the system-on-chip, wherein the frame data is provided to the at least one second resource using an elastic memory buffer and a first data path included in the system-on-chip;

providing, by the first resource of the system-on-chip, a token to the at least one second resource of the system-on-chip, wherein the token is provided to the at least one second resource using a second data path included in the system-on-chip; and

using, by a processor of the system-on-chip, at least the token provided using the second data path to:

i) synchronize production of sub-frames of image pixel data provided by the first resource to the at least one second resource; and

ii) synchronize consumption of the sub-frames of image pixel data received by the at least one second resource from the elastic memory buffer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2019
From: DODGE, BENJAMIN; REDGRAVE, JASON RUPERT; MA, XIAOYU
To: GOOGLE LLC
Reel/Frame 051297/0773 →
Continuity (2)
Provisional Application 62594264 · Dec 4, 2017
Related Publication 20200193554A1 · Jun 18, 2020