IP Library Granted Patent US 11,698,529
Granted Patent B2
US 11,698,529 · App. 16/506,479 · Granted Jul 11, 2023

Systems and methods for distributing a neural network across multiple computing devices

Inventors: Liangzhen Lai (Fremont, CA); Pierce I-Jen Chuang (Sunnyvale, CA); Vikas Chandra (Fremont, CA); Ganesh Venkatesh (San Jose, CA)
Assignee: Meta Platforms Technologies, LLC
G02B27/017G06N3/04G06N3/045H04N13/106G02B2027/014G02B2027/0138
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,698,529
App. No.
16/506,479
Granted
Jul 11, 2023
Kind
B2
Abstract

Disclosed herein is a method for using a neural network across multiple devices. The method can include receiving, by a first device configured with a first one or more layers of a neural network, input data for processing via the neural network implemented across the first device and a second device. The method can include outputting, by the first one or more layers of the neural network implemented on the first device, a data set that is reduced in size relative to the input data while identifying one or more features of the input data for processing by a second one or more layers of the neural network. The method can include communicating, by the first device, the data set to the second device for processing via the second one or more layers of the neural network implemented on the second device.

Claims (34)

1. A method comprising:

receiving, by processing circuitry of a first device configured with a first one or more layers of a neural network trained to identity one or more features in input data, input data for processing via the neural network implemented across the processing circuitry of the first device and processing circuitry of a second device;

outputting, by the first one or more layers of the neural network implemented on processing circuitry of the first device, a data set that is reduced in size relative to a size of the input data while identifying one or more features of the input data for processing by a second one or more layers of the neural network;

determining that an accuracy associated with a first feature of the one or more features in the data set is outside a range of acceptable values;

determining that an accuracy associated with a second feature of the data set is within the range of acceptable values; and

communicating, by the processing circuitry of the first device responsive to the determination that the accuracy associated with the first feature is outside the range of acceptable values, a first portion of the data set that is reduced in size relative to the size of the input data and associated with the first feature to the second device for processing via the second one or more layers of the neural network implemented on the processing circuitry of the second device while performing an action on the second feature on first device and foregoing communicating to the second device a second portion of the data set associated with the second feature responsive to the determination that the accuracy associated with the second feature is within the range of acceptable values.

2. The method of claim 1 , further comprising reducing, by the first one or more layers, the data set by compressing the data set for transmission via a network to the processing circuitry of the second device.

3. The method of claim 1 , wherein the second one or more layers determine if a particular feature is one of the one or more features within the input data.

4. The method of claim 3 , further comprising receiving, by the processing circuitry of the first device, an indication from the processing circuitry of the second device that the particular feature was detected by the second one or more layers.

5. The method of claim 1 , further comprising detecting, by processing circuitry of the first device, that a particular one of the one or more features meets a threshold of accuracy to take an action by the processing circuitry of the first device.

6. The method of claim 5 , further comprising performing, by the processing circuitry of the first device responsive to the detection, the action with respect to the particular one of the one or more features.

7. The method of claim 6 , further comprising performing the action without communicating the data set to the processing circuitry of the second device.

8. A method comprising

receiving, by a processor of a wearable head display, input data captured by the wearable head display;

generating, by a first one or more layers of a neural network implemented on the processor and trained to identify one or more features in input data, a data set that is reduced in size relative to a size of the input data while identifying one or more features of the input data for processing by a second one or more layers of the neural network;

determining an accuracy associated with a first feature of the one or more features in the data set is outside a range of acceptable values; and

determining an accuracy associated with a second feature of the data set is within the range of acceptable values;

in response to the accuracy associated with the first feature of the one or more features in the data set is outside a range of acceptable values:

communicating, by the processor of the wearable head display, first portion of the data set that is reduced in size relative to the size of the input data and associated with the first feature to a second device for processing via the second one or more layers of the neural network implemented on processing circuitry of the second device;

in response to the accuracy associated with the second feature of the data set being within the range of acceptable values:

performing, by the processor, an action with respect to the second feature on the processor of the wearable head display instead of communicating the portion of the data set associated with the second feature to the second device.

9. The method of claim 8 , further comprising performing the action comprising modifying an image being displayed via the wearable head display.

10. The method of claim 8 , further comprises generating, by the first one or more layers implemented on the processor, the portion of the data set as a second data set that is reduced in size relative to a second input data while identifying a second one or more of features in the second input data.

11. The method of claim 10 , further comprises determining if an accuracy associated with a specific one of the second one or more features is outside a range of acceptable values.

12. The method of claim 11 , further comprising communicating, by the processor responsive to the determination, the second data set to the second device implementing the second one or more layers of the neural network.

13. The method of claim 12 , further comprising receiving, by the processor, from the second device an indication of a result of processing of the second data set by the second one or more layers.

14. A system comprising:

a first device comprising processing circuitry configured to receive input data for processing via a neural network implemented across the processing circuitry of the first device and processing circuitry of a second device, wherein the neural network is trained to identify one or more features within input data;

where a first one or more layers of the neural network implemented on the processing circuitry of the first device is configured to output a data set that is reduced in size relative to a size of the input data while identifying one or more features of the input data for processing by a second one or more layers of the neural network, responsive to a determination that an accuracy associated with a first feature of the one or more features in the data set is outside a range of acceptable values and a second feature of one or more feature in the data set is within the range of acceptable values; and

wherein the processing circuitry of the first device is configured to communicate a first portion of the data set that is reduced in size relative to the size of the input data and associated with the first feature to the processing circuitry of the second device for processing via the second one or more layers of the neural network implemented on the processing circuitry of the second device while performing an action on the second feature on the first device and foregoing communicating to the second device a second portion of the data set associated with the second feature.

15. The system of claim 14 , wherein the first one or more layers is further configured to reduce the data set by compressing the data set for transmission via a network to processing circuitry of the second device.

16. The system of claim 14 , wherein the second one or more layers is further configured to detect the specific one of the one or more features within the input data.

17. The system of claim 16 , wherein the first device is further configured to receive an indication from the second device that the specific one of the one or more features was detected by the second one or more layers.

18. The system of claim 14 , wherein the first device is further configured to perform, responsive to a determination that the accuracy associated with the specific one of the one or more features is within the range of acceptable values, an action with respect to the specific one of the one or more features without communicating the data set to the second device.

Assignments (2)
CHANGE OF NAME Recorded Jul 22, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060816/0634 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 11, 2019
From: LAI, LIANGZHEN; CHUANG, PIERCE I-JEN; CHANDRA, VIKAS; VENKATESH, GANESH
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 050693/0433 →