Data processing architecture for improved data flow
Disclosed are methods, apparatus and systems for improving data management and workload distribution in pattern recognition systems. An example method of managing data for a sign language translation system includes receiving multiple sets of data acquired by one or more data acquisition devices. Each set of data including an image frame that illustrates at least a part of a gesture. The method includes determining, for each of the multiple sets of data, a plurality of attribute values defined by a customized template. The method includes accessing the multiple sets of data, by a plurality of processing units, based on a location indicated by the attributes for recognizing the at least a part of a gesture.
1. An apparatus in a sign language processing system, comprising:
a first processing unit and a second processing unit, and
a memory including instructions stored thereupon, the instructions upon execution by the first processing unit causing the first processing unit to:
receive, by a first thread of a first processing unit, a set of data captured by a capture device, the set of data including an image frame that illustrates a gesture representing a letter, a word, or a phrase in a sign language;
eliminate, by the first thread of the first processing unit, background information in the image frame to obtain one or more areas of interest;
prepare, by a second thread of the first processing unit concurrently as the set of data is preprocessed, a set of resources for a gesture recognition operation;
invoke, by the second thread of the first processing unit, a first neural network to be executed on a second processing unit to carry out the gesture recognition operation on the one or more areas of interest using the set of resources; and
receive, by the first thread of the first processing unit, a subsequent set of data captured by the capture device, the subsequent set of data received concurrently as the gesture recognition operation is being performed,
wherein the instructions upon execution by the first or the second processing unit cause the first or the second processing unit to:
determine a performance result of the gesture recognition operation; and
dynamically adjust the set of resources for subsequent processing of data from the capture device to improve the performance result that includes a utilization rate of the first or the second processing unit, and
wherein the instructions upon execution by the second processing unit cause the second processing unit to dynamically adjust the set of resources using a second neural network by performing a machine learning procedure based on the performance result.
2. The apparatus of claim 1 , comprising a third processing unit, and wherein the instructions upon execution by the first processing unit cause the first processing unit to:
receive, by a fourth thread of the first processing unit, a second set of data obtained from a second capture device, the second set of data including a second image frame that illustrates the gesture;
eliminate, by the fourth thread of the first processing unit, background information in the second image frame to obtain one or more areas of interest in the second image frame;
prepare, by a fifth thread of the first processing unit concurrently as the second set of data is preprocessed, a second set of resources for the gesture recognition operation;
invoke, by the fifth thread of the first processing unit, a third neural network to be executed on the third processing unit to carry out the gesture recognition operation on the one or more areas of interest in the second image frame using the second set of resources; and
receive, by the fourth thread of the first processing unit, a subsequent set of data captured by the second capture device concurrently as the gesture recognition operation is being performed.
3. The apparatus of claim 2 , wherein the second processing unit is same as the third processing unit.
4. A method for improving computational efficiency of a computer system for use in a sign language translation system, comprising:
receiving, by a first thread of a first processing unit, a set of data captured by a capture device, the set of data including an image frame that illustrates a gesture representing a letter, a word, or a phrase in a sign language;
eliminating, by the first thread of the first processing unit, background information from the image frame to obtain one or more areas of interest;
preparing, by a second thread of the first processing unit concurrently as the set of data is preprocessed, a set of resources for a gesture recognition operation;
invoking, by the second thread of the first processing unit, a first neural network to be executed on a second processing unit to carry out the gesture recognition operation on the one or more areas of interest using the set of resources;
receiving, by the first thread of the first processing unit, a subsequent set of data captured by the capture device concurrently as the gesture recognition operation is being performed;
determining a performance result of the gesture recognition operation, the performance result including a utilization rate of the computer system; and
dynamically adjusting the set of resources for subsequent processing of data from the capture device to improve the performance result, wherein the set of resources is dynamically adjusted by a second neural network performing a machine learning procedure based on the performance result.
5. The method of claim 4 , comprising:
receiving, by a fourth thread of the first processing unit, a second set of data obtained from a second capture device, the second set of data including a second image frame that illustrates the gesture;
eliminating, by the fourth thread of the first processing unit, background information in the second image frame to obtain one or more areas of interest in the second image frame;
preparing, by a fifth thread of the first processing unit concurrently as the fourth thread obtains the second set of data, a second set of resources for the gesture recognition operation;
invoking, by the fifth thread of the first processing unit, a third neural network to be executed on a third processing unit to carry out the gesture recognition operation on the one or more areas of interest in the second image frame using the second set of resources; and
receiving, by the fourth thread of the first processing unit, a subsequent set of data captured by the second capture device concurrently as the gesture recognition operation is being performed.
6. The method of claim 5 , wherein the second processing unit is same as the third processing unit.