Methods and systems for dynamic control of an acoustic field
Disclosed are methods and systems for implementing dynamic control of acoustic fields, which involves the use of machine learning through a Markov Decision Process. The machine learning algorithm monitors an attribute of a system and maintains the attribute within a range by altering a signal sent to an acoustic transducer. The acoustic transducer is configured to alter the attribute of the system. The attribute may be an attribute of the acoustic field produced by the acoustic transducer, or may be an attribute of another element of the system, such as a fluid, a rotor, or a cell mass.
1 . A non-transitory computer readable medium having a computer program stored thereon for controlling an acoustic field to rotate a rotor, the computer program comprising instructions for causing one or more processors to:
transmit a signal to an acoustic transducer causing the acoustic transducer to generate an acoustic field for rotating a rotor, wherein the signal has a plurality of characteristics comprising amplitude, frequency and phase;
set an attribute of the rotor, wherein the attribute comprises a range of rotor speed; and
monitor the attribute of the rotor and execute a machine-learning algorithm to dynamically control the attribute of the rotor, wherein the machine-learning algorithm uses a Markov Decision Process to:
(i) measure the attribute of the rotor using a sensor upon altering the attribute of the rotor by an amount by adjusting at least a first characteristic of the plurality of characteristics of the signal;
(ii) calculate an output of the acoustic field associated with altering the attribute of the rotor by the amount;
(iii) calculate a resulting reward based on (a) altering the attribute of the rotor by an amount and (b) a resulting output of the acoustic field associated with altering the attribute of the rotor by the amount;
(iv) calculate a maximal reward value based upon an accumulation of the resulting reward of step (iii) and additional resulting rewards that are previously calculated;
(v) continuously alter, based on a control policy and the maximal reward value, the attribute of the rotor by (a) further adjusting the first characteristic or (b) adjusting a second characteristic of the plurality of characteristics of the signal that is different from the first characteristic; and
(vi) repeat steps (i) through (v) to maintain a stable rotor speed within the range.
2 . The non-transitory computer readable medium of claim 1 , further comprising updating the control policy based upon on whether the maximal reward value has been achieved.
3 . The non-transitory computer readable medium of claim 2 , wherein updating the control policy based upon on whether the maximal reward value has been achieved is performed between steps (iv) and (v) or between steps (v) and (vi).
4 . The non-transitory computer readable medium of claim 1 , wherein the attribute is altered by adjusting at least one of the plurality of characteristics comprising amplitude, frequency and phase.
5 . The non-transitory computer readable medium of claim 1 , wherein the output of the acoustic field is an amount of energy produced by the acoustic field.
6 . The non-transitory computer readable medium of claim 1 , wherein the acoustic field is configured to interact with a fluid in which the rotor is disposed.
7 . The non-transitory computer readable medium of claim 6 , wherein the output of the acoustic field is a temperature of the fluid.
8 . The non-transitory computer readable medium of claim 1 , wherein the computer program further comprises instructions for causing the one or more processors to calculate one or more parameters of the acoustic field upon transmitting the signal to the acoustic transducer.
9 . The non-transitory computer readable medium of claim 1 , wherein the machine-learning algorithm further uses the Markov Decision Process to:
confirm whether the attribute of the rotor remains within a first limit upon altering the attribute of the rotor by the amount; and
confirm whether the output of the acoustic field associated with altering the attribute of the rotor by the amount remains within a second limit upon altering the attribute of the rotor by the amount.
10 . A method for controlling a fluid characteristic within a medium through an acoustic field, the method comprising the steps of:
transmitting a signal to an acoustic transducer causing the acoustic transducer to generate an acoustic field to rotate a rotor, wherein the signal has a plurality of characteristics comprising amplitude, frequency and phrase phase;
calculating one or more parameters of the acoustic field upon transmitting the signal to the acoustic transducer;
setting a desired attribute of the rotor upon being subject to the acoustic field, wherein the desired attribute comprises a range of fluid speed; and
monitoring the attribute of the rotor and executing a machine-learning algorithm to dynamically control the attribute of the rotor, wherein the machine-learning algorithm uses a Markov Decision Process to:
(i) measure the attribute of the rotor using a sensor upon altering the attribute of the rotor by an amount by adjusting at least a first characteristic of the plurality of characteristics of the signal;
(ii) calculate an output of the acoustic field associated with altering the attribute of the rotor by the amount;
(iii) calculate a resulting reward based on (a) altering the attribute of the rotor by an amount and (b) a resulting output of the acoustic field associated with altering the attribute of the rotor by the amount;
(iv) calculate a maximal reward value based upon an accumulation of the resulting reward of step (iii) and additional resulting rewards that are previously calculated;
(v) continuously alter, based on a control policy and the maximal reward value, the attribute of the rotor by (a) further adjusting the first characteristic or (b) adjusting a second characteristic of the plurality of characteristics of the signal that is different from the first characteristic; and
(vi) repeat steps (i) through (v) to maintain a stable fluid speed within the range.
11 . The method of claim 10 , further comprising updating the control policy based upon on whether the maximal reward value has been achieved.
12 . The method of claim 11 , wherein updating the control policy based upon on whether the maximal reward value has been achieved is performed between steps (iv) and (v) or between steps (v) and (vi).
13 . A system for controlling the behavior of a cell mass, the system comprising:
a medium in which the cell mass is immersed;
a rotor configured to alter a characteristic of the medium;
an acoustic transducer configured to alter the characteristic of the medium through at least the rotor;
a sensor configured to measure the characteristic; and
a non-transitory computer readable medium having a computer program stored thereon, the computer program comprising instructions for causing one or more processors to:
transmit a signal to the acoustic transducer, wherein the acoustic transducer is configured to generate an acoustic field to rotate a rotor, wherein the signal has a plurality of characteristics comprising amplitude, frequency and phase;
set an attribute of the rotor upon being subject to the acoustic field, wherein the attribute comprises a range of cell mass rotation speed; and
monitor the attribute of the rotor and execute a machine-learning algorithm to dynamically control the attribute of the rotor, wherein the machine-learning algorithm uses a Markov Decision Process to:
(i) measure the attribute of the rotor using a sensor upon altering the attribute of the rotor by an amount by adjusting at least a first characteristic of the plurality of characteristics of the signal;
(ii) calculate an output of the acoustic field associated with altering the attribute of the rotor by the amount;
(iii) calculate a resulting reward based on (a) altering the attribute of the rotor by an amount and (b) a resulting output of the acoustic field associated with altering the attribute of the rotor by the amount;
(iv) calculate a maximal reward value based upon an accumulation of the resulting reward of step (iii) and additional resulting rewards that are previously calculated;
(v) continuously alter, based on a control policy and the maximal reward value, the attribute of the rotor by (a) further adjusting the first characteristic or (b) adjusting a second characteristic another one of the plurality of characteristics of the signal that is different from the first characteristic; and
(vi) repeat steps (i) through (v) while maintaining a stable cell mass rotation speed within the range.
14 . The system of claim 13 , the non-transitory computer readable medium further configured to update the control policy based upon on whether the maximal reward value has been achieved.
15 . The system of claim 14 , wherein updating the control policy based upon on whether the maximal reward value has been achieved is performed between steps (iv) and (v) or between steps (v) and (vi).
16 . The system of claim 13 , wherein the medium is a fluid.
17 . The system of claim 16 , wherein the characteristic is altered by altering the attribute of the rotor by the amount.
18 . The system of claim 13 , wherein the cell mass is supported by a scaffold structure coupled to the rotor.
19 . A non-transitory computer readable medium having a computer program stored thereon for controlling an acoustic field, the computer program comprising instructions for causing one or more processors to:
transmit a signal to an acoustic transducer, wherein the acoustic transducer is configured to generate an acoustic field, wherein the signal has a plurality of characteristics comprising amplitude, frequency and phase;
set at least two attributes of the acoustic field, wherein the at least two attributes comprise corresponding ranges of acoustic output; and
monitor the two or more attributes of the acoustic field and execute a machine-learning algorithm to dynamically control one of the at least two attributes of the acoustic field, wherein the machine-learning algorithm uses a Markov Decision Process to:
(i) measure the attribute of the one of the at least two attributes of the acoustic field using a sensor upon altering the one of the at least two attributes of the acoustic field by an amount;
(ii) calculate an output of the other of the at least two attributes of the acoustic field associated with altering the one of the at least two attributes of the acoustic field by the amount;
(iii) calculate a resulting reward based on (a) altering the one of the at least two attributes of the acoustic field by the amount and (b) the output of the other of the at least two attributes of the acoustic field associated with altering the one of the at least two attributes of the acoustic field by the amount;
(iv) calculate a maximal reward value based upon an accumulation of the resulting reward of step (iii) and additional resulting rewards that are previously calculated;
(v) continuously alter, based on a control policy and the maximal reward value, the one of the at least two attributes by adjusting at least one of the plurality of characteristics of the signal; and
(vi) repeat steps (i) through (v) to maintain a stable acoustic output such that the two or more attributes of the acoustic field remain within the corresponding ranges.
20 . The system of claim 19 , further comprising updating the control policy based upon on whether the maximal reward value has been achieved.
21 . The system of claim 20 , wherein updating the control policy based upon on whether the maximal reward value has been achieved is performed between steps (iv) and (v) or between steps (v) and (vi).
22 . The non-transitory computer readable medium of claim 19 , wherein the machine-learning algorithm comprises a neural network.
23 . The non-transitory computer readable medium of claim 19 , wherein the machine-learning algorithm comprises fuzzy logic.