System and method for multi-object micro-assembly control with the aid of a digital computer
System and method that allow to jointly cause movement of multiple micro-and-nano-objects to desired positions are described. A high speed camera tracks the locations of the objects. An array of photo-transistor-controlled electrodes is used to generate a dynamic potential energy landscape for manipulating objects with both DEP and EP forces, and a video projector is used actuate the array. One or more computing devices are used to: process images captured by the camera to estimate positions of the objects; generate desired trajectories of the objects using an objective function; compare the desired chiplet positions with current positions and generate input signals to minimize the error between them; and map the control inputs to images that are projected on the array using a video project. The projected images activate or deactivate electrodes, as indicated by the control inputs.
1. A system for multi-object micro-assembly control with the aid of a digital computer, comprising:
a plurality of processors configured to execute computer-executable code, the processors configured to:
obtain one or more parameters of a system for positioning a plurality of chiplets, each of the chiplets comprising a micro-object, the system comprising a plurality of electrodes, each of the electrodes controlled by a photo-transistor, the electrodes configured to induce a movement of the chiplets when the chiplets are suspended in a fluid proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
model capacitance between the chiplets and the electrodes and capacitance between the chiplets based on the parameters of the system;
estimate positions of the chiplets based on images taken by at least one camera;
receive further positions of at least some of the chiplets;
perform model predictive control (MPC) optimization to derive based on the capacitance modeling a control scheme for moving the at least some chiplets from the positions to the further positions and potentials of the electrode potentials necessary for the at least some chiplets to travel along the trajectories, wherein the trajectories are parametrized as smooth, time-dependent functions during the MPC optimization and the electrode potentials are parametrized as smooth time-and-space-dependent functions during the MPC optimization, wherein the processors work in parallel to perform the MPC optimization, wherein the trajectory and the potentials are given a neural network representation, and wherein the MPC optimization is performed using automatic differentiation;
map the electrode potentials in the control scheme to one or more further images;
control a video projector to project the one or more further images to the photo-transistors, wherein the phototransistors control the electrodes to generate the electrode potentials in the control scheme based on the projected images.
2. A system according to claim 1 , wherein the automatic differentiation is performed using a sequential quadratic programming (SQP) algorithm.
3. A system according to claim 1 , wherein the processors comprise at least one of one or more graphics processing units (GPUs) and one or more of tensor processing units (TPUs).
4. A system according to claim 1 , the processors further configured to:
set system dynamics constraints, wherein the trajectories are determined in accordance with the system dynamics constraints.
5. A system according to claim 1 , the processors further configured to:
perform a plurality of simulations of the capacitance between the electrodes and the micro-objects; and
define a function describing the capacitance between each of the micro-objects and each of the electrodes as a function of a distance between that micro-object and that electrode.
6. A system according to claim 5 , the processors further configured to:
perform a plurality of simulations of the capacitance between the micro-objects; and
define a function describing the capacitance between each of the micro-objects and at least another one of the micro-objects as a function of the distance between those micro-objects.
7. A system according to claim 6 , wherein each of the functions are defined using at least one error function.
8. A system according to claim 1 , wherein at least one of the chiplets remains at that chiplet's position while the at least some chiplets are moved to the desired positions of those chiplets.
9. A system according to claim 1 , wherein at least some of the electrodes are rectangular.
10. A method for multi-object micro-assembly control with the aid of a digital computer, comprising:
obtaining by one or more of a plurality of processors configured to execute computer-executable code one or more parameters of a system for positioning a plurality of chiplets, each of the chiplets comprising a micro-object, the system comprising a plurality of electrodes, each of the electrodes controlled by a photo-transistor, the electrodes configured to induce a movement of the chiplets when the chiplets are suspended in a fluid proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
modeling by one or more of the plurality of processors capacitance between the chiplets and the electrodes and capacitance between the chiplets based on the parameters of the system;
estimating by one or more of the plurality of processors positions of the chiplets based on images taken by at least one camera;
receiving by one or more of the plurality of processors further positions of at least some of the chiplets;
performing by the plurality of processors model predictive control (MPC) optimization to derive based on the capacitance modeling a control scheme for moving the at least some chiplets from the positions to the further positions and potentials of the electrode potentials necessary for the at least some chiplets to travel along the trajectories, wherein the trajectories are parametrized as smooth, time-dependent functions during the MPC optimization and the electrode potentials are parametrized as smooth time-and-space-dependent functions during the MPC optimization, wherein the processors work in parallel to perform the MPC optimization, wherein the trajectory and the potentials are given a neural network representation, and wherein the MPC optimization is performed using automatic differentiation;
mapping by one or more of the plurality of processors the electrode potentials in the control scheme to one or more further images;
controlling by one or more of the plurality of processors a video projector to project the one or more further images to the photo-transistors, wherein the phototransistors control the electrodes to generate the electrode potentials in the control scheme based on the projected images.
11. A method according to claim 10 , wherein the automatic differentiation is performed using a sequential quadratic programming (SQP) algorithm.
12. A method according to claim 10 , wherein the processors comprise at least one of one or more graphics processing units (GPUs) and one or more of tensor processing units (TPUs).
13. A method according to claim 10 , further comprising:
setting system dynamics constraints, wherein the trajectories are determined in accordance with the system dynamics constraints.
14. A method according to claim 10 , further comprising:
performing a plurality of simulations of the capacitance between the electrodes and the micro-objects; and
defining a function describing the capacitance between each of the micro-objects and each of the electrodes as a function of a distance between that micro-object and that electrode.
15. A method according to claim 14 , further comprising:
performing a plurality of simulations of the capacitance between the micro-objects; and
defining a function describing the capacitance between each of the micro-objects and at least another one of the micro-objects as a function of the distance between those micro-objects.
16. A method according to claim 15 , wherein each of the functions are defined using at least one error function.
17. A method according to claim 10 , wherein at least one of the chiplets remains at that chiplet's position while the at least some chiplets are moved to the desired positions of those chiplets.
18. A method according to claim 10 , wherein at least some of the electrodes are rectangular.