IP Library Granted Patent US 10,558,204
Granted Patent B2
US 10,558,204 · App. 15/469,433 · Granted Feb 11, 2020

System and method for scalable real-time micro-object position control with the aid of a digital computer

Inventors: Ion Matei (Sunnyvale, CA); Jeng Ping Lu (Fremont, CA); Saigopal Nelaturi (Mountain View, CA); Julie A. Bert (East Palo Alto, CA); Lara S. Crawford (Belmont, CA); Armin R. Volkel (Mountain View, CA); Eugene M. Chow (Palo Alto, CA)
Assignee: Palo Alto Research Center Incorporated
G05B19/41885G05B17/02G06T7/155G06T7/194G06T7/74H04N7/183G05B2219/32359G06T2207/20021G06T2207/30164
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,558,204
App. No.
15/469,433
Granted
Feb 11, 2020
Kind
B2
Abstract

The system and method described below allow for real-time control over positioning of a micro-object. A movement of at least one micro-object suspended in a medium can be induced by a generation of one or more forces by electrodes proximate to the micro-object. Prior to inducing the movement, a simulation is used to develop a model describing a parameter of an interaction between each of the electrodes and the micro-object. A function describing the forces generated by an electrode and an extent of the movement induced due to the forces is generated using the model. The function is used to design closed loop policy control scheme for moving the micro-object towards a desired position. The position of the micro-object is tracked and taken into account when generating control signals in the scheme.

Claims (62)

1. A system for scalable real-time micro-object position control with the aid of a digital computer, comprising:

at least one processor configured to execute computer-executable code and further configured to:

obtain one or more parameters of a system for positioning of at least one micro-object, the system comprising a plurality of electrodes, the electrodes configured to induce a movement of the micro-object when the micro-object is suspended in a fluid proximate to the electrodes upon a generation of one or more forces by the electrodes;

model a capacitance of an interaction between each of the electrodes and the micro-object as a function of a distance between that electrode and the micro-object using the system parameters;

obtain a position of the micro-object within the fluid;

obtain using the modeled capacitance a further function describing a relationship between the forces generated by one of the electrodes for a predefined period of time and an extent of the movement induced by the forces generated by the one electrode, comprising:

map a plurality of values for the one or more forces generated by the one electrode to the extent of the movement of the micro-object induced by the generation of each of the voltage values; and

derive the further function from the mappings; and

generate a control signal for each of the electrodes, the control signal comprising a command to generate one or more of the forces for the predefined period of time by that electrode, using the obtained position, the further function, and a desired location of the micro-object.

2. A system according to claim 1 , wherein the electrodes are spiral electrodes and are comprised in an electrode unit.

3. A system according to claim 1 , wherein the electrodes are periodically spaced.

4. A system according to claim 1 , the at least one processor further configured to:

perform a plurality of simulations of the interaction parameters at a plurality of values of the distance, wherein the modeling is done based on the simulations.

5. A system according to claim 1 , the at least one processor further configured to:

apply the control signals to the electrodes, wherein the electrodes generate the one or more forces commanded by the control signals upon the application and induce the movement of the micro-object;

identify a further position of the micro-object following the application of the control signals to the electrodes; and

generate further control signals for the electrodes using the interaction parameter, the further position of the micro-object, and the desired position of the micro-object following the application.

6. A system according to claim 5 , the at least one processor further configured to:

receive raw data captured by a high-speed, high-resolution camera monitoring the micro-object, the raw data having been captured before and after the application of the control signals;

convert the raw data into a plurality of images using a frame grabber;

perform background extraction on the images converted from the raw data captured before the application of the control signals;

identify foreground components in at least one of the images converted from the raw data captured after the application of the signal using a result of the background extraction;

identify the foreground components whose size is greater than or equals to a predefined size; and

perform a template matching algorithm in a neighborhood associated with the foreground components to identify the position of the micro-object.

7. A system according to claim 5 , the at least one processor further configured to:

apply a morphological opening to the foreground components to identify the foreground components whose size is greater than or equal to the predefined size.

8. A system according to claim 5 , the at least one processor further configured to:

define a window around the identified position,

wherein the template matching algorithm is run in a neighborhood defined by the window to determine a subsequent position of the micro-object.

9. A system according to claim 1 , the at least one processor further configured to:

perform a plurality of simulations to obtain the mappings.

10. A method for scalable real-time micro-object position control with the aid of a digital computer, comprising the steps of:

obtaining one or more parameters of a system for positioning of at least one micro-object, the system comprising a plurality of electrodes, the electrodes configured to induce a movement of the micro-object when the micro-object is suspended in a fluid proximate to the electrodes upon a generation of one or more forces by the electrodes;

modeling a capacitance of an interaction between each of the electrodes and the micro-object as a function of a distance between a location of that electrode and a location of the micro-object using the system parameters;

obtaining a position of the micro-object within the fluid;

obtaining using the modeled capacitance a further function describing a relationship between the forces generated at one of the electrodes for a predefined period of time and an extent of the movement induced by the forces generated by the one electrode, comprising:

mapping a plurality of values for the one or more forces generated by the one electrode to the extent of the movement of the micro-object induced by the generation of each of the voltage values; and

deriving the further function from the mappings; and

generating a control signal for each of the electrodes, the control signal comprising a command to generate one or more of the forces for the predefined period of time by that electrode, using the obtained position, the further function, and a desired location of the micro-object,

wherein the steps are performed by a computer.

11. A method according to claim 10 , wherein the electrodes are spiral electrodes and are comprised in an electrode unit.

12. A method according to claim 10 , wherein the electrodes are periodically spaced.

13. A method according to claim 10 , further comprising:

performing a plurality of simulations of the interaction parameters at a plurality of values of the distance, wherein the modeling is done based on the simulations.

14. A method according to claim 10 , further comprising:

applying the control signals to the electrodes, wherein the electrodes generate the one or more forces commanded by the control signals upon the application and induce the movement of the micro-object;

identifying a further position of the micro-object following the application of the control signals to the electrodes; and

generating further control signals for the electrodes using the interaction parameter, the further position of the micro-object, and the desired position of the micro-object following the application.

15. A method according to claim 14 , further comprising:

receiving raw data captured by a high-speed, high-resolution camera monitoring the micro-object, the raw data having been captured before and after the application of the control signals;

converting the raw data into a plurality of images using a frame grabber;

performing background extraction on the images converted from the raw data captured before the application of the control signals;

identifying foreground components in at least one of the images converted from the raw data captured after the application of the signal using a result of the background extraction;

identifying the foreground components whose size is greater than or equals to a predefined size; and

performing a template matching algorithm in a neighborhood associated with the foreground components to identify the position of the micro-object.

16. A method according to claim 15 , further comprising:

applying a morphological opening to the foreground components to identify the foreground components whose size is greater than or equal to the predefined size.

17. A method according to claim 15 , further comprising:

defining a window around the identified position,

wherein the template matching algorithm is run in a neighborhood defined by the window to determine a subsequent position of the micro-object.

18. A method according to claim 10 , comprising:

performing a plurality of simulations to obtain the mappings.

Assignments (10)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2025
From: XEROX CORPORATION
To: GENESEE VALLEY INNOVATIONS, LLC
Reel/Frame 073562/0677 →
SECOND LIEN NOTES PATENT SECURITY AGREEMENT Recorded Jul 2, 2025
From: XEROX CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 071785/0550 →
FIRST LIEN NOTES PATENT SECURITY AGREEMENT Recorded Apr 11, 2025
From: XEROX CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 070824/0001 →
SECURITY INTEREST Recorded Feb 13, 2024
From: XEROX CORPORATION
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 066741/0001 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT RF 064760/0389 Recorded Feb 13, 2024
From: CITIBANK, N.A., AS COLLATERAL AGENT
To: XEROX CORPORATION
Reel/Frame 068261/0001 →
SECURITY INTEREST Recorded Nov 20, 2023
From: XEROX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 065628/0019 →
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVAL OF US PATENTS 9356603, 10026651, 10626048 AND INCLUSION OF US PATENT 7167871 PREVIOUSLY RECORDED ON REEL 064038 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 28, 2023
From: PALO ALTO RESEARCH CENTER INCORPORATED
To: XEROX CORPORATION
Reel/Frame 064161/0001 →
SECURITY INTEREST Recorded Jun 22, 2023
From: XEROX CORPORATION
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 064760/0389 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2023
From: PALO ALTO RESEARCH CENTER INCORPORATED
To: XEROX CORPORATION
Reel/Frame 064038/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2017
From: MATEI, ION; LU, JENG PING; NELATURI, SAIGOPAL; BERT, JULIE A.; CRAWFORD, LARA S.; VOLKEL, ARMIN R.; CHOW, EUGENE M.
To: PALO ALTO RESEARCH CENTER INCORPORATED
Reel/Frame 041787/0498 →