IP Library Granted Patent US 12,242,169
Granted Patent B2
US 12,242,169 · App. 18/128,532 · Granted Mar 4, 2025

High-speed autofocus control

Inventors: Peter DiMeo (Boston, MA); Jingyang Yan (Boston, MA); Xian Du (Boston, MA)
Assignee: UNIVERSITY OF MASSACHUSETTS
G03B13/36G03B3/10H04N23/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,242,169
App. No.
18/128,532
Granted
Mar 4, 2025
Kind
B2
Abstract

Various examples are provided related to high-speed autofocus control. In one example, a method includes obtaining a first image of a target with a camera; adjusting focus of the camera by a specified AF bin step size; obtaining a second image of the target; adjusting focus of the camera by the specified AF bin step size; obtaining a third image of the target; determining an optimal focus using data of the second and third images; and adjusting focus of the camera to the optimal focus. In another example, a method includes generating an input vector comprising a sequence of input-output pairs associated with a piezoelectric controlled motion stage that can position a camera relative to a target; determine a control input for the motion stage using a LSTM backpropagation network trained to minimize a cost function over a defined prediction horizon; and applying the control input to the motion stage.

Claims (30)

1. A method for autofocusing, comprising:

obtaining a first image of a target with a camera;

adjusting focus of the camera by a specified autofocus (AF) bin step size;

obtaining a second image of the target with the camera;

adjusting focus of the camera by the specified AF bin step size;

obtaining a third image of the target with the camera;

determining an optimal focus based at least in part upon focus measure (FM) data of the second and third images, the optimal focus determined using a direct Gaussian mean calculation; and

adjusting focus of the camera to the optimal focus.

2. The method of claim 1 , wherein the FM data comprises Gaussian derivative FM scores.

3. The method of claim 2 , wherein the optimal focus is determined in response to the FM score of the third image being less than or equal to the FM score of the second image, wherein the optimal focus is based at least in part upon FM data of the first, second and third images.

4. The method of claim 2 , comprising:

adjusting focus of the camera by the specified AF bin step size; and

obtaining a fourth image of the target with the camera in response to the FM score of the third image being less than the FM score of the second image, wherein the optimal focus is based at least in part upon FM data of the second, third and fourth images.

5. The method of claim 1 , wherein the specified AF bin step size is determined based at least in part upon a defocus gradient of a Gaussian FM curve associated with the camera.

6. The method of claim 5 , wherein the specified AF bin step size is selected based upon standard deviation of the Gaussian FM curve.

7. The method of claim 1 , comprising obtaining an image of the target with the camera at the optimal focus.

8. The method of claim 7 , wherein the target is an item or component for inspection.

9. The method of claim 7 , wherein the method for autofocus is initiated in response to detecting the target.

10. A method for autofocus, comprising:

generating an input vector comprising a sequence of input-output pairs associated with a piezoelectric controlled motion stage configured to position a camera relative to a target;

determine a control input for the piezoelectric controlled motion stage using a long short-term memory (LSTM) backpropagation network trained to minimize a cost function over a defined prediction horizon; and

applying the control input to the piezoelectric controlled motion stage, the control input shaped by an S-curve control input profile.

11. The method of claim 10 , wherein the piezoelectric controlled motion stage is a piezoelectric motion stage.

12. The method of claim 10 , wherein the piezoelectric controlled motion stage comprises piezoelectric actuators.

13. The method of claim 10 , wherein the input vector comprises position information of the piezoelectric controlled motion stage and focus scores of images captured by the camera at corresponding positions.

14. The method of claim 10 , wherein the cost function comprises a squared error between a reference signal and a predicted output of the LSTM backpropagation network and a weighted squared change of the control input.

15. The method of claim 10 , wherein the LSTM backpropagation network is trained using a backpropagation through time (BPTT) algorithm comprising pre-allocation for expedited gradient calculation.

16. The method of claim 15 , wherein the BPTT algorithm comprises calculating derivatives of predicted focus measurement to generated intermediate control input by applying a chain rule through time.

17. The method of claim 10 , comprising training the LSTM backpropagation network with training data generated using a pulse-train method applied to the piezoelectric controlled motion stage.

18. The method of claim 17 , wherein the pulse-train method comprises applying a series of pulses with random amplitudes to capture nonlinear dynamics of the piezoelectric controlled motion stage.

Assignments (2)
CONFIRMATORY LICENSE Recorded Feb 19, 2025
From: UNIVERSITY OF MASSACHUSETTS AMHERST
To: NATIONAL SCIENCE FOUNDATION
Reel/Frame 070253/0745 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2023
From: DIMEO, PETER; YAN, JINGYANG; DU, XIAN
To: UNIVERSITY OF MASSACHUSETTS
Reel/Frame 063508/0849 →
Continuity (2)
Provisional Application 63326443 · Apr 1, 2022
Related Publication 20230314909A1 · Oct 5, 2023
References Cited (7)
US 20180060665A1 · Song · 2018 [cited by examiner]
S. Xie and J. Ren, “Recurrent-Neural-Network-Based Predictive Control of Piezo Actuators for Trajectory Tracking,” IEEE/ASME Transactions on Mechatronics, vol. 24, No. 6, pp. 2885-2896, Dec. 2019, doi: 10.1109/TMECH.201… [cited by applicant]
A. Shajkofci and M. Liebling, “DeepFocus: a Few-Shot Microscope Slide Auto-Focus using a Sample Invariant CNN-based Sharpness Function,” arXiv:2001.00667 [cs, eess] (2020). [cited by applicant]
W. Liu, L. Cheng, C. Zhou, Z. Hou, and M. Tan, “Neural-network based model predictive control for piezoelectric-actuated stick-slip micro-positioning devices,” in 2016 IEEE International Conference on Advanced Intellige… [cited by applicant]
Yan, et al., “LSTM-based Model Predictive Control of Piezoelectric Motion Stages for High-Speed Autofocus”, IEEE Transaction on Industrial Electronics, vol. 70, No. 6, Jun. 2023. [cited by applicant]
DiMeo, et al., “Fast and accurate autofocus control using Gaussian standard deviation and gradient-based binning”, Optics Express, vol. 29, No. 13, Jun. 21, 2021. [cited by applicant]
Cheng, et al., “Neural-Network-Based Nonlinear Model Predictive Control for Piezoelectric Actuators”, IEEE Transactions on Industrial Electronics, vol. 62, No. 12, Dec. 2015. [cited by applicant]