IP Library Granted Patent US 50,624
Granted Patent E1
US 50,624 · App. 18/891,778 · Granted Oct 7, 2025

Method and apparatus for super resolution imaging and eye tracking devices

Inventor: Masoud Vaziri (Richardson, TX)
Assignee: Optics Innovation LLC
G06T3/4053G02B27/0093G06T3/4084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 50,624
App. No.
18/891,778
Granted
Oct 7, 2025
Kind
E1
Abstract

An eyewear camera as a system is introduced that includes many subsystems. These subsystems include: scene imaging, control methods, tracking a user's eye, methods and techniques to increase the resolution of an image captured by a scene camera, methods to create a viewfinder, and methods to capture an image of a user's eye while simultaneously projecting an image into the user's eye. Such an eyewear will allow a user to capture a scene effortlessly, select an object within the scene, get extra information about the object via a digital personal assistant, or even modify a subset of the scene. Specific applications will be discussed that include visual aid for people with low vision and upgrading existing security cameras via proposed single camera super resolution techniques.

Claims (37)

1. A security imaging system comprising:

a camera for capturing at least one image of a scene wherein the scene includes a poster of at least one predetermined resolution test pattern;

an electronic storage unit for storing at least one image of the at least one predetermined resolution test pattern; and

at least one processor, in communication with the camera and the electronic storage unit, the at least one processor configured to:

receive the at least one image of the scene from the camera, and

initiate, upon detecting a motion in the scene based upon the at least one image of the scene, an iterative bandwidth extrapolation procedure to increase the image resolution of a subset of the least one image of the scene based upon the at least one image of the scene and the at least one image of the at least one predetermined resolution test pattern wherein the one subset of the at least one image of the scene has no overlap with the image of the poster in the at least one image of the scene.

2. The security imaging system of claim 1 further comprising an enclosure wherein the camera, the electronic storage unit and the at least one processor are disposed on or in the enclosure.

3. The security imaging system of claim 1 wherein the camera resides in a first enclosure and the at least one processor resides in a second enclosure.

4. The security imaging system of claim 3 wherein the camera is in communication with the at least one processor via at least one wireless link.

5. The security imaging system of claim 4 wherein the at least one wireless link is provided by a wireless router.

6. The security imaging system of claim 5 wherein the wireless router is a mesh wireless router.

7. The security imaging system of claim 1 wherein the at least one processor is further configured to perform face recognition and object recognition in the at least one image of the scene.

8. The security imaging system of claim 1 further comprising a second camera for capturing a second image of the scene.

9. The security imaging system of claim 8 wherein the first and the second cameras have rectangular pixels and the ratio of the length to the width of each pixel is equal or larger than 2.

10. The security imaging system of claim 1 wherein the at least one processor is further configured to:

recognize a face in the at least one image of the scene,

maintain a history of the at least one recognized face, and

take an action when a new face is detected.

11. The security imaging system of claim 1 wherein the camera is a CCTV camera.

12. An imaging method comprising:

capturing, via a camera, at least one image of a scene wherein the scene includes a poster of at least one predetermined resolution test pattern;

storing at least one image of the at least one predetermined resolution test pattern; and

initiating, via at least one processor and upon detecting a motion in the scene, an iterative image resolution enhancement procedure to increase the image resolution of a subset of the least one image of the scene based upon the at least one image of the scene and the at least one image of the at least one predetermined resolution test pattern wherein the one subset of the at least one image of the scene has no overlap with the image of the poster in the at least one image of the scene.

13. The imaging method of claim 12 wherein the iterative image resolution enhancement procedure includes at least a bandwidth extrapolation technique.

14. The imaging method of claim 12 further comprising:

initiating a face recognition procedure based upon the at least one image of the scene,

maintaining a history of the at least one recognized face, and

taking an action when a new face is detected.

15. The imaging method of claim 12 wherein the camera resides in a first enclosure and the at least one processor resides in a second enclosure and the camera and the at least one processor are in communication via a wireless link.

16. The imaging method of claim 12 wherein the camera is a CCTV camera.

17. A security imaging system comprising:

a CCTV camera for capturing at least one image of a scene wherein the scene includes a poster of at least one predetermined resolution test pattern;

an electronic storage unit for storing at least one image of the at least one predetermined resolution test pattern; and

at least one processor in communication with the CCTV camera and the electronic storage unit, the at least one processor configured to:

receive the at least one image of the scene from the CCTV camera,

initiate, upon detecting a motion in the scene based upon the at least one image of the scene, an iterative bandwidth extrapolation procedure to increase the image resolution of a subset of the least one image of the scene based upon the at least one image of the scene and the at least one image of the at least one predetermined resolution test pattern wherein the one subset of the at least one image of the scene has no overlap with the image of the poster in the at least one image of the scene.

18. The security imaging system of claim 15 wherein the iterative bandwidth extrapolation procedure uses at least a portion of the spatial frequency spectra of the at least one image of the at least one predetermined resolution test pattern in order to increase the image resolution of the subset of the at least one image of the scene.

Assignments (5)
NUNC PRO TUNC ASSIGNMENT Recorded Aug 20, 2025
From: VAZIRI, MASOUD
To: U & LIFE CORPORATION
Reel/Frame 072073/0653 →
MERGER Recorded Aug 20, 2025
From: U & LIFE CORPORATION
To: I2I, INC.
Reel/Frame 072073/0834 →
MERGER Recorded Aug 20, 2025
From: U & LIFE CORPORATION
To: I2I, INC.
Reel/Frame 072074/0011 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2025
From: IPAL INC.
To: OPTICS INNOVATION LLC
Reel/Frame 072074/0331 →
CHANGE OF NAME Recorded Aug 20, 2025
From: I2I, INC.
To: IPAL INC.
Reel/Frame 072554/0403 →
Continuity (3)
Provisional Application 62881961 · Aug 2, 2019
Provisional Application 62769438 · Nov 19, 2018
Reissue 16687675 · Nov 18, 2019
References Cited (160)
US 4028725A · Lewis · 1977 [cited by applicant]
US 4907296A · Blecha · 1990 [cited by applicant]
US 5262871A · Wilder et al. · 1993 [cited by applicant]
US 5856811A · Shih · 1999 [cited by applicant]
US 5859921A · Suzuki · 1999 [cited by applicant]
US 6163336A · Richards · 2000 [cited by applicant]
US 6198485B1 · Mack et al. · 2001 [cited by applicant]
US 6285799B1 · Dance · 2001 [cited by examiner]
US 6307526B1 · Mann · 2001 [cited by applicant]
US 6434280B1 · Peleg · 2002 [cited by applicant]
US 6486799B1 · Still et al. · 2002 [cited by applicant]
US 6661495B1 · Popovich · 2003 [cited by applicant]
US 6766067B2 · Freeman · 2004 [cited by applicant]
US 6850629B2 · Jeon · 2005 [cited by applicant]
US 7023464B1 · Harada et al. · 2006 [cited by applicant]
US 7331671B2 · Hammond · 2008 [cited by applicant]
US 7391887B2 · Durnell · 2008 [cited by applicant]
US 7492926B2 · Kang · 2009 [cited by applicant]
US 7697024B2 · Currivan et al. · 2010 [cited by applicant]
US 7715658B2 · Cho · 2010 [cited by applicant]
US 7894666B2 · Mitarai et al. · 2011 [cited by applicant]
US 8014632B2 · Matsumoto · 2011 [cited by applicant]
US 8139089B2 · Doyle et al. · 2012 [cited by applicant]
US 8159519B2 · Kurtz · 2012 [cited by examiner]
US 8305899B2 · Luo et al. · 2012 [cited by applicant]
US 8432492B2 · Deigmoeller et al. · 2013 [cited by applicant]
US 8666196B2 · Young · 2014 [cited by examiner]
US 8872910B1 · Vaziri · 2014 [cited by applicant]
US 9131223B1 · Rangarajan · 2015 [cited by examiner]
US 9438491B1 · Van Broeck · 2016 [cited by applicant]
US 9438819B2 · Van Broeck · 2016 [cited by applicant]
US 9618746B2 · Browne · 2017 [cited by applicant]
US 9674490B2 · Koravadi · 2017 [cited by applicant]
US 9727790B1 · Vaziri · 2017 [cited by applicant]
US 9779311B2 · Lee et al. · 2017 [cited by applicant]
US 9811946B1 · Hung · 2017 [cited by examiner]
US 9858676B2 · Bostick et al. · 2018 [cited by applicant]
US 9864372B2 · Chen et al. · 2018 [cited by applicant]
US 10039445B1 · Torch · 2018 [cited by applicant]
US 10064552B1 · Vaziri · 2018 [cited by applicant]
US 10708514B2 · Haltmaier · 2020 [cited by applicant]
US 11189017B1 · Bagai · 2021 [cited by applicant]
US 11287262B2 · Dooley · 2022 [cited by applicant]
US 20030122930A1 · Schofield · 2003 [cited by applicant]
US 20040212882A1 · Liang · 2004 [cited by applicant]
US 20040218834A1 · Bishop · 2004 [cited by applicant]
US 20060033992A1 · Solomon · 2006 [cited by applicant]
US 20060279585A1 · Milanfar · 2006 [cited by examiner]
US 20070041663A1 · Cho · 2007 [cited by applicant]
US 20070115349A1 · Currivan et al. · 2007 [cited by applicant]
US 20080010060A1 · Asano · 2008 [cited by applicant]
US 20080030592A1 · Border · 2008 [cited by applicant]
US 20080036875A1 · Jones et al. · 2008 [cited by applicant]
US 20080198324A1 · Fuziak · 2008 [cited by applicant]
US 20080291295A1 · Kato · 2008 [cited by applicant]
US 20080297589A1 · Kurtz · 2008 [cited by applicant]
US 20090189974A1 · Deering · 2009 [cited by applicant]
US 20100053555A1 · Enriquez et al. · 2010 [cited by applicant]
US 20100103276A1 · Border · 2010 [cited by applicant]
US 20100128135A1 · Filipovich · 2010 [cited by applicant]
US 20100157078A1 · Atanassov · 2010 [cited by applicant]
US 20100157079A1 · Atanassov · 2010 [cited by applicant]
US 20100208207A1 · Connell, II · 2010 [cited by applicant]
US 20100240988A1 · Varga et al. · 2010 [cited by applicant]
US 20100254630A1 · Mi et al. · 2010 [cited by applicant]
US 20100277619A1 · Scarff · 2010 [cited by applicant]
US 20100289941A1 · Ito et al. · 2010 [cited by applicant]
US 20100290668A1 · Friedman · 2010 [cited by applicant]
US 20100290685A1 · Wein · 2010 [cited by applicant]
US 20110263946A1 · El Kaliouby · 2011 [cited by applicant]
US 20110279666A1 · Strombom et al. · 2011 [cited by applicant]
US 20120257005A1 · Browne · 2012 [cited by applicant]
US 20130121525A1 · Chen · 2013 [cited by applicant]
US 20130242057A1 · Hong · 2013 [cited by applicant]
US 20140146153A1 · Birnkrant · 2014 [cited by applicant]
US 20140253738A1 · Mullis · 2014 [cited by examiner]
US 20140267890A1 · Lelescu · 2014 [cited by examiner]
US 20140313335A1 · Koravadi · 2014 [cited by applicant]
US 20150009550A1 · Misago · 2015 [cited by applicant]
US 20150209002A1 · De Beni · 2015 [cited by applicant]
US 20160012280A1 · Ito · 2016 [cited by applicant]
US 20160179093A1 · Prokorov · 2016 [cited by applicant]
US 20160225192A1 · Jones · 2016 [cited by applicant]
US 20170007351A1 · Yu · 2017 [cited by applicant]
US 20170019599A1 · Muramatsu · 2017 [cited by examiner]
US 20170099479A1 · Browd · 2017 [cited by applicant]
US 20170142312A1 · Dal Mutto · 2017 [cited by applicant]
US 20170181802A1 · Sachs · 2017 [cited by applicant]
US 20170225336A1 · Deyle · 2017 [cited by examiner]
US 20170322410A1 · Watson · 2017 [cited by applicant]
US 20170360578A1 · Shin · 2017 [cited by applicant]
US 20170371142A1 · Anthony · 2017 [cited by examiner]
US 20180012413A1 · Jones · 2018 [cited by applicant]
US 20180330473A1 · Foi · 2018 [cited by applicant]
US 20190175214A1 · Wood · 2019 [cited by applicant]
US 20190254754A1 · Johnson · 2019 [cited by applicant]
US 20200041261A1 · Bernstein · 2020 [cited by examiner]
US 20200077033A1 · Chan · 2020 [cited by applicant]
US 20200117025A1 · Sauer · 2020 [cited by applicant]
US 20200330179A1 · Ton · 2020 [cited by applicant]
US 20210067764A1 · Shau · 2021 [cited by applicant]
CN 106527039A · 2017 [cited by examiner]
DE 102014003348A1 · 2014 [cited by examiner]
GB 2500417A · 2013 [cited by examiner]
WO WO2011004378A1 · 2011 [cited by examiner]
WO WO2012018282A1 · 2012 [cited by examiner]
WO WO2013167901A1 · 2013 [cited by examiner]
WO WO2014169197A1 · 2014 [cited by examiner]
Saito et al., Very High Resolution Image Acquisition Through Image Reconstruction From Lower Resolution Images Taken With Multiple Cameras, Singapore ICCS/ISITA, IEEE 1992 (Year: 1992). [cited by examiner]
“A High Speed Eye Tracking System with Robust Pupil Center Estimation Algorithm”, Proceedings of the 29th Ammal International Conference of the IEEE EMBS, 25 Kyon, France, pp. 3331-3334, Aug. 2007. [cited by applicant]
“A Novel Approach of Video-Based Pupil Tracking”, Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, Tex., USA—pp. 1255-1262, Oct. 2009. [cited by applicant]
Barbara Zitova et al.; Image Registration Methods: a Survey; Department of Image Processing; Institute of Information Theory and Automation Academy of Sciences of the Szech Republic; Image and Vision Computing; pp. 977-… [cited by applicant]
B. K. Gunturk, “Super-resolution imaging”, in Compu-tational Photography Methods and Applications, by R. Lukac, CRC Press, 2010 [Abstract Provided]. [cited by applicant]
Cheng et al. Developing a Real-Time Identify-and-Locate System for the Blind. Workshop on Computer Vision Applications for the Visually Impaired, James Coughlan and Roberto Manduchi, Oct. 2008, Marseille, France. [cited by applicant]
D. Li, D. Winfield and D. Parkhurst, “Starburst: A Hybrid algorithm for video based eye tracking combining feature-based and model-based approaches”, Iowa State Univer-sity, Ames, Iowa. [cited by applicant]
Extrema.m, http:/lwww.malhworks.com/mallabcentral/fileexchange/12275-extrema-m-extrema2-m, Sep. 14, 2006. [cited by applicant]
Eyelink User Manual, SR Research Lid., Version 1.3.0, Copyright 2005-2008, 134 pages. [cited by applicant]
Eyelink Data Viewer User's Manual, SR Research Lid., Copyright 2002-2008, Document Version 1.8.402, 149 pages. [cited by applicant]
Fritz Gleyo, Microsoft May Make Life-Sized Cortana In Person For HoloLens, (Dec. 14, 2015). [cited by applicant]
Guestrin et al. “General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections”, IEEE Trans. Biomedical Eng., vol. 53, No. 6, pp. 1124-1133, Jun. (2006). [cited by applicant]
Jessi Hempel, Project HoloLens: Our Exclusive Hands-On With Microsoft's Holographic Goggles, (Jan. 21, 2015). [cited by applicant]
J. Goodman, Introduction to Fourier Optics, Second Ed., 160-165, McGraw-Hill, 1988. [cited by applicant]
Jordan Novet, Microsoft could build a life-sized Cortana for HoloLens, https://www.technologyrecord.com/Article/introducing-microsoft-hololens-development-edition-48296. [cited by applicant]
Kennet Kubala et al.; Reducing Complexity in Computational Imaging Systems; CDM Optics, Inc.; Sep. 8, J003; vol. 11, No. 18; Optics Express; pp. 2102-2108. [cited by applicant]
Lees et al. (Ultrasound Imaging in Three and Four Dimensions, Seminars in Ultrasound, CT, and MR/, vol. 22, No. 1 Feb. 2001: pp. 85-105, (Year: 2001). [cited by applicant]
Lindsay James, Introducing Microsoft HoloLens Development Edition, https://blogs.windows.com/ :levices/2015/04/30/build-2015-a-closer-look-at-the-microsoft-hololens-hardware/. [cited by applicant]
Malcolm et al. Combining topdown processes to guide eye movements during real-world scene search. Journal of Vision, 10(2):4, p. 1-11 (2010). [cited by applicant]
Maria E. Angelopoulou et al.; FPGA-based Real-lime Super-Resolution on an Adaptive Image Sensor; Department of Electrical and Electronic Engineering, Imperial College London; 9 pages. [cited by applicant]
Moreno et al. Classification of visual and linguistic tasks using eye-movement features; Journal of Vision (2014) 14(3):11, 1-18. [cited by applicant]
P. C. Hansen, J. G. Nagy, D. P. O'Leary, Deblurring Images: matrices, Spectra and Filtering, SIAM (2006) [Abstract Provided]. [cited by applicant]
P. Milanfar, Super-Resolution Imaging, CRC Press (2011) [Abstract Provided]. [cited by applicant]
Pravin Bhat et al.; Using Photographs to Enhance Videos of a Static Scene; University of Washington; Microsoft Research; Adobe Systems; University of California; The Eurographics Association 2007; pp. 1-12. [cited by applicant]
R. W. Gerchberg, “Super-resolution through energy reduction”, Optica Acta, vol. 21, No. 9, pp. 709-720 (1974). [cited by applicant]
S. Chaudhuri, Super-Resolution Imaging, Kluwer Aca-demic Publishers (2001) [Abstract Provided]. [cited by applicant]
Sang-Hyuck Lee et al.; Breaking Diffraction Limit of a Small F-Number Compact Camera Using Wavefront Coding; Center for Information Storage Device; Department of Mechanical Engineering, Yonsei University, Shinchondong, … [cited by applicant]
S.-W Jung and S.-J. Ko, “Image deblurring using multi-exposed images” in Computational Photography 65 Methods and Applications, by R. Lukac, CRC Press, 2010. [Abstract Provided]. [cited by applicant]
Todd Holmdahl, Build 2015: A closer look at the Microsoft HoloLens hardware, https://blogs.windows.com/ :levices/2015/04/30/build-2015-a-closer-look-at-the-microsoft-hololens-hardware/. [cited by applicant]
V. Barmore, Iterative-Interpolation Super-Resolution Image Reconstruction, Springer (2009) [Abstract Provided]. [cited by applicant]
W. Thomas Cathey et al.; New Paradigm for Imaging Systems; Optical Society of America; Applied Optics; Oct. 10, 2002; vol. 41, No. 29; pp. 6080-6092. [cited by applicant]
William T. Freeman et al.; Example-Based Super-Resolution; Mitsubishi Electric Research Labs; March/April J002; IEEE Computer Graphics and Applications; pp. 56-65. [cited by applicant]
John Bardsley et al.; Blind Iterative Restoration of Images With Spatially-Varying Blur; 9 pages. [cited by applicant]
Sang-Hyuck Lee et al.; Breaking Diffraction Limit of a Small F-Number Compact Camera Using Wavefront Coding; Center for Information Storage Device; Department of Mechanical Engineering, Yonsei University, Shinchondong, … [cited by applicant]
Tod R. Lauer; Deconvolution With a Spatially-Variant PSF; National Optical Astronomy Observatory; Tucson, AZ; arXiv:astro-ph/0208247v1; Aug. 12, 2002; 7 pages. [cited by applicant]
Suk Hwan Lim and Amnon Silverstein; Estimation and Removal of Motion Blur by Capturing Two Images With Different Exposures; HP Laboratories and NVidia Corp.; HPL-2008-170; Oct. 21, 2008; 8 pages. [cited by applicant]
William T. Freeman et al.; Example-Based Super-Resolution; Mitsubishi Electric Research Labs; Mar./Apr. 2002; IEEE Computer Graphics and Applications; pp. 56-65. [cited by applicant]
Edward R. Dowski, Jr. et al.; Extended Depth of Field Through Wave-Front Coding; Apr. 10, 1995; Optical Society of America; vol. 34, No. 11; Applied Optics pp. 1859-1866. [cited by applicant]
Maria E. Angelopoulou et al.; FPGA-based Real-time Super-Resolution on an Adaptive Image Sensor Department of Electrical and Electronic Engineering, Imperial College London; 9 pages. [cited by applicant]
Patrick Vandewalle et al.; A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution; Ecole Polytechnique Federal de Lausanne, School of Computer and Communication Sciences; Depa… [cited by applicant]
Danny Keren et al.; Image Sequence Enhancement Using Sub-pixel Displacements; Department of computer science; The Hebrew University of Jerusalem; 1988 IEEE; pp. 742-746. [cited by applicant]
Athanasios Papoulis; A New Algorithm in Spectral Analysis and Band-Limited Extrapolation; IEEE Transactions on Circuits and Systems, Sep. 1975; vol. CAS-22, No. 9; pp. 735-742. [cited by applicant]
W. Thomas Cathey ey al.; New Paradigm for Imaging Systems; Optical Society of America; Applied Optics; Oct. 10, 2002; vol. 41, No. 29; pp. 6080-6092. [cited by applicant]
Oliver Bowen et al.; Real-Time Image Super Resolution Using an FPGA; Department of Electrical and Electronic Engineering; Imperial College London; 2008 IEEE; pp. 89-94. [cited by applicant]
Kennet Kubala et al.; Reducing Complexity in Computational Imaging Systems; CDM Optics, Inc.; Sep. 8, 2003; vol. 11, No. 18; Optics Express; pp. 2102-2108. [cited by applicant]
Maria E. Angelopoulou et al.; Robust Real-Time Super-Resolution on FPGA and an Application to Video Enhancement; Imperial College London; ACM Journal Name; Sep. 2008; vol. V, No. N; pp. 1-27. [cited by applicant]
Eran Gur and Zeev Zalevsky; Single-Image Digital Super-Resolution a Revised Gerchberg-Papoulis Algorithm AENG International Journal of Computer Science; Nov. 17, 2007; pp. 1-5. [cited by applicant]
Lisa Gottesfeld Brown; A Survey of Image Registration Techniques; Department of Computer Science Columbia University; Jan. 12, 1992; pp. 1-60. [cited by applicant]
Pravin Bhat Ei Al.; Using Photographs to Enhance Videos of a Static Scene; University of Washington; Microsoft Research; Adobe Systems; University of California; The Eurographics Association 2007; pp. 1-12. [cited by applicant]
Guestrin et al. “General Theory of Remote Gaze Estimation Using the Pupil Center and Comeal Reflections”, IEEE Frans. Biomedical Eng., vol. 53, No. 6, pp. 1124-1133, June (2006). [cited by applicant]
Malcolm et al. Combining topdown processes to guide eye movements during real-world scene search. Journal of Mision, 10(2):4, p. 1-11 (2010). [cited by applicant]
Extrema.m, http://www.mathworks.com/matlabcentral/fileexchange/12275-extrema-m-extrema2-m, Sep. 14, 2006. [cited by applicant]