IP Library Granted Patent US 12,336,782
Granted Patent B2
US 12,336,782 · App. 18/431,800 · Granted Jun 24, 2025

Method and apparatus for a compact and high resolution mind-view communicator

Inventor: Masoud Vaziri (Richardson, TX)
Assignee: OPTICS INNOVATION LLC
A61B5/0022A61B5/1128A61B5/163A61B5/168A61B5/369A61B5/6803A61N1/3606G06V40/19G06V40/193G09B19/00H04N5/76H04N5/775H04N7/22H04N9/8205H04N21/2187H04N23/45H04N23/54H04N23/56H04N23/57A61B5/1107A61B5/1112A61B2560/0242H04N7/181
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,336,782
App. No.
18/431,800
Granted
Jun 24, 2025
Kind
B2
Abstract

An embodiment of a mind-view communication apparatus includes a first portable unit and a second portable unit. The first portable unit includes an eyeglass frame, at least one first optical unit disposed on the eyeglass frame for capturing at least one scene image corresponding to a field of view of a user, and at least one second optical unit disposed on the eyeglass frame for capturing at least one eye image corresponding to at least a portion of at least one eye of the user. The second portable unit is in communication with the first portable unit and includes at least one processor configured for receiving the at least one scene image and the at least one eye image, determining a direction within the field of view to which the at least one eye is directed based upon the at least one eye image, and generating a subset of the at least one scene image based on the determined direction.

Claims (35)

1. An imaging system comprising:

a first visible light camera having a first field of view;

a second visible light camera having a second field of view;

an interface configured to present a plurality of selectable imaging options, the plurality of selectable imaging options including:

a first imaging option corresponding to using the first visible light camera;

a second imaging option corresponding to using the second visible light camera; and

a third imaging option corresponding to using the first visible light camera and the second visible light camera;

a frame configured to be carried or worn by a user, the first visible light camera and the second visible light camera being coupled to the frame such that the first field of view of the first visible light camera and the second field of view of the second visible light camera at least partially overlap; and

at least one processor configured to:

responsive to receiving a selection associated with the first imaging option, cause the first visible light camera to generate first image data that is associated with a scene;

responsive to receiving a selection associated with the second imaging option, cause the second visible light camera to generate second image data that is associated with the scene; and

responsive to receiving a selection associated with the third imaging option, (i) cause the first visible light camera to generate third image data that is associated with the scene, (ii) cause the second visible light camera to generate fourth image data that is associated with the scene, and (iii) cause the interface to display a visual image of the scene using at least a portion of the third image data and at least a portion of the fourth image data.

2. The imaging system of claim 1 , wherein the third image data is reproducible as a first image having a first image resolution, wherein the fourth image data is reproducible as a second image having a second image resolution that is higher than the first image resolution, and wherein the first image overlaps with the second image in an overlap area of the first image, and wherein the displayed visual image of the scene is an enhanced first image such that at least a portion of the enhanced first image not within the overlap area of the first image has a third image resolution that is higher than the first image resolution.

3. The imaging system of claim 2 , wherein the at least a portion of the third image data corresponds to the at least a portion of the fourth image data.

4. The imaging system of claim 2 , wherein the at least a portion of the enhanced first image is the entire enhanced first image.

5. The imaging system of claim 2 , wherein the second field of view is smaller than the first field of view.

6. The imaging system of claim 5 , wherein the second image is completely within and a subset of the first image.

7. The imaging system of claim 5 , wherein at least a portion of the first image surrounds the second image.

8. The imaging system of claim 2 , further comprising storing the visual image in a memory device of the imaging system.

9. The imaging system of claim 2 , wherein responsive to the selection associated with the third imaging option, the first visible light camera generates the third image data at the same time or about the same time that the second visible light camera generates the fourth image data.

10. The imaging system of claim 2 , wherein the first visible light camera includes a first color filter array and the second visible light camera includes no color filter, and wherein the first image is a color image, and the second image is a black and white image.

11. The imaging system of claim 2 , wherein the at least a portion of the enhanced first image is generated by iterative bandwidth extrapolation.

12. The imaging system of claim 2 , wherein the first visible light camera and the second visible light camera are coupled to the frame such that the first visible light camera and the second visible light camera are positioned at a predetermined distance apart.

13. The imaging apparatus system of claim 12 , wherein the predetermined distance is less than about six inches.

14. The imaging system of claim 1 , wherein the at least one processor is further configured to, based at least in part on the third image data and the fourth image data, extract distance information associated with an object in the scene.

15. The imaging system of claim 14 , wherein the displayed visual image of the scene includes the object and has a lowered resolution version of a background portion of the scene.

16. The imaging system of claim 15 , wherein the lowered resolution version of the background portion of the scene causes the background portion of the scene to appear blurred.

17. The imaging system of claim 15 , wherein the background portion of the scene generally surrounds the object in the scene.

18. The imaging system of claim 16 , wherein the background portion is blurred by pixel binning, applying a low pass filter, or a combination thereof.

19. The imaging system of claim 1 , wherein the at least one processor is further configured to, based at least in part on the third image data and the fourth image data, extract distance information associated with a feature in the scene.

20. The imaging system of claim 19 , wherein the displayed visual image of the scene includes the feature and has a lowered resolution version of a region peripheral to the feature.

21. The imaging system of claim 20 , wherein the frame is an eyeglass frame.

22. The imaging system of claim 20 , wherein the region peripheral to the feature generally surrounds the feature in the scene.

23. The imaging system of claim 20 , wherein the resolution of the region peripheral to the feature is lowered by pixel binning, applying a low pass filter, or a combination thereof.

24. The imaging system of claim 20 , wherein the at least one processor is further configured to execute a face recognition procedure to identify a face in the feature.

Assignments (6)
NUNC PRO TUNC ASSIGNMENT Recorded Feb 5, 2024
From: VAZIRIR, MASOUD
To: U & LIFE CORPORATION
Reel/Frame 066386/0629 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2024
From: IPAL INC.
To: VAZIRI, MOJTABA
Reel/Frame 066387/0025 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2024
From: VAZIRI, MOJTABA
To: OPTICS INNOVATION LLC
Reel/Frame 066387/0054 →
MERGER Recorded Feb 5, 2024
From: U & LIFE CORPORATION
To: I2I INC.
Reel/Frame 066494/0104 →
MERGER Recorded Feb 5, 2024
From: U & LIFE CORPORATION
To: I2I INC.
Reel/Frame 066494/0129 →
CHANGE OF NAME Recorded Feb 5, 2024
From: I2I INC.
To: IPAL INC.
Reel/Frame 066494/0371 →
Continuity (11)
Continuation 17887438 · Aug 13, 2022
Continuation 16908514 · Jun 22, 2020
Continuation 16120326 · Sep 3, 2018
Continuation 15400399 · Jan 6, 2017
Continuation 13175421 · Jul 1, 2011
Continuation In Part 12794283 · Jun 4, 2010
Provisional Application 61471397 · Apr 4, 2011
Provisional Application 61471376 · Apr 4, 2011
Provisional Application 61369618 · Jul 30, 2010
Provisional Application 61184232 · Jun 4, 2009
Related Publication 20240172940A1 · May 30, 2024
References Cited (156)
US 4028725A · Lewis · 1977 [cited by applicant]
US 4907296A · Blecha · 1990 [cited by examiner]
US 5262871A · Wilder · 1993 [cited by applicant]
US 5856811A · Shih · 1999 [cited by examiner]
US 5859921A · Suzuki · 1999 [cited by applicant]
US 6091546A · Spitzer · 2000 [cited by applicant]
US 6163336A · Richards · 2000 [cited by applicant]
US 6198485B1 · Mack · 2001 [cited by applicant]
US 6307526B1 · Mann · 2001 [cited by applicant]
US 6434280B1 · Peleg · 2002 [cited by applicant]
US 6486799B1 · Still · 2002 [cited by applicant]
US 6661495B1 · Popovich · 2003 [cited by applicant]
US 6766067B2 · Freeman · 2004 [cited by applicant]
US 6850629B2 · Jeon · 2005 [cited by applicant]
US 7023464B1 · Harada · 2006 [cited by applicant]
US 7331671B2 · Hammond · 2008 [cited by applicant]
US 7391887B2 · Durnell · 2008 [cited by applicant]
US 7492926B2 · Kang · 2009 [cited by applicant]
US 7538326B2 · Johnson · 2009 [cited by applicant]
US 7697024B2 · Currivan · 2010 [cited by applicant]
US 7715658B2 · Cho · 2010 [cited by applicant]
US 7894666B2 · Mitarai · 2011 [cited by applicant]
US 8014632B2 · Matsumoto · 2011 [cited by applicant]
US 8139089B2 · Doyle · 2012 [cited by applicant]
US 8159519B2 · Kurtz · 2012 [cited by applicant]
US 8189100B2 · Li · 2012 [cited by applicant]
US 8305899B2 · Luo · 2012 [cited by applicant]
US 8432492B2 · Deigmoeller · 2013 [cited by applicant]
US 8872910B1 · Vaziri · 2014 [cited by applicant]
US 9230140B1 · Ackley · 2016 [cited by applicant]
US 9438491B1 · Broeck · 2016 [cited by applicant]
US 9438819B2 · Broeck · 2016 [cited by applicant]
US 9618746B2 · Browne · 2017 [cited by applicant]
US 9674490B2 · Koravadi · 2017 [cited by applicant]
US 9727790B1 · Vaziri · 2017 [cited by applicant]
US 9779311B2 · Lee · 2017 [cited by applicant]
US 9858676B2 · Bostick · 2018 [cited by applicant]
US 9864372B2 · Chen · 2018 [cited by applicant]
US 10039445B1 · Torch · 2018 [cited by applicant]
US 10064552B1 · Vaziri · 2018 [cited by applicant]
US 10708514B2 · Haltmaier · 2020 [cited by applicant]
US 11189017B1 · Bagai · 2021 [cited by applicant]
US 11287262B2 · Dooley · 2022 [cited by applicant]
US 20030122930A1 · Schofield · 2003 [cited by applicant]
US 20040212882A1 · Liang · 2004 [cited by applicant]
US 20040218834A1 · Bishop · 2004 [cited by applicant]
US 20050046698A1 · Knight · 2005 [cited by applicant]
US 20060033992A1 · Solomon · 2006 [cited by applicant]
US 20070041663A1 · Cho · 2007 [cited by applicant]
US 20070115349A1 · Currivan · 2007 [cited by applicant]
US 20080010060A1 · Asano · 2008 [cited by applicant]
US 20080030592A1 · Border · 2008 [cited by applicant]
US 20080036875A1 · Jones · 2008 [cited by applicant]
US 20080198324A1 · Fuziak · 2008 [cited by applicant]
US 20080291295A1 · Kato · 2008 [cited by applicant]
US 20080297589A1 · Kurtz · 2008 [cited by applicant]
US 20090136158A1 · Tamura · 2009 [cited by applicant]
US 20090189974A1 · Deering · 2009 [cited by applicant]
US 20090210786A1 · Suzuki · 2009 [cited by applicant]
US 20090219224A1 · Elg · 2009 [cited by applicant]
US 20100053555A1 · Enriquez · 2010 [cited by applicant]
US 20100103276A1 · Border · 2010 [cited by applicant]
US 20100128135A1 · Filipovich · 2010 [cited by applicant]
US 20100149073A1 · Chaum · 2010 [cited by applicant]
US 20100157078A1 · Atanassov · 2010 [cited by applicant]
US 20100157079A1 · Atanassov · 2010 [cited by applicant]
US 20100165287A1 · Pienimaa · 2010 [cited by applicant]
US 20100208207A1 · Connell, II · 2010 [cited by applicant]
US 20100240988A1 · Varga · 2010 [cited by applicant]
US 20100254630A1 · Ali · 2010 [cited by applicant]
US 20100277619A1 · Scarff · 2010 [cited by applicant]
US 20100289941A1 · Ito · 2010 [cited by applicant]
US 20100290668A1 · Friedman · 2010 [cited by applicant]
US 20100290685A1 · Wein · 2010 [cited by applicant]
US 20110064327A1 · Dagher · 2011 [cited by applicant]
US 20110263946A1 · El Kaliouby · 2011 [cited by applicant]
US 20110279666A1 · Stromborn · 2011 [cited by applicant]
US 20120086780A1 · Sharma · 2012 [cited by applicant]
US 20120257005A1 · Browne · 2012 [cited by applicant]
US 20130106911A1 · Salsman · 2013 [cited by applicant]
US 20130121525A1 · Chen · 2013 [cited by applicant]
US 20130242057A1 · Hong · 2013 [cited by applicant]
US 20140146153A1 · Birnkrant · 2014 [cited by examiner]
US 20140267890A1 · Lelescu · 2014 [cited by applicant]
US 20140313335A1 · Koravadi · 2014 [cited by applicant]
US 20150009550A1 · Misago · 2015 [cited by examiner]
US 20150209002A1 · De Beni · 2015 [cited by applicant]
US 20160012280A1 · Ito · 2016 [cited by applicant]
US 20160179093A1 · Prokorov · 2016 [cited by applicant]
US 20160225192A1 · Jones · 2016 [cited by examiner]
US 20170007351A1 · Yu · 2017 [cited by examiner]
US 20170019599A1 · Muramatsu · 2017 [cited by applicant]
US 20170099479A1 · Browd · 2017 [cited by examiner]
US 20170142312A1 · Dal Mutto · 2017 [cited by applicant]
US 20170181802A1 · Sachs · 2017 [cited by examiner]
US 20170225336A1 · Deyle · 2017 [cited by applicant]
US 20170322410A1 · Watson · 2017 [cited by examiner]
US 20170360578A1 · Shin · 2017 [cited by applicant]
US 20180012413A1 · Jones · 2018 [cited by examiner]
US 20180188892A1 · Levac · 2018 [cited by applicant]
US 20180330473A1 · Foi · 2018 [cited by applicant]
US 20190175214A1 · Wood · 2019 [cited by applicant]
US 20190254754A1 · Johnson · 2019 [cited by examiner]
US 20190272336A1 · Ciecko · 2019 [cited by applicant]
US 20200041261A1 · Bernstein · 2020 [cited by applicant]
US 20200077033A1 · Chan · 2020 [cited by applicant]
US 20200117025A1 · Sauer · 2020 [cited by examiner]
US 20200330179A1 · Ton · 2020 [cited by examiner]
US 20210067764A1 · Shau · 2021 [cited by examiner]
Guestrin et al. “General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections”, IEEE Trans. Biomedical Eng., vol. 53, No. 6, pp. 1124-1133, Jun. (2006). [cited by applicant]
“A High Speed Eye Tracking System with Robust Pupil Center Estimation Algorithm”, Proceedings of the 29th Ammal International Conference of the IEEE EMBS, 25 Kyon, France, pp. 3331-3334, Aug. 2007. [cited by applicant]
“A Novel Method of Video-Based Pupil Tracking”, Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, Tex., USA-pp. 1255-1262, Oct. 2009. [cited by applicant]
A Zandifar, R. Duraiswami, L.S. Davis, A video-based framework for the analysis of presentations/posters, 2003 (Year: 2003) 10 pages. [cited by applicant]
B. K. Gunturk, “Super-resolution imaging”, in Computational Photography Methods and Applications, by R. Lukac, CRC Press, 2010 [Abstract Provided]. [cited by applicant]
Cheng et al. Developing a Real-Time Identify-and-Locate System for the Blind. Workshop on Computer Vision Applications for the Visually Impaired, James Coughlan and Roberto Manduchi, Oct. 2008, Marseille, France. [cited by applicant]
D. Li, D. Winfield and D. Parkhurst, “Starbust: A Hybrid algorithm for video based eye tracking combining feature-based and model-based approaches”, Iowa State Univer-sity, Ames, Iowa. [cited by applicant]
Fritz Gleyo, Microsoft May Make Life-Sized Cortana In Person For HoloLens, (Dec. 14, 2015). [cited by applicant]
Jessi Hempel, Project HoloLens: Our Exclusive Hands-On With Microsoft's Holographic Goggles, (Jan. 21, 2015). [cited by applicant]
J. Goodman, Introduction to Fourier Optics, rd edition, 160-165, McGraw-Hill, 1988. [cited by applicant]
Jordan Novet, Microsoft could build a life-sized Cortana for HoloLens, https://www.technologyrecord.com/Article/introducing-microsoft-hololens-development-edition-48296. [cited by applicant]
Lees et al. (Ultrasound Imaging in Three and Four Dimensions, Seminars in Ultrasound, CT, and MR/, vol. 22, No. 1 (Feb.), 2001: pp. 85-105, (Year: 2001). [cited by applicant]
Lindsay James, Introducing Microsoft HoloLens Development Edition, https://blogs.windows.com/ :levices/2015/04/30/build-2015-a-closer-look-at-the-microsoft-hololens-hardware/. [cited by applicant]
Moreno et al. Classification of visual and linguistic tasks using eye-movement features; Journal of Vision (2014) 14(3):11, 1-18. [cited by applicant]
P. C. Hansen, J. G. Nagy, D. P. O'Leary, Deblurring Images: matrices, Spectra and Filtering, SIAM (2006) [Abstract Provided]. [cited by applicant]
P. Milanfar, Super-Resolution Imaging, CRC Press (2011) [Abstract Provided]. [cited by applicant]
R. W. Gerchberg, “Super-resolution through error energy reduction”, Optica Acta, vol. 21, No. 9, pp. 709-720,(1974). [cited by applicant]
S. Chaudhuri, Super-Resolution Imaging, Kluwer Aca-demic Publishers (2001) [Abstract Provided]. [cited by applicant]
Sawhney, H. et al.; “Hybrid Stereo Camera: An IBR Approach for Synthesis of Very High Resolution Stereoscopic Image Sequences”; AC SIGGRAPH, pp. 451-460; 2001 (10 pages). [cited by applicant]
S.-W Jung and S.-J. Ko, “Image deblurring using multi-exposed images” in Computational Photography 65 Methods and Applications, by R. Lukac, CRC Press, 2010. [Abstract Provided]. [cited by applicant]
Todd Holmdahl, Build 2015: A closer look at the Microsoft HoloLens hardware, https://blogs.windows.com/ :levices/2015/04/30/build-2015-a-closer-look-at-the-microsoft-hololens-hardware/. [cited by applicant]
V. Barmore, Iterative-Interpolation Super-Resolution Image Reconstruction, Springer (2009) [Abstract Provided]. [cited by applicant]
Zitnick, L. et al.; “Stereo for Image-Based Rendering Using Image Over-Segmentation”; International Journal of Computer Visions; 2006 (32 pages). [cited by applicant]
Z. Zalevsky, D. Mendlovic, Optical Superresolution, 2004 (Year: 2004) 261 pages. [cited by applicant]
Papoulis, A.; A New Algorithm in Spectral Analysis and Band-Limited Extrapolation; IEEE Transactions on Circuits and Systems, Sep. 1975; vol. CAS-22, No. 9; pp. 735-742. [cited by applicant]
Zitova, B. et al.; Image Registration Methods: a Survey; Department of Image Processing; Institute of Information Theory and Automation Academy of Sciences of the Szech Republic; Image and Vision Computing; pp. 977-1000. [cited by applicant]
Keren, D. et al.; Image Sequence Enhancement Using Sub-pixel Displacements; Department of computer science; The Hebrew University of Jerusalem; 1988 IEEE; pp. 742-746. [cited by applicant]
Dowski, Jr., E. et al.; Extended Depth of Field Through Wave-Front Coding; Apr. 10, 1995; Optical Society of America; vol. 34, No. 11; Applied Optics pp. 1859-1866. [cited by applicant]
Eran Gur and Zeev Zalevsky; Single-Image Digital Super-Resolution a Revised Gerchberg-Papoulis Algorithm; AENG International Journal of Computer Science; Nov. 17, 2007; pp. 1-5. [cited by applicant]
Extrema.m; http:/lwww.malhworks.com/mallabcentral/fileexchange/12275-extrema-m-extrema2-m; Sep. 14, 2006. [cited by applicant]
Eyelink User Manual, SR Research Lid., Copyright 2005-2008, 134 pages. [cited by applicant]
Eyelink Data Viewer User's Manual, SR Research Lid., Copyright 2002-2008, 149 pages. [cited by applicant]
Guestrin et al. “General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections”, IEEE Trans. Biomedical Eng., vol. 53, No. 6, pp. 1124-1133, June (2006). [cited by applicant]
John Bardsley et al.; “Blind Iterative Restoration of Images With Spatially-Varying Blur”; 9 pages. [cited by applicant]
Kennet Kubala et al.; Reducing Complexity in Computational Imaging Systems; COM Optics, Inc.; Sep. 8, J003; vol. 11, No. 18; Optics Express; pp. 2102-2108. [cited by applicant]
Lisa Gottesfeld Brown; A Survey of Image Registration Techniques; Department of Computer Science; Columbia University; Jan. 12, 1992; pp. 1-60. [cited by applicant]
Malcolm et al. Combining topdown processes to guide eye movements during real-world scene search. Journal of Vision, 10(2):4, p. 1-11 (2010). [cited by applicant]
Maria E. Angelopoulou et al.; FPGA-based Real-time Super-Resolution on an Adaptive Image Sensor; Department of Electrical and Electronic Engineering, Imperial College London; 9 pages. [cited by applicant]
Maria E. Angelopoulou et al.; Robust Real-Time Super-Resolution on FPGA and an Application to Video Enhancement; Imperial College London; ACM Journal Name; Sep. 2008; vol. V, No. N; pp. 1-27. [cited by applicant]
Oliver Bowen et al.; Real-Time Image Super Resolution Using an FPGA; Department of Electrical and Electronic Engineering; Imperial College London; 2008 IEEE; pp. 89-94. [cited by applicant]
Patrick Vandewalle et al.; A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution; Ecole Polytechnique Federal de Lausanne, School of Computer and Communication Sciences; Depa… [cited by applicant]
Pravin Bhat et al.; Using Photographs to Enhance Videos of a Static Scene; University of Washington; Microsoft Research; Adobe Systems; University of California; The Eurographics Association 2007; pp. 1-12. [cited by applicant]
Sang-Hyuck Lee et al.; Breaking Diffraction Limit of a Small F-Number Compact Camera Using Wavefront Coding; Center for Information Storage Device; Department of Mechanical Engineering, Yonsei University, Shinchondong, … [cited by applicant]
Suk Hwan Lim and Amnon Silverstein; Estimation and Removal of Motion Blur by Capturing Two Images With Different Exposures; HP Laboratories and NVidia Corp.; HPL-2008-170; Oct. 21, 2008; 8 pages. [cited by applicant]
Tod R. Lauer; Deconvolution With a Spatially-Variant PSF; National Optical Astronomy Observatory; Tucson, AZ; arXiv:astro-ph/0208247v1; Aug. 12, 2002; 7 pages. [cited by applicant]
W. Thomas Cathey et al.; New Paradigm for Imaging Systems; Optical Society of America; Applied Optics; Oct. 10, 2002; vol. 41, No. 29; pp. 6080-6092. [cited by applicant]
William T. Freeman et al.; Example-Based Super-Resolution; Mitsubishi Electric Research Labs; Mar./Apr. 2002; IEEE Computer Graphics and Applications; pp. 56-65. [cited by applicant]