Granted Patent
B2
US 12,472,000 · App. 18/505,385 · Granted Nov 18, 2025
Augmented reality system for monitoring size and laterality of physical implants during surgery and for billing and invoicing
A61B34/10A61B17/155A61B17/157A61B17/1703A61B17/1742A61B17/1764A61B17/1775A61B17/1778A61B34/74A61B90/36A61B90/37A61F2/32A61F2/3859A61F2/389A61B2017/00216A61B17/15A61B17/1666A61B17/17A61B2017/568A61B2034/104A61B2034/107A61B2034/2048A61B2034/2055A61B2034/2059A61B2090/365A61B2090/368A61B2090/372A61B2090/373A61B2090/374A61B2090/376A61B2090/3762A61B2090/502A61F2002/4018A61F2002/4205A61F2002/4207
View Patent ↗
Loading inventors, assignments & file history…
Abstract
Devices and methods for performing a surgical step or surgical procedure with visual guidance using an optical head mounted display are disclosed.
Claims (22)
1 . A system for use during surgery comprising,
at least one computer processor;
at least one see-through head mounted display;
at least one camera integrated into or attached to the at least one see-through head mounted display; and
at least one optical marker,
wherein the at least one optical marker is included in a sterile implant component package or attached to an implant component packaging label of a physical implant component in a sterile implant component package,
wherein the at least one optical marker includes an information about at least one of a version, type, model, lot number, bill, invoice, or a combination thereof for the physical implant component,
wherein the at least one camera is configured to acquire an image of the at least one optical marker,
wherein the at least one computer processor is configured to identify the physical implant component used during surgery based on the image of the at least one optical marker,
wherein the at least one computer processor is configured to process the information about the at least one of a version, type, model, lot number, bill, invoice, or a combination thereof for the physical implant component,
wherein the at least one computer processor is configured to trigger an alert or an action, wherein the alert or the action includes an information about the bill, the invoice, or combinations thereof for the physical implant component used during surgery.
2 . The system of claim 1 , wherein the at least one optical marker comprises a QR code, a bar code, an alphanumeric information or a combination thereof.
3 . The system of claim 1 , wherein the at least one see-through head mounted display is an optical see-through head mounted display.
4 . The system of claim 1 , wherein the at least one see-through head mounted display is a video see-through head mounted display.
5 . The system of claim 1 , wherein the implant component is a spinal implant, a knee implant, a hip implant, a shoulder implant, or a vascular implant.
6 . The system of claim 5 , wherein the vascular implant is a coronary stent, a carotid stent, an aortic stent, or a femoral artery stent.
7 . The system of claim 1 , wherein the system comprises at least one user interface.
8 . The system of claim 1 , wherein the at least one user interface comprises a graphical user interface, a voice recognition, a gesture recognition, a virtual interface displayed by the at least one see-through head mounted display, a virtual keyboard displayed by the at least one see through head mounted display, a physical keyboard, a physical computer mouse, a physical track pad, or a combination thereof.
9 . The system of claim 7 , wherein the at least one user interface is configured for billing purposes.
10 . The system of claim 7 , wherein the at least one user interface is configured for generating an invoice.
11 . The system of claim 1 , wherein the alert or the action comprises generating the bill or the invoice.
12 . The system of claim 11 , wherein the system is configured to monitor payment.
Continuity (19)
Provisional Application
62453484
· Feb 1, 2017
Provisional Application
62445691
· Jan 12, 2017
Provisional Application
62425019
· Nov 21, 2016
Provisional Application
62406379
· Oct 10, 2016
Provisional Application
62393054
· Sep 11, 2016
Provisional Application
62378242
· Aug 23, 2016
Provisional Application
62354780
· Jun 26, 2016
Provisional Application
62331995
· May 5, 2016
Provisional Application
62323716
· Apr 17, 2016
Provisional Application
62318157
· Apr 4, 2016
Provisional Application
62307476
· Mar 12, 2016
References Cited (291)
US 5526812A
· Dumoulin et al.
· 1996
[cited by applicant]
US 5676673A
· Ferre et al.
· 1997
[cited by applicant]
US 5800352A
· Ferre et al.
· 1998
[cited by applicant]
US 5803089A
· Ferre et al.
· 1998
[cited by applicant]
US 5829444A
· Ferre et al.
· 1998
[cited by applicant]
US 5873822A
· Ferre et al.
· 1999
[cited by applicant]
US D415146S
· Hori
· 1999
[cited by applicant]
US 5967980A
· Ferre et al.
· 1999
[cited by applicant]
US 6175756B1
· Ferre et al.
· 2001
[cited by applicant]
US 6341231B1
· Ferre et al.
· 2002
[cited by applicant]
US 6396497B1
· Reichlen
· 2002
[cited by applicant]
US 6445943B1
· Ferre et al.
· 2002
[cited by applicant]
US 6599247B1
· Stetten
· 2003
[cited by applicant]
US 6714810B2
· Grzeszczuk et al.
· 2004
[cited by applicant]
US 7130676B2
· Barrick
· 2006
[cited by applicant]
US 7774044B2
· Sauer et al.
· 2010
[cited by applicant]
US 7812815B2
· Banerjee et al.
· 2010
[cited by applicant]
US 8320612B2
· Knobel et al.
· 2012
[cited by applicant]
US 8730266B2
· Brown et al.
· 2014
[cited by applicant]
US 8989843B2
· Chien
· 2015
[cited by applicant]
US 9068820B2
· Kosmecki et al.
· 2015
[cited by applicant]
US 9068824B2
· Findeisen et al.
· 2015
[cited by applicant]
US 9123155B2
· Cunningham et al.
· 2015
[cited by applicant]
US 9183560B2
· Abelow
· 2015
[cited by applicant]
US 9215293B2
· Miller
· 2015
[cited by applicant]
US 9299138B2
· Zellner et al.
· 2016
[cited by applicant]
US 9310559B2
· Macnamara
· 2016
[cited by applicant]
US 9311284B2
· Warila et al.
· 2016
[cited by applicant]
US 9389424B1
· Schowengerdt
· 2016
[cited by applicant]
US 9417452B2
· Schowengerdt et al.
· 2016
[cited by applicant]
US 9429752B2
· Schowengerdt et al.
· 2016
[cited by applicant]
US 9503681B1
· Popescu et al.
· 2016
[cited by applicant]
US 9547940B1
· Sun et al.
· 2017
[cited by applicant]
US 9582717B2
· Lee et al.
· 2017
[cited by applicant]
US 9792721B2
· Kosmecki et al.
· 2017
[cited by applicant]
US 9861446B2
· Lang
· 2018
[cited by applicant]
US 9980780B2
· Lang
· 2018
[cited by applicant]
US 20010041838A1
· Holupka et al.
· 2001
[cited by applicant]
US 20020082498A1
· Wendt et al.
· 2002
[cited by applicant]
US 20020163499A1
· Sauer
· 2002
[cited by applicant]
US 20050113846A1
· Carson
· 2005
[cited by applicant]
US 20050215879A1
· Chuanggui
· 2005
[cited by applicant]
US 20050267353A1
· Marquart et al.
· 2005
[cited by applicant]
US 20050281465A1
· Marquart et al.
· 2005
[cited by applicant]
US 20060142739A1
· DiSilestro et al.
· 2006
[cited by applicant]
US 20070015999A1
· Heldreth et al.
· 2007
[cited by applicant]
US 20070035511A1
· Banerjee et al.
· 2007
[cited by applicant]
US 20070038944A1
· Carignano et al.
· 2007
[cited by applicant]
US 20070236514A1
· Agusanto et al.
· 2007
[cited by applicant]
US 20070276234A1
· Shahidi
· 2007
[cited by applicant]
US 20090068620A1
· Knobel et al.
· 2009
[cited by applicant]
US 20090089081A1
· Haddad
· 2009
[cited by applicant]
US 20090138019A1
· Wasielewski
· 2009
[cited by applicant]
US 20090267805A1
· Jin et al.
· 2009
[cited by applicant]
US 20110190637A1
· Knobel et al.
· 2011
[cited by applicant]
US 20130093829A1
· Rosenblatt et al.
· 2013
[cited by applicant]
US 20130116574A1
· Knobel et al.
· 2013
[cited by applicant]
US 20130296682A1
· Clavin et al.
· 2013
[cited by applicant]
US 20130326364A1
· Latta et al.
· 2013
[cited by applicant]
US 20140081659A1
· Nawana et al.
· 2014
[cited by applicant]
US 20140085203A1
· Kobayashi
· 2014
[cited by applicant]
US 20140135746A1
· Schoepp
· 2014
[cited by applicant]
US 20140198190A1
· Okumu
· 2014
[cited by applicant]
US 20150100067A1
· Cavanagh et al.
· 2015
[cited by applicant]
US 20150206218A1
· Banerjee et al.
· 2015
[cited by applicant]
US 20150366628A1
· Ingmanson
· 2015
[cited by applicant]
US 20160163105A1
· Hong et al.
· 2016
[cited by applicant]
US 20160228193A1
· Moctezuma de la Barrera et al.
· 2016
[cited by applicant]
US 20160287337A1
· Aram et al.
· 2016
[cited by applicant]
US 20160324580A1
· Esterberg
· 2016
[cited by applicant]
US 20180049622A1
· Ryan et al.
· 2018
[cited by applicant]
US 20190000564A1
· Navab et al.
· 2019
[cited by applicant]
US 20190110842A1
· Lang
· 2019
[cited by applicant]
US 20190216452A1
· Nawana et al.
· 2019
[cited by applicant]
US 20190262078A1
· Lang
· 2019
[cited by applicant]
US 20200246074A1
· Lang
· 2020
[cited by applicant]
CN 101262830A
· 2008
[cited by applicant]
CN 101420911A
· 2009
[cited by applicant]
EP 1028659B1
· 2004
[cited by applicant]
GB 2498833B
· 2016
[cited by applicant]
WO 1993025157A1
· 1993
[cited by applicant]
WO 2005088539A2
· 2005
[cited by applicant]
WO 2010034117A1
· 2010
[cited by applicant]
WO 2014057352A1
· 2014
[cited by applicant]
WO 2015110859A1
· 2015
[cited by applicant]
WO 2015145395A1
· 2015
[cited by applicant]
WO 2016028828A1
· 2016
[cited by applicant]
WO 2016162789A2
· 2016
[cited by applicant]
WO 2016195401A1
· 2016
[cited by applicant]
WO 2016207628A1
· 2016
[cited by applicant]
WO 2017160651A1
· 2017
[cited by applicant]
WO 2018052966A8
· 2018
[cited by applicant]
WO 2018085417A1
· 2018
[cited by applicant]
WO 2018085691A1
· 2018
[cited by applicant]
3D Optical Microscopy for Orthopedic Implants; Bruker Nano Surfaces, Jun. 17, 2016.
[cited by applicant]
“A Look into the Body—Augmented Reality in Computer Aided Surgery,” Department of Informatics, Research—Highlights, Technische Universitat Munchen.
[cited by applicant]
Abe et al., “A Novel 3D Guidance System Using Augmented Reality for Percutaneous Vertebroplasty,” Journal of Neurosurgery, Oct. 2013, vol. 19, pp. 492-501.
[cited by applicant]
Aguerreche et al., “Reconfigurable Tangible Devices for 3D Virtual Object Manipulation by Single or Multiple Users.” VRST 2010, Nov. 22-24, 2010, Hong Kong, Hong Kong SAR China. inria-00534095.
[cited by applicant]
Aichert et al., “Image-Based Tracking of the Teeth for Orthodontic Augmented Reality,” Medical Image Computing and Computer-Assisted Intervention, Lecture Notes in Computer Science, 2012, vol. 7511, pp. 601-608.
[cited by applicant]
Anderson et al., “Virtual annotations of the surgical field through an augmented reality transparent display,” The Visual Computer, Nov. 2016, vol. 32, Issue 11, pp. 1481-1498.
[cited by applicant]
Armstrong et al., “A Heads-Up Display for Diabetic Limb Salvage Surgery: A View Through the Google Looking Glass,” Journal of Diabetes Science and Technology, 2014, vol. 8, No. 5, pp. 951-956.
[cited by applicant]
Azuma, Ronald T., “A survey of augmented reality,” Presence: Teleoperators and Virtual Environments, Aug. 1997, vol. 6, No. 4, pp. 355-385.
[cited by applicant]
Bajura et al., “Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery Within the Patient.,” ACM SIGGRAPH Computer Graphics, Jul. 1992, vol. 26, pp. 203-210.
[cited by applicant]
Baker et al., “The Emergence of Augmented Reality in Orthopaedic Surgery and Education,” The Orthopaedic Journal at Harvard Medical School, Jun. 2015, vol. 16, pp. 8-16.
[cited by applicant]
Bauer et al., “Joint ToF Image Denoising and Registration with a CT Surface in Radiation Therapy,” Scale Space and Variational Methods in Computer Vision, Lecture Notes in Computer Science, Springer, May/Jun. 2011, vol.…
[cited by applicant]
Bauer et al., “Multi-Modal Surface Registration for Markerless Initial Patient Setup in Radiation Therapy Using Microsoft's Kinect Sensor,” 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops…
[cited by applicant]
Bauer et al., “Real-Time Range Imaging in Health Care: A Survey,” Time-of-Flight and Depth Imaging, Sensors, Algorithms, and Applications. Lecture Notes in Computer Science, 2017, vol. 8200, pp. 228-254.
[cited by applicant]
Bauer, Sebastian, Doctoral Thesis, “Rigid and Non-Rigid Surface Registration for Range Imaging Applications in Medicine,” um:nbn:de:bvb:29-opus4-54665, Nov. 27, 2014, pp. 1-198.
[cited by applicant]
Benford et al., “User embodiment in collaborative virtual environments,” CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 7-11, 1995, pp. 242-249.
[cited by applicant]
Besl et al., “A method for registration of 3-D shapes,” IEEE Trans PAMI, Feb. 1992, vol. 14, pp. 239-256.
[cited by applicant]
Bichlmeier et al., “Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality.,” 2007 6th IEEE and ACM International Symposium on Mixed and…
[cited by applicant]
Bichlmeier et al., “Virtually Extended Surgical Drilling Device: Virtual Mirror for Navigated Spine Surgery,” MICCAI 2007, Part I, LNCS 4791, 2007, pp. 434-441.
[cited by applicant]
Billinghurst et al., “Collaborative Augmented Reality,” Communications of the ACM, 2002, vol. 45, pp. 64-70.
[cited by applicant]
Billinghurst et al., “Collaborative Mixed Reality,” Proceedings of the First International Symposium on Mixed Reality (ISMR '99), Mixed Reality—Merging Real and Virtual Worlds, 1999, pp. 261-284.
[cited by applicant]
Billinghurst et al., “Experiments with Face-to-Face Collaborative AR Interfaces,” Virtual Reality Journal, 2002, vol. 6, pp. 107-121.
[cited by applicant]
Billinghurst et al., “The MagicBook: A Transitional AR Interface,” Computers and Graphics, Oct. 2001, pp. 745-753.
[cited by applicant]
Birkfellner et al., “A Head-Mounted Operating Binocular for Augmented Reality Visualization in Medicine—Design and Initial Evaluation,” IEEE Transactions on Medical Imaging, vol. 21, Aug. 2002, pp. 991-997.
[cited by applicant]
Birkfellner et al., “Computer-enhanced stereoscopic vision in a head-mounted operating binocular,” Physics in Medicine & Biology, Feb. 7, 2003, vol. 48, pp. N49-57.
[cited by applicant]
Birkfellner et al., “In-Vitro Assessment of a Registration Protocol for Image Guided Implant Dentistry,” Clinical Oral Implants Research, Feb. 2001, vol. 12, pp. 69-78.
[cited by applicant]
Blackwell et al., “An Image Overlay System for Medical Data Visualization,” Medical Image Analysis, 2000, vol. 4, pp. 67-72.
[cited by applicant]
Blackwell et al., “An Image Overlay System for Medical Data Visualization,” Medical Image Computing and Computer-Assisted Intervention—MICCAI '98, MICCAI 1998, Lecture Notes in Computer Science, Oct. 11-13, 1998, vol. 1…
[cited by applicant]
Blackwell et al., “Augmented Reality and Its Future in Orthopaedics,” Clinical Orthopaedics & Related Research, Sep. 1998, vol. 354, pp. 111-122.
[cited by applicant]
Catani et al., Knee Surgery Using Computer Assisted Surgery and Robotics, [Textbook], 2013, Springer Berlin, Heidelberg, Germany.
[cited by applicant]
Chandak, Ashok S., “MEMS Based Wireless Controlled Robot with Voice and Video Camera,” International Journal of Scientific & Engineering Research, Apr. 2014, vol. 5, p. 456.
[cited by applicant]
Charbonnier et al., “Real Virtuality: Perspectives offered by the combination of Virtual Reality headsets and Motion Capture,” Artanim, Real Virtuality White Paper, Aug. 23, 2015, pp. 1-8.
[cited by applicant]
Chen et al., “Development of a surgical navigation system based on augmented reality using an optical seethrough head-mounted display”, Journal of Biomedical Informatics, 2015, vol. 55, pp. 124-131.
[cited by applicant]
Cruz-Neira et al., “The Cave: Audio Visual Experience Automatic Virtual Environment.,” Communications of the ACM, Jun. 1992, vol. 35, pp. 64-72.
[cited by applicant]
Cui et al., “KinectAvatar: Fully Automatic Body Capture Using a Single Kinect,” Computer Vision—ACCV 2012 Workshops, ACCV 2012, Lecture Notes in Computer Science, Nov. 2012, vol. 7729, pp. 133-147.
[cited by applicant]
Daniel et al., “Augmented Reality for Assistance of Total Knee Replacement,” Journal of Electrical and Computer Engineering, 2016, vol. 2016, Article 9358369, pp. 1-6.
[cited by applicant]
Davies et al., “Computer Assisted Orthopaedic Surgery,” 8th Annual Meeting of CAOS—International Proceedings, 2008, pp. 1-490.
[cited by applicant]
De Lambert et al., “Electromagnetic Tracking for Registration and Navigation in Endovascular Aneurysm Repair: A Phantom Study”, European Journal of Vascular and Endovascular Surgery, 2012, vol. 43, pp. 684-689.
[cited by applicant]
Draelos, Mark Theodore, “The Kinect Up Close: Modifications for Short-Range Depth Imaging,” NC State Theses and Dissertations, Mar. 26, 2012, pp. 1-101.
[cited by applicant]
Elmi-Terander et al., “Surgical Navigation Technology Based on Augmented Reality and Integrated 3D Intraoperative Imaging,” Spine, 2016, vol. 41, pp. E1303-1311.
[cited by applicant]
Ferrari et al., “Video See-Through in the Clinical Practice,” Proceedings of the 1st International Workshop on Engineering Interactive Computing Systems for Medicine and Health Care, EICS4Med 2011, Jun. 2011, pp. 19-24.
[cited by applicant]
Fischer et al., “Medical Augmented Reality Based on Commercial Image Guided Surgery,” EGVE '04: Proceedings of the 10th Eurographics Symposium on Virtual Environments, Jun. 2004, pp. 83-86.
[cited by applicant]
Fitzmaurice et al., “Bricks: Laying the Foundations for Graspable User Interfaces.,” CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 7-11, 1995, pp. 442-449.
[cited by applicant]
Flusser et al., “Image Fusion: Principles, Methods and Applications,” Tutorial EUSIPCO 2007 Lecture Notes, 2007, pp. 1-60.
[cited by applicant]
Fritz et al., “Augmented Reality Visualization with Image Overlay for MRI-Guided Intervention: Accuracy for Lumbar Spinal Procedures with a 1.5-T MRI System,” American Journal of Roentgenology, Mar. 2012, vol. 198, pp. …
[cited by applicant]
Fritz et al., “Augmented Reality Visualization with Use of Image Overlay Technology for MR Imaging-Guided Interventions: Assessment of Performance in Cadaveric Shoulder and Hip Arthrography at 1.5T,” Radiology, Oct. 201…
[cited by applicant]
Garon et al., “Real-time High Resolution 3D Data on the HoloLens,” 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR—Adjunct), 2016, pp. 189-191.
[cited by applicant]
Garrido-Jurado et al., “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, 2014, vol. 47, pp. 2280-2292.
[cited by applicant]
Gavaghan et al., “Augmented Reality Image Overlay Projection for Image Guided Open Liver Ablation of Metastatic Liver Cancer,” Augmented Environments for Computer-Assisted Interventions, AE-CAI 2011, Lecture Notes in Co…
[cited by applicant]
Gee et al., “Processing and visualizing three-dimensional ultrasound data,” The British Journal of Radiology, 2004, vol. 77, S186-S193.
[cited by applicant]
George et al., “Low Cost Augmented Reality for Training of MRI-Guided Needle Biopsy of the Spine,” Studies in Health Technology and Informatics, 2008, vol. 132, pp. 138-140.
[cited by applicant]
Germano, Isabelle M., Advanced Techniques in Image-Guided Brain and Spine Surgery, [Textbook], 2002, Thieme Medical Publishers, Inc., New York.
[cited by applicant]
González et al., “Smart Multi-Level Tool for Remote Patient Monitoring Based on a Wireless Sensor Network and Mobile Augmented Reality,” Sensors, 2014, vol. 14, pp. 17212-17234.
[cited by applicant]
Gorbet et al., “Triangles: Tangible Interface for Manipulation and Exploration of Digital Information Topography,” CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Apr. 18-23, 1998, p…
[cited by applicant]
Gromov et al., “What is the optimal alignment of the tibial and femoral components in knee arthroplasty? An overview of the literature,” Acta Orthopaedica, 2014, vol. 85, pp. 480-487.
[cited by applicant]
Hayashibe et al., “Surgical Navigation Display System Using Volume Rendering of Intraoperatively Scanned CT Images,” Computer Aided Surgery, Sep. 2006, vol. 11, pp. 240-246.
[cited by applicant]
Hinterstoisser et al., “Gradient Response Maps for Real-Time Detection of Texture-Less Objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, May 2012, vol. 34, pp. 876-888.
[cited by applicant]
Hinterstoisser et al., “Learning Real-Time Perspective Patch Rectification,” International Journal of Computer Vision, 2011, vol. 91, pp. 107-130.
[cited by applicant]
Hinterstoisser et al., “Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes,” Computer Vision—ACCV 2012 Workshops, ACCV 2012, Lecture Notes in Computer Science, Nov…
[cited by applicant]
Hinterstoisser et al., “Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes” 2011 International Conference on Computer Vision, Barcelona, Spain, 2011, pp. 858-865.
[cited by applicant]
Hoff, William A., “Fusion of Data from Head-Mounted and Fixed Sensors,” First International Workshop on Augmented Reality, Nov. 1, 1998, pp. 1-20.
[cited by applicant]
Holographic weapon sight—Wikipedia https://en.wikipedia.org/wiki/Holographic_weapon_sight retrieved on Nov. 22, 2016.
[cited by applicant]
Hu et al., “A Convenient Method of Video See-through Augmented Reality Based on Image-Guided Surgery System,” 2013 Seventh International Conference on Internet Computing for Engineering and Science, Shanghai, China, 201…
[cited by applicant]
Hua et al., “A 3D Integral Imaging Optical See-Through Head-Mounted Display,” Optics Express, Jun. 2, 2014, vol. 22, pp. 13484-13491.
[cited by applicant]
Ishii et al., “Iterative Design of Seamless Collaboration Media,” Communications of the ACM, Aug. 1994, vol. 37, pp. 83-97.
[cited by applicant]
Jamali et al., “Utilising Mobile-Augmented Reality for Learning Human Anatomy,” Procedia—Social and Behavioral Sciences, 2015, vol. 197, pp. 659-668.
[cited by applicant]
Ji et al., “Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance,” Real-Time Imaging, 2002, vol. 8, pp. 357-377.
[cited by applicant]
Jiang et al., “A Robust Automated Markerless Registration Framework for Neurosurgery Navigation,” The International Journal of Medical Robotics and Computer Assisted Surgery, 2015, vol. 11, pp. 436-447.
[cited by applicant]
Jolesz, Ferenc A., Intraoperative Imaging and Image-Guided Therapy, [Textbook], Jan. 14, 2014, Springer Science & BusinessMedia, New York, NY.
[cited by applicant]
Kanade et al., “Simulation, Planning, and Execution of Computer-Assisted Surgery,” Proceedings of the NSF Grand Challenges Workshop, 1996, pp. 1-3.
[cited by applicant]
Kato et al., “Marker tracking and HMD calibration for a video-based augmented reality conferencing system,” Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR '99), San Francisco, CA, USA, 19…
[cited by applicant]
Kersten-Oertel et al., “The State of the Art of Visualization in Mixed Reality Image Guided Surgery,” Computerized Medical Imaging and Graphics, 2013, vol. 37, pp. 98-112.
[cited by applicant]
Kim et al., “Registration Accuracy Enhancement of a Surgical Navigation System for Anterior Cruciate Ligament Reconstruction: A Phantom and Cadaveric Study,” The Knee, 2017, vol. 24, pp. 329-339.
[cited by applicant]
Kolodzey et al., “Wearable technology in the operating room: a systematic review”; BMJ Innovations, 2017, vol. 3, pp. 55-63.
[cited by applicant]
Kumar et al., “A Portable Wireless Head Movement Controlled Human-Computer Interface for People with Disabilities,” International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering, …
[cited by applicant]
Kutter et al., “Real-time Volume Rendering for High Quality Visualization in Augmented Reality,” International Workshop on Augmented Environments for Medical Imaging Including Augmented Reality in Computer-Aided Surgery…
[cited by applicant]
Lamata et al., “Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances,” Augmented Reality, Jan. 2010, pp. 73-98.
[cited by applicant]
Liao et al., “3-D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay,” IEEE Transactions on Biomedical Engineering, Jun. 2010, vol. 57, pp. 1476-1486.
[cited by applicant]
Liao et al., “Surgical Navigation by Autostereoscopic Image Overlay of Integral Videography,” IEEETransactions on Information Technology in Biomedicine, Jun. 2004, vol. 8, pp. 114-121.
[cited by applicant]
Liévin et al., “Stereoscopic Augmented Reality System for Computer-Assisted Surgery,” International Congress Series, 2001, vol. 1230, pp. 107-111.
[cited by applicant]
Linte et al., “On Mixed Reality Environments for Minimally Invasive Therapy Guidance: Systems Architecture, Successes and Challenges in Their Implementation From Laboratory to Clinic,” Computurized Medical Imaging Graph…
[cited by applicant]
Liu et al., “An Optical See-Through Head Mounted Display with Addressable Focal Planes,” 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, 2008, pp. 33-42.
[cited by applicant]
Lorensen et al., “Marching cubes: A high resolution 3d surface construction algorithm,” Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '87), 1987, pp. 163-169.
[cited by applicant]
Maier-Hein et al., “Optical Techniques for 3D Surface Reconstruction in Computer-Assisted Laparoscopic Surgery,” Medical Image Analysis, 2013, vol. 17, pp. 974-996.
[cited by applicant]
Maier-Hein et al., “Towards Mobile Augmented Reality for On-Patient Visualization of Medical Images,” Bildverarbeitung für die Medizin 2011: Algorithmen—Systeme—Anwendungen, 2011, pp. 389-393.
[cited by applicant]
Masamune et al., “An Image Overlay System with Enhanced Reality for Percutaneous Therapy Performed Inside CT Scanner,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2002, Lecture Notes in Computer Sc…
[cited by applicant]
Maurer, Jr. et al., “Augmented-Reality Visualization of Brain Structures with Stereo and Kinetic Depth Cues: System Description and Initial Evaluation with Head Phantom,” Medical Imaging 2001: Visualization, Display, an…
[cited by applicant]
Medeiros et al., “Proposal and evaluation of a tablet-based tool for 3D virtual environments.,” SBC Journal on 3D Interactive Systems, 2013, vol. 4, pp. 30-40.
[cited by applicant]
Melzer, James E., “Head-Mounted Displays,” The Avionics Handbook, 2001, pp. 1-21.
[cited by applicant]
Menozzi et al., “Development of Vision-Aided Navigation for a Wearable Outdoor Augmented Reality System,” 2014 IEEE/ION Position, Location and Navigation Symposium—PLANS 2014, Monterey, CA, USA, 2014, pp. 460-472.
[cited by applicant]
Micro Vision 2015 Annual Report and Proxy Statement for 2016 Annual Meeting of Shareholders, 2016, pp. 1-103.
[cited by applicant]
Moore et al., “Image Guidance for Spinal Facet Injections Using Tracked Ultrasound,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2009, 2009, vol. 5761, pp. 516-523.
[cited by applicant]
Müller et al., “Automatic Multi-Modal ToF/CT Organ Surface Registration,” Bildverarbeitung für die Medizin, 2011, pp. 154-158.
[cited by applicant]
Newcombe et al., “KinectFusion: Real-time dense surface mapping and tracking,” 2011 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, 2011, pp. 127-136.
[cited by applicant]
Nicolau et al., “Augmented Reality in Laparoscopic Surgical Oncology,” Surgical Oncology, 2011, vol. 20, pp. 189-201.
[cited by applicant]
Nikou et al., “Augmented Reality Imaging Technology for Orthopaedic Surgery,” Operative Techniques in Orthopaedics, Jan. 2000, vol. 10, pp. 82-86.
[cited by applicant]
Noonan et al., “The Design and Initial Calibration of an Optical Tracking System Using the Microsoft Kinect,” IEEE Nuclear Science Symposium Conference Record, 2011, pp. 3614-3617.
[cited by applicant]
Okamura, Allison, “Tracking and Surgical Navigation, Registration,” Stanford Lecture 8: ME 328: Medical Robotics, Spring 2013, pp. 1-19.
[cited by applicant]
Ortega et al., “Usefulness of a head mounted monitor device for viewing intraoperative fluoroscopy during orthopaedic procedures,” Archives of Orthopaedic and Trauma Surgery, 2008, vol. 128, pp. 1123-1126.
[cited by applicant]
Paprosky et al., “Intellijoint HIP®: a 3D mini-optical navigation tool for improving intraoperative accuracy during total hip arthroplasty,” Medical Devices: Evidence and Research, 2016, vol. 9, pp. 401-408.
[cited by applicant]
Pauly et al., “Machine Learning-Based Augmented Reality for Improved Surgical Scene Understanding,” Computerized Medical Imaging and Graphics, 2015, vol. 41, pp. 55-60.
[cited by applicant]
Peters et al., Image-Guided Interventions, Technology and Applications, [Textbook], 2008, Springer Science and Business Media, New York, NY.
[cited by applicant]
Ponce et al., “Emerging Technology in Surgical Education: Combining Real-Time Augmented Reality and Wearable Computing Devices,” The Cutting Edge, Nov. 2014, vol. 37, pp. 751-757.
[cited by applicant]
Qian et al., “Comprehensive Tracker Based Display Calibration for Holographic Optical See-Through Head-Mounted Display,” ArXiv preprint, 2017, arXiv:1703.05834.
[cited by applicant]
Ren et al., “Marker-Based Surgical Instrument Tracking Using Dual Kinect Sensors,” IEEE Transactions on Automation Science and Engineering, Jul. 2014, vol. 11, pp. 921-924.
[cited by applicant]
Rhodes, Bradley, “A brief history of wearable computing,” MIT Wearable Computing Project, 1997.
[cited by applicant]
Rinaldi et al., Computer-Guided Applications for Dental Implants, Bone Grafting, and Reconstructive Surgery, [Textbook], 2015, Elsevier Inc., Amsterdam, The Netherlands.
[cited by applicant]
Robinett et al., “A Computational Model for the Stereoscopic Optics of a Head-Mounted Display,” Proceedings of SPIE, Stereoscopic Displays and Applications II, 1991, vol. 1457, pp. 140-160.
[cited by applicant]
Rolland et al., “A Comparison of Optical and Video See-through Head-mounted Displays,” Proceedings of SPIE, 1995, vol. 2351, Telemanipulator and Telepresence Technologies, pp. 293-307.
[cited by applicant]
Rolland et al., “Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization,” Presence: Teleoperators and Virtual Environments, Jun. 2000, vol. 9, pp. 287-309.
[cited by applicant]
Rosenthal et al., “Augmented Reality Guidance for Needle Biopsies: A Randomized, Controlled Trial in Phantoms,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2001, Lecture Notes in Computer Science, …
[cited by applicant]
Rosman et al., “Articulated Motion Segmentation of Point Clouds by Group-Valued Regularization,” Eurographics Workshop on 3D Object Retrieval, EG 3DOR, May 2012, pp. 77-84.
[cited by applicant]
Sanko, “Microvision's Nomad Augmented Vision System: The How and the Why” SID Pacific Northwest Chapter Meeting, Jun. 11, 2003.
[cited by applicant]
Sauer et al., “An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2002, Lecture Notes i…
[cited by applicant]
Sauer et al., “Augmented Workspace: Designing an AR Testbed,” Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000), 2000, Munich, Germany, pp. 47-53.
[cited by applicant]
Schramm, Mike, “Kinect: The Company Behind the Tech Explains How it Works,” Jun. 19, 2010, https://www.engadget.com/2010/06/19/kinect-how-it-works-from-the-company-behind-the-tech/?guccounter= I&guce_referrer=aHR0cHM6Ly…
[cited by applicant]
Scuderi et al., “Total Knee Arthroplasty with a Novel Navigation System Within the Surgical Field,” Orthopedic Clinics, Apr. 2014, vol. 45, pp. 167-173.
[cited by applicant]
Shen et al., “3D Augmented Reality with Integral Imaging Display,” Proceedings of SPIE—The International Society for Optical Engineering, 2016, vol. 9867, pp. 267-270.
[cited by applicant]
Sherstyuk et al., “Dynamic Eye Convergence for Head-Mounted Displays Improves User Performance in Virtual Environments,” Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, 2012, pp. 23-30.
[cited by applicant]
State et al., “Stereo Imagery from the UNC Augmented Reality System for Breast Biopsy Guidance,” Medicine Meets Virtual Reality 11, 2003, pp. 325-328, IOS Press.
[cited by applicant]
Tan et al., “A Versatile Learning-Based 3D Temporal Tracker: Scalable, Robust, Online,” 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 693-701.
[cited by applicant]
Tong et al., “Scanning 3D Full Human Bodies Using Kinects,” IEEE Transactions on Visualization and Computer Graphics, Apr. 2012, vol. 18, pp. 643-650.
[cited by applicant]
Traub et al., “Hybrid Navigation Interface for Orthopedic and Trauma Surgery,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006, Lecture Notes in Computer Science, 2006, vol. 4190, pp. 373-380.
[cited by applicant]
Trevisan et al., “Towards Markerless Augmented Medical Visualization,” AMI-ARCS '04, 2004, pp. 57-66.
[cited by applicant]
Vagvolgyi et al., “Video to CT Registration for Image Overlay on Solid Organs,” Procedural Augmented Reality in Medical Imaging and Augmented Reality in Computer-Aided Surgery (AMIARCS), 2008, pp. 78-86.
[cited by applicant]
Van Lindert et al., “The use of a head-mounted display for visualization in neuroendoscopy,” Computer Aided Surgery, 2004, vol. 9, pp. 251-256.
[cited by applicant]
Vercauteren et al., “Real Time Autonomous Video Image Registration for Endomicroscopy: Fighting the Compromises,” Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XV, 2008, vol. 6861, …
[cited by applicant]
Vogt et al., “Reality Augmentation for Medical Procedures: System Architecture, Single Camera Marker Tracking, and System Evaluation,” International Journal of Computer Vision, 2006, vol. 70, pp. 179-190.
[cited by applicant]
Vogt, Sebastian, “Real-Time Augmented Reality for Image-Guided Interventions,” PhD Thesis, Der Technischen Fakultät der Universität Erlangen-Nürnberg, 2009, pp. 1-211.
[cited by applicant]
Wang et al., “Augmented Reality 3D Displays with Micro Integral Imaging,” Journal of Display Technology, 2014, vol. 11, pp. 889-893.
[cited by applicant]
Wang et al., “3D Modeling from Wide Baseline Range Scans Using Contour Coherence,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 4018-4025.
[cited by applicant]
Wang et al., “Augmented Reality Navigation with Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery,” IEEE Transactions on Biomedical Engineering, Apr. 2014, vol. 61, pp. 1295-1304.
[cited by applicant]
Wang et al., “Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: a pilot study,” International Orthopaedics, 2016, vol. 40, pp. 1941-1947.
[cited by applicant]
Watsen et al., “A Handheld Computer as an Interaction Device to a Virtual Environment,” Proceedings of the International Projection Technologies Workshop, Stuttgart, Germany, May 10-11, 1999, pp. 303-304.
[cited by applicant]
Weiss et al., “Augmented Reality Visualization Using Image-Overlay for MR-Guided Interventions: System Description, Feasibility, and Initial Evaluation in a Spine Phantom,” AJR—American Journal of Roentgenology, Mar. 20…
[cited by applicant]
Wellner, Pierre, “Interacting with Paper on the DigitalDesk,” Communications of the ACM, 1993, vol. 36, pp. 87-96.
[cited by applicant]
Wilson et al., “Validation of Three-Dimensional Models of the Distal Femur Created from Surgical Navigation Point Cloud Data,” CAOS 2015, 2015, pp. 1-4.
[cited by applicant]
Yamazaki et al., “Gesture Laser and Gesture Laser Car-Development of an Embodied Space to Support Remote Instruction.,” Proceedings of the Sixth European Conference on Computer Supported Cooperative Work—ECSCW '99, Sep.…
[cited by applicant]
Yang et al., “Exploring collaborative navigation: the effect of perspectives on group performance,” Proceedings of the 4th International Conference on Collaborative Virtual Environments, CVE '02, 2002, pp. 135-142.
[cited by applicant]
Ye et al., “Accurate 3D Pose Estimation from a Single Depth Image,” 2011 International Conference on Computer Vision, Barcelona, Spain, 2011, pp. 731-738.
[cited by applicant]
Yoon et al., “Technical Feasibility and Safety of an Intraoperative Head-Up Display Device During Spine Instrumentation,” The International Journal of Medical Robotics and Computer Assisted Surgery, 2017, vol. 13, p. e1…
[cited by applicant]