Granted Patent
B1
US 12,433,761 · App. 18/099,736 · Granted Oct 7, 2025
Systems and methods for determining the shape of spinal rods and spinal interbody devices for use with augmented reality displays, navigation systems and robots in minimally invasive spine procedures
Inventors:
Chuang-Jang Chiou (Bedford, MA); Daniel Steines (Lexington, MA); Philipp K. Lang (Franconia, NH)
Assignee:
OnPoint Medical, Inc.
View Patent ↗
Loading inventors, assignments & file history…
Abstract
Systems, techniques, devices and methods are described for determining an axis, a curvature, and/or a 3D shape of spinal rods for optional display on a computer monitor and/or an augmented reality display. Systems, techniques, devices and methods are described for determining a leading edge, a curvature, and/or a 3D shape of interbody devices for optional display on a computer monitor and/or an augmented reality display.
Claims (48)
1. A system for performing spinal fusion surgery, the system comprising:
a tracking system for tracking one or more markers;
a computer processor;
a physical spinal rod having a length;
an inserter configured for insertion of the physical spinal rod; and
an engagement member;
wherein the rod inserter is configured for connecting to a proximal end of the physical spinal rod,
wherein the rod inserter comprises a first marker attached to the rod inserter,
wherein the engagement member comprises a second marker attached to the engagement member,
wherein the engagement member is configured to slideably engage with the physical spinal rod,
wherein the tracking system is configured to track the engagement member during movement of the engagement member from the proximal end to a distal end or from the distal end to the proximal end along a length of the physical spinal rod, and
wherein the computer processor is configured to determine an axis, a curvature, a shape or a combination thereof of the physical spinal rod based on tracking information of the engagement member in relationship to the spinal rod connected to the tracked rod inserter.
2. The system of claim 1 ,
wherein the computer processor is configured to receive information about a radius or a diameter of the physical spinal rod,
wherein the computer processor is configured to determine a shape of the physical spinal rod based on the tracking information generated by the movement of the second marker attached to the engagement member and the information about the radius or the diameter.
3. The system of claim 2 , wherein the axis is a central axis of the physical spinal rod and the shape is determined, by the computer processor, based on the radius or the diameter and the central axis.
4. The system of claim 1 , wherein the system comprises an augmented reality device, wherein the system is configured to display, by the augmented reality device, a virtual spinal rod based on the determined axis, curvature, or shape, or combination thereof of the physical spinal rod, wherein the virtual spinal rod is a virtual representation of the physical spinal rod.
5. The system of claim 4 , wherein the virtual representation comprises at least a portion of a surface of the physical spinal rod.
6. The system of claim 4 , wherein the augmented reality device comprises an optical see-through head mounted device or a video see-through head mounted device.
7. The system of claim 4 , wherein the system is configured to display at least a portion of at least one virtual spinal screw, interbody device, physical plate or combination thereof implanted in a patient, wherein the at least one virtual device is virtual representations of a physical spinal screw, a physical interbody device, or a physical plate or combination thereof.
8. The system of claim 4 , wherein system is configured to display at least a portion of a virtual screw corresponding to at least a portion of physical spinal screw, at least a portion of a virtual interbody device corresponding to at least a portion of a physical interbody device, at least a portion of a virtual plate corresponding to at least a portion of a physical plate, at least a portion of a virtual spinal rod corresponding to at least a portion of the physical spinal rod, or a combination thereof.
9. The system of claim 8 ,
wherein the at least a portion of the virtual screw corresponding to at least a portion of physical spinal screw is superimposed, aligned or superimposed and aligned with the at least portion of the physical screw, or
wherein the at least a portion of the virtual interbody device corresponding to at least a portion of the physical interbody device is superimposed, aligned or superimposed and aligned with the at least portion of the physical interbody device, or
wherein the at least a portion of the virtual plate corresponding to at least a portion of the physical plate is superimposed, aligned or superimposed and aligned with the at least portion of the physical plate, or
wherein the at least a portion of the virtual spinal rod corresponding to at least a portion of the physical rod is superimposed, aligned or superimposed and aligned with the at least portion of the physical spinal rod.
10. The system of claim 9 , wherein the physical interbody device is attached to an inserter, or wherein the physical spinal rod is attached to the inserter.
11. The system of claim 4 , wherein the system is configured to display a receiving portion of a virtual screw tulip, and wherein the receiving portion of the virtual screw tulip is configured to receive the virtual spinal rod.
12. The system of claim 11 , wherein the display of the virtual screw tulip comprises a virtual representation of a thread or of a fastening mechanism for a locking screw for fastening the physical spinal rod.
13. The system of claim 4 , wherein the augmented reality device is configured to provide a magnified view of the virtual spinal rod, a virtual spinal screw, a virtual screw tulip, a spinal rod receiving portion of the virtual screw tulip, or a combination thereof.
14. The system of claim 11 , wherein the system is configured to facilitate advancing the virtual spinal rod towards the receiving portion of one or more virtual screw tulip, wherein the system is configured to keep the virtual spinal rod superimposed and aligned with the physical spinal rod during the advancing.
15. The system of claim 1 , wherein the tracking system is configured to determine one or more coordinates of the one or more markers.
16. The system of claim 1 , wherein the one or more markers comprise at least one optical marker, at least one geometric pattern, at least one retroreflective marker, at least one infrared marker, at least one radiofrequency emitting and/or receiving marker, at least one light emitting diode, at least one inertial measurement unit or a combination thereof.
17. A system for performing spinal surgery, the system comprising:
a tracking system for tracking one or more markers;
a computer processor;
an inserter configured for insertion of a physical spinal interbody device;
a physical spinal interbody device;
a registration device;
wherein the inserter is configured for connecting to the physical spinal interbody device,
wherein the inserter comprises a first marker attached to inserter,
wherein the registration device comprises a second marker attached to the registration device,
wherein the registration device comprises a surface for slideable, rotatable, and/or moveable engagement of the physical spinal interbody device with the registration device,
wherein the tracking system is configured to track the inserter during movement of the physical spinal interbody device on the surface of the registration device,
wherein the computer processor is configured to determine at least a portion of a shape of the physical spinal interbody device based on tracking information of the inserter with the connected physical interbody device in relationship to the tracking information of the second marker attached to the registration device.
18. The system of claim 17 , wherein the at least a portion of the shape is an edge of the physical spinal interbody device, wherein the edge is at least a portion of a leading edge of the physical spinal interbody device configured for insertion into an intervertebral disc space.
19. The system of claim 18 , wherein the computer processor is configured to determine a dimension of the physical spinal interbody device based on a distance between the leading edge of the physical spinal interbody device and the attachment of the physical spinal interbody device to the inserter.
20. The system of claim 19 , wherein the dimension is a width or a depth of the physical spinal interbody device.
Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST
Recorded May 12, 2023
From: CHIOU, CHUANG-JANG; STEINES, DANIEL; LANG, PHILIPP K.
To: ONPOINT MEDICAL, INC.
Reel/Frame 063629/0024 →
Continuity (1)
Provisional Application
63301169
· Jan 20, 2022
References Cited (317)
US 5526812A
· Dumoulin et al.
· 1996
[cited by applicant]
US 5676673A
· Ferre et al.
· 1997
[cited by applicant]
US 5800352A
· Ferre et al.
· 1998
[cited by applicant]
US 5803089A
· Ferre et al.
· 1998
[cited by applicant]
US 5829444A
· Ferre et al.
· 1998
[cited by applicant]
US 5873822A
· Ferre et al.
· 1999
[cited by applicant]
US D415146S
· Hori
· 1999
[cited by applicant]
US 5967980A
· Ferre et al.
· 1999
[cited by applicant]
US 6175756B1
· Ferre et al.
· 2001
[cited by applicant]
US 6341231B1
· Ferre et al.
· 2002
[cited by applicant]
US 6396497B1
· Reichlen
· 2002
[cited by applicant]
US 6445943B1
· Ferre et al.
· 2002
[cited by applicant]
US 6599247B1
· Stetten
· 2003
[cited by applicant]
US 6714810B2
· Grzeszczuk et al.
· 2004
[cited by applicant]
US 7130676B2
· Barrick
· 2006
[cited by applicant]
US 7774044B2
· Sauer et al.
· 2010
[cited by applicant]
US 7812815B2
· Banerjee et al.
· 2010
[cited by applicant]
US 8320612B2
· Knobel et al.
· 2012
[cited by applicant]
US 8730266B2
· Brown et al.
· 2014
[cited by applicant]
US 8989843B2
· Chien
· 2015
[cited by applicant]
US 9068820B2
· Kosmecki et al.
· 2015
[cited by applicant]
US 9068824B2
· Findeisen et al.
· 2015
[cited by applicant]
US 9123155B2
· Cunningham et al.
· 2015
[cited by applicant]
US 9183560B2
· Abelow
· 2015
[cited by applicant]
US 9215293B2
· Miller
· 2015
[cited by applicant]
US 9299138B2
· Zellner et al.
· 2016
[cited by applicant]
US 9310559B2
· Macnamara
· 2016
[cited by applicant]
US 9311284B2
· Warila et al.
· 2016
[cited by applicant]
US 9389424B1
· Schowengerdt
· 2016
[cited by applicant]
US 9417452B2
· Schowengerdt et al.
· 2016
[cited by applicant]
US 9429752B2
· Schowengerdt et al.
· 2016
[cited by applicant]
US 9503681B1
· Popescu et al.
· 2016
[cited by applicant]
US 9547940B1
· Sun et al.
· 2017
[cited by applicant]
US 9582717B2
· Lee et al.
· 2017
[cited by applicant]
US 9792721B2
· Kosmecki et al.
· 2017
[cited by applicant]
US 9861446B2
· Lang
· 2018
[cited by applicant]
US 9901463B2
· Mahfouz
· 2018
[cited by applicant]
US 9913692B2
· Arata et al.
· 2018
[cited by applicant]
US 9918658B2
· McCaulley et al.
· 2018
[cited by applicant]
US 9980780B2
· Lang
· 2018
[cited by applicant]
US 20020016349A1
· Heywang et al.
· 2002
[cited by applicant]
US 20020082498A1
· Wendt et al.
· 2002
[cited by applicant]
US 20050028146A1
· Quick
· 2005
[cited by applicant]
US 20050113846A1
· Carson
· 2005
[cited by applicant]
US 20050215879A1
· Chuanggui
· 2005
[cited by applicant]
US 20050267353A1
· Marquart et al.
· 2005
[cited by applicant]
US 20050281465A1
· Marquart et al.
· 2005
[cited by applicant]
US 20070015999A1
· Heldreth et al.
· 2007
[cited by applicant]
US 20070038944A1
· Carignano et al.
· 2007
[cited by applicant]
US 20070236514A1
· Agusanto et al.
· 2007
[cited by applicant]
US 20070276234A1
· Shahidi
· 2007
[cited by applicant]
US 20090267805A1
· Jin et al.
· 2009
[cited by applicant]
US 20110190637A1
· Knobel et al.
· 2011
[cited by applicant]
US 20130093829A1
· Rosenblatt et al.
· 2013
[cited by applicant]
US 20130116574A1
· Knobel et al.
· 2013
[cited by applicant]
US 20130296682A1
· Clavin et al.
· 2013
[cited by applicant]
US 20130326364A1
· Latta et al.
· 2013
[cited by applicant]
US 20140081659A1
· Nawana et al.
· 2014
[cited by applicant]
US 20140085203A1
· Kobayashi
· 2014
[cited by applicant]
US 20140135746A1
· Schoepp
· 2014
[cited by applicant]
US 20140198190A1
· Okumu
· 2014
[cited by applicant]
US 20150045657A1
· Kim
· 2015
[cited by applicant]
US 20150100067A1
· Cavanagh et al.
· 2015
[cited by applicant]
US 20150206218A1
· Banerjee et al.
· 2015
[cited by applicant]
US 20150366628A1
· Ingmanson
· 2015
[cited by applicant]
US 20160163105A1
· Hong et al.
· 2016
[cited by applicant]
US 20160228193A1
· Moctezuma de la Barrera et al.
· 2016
[cited by applicant]
US 20160287337A1
· Aram et al.
· 2016
[cited by applicant]
US 20160324580A1
· Esterberg
· 2016
[cited by applicant]
US 20180049622A1
· Ryan et al.
· 2018
[cited by applicant]
US 20190000564A1
· Navab et al.
· 2019
[cited by applicant]
US 20190110842A1
· Lang
· 2019
[cited by applicant]
US 20190216452A1
· Nawana et al.
· 2019
[cited by applicant]
US 20190246088A1
· Casas
· 2019
[cited by applicant]
US 20190262078A1
· Lang
· 2019
[cited by applicant]
US 20200107002A1
· Casas
· 2020
[cited by applicant]
US 20200246074A1
· Lang
· 2020
[cited by applicant]
US 20220159227A1
· Casas
· 2022
[cited by applicant]
CN 118102988A
· 2024
[cited by examiner]
EP 1028659
· 2004
[cited by applicant]
GB 2498833
· 2016
[cited by applicant]
WO WO1993025157
· 1993
[cited by applicant]
WO WO2005088539
· 2005
[cited by applicant]
WO WO2010034117
· 2010
[cited by applicant]
WO WO2014057352
· 2014
[cited by applicant]
WO WO2015110859
· 2015
[cited by applicant]
WO WO2015145395
· 2015
[cited by applicant]
WO WO2016028828
· 2016
[cited by applicant]
WO WO2016162789
· 2016
[cited by applicant]
WO WO2016195401
· 2016
[cited by applicant]
WO WO2016207628
· 2016
[cited by applicant]
WO WO2017160651
· 2017
[cited by applicant]
WO WO2018085417
· 2018
[cited by applicant]
WO WO2018085691
· 2018
[cited by applicant]
WO WO2018052966
· 2018
[cited by applicant]
3D Optical Microscopy for Orthopedic Implants; Bruker Nano Surfaces, Jun. 17, 2016.
[cited by applicant]
A Look into the Body—Augmented Reality in Computer Aided Surgery, Department of Informatics, Research-Highlights; Technische Universitat Munchen.
[cited by applicant]
Abe et al., “A Novel 3D Guidance System Using Augmented Reality for Percutaneous Vertebroplasty”, Journal of Neurological Spine, vol. 19, pp. 492-501, Oct. 2013.
[cited by applicant]
Aguerreche L. et al., “Reconfigurable Tangible Devices for 3D Virtual Object Manipulation by Single or Multiple Users.” VRST 2010, Nov. 2010, Hong Kong, Hong Kong SAR China. ffinria-00534095.
[cited by applicant]
Aichert et al., “Image-Based Tracking of the Teeth for Orthodontic Augmented Reality”, Medical Image Computing and Computer-Assisted Intervention, Lecture Notes in Computer Science, vol. 7511, Springer, pp. 601-608, 201…
[cited by applicant]
Anderson et al., “Virtual annotations of the surgical field through an augmented reality transparent display”, The Visual Computer, vol. 32, Issue 11, pp. 1481-1498, Nov. 2016.
[cited by applicant]
Armstrong et al., “A Heads-Up Display for Diabetic Limb Salvage Surgery: A View Through the Google Looking Glass”; Journal of Diabetes Science and Technology 2014, vol. 8(5) 951-956.
[cited by applicant]
Azura, R., “A survey of augmented reality.” Teleoperators and Virtual Environments, vol. 6, Issue 4, Aug. 1997, pp. 355-385.
[cited by applicant]
Bajura, M., et al., “Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery Within the Patient.”, In Proceedings of SIGGRAPH '92, 1992, New York: ACM Press, pp. 203-210.
[cited by applicant]
Baker et al., “The Emergence of Augmented Reality in Orthopaedic Surgery and Education”, The Orthopaedic Journal at Harvard Medical School, vol. 16, pp. 8-16, Jun. 2015.
[cited by applicant]
Bauer et al., “Joint ToF Image Denoising and Registration with a CT Surface in Radiation Therapy”, Scale Space and Variational Methods in Computer Vision, Lecture Notes in Computer Science, Springer, vol. 6667, pp. 98-1…
[cited by applicant]
Bauer et al., “Multi-Modal Surface Registration for Markerless Initial Patient Setup in Radiation Therapy Using Microsoft's Kinect Sensor”, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops…
[cited by applicant]
Bauer et al., “Real-Time Range Imaging in Health Care: A Survey”, Time-of-Flight and Depth Imaging, Sensors, Algorithms, and Applications. Lecture Notes in Computer Science, vol. 8200, pp. 228-254, 2017.
[cited by applicant]
Bauer, Sebastian, Doctoral Thesis, “Rigid and Non-Rigid Surface Registration for Range Imaging Applications in Medicine”, urn:nbn:de:bvb:29-opus4-54665, Nov. 27, 2014.
[cited by applicant]
Benford, S. et al., “User embodiment in collaborative virtual environments”, Proceedings of the SIGCHI conference on Human factors in computing systems, CHI '95, pp. 242-249, 1995.
[cited by applicant]
Besl PJ, McKay ND. 2, 1992. A method for registration of 3-D shapes. IEEE Trans PAMI, vol. 14, pp. 239-256.
[cited by applicant]
Bichlmeier C., et al. “Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality.”, IEEE 2007, 2007 6th IEEE and ACM International Symposiu…
[cited by applicant]
Bichlmeier et al., “Virtually Extended Surgical Drilling Device: Virtual Mirror for Navigated Spine Surgery”; MICCAI 2007, Part I, LNCS 4791, pp. 434-441.
[cited by applicant]
Billinghurst, et al., “The MagicBook: A Transitional AR Interface.”, Computers and Graphics, Nov. 2001, pp. 745-753.
[cited by applicant]
Billinghurst, M., et al., “Collaborative Mixed Reality”, First International Symposium on Mixed Reality (ISMR '99). Mixed Reality—Merging Real and Virtual Worlds, pp. 261-284. Berlin: Springer Verlag.
[cited by applicant]
Billinghurst, M., et al., “Experiments with Face to Face Collaborative AR Interfaces.”, Virtual Reality Journal, vol. 4, No. 2, (2002).
[cited by applicant]
Birkfellner et al., “A Head-Mounted Operating Binocular for Augmented Reality Visualization in Medicine—Design and Initial Evaluation”, IEEE Transactions on Medical Imaging, vol. 21, No. 8, pp. 991-997, Aug. 2002.
[cited by applicant]
Birkfellner et al., “Computer-enhanced stereoscopic vision in a head-mounted operating binocular”, Physics in Medicine & Biology, vol. 48, No. 3, pp. 49-57, Feb. 7, 2003.
[cited by applicant]
Birkfellner et al., “In-Vitro Aassessment of a Registration Protocol for Image Guided Implant Dentistry”, Clinical Oral Implants Research, vol. 12, Issue 1, pp. 69-78, Feb. 2001.
[cited by applicant]
Blackwell et al., “An Image Overlay System for Medical Data Visualization”, In: Wells W.M., Colchester A., Delp S. (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI'98. MICCAI 1998. Lecture Notes i…
[cited by applicant]
Blackwell et al., “An Image Overlay System for Medical Data Visualization”, Medical Image Analysis vol. 4, pp. 67-72, 2000.
[cited by applicant]
Blackwell et al., “Augmented Reality and Its Future in Orthopaedics”, Clinical Orthopaedics & Related Research, vol. 354, pp. 111-122, Sep. 1998.
[cited by applicant]
Castillo et al., “Augmented Reality for Assistance of Total Knee Replacement”, Journal of Electrical and Computer Engineering, vol. 2016, Article 9358369, pp. 1-6, 2016.
[cited by applicant]
Catani et al., “Knee Surgery Using Computer Assisted Surgery and Robotics”, Springer Heidelberg Publishing, Book, pp. 1-221, 2013.
[cited by applicant]
Chandak, “MEMS Based Wireless Controlled Robot with Voice and Video Camera”; International Journal of Scientific & Engineering Research, vol. 5, Issue 4, Apr. 2014.
[cited by applicant]
Charbonnier et al., “Real Virtuality: Perspectives offered by the combination of Virtual Reality headsets and Motion Capture”, Artanim, Real Virtuality White Paper, Aug. 23, 2015.
[cited by applicant]
Chen et al., “Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display”; Journal of Biomedical Informatics 55 (2015) 124-131.
[cited by applicant]
Cruz-Neira C. et al., “The cave: audio visual experience automatic virtual environment.”, Commun. ACM, vol. 35, No. 6, pp. 64-72, Jun. 1992.
[cited by applicant]
Cui et al., “KinectAvatar: Fully Automatic Body Capture Using a Single Kinect”, ACCV'12 Proceedings of the 11th International Conference on Computer Vision—vol. 2, pp. 133-147, Nov. 2012.
[cited by applicant]
Daniel and Ramos, “Augmented Reality for Assistance of Total Knee Replacement”, Journal of Electrical and Computer Engineering, vol. 2016, Article ID 9358369, Hindawi Publishing Corporation.
[cited by applicant]
Davies et al., “Computer Assisted Orthopaedic Surgery”, 8th Annual Meeting of CAOS-International Proceedings, Apr. 2008.
[cited by applicant]
deLambert et al., “Electromagnetic Tracking for Registration and Navigation in Endovascular Aneurysm Repair: A Phantom Study” European Journal of Vascular and Endovascular Surgery, vol. 43, pp. 684-689, 2012.
[cited by applicant]
Draelos, Mark, “The Kinect Up Close: Modifications for Short-Range Depth Imaging”, NC State Theses and Dissertations, pp. 1-88, Mar. 26, 2012.
[cited by applicant]
Elmi-Terander et al., “Surgical Navigation Technology Based on Augmented Reality and Integrated 3D Intraoperative Imaging”; Spine Surgery, vol. 41, No. 21, pp. E1303-1311, 2016.
[cited by applicant]
Ferrari et al., “Video See-Through in the Clinical Practice”, 1st International Workshop on Engineering Interactive Computing Systems for Medicine and Health Care, EICS4Med. vol. 727, pp. 19-24, 2011.
[cited by applicant]
Fischer et al., “Medical Augmented Reality Based on Commercial Image Guided Surgery”, European Association for Computer Graphics, Proceedings of the 10th Eurographics Symposium on Virtual Environments, pp. 83-86, Jun. 2…
[cited by applicant]
Fitzmaurice, G., et al., “Bricks: Laying the Foundations for Graspable User Interfaces.”, Proceedings of Conference on Human Factors in Computing Systems (CHI '95), Denver, Colorado, ACM Press, 442-449, (1995).
[cited by applicant]
Flusser et al., “Image Fusion: Principles, Methods and Applications”, Tutorial EISIPCO 2007 Lecture Notes.
[cited by applicant]
Fritz et al., “Augmented Reality Visualization with Image Overlay for MRI-Guided Intervention: Accuracy for Lumbar Spinal Procedures with a 1.5-T MRI System”, Vascular and Interventional Radiology, AJR: 198, Mar. 2012.
[cited by applicant]
Fritz et al., “Augmented Reality Visualization with Use of Image Overlay Technology for MR Imaging - guided Interventions: Assessment of Performance in Cadaceric Shoulder and Hip Arthrography at 1.5T”; Radiology: vol. 2…
[cited by applicant]
Garon, Mathieu; Boulet, Pierre-Olivier; Doiron, Jean-Philippe; Beaulieu, Luc; Lalonde, Jean-François (2016): Real-time High Resolution 3D Data on the HoloLens. In: International Symposium on Mixed and Augmented Reality …
[cited by applicant]
Garrido-Jurado, S.; Muñoz-Salinas, R.; Madrid-Cuevas, F. J.; Marín-Jiménez, M. J. (2014): Automatic generation and detection of highly reliable fiducial markers under occlusion. In: Pattern Recognition 47 (6), S. 2280-2…
[cited by applicant]
Gavaghan et al., “Augmented Reality Image Overlay Projection for Image Guided Open Liver Ablation of Metastatic Liver Cancer”; C.A. Linte et al. (Eds.): AE-CAI 2011, LNCS, pp. 36-46, 2012.
[cited by applicant]
Gee A, et al., “Processing and visualizing three-dimensional ultrasound data.”, The British Journal of Radiology, vol. 77, S186-S193, (2004).
[cited by applicant]
George et al., “Low Cost Augmented Reality for Training of MRI-Guided Needle Biopsy of the Spine”, Medicine Meets Virtual Reality 16, pp. 138-140, IOS Press, 2008.
[cited by applicant]
Germano et al., Advanced Techniques in Image-Guided Brain and Spine Surgery, Thieme Medical Publishers, Incorporated, 2002.
[cited by applicant]
Gonzalez, Smart Multi-Level Tool for Remote Patient Monitoring Based on a Wireless Sensor Network and Mobile Augmented Reality, Sensors, Sep. 2014; 14(9): 17212-17234.
[cited by applicant]
Gorbert, M. et al., “Triangles: Tangible Interface for Manipulation and Exploration of Digital Information Topography.”, Proceedings of CHI '98, Apr. 18-23, 1998, © 1998 ACM.
[cited by applicant]
Gromov et al., “What is the optimal alignment of the tibial and femoral components in knee arthroplasty?: An overview of the literature”; Acta Orthopaedica 2014; 85(5): 480-487.
[cited by applicant]
Hayashibe et al., “Surgical Navigation Display System Using Volume Rendering of Intraoperatively Scanned CT Images”, Computer Aided Surgery, vol. 11, No. 5, pp. 240-246, Sep. 2006.
[cited by applicant]
Hinterstoisser et al., “Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes” In 2011 International Conference on Computer Vision, pp. 858-865, Nov. 6, 2011.
[cited by applicant]
Hinterstoisser et al., “Gradient Response Maps for Real-Time Detection of Texture-Less Objects”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE Transactions on Pattern Analysis and Machine Intel…
[cited by applicant]
Hinterstoisser et al., “Real-Time Learning of Accurate Patch Rectification”, 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 2009, pp. 2945-2952, doi: 10.1109/CVPR.2009.5206794.
[cited by applicant]
Hinterstoisser et al., “Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes” Computer Vision—ACCV 2012: 11th Asian Conference on Computer Vision, Daejeon, Korea, No…
[cited by applicant]
Hoff, “Fusion of Data from Head-Mounted and Fixed Sensors”; First International Workshop on Augmented Reality, 1, 1998, pp. 1-15.
[cited by applicant]
Holographic weapon sight—Wikipedia https://en.wikipedia.org/wiki/Holographic_weapon_sight retrieved on Nov. 22, 2016.
[cited by applicant]
Hu et al., “A Convenient Method of Video See-through Augmented Reality Based on Image-Guided Surgery System”, Internet Computing for Engineering and Science, 2013 Seventh International Conference on Internet Computing f…
[cited by applicant]
Hua et al., “A 3D Integral Imaging Optical See-Through Head-Mounted Display”, Optical Society of America, vol. 22, No. 11, pp. 1-8, Jun. 2, 2014.
[cited by applicant]
Ishii, H., et al., “Iterative Design of Seamless Collaboration Media.”, Communications of the ACM, vol. 37, No. 8, Aug. 1994, pp. 83-97.
[cited by applicant]
Ji et al., “Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance”; Real-Time Imaging 8, pp. 357-377, 2002.
[cited by applicant]
Jiang et al., “A Robust Automated Markerless Registration Framework for Neurosurgery Navigation”, The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 11, pp. 436-447, Oct. 19, 2014.
[cited by applicant]
Jolesz, Ferenc A., “Intraoperative Imaging and Image-Guided Therapy”, Springer Science & Business Media, 893 pages, Jan. 14, 2014.
[cited by applicant]
Kanade et al., “Simulation, Planning, and Execution of Computer-Assisted Surgery”, Proceedings of the NSF Grand Challenges Workshop, 1996.
[cited by applicant]
Kato, H.; Billinghurst, M. (1999): Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Augmented Reality, 1999. (IWAR '99) Proceedings. 2nd IEEE and ACM International Worksho…
[cited by applicant]
Kersten-Oertel et al., “The State of the Art of Visualization in Mixed Reality Image Guided Surgery”, Computerized Medical Imaging and Graphics, vol. 37, pp. 98-112, Jan. 2013.
[cited by applicant]
Kim et al., “Registration Accuracy Enhancement of a Surgical Navigation System for Anterior Cruciate Ligament Reconstruction: A Phantom and Cadaveric Study”, The Knee, vol. 24, pp. 329-339, 2017.
[cited by applicant]
Kolodzey et al., “Wearable technology in the operating room: a systematic review”; GMJ Innov 2017; 3:55-63.
[cited by applicant]
Kumar et al., “A Portable Wireless Head Movement Controlled Human-Computer Interface for People with Disabilities”, International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering, …
[cited by applicant]
Kutter et al., “Real-time Volume Rendering for High Quality Visualization in Augmented Reality”, International Workshop on Augmented Environments for Medical Imaging including Augmented Reality in Computer-aided Surgery…
[cited by applicant]
Lamata et al., “Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances”; Augmented Reality, Jan. 2010.
[cited by applicant]
Liao et al., “3-D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay”, IEEE Transactions on Biomedical Engineering, vol. 57, No. 6, pp. 1476-1486, Jun. 2010.
[cited by applicant]
Liao et al., “Surgical Navigation by Autostereoscopic Image Overlay of Integral Videography”, IEEE Transactions on Information Technology in Biomedicine, vol. 8, No. 2, pp. 114-121, Jun. 2004.
[cited by applicant]
Lievin et al., “Stereoscopic Augmented Reality System for Computer-Assisted Surgery”, International Congress Series, vol. 1230, pp. 107-111, Jun. 2001.
[cited by applicant]
Lindert et al., “The use of a head-mounted display for visualization in neuroendoscopy”, Computer Aided Surgery, 2004; 9(6): 251-256.
[cited by applicant]
Linte et al., “On Mixed Reality Environments for Minimally Invasive Therapy Guidance: Systems Architecture, Successes and Challenges in their Implementation from Laboratory to Clinic”, Comput Med Imaging Graph, Mar. 201…
[cited by applicant]
Liu et al., “An Optical See-Through Head Mounted Display with Addressable Focal Planes” IEEE International Symposium on Mixed and Augmented Reality, Cambridge, UK, pp. 33-42, Oct. 3, 2008.
[cited by applicant]
Lorensen WE, Cline HE. [ed.], in M.C. Stone. 1987. Marching cubes: A high resolution 3d surface construction algorithm. Proceedings of SIGGRAPH 87. pp. 163-169.
[cited by applicant]
Maier-Hein et al., “Optical Techniques for 3D Surface Reconstruction in Computer-Assisted Laparoscopic Surgery”, Medical Image Analysis, vol. 17, pp. 974-996, May 3, 2013.
[cited by applicant]
Maier-Hein, L. et al., “Towards Mobile Augmented Reality for On-Patient Visualization of Medical Images.”, Bildverarbeitung für die Medizin 2011: Algorithmen—Systeme—Anwendungen Proceedings des Workshops in Lubeck, vol.…
[cited by applicant]
Masamune et al., “An Image Overlay System with Enhanced Reality for Percutaneous Therapy Performed Inside CT Scanner”, Medical Image Computing and Computer-Assisted Intervention, Lecture Notes in Computer Science, vol. …
[cited by applicant]
Maurer et al., “Augmented-Reality Visualization of Brain Structures with Stereo and Kinetic Depth Cues: System Description and Initial Evaluation with Head Phantom”, Proceedings, vol. 4319, Medical Imaging 2001: Visuali…
[cited by applicant]
Medeiros D. et al., “Proposal and evaluation of a tablet-based tool for 3D virtual environments.”, SBC Journal on 3D Interactive Systems, vol. 4, No. 2, pp. 30-40, 2013.
[cited by applicant]
Melzer, “Head-Mounted Displays”, The Avionics Handbook, 2001.
[cited by applicant]
Menozzi et al., “Development of Vision-Aided Navigation for a Wearable Outdoor Augmented Reality System”, IEEE Plans, Position Location and Navigation Symposium, Article No. 6851442, pp. 760-772, 2014.
[cited by applicant]
Micro Vision 2015 Annual Report and Proxy Statement for 2016 Annual Meeting of Shareholders.
[cited by applicant]
Moore et al., “Image Guidance for Spinal Facet Injections Using Tracked Ultrasound”, MICCAI 2009, Part I, LNCS 5761, pp. 516-523 2009.
[cited by applicant]
Muller et al., “Automatic Multi-Modal ToF/CT Organ Surface Registration”, Bildverarbeitung für die Medizin, pp. 154-158, Mar. 2011.
[cited by applicant]
Newcombe et al., “KinectFusion. Real-time dense surface mapping and tracking” In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, S. pp. 127-136, Oct. 26, 2011.
[cited by applicant]
Nicolau et al., “Augmented Reality in Laparoscopic Surgical Oncology.”, Surgical Oncology, vol. 20, Issue 3, pp. 189-201, Sep. 1, 2011.
[cited by applicant]
Nikou et al., “Augmented Reality Imaging Technology for Orthopaedic Surgery”, Operative Techniques in Orthopaedics, vol. 10, No. 1, pp. 82-86, Jan. 1, 2000.
[cited by applicant]
Noonan et al., “The Design and Initial Calibration of an Optical Tracking System Using the Microsoft Kinect”, IEEE Nuclear Science Symposium Conference Record, pp. 3614-3617, Oct. 2011.
[cited by applicant]
Okamura, Allison, “Tracking and Surgical Navigation, Registration”, Stanford Lecture 8: ME 328: Medical Robotics, pp. 1-19, Spring 2013.
[cited by applicant]
Ortega et al., “Usefulness of a head mounted monitor device for viewing intraoperative fluoroscopy during orthopaedic procedures”, Arch Orthop Trauma Surg, vol. 128, pp. 1123-1126, Oct. 2008.
[cited by applicant]
Paprosky et al., “Intellijoint HIP: a 3D mini-optical navigation tool for improving intraoperative accuracy during total hip arthroplasty”, Medical Devices: Evidence and Research, pp. 401-408, Nov. 18, 2016.
[cited by applicant]
Pauly et al., “Machine Learning-Based Augmented Reality for Improved Surgical Scene Understanding”, Computerized Medical Imaging and Graphics, vol. 1280, pp. 1-6, Jun. 2014.
[cited by applicant]
Peters et al., “Image-Guided Interventions, Technology and Applications”, Springer Science and Business Media, 576 pages, 2018.
[cited by applicant]
Ponce et al., “Emerging Technology in Surgical Education: Combining Real-Time Augmented Reality and Wearable Computing Devices”, The Cutting Edge, vol. 37, No. 11, Nov. 2014.
[cited by applicant]
Qian et al., Comprehensive Tracker Based Display Calibration for Holographic Optical See-Through Head-Mounted Display, 2017.
[cited by applicant]
Ren et al., “Marker-Based Surgical Instrument Tracking Using Dual Kinect Sensors”, IEEE Transactions on Automation Science and Engineering, vol. 11, No. 3, pp. 921-924, Jul. 2014.
[cited by applicant]
Rhodes, “A brief history of wearable computing”, MIT Wearable Computing Project, Feb. 1997.
[cited by applicant]
Rinaldi et al., “Computer-Guided Applications for Dental Implants, Bone Grafting, and Reconstructive Surgery”, Elsevier Inc., 556 pages, 2016.
[cited by applicant]
Robinett et al., “A Computational Model for the Stereoscopic Optics of a Head-Mounted Display”, Proceedings vol. 1457, Stereoscopic Displays and Applications II, pp. 140-160, 1991.
[cited by applicant]
Rolland et al., “A Comparison of Optical and Video See-through Head-mounted Displays”, Proceedings vol. 2351, Telemanipulator and Telepresence Technologies, pp. 293-307, Dec. 21, 1995.
[cited by applicant]
Rolland et al., “Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization”, Presence: Teleoperators and Virtual Environments, vol. 9, Issue 3, pp. 287-309, Jun. 2000.
[cited by applicant]
Rosenthal et al., “Augmented Reality Guidance for Needle Biopsies: A Randomized, Controlled Trial in Phantoms”; MICCAI 2001, The Netherlands, Oct. 14-17, 2001, Proceedings, vol. 2208, pp. 240-248, Oct. 2, 2001.
[cited by applicant]
Rosman et al., “Articulated Motion Segmentation of Point Clouds by Group-Valued Regularization”, Eurographics Workshop on 3D Object Retrieval, EG 3DOR, pp. 77-84, May 2012.
[cited by applicant]
Salmi Jamali, S. et al., “Utilising Mobile-Augmented Reality for Learning Human Anatomy.”, 7th World Conference on Educational Sciences, (WCES-2015), Feb. 5-7, 2015, Novotel Athens Convention Center, Athens, Greece, Pro…
[cited by applicant]
Sanko, “Microvision's Nomad Augmented Vision System: The How and the Why”; SID Pacific Northwest Chapter Meeting, Jun. 11, 2003.
[cited by applicant]
Sauer et al., “An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial”, Proceedings of the 5th International Conference on Medical Image Computing and Computer…
[cited by applicant]
Sauer et al., “Augmented Workspace: Designing an AR Testbed”, Proceedings IEEE and ACM International Symposium on Augmented Reality, pp. 47-53, Munich 2000.
[cited by applicant]
Schramm, Kinect: The Company Behind the Tech Explains How it Works, Jun. 19, 2010, https://www.engadget.com/2010/06/19/kinect-how-it-works-from-the-company-behind-the-tech/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29…
[cited by applicant]
Scuderi et al., “Total Knee Arthroplasty with a Novel Navigation System Within the Surgical Field”, Orthopedic Clinics, vol. 45, Issue 2, pp. 167-173, Apr. 2014.
[cited by applicant]
Shen et al., “3D Augmented Reality with Integral Imaging Display”, Proceedings of SPIE—The International Society for Optical Engineering, vol. 9867, Article No. 9867OY, Apr. 2016.
[cited by applicant]
Sherstyuk et al., “Dynamic Eye Convergence for Head-Mounted Displays Improves User Performance in Virtual Environments”, Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 23-30, Mar. 20…
[cited by applicant]
State et al., “Stereo Imagery from the UNC Augmented Reality System for Breast Biopsy Guidance”, MMVR 2003.
[cited by applicant]
Tan, D. J.; Tombari, F.; Ilic, S.; Navab, N. (2015): A Versatile Learning-Based 3D Temporal Tracker. Scalable, Robust, Online. In: 2015 IEEE International Conference on Computer Vision (ICCV), S. 693-701.
[cited by applicant]
Tong et al., “Scanning 3D Full Human Bodies Using Kinects”, IEEE Transactions on Visualization and Computer Graphics, vol. 18, Issue 4, pp. 643-650, Apr. 1, 2012.
[cited by applicant]
Traub, J., Stefan, P., Heining, S.M., Sielhorst, T., Riquarts, C., Eulerz, E., Navab, N. (2006): Hybrid Navigation Interface for Orthopedic and Trauma Surgery. R. Larsen, M. Nielsen, and J. Sporring (Eds.): MICCAI 2006,…
[cited by applicant]
Trevisan et al., “Towards Markerless Augmented Medical Visualization”, AMI-ARCS, pp. 57-66, 2004.
[cited by applicant]
Vagvolgyi et al., “Video to CT Registration for Image Overlay on Solid Organs”, Procedural Augmented Reality in Medical Imaging and Augmented Reality in Computer-Aided Surgery (AMIARCS) pp. 78-86, 2008.
[cited by applicant]
Vercauteren et al., “Real Time Autonomous Video Image Registration for Endomicroscopy: Fighting the Compromises”, Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XV., vol. 6861, pp. 6…
[cited by applicant]
Vogt et al., “Reality Augmentation for Medical Procedures: System Architecture, Single Camera Marker Tracking, and System Evaluation”, International Journal of Computer Vision, vol. 70, No. 2, pp. 179-190, 2006.
[cited by applicant]
Vogt, Sebastian, “Real-Time Augmented Reality for Image-Guided Interventions”, PhD Thesis, Nürnberg: Der Technischen Fakultät der Universität Erlangen, 2009.
[cited by applicant]
Wang et al., “3D Modeling from Wide Baseline Range Scans Using Contour Coherence”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4018-4025, 2014.
[cited by applicant]
Wang et al., “Augmented Reality 3D Displays with Micro Integral Imaging”; Journal of Display Technology, vol. 11, No. 11, Nov. 1, 2015.
[cited by applicant]
Wang et al., “Augmented Reality Navigation with Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery”, IEEE Transactions on Biomedical Engineering, vol. 61, No. 4, pp. 1295-1304, Apr. 2014.
[cited by applicant]
Wang H. et al., “Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: a pilot study”, Intl. Orthop. (SICOT) vol. 40, pp. 1941-1947, Sep. 2016.
[cited by applicant]
Watsen, K., et al., “A Handheld Computer as an Interaction Device to a Virtual Environment.”, Proceedings of the International Projection Technologies Workshop, Stuttgart, Germany, pp. 1-7, May 10-11, 1999.
[cited by applicant]
Weiss et al., “Augmented Reality Visualization Using Image-Overlay for MR-Guided Interventions: System Description, Feasibility, and Initial Evaluation in a Spine Phantom”, Musculoskeletal Imaging, AJR:196, Mar. 2011, D…
[cited by applicant]
Wellner, P., “Interacting with Paper on the DigitalDesk.”, Communications of the ACM. vol. 36, No. 7, pp. 87-96, Jul. 1, 1993.
[cited by applicant]
Wilson et al., “Validation of Three-Dimensional Models of the Distal Femur Created from Surgical Navigation Point Cloud Data”; CAOS 2015.
[cited by applicant]
Xiaojun et al., “Development of a Surgical Navigation System Based on Augmented Reality Using an Optical See-Through Head-Mounted Display”, Journal of Biomedical Informatics, vol. 55, pp. 124-131, 2015.
[cited by applicant]
Yamazaki, K et al., “Gesture Laser and Gesture Laser Car—Development of an Embodied Space to Support Remote Instruction.”, In Proceedings of the Sixth European Conference on Computer Supported Cooperative Work—ECSC W'99…
[cited by applicant]
Yang H. et al., “Exploring Collaborative Navigation: the Effect of Perspectives on Group Performance”, Proceedings of the 4th international conference on Collaborative virtual environments, pp. 135-142, Sep. 30, 2002.
[cited by applicant]
Ye et al., “Accurate 3D Pose Estimation from a Single Depth Image”, IEEE International Conference on Computer Vision (ICCV), pp. 731-738, Nov. 2011.
[cited by applicant]
Yoon et al., “Technical Feasibility and Safety of an Intraoperative Head-Up Display Device During Spine Instrumentation”, The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 13, No. 3, pp. …
[cited by applicant]