IP Library › Granted Patent US 12,514,650
Granted Patent B2
US 12,514,650 · App. 18/035,861 · Granted Jan 6, 2026

Deep brain path planning using augmented reality

Inventors: Adrianus Johannes Maria Denissen (Moergestel, NL); Raymond Van Ee (Geldrop, NL); Rudolf Mathias Johannes Nicolaas Lamerichs (Liempde, NL); Jean-Marc Huijskens (Eindhoven, NL); Timmy Robertus Maria Leufkens (Upplands Väsby, SE)
Assignee: KONINKLIJKE PHILIPS N.V.
A61B34/20A61B34/10A61B90/37A61N5/1049G01R33/4806G06T19/006A61B2034/105A61B2034/107A61B2034/2055A61B2090/365A61N2005/1055G06T2210/41
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,514,650
App. No.
18/035,861
Filed
May 8, 2023
Granted
Jan 6, 2026
Kind
B2
Art Unit
3796
USPC
606/1
Abstract

Disclosed herein is a medical system ( 200, 300 ) comprising an augmented reality system ( 204 ) configured for rendering virtual objects within a visual three-dimensional field of view of an operator ( 214 ). Execution of machine executable instructions ( 230 ) causes the computational system to: receive ( 400 ) segmented medical imaging data ( 100, 232 ) descriptive of a continuous volume of a subject ( 210 ) comprising at least a portion of a brain ( 102 ) of the subject and a cranial surface ( 106 ) of the subject; control ( 402 ) the augmented reality system to determine a registration ( 234 ) between the subject and the segmented medical imaging data; receive ( 404 ) a target location ( 104 ) within the at least a portion of the brain; discretize ( 406 ) the cranial surface to define multiple entry locations ( 108 ); determine ( 408 ) a straight path ( 110 ) for each of the multiple entry locations that extends to the target location; assign ( 410 ) a path score ( 238 ) to the straight path for each of the multiple entry locations using the segmented medical imaging data; calculate ( 412 ) a three-dimensional path score surface defined by the cranial surface and is descriptive of the path score of each of the multiple entry locations; and render ( 414 ) the three-dimensional path score surface using the augmented reality system, wherein the rendered three-dimensional path score surface is positioned in the visual three-dimensional field of view of the operator using the registration.

Claims (60)

1 . A medical system comprising:

a memory storing machine executable instructions; and

a processor configured to execute the machine executable instructions to cause the processor to:

receive segmented medical imaging data descriptive of a continuous volume of a subject comprising at least a portion of a brain of the subject and a cranial surface of the subject;

determine a registration between the subject and the segmented medical imaging data in augmented reality;

receive a target location within the at least a portion of the brain;

discretize the cranial surface to define multiple entry locations;

for each entry location of the multiple entry locations,

determine a straight path for the entry location that extend from the cranial surface to the target location, and

assign a path score to the straight path for the entry location using the segmented medical imaging data;

calculate a three-dimensional path score surface defined by the cranial surface and is descriptive of each path score assigned for the multiple entry locations;

and

render the three-dimensional path score on the cranial surface of the subject in the augmented reality, wherein the rendered three-dimensional path score surface is positioned in a visual three-dimensional field of view of an operator using the registration.

2 . The medical system of claim 1 , wherein the cranial surface is defined as an exterior surface of the subject.

3 . The medical system of claim 1 , wherein the processor is configured to execute the machine executable instructions to further cause the processor to:

determine an optimal path selected using the path score of each of the multiple entry locations, and

render a three-dimensional medical implement positioning guide in the visual three-dimensional field of view of the operator, wherein the three-dimensional implement positioning guide is aligned with the optimal path.

4 . The medical system of claim 3 , wherein the processor is configured to execute the machine executable instructions to further cause the processor to:

to detect an implement location descriptive of a three-dimensional location of a medical implement; and

render a tool aligned indicator in the visual three-dimensional field of view of the operator if the implement location is aligned with the optimal path.

5 . The medical system of claim 3 , wherein the three-dimensional medical implement is one or more of: a biopsy needle, a deep brain stimulation implantable device, a pulse generator insertion needle, a needle, and a surgical needle.

6 . The medical system of claim 1 , wherein the machine executable instructions include at least one machine executable instruction configured to modify the segmented medical imaging data in response to at least one of a puncture of a skull or draining of cerebral fluid of the subject, wherein the three-dimensional path score is corrected using the at least one machine executable instruction.

7 . The medical system of claim 1 , wherein the processor is configured to execute the machine executable instructions to further cause the processor to:

receive intermediate magnetic resonance imaging data descriptive of the continuous volume of a subject;

register the intermediate magnetic resonance imaging data to the segmented medical imaging data; and

modify the segmented medical imaging data using the registration for a deformable brain model, wherein the modified segmented medical imaging data is used to recalculate the three-dimensional path score surface.

8 . The medical system of claim 1 , wherein the processor is configured to execute the machine executable instructions to further cause the processor to:

receive a selection of the one or more of the multiple entry locations; and

construct control commands to irradiate the target location along the selection of the one or more of the multiple entry locations.

9 . The medical system of claim 8 , wherein the processor is configured to execute the machine executable instructions to further cause the processor to execute the control commands to irradiate the target location.

10 . The medical system of claim 1 , wherein the segmented medical imaging data comprises one or more of: segmented T2 weighted magnetic resonance imaging data, segmented magnetic resonance imaging data, segmented computed tomography magnetic resonance imaging data, segmented functional magnetic resonance imaging data, magnetic resonance angiography data, and combinations thereof.

11 . The medical system of claim 1 , wherein the segmented medical imaging data assigns multiple tissue types three-dimensionally within the continuous volume, wherein the respective multiple tissue types are assigned a numerical damage value, and wherein the path score is calculated by determining a distance traveled through the respective multiple tissue types using the segmented medical imaging data times its numerical damage score.

12 . The medical system of claim 1 , wherein the segmented medical imaging data identifies critical anatomical structures, and wherein the processor is configured to execute of the machine executable instructions to further cause the processor to exclude any straight path through any of the critical anatomical structures.

13 . The medical system of claim 1 , wherein the segmented medical imaging data further comprises functional magnetic resonance imaging data descriptive of critical brain function regions, and wherein the processor is configured to execute the machine executable instructions to further cause the processor to:

exclude any straight path through the critical brain function regions;

wherein each of the critical brain function regions is assigned a numerical brain function damage value, and wherein the path score is calculated at least partially by determining a distance traveled through each of the critical brain function regions times its numerical brain function damage score; and

combinations thereof.

14 . The medical system of claim 1 , wherein real-time feedback on insertion of a needle into the subject is received by the operator of the system using AR goggles.

15 . The medical system of claim 14 , wherein the real-time feedback reflects insertion angle of the needle, wherein changes in the insertion angle of the needle is expressed as a change of color.

16 . The medical system of claim 14 , wherein the real-time feedback reflects speed of insertion of the needle, and wherein the speed of insertion of the needle is expressed as a change of a color.

17 . A non-transitory computer readable medium comprising machine executable instructions which, when executed by a processor, cause the processor to:

receive segmented medical imaging data descriptive of a continuous volume of a subject comprising at least a portion of a brain of the subject and a cranial surface of the subject;

determine a registration between the subject and the segmented medical imaging data in augmented reality;

receive a target location within the at least a portion of the brain;

discretize the cranial surface to define multiple entry locations;

for each entry location of the multiple entry locations,

determine a straight path for the entry location that extends from the cranial surface to the target location, and

assign a path score to the straight path for the entry location using the segmented medical imaging data;

calculate a three-dimensional path score surface defined by the cranial surface and is descriptive of each path score assigned for the multiple entry locations; and

render the three-dimensional path score on the cranial surface of the subject in the augmented reality, wherein the rendered three-dimensional path score surface is positioned in a visual three-dimensional field of view of the an operator using the registration.

18 . A method of operating a medical system the method comprising:

receiving segmented medical imaging data descriptive of a continuous volume of a subject comprising at least a portion of a brain of the subject and a cranial surface of the subject;

determining a registration between the subject and the segmented medical imaging data in augmented reality;

receiving a target location within the at least a portion of the brain;

discretizing the cranial surface to define multiple entry locations;

for each entry location of the multiple entry locations,

determining a straight path for the entry location that extends from the cranial surface to the target location, and

assigning a path score to the straight path for the entry location using the segmented medical imaging data;

calculating a three-dimensional path score surface defined by the cranial surface and is descriptive of each path score assigned for the multiple entry locations; and

rendering the three-dimensional path score on the cranial surface of the subject in using the augmented reality, wherein the rendered three-dimensional path score surface is positioned in the visual three-dimensional field of view of the operator using the registration.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 8, 2023
From: DENISSEN, ADRIANUS JOHANNES MARIA; VAN EE, RAYMOND; LAMERICHS, RUDOLF MATHIAS JOHANNES NICOLAAS; HUIJSKENS, JEAN-MARC; LEUFKENS, TIMMY ROBERTUS MARIA
To: KONINKLIJKE PHILIPS N.V.
Reel/Frame 063567/0121 →
Priority Claims (1)
EP 20206434 · Nov 9, 2020 · regional
Continuity (1)
Related Publication 20230404678A1 · Dec 21, 2023
References Cited (18)
US 20090005678A1 · Schmiedehausen · 2009 [cited by applicant]
US 20110245625A1 · Trovato · 2011 [cited by applicant]
US 20170065349A1 · Ourselin · 2017 [cited by examiner]
US 20170215968A1 · Rodriguez-Ponce · 2017 [cited by applicant]
US 20170265943A1 · Sela · 2017 [cited by applicant]
US 20180078316A1 · Schaewe · 2018 [cited by applicant]
US 20180140362A1 · Cali · 2018 [cited by applicant]
US 20180177406A1 · Kim · 2018 [cited by applicant]
EP 0945814A2 · 1999 [cited by applicant]
EP 0945814A3 · 2001 [cited by applicant]
EP 3298968A1 · 2018 [cited by applicant]
WO WO2010080611A2 · 2010 [cited by applicant]
WO WO2014139024A1 · 2014 [cited by applicant]
WO WO2016083927A1 · 2016 [cited by applicant]
WO WO2016083927A8 · 2016 [cited by applicant]
WO WO2017144107A1 · 2017 [cited by applicant]
WO WO2020167997A1 · 2020 [cited by applicant]
PCT International Search Report, International application No. PCT/EP2021/080568, Jan. 4, 2022. [cited by applicant]