Method and system for mesh-to-image registration using raycasting
View Patent ↗A method of registering a digital image with a polygonal mesh comprising the steps of providing a digital image, providing a polygonal mesh comprising a plurality of vertices that approximates an object in the digital image, propagating a ray from a vertex of the polygonal mesh into the digital image and calculating a line integral of opacity values wherein each opacity value corresponds to the intensity of a pixel of the image through which said ray passes, terminating the ray at a point in the image when an opacity threshold is reached, and adding the termination point to a set of closest points. A registration is computed between the vertices of the polygonal mesh and the set of closest points, and the registration is applied to the polygonal mesh to obtain a new set of vertices.
1. A method of registering a digital image with a polygonal mesh comprising the steps of:
providing a digital image comprising a plurality of intensities corresponding to a domain of points in a D-dimensional space;
providing a polygonal mesh comprising a plurality of vertices that approximates an object in the digital image;
propagating a ray from a vertex of the polygonal mesh in a direction into the digital image and calculating a line integral of opacity values wherein each opacity value corresponds to the intensity of a pixel of the image through which said ray passes;
terminating the ray at a point in the image when an opacity threshold is reached; and
adding the termination point to a set of closest points.
2. The method of claim 1 , wherein the steps of propagating a ray from a vertex of the polygonal mesh into the digital image, terminating the ray at a point in the image, and adding the termination point to a set of closest points are repeated for each vertex in the polygonal mesh, and wherein the method further comprises the steps of:
computing a registration between the vertices of the polygonal mesh and the set of closest points;
applying the registration to the polygonal mesh to obtain a new set of vertices; and
calculating an error between the set of closest points and the new set of vertices.
3. The method of claim 2 , further comprising, if the error is greater than a predefined threshold, repeating the steps of propagating rays from the mesh vertices into the image, terminating each ray at a point in the image, adding each termination point to a set of closest points, computing a registration between the vertices of the polygonal mesh and the set of closest points, applying the registration to the polygonal mesh to obtain a new set of vertices, and calculating an error between the set of closest points and the new set of vertices.
4. The method of claim 1 , wherein the ray can be subdivided into n segments of length d, and the line integral of opacity values can be defined by
∑
i
=
0
n
c
(
s
(
x
→
(
id
)
)
)
d
∏
j
=
0
i
-
1
exp
(
-
τ
(
s
(
x
→
(
jd
)
)
)
d
)
,
wherein i,j are ray segment counters, {right arrow over (x)}(kd) represents the position of the ray at a distance kd of the k th segment, the scalar intensity value corresponding to this position on the ray is denoted by s({right arrow over (x)}(kd)), τ(s({right arrow over (x)}(kd))) represents an absorption look-up table value at said point, and c(s({right arrow over (x)}(kd))) represents an emission look-up table at said point.
5. The method of claim 1 , wherein said ray is terminated if a predefined distance threshold is reached before said opacity threshold is reached.
6. The method of claim 5 , wherein the starting point of said ray is offset in a negative direction by said distance threshold.
7. The method of claim 1 , wherein the direction of said ray is determined by a vector normal to a plane defined by those vertices nearest to the starting vertex of said ray.
8. The method of claim 1 , wherein the direction of said ray is determined by taking an average of normals of triangles neighboring said vertex.
9. The method of claim 1 , wherein the direction of said ray is determined from a stored normal vector associated with the vertex.
10. The method of claim 1 , further comprising the step of imposing an angular threshold on the direction of the ray, wherein if the angle between the ray and a normal to the termination point is greater than a predefined threshold, said termination point is excluded from the set of closest points.
11. A method of registering an object in a pair of digital images, said method comprising the steps of:
providing a first digital image and a second digital image, each digital image comprising a plurality of intensities corresponding to a domain of points in a D-dimensional space;
extracting an object from said first image, and representing said object by a set of oriented points, wherein each oriented point has an associated direction vector;
for each oriented point in the first image,
propagating a ray from said oriented point of the first image in the direction of said direction vector into the second digital image and calculating a line integral of opacity values wherein each opacity value corresponds to the intensity of a pixel of the second image through which said ray passes;
terminating the ray at a point in the second image when an opacity threshold is reached;
adding the termination point to a set of closest points;
computing a registration between the set of oriented points of-the first image and the set of closest points; and
applying the registration to the set of oriented points to obtain a new set of oriented points.
12. The method of claim 11 , wherein said object comprises a subset of points in said first image, and wherein the intensities of the points of said object are in a predetermined range.
13. A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for registering a digital image with a polygonal mesh, said method comprising the steps of:
providing a digital image comprising a plurality of intensities corresponding to a domain of points in a D-dimensional space;
providing a polygonal mesh comprising a plurality of vertices that approximates an object in the digital image;
propagating a ray from a vertex of the polygonal mesh in a direction into the digital image and calculating a line integral of opacity values wherein each opacity value corresponds to the intensity of a pixel of the image through which said ray passes;
terminating the ray at a point in the image when an opacity threshold is reached; and
adding the termination point to a set of closest points.
14. The computer readable program storage device of claim 13 , wherein the steps of propagating a ray from a vertex of the polygonal mesh into the digital image, terminating the ray at a point in the image, and adding the termination point to a set of closest points are repeated for each vertex in the polygonal mesh, and wherein the method further comprises the steps of:
computing a registration between the vertices of the polygonal mesh and the set of closest points;
applying the registration to the polygonal mesh to obtain a new set of vertices; and
calculating an error between the set of closest points and the new set of vertices.
15. The computer readable program storage device of claim 14 , said method further comprising the step of, if the error is greater than a predefined threshold, repeating the steps of propagating rays from the mesh vertices into the image, terminating each ray at a point in the image, adding each termination point to a set of closest points, computing a registration between the vertices of the polygonal mesh and the set of closest points, applying the registration to the polygonal mesh to obtain a new set of vertices, and calculating an error between the set of closest points and the new set of vertices.
16. The computer readable program storage device of claim 13 , wherein the ray can be subdivided into n segments of length d, and the line integral of opacity values can be defined by
∑
i
=
0
n
c
(
s
(
x
→
(
id
)
)
)
d
∏
j
=
0
i
-
1
exp
(
-
τ
(
s
(
x
→
(
jd
)
)
)
d
)
,
wherein i,j are ray segment counters, {right arrow over (x)}(kd) represents the position of the ray at a distance kd of the k th segment, the scalar intensity value corresponding to this position on the ray is denoted by s({right arrow over (x)}(kd)), τ(s({right arrow over (x)}(kd))) represents an absorption look-up table value at said point, and c(s({right arrow over (x)}(kd))) represents an emission look-up table at said point.
17. The computer readable program storage device of claim 13 , wherein said ray is terminated if a predefined distance threshold is reached before said opacity threshold is reached.
18. The computer readable program storage device of claim 17 , wherein the starting point of said ray is offset in a negative direction by said distance threshold.
19. The computer readable program storage device of claim 13 , wherein the direction of said ray is determined by a vector normal to a plane defined by those Vertices nearest to the starting vertex of said ray.
20. The computer readable program storage device of claim 13 , said method further comprising the step of imposing an angular threshold on the direction of the ray, wherein if the angle between the ray and a normal to the termination point is greater than a predefined threshold, said termination point is excluded from the set of closest points.
21. The computer readable program storage device of claim 13 , wherein the direction of said ray is determined by taking an average of normals of triangles neighboring said vertex.
22. The computer readable program storage device of claim 13 , wherein the direction of said ray is determined from a stored normal vector associated with the vertex.