IP Library Granted Patent US 10,401,156
Granted Patent B2
US 10,401,156 · App. 16/110,023 · Granted Sep 3, 2019

System and method for quantifying deformation, disruption, and development in a sample

Inventors: John J. Boyle (St. Louis, MO); Guy M. Genin (St. Louis, MO); Maiko Kume (St. Louis, MO); Robert B. Pless (St. Louis, MO); Stavros Thomopoulos (St. Louis, MO)
Assignee: WASHINGTON UNIVERSITY
G01B11/16G06T7/30G06T2207/10004G06T2207/30024G06T2207/30108
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,401,156
App. No.
16/110,023
Granted
Sep 3, 2019
Kind
B2
Abstract

A computer-implemented method for determining a quantification of the deformation of the sample is implemented using a computer device in communication with a memory. The method includes receiving, by the computer device, a first image of the sample and a second image of the sample. The method also includes registering the first image to the second image using a warping function. The warping function maps a plurality of pixels in the first image to a plurality of pixels in the second image. A first displacement field for the sample is determined based on the warping function, where the first displacement field includes at least a portion of the warping function. A first quantification of the deformation of the sample is determined based at least in part on the displacement field.

Claims (641)

1. A computer-implemented method for determining a first quantification of a deformation on a surface or on incremental regions of a sample, said method implemented using a computing device in communication with a memory, said method comprising:

receiving, by the computing device, a first stereo image and a second stereo image of the sample, each stereo image comprising a camera 1 image and a camera 2 image obtained by first and second cameras, respectively;

registering the first stereo image to the second stereo image, wherein registering the first stereo image to the second stereo image comprises:

using a first warping function W 1 (x;p) to map a plurality of pixels from the camera 1 image of the first stereo image to the camera 1 image of the second stereo image; and

using a second warping function W 2 (x;p) to map a plurality of pixels from the camera 2 image of the first stereo image to the camera 2 image of the second stereo image;

wherein x is a 3D position within a world coordinate system and p is a set of shared warp parameters;

determining a first deformation gradient tensor for the sample based on the first and second warping functions, the first deformation gradient tensor including at least a portion of the first and second warping functions; and

determining the first quantification of the deformation of the sample based at least in part on the first deformation gradient tensor;

wherein the first and second warping functions each comprise an affine warp with a linear translation.

2. The method of claim 1 , further comprising optimizing the first and second warping functions to minimize a difference between each mapped pixel value of the plurality of pixels in the first stereo image and each corresponding pixel value of the plurality of pixels in the second stereo image using an energy minimization method.

3. The method of claim 1 , wherein the first and second warping functions are expressed as:

W

1

(

x

;

p

)

=

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

Λ

i

C

i

R

1

p

K

2

[

X

2

Y

2

1

]

+

T

1

p

;

and

W

2

(

x

;

p

)

=

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

Λ

i

C

i

R

1

p

T

R

12

K

1

[

X

1

Y

1

1

]

+

T

1

p

+

T

12

wherein:

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

is a matrix containing a rotation matrix R i and a translation vector

[

T

i

,

x

T

i

,

y

T

i

,

z

]

describing a translation and rotation from the camera 1 and camera 2 images of the first stereo image to corresponding camera 1 and camera 2 images of the second stereo image;

Λ i is any deformation tensor, scalar, or vector describing a deformation from the camera 1 and camera 2 images of the first stereo image to corresponding camera 1 and camera 2 images of the second stereo image;

C i is a normalization factor relating a normalized coordinate system to a world coordinate system;

R 1p is a 3D rotation matrix describing a rotation from the world coordinate system to a coordinate system of camera 1;

R 12 is a 3D rotation matrix describing a rotation from the coordinate system of camera 1 to a coordinate system of camera 2;

K 1 and K 2 are calibrations for cameras 1 and 2, respectively;

[

X

1

Y

1

1

]

and

[

X

2

Y

2

1

]

are positions within the coordinate systems of cameras 1 and 2, respectively;

T 1p is a 3D translation matrix describing a translation from the world coordinate system to the coordinate system of camera 1; and

T 12 is a 3D translation matrix describing a translation from the coordinate system of camera 1 to the coordinate system of camera 2.

4. The method of claim 1 , further comprising:

determining a displacement field by comparing the first stereo image to the second stereo image;

determining a second deformation gradient tensor for the sample based on a fit or differentiation of the displacement field;

determining a second quantification of the deformation of the sample based at least in part on the second deformation gradient tensor; and

determining a strain concentration tensor comprising the difference between the first quantification of the deformation of the sample and the second quantification of the deformation of the sample.

5. A computing device for determining a first quantification of a deformation on a surface or on incremental regions of a sample, said computing device comprising a processor communicatively coupled to a memory, said computing device programmed to:

receive, by the computing device, a first stereo image and a second stereo image of the sample, each stereo image comprising a camera 1 image and a camera 2 image obtained by first and second cameras, respectively;

register the first stereo image to the second stereo image, wherein registering the first stereo image to the second stereo image comprises:

using a first warping function W 1 (x;p) to map a plurality of pixels from the camera 1 image of the first stereo image to the camera 1 image of the second stereo image; and

using a second warping function W 2 (x;p) to map a plurality of pixels from the camera 2 image of the first stereo image to the camera 2 image of the second stereo image;

wherein x is a 3D position within a world coordinate system and p is a set of shared warp parameters;

determine a first deformation gradient tensor for the sample based on the first and second warping functions, the first deformation gradient tensor including at least a portion of the first and second warping functions; and

determine the first quantification of the deformation of the sample based at least in part on the first deformation gradient tensor;

wherein the first and second warping functions each comprise an affine warp with a linear translation.

6. The computing device of claim 5 , wherein the computing device is further programmed to optimize the first and second warping functions to minimize a difference between each mapped pixel value of the plurality of pixels in the first stereo image and each corresponding pixel value of the plurality of pixels in the second stereo image using an energy minimization method.

7. The computing device of claim 5 , wherein the first and second warping functions are expressed as:

W

1

(

x

;

p

)

=

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

Λ

i

C

i

R

1

p

K

2

[

X

2

Y

2

1

]

+

T

1

p

;

and

W

2

(

x

;

p

)

=

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

Λ

i

C

i

R

1

p

T

R

12

K

1

[

X

1

Y

1

1

]

+

T

1

p

+

T

12

wherein:

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

is a matrix containing a rotation matrix R i and a translation vector

[

T

i

,

x

T

i

,

y

T

i

,

z

]

describing a translation and rotation from the camera 1 and camera 2 images of the first stereo image to corresponding camera 1 and camera 2 images of the second stereo image;

Λ i is any deformation tensor, scalar, or vector describing a deformation from the camera 1 and camera 2 images of the first stereo image to corresponding camera 1 and camera 2 images of the second stereo image;

C i is a normalization factor relating a normalized coordinate system to a world coordinate system;

R 1p is a 3D rotation matrix describing a rotation from the world coordinate system to a coordinate system of camera 1;

R 12 is a 3D rotation matrix describing a rotation from the coordinate system of camera 1 to a coordinate system of camera 2;

K 1 and K 2 are calibrations for cameras 1 and 2, respectively;

[

X

1

Y

1

1

]

and

[

X

2

Y

2

1

]

are positions within the coordinate systems of cameras 1 and 2, respectively;

T 1p is a 3D translation matrix describing a translation from the world coordinate system to the coordinate system of camera 1; and

T 12 is a 3D translation matrix describing a translation from the coordinate system of camera 1 to the coordinate system of camera 2.

8. The computing device of claim 5 , further comprising:

determining a displacement field by comparing the first stereo image to the second stereo image;

determining a second deformation gradient tensor for the sample based on a fit or differentiation of the displacement field;

determining a second quantification of the deformation of the sample based at least in part on the second deformation gradient tensor; and

determining a strain concentration tensor comprising the difference between the first quantification of the deformation of the sample and the second quantification of the deformation of the sample.

9. A system for determining a determining a first quantification of a deformation on a surface or on incremental regions of a sample, said system comprising:

an imaging device communicatively coupled to a computing device; and

the computing device comprising a processor communicatively coupled to a memory, said computing device programmed to:

receive, in the memory, a first stereo image and a second stereo image of the sample, each stereo image comprising a camera 1 image and a camera 2 image obtained by first and second cameras, respectively;

register the first stereo image to the second stereo image, wherein registering the first stereo image to the second stereo image comprises:

using a first warping function W 1 (x;p) to map a plurality of pixels from the camera 1 image of the first stereo image to the camera 1 image of the second stereo image; and

using a second warping function W 2 (x;p) to map a plurality of pixels from the camera 2 image of the first stereo image to the camera 2 image of the second stereo image;

wherein x is a 3D position within a world coordinate system and p is a set of shared warp parameters;

determine a first deformation gradient tensor for the sample based on the first and second warping functions, the first deformation gradient tensor including at least a portion of the first and second warping functions; and

determine the first quantification of the deformation of the sample based at least in part on the first deformation gradient tensor;

wherein the first and second warping functions each comprises an affine warp with a linear translation.

10. The system of claim 9 , wherein the computing device is further programmed to optimize the first and second warping functions to minimize a difference between each mapped pixel value of the plurality of pixels in the first stereo image and each corresponding pixel value of the plurality of pixels in the second stereo image using an energy minimization method.

11. The system of claim 9 , wherein the first and second warping functions are expressed as:

W

1

(

x

;

p

)

=

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

Λ

i

C

i

R

1

p

K

2

[

X

2

Y

2

1

]

+

T

1

p

;

and

W

2

(

x

;

p

)

=

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

Λ

i

C

i

R

1

p

T

R

12

K

1

[

X

1

Y

1

1

]

+

T

1

p

+

T

12

wherein:

[

.

.

.

T

i

,

x

.

R

i

.

T

i

,

y

.

.

.

T

i

,

z

]

is a matrix containing a rotation matrix R i and a translation vector

[

T

i

,

x

T

i

,

y

T

i

,

z

]

describing a translation and rotation from the camera 1 and camera 2 images of the first stereo image to corresponding camera 1 and camera 2 images of the second stereo image;

Λ i is any deformation tensor, scalar, or vector describing a deformation from the camera 1 and camera 2 images of the first stereo image to corresponding camera 1 and camera 2 images of the second stereo image;

C i is a normalization factor relating a normalized coordinate system to a world coordinate system;

R 1p is a 3D rotation matrix describing a rotation from the world coordinate system to a coordinate system of camera 1;

R 12 is a 3D rotation matrix describing a rotation from the coordinate system of camera 1 to a coordinate system of camera 2;

K 1 and K 2 are calibrations for cameras 1 and 2, respectively;

[

X

1

Y

1

1

]

and

[

X

2

Y

2

1

]

are positions within the coordinate systems of cameras 1 and 2, respectively;

T 1p is a 3D translation matrix describing a translation from the world coordinate system to the coordinate system of camera 1; and

T 12 is a 3D translation matrix describing a translation from the coordinate system of camera 1 to the coordinate system of camera 2.

12. The system of claim 9 , further comprising:

determining a displacement field by comparing the first stereo image to the second stereo image;

determining a second deformation gradient tensor for the sample based on a fit or differentiation of the displacement field;

determining a second quantification of the deformation of the sample based at least in part on the second deformation gradient tensor; and

determining a strain concentration tensor comprising the difference between the first quantification of the deformation of the sample and the second quantification of the deformation of the sample.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE SPELLING OF THE THIRD ASSIGNORS LAST NAME PREVIOUSLY RECORDED AT REEL: 048790 FRAME: 0028. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 5, 2019
From: KUME, MAIKO; THOMOPOULOS, STAVROS; GENIN, GUY M.; PLESS, ROBERT B.; BOYLE, JOHN J.
To: WASHINGTON UNIVERSITY
Reel/Frame 048810/0533 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2019
From: KUME, MAIKO; THOMOPOULOS, STAVROS; GENN, GUY M.; PLESS, ROBERT B.; BOYLE, JOHN J.
To: WASHINGTON UNIVERSITY
Reel/Frame 048790/0028 →
CONFIRMATORY LICENSE Recorded Oct 3, 2018
From: WASHINGTON UNIVERSITY
To: NATIONAL SCIENCE FOUNDATION
Reel/Frame 047186/0501 →
Continuity (3)
Continuation 15126852
Provisional Application 61954313 · Mar 17, 2014
Related Publication 20190219382A1 · Jul 18, 2019