IP Library › Granted Patent US 11,504,055
Granted Patent B2
US 11,504,055 · App. 16/639,122 · Granted Nov 22, 2022

Systems and methods for analyzing cutaneous conditions

Inventors: Jeremy Mark Massey (Singapore, SG); Syed Haroon Alam (Lahore, PK); Paul Lorenz Bigliardi (Minneapolis, MN); Mei Bigliardi-Qi (Minneapolis, MN)
Assignee: IKO PTE. LTD.
A61B5/441A61B5/0064G06T7/0014G06T17/20G06T2207/10024G06T2207/10028G06T2207/30088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,504,055
App. No.
16/639,122
Granted
Nov 22, 2022
Kind
B2
Abstract

The following disclosure discusses systems and methods of detecting and analyzing cutaneous conditions. According to one embodiment, a 2D image of the cutaneous condition and a set of 3D point clouds associated with the 2D image are captured using an image capturing device. The 2D image and the set of 3D point clouds are sent to a computing device. The computing device generates a 3D surface according to the set of 3D point clouds. Subsequently, the computing device receives a depth map for the 2D image based upon the 3D surface from another computing device such that the depth map comprises depth data for each pixel of the 2D image. The cutaneous condition may then be measured and analyzed based upon the depth map using the computing device.

Claims (175)

1. A method of analyzing a cutaneous condition comprising:

receiving from an image capturing device a two-dimensional (2D) image of the cutaneous condition and a set of three-dimensional (3D) point clouds associated with the 2D image using a computing device;

generating a 3D surface according to the set of 3D point clouds using the computing device;

generating a depth map for the 2D image based on the 3D surface using a second computing device, wherein the depth map comprises a depth value for each pixel of the 2D image; and

analyzing the cutaneous condition based upon the 2D image and the depth map using the computing device;

wherein dimensions of each pixel in a desired area or path in the 2D image are calculated for the analysis:

wherein horizontal and vertical dimensions of the each pixel in the desired area or path in the 2D image are calculated using pixel-space corner vectors for each pixel in the desired area or path in the 2D image, according to the formulas below:

h t V =V 2 −V 1

h t =√{square root over ( h t V ·x 0 2 +h t V y 0 2 +h t V z 0 2 )}

h b V =V 4 −V 3

h b =√{square root over ( h b V ·x 0 2 +h b V y 0 2 +h b V z 0 2 )}

v l V =V 1 −V 3

v l =√{square root over ( v l V ·x 0 2 +v l V y 0 2 +v l V z 0 2 )}

v b V =V 2 −V 4

v b =√{square root over ( v b V ·x 0 2 +v b V y 0 2 +v b V z 0 2 )}

wherein,

V 1 , V 2 , V 3 , V 4 represent pixel-space corner vectors for a given pixel

x 0 , y 0 , z 0 represent position vector of the given pixel

h t represents top horizontal dimension of the given pixel

h b represents bottom horizontal dimension of the given pixel

v l represents left vertical dimension of the given pixel

v r represents right vertical dimension of the given pixel;

wherein the pixel-space corner vectors for the each pixel in the desired area or path in the 2D image are calculated using 3D camera-space position of the each pixel in the desired area or path in the 2D image according to the formula below:

V n =(Pos n N −Pos)/2

wherein,

Pos represents position vector of the given pixel

Pos n N represents position vectors of pixels located at corners of the give pixel

V n represents pixel-space corner vectors for the given pixel

n is 1, 2, 3 or 4;

wherein 3D camera-space position of the each pixel in the desired area or path in the 2D image is calculated according to the formulas below:

x

=

tan

⁢

⁢

θ

H

2

·

D

·

(

C

h

·

2

R

h

-

1

)

y

=

tan

⁢

⁢

θ

V

2

·

D

·

(

C

v

·

2

R

v

-

1

)

z

=

D

wherein,

D represents depth of the given pixel

C represents coordinates of the given pixel, and C h and C v represent horizontal and vertical coordinate value respectively

R represents resolution of total image, and R h and R v represent horizontal and vertical resolution respectively

Θ H and Θ V represent horizontal and vertical 2D camera field-of-view angles.

2. A system for analyzing a cutaneous condition comprising:

an image capturing device that captures a two-dimensional (2D) image of the cutaneous condition and a set of three-dimensional (3D) point clouds associated with the 2D image;

a computing device communicatively coupled to the image capturing device and a second computing device, wherein the computing device is configured to:

receive the 2D image and the set of 3D point clouds from the image capturing device;

generate a 3D surface according to the set of 3D point clouds;

generate a depth map for the 2D image based on the 3D surface using the second computing device, wherein the depth map comprises depth data for each pixel of the 2D image; and

analyze the cutaneous condition based on the depth map;

wherein dimensions of each pixel in a desired area or path in the 2D image are calculated for the analysis;

wherein horizontal and vertical dimensions of each pixel in the desired area or path in the 2D image are calculated using pixel-space corner vectors for each pixel in the desired area or path in the 2D image, according to the formulas below:

h t V =V 2 −V 1

h t =√{square root over ( h t V ·x 0 2 +h t V y 0 2 +h t V z 0 2 )}

h b V =V 4 −V 3

h b =√{square root over ( h b V ·x 0 2 +h b V y 0 2 +h b V z 0 2 )}

v l V =V 1 −V 3

v l =√{square root over ( v l V ·x 0 2 +v l V y 0 2 +v l V z 0 2 )}

v b V =V 2 −V 4

v b =√{square root over ( v b V ·x 0 2 +v b V y 0 2 +v b V z 0 2 )}

wherein,

V 1 , V 2 , V 3 , V 4 represent pixel-space corner vectors for a given pixel

x 0 , y 0 , z 0 represent position vector of the given pixel

h t represents top horizontal dimension of the given pixel

h b represents bottom horizontal dimension of the given pixel

v l represents left vertical dimension of the given pixel

v r represents right vertical dimension of the given pixel;

wherein the pixel-space corner vectors for the each pixel in the desired area or path in the 2D image are calculated using 3D camera-space position of the each pixel in the desired area or path in the 2D image according to the formula below:

V n =(Pos n N −Pos)/2

wherein,

Pos represents position vector of the given pixel

Pos n N represents position vectors of pixels located at corners of the give pixel

V n represents pixel-space corner vectors for the given pixel

n is 1, 2, 3 or 4;

wherein 3D camera-space position of the each pixel in the desired area or path in the 2D image is calculated according to the formulas below:

x

=

tan

⁢

⁢

θ

H

2

·

D

·

(

C

h

·

2

R

h

-

1

)

y

=

tan

⁢

⁢

θ

V

2

·

D

·

(

C

v

·

2

R

v

-

1

)

z

=

D

wherein,

D represents depth of the given pixel

C represents coordinates of the given pixel, and C h and C v represent horizontal and vertical coordinate value respectively

R represents resolution of total image, and R h and R v represent horizontal and vertical resolution respectively

Θ H and Θ V represent horizontal and vertical 2D camera field-of-view angles.

3. The system as defined in claim 2 , wherein each 3D point in each 3D point cloud in the set of 3D point clouds corresponds to at least one pixel of the 2D image.

4. The system as defined in claim 2 , wherein the second computing device is configured to calculate the depth map by implementing a ray-casting algorithm or a ray-tracing algorithm according to the 2D image and the 3D surface.

5. The system as defined in claim 2 , wherein the image capturing device is configured to capture the 2D image and the set of 3D point clouds.

6. The system as defined in claim 5 wherein the image capturing device is configured to revolve along at least one axis around a subject with the cutaneous condition to capture a set of 2D images and associated sets of 3D images.

7. The system as defined in claim 5 further comprising a memory device, communicatively coupled to the image capturing device, configured to store the 2D image and the set of 3D point clouds and store the depth map.

8. The system as defined in claim 5 , wherein the image capturing device comprises a two-dimensional (2D) camera configured to:

capture a colored 2D image of the cutaneous condition, and

capture a monochromatic 2D image of the cutaneous condition.

9. The system as defined in claim 8 , wherein the computing device is configured to generate a set of pixel dimensions for each pixel of the 2D image based upon the depth value of each pixel, an angle of horizontal field of view of the 2D camera, and an angle of vertical field of view of the 2D camera.

10. The system as defined in claim 5 , wherein the image capturing device comprises a three-dimensional (3D) camera, and wherein the 3D camera captures the set of 3D point clouds corresponding to the 2D image.

11. The system as defined in claim 2 , wherein the 3D surface is an interpolated 3D surface mesh, and wherein the 3D surface depicts contours of the cutaneous condition.

12. The system as defined in claim 2 , wherein the computing device is configured to generate the 3D surface by filtering out a set of aberrations in at least one 3D point cloud of the set of 3D point clouds, and wherein the 3D surface is generated by using at least one interpolation algorithm using the computing device.

13. The system as defined in claim 2 , wherein the computing device is configured to analyze the cutaneous condition by measuring a size of the cutaneous condition, determine a variance in the size of the cutaneous condition, and automatically diagnose the cutaneous condition according to the variance in the size of the cutaneous condition.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2020
From: MASSEY, JEREMY MARK; ALAM, SYED HAROON; BIGLIARDI, PAUL LORENZ; BIGLIARDI-QI, MEI
To: IKO PTE. LTD.
Reel/Frame 052398/0655 →
Priority Claims (1)
SG 10201706752X · Aug 17, 2017 · national
Continuity (1)
Related Publication 20200205723A1 · Jul 2, 2020