Intelligent body measurement
A method for estimating the body measurements of a subject from at least two photographic images of the subject. The method includes capturing the photographic images using a digital imaging device, for example, a mobile device camera, and estimating the body measurements of the subject using heatmap data generated by an intelligent computing system such as a trained neural network.
1. A method for estimating body measurements of a subject, comprising:
receiving at least two images containing the subject;
generating landmark heatmap data from the at least two images containing the subject;
relating the landmark heatmap data to an abstract three-dimensional body model to determine a shape of the subject's body,
wherein relating the landmark heatmap data to the abstract three-dimensional body model to determine the shape of the subject's body comprises extracting a plurality of contour points from the landmark heatmap data, and wherein the plurality of contour points comprises at least one control point,
generating the abstract three-dimensional body model using the plurality of contour points, wherein the abstract three-dimensional body model comprises a plurality of body slices, and wherein each slice of the plurality of body slices represents an anthropometric measurement of a body that influences body shape,
projecting each body slice of the plurality of body slices as a corresponding two-dimensional landmark,
generating at least one main cross-section value by matching the at least one control point to the corresponding two-dimensional landmark, wherein the at least one main cross-section value is used to construct a cross-section curve,
extracting a vertex from the cross-section curve, and
constructing a surface of a part of the subject's body using the vertex from the cross-section curve; and
estimating the body measurements of the subject based on the shape of the subject's body.
2. The method of claim 1 , wherein the at least two images containing the subject are generated from a digital imaging device.
3. The method of claim 2 , wherein the digital imaging device comprises a mobile device camera.
4. The method of claim 1 , wherein the landmark heatmap data is generated using an intelligent computing system.
5. The method of claim 4 , wherein the intelligent computing system comprises a trained neural network.
6. The method of claim 1 , wherein generating the landmark heatmap data from the at least two images containing the subject comprises:
generating a first portion of the landmark heatmap data using a first intelligent computing system, wherein the first intelligent computing system is trained to analyze front-view photographic images, wherein one of the at least two images containing the subject is a front-view image containing the subject, and wherein the first intelligent computing system analyzes the front-view image containing the subject to generate the first portion of the landmark heatmap data; and
generating a second portion of the landmark heatmap data using a second intelligent computing system, wherein the second intelligent computing system is trained to analyze side-view photographic images, wherein another of the at least two images containing the subject is a side-view image containing the subject, and wherein the second intelligent computing system analyzes the side-view image containing the subject to generate the second portion of the landmark heatmap data.
7. The method of claim 6 , wherein generating the first portion of the landmark heatmap data using the first intelligent computing system comprises:
computing two-dimensional probability map distribution data using the front-view image containing the subject, wherein the two-dimensional probability map distribution data represents the probability that a pixel is associated with a part of the subject's body; and
calculating a two-dimensional coordinate value (x i , y i ) for a keypoint i using the formula:
x i ,y i =argmax( H i ),
where, H i is a two-dimensional probability map of the i-th keypoint, wherein the i-th keypoint represents an area in the front-view image containing the subject where the part of the subject's body may be located.
8. The method of claim 6 , wherein generating the second portion of the landmark heatmap data using the second intelligent computing system comprises:
computing two-dimensional probability map distribution data using the side-view image containing the subject, wherein the two-dimensional probability map distribution data represents the probability that a pixel is associated with a part of the subject's body; and
calculating a two-dimensional coordinate value (x i , y i ) for a keypoint i using the formula:
x i ,y i =argmax( H i ),
where, H i is a two-dimensional probability map of the i-th keypoint, wherein the i-th keypoint represents an area in the side-view image containing the subject where the part of the subject's body may be located.
9. The method of claim 6 , wherein the first intelligent computing system and the second intelligent computing system each comprise a trained neural network.
10. The method of claim 1 , wherein projecting each body slice of the plurality of body slices as the two-dimensional landmark comprises using a virtual camera to project each body slice of the plurality of body slices as the two-dimensional landmark.
11. The method of claim 10 , further comprising:
proportionally aligning a scale of the three-dimensional body model relative to the virtual camera used to project each body slice of the plurality of body slices, wherein a height of the subject is used to proportionally align the scale of the three-dimensional body model relative to the virtual camera.
12. The method of claim 1 , further comprising:
approximating the plurality of body slices using B-spline curves.
13. The method of claim 12 , wherein the plurality of body slices is constructed as linear combinations of the B-spline curves and the at least one control point using the formula:
C ( t )=Σ i−0 n N i,p ( t )·( P i )
where basis functions are described using a recursion formula:
N
i
,
0
(
t
)
=
{
1
,
t
i
≤
t
≤
t
i
+
1
0
,
in
other
case
N
i
,
j
(
t
)
=
t
-
t
i
t
i
+
j
-
t
i
·
N
i
,
j
-
1
(
t
)
+
t
i
+
j
+
1
-
t
t
i
+
j
+
1
-
t
i
+
1
·
N
i
+
j
+
1
(
t
)
;
j
=
1
,
p
.
14. The method of claim 13 , wherein the recursion formula comprises a Cox de Boor recursion formula.
15. The method of claim 1 , wherein estimating the body measurements of the subject based on the shape of the subject's body comprises:
calculating a perimeter of the cross-section curve using a curvilinear integral formula:
L (γ)=∫ dγ,
where γ is the cross-section curve, and L(γ) is the perimeter.
16. The method of claim 1 , wherein the body measurements of the subject comprise chest volume, waist volume, hip volume, front shoulders, front chest, front waist, front hips, shoulder-to-waist length, sleeve length, body length, leg length, outside leg length, and inside leg length.
17. The method of claim 1 , further comprising:
detecting quality criteria of at least one of the at least two images containing the subject to determine whether the at least one of the at least two images containing the subject meets pre-defined quality standards.
18. The method of claim 17 , wherein detecting the quality criteria of the at least one of the at least two images containing the subject comprises:
generating segment heatmap data from the at least one of the at least two images containing the subject;
generating mask heatmap data from the at least one of the at least two images containing the subject;
extracting features from the landmark heatmap data of the at least one of the at least two images containing the subject; and
analyzing the features extracted from the landmark heatmap data to determine whether the at least one of the at least two images satisfies a first pre-defined criterion.
19. The method of claim 18 , wherein the features extracted from the landmark heatmap data comprise number of zero landmarks, landmark spread, and landmark max.
20. The method of claim 19 , wherein the first pre-defined criterion comprises the number of zero landmarks equals zero.
21. The method of claim 18 , further comprising:
extracting features from the segment heatmap data of the at least one of the at least two images containing the subject;
extracting features from the mask heatmap data of the at least one of the at least two images containing the subject;
analyzing the features extracted from the segment heatmap data; and
analyzing the features extracted from the mask heatmap data.
22. The method of claim 21 , wherein the features extracted from the segment heatmap data comprise segment quality, segment max, segment poles, and number of zero segments.
23. The method of claim 21 , wherein the features extracted from the mask heatmap data comprise mask quality.
24. A system for estimating body measurements of a subject, comprising:
at least one server capable of storing or processing, or both storing and processing a plurality of photographic images of a subject;
at least one intelligent computing system coupled to the at least one server, wherein the at least one intelligent computing system capable of analyzing the plurality of photographic images of the subject to generate landmark heatmap, contour segment heatmap data, and mask heatmap data; and
a module capable of relating the landmark heatmap data to an abstract three-dimensional body model to determine a shape of the subject's body and estimating body measurements of the subject based on at least the landmark heatmap data,
wherein the abstract three-dimensional body model is generated using a plurality of contour points, and wherein the plurality of contour points is extracted from the landmark heatmap data, and the plurality of contour points comprises at least one control point,
wherein the abstract three-dimensional body model comprises a plurality of body slices and each slice of the plurality of body slices represents an anthropometric measurement of a body that influences body shape,
wherein each body slice of the plurality of body slices is projected as a corresponding two-dimensional landmark,
wherein at least one main cross-section value is generated by matching the at least one control point to the corresponding two-dimensional landmark, and wherein the at least one main cross-section value is used to construct a cross-section curve,
wherein a vertex is extracted from the cross-section curve, and
wherein a surface of a part of the subject's body is constructed using the vertex from the cross-section curve.
25. The system of claim 24 , further comprising:
at least one detector coupled to the least one intelligent computing system, the at least one detector capable of detecting quality of the plurality of photographic images based on the landmark heatmap data, the contour segment heatmap data, and the mask heatmap data.
26. The system of claim 24 , wherein the at least one intelligent computing system is a trained neural network.