Granted Patent B2

US 12,731,284 · App. 18/727,184 · Granted Sep 8, 2026

Inventor: Yadong Pan (Tokyo, JP)

Assignee: NEC CORPORATION

G06T7/73G06V10/82G06V40/10G06T2207/20084G06T2207/30196

Abstract

Claims (104)

1 . A pose estimation apparatus comprising:

at least one memory that is configured to store instructions; and

at least one processor that is configured to execute the instructions to:

acquire a target image in which one or more persons are captured;

detect key-points from the target image;

compute one or more selection factors based on the key-points, the one or more selection factors including density, resolution, or both of persons in the target image;

select an algorithm for key-point association from predefined algorithms for key-point association based on the one or more selection factors;

perform key-point association on the key-points using the selected algorithm to divide the key-points into one or more key-point groups each of which includes the key-points of a same person as each other; and

for each of the key-point groups, estimate a pose of the person corresponding to the key-point group based on the key-points included in the key-point group.

2 . The pose estimation apparatus according to claim 1 ,

wherein types of the key-points include right shoulder and left shoulder, and

wherein the computation of the density of persons in the target image includes:

for each of the key-points of left shoulder, liking the key-point of left shoulder with the key-point of right shoulder closest thereto;

for each of the key-point of right shoulder that is linked with multiple key-points of left shoulder, deleting a longest link having the key-point of right shoulder when a length of the longest link is more than predetermined times of a length of a shortest link having the key-point of right shoulder; and

computing the density of persons in the target image based on a number of the key-points of right shoulder that are linked with multiple key-points of left shoulder.

3 . The pose estimation apparatus according to claim 1 ,

wherein types of the key-points include right shoulder and left shoulder, and

wherein the computation of the resolution of persons in the target image includes:

for each of the key-points of left shoulder, liking the key-point of left shoulder with the key-point of right shoulder closest thereto;

for each of the key-point of right shoulder that is linked with multiple key-points of left shoulder, deleting a longest link having the key-point of right shoulder when a length of the longest link is more than predetermined times of a length of a shortest link having the key-point of right shoulder; and

computing the resolution of persons in the target image based on a number of the links remained after the deletion and a number of the links whose lengths are less than a threshold that is defined based on one of dimensions of the target image.

4 . The pose estimation apparatus according to claim 1 ,

wherein the predefined algorithms include mid-point algorithm, direction map algorithm, location map algorithm, or two or three of them.

5 . The pose estimation apparatus according to claim 1 , wherein the selection algorithm for key-point association includes:

determining whether or not the selection factor is less than a threshold of the selection factor;

selecting a first algorithm for key-point association when the selection factor is less than the threshold of the selection factor; and

selecting a second algorithm for key-point association when the selection factor is not less than the threshold of the selection factor.

6 . The pose estimation apparatus according to claim 5 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the resolution of persons in the target image is less than a threshold of the resolution;

selecting mid-point algorithm when the resolution of persons in the target image is less than the threshold of the resolution; and

selecting direction map algorithm or location map algorithm when the resolution of persons in the target image is not less than the threshold of the resolution.

7 . The pose estimation apparatus according to claim 5 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the density of persons in the target image is less than a threshold of the density;

selecting mid-point algorithm or direction map algorithm when the density of persons in the target image is less than the threshold of the density; and

selecting location map algorithm when the density of persons in the target image is not less than the threshold of the density.

8 . A pose estimation method performed by one or more computers comprising:

acquiring a target image in which one or more persons are captured;

detect key-points from the target image;

computing one or more selection factors based on the key-points, the one or more selection factors including density, resolution, or both of persons in the target image;

selecting an algorithm for key-point association from predefined algorithms for key-point association based on the one or more selection factors;

performing key-point association on the key-points using the selected algorithm to divide the key-points into one or more key-point groups each of which includes the key-points of a same person as each other; and

for each of the key-point groups, estimating a pose of the person corresponding to the key-point group based on the key-points included in the key-point group.

9 . The pose estimation method according to claim 8 ,

wherein types of the key-points include right shoulder and left shoulder, and

wherein the computation of the density of persons in the target image includes:

for each of the key-points of left shoulder, liking the key-point of left shoulder with the key-point of right shoulder closest thereto;

for each of the key-point of right shoulder that is linked with multiple key-points of left shoulder, deleting a longest link having the key-point of right shoulder when a length of the longest link is more than predetermined times of a length of a shortest link having the key-point of right shoulder; and

computing the density of persons in the target image based on a number of the key-points of right shoulder that are linked with multiple key-points of left shoulder.

10 . The pose estimation method according to claim 8 ,

wherein types of the key-points include right shoulder and left shoulder, and

wherein the computation of the resolution of persons in the target image includes:

for each of the key-points of left shoulder, liking the key-point of left shoulder with the key-point of right shoulder closest thereto;

for each of the key-point of right shoulder that is linked with multiple key-points of left shoulder, deleting a longest link having the key-point of right shoulder when a length of the longest link is more than predetermined times of a length of a shortest link having the key-point of right shoulder; and

computing the resolution of persons in the target image based on a number of the links remained after the deletion and a number of the links whose lengths are less than a threshold that is defined based on one of dimensions of the target image.

11 . The pose estimation method according to claim 8 ,

wherein the predefined algorithms include mid-point algorithm, direction map algorithm, location map algorithm, or two or three of them.

12 . The pose estimation method according to claim 8 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the selection factor is less than a threshold of the selection factor;

selecting a first algorithm for key-point association when the selection factor is less than the threshold of the selection factor; and

selecting a second algorithm for key-point association when the selection factor is not less than the threshold of the selection factor.

13 . The pose estimation method according to claim 12 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the resolution of persons in the target image is less than a threshold of the resolution;

selecting mid-point algorithm when the resolution of persons in the target image is less than the threshold of the resolution; and

selecting direction map algorithm or location map algorithm when the resolution of persons in the target image is not less than the threshold of the resolution.

14 . The pose estimation method according to claim 12 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the density of persons in the target image is less than a threshold of the density;

selecting mid-point algorithm or direction map algorithm when the density of persons in the target image is less than the threshold of the density; and

selecting location map algorithm when the density of persons in the target image is not less than the threshold of the density.

15 . A non-transitory computer-readable storage medium storing a program that causes one or more computers to execute:

acquiring a target image in which one or more persons are captured;

detect key-points from the target image;

computing one or more selection factors based on the key-points, the one or more selection factors including density, resolution, or both of persons in the target image;

selecting an algorithm for key-point association from predefined algorithms for key-point association based on the one or more selection factors;

performing key-point association on the key-points using the selected algorithm to divide the key-points into one or more key-point groups each of which includes the key-points of a same person as each other; and

for each of the key-point groups, estimating a pose of the person corresponding to the key-point group based on the key-points included in the key-point group.

16 . The storage medium according to claim 15 ,

wherein types of the key-points include right shoulder and left shoulder, and

wherein the computation of the density of persons in the target image includes:

for each of the key-points of left shoulder, liking the key-point of left shoulder with the key-point of right shoulder closest thereto;

for each of the key-point of right shoulder that is linked with multiple key-points of left shoulder, deleting a longest link having the key-point of right shoulder when a length of the longest link is more than predetermined times of a length of a shortest link having the key-point of right shoulder; and

computing the density of persons in the target image based on a number of the key-points of right shoulder that are linked with multiple key-points of left shoulder.

17 . The storage medium according to claim 15 ,

wherein types of the key-points include right shoulder and left shoulder, and

wherein the computation of the resolution of persons in the target image includes:

for each of the key-points of left shoulder, liking the key-point of left shoulder with the key-point of right shoulder closest thereto;

for each of the key-point of right shoulder that is linked with multiple key-points of left shoulder, deleting a longest link having the key-point of right shoulder when a length of the longest link is more than predetermined times of a length of a shortest link having the key-point of right shoulder; and

computing the resolution of persons in the target image based on a number of the links remained after the deletion and a number of the links whose lengths are less than a threshold that is defined based on one of dimensions of the target image.

18 . The storage medium according to claim 15 ,

wherein the predefined algorithms include mid-point algorithm, direction map algorithm, location map algorithm, or two or three of them.

19 . The storage medium according to claim 15 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the selection factor is less than a threshold of the selection factor;

selecting a first algorithm for key-point association when the selection factor is less than the threshold of the selection factor; and

selecting a second algorithm for key-point association when the selection factor is not less than the threshold of the selection factor.

20 . The storage medium according to claim 19 ,

wherein the selection algorithm for key-point association includes:

determining whether or not the resolution of persons in the target image is less than a threshold of the resolution;

selecting mid-point algorithm when the resolution of persons in the target image is less than the threshold of the resolution; and

selecting direction map algorithm or location map algorithm when the resolution of persons in the target image is not less than the threshold of the resolution.

Assignments (1)

Continuity (1)

References Cited (12)