IP Library Granted Patent US 11,087,476
Granted Patent B2
US 11,087,476 · App. 16/890,087 · Granted Aug 10, 2021

Trajectory tracking method and apparatus, computer device, and storage medium

Inventors: Changwei He (Shenzhen, CN); Chengjie Wang (Shenzhen, CN); Jilin Li (Shenzhen, CN); Yabiao Wang (Shenzhen, CN); Yandan Zhao (Shenzhen, CN); Yanhao Ge (Shenzhen, CN); Hui Ni (Shenzhen, CN); Yichao Xiong (Shenzhen, CN); Zhenye Gan (Shenzhen, CN); Yongjian Wu (Shenzhen, CN); Feiyue Huang (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G06T7/246G06K9/00295G06K9/6215G06T7/62G06T7/73G06T2207/10016G06T2207/30201G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,087,476
App. No.
16/890,087
Granted
Aug 10, 2021
Kind
B2
Abstract

A trajectory tracking method is provided for a computer device. The method includes performing motion tracking on head images in a plurality of video frames, to obtain motion trajectories corresponding to the head images; acquiring face images corresponding to the head images in the video frames, to obtain face image sets corresponding to the head images; determining from the face image sets corresponding to the head images, at least two face image sets having same face images; and combining motion trajectories corresponding to the at least two face image sets having same face images, to obtain a final motion trajectory of trajectory tracking.

Claims (82)

1. A trajectory tracking method for a computer device, comprising:

performing motion tracking on head images in a plurality of video frames, to obtain motion trajectories corresponding to the head images;

acquiring face images corresponding to the head images in the video frames, to obtain face image sets corresponding to the head images;

determining from the face image sets corresponding to the head images, at least two face image sets having same face images; and

combining motion trajectories corresponding to the at least two face image sets having same face images, to obtain a final motion trajectory of trajectory tracking.

2. The trajectory tracking method according to claim 1 , wherein:

before the performing motion tracking on head images in a plurality of video frames, the method further comprises:

performing head detection and face detection on the plurality of video frames, to determine head images and face images in the video frames; and

determining head images and face images that are in the video frames and belong to a same person, and establishing correspondences between the head images and the face images that belong to a same person; and

the acquiring face images corresponding to the head images in the video frames comprises:

acquiring according to the established correspondences, the face images corresponding to the head images in the video frames.

3. The trajectory tracking method according to claim 2 , wherein the determining head images and face images that are in the video frames and belong to a same person comprises:

obtaining the area overlap ratios between the head images and the face images that are determined from the video frames; and

determining according to the area overlap ratios obtained from the video frames, head images and face images that are in the video frames and belong to a same person.

4. The trajectory tracking method according to claim 3 , wherein the determining according to the area overlap ratios obtained from the video frames, head images and face images that are in the video frames and belong to a same person comprises:

performing bipartite graph optimal matching on the head images and the face images in the video frames according to the area overlap ratios obtained from the video frames; and

determining that head images and face images that are in the video frames and match each other belong to a same person.

5. The trajectory tracking method according to claim 1 , wherein the determining from the face image sets corresponding to the head images, at least two face image sets having the same face images comprises:

selecting two face image sets from the face image sets corresponding to the head images;

obtaining similarities among face images in the two face image sets, to obtain a plurality of similarities; and

determining, in a case that a similarity reaching a first preset similarity exists in the plurality of similarities, that the two face image sets have same face images.

6. The trajectory tracking method according to claim 5 , wherein before the determining that the two face image sets have same face images, the method further comprises:

obtaining an average similarity of the plurality of similarities in a case that none of the plurality of similarities reaches the first preset similarity; and

determining, in a case that the average similarity reaches a second preset similarity, that the two face image sets have same face images, the second preset similarity being less than the first preset similarity.

7. The trajectory tracking method according to claim 5 , wherein the obtaining similarities among face images in the two face image sets comprises:

separately selecting one face image from the two face image sets, to obtain two face images;

separately obtaining multidimensional image features of the two face images, to obtain a first image feature set and a second image feature set; and

obtaining a distance between the first image feature set and the second image feature set, and using the obtained distance as a similarity between the two face images.

8. A computer device, comprising:

a memory storing computer-readable instructions; and

a processor coupled to the memory and, when executing the computer-readable instructions, configured to perform:

performing motion tracking on head images in a plurality of video frames, to obtain motion trajectories corresponding to the head images;

acquiring face images corresponding to the head images in the video frames, to obtain face image sets corresponding to the head images;

determining from the face image sets corresponding to the head images, at least two face image sets having same face images; and

combining motion trajectories corresponding to the at least two face image sets having same face images, to obtain a final motion trajectory of trajectory tracking.

9. The computer device according to claim 8 , wherein:

before the performing motion tracking on head images in a plurality of video frames, the processor is further configured to perform:

performing head detection and face detection on the plurality of video frames, to determine head images and face images in the video frames; and

determining head images and face images that are in the video frames and belong to a same person, and establishing correspondences between the head images and the face images that belong to a same person; and

the acquiring face images corresponding to the head images in the video frames comprises:

acquiring according to the established correspondences, the face images corresponding to the head images in the video frames.

10. The computer device according to claim 9 , wherein the determining head images and face images that are in the video frames and belong to a same person comprises:

obtaining the area overlap ratios between the head images and the face images that are determined from the video frames; and

determining according to the area overlap ratios obtained from the video frames, head images and face images that are in the video frames and belong to a same person.

11. The computer device according to claim 10 , wherein the determining according to the area overlap ratios obtained from the video frames, head images and face images that are in the video frames and belong to a same person comprises:

performing bipartite graph optimal matching on the head images and the face images in the video frames according to the area overlap ratios obtained from the video frames; and

determining that head images and face images that are in the video frames and match each other belong to a same person.

12. The computer device according to claim 8 , wherein the determining from the face image sets corresponding to the head images, at least two face image sets having the same face images comprises:

selecting two face image sets from the face image sets corresponding to the head images;

obtaining similarities among face images in the two face image sets, to obtain a plurality of similarities; and

determining, in a case that a similarity reaching a first preset similarity exists in the plurality of similarities, that the two face image sets have same face images.

13. The computer device according to claim 12 , wherein before the determining that the two face image sets have same face images, the processor is further configured to perform:

obtaining an average similarity of the plurality of similarities in a case that none of the plurality of similarities reaches the first preset similarity; and

determining, in a case that the average similarity reaches a second preset similarity, that the two face image sets have same face images, the second preset similarity being less than the first preset similarity.

14. The computer device according to claim 12 , wherein the obtaining similarities among face images in the two face image sets comprises:

separately selecting one face image from the two face image sets, to obtain two face images;

separately obtaining multidimensional image features of the two face images, to obtain a first image feature set and a second image feature set; and

obtaining a distance between the first image feature set and the second image feature set, and using the obtained distance as a similarity between the two face images.

15. A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor to perform:

performing motion tracking on head images in a plurality of video frames, to obtain motion trajectories corresponding to the head images;

acquiring face images corresponding to the head images in the video frames, to obtain face image sets corresponding to the head images;

determining from the face image sets corresponding to the head images, at least two face image sets having same face images; and

combining motion trajectories corresponding to the at least two face image sets having same face images, to obtain a final motion trajectory of trajectory tracking.

16. The non-transitory computer-readable storage medium according to claim 15 , wherein:

before the performing motion tracking on head images in a plurality of video frames, the processor is further to perform:

performing head detection and face detection on the plurality of video frames, to determine head images and face images in the video frames; and

determining head images and face images that are in the video frames and belong to a same person, and establishing correspondences between the head images and the face images that belong to a same person; and

the acquiring face images corresponding to the head images in the video frames comprises:

acquiring according to the established correspondences, the face images corresponding to the head images in the video frames.

17. The non-transitory computer-readable storage medium according to claim 16 , wherein the determining head images and face images that are in the video frames and belong to a same person comprises:

obtaining the area overlap ratios between the head images and the face images that are determined from the video frames; and

determining according to the area overlap ratios obtained from the video frames, head images and face images that are in the video frames and belong to a same person.

18. The non-transitory computer-readable storage medium according to claim 17 , wherein the determining according to the area overlap ratios obtained from the video frames, head images and face images that are in the video frames and belong to a same person comprises:

performing bipartite graph optimal matching on the head images and the face images in the video frames according to the area overlap ratios obtained from the video frames; and

determining that head images and face images that are in the video frames and match each other belong to a same person.

19. The non-transitory computer-readable storage medium according to claim 15 , wherein the determining from the face image sets corresponding to the head images, at least two face image sets having the same face images comprises:

selecting two face image sets from the face image sets corresponding to the head images;

obtaining similarities among face images in the two face image sets, to obtain a plurality of similarities; and

determining, in a case that a similarity reaching a first preset similarity exists in the plurality of similarities, that the two face image sets have same face images.

20. The non-transitory computer-readable storage medium according to claim 19 , wherein before the determining that the two face image sets have same face images, the method further comprises:

obtaining an average similarity of the plurality of similarities in a case that none of the plurality of similarities reaches the first preset similarity; and

determining, in a case that the average similarity reaches a second preset similarity, that the two face image sets have same face images, the second preset similarity being less than the first preset similarity.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2020
From: HE, CHANGWEI; WANG, CHENGJIE; LI, JILIN; WANG, YABIAO; ZHAO, YANDAN; GE, YANHAO; NI, HUI; XIONG, YICHAO; GAN, ZHENYE; WU, YONGJIAN; HUANG, FEIYUE
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 052809/0059 →
Priority Claims (1)
CN 201810267283.4 · Mar 28, 2018 · national
Continuity (2)
Continuation PCTCN2019078529 · Mar 18, 2019
Related Publication 20200294250A1 · Sep 17, 2020