IP Library Granted Patent US 10,810,746
Granted Patent B2
US 10,810,746 · App. 15/975,164 · Granted Oct 20, 2020

Target tracking hardware implementation system and method

Inventors: Xijie Jia (Beijing, CN); Di Wu (Beijing, CN); Hanbo Sun (Beijing, CN)
Assignee: XILINX TECHNOLOGY BEIJING LIMITED
G06T7/248G06K9/00711G06K9/00744G06K9/4671G06K9/6202G06K9/6247G06K9/6256G06K2009/3291G06K2209/21G06T2207/10016G06T2207/20081G06T2207/30232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,810,746
App. No.
15/975,164
Granted
Oct 20, 2020
Kind
B2
Abstract

A target tracking hardware implementation system and method are provided. The target tracking hardware implementation system ( 200 ) includes: an intercepting and scaling module ( 210 ) configured to intercept a target image and to-be-checked images of a plurality of sizes from a video in an external storage module ( 300 ), and normalize the images to a specified size by scaling; a feature extracting module ( 220 ) configured to extract features of the normalized images; a feature managing module ( 230 ) configured to access and update a matching template based on the extracted image features; a Kernelized Correlation Filter (KCF) calculating module ( 240 ) configured to calculate the KCF response of each of the to-be-checked images based on the extracted image features and the updated matching template, and select the to-be-checked image with the largest KCF response as the tracked target. The system and method integrate the advantages of the existing KCF algorithm and are implemented on a parallel hardware platform, have the advantages of being small in size, low in power consumption and high in real-timeliness.

Claims (56)

1. A target tracking hardware implementation system, comprising:

an intercepting and scaling circuit configured to intercept a target image and to-be-checked images of a plurality of sizes from a video in an external storage module, and normalize the target image and the to-be-checked images to a specified size by scaling;

a feature extractor configured to extract image features of the normalized images;

a feature manager configured to read a history train matching template from the external storage module and update a feature train matching template using a feature vector of the normalized target image, wherein the feature manager employs a ping-pong buffer structure to access and use the extracted image features; and

a Kernelized Correlation Filter (KCF) calculator configured to calculate a KCF response of each of the normalized to-be-checked images based on the extracted image features and the updated feature train matching template, and select a to-be-checked image with a largest KCF response as a tracked target.

2. The system of claim 1 , wherein the video is a high-resolution video.

3. The system of claim 1 , wherein the feature extractor is further configured to extract a Histogram of Gradient (HOG) feature, and perform normalization, Principal Component Analysis (PCA) dimension reduction, and Hanning window weighting on the extracted HOG feature.

4. The system of claim 1 , wherein the intercepting and scaling circuit is further configured to:

acquire a previous frame from the video as a training image frame, acquire a current frame as a to-be-checked image frame;

extract a target position from the training image frame, and extract to-be- checked positions of a plurality of sizes based on the target position; and

intercept the target image from the training image frame and normalize the target image to a specified size by scaling, intercept the to-be-checked images at the to-be-checked positions from the to-be-checked image frame and normalize the to-be-checked images to the same specified size by scaling.

5. The system of claim 1 , wherein

the feature extractor is further configured to extract the feature vector of the normalized target image and feature vectors of the normalized to-be-checked images and send the extracted feature vectors to the feature manager for storage;

the KCF calculator is further configured to calculate and generate a KCF training coefficient based on the feature vector of the normalized target image read from the feature manager on a basis of a discrete Fourier transform, thereby updating a KCF training coefficient matching template; use the updated KCF training coefficient matching template to calculate KCF responses of the feature vectors of the normalized to-be-checked images, based on the updated feature train matching template read from the feature manager and the feature vectors of the normalized to-be-checked images read from the feature manager; and use the to-be-checked image with the largest KCF response as the tracked target to convert a tracked target size and an offset distance.

6. A target tracking method, comprising:

intercepting a target image and to-be-checked images of a plurality of sizes from a video in an external storage module and normalizing the target image and the to-be-checked images to a specified size by scaling by an intercepting and scaling module;

extracting image features of the normalized images by a feature extracting module;

reading a history train matching template from the external storage module and updating a feature train matching template using a feature vector of the normalized target image by a feature managing module, wherein the feature managing module employs a ping-pong buffer structure to access and use the extracted image features; and

calculating a KCF response of each of the normalized to-be-checked images based on the extracted image features and the updated feature train matching template, and selecting a to-be-checked image with a largest KCF response as a tracked target by a Kernelized Correlation Filter (KCF) calculating module.

7. The method of claim 6 , wherein the video is a high-resolution video.

8. The method of claim 6 , wherein the feature extracting module is further configured to extract a Histogram of Gradient (HOG) feature, and perform normalization, Principal Component Analysis (PCA) dimension reduction, and Hanning window weighting on the extracted HOG feature.

9. The method of claim 6 , wherein the step of intercepting further comprises:

acquiring a previous frame from the video as a training image frame, acquiring a current frame as a to-be-checked image frame;

extracting a target position from the training image frame, and extracting to-be- checked positions of a plurality of sizes based on the target position; and

intercepting the target image from the training image frame and normalizing the target image to a specified size by scaling, intercepting the to-be-checked images at the to-be-checked positions from the to-be-checked image frame and normalizing the to-be-checked images to the same specified size by scaling.

10. The method of claim 6 , wherein:

the step of extracting features of the normalized images further comprises: extracting the feature vector of the normalized target image and feature vectors of the normalized to-be-checked images for storage;

the step of calculating the KCF response of each of the normalized to-be-checked images based on the extracted image features and the updated feature train matching template and selecting the to-be-checked image with the largest KCF response as the tracked target further comprises: calculating and generating a KCF training coefficient based on the feature vector of the normalized target image on a basis of a discrete Fourier transform, thereby updating a KCF training coefficient matching template; using the updated KCF training coefficient matching template to calculate KCF responses of the feature vectors of the to-be-checked normalized images based on the updated feature train matching template and the feature vector of the normalized to-be-checked images; and using the to-be-checked image with the largest KCF response as the tracked target to convert a tracked target size and an offset distance.

11. The method of claim 6 , further comprising: using a plurality of sets of computing resources to traverse a plurality of targets in parallel until the tracking of all the targets is completed.

12. The method of claim 9 , further comprising: using the current frame as a training image frame, reading a next frame from the video as the to-be-checked image frame, and performing the control method by sequentially traversing all image frames in the video until an end of the video.

13. A non-transitory computer-readable medium, being configured to record instructions executable by a processor, the instructions, when executed by the processor, cause the processor to perform a target tracking method, the method comprising the following operations:

intercepting a target image and to-be-checked images of a plurality of sizes from a video in an external storage module and normalizing the target image and the to-be-checked images to a specified size by scaling;

extracting image features of the normalized images;

reading a history train matching template from the external storage module and updating a feature train matching template using a feature vector of the normalized target image by a feature managing module that employs a ping-pong buffer structure to access and use the extracted image features; and

calculating a Kernelized Correlation Filter (KCF) response of each of the normalized to-be-checked images based on the extracted image features and the updated feature train matching template, and selecting a to-be-checked image with a largest KCF response as a tracked target.

14. A target tracking hardware implementation system, comprising:

an intercepting and scaling circuit configured to intercept a target image and to-be-checked images of a plurality of sizes from a video in an external storage module, and normalize the target image and the to-be-checked images to a specified size by scaling;

a feature extractor configured to extract image features of the normalized images;

a feature manager configured to read a history train matching template from the external storage module and update a feature train matching template using a feature vector of the extracted target image; and

a Kernelized Correlation Filter (KCF) calculator configured to calculate a KCF response of each of the normalized to-be-checked images based on the extracted image features and the updated feature train matching template, and select a to-be-checked image with a largest KCF response as a tracked target,

wherein the feature extractor is further configured to extract the feature vector of the normalized target image and feature vectors of the normalized to-be-checked images and send the extracted feature vectors to the feature manager for storage; and

wherein the KCF calculator is further configured to calculate and generate a KCF training coefficient based on the feature vector of the extracted target image read from the feature manager on a basis of a discrete Fourier transform, thereby updating a KCF training coefficient matching template; use the updated KCF training coefficient matching template to calculate KCF responses of the feature vectors of the normalized to-be-checked images, based on the updated feature train matching template read from the feature manager and the feature vectors of the normalized to-be-checked images read from the feature manager; and use the normalized to-be-checked image with the largest KCF response as the tracked target to convert a tracked target size and an offset distance.

15. The system of claim 14 , wherein the video is a high-resolution video.

16. The system of claim 14 , wherein the feature extractor is further configured to extract a Histogram of Gradient (HOG) feature, and perform normalization, Principal Component Analysis (PCA) dimension reduction, and Hanning window weighting on the extracted HOG feature.

17. A target tracking method, comprising:

intercepting a target image and to-be-checked images of a plurality of sizes from a video in an external storage module and normalizing the target image and the to-be-checked images to a specified size by scaling by an intercepting and scaling module;

extracting image features of the normalized images by a feature extracting module including extracting a feature vector of the normalized target image and feature vectors of the normalized to-be-checked images for storage;

reading a history train matching template from the external storage module and updating a feature train matching template using the feature vector of the normalized target image by a feature managing module; and

calculating a KCF response of each of the normalized to-be-checked images based on the extracted image features and the updated feature train matching template, and selecting a to-be-checked image with a largest KCF response as a tracked target by a Kernelized Correlation Filter (KCF) calculating module, including:

calculating and generating a KCF training coefficient based on the feature vector of the normalized target image on a basis of a discrete Fourier transform, thereby updating a KCF training coefficient matching template; using the updated KCF training coefficient matching template to calculate KCF responses of the feature vectors of the normalized to-be-checked images based on the updated feature train matching template and the feature vector of the normalized to-be-checked images; and using the to-be-checked image with the largest KCF response as the tracked target to convert a tracked target size and an offset distance.

18. The method of claim 17 , wherein the video is a high-resolution video.

19. The method of claim 17 , wherein the feature extracting module is further configured to extract a Histogram of Gradient (HOG) feature, and perform normalization, Principal Component Analysis (PCA) dimension reduction, and Hanning window weighting on the extracted HOG feature.

20. The method of claim 17 , wherein the step of intercepting and normalizing further comprises:

acquiring a previous frame from the video as a training image frame, acquiring a current frame as a to-be-checked image frame;

extracting a target position from the training image frame, and extracting to-be-checked positions of a plurality of sizes based on the target position; and

intercepting the target image from the training image frame and normalizing the target image to a specified size by scaling, intercepting the to-be-checked images at the to-be-checked positions from the to-be-checked image frame and normalizing the to-be-checked images to the same size specified by scaling.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2020
From: BEIJING DEEPHI INTELLIGENT TECHNOLOGY CO., LTD.
To: XILINX TECHNOLOGY BEIJING LIMITED
Reel/Frame 053581/0037 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 25, 2018
From: BEIJING DEEPHI TECHNOLOGY CO., LTD.
To: BEIJING DEEPHI INTELLIGENT TECHNOLOGY CO., LTD.
Reel/Frame 045906/0478 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 9, 2018
From: JIA, XIJIE; WU, DI; SUN, HANBO
To: BEIJING DEEPHI TECHNOLOGY CO., LTD.
Reel/Frame 045755/0071 →
Priority Claims (1)
CN 2017 1 1067602 · Nov 3, 2017 · national
Continuity (1)
Related Publication 20190139232A1 · May 9, 2019