Granted Patent B2

US 12,400,119 · App. 17/972,831 · Granted Aug 26, 2025

Inventors: Moon Hyun Cha (Suwon-si, KR); Il Chae Jung (Seoul, KR); Bo Hyung Han (Seoul, KR); Daeyoung Park (Ulsan, KR); Changwook Jeong (Ulsan, KR)

Assignees: Samsung Electronics Co., Ltd.; Seoul National University R&DB Foundation

G06N3/082G06N3/045G06T7/20

Abstract

Claims (39)

1. An object tracking learning system, the learning system comprising:

a first neural network module configured to express and learn a first parameter for an input image from a first type to a second type and to output a learned result as a first learning result;

a second neural network module configured to remove and learn a connection of a part of a second parameter for the input image and to output the learned result as a second learning result;

a prediction module configured to generate a prediction value for an object of the input image from a summation result obtained by summing the first learning result and the second learning result; and

an optimization module configured to update the first parameter and the second parameter based on the prediction value.

2. The learning system of claim 1 , wherein the first neural network module includes a (1-1)-th quantization unit configured to quantize a parameter having the first type of a real number type into a parameter having the second type of an integer type.

3. The learning system of claim 2 , wherein the (1-1)-th quantization unit quantizes the parameter having the first type into the parameter having the second type in response to a center and width of an target interval for the parameter of the first type.

4. The learning system of claim 2 , wherein the first neural network module further includes a (1-2)-th quantization unit configured to quantize an activation value having the first type into an activation value having the second type.

5. The learning system of claim 1 , wherein the second neural network module includes a channel pruning unit configured to prune a channel corresponding to a part of the second parameter having the first type.

6. The learning system of claim 1 , wherein the first neural network module includes a (1-1)-th quantization unit configured to quantize a first parameter having the first type of a real number type into a first parameter having the second type of an integer type, and

wherein the second neural network module includes a channel pruning unit configured to prune a channel corresponding to a part of the second parameter having the first type.

7. The learning system of claim 1 , wherein the optimization module includes:

a loss adjusting unit configured to adjust a loss of each of the first learning result, the second learning result, and the prediction value; and

a backpropagation performing unit configured to perform backpropagation based on the adjusted loss.

8. The learning system of claim 1 , further comprising:

a pre-training module configured to perform pre-training on a test image by performing pre-training on the first neural network module and the second neural network module and to set an initial value of a parameter for the input image.

9. The learning system of claim 8 , further comprising:

an online tracking module configured to allow the second neural network module to learn the input image, which is a streaming image, in real time.

10. The learning system of claim 9 , wherein the prediction module includes:

a learning result summation unit configured to generate the summation result by summing the first learning result of the first neural network module, which is pre-trained, and the second learning result of the second neural network module, which is learned online.

11. The learning system of claim 1 , further comprising:

a neural network design module configured to differently set a combination of the first neural network module and the second neural network module based on a first standard.

12. The learning system of claim 1 , wherein the first learning result is expressed in an integer type,

wherein the second learning result is expressed in a real number type, and

wherein the summation result is expressed by mixing the integer type and the real number type.

13. A hybrid neural network-based object tracking learning method, the method comprising:

expressing a first parameter for an input image from a first type to a second type and outputting an expressed result as a first learning result;

removing a connection of a part of a second parameter for the input image and outputting a removed result as a second learning result;

generating a prediction value based on a summation result obtained by summing the first learning result and the second learning result; and

updating the first parameter and the second parameter based on the prediction value.

14. The method of claim 13 , wherein the outputting of the expressed result as the first learning result includes:

quantizing a parameter having the first type of a real number type into a parameter having the second type of an integer type, and

wherein the outputting of the removed result as the second learning result includes:

pruning a channel corresponding to the part of the second parameter.

15. The method of claim 13 , wherein the generating of the prediction value includes:

summing the first learning result of a first neural network module, which is pre-trained, and the second learning result of a second neural network module, which is learned online, as the summation result.

16. The method of claim 13 , wherein the first learning result is expressed in an integer type,

wherein the second learning result is expressed in a real number type, and

wherein the summation result obtained by summing the first learning result and the second learning result is expressed by mixing the integer type and the real number type.

Assignments (1)

Priority Claims (2)

Continuity (1)

References Cited (35)