IP Library › Granted Patent US 9,582,726
Granted Patent B2
US 9,582,726 · App. 14/749,387 · Granted Feb 28, 2017

Systems and methods for image processing in a deep convolution network

Inventors: Yunke Pan (San Diego, CA); Dashan Gao (San Diego, CA); Xin Zhong (San Diego, CA); Shuxue Quan (San Diego, CA)
Assignee: QUALCOMM Incorporated
G06K9/00986G06T5/20G06T11/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,582,726
App. No.
14/749,387
Granted
Feb 28, 2017
Kind
B2
Abstract

A method performed by an electronic device is described. The method includes interleaving multiple input image channels to produce an interleaved multi-channel input. The method also includes loading the interleaved multi-channel input to a single-instruction multiple data (SIMD) processor. The method further includes convolving the interleaved multi-channel input with a multi-channel filter.

Claims (51)

1. A method performed by an electronic device, comprising:

interleaving a plurality of input image channels to produce an interleaved multi-channel input;

loading the interleaved multi-channel input to a single-instruction multiple data (SIMD) processor;

interleaving multiple filter channels to produce a multi-channel filter; and

convolving the interleaved multi-channel input with the multi-channel filter.

2. The method of claim 1 , further comprising performing object recognition based on the convolution.

3. The method of claim 1 , further comprising:

storing a maximum number of filter coefficients of a filter kernel in registers of the SIMD processor; and

applying the maximum number of filter coefficients in convolution operations without reloading any of the maximum number of filter coefficients from a cache.

4. The method of claim 1 , further comprising, for each input element, loading the element only once for multiple output elements.

5. The method of claim 1 , wherein convolving the interleaved multi-channel input with the multi-channel filter comprises multiplying an image input in parallel with multiple filter coefficients and accumulating sums in parallel.

6. The method of claim 1 , wherein the convolution is Deep Convolution Network (DCN) convolution.

7. The method of claim 1 , further comprising storing a kernel of the multi-channel filter in registers.

8. The method of claim 1 , further comprising storing the interleaved multi-channel input in at least one of a level 1 (L1) cache and a level 2 (L2) cache.

9. The method of claim 1 , further comprising storing an output in a level 1 (L1) cache.

10. An electronic device, comprising:

a single-instruction multiple data (SIMD) processor configured

to interleave a plurality of input image channels to produce an interleaved multi-channel input,

to load the interleaved multi-channel input to the SIMD processor,

to interleave multiple filter channels to produce a multi-channel filter, and

to convolve the interleaved multi-channel input with the multi-channel filter.

11. The electronic device of claim 10 , wherein the SIMD processor is configured to perform object recognition based on the convolution.

12. The electronic device of claim 10 , wherein the SIMD processor is configured to:

store a maximum number of filter coefficients of a filter kernel in registers of the SIMD processor; and

apply the maximum number of filter coefficients in convolution operations without reloading any of the maximum number of filter coefficients from a cache.

13. The electronic device of claim 10 , wherein the SIMD processor is configured to, for each input element, load the element only once for multiple output elements.

14. The electronic device of claim 10 , wherein the SIMD processor is configured to multiply an image input in parallel with multiple filter coefficients and to accumulate sums in parallel.

15. The electronic device of claim 10 , wherein the SIMD processor is configured to perform Deep Convolution Network (DCN) convolution.

16. The electronic device of claim 10 , wherein the SIMD processor is configured to store a kernel of the multi-channel filter in registers.

17. The electronic device of claim 10 , further comprising a memory, wherein the memory is configured to store the interleaved multi-channel input in at least one of a level 1 (L1) cache and a level 2 (L2) cache.

18. The electronic device of claim 10 , further comprising a memory, wherein the memory is configured to store an output in a level 1 (L1) cache.

19. An apparatus, comprising:

means for interleaving a plurality of input image channels to produce an interleaved multi-channel input;

means for loading the interleaved multi-channel input to a parallel processing means;

means for interleaving multiple filter channels to produce a multi-channel filter; and

means for convolving the interleaved multi-channel input with the multi-channel filter.

20. The apparatus of claim 19 , further comprising:

means for storing a maximum number of filter coefficients of a filter kernel in registers of the parallel processing means; and

means for applying the maximum number of filter coefficients in convolution operations without reloading any of the maximum number of filter coefficients from a cache.

21. The apparatus of claim 19 , further comprising means for, for each input element, loading the element only once for multiple output elements.

22. The apparatus of claim 19 , wherein the means for convolving comprises means for performing Deep Convolution Network (DCN) convolution.

23. A computer-program product, comprising a non-transitory computer-readable medium having instructions thereon, the instructions comprising:

code for causing an electronic device to interleave a plurality of input image channels to produce an interleaved multi-channel input;

code for causing the electronic device to load the interleaved multi-channel input to a single-instruction multiple data (SIMD) processor;

code for causing the electronic device to interleave multiple filter channels to produce a multi-channel filter; and

code for causing the electronic device to convolve the interleaved multi-channel input with a multi-channel filter.

24. The computer-program product of claim 23 , further comprising:

code for causing the electronic device to store a maximum number of filter coefficients of a filter kernel in registers of the SIMD processor; and

code for causing the electronic device to apply the maximum number of filter coefficients in convolution operations without reloading any of the maximum number of filter coefficients from a cache.

25. The computer-program product of claim 23 , further comprising code for causing the electronic device to, for each input element, load the element only once for multiple output elements.

26. The computer-program product of claim 23 , wherein the code for causing the electronic device to convolve comprises code for causing the electronic device to perform Deep Convolution Network (DCN) convolution.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2015
From: PAN, YUNKE; GAO, DASHAN; ZHONG, XIN; QUAN, SHUXUE
To: QUALCOMM INCORPORATED
Reel/Frame 035922/0287 →
Continuity (1)
Related Publication 20160379073A1 · Dec 29, 2016