IP Library › Granted Patent US 12,725,017
Granted Patent B2
US 12,725,017 · App. 16/923,447 · Granted Sep 1, 2026

Method and system for implementing a variable accuracy neural network

Inventors: Stefanos Laskaridis (Staines, GB); Hyeji Kim (Staines, GB); Stylianos Venieris (Staines, GB)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06N3/063G06N3/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,725,017
App. No.
16/923,447
Filed
Jul 8, 2020
Granted
Sep 1, 2026
Kind
B2
Art Unit
2122
USPC
706/19
Abstract

Disclosed is an electronic apparatus. The electronic apparatus includes a memory storing at least one instruction, and a processor coupled to the memory and configured to control the electronic apparatus, the processor configured to identify one of a plurality of exit points included in a neural network based on at least one constraint in at least one of processing or the electronic apparatus, process the input data via the neural network and obtain processing results output from the identified exit point as output data.

Claims (44)

1 . A method for controlling an electronic apparatus, the method comprising:

receiving input data;

identifying one of a plurality of exit points included in a neural network based on a time taken to reach each exit point to provide a processing result and based on a memory capacity of the electronic apparatus, the neural network providing progressive hierarchical inference by having at least a coarse-grained domain and a fine-grained domain, and coarse-grained results output from exit points in the coarse-grained domain are further processed in the fine-grained domain to obtain fine-grained results;

processing the input data via the neural network;

obtaining processing results output from the identified exit point as output data; and

using knowledge distillation, jointly training a plurality of classifiers corresponding to the plurality of exit points.

2 . The method as claimed in claim 1 , wherein the plurality of exit points are equidistantly spaced in the neural network.

3 . The method as claimed in claim 1 , wherein the identifying comprises identifying one of the plurality of exit points based on accuracy.

4 . The method as claimed in claim 1 , wherein:

the neural network includes the coarse-grained domain corresponding to a starting part of the neural network and the fine-grained domain corresponding to a deeper part of the neural network following the starting part;

classifiers in the coarse-grained domain generate coarser predictions then classifiers in the fine-grained domain; and

a plurality of exit points are positioned in the coarse-grained domain or the fine-grained domain.

5 . The method as claimed in claim 1 , wherein the obtaining comprises, based on a confidence of the processing result being greater than or equal to a predetermined confidence level, obtaining the processing result as the output data.

6 . The method as claimed in claim 5 , further comprising:

based on the confidence of the processing result being less than the predetermined confidence level, further processing the input data through a neural network after the identified exit point.

7 . The method as claimed in claim 1 , further comprising:

receiving a command to select a low-latency mode,

wherein the identifying comprises identifying one of the plurality of exit points based on time constraints corresponding to the low-latency mode.

8 . The method as claimed in claim 1 , further comprising:

receiving a command to select a confidence-based mode,

wherein the identifying comprises identifying one of the plurality of exit points based on the confidence level corresponding to the confidence-based mode.

9 . The method as claimed in claim 8 , further comprising:

receiving an additional processing command for the processing result;

additionally processing the input data through a neural network after the identified exit point; and

obtaining the additionally-processed data from the neural network as the output data.

10 . The method as claimed in claim 1 , further comprising:

calibrating the neural network based on at least one of a processing capacity of the electronic apparatus, the memory capacity of the electronic apparatus, or a power capacity of the electronic apparatus.

11 . The method as claimed in claim 10 , wherein the calibrating comprises reducing a number of classes used for processing the input data and outputting the processing result.

12 . The method as claimed in claim 10 wherein the calibrating comprises partitioning the neural network into a first portion configured to be executed by the electronic apparatus and a second portion configured to be executed by a remote server, and

wherein the obtaining comprises processing the input data through the first portion.

13 . The method as claimed in claim 12 , wherein based on a confidence of the processing result through the first portion being greater than or equal to the predetermined confidence level, the processing result through the first portion is obtained as the output data.

14 . The method as claimed in claim 13 , further comprising:

based on the confidence of the processing result through the first portion being less than the predetermined confidence level, transmitting the processing result through the first portion to the remote server; and

receiving, from the remote server, a result obtained by additionally processing, through the second portion, the processing result through the first portion.

15 . An electronic apparatus comprising:

a memory storing at least one instruction; and

a processor coupled to the memory and configured to control the electronic apparatus,

wherein the processor is configured to:

identify one of a plurality of exit points included in a neural network based on a time taken to reach each exit point to provide a processing result and based on a memory capacity of the electronic apparatus, the neural network providing progressive hierarchical inference by having at least a coarse-grained domain and a fine-grained domain, and coarse-grained results output from exit points in the coarse-grained domain are further processed in the fine-grained domain to obtain fine-grained results;

process the input data via the neural network;

obtain processing results output from the identified exit point as output data; and

jointly train, using knowledge distillation, a plurality of classifiers corresponding to the plurality of exit points.

16 . The electronic apparatus as claimed in claim 15 , wherein the processor, based on a confidence of the processing result being greater than or equal to a predetermined confidence level, is configured to obtain the processing result as the output data.

17 . The electronic apparatus as claimed in claim 16 , wherein the processor, based on the confidence of the processing result being less than the predetermined confidence level, is configured to further process the input data through a neural network after the identified exit point.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2020
From: LASKARIDIS, STEFANOS; KIM, HYEJI; VENIERIS, STYLIANOS
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 053150/0172 →
Priority Claims (3)
GR 20190100291 · Jul 11, 2019 · national
GB 2005029 · Apr 6, 2020 · national
KR 10-2020-0066486 · Jun 2, 2020 · national
Continuity (1)
Related Publication 20210012194A1 · Jan 14, 2021
References Cited (52)
US 20170169326A1 · Diamos et al. · 2017 [cited by applicant]
US 20180129947A1 · Cambot et al. · 2018 [cited by applicant]
US 20180157966A1 · Henry · 2018 [cited by applicant]
US 20190034761A1 · Huang et al. · 2019 [cited by applicant]
US 20190180168A1 · Barad · 2019 [cited by examiner]
CN 1112693A · 1995 [cited by applicant]
CN 108549926A · 2018 [cited by applicant]
JP 3743247 · 2006 [cited by applicant]
KR 1020180062004 · 2018 [cited by applicant]
KR 1020180138441 · 2018 [cited by applicant]
KR 1020190013162 · 2019 [cited by applicant]
WO 2018067962 · 2018 [cited by applicant]
Teerapittayanon, S., McDanel, B., & Kung, H. T. (Dec. 2016). Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR) (pp. 2464-2469). I… [cited by examiner]
Li, E., Zhou, Z., & Chen, X. (Aug. 2018). Edge intelligence: On-demand deep learning model co-inference with device-edge synergy. In Proceedings of the 2018 Workshop on Mobile Edge Communications (pp. 31-36). (Year: 201… [cited by examiner]
Tsai, C. H., Chih, Y. T., Wong, W. H., & Lee, C. Y. (2015). A hardware-efficient sigmoid function with adjustable precision for a neural network system. IEEE Transactions on Circuits and Systems II: Express Briefs, 62(1… [cited by examiner]
Teerapittayanon, S., McDanel, B., & Kung, H. T. (Jun. 2017). Distributed deep neural networks over the cloud, the edge and end devices. In 2017 IEEE 37th international conference on distributed computing systems (ICDCS)… [cited by examiner]
Peluso, Valentino, and Andrea Calimera. “Scalable-effort convnets for multilevel classification.” In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1-8. IEEE, 2018. (Year: 2018). [cited by examiner]
Yan, Z. et al., “HD-CNN: hierarchical deep convolutional neural network for large scale visual recognition,” downloaded from <arxiv.org/abs/1410.0736> (May 16, 2015) pp. 4321-4329. (Year: 2015). [cited by examiner]
S. Teerapittayanon et al. “BranchyNet: Fast inference via early exiting from deep neural networks,” 23rd International Conference on Pattern Recognition (ICPR), 2016, 6 pages. [cited by applicant]
Gao Huang et al. “Multi-Scale Dense Networks for Resource Efficient Image Classification,” International Conference on Learning Representations (ICLR), 2018, 14 pages. [cited by applicant]
Y. Kaya et al. “How to Stop Off-the-Shelf Deep Neural Networks from Overthinking,” International Conference on Machine Learning (ICML), Oct. 16, 2018, 8 pages. [cited by applicant]
Jia Deng et al. “What Does Classifying More Than 10,000 Image Categories Tell Us?” ECCV, 2010, pp. 71-84. [cited by applicant]
Jia Deng et al. “Hedging Your Bets: Optimizing Accuracy-Specificity Trade-offs in Large Scale Visual Recognition,” CVPR, 2012, pp. 1-8. [cited by applicant]
A. Bilal et al. “Do Convolutional Neural Networks Learn Class Hierarchy?” IEEE Transactions on Visualization and Computer Graphics, 24(1), Oct. 17, 2017, pp. 152-162. [cited by applicant]
Mohammed J. Saberian et al. “HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition”, ICCV, 2015, 9 pages. [cited by applicant]
Paul Viola et al. “Robust Real-Time Face Detection”, IJCV, 57(2), 2004, pp. 137-154. [cited by applicant]
Nuno Vasconcelos et al. “Boosting Classifier Cascades”, NIPS, 2010, 9 pages. [cited by applicant]
Zhixiang (Eddie) Xu et al. “Classifier Cascades and Trees for Minimizing Feature Evaluation Cost”, Journal of Machine Learning Research (JMLR), Jun. 2014, pp. 2113-2144. [cited by applicant]
H. Li et al. “A Convolutional Neural Network Cascade for Face Detection”, CVPR, 2015, 10 pgs. [cited by applicant]
Anelia Angelova et al. “Real-Time Pedestrian Detection With Deep Network Cascades”, BMVC, 2015, pp. 1-12. [cited by applicant]
Ali Diba et al. “Weakly Supervised Cascaded Convolutional Networks”, CVPR, 2017, 9 pgs. [cited by applicant]
Alexandros Kouris et al. “CascadeCNN: Pushing the Performance Limits of Quantisation in Convolutional Neural Networks”, 2018 28 [cited by applicant]
Christian Szegedy et al. “Going Deeper with Convolutions”, CVPR, 2015, 9 pages. [cited by applicant]
Gao Huang et al. “Densely Connected Convolutional Networks”, CVPR, 2017, 9 pages. [cited by applicant]
Valentino Peluso et al. “Scalable-Effort ConvNets for Multilevel Classification” ICCAD, 2018, 8 pages. [cited by applicant]
Xin Wang et al. “SkipNet: Learning Dynamic Routing in Convolutional Networks” ECCV, 2018, 16 pages. [cited by applicant]
Zuxuan Wu et al. “BlockDrop: Dynamic Inference Paths in Residual Networks”, CVPR, 2018, 10 pages. [cited by applicant]
Michael Figurnov et al. “Spatially Adaptive Computation Time for Residual Networks”, CVPR, 2017, 10 pages. [cited by applicant]
Alex Graves et al. “Adaptive Computation Time for Recurrent Neural Networks”, NIPS Deep Learning Symposium, Apr. 18, 2016, 19 pages. [cited by applicant]
Chuan Guo et al. “On Calibration of Modern Neural Networks”, ICML, 2017, 10 pages. [cited by applicant]
Jia Deng, et al “ImageNet: A Large-Scale Hierarchical Image Database”, 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248-255. [cited by applicant]
Search Report and Written Opinion issued Oct. 7, 2020 in counterpart International Patent Application No. PCT/KR2020/008980. [cited by applicant]
Li, En et al., “Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy,” arXiv:1806.0784v4, Dec. 27, 2018, pp. 1-10[retrieved on Sep. 25, 2020]. [cited by applicant]
Cambazoglu, B. Barla, et al., “Early Exit Optimizations for Additive Machine Learned Ranking Systems,” Proceedings of the third ACM international conference on Web search and data mining (WSCM), Feb. 6, 2010 pp. 411-420. [cited by applicant]
Extended Search Report and Written Opinion dated Apr. 25, 2022 issued in counterpart European Patent Application No. 20836673.2. [cited by applicant]
Search and Examination Report issued Dec. 17, 2020 in corresponding United Kingdom Patent Application No. 2005029.0. [cited by applicant]
Teerapittayanon Surat et al, “Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices”, Proceedings of the International Conference On Distributed Computing Systems, 2017, IEEE, pp. 328-339. [cited by applicant]
Chinese Office Action dated Oct. 30, 2024 for CN Application No. 202080051012.0. [cited by applicant]
Office Action for CN Application No. 202080051012.0 dated Jun. 26, 2025 and English translation, 17 pages. [cited by applicant]
Office Action for EP Application No. 20836673.2 dated Jul. 1, 2025, 12 pages. [cited by applicant]
Amir R. Zamir et al, “Feedback Networks” 2017 IEEE Conference on Computer Vision and Pattern Recognition, Jul. 21, 2017, pp. 1808-1817. [cited by applicant]
Chinese Office Action dated Mar. 12, 2025 for CN Application No. 202080051012.0. [cited by applicant]