IP Library › Granted Patent US 12,282,587
Granted Patent B2
US 12,282,587 · App. 17/894,586 · Granted Apr 22, 2025

Electronic device and control method thereof

Inventors: Jun Ho Huh (Suwon-si, KR); Donguk Kim (Suwon-si, KR); Choonghoon Lee (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06F21/74G06F18/251G06F21/62G06N3/063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,282,587
App. No.
17/894,586
Granted
Apr 22, 2025
Kind
B2
Abstract

An electronic device includes a memory and a processor. The memory stores information related to an artificial intelligence model that performs arithmetic operations through neural network layers. The processor pre-processes raw data and obtains output data by inputting the pre-processed raw data to the artificial intelligence model. First data used for pre-processing the raw data and second data used in a portion of the neural network layers are stored in a secure area of the memory, and third data used in remaining layers of the neural network layers other than the portion is stored in a non-secure area of the memory.

Claims (23)

1. An electronic device comprising:

a memory configured to store information related to an artificial intelligence model that performs arithmetic operations through a plurality of neural network layers; and

a processor configured to pre-process raw data using first data and obtain output data by inputting the pre-processed raw data to the artificial intelligence model in which second data is used in a portion of the plurality of neural network layers of the artificial intelligence model and third data is used in remaining layers of the plurality of neural network layers other than the portion of the plurality of neural network layers,

wherein the first data used for pre-processing the raw data and the second data used in the portion of the plurality of neural network layers are stored in a secure area of the memory, and

the third data used in the remaining layers of the plurality of neural network layers other than the portion of the plurality of neural network layers is stored in a non-secure area of the memory,

wherein the remaining layers comprise a last layer of the plurality of neural network layers and the second data includes information regarding parameters used in the last layer,

wherein the first data includes at least one of data used for filtering of raw data to be input to the artificial intelligence model among a plurality of raw data, or data used for conversion of raw data into input data to be input to the artificial intelligence model, and

wherein the processor operates as a secure world, reads the raw data stored in the non-secure area, and pre-processes the raw data based on the first data, and operates as a normal world, performs an arithmetic operation on the pre-processed data based on the third data, and obtains feature map information output from a previous layer of the last layer.

2. The electronic device as claimed in claim 1 , wherein the processor operates as the secure world, performs an arithmetic operation on the feature map information based on the second data, and obtains, as the output data, data output from the last layer.

3. The electronic device as claimed in claim 1 , wherein the remaining layers comprise at least one layer that is determined based on a size of parameter information used in each of the plurality of neural network layers, and the second data includes a parameter used in the at least one layer.

4. The electronic device as claimed in claim 1 , wherein the memory includes:

a first memory used as the secure area; and

a second memory used as the non-secure area.

5. The electronic device as claimed in claim 1 , further comprising a sensor,

wherein based on at least one raw data being obtained through the sensor, the processor filters the obtained raw data based on the first data and converts the filtered raw data into input data to be input to the artificial intelligence model.

6. A control method of an electronic device, the control method comprising:

pre-processing raw data based on first data stored in a secure area of a memory included in the electronic device; and

obtaining output data by applying the pre-processed raw data to an artificial intelligence model based on second data used in a portion of a plurality of neural network layers of the artificial intelligence model and stored in the secure area, and third data used in remaining layers of the plurality of neural network layers other than the portion of the plurality of neural network layers and stored in a non-secure area of the memory,

wherein the remaining layers comprise a last layer of the plurality of neural network layers and the second data includes information regarding parameters used in the last layer,

wherein the first data includes at least one of data used for filtering of raw data to be input to the artificial intelligence model among a plurality of raw data, or data used for conversion of raw data into input data to be input to the artificial intelligence model,

wherein in the pre-processing, the electronic device operates as a secure world, reads the raw data stored in the non-secure area, and pre-processes the raw data based on the first data, and

in the obtaining, the electronic device operates as a normal world, performs an arithmetic operation on the preprocessed data based on the third data, and obtains feature map information output from a previous layer of the last layer.

7. The control method as claimed in claim 6 , wherein in the obtaining, the electronic device operates as the normal world, performs an arithmetic operation on the feature map information based on the second data, and obtains, as the output data, data output from the last layer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2022
From: HUH, JUN HO; KIM, DONGUK; LEE, CHOONGHOON
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 060889/0141 →
Priority Claims (1)
KR 10-2020-0022062 · Feb 24, 2020 · national
Continuity (2)
Continuation PCTKR2021000465 · Jan 13, 2021
Related Publication 20220405430A1 · Dec 22, 2022
References Cited (32)
US 10002402B2 · Liu et al. · 2018 [cited by applicant]
US 11244222B2 · Kuo · 2022 [cited by applicant]
US 20170289409A1 · Min · 2017 [cited by examiner]
US 20180144465A1 · Hsieh et al. · 2018 [cited by applicant]
US 20180158177A1 · Lannes et al. · 2018 [cited by applicant]
US 20180299841A1 · Appu · 2018 [cited by examiner]
US 20180300556A1 · Varerkar · 2018 [cited by examiner]
US 20190042878A1 · Sheller et al. · 2019 [cited by applicant]
US 20190042937A1 · Sheller et al. · 2019 [cited by applicant]
US 20190045207A1 · Chen · 2019 [cited by examiner]
US 20190087689A1 · Chen · 2019 [cited by applicant]
US 20190108618A1 · Hwang · 2019 [cited by examiner]
US 20190130250A1 · Park et al. · 2019 [cited by applicant]
US 20190197396A1 · Rajkumar · 2019 [cited by examiner]
US 20190392194A1 · Croxford · 2019 [cited by examiner]
US 20190392305A1 · Gu · 2019 [cited by examiner]
US 20200379454A1 · Trinh · 2020 [cited by examiner]
US 20210133361A1 · Fishel · 2021 [cited by examiner]
US 20210232933A1 · Gu · 2021 [cited by examiner]
US 20220114014A1 · Chen · 2022 [cited by examiner]
CN 110148935A · 2019 [cited by applicant]
CN 109918951B · 2020 [cited by applicant]
JP 201885135A · 2018 [cited by applicant]
JP 6954084B2 · 2021 [cited by applicant]
JP 7026298B2 · 2022 [cited by applicant]
KR 101972231B1 · 2019 [cited by applicant]
KR 1020200000361A · 2020 [cited by applicant]
WO 2018008605A1 · 2018 [cited by applicant]
Gu et al., “Securing Input Data of Deep Learning Inference Systems via Partitioned Enclave Execution”, ArXiv, Jul. 3, 2018, 14 total pages, arXiv:1807.00969v1. [cited by applicant]
Communication dated Jun. 29, 2023, issued by European Patent Office for European Patent Application No. 21760057.6. [cited by applicant]
Communication dated Apr. 22, 2021, issued by the International Searching Authority in counterpart International Application No. PCT/KR2021/000465 (PCT/ISA/210 and PCT/ISA/237). [cited by applicant]
Communication dated Feb. 3, 2025, issued by European Patent Office in European Patent Application No. 21760057.6. [cited by applicant]