IP Library Granted Patent US 12,639,596
Granted Patent B2
US 12,639,596 · App. 17/356,145 · Granted May 26, 2026

Multi-instance learning framework for artificial intelligence (AI) household inference models

Inventors: Tomasz Palczewski (Danville, CA); Lenin Mookiah (Houston, TX); Yingnan Zhu (Irvine, CA); Hari Nayar (Hayward, CA); Praveen Pratury (Mountain House, CA)
Assignee: Samsung Electronics Co., Ltd.
G06N5/04G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,639,596
App. No.
17/356,145
Granted
May 26, 2026
Kind
B2
Abstract

A method includes obtaining, using at least one processor of an electronic device, one or more instance level supervised artificial intelligence (AI) models. The method also includes obtaining, using the at least one processor, aggregated level label information related to the one or more instance level supervised AI models. The method further includes obtaining, using the at least one processor, instance level feature information related to the one or more instance level supervised AI models. In addition, the method includes training, using the at least one processor, the one or more instance level supervised AI models using the instance level feature information and the aggregated level label information to obtain one or more trained instance level supervised AI models.

Claims (71)

1 . A method comprising:

obtaining, using at least one processor of an electronic device, one or more instance level supervised artificial intelligence (AI) models;

obtaining, using the at least one processor, aggregated level label information related to the one or more instance level supervised AI models;

obtaining, using the at least one processor, instance level feature information related to the one or more instance level supervised AI models; and

training, using the at least one processor, the one or more instance level supervised AI models using the instance level feature information and the aggregated level label information to obtain one or more trained instance level supervised AI models;

wherein the training includes performing transfer learning using the aggregated level label information; and

wherein training the one or more instance level supervised AI models comprises:

generating, by the one or more instance level supervised AI models, instance level predictions using the instance level feature information;

grouping, by an aggregation layer of the one or more instance level supervised AI models, the instance level predictions into bags;

providing, to each bag, a histogram of aggregated level information as an aggregated level prediction;

aggregating, by the aggregation layer, the instance level predictions from the bags into aggregated level predictions comprising histogram information;

adding one or more constraints on the aggregated level predictions comprising the histogram information;

extracting the aggregated level label information corresponding to the aggregated level predictions from the bags; and

using the aggregated level predictions and the aggregated level label information to modify the one or more instance level supervised AI models.

2 . The method of claim 1 , wherein training the one or more instance level supervised AI models further comprises:

iteratively comparing the aggregated level predictions with labels from the aggregated level label information; and

adjusting weights of the one or more instance level supervised AI models based on the comparison.

3 . The method of claim 2 , wherein, in each of a plurality of training iterations, the histogram information and the aggregated level predictions are adjusted by altering one or more weights in a loss function pertaining to the histogram information and the aggregated level predictions.

4 . The method of claim 2 , wherein training the one or more instance level supervised AI models further comprises minimizing a loss function that includes one or more aggregated level parameters.

5 . The method of claim 4 , wherein:

the loss function further includes instance level parameters; and

minimizing the loss function comprises minimizing a sum of the aggregated level parameters and the instance level parameters.

6 . The method of claim 1 , wherein the transfer learning provides for modifications of a loss function and the adding of the one or more constraints to improve precision of the one or more instance level supervised AI models.

7 . The method of claim 1 , wherein the aggregated level label information includes labels associated with aggregated groups of data and does not include labels associated with single instance level data.

8 . The method of claim 1 , wherein the one or more constraints impose one or more restrictions on at least one of the bags.

9 . The method of claim 1 , wherein the one or more instance level supervised AI models are associated with household demographic information.

10 . An electronic device comprising:

at least one memory configured to store instructions; and

at least one processing devices configured when executing the instructions to:

obtain one or more instance level supervised artificial intelligence (AI) models;

obtain aggregated level label information related to the one or more instance level supervised AI models;

obtain instance level feature information related to the one or more instance level supervised AI models; and

train the one or more instance level supervised AI models using the instance level feature information and the aggregated level label information to obtain one or more trained instance level supervised AI models;

wherein, to train the one or more instance level supervised AI models, the at least one processing device is configured to perform transfer learning using the aggregated level label information; and

wherein, to train the one or more instance level supervised AI models, the at least one processing device is configured to:

generate, by the one or more instance level supervised AI models, instance level predictions using the instance level feature information;

group, by an aggregation layer of the one or more instance level supervised AI models, the instance level predictions into bags;

provide, to each bag, a histogram of aggregated level information as an aggregated level prediction;

aggregate, by the aggregation layer, the instance level predictions from the bags into aggregated level predictions comprising histogram information;

add one or more constraints on the aggregated level predictions comprising the histogram information;

extract the aggregated level label information corresponding to the aggregated level predictions from the bags; and

use the aggregated level predictions and the aggregated level label information to modify the one or more instance level supervised AI models.

11 . The electronic device of claim 10 , wherein, to train the one or more instance level supervised AI models, the at least one processing device is further configured to:

iteratively compare the aggregated level predictions with labels from the aggregated level label information; and

adjust weights of the one or more instance level supervised AI models based on the comparison.

12 . The electronic device of claim 11 , wherein the at least one processing devices is configured when executing the instructions, in each of a plurality of training iterations, to adjust the histogram information and the aggregated level predictions by altering one or more weights in a loss function pertaining to the histogram information and the aggregated level predictions.

13 . The electronic device of claim 11 , wherein, to train the one or more instance level supervised AI models, the at least one processing device is configured to minimize a loss function that includes one or more aggregated level parameters.

14 . The electronic device of claim 13 , wherein:

the loss function further includes instance level parameters; and

to minimize the loss function, the at least one processing device is configured to minimize a sum of the aggregated level parameters and the instance level parameters.

15 . The electronic device of claim 10 , wherein the transfer learning provides for modifications of a loss function and the addition of the one or more constraints to improve precision of the one or more instance level supervised AI models.

16 . The electronic device of claim 10 , wherein the aggregated level label information includes labels associated with aggregated groups of data and does not include labels associated with single instance level data.

17 . The electronic device of claim 10 , wherein: the one or more constraints impose one or more restrictions on at least one of the bags.

18 . A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of an electronic device to:

obtain one or more instance level supervised artificial intelligence (AI) models;

obtain aggregated level label information related to the one or more instance level supervised AI models;

obtain instance level feature information related to the one or more instance level supervised AI models; and

train the one or more instance level supervised AI models using the instance level feature information and the aggregated level label information to obtain one or more trained instance level supervised AI models;

wherein the instructions that when executed cause the at least one processor to train the one or more instance level supervised AI models comprise instructions that when executed cause the at least one processor to perform transfer learning using the aggregated level label information; and

wherein the instructions that when executed cause the at least one processor to train the one or more instance level supervised AI models comprise instructions that when executed cause the at least one processor to:

generate, by the one or more instance level supervised AI models, instance level predictions using the instance level feature information;

group, by an aggregation layer of the one or more instance level supervised AI models, the instance level predictions into bags;

provide, to each bag, a histogram of aggregated level information as an aggregated level prediction;

aggregate, by the aggregation layer, the instance level predictions from the bags into aggregated level predictions comprising histogram information;

add one or more constraints on the aggregated level predictions comprising the histogram information;

extract the aggregated level label information corresponding to the aggregated level predictions from the bags; and

use the aggregated level predictions and the aggregated level label information to modify the one or more instance level supervised AI models.

19 . The non-transitory machine-readable medium of claim 18 , wherein the instructions that when executed cause the at least one processor to train the one or more instance level supervised AI models further comprise instructions that when executed cause the at least one processor to:

iteratively compare the aggregated level predictions with labels from the aggregated level label information; and

adjust weights of the one or more instance level supervised AI models based on the comparison.

20 . The non-transitory machine-readable medium of claim 19 , wherein the instructions when executed cause the at least one processor, in each of a plurality of training iterations, to adjust the histogram information and the aggregated level predictions by altering one or more weights in a loss function pertaining to the histogram information and the aggregated level predictions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2021
From: PALCZEWSKI, TOMASZ; MOOKIAH, LENIN; ZHU, YINGNAN; NAYAR, HARI; PRATURY, PRAVEEN
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 056642/0791 →
Continuity (1)
Related Publication 20220414494A1 · Dec 29, 2022
References Cited (12)
US 9317781B2 · Tu et al. · 2016 [cited by applicant]
US 9495361B2 · Allen et al. · 2016 [cited by applicant]
US 9886094B2 · Wang et al. · 2018 [cited by applicant]
US 10719780B2 · Lee et al. · 2020 [cited by applicant]
US 10785533B2 · Pecjak et al. · 2020 [cited by applicant]
US 20150356455A1 · Wu et al. · 2015 [cited by applicant]
US 20170063893A1 · Franc · 2017 [cited by examiner]
US 20220237788A1 · Shaul · 2022 [cited by examiner]
M. Fang, J. Yin and X. Zhu, “Transfer Learning across Networks for Collective Classification,” 2013 IEEE 13th International Conference on Data Mining, Dallas, TX, USA, 2013, pp. 161-170, doi: 10.1109/ICDM.2013.116. keyw… [cited by examiner]
Kotzias, Dimitrios, et al. “From group to individual labels using deep features.” Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 2015. (Year: 2015). [cited by examiner]
Zhao, Yu, et al. “Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution.” Proceedings of the IEEE/CVF conference on computer vision and pattern r… [cited by examiner]
Carbonneau et al., “Multiple Instance Learning: A Survey of Problem Characteristics and Applications”, https://arxiv.org/pdf/1612.03365.pdf, 2016, 37 pgs. [cited by applicant]