IP Library Granted Patent US 12,334,223
Granted Patent B2
US 12,334,223 · App. 18/245,965 · Granted Jun 17, 2025

Learning apparatus, mental state sequence prediction apparatus, learning method, mental state sequence prediction method and program

Inventors: Shuhei Yamamoto (Tokyo, JP); Hiroyuki Toda (Tokyo, JP); Takeshi Kurashima (Tokyo, JP); Tomu Tominaga (Tokyo, JP)
Assignee: NIPPON TELEGRAPH AND TELEPHONE CORPORATION
G16H50/20G16H20/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,334,223
App. No.
18/245,965
Granted
Jun 17, 2025
Kind
B2
Abstract

A learning device includes: a psychological state data preprocessing unit that calculates a duration of a psychological state from psychological state sequence data, and generates preprocessed psychological state sequence data including the psychological state and the duration; and a learning unit that learns a psychological state sequence prediction model, using input sequence data including behavior sequence data and the preprocessed psychological state sequence data, and correct sequence data that is preprocessed psychological state sequence data at a time later than the input sequence data.

Claims (44)

1. A learning apparatus comprising:

a memory; and

a processor configured to execute:

scanning each of behavior data columns that are extracted from a behavior sequence database,

replacing a missing value or an unexpected value with another value in a case where there is the missing value or the unexpected value, thereby generating past behavior sequence data;

calculating a duration of a mental state from mental state sequence data, and generating preprocessed mental state sequence data including the mental state and the duration; and

learning a mental state sequence prediction model, using input sequence data including the behavior sequence data and the preprocessed mental state sequence data, and correct sequence data that is preprocessed mental state sequence data at a time later than the input sequence data,

wherein the mental state is defined by mood data including valence and arousal, together with information about dates and times when a user makes self-reports, said valence indicating degree of positiveness and negativity of emotion of the user at that time, and said arousal indicating degree of excitement in the emotion of the user, and

wherein the processor is further configured to execute:

calculating the duration of the mental state by calculating a difference of an entry of the mood data and next entry of the mood data.

2. The learning apparatus according to claim 1 , wherein the processor is further configured to execute:

performing soft clustering on the preprocessed mental state sequence data obtained at the generating, and outputting a parameter set related to a mental state and a duration of each component,

wherein the learning includes learning the mental state sequence prediction model from a probability of belonging to each component and the parameter set, the probability of belonging being obtained by the mental state sequence prediction model from the input sequence data.

3. The learning apparatus according to claim 1 , wherein the processor is further configured to execute:

replacing the missing value or the unexpected value with the another value by inserting the mean value of a corresponding column in the behavior sequence database or “0” in the case of numerical data, and inserting a character string indicating that a value is missing in the case of character string data.

4. A mental state sequence prediction apparatus comprising:

a memory; and

a processor configured to execute:

scanning each of behavior data columns that are extracted from a behavior sequence database,

replacing a missing value or an unexpected value with another value in a case where there is the missing value or the unexpected value, thereby generating past behavior sequence data;

calculating a duration of a mental state from mental state sequence data, and generating preprocessed mental state sequence data including the mental state and the duration; and

predicting a future mental state and a duration of the future mental state, by inputting the past behavior sequence data and past preprocessed mental state sequence data obtained at the generating, to a mental state sequence prediction model learned with use of input sequence data including behavior sequence data and the preprocessed mental state sequence data, and correct sequence data that is preprocessed mental state sequence data at a time later than the input sequence data,

wherein the mental state is defined by mood data including valence and arousal, together with information about dates and times when a user makes self-reports, said valence indicating degree of positiveness and negativity of emotion of the user at that time, and said arousal indicating degree of excitement in the emotion of the user, and

wherein the processor is further configured to execute:

calculating the duration of the mental state by calculating a difference of an entry of the mood data and the next entry of the mood data.

5. The mental state sequence prediction apparatus according to claim 4 , wherein

the predicting includes predicting a future mental state and a duration of the future mental state, on a basis of a parameter set and a probability of belonging to a component, the parameter set being related to a mental state and a duration of each component, the parameter set being obtained by soft clustering performed on the preprocessed mental state sequence data during learning, the probability of belonging being obtained by the mental state sequence prediction model.

6. A learning method comprising:

scanning each of behavior data columns that are extracted from a behavior sequence database,

replacing a missing value or an unexpected value with another value in a case where there is the missing value or the unexpected value, thereby generating behavior sequence data;

calculating a duration of a mental state from mental state sequence data, and generating preprocessed mental state sequence data including the mental state and the duration; and

learning a mental state sequence prediction model, using input sequence data including the behavior sequence data and the preprocessed mental state sequence data, and correct sequence data that is preprocessed mental state sequence data at a time later than the input sequence data,

wherein the mental state is defined by mood data including valence and arousal, together with information about dates and times when a user makes self-reports, said valence indicating degree of positiveness and negativity of emotion of the user at that time, and said arousal indicating degree of excitement in the emotion of the user, and

wherein the processor is further configured to execute:

calculating the duration of the mental state by calculating a difference of an entry of the mood data and the next entry of the mood data.

7. A mental state sequence prediction method comprising:

scanning each of behavior data columns that are extracted from a behavior sequence database,

replacing a missing value or an unexpected value with another value in a case where there is the missing value or the unexpected value, thereby generating past behavior sequence data;

calculating a duration of a mental state from mental state sequence data, and generating preprocessed mental state sequence data including the mental state and the duration; and

predicting a future mental state and a duration of the future mental state, by inputting past behavior sequence data and past preprocessed mental state sequence data obtained at the generating, to a mental state sequence prediction model learned with use of input sequence data including behavior sequence data and the preprocessed mental state sequence data, and correct sequence data that is preprocessed mental state sequence data at a time later than the input sequence data,

wherein the mental state is defined by mood data including valence and arousal, together with information about dates and times when a user makes self-reports, said valence indicating degree of positiveness and negativity of emotion of the user at that time, and said arousal indicating degree of excitement in the emotion of the user, and

wherein the processor is further configured to execute:

calculating the duration of the mental state by calculating a difference of an entry of the mood data and the next entry of the mood data.

8. A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which when executed, cause a computer to function as the learning apparatus according to claim 1 .

Assignments (2)
CHANGE OF NAME Recorded Aug 15, 2025
From: NIPPON TELEGRAPH AND TELEPHONE CORPORATION
To: NTT, INC.
Reel/Frame 072473/0885 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2023
From: YAMAMOTO, SHUHEI; TODA, HIROYUKI; KURASHIMA, TAKESHI; TOMINAGA, TOMU
To: NIPPON TELEGRAPH AND TELEPHONE CORPORATION
Reel/Frame 063035/0719 →
Continuity (1)
Related Publication 20230368920A1 · Nov 16, 2023
References Cited (15)
US 7917302B2 · Rognes · 2011 [cited by examiner]
US 11302448B1 · Jain · 2022 [cited by examiner]
US 20140156698A1 · Stivoric · 2014 [cited by examiner]
US 20170258390A1 · Howard · 2017 [cited by examiner]
US 20220000405A1 · Tsuji · 2022 [cited by examiner]
CN 101460625A · 2009 [cited by examiner]
CN 111419249B · 2023 [cited by examiner]
Decoding Spontaneous Emotional States in the Human Brain, Philip A. Kragel,Annchen R. Knodt,Ahmad R. Hariri,Kevin S. LaBar, Published: Sep. 14, 2016, PLOSBiology (Year: 2016). [cited by examiner]
Using Machine Learning and Smartphone and Smartwatch Data to Detect Emotional States and Transitions: Exploratory Study, Madeena Sultana1,2*, PhD; Majed Al-Jefri1,3*, PhD; Joon Lee1,2,3, PhD, JMIR MHealth and Unhealth, … [cited by examiner]
A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors, Agata Kolakowska * , Wioleta Szwoch and Mariusz Szwoch, Sensors 2020 (Year: 2020). [cited by examiner]
DeepMood: Forecasting Depressed Mood Based on Self-Reported Histories via Recurrent Neural Networks, 2017 International World Wide Web Conference Committee (IW3C2), published under Creative Commons CC BY 4.0 License. WW… [cited by examiner]
E. Nosakhare and R. Picard: Probabilistic Latent Variable Modeling for Assessing Behavioral Influences on Well-Being. In Proc. of KDD, 2019. [cited by applicant]
S. Yan et al.: Estimating Individualized Daily Self-Reported Affect with Wearable Sensors. In Proc. of ICHI, 2019. [cited by applicant]
D. Spathis, S. Servia-Rodriguez, K. Farrahi, C. Mascolo, and J. Rentflow: Sequence Multi-task Learning to Forecast Mental Wellbeing from Sparse Self-reported Data. In Proc. of KDD, 2019. [cited by applicant]
J.P. Pollak, P. Adams, G. Gay: PAM: A Photographic Affect Meter for Frequent, In Situ Measurement of Affect. In Proc. of CHI, 2011. [cited by applicant]