IP Library › Granted Patent US 12,314,673
Granted Patent B2
US 12,314,673 · App. 18/505,708 · Granted May 27, 2025

Generative language model for few-shot aspect-based sentiment analysis

Inventors: Ehsan Hosseini-Asl (Palo Alto, CA); Wenhao Liu (Redwood City, CA)
Assignee: Salesforce, Inc.
G06F40/30G06F40/284G06N3/04G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,314,673
App. No.
18/505,708
Granted
May 27, 2025
Kind
B2
Abstract

Sentiment analysis is a task in natural language processing. The embodiments are directed to using a generative language model to extract an aspect term, aspect category and their corresponding polarities. The generative language model may be trained as a single, joint, and multi-task model. The single-task generative language model determines a term polarity from the aspect term in the sentence or a category polarity from an aspect category in the sentence. The joint-task generative language model determines both the aspect term and the term polarity or the aspect category and the category polarity. The multi-task generative language model determines the aspect term, term polarity, aspect category and category polarity of the sentence.

Claims (42)

1. A system for generating a sentiment analysis, the system comprising:

a memory configured to store an aspect-based sentiment analysis (ABSA) generative language model; and

a processor coupled to the memory and configured to execute instructions that cause the ABSA generative model to perform operations, the operations comprising:

receiving a sentence expressing a sentiment of a user; and

generating at least one pair including an aspect term in the sentence and a term polarity associated with the aspect term.

2. The system of claim 1 , wherein the ABSA generative language model is a generative pre-trained transformer (GPT) model.

3. The system of claim 1 , wherein the aspect term is at least one word or a span of text in the sentence that is associated with the sentiment of the aspect term.

4. The system of claim 1 , further comprising:

training the ABSA generative language model to determine the aspect term using an aspect term extraction task.

5. The system of claim 1 , further comprising:

training the ABSA generative language model to determine the term polarity of the aspect term using an aspect term polarity task.

6. The system of claim 5 , wherein a training dataset includes the sentence appended with the aspect term.

7. The system of claim 1 , further comprising:

receiving an aspect category from a predefined set of categories; and

generating, using the ABSA generative language model, the sentence, and the aspect category, a category polarity associated with the aspect category, wherein the category polarity corresponds to the sentiment.

8. The system of claim 7 , wherein the aspect category is conditioned on other terms in the sentence.

9. A system for generating a sentiment analysis, the system comprising:

a memory configured to store an aspect-based sentiment analysis (ABSA) generative language model; and

a processor coupled to the memory and configured to execute instructions that cause the ABSA generative model to perform operations, the operations comprising:

receiving a sentence expressing a sentiment of a user and an aspect category from a predefined set of categories; and

generating, using the ABSA generative language model, the sentence, and the aspect category, a category polarity associated with the aspect category, wherein the category polarity corresponds to the sentiment.

10. The system of claim 9 , further comprising:

training the ABSA generative language model to determine the category polarity associated with the aspect category using an aspect category polarity task.

11. The system of claim 10 , wherein a training dataset includes the sentence appended with the aspect category and the category polarity.

12. The system of claim 9 , further comprising:

receiving, together with the aspect category an aspect term; and

generating, using the ABSA generative language model, the sentence, and the aspect term, a term polarity associated with the aspect term, wherein the term polarity corresponds to the sentiment of the aspect term.

13. The system of claim 12 , further comprising:

training the ABSA generative language model to determine the term polarity associated with the aspect term using an aspect term polarity task.

14. A non-transitory computer readable medium having instructions stored thereon, that when executed by a processor, cause the processor to perform operations, the operations comprising:

receiving, at an aspect-based sentiment analysis (ABSA) generative language model, a sentence expressing a sentiment of a user; and

generating, at the ABSA generative language model, at least one pair including an aspect term in the sentence and a term polarity associated with the aspect term.

15. The non-transitory computer readable medium of claim 14 , wherein the ABSA generative language model is a generative pre-trained transformer (GPT) model.

16. The non-transitory computer readable medium of claim 14 , wherein the aspect term is at least one word or a span of text in the sentence that is associated with the sentiment of the aspect term.

17. The non-transitory computer readable medium of claim 14 , further comprising:

training the ABSA generative language model to determine the aspect term using an aspect term extraction task.

18. The non-transitory computer readable medium of claim 14 , further comprising:

training the ABSA generative language model to determine the term polarity of the aspect term using an aspect term polarity task.

19. The non-transitory computer readable medium of claim 18 , wherein a training dataset includes the sentence appended with the aspect term and the term polarity.

20. The non-transitory computer readable medium of claim 14 , further comprising:

receiving an aspect category from a predefined set of categories; and

generating, using the ABSA generative language model, the sentence, and the aspect category, a category polarity associated with the aspect category, wherein the category polarity corresponds to the sentiment.

Assignments (2)
CHANGE OF NAME Recorded Mar 26, 2025
From: SALESFORCE.COM, INC.
To: SALESFORCE, INC.
Reel/Frame 070633/0053 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2023
From: HOSSEINI-ASL, EHSAN; LIU, WENHAO
To: SALESFORCE.COM, INC.
Reel/Frame 065512/0573 →
Continuity (3)
Continuation 17468950 · Sep 8, 2021
Provisional Application 63189647 · May 17, 2021
Related Publication 20240078389A1 · Mar 7, 2024
References Cited (37)
US 10726207B2 · He · 2020 [cited by examiner]
US 10755174B2 · He · 2020 [cited by examiner]
US 20180032508A1 · Matskevich · 2018 [cited by examiner]
US 20180053107A1 · Wang · 2018 [cited by examiner]
US 20200073937A1 · Zhao · 2020 [cited by examiner]
US 20200265196A1 · Ravi · 2020 [cited by examiner]
US 20210089936A1 · Zhao · 2021 [cited by examiner]
US 20210248425A1 · Zong · 2021 [cited by examiner]
US 20210312124A1 · Agarwal · 2021 [cited by examiner]
US 20220245343A1 · Brun · 2022 [cited by examiner]
Li, Kun, et al. “Conditional augmentation for aspect term extraction via masked sequence-to-sequence generation.” arXiv preprint arxiv:2004.14769 (2020). (Year: 2020). [cited by examiner]
Sun, Chi, Luyao Huang, and Xipeng Qiu. “Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence.” arXiv preprint arXiv:1903.09588 (2019). (Year: 2019). [cited by examiner]
Pang et al., “Thumbs up? Sentiment Classification using Machine Learning Techniques.” May 28, 2002. arxiv preprint https://arxiv.org/abs/cs/0205070, pp. 1-9. [cited by applicant]
Turney, “Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews.” 2002. arXiv preprint https://arxiv.org/abs/cs/0212032, pp. 1-8. [cited by applicant]
Chevalier et al., “The effect of word of mouth on sales: Online book reviews.” In Journal of marketing research, 2006, 43(3):345-354. [cited by applicant]
Pontiki et al., “SemEval—2014 task 4: Aspect based sentiment analysis.” In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). 2014. pp. 27-35. [cited by applicant]
Pontiki et al., “Semeval—2016 Task 5: Aspect Based Sentiment Analysis.” In International workshop on semantic evaluation. Jan. 2016. pp. 19-30. [cited by applicant]
Rajpurkar et al., “SQuAD: 100,000+ Questions for Machine Comprehension of Text.” Oct. 11, 2016. arXiv preprint https://arxiv.org/abs/1606.05250, pp. 1-10. [cited by applicant]
Xu et al., “BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis.” May 4, 2019. arXiv preprint https://arxiv.org/abs/1904.02232, pp. 1-12. [cited by applicant]
Radford et al., “Improving Language Understanding by Generative Pre-Training.” 2018. https://s3-US-west-2.amazonaws.com/openai-assets/research-covers/languageunsupervised/language_understanding_paper.pdf, pp. 1-12. [cited by applicant]
Brown et al., “Language Models are Few-Shot Learners.” Jul. 22, 2020. arXiv preprint https://arxiv.org/abs/2005.14165, pp. 1-75. [cited by applicant]
Liu, “Sentiment analysis and opinion mining.” Synthesis lectures on human language technologies. 2012. 5(1):1-167. [cited by applicant]
Liu et al., “A Survey of Opinion Mining and Sentiment Analysis.” 2012. In Mining Text Data, pp. 415-463. [cited by applicant]
Cambria et al., “Sentic computing.” marketing. 2012. 59(2):557-577. [cited by applicant]
Hu et al., “Mining and summarizing customer reviews.” In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. 2004. pp. 168-177. [cited by applicant]
Samha et al., “Aspect-Based Opinion Extraction from Customer Reviews.” 2014. arXiv preprint https://arxiv.org/abs/1404.1982. pp. 1-12. [cited by applicant]
Xu et al., “Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction.” May 11, 2018. arXiv preprint https://arxiv.org/abs/1805.04601, pp. 1-7. [cited by applicant]
Li et al., “Exploiting Coarse-to-Fine Task Transfer for Aspect-Level Sentiment Classification.” 2019. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4253-4260. [cited by applicant]
Hochreiter et al., “Long Short-Term Memory.” 1997. Neural Computation, 9(8):1735-1780, pp. 1-32. [cited by applicant]
Devlin et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” May 24, 2019. arXiv preprint https://arXiv:1810.04805, pp. 1-16. [cited by applicant]
Karimi et al., “Improving BERT Performance for Aspect-Based Sentiment Analysis.” Mar. 1, 2021. arXiv preprint https://arxiv.org/abs/2010.11731, pp. 1-6. [cited by applicant]
Lafferty et al., “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data.” Jun. 28, 2001. pp. 1-10. [cited by applicant]
Reddy et al., “Does BERT Understand Sentiment? Leveraging Comparisons between Contextual and Non-Contextual Embeddings to Improve Aspect-based Sentiment Models.” Nov. 23, 2020. arXiv preprint https://arxiv.org/abs/2011.… [cited by applicant]
Pennington et al., “GloVe: Global Vectors for Word Representation.” In Proceedings of the 2014 Conference on empirical methods in natural language processing (EMNLP). Oct. 25-29, 2014. Doha, Qatar. pp. 1532-1543. [cited by applicant]
Bengio et al., “A Neural Probabilistic Language Model.” 2003. Journal of Machine Learning Research 3. 2003. pp. 1137-1155. [cited by applicant]
Li et al., “Conditional augmentation for aspect term extraction via masked sequence-to-sequence generation.” arXivpreprint arXiv:2004.14769 (2020). [cited by applicant]
Sun et al., “Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence.” arXivpreprint arXiv:1903.09588 (2019). [cited by applicant]