IP Library Granted Patent US 12,254,872
Granted Patent B2
US 12,254,872 · App. 18/392,402 · Granted Mar 18, 2025

Search system and method having civility score

Inventors: Robin J. Clark (Vienna, VA); Ali Taleb Zadeh Kasgari (Vienna, VA); Stefanos Poulis (Vienna, VA)
Assignee: SEEKR TECHNOLOGIES INC.
G10L15/183G10L15/22G10L15/26G10L25/48G10L15/16G10L25/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,254,872
App. No.
18/392,402
Granted
Mar 18, 2025
Kind
B2
Abstract

A scoring system and method identifies personal attacks in a piece of audio content and generates a civility score for the piece of audio content that can differentiate between personal attacks and vernacular/casual banter. The piece of audio content may be a podcast.

Claims (26)

1. A system, comprising:

a computer system having a processor and a plurality of lines of instructions that are executed by the processor that is configured to:

retrieve a plurality of podcasts;

generate, for each podcast using a trained machine learning model, a civility score wherein the civility score for each podcast indicates a quantity of personal attacks in the podcast; and

communicate, via an application programming interface, the civility score for a particular podcast from the computer system to a second computer system.

2. The system of claim 1 , wherein the second computer system is an advertisement system that sends a request for the civility score for the particular podcast.

3. The system of claim 2 , wherein the advertisement system selects an advertisement for the particular podcast based on the civility score.

4. The system of claim 1 , wherein the processor is further configured to generate a user interface that displays an image representing the particular podcast, the civility score for the particular podcast and a change in civility since a prior episode of the particular podcast.

5. The system of claim 1 , wherein the personal attack is directed to a third party and is one of a profanity, derogatory language, a negative evaluation and a negative perspective on a situation.

6. The system of claim 1 , wherein the processor is further configured to separate a transcribed podcast into a plurality of pieces of data and train the trained machine learning model using a training data set formed based on the plurality of pieces of data.

7. The system of claim 6 , wherein each piece of data is a portion of the transcribed podcast when one person is talking during the transcribed podcast.

8. The system of claim 1 , wherein the trained machine learning model is a transformer model.

9. The system of claim 6 , wherein the processor configured to train the trained machine learning model is further configured to invoke a plurality of large language models to generate a label for each piece of data of the transcribed podcast for each large language model and aggregate the labels from each large language model into a final label for each piece of data that form the training data set.

10. A method, comprising:

retrieving, by a computer system having a processor and a plurality of lines of instructions that are executed by the processor, a plurality of podcasts;

generating, by the computer system for each podcast using a trained machine learning model, a civility score wherein the civility score for each podcast indicates a quantity of personal attacks in the podcast; and

communicating, via an application programming interface of the computer system, the civility score for a particular podcast from the computer system to a second computer system.

11. The method of claim 10 further comprising sending, by the second computer system that is an advertisement system, a request for the civility score for the particular podcast.

12. The method of claim 11 further comprising selecting, by the advertisement system, an advertisement for the particular podcast based on the civility score.

13. The method of claim 10 further comprising generating, by the computer system, a user interface that displays an image representing the particular podcast, the civility score for the particular podcast and a change in civility since a prior episode of the particular podcast.

14. The method of claim 10 , wherein the personal attack is directed to a third party and is one of a profanity, derogatory language, a negative evaluation and a negative perspective on a situation.

15. The method of claim 10 further comprising separating, by the computer system, a transcribed podcast into a plurality of pieces of data and training, by the computer system, the trained machine learning model using a training data set formed based on the plurality of pieces of data.

16. The method of claim 15 , wherein each piece of data is a portion of the transcribed podcast when one person is talking during the transcribed podcast.

17. The method of claim 15 , wherein training the trained machine learning model further comprises invoking a plurality of large language models to generate a label for each piece of data of the transcribed podcast for each large language model and aggregating the labels from each large language model into a final label for each piece of data that form the training data set.

18. The system of claim 1 , wherein the particular podcast is one of a podcast episode and an entire podcast.

19. The method of claim 10 , wherein the particular podcast is one of a podcast episode and an entire podcast.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2025
From: CLARK, ROBIN J.; KASGARI, ALI TALEB ZADEH; POULIS, STEFANOS
To: SEEKR TECHNOLOGIES INC.
Reel/Frame 070178/0800 →
Continuity (3)
Continuation 18243588 · Sep 7, 2023
Continuation In Part 18220437 · Jul 11, 2023
Related Publication 20250022461A1 · Jan 16, 2025
References Cited (85)
US 5696962A · Kupiec · 1997 [cited by applicant]
US 5706507A · Schloss · 1998 [cited by examiner]
US 5909510A · Nakayama · 1999 [cited by applicant]
US 6026388A · Liddy et al. · 2000 [cited by applicant]
US 6119114A · Smadja · 2000 [cited by applicant]
US 6226668B1 · Silverman · 2001 [cited by applicant]
US 6601075B1 · Huang et al. · 2003 [cited by applicant]
US 6651057B1 · Jin et al. · 2003 [cited by applicant]
US 6807565B1 · Dodrill et al. · 2004 [cited by applicant]
US 6847969B1 · Mathal et al. · 2005 [cited by applicant]
US 6990514B1 · Dodrill et al. · 2006 [cited by applicant]
US 7062485B1 · Jin et al. · 2006 [cited by applicant]
US 7076484B2 · Dworkis et al. · 2006 [cited by applicant]
US 7120925B2 · D'Souza et al. · 2006 [cited by applicant]
US 7197497B2 · Cossock · 2007 [cited by applicant]
US 7240067B2 · Timmons · 2007 [cited by applicant]
US 7313622B2 · Lee et al. · 2007 [cited by applicant]
US 7475404B2 · Hamel · 2009 [cited by applicant]
US 7516123B2 · Betz et al. · 2009 [cited by applicant]
US 7606810B1 · Jeavons · 2009 [cited by applicant]
US 7827125B1 · Rennison · 2010 [cited by applicant]
US 7836060B1 · Rennison · 2010 [cited by applicant]
US 7870117B1 · Rennison · 2011 [cited by applicant]
US 7925973B2 · Allaire et al. · 2011 [cited by applicant]
US 7933893B2 · Walker et al. · 2011 [cited by applicant]
US 8001064B1 · Rennison · 2011 [cited by applicant]
US 8060518B2 · Timmons · 2011 [cited by applicant]
US 8195666B2 · Jeavons · 2012 [cited by applicant]
US 8219911B2 · Clarke-Martin et al. · 2012 [cited by applicant]
US 10733452B2 · Attorre · 2020 [cited by applicant]
US 11893981B1 · Clark · 2024 [cited by examiner]
US 20010021934A1 · Yokoi · 2001 [cited by applicant]
US 20020007393A1 · Hamel · 2002 [cited by applicant]
US 20020062340A1 · Kloecker et al. · 2002 [cited by applicant]
US 20020095336A1 · Trifon et al. · 2002 [cited by applicant]
US 20020147578A1 · O'Neil et al. · 2002 [cited by applicant]
US 20020169669A1 · Stetson et al. · 2002 [cited by applicant]
US 20020169771A1 · Melmon et al. · 2002 [cited by applicant]
US 20030191816A1 · Landress et al. · 2003 [cited by applicant]
US 20030212954A1 · Patrudu · 2003 [cited by applicant]
US 20030217052A1 · (Il]et al. · 2003 [cited by applicant]
US 20040049574A1 · Watson et al. · 2004 [cited by applicant]
US 20040117400A1 · McCrystal et al. · 2004 [cited by applicant]
US 20050091200A1 · Melton et al. · 2005 [cited by applicant]
US 20050144158A1 · Capper · 2005 [cited by examiner]
US 20050188402A1 · de Andrade et al. · 2005 [cited by applicant]
US 20060031419A1 · Huat · 2006 [cited by applicant]
US 20060047632A1 · Zhang · 2006 [cited by applicant]
US 20060117348A1 · D'Souza et al. · 2006 [cited by applicant]
US 20070033531A1 · Marsh · 2007 [cited by applicant]
US 20070038567A1 · Allaire et al. · 2007 [cited by applicant]
US 20070038931A1 · Allaire et al. · 2007 [cited by applicant]
US 20070260671A1 · Harinstein et al. · 2007 [cited by applicant]
US 20080010142A1 · O'Brien et al. · 2008 [cited by applicant]
US 20080104113A1 · Wong · 2008 [cited by applicant]
US 20080221983A1 · Ausiannik et al. · 2008 [cited by applicant]
US 20090024574A1 · Timmons · 2009 [cited by applicant]
US 20090197581A1 · Gupta et al. · 2009 [cited by applicant]
US 20090248668A1 · Zheng · 2009 [cited by applicant]
US 20100100545A1 · Jeavons · 2010 [cited by applicant]
US 20100313116A1 · Hyman · 2010 [cited by applicant]
US 20110166918A1 · Allaire et al. · 2011 [cited by applicant]
US 20110191163A1 · Allaire et al. · 2011 [cited by applicant]
US 20120078895A1 · Chu-Carroll · 2012 [cited by applicant]
US 20120143792A1 · Wang · 2012 [cited by applicant]
US 20130318063A1 · Ayzenshtat · 2013 [cited by applicant]
US 20150095014A1 · Marimuthu · 2015 [cited by examiner]
US 20160021037A1 · Hewitt · 2016 [cited by applicant]
US 20180101534A1 · Alexander, Jr. · 2018 [cited by applicant]
US 20190065744A1 · Gaustad · 2019 [cited by applicant]
US 20190082224A1 · Bradley · 2019 [cited by applicant]
US 20190147062A1 · Kim · 2019 [cited by applicant]
US 20190163327A1 · Otero · 2019 [cited by applicant]
US 20200125639A1 · Doyle · 2020 [cited by applicant]
US 20200126533A1 · Doyle · 2020 [cited by applicant]
US 20210004420A1 · Mittal · 2021 [cited by applicant]
US 20210019339A1 · Ghulati · 2021 [cited by applicant]
WO WO0077690A1 · 2000 [cited by applicant]
Baulepur, “Aligning Language Models with Factuality and Truthfulness” Thesis submitted in partial fulfillment of Bachelor of Science in Computer Science, University of Illinois at Urbana—Champaign, 2023, 50 pages. [cited by applicant]
Azaria, et al., “The Internal State of an LLM Knows When its Lying”, School of Computer Science, Ariel University, Israel and Machine Learning Dept., Carnegie Mellon University, Pittsburgh, PA, Apr. 2023, 10 pages. [cited by applicant]
Lee, et al., “Linguistic Properties of Truthful Response,” University of Pennsylvania, PA, USA., Jun. 2023, 6 pages. [cited by applicant]
Poulis, “Algorithms for Interactive Machine Learning”, Dissertation submitted in partial fulfillment of degree of Doctor of Philosophy in Computer Science, University of California, San Diego, 2019, 148 pages. [cited by applicant]
Yang, et al., “RefGPT: Reference—Truthful & Customized Dialogues Generation by GPTs and for GPTs”, Shanghai Jiao Tong University, Hong Kong Polytechnical University, Beijing University of Posts and Telecommunications, M… [cited by applicant]
Pan, et al., “On the Risk of Misinformation Pollution with Large Language Models”, National University of Singapore, University of California, Santa Barbara, University of Waterloo, MBZUAI, Zhejiang University, May 2023… [cited by applicant]
McKenna, et al., “Sources of Hallucination by Large Language Models on Inference Tasks”, University of Edinburgh, Google Research, Macquarie University, May 2023, 17 pages. [cited by applicant]
Cited By (6)
US 1,094,414 US 1,096,797 US 1,109,183 US 12,499,387 US 12,511,557 US 12,579,475