IP Library › Granted Patent US 12,363,057
Granted Patent B1
US 12,363,057 · App. 16/518,451 · Granted Jul 15, 2025

System and method for processing of speech content in email messages

Inventor: Justin Khoo (San Francisco, CA)
H04L51/10G10L13/00H04L51/224
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,363,057
App. No.
16/518,451
Granted
Jul 15, 2025
Kind
B1
Abstract

A system and method to provide email content within a separate part of an email, where the content is optimized for speaking and audio playback. Embodiments of the invention are methods for a speech-enabled email client to identify and use substitute speech content when outputting audio or reading to a user instead of the regular HTML or text parts of an email.

Claims (29)

1. A method performed by a speech-enabled email client, comprising:

receiving, at the speech-enabled email client, an email sent to a user, the email comprising a plurality of plain text headers including a “from” header containing a sender name, the email further comprising data identifying a DNS record,

wherein the DNS record is a TXT record that includes an attribute specifying a resource location comprising a location of a pre-recorded DNS-based binary audio file on a network, the DNS record associated with the sender name in the “from” header, and

wherein the DNS record includes data specifying that the DNS-based binary audio file is usable for audibly rendering the associated sender name;

the speech-enabled email client determining whether the email contains the data identifying the DNS-based binary audio file; and

when the speech-enabled email client determines that the email contains the data identifying the DNS-based binary audio file for the “from” header and that the sender domain is included in a whitelist, in response to a user request to the speech-enabled email client, retrieving the DNS-based binary audio file via DNS resolution and playing the DNS-based binary audio file instead of converting the “from” header text to speech using TTS.

2. The method of claim 1 , wherein the user request is a request to play a summary of a list of emails.

3. The method of claim 1 , wherein the user request is a request to play at least the email.

4. The method of claim 1 , wherein the location of the pre-recorded DNS-based binary audio file is stored as a URL.

5. The method of claim 1 , wherein the DNS record includes phoneme-based pronunciation data for the sender name in the “from” header.

6. The method of claim 5 , wherein the speech-enabled email client prioritizes retrieval of a pre-recorded binary audio file over phoneme-based pronunciation data when both exist in a DNS record.

7. The method of claim 1 , wherein the DNS record is associated with the sender domain.

8. The method of claim 1 , wherein the speech-enabled email client retrieves the DNS record for the sender domain only if the email has been authenticated using an authentication scheme.

9. The method of claim 8 wherein the authentication scheme is DomainKeys Identified Mail (DKIM).

10. The method of claim 9 wherein the location of the pre-recorded binary audio file is a TXT record associated with a subdomain of the sender domain.

11. A system, comprising:

a speech-enabled email client, that:

receives, at the speech-enabled email client, an email sent to a user, the email comprising a plurality of plain text headers including a “from” header containing a sender name, the email further comprising data identifying a DNS record,

wherein the DNS record is a TXT record that includes an attribute specifying a resource location comprising a location of a pre-recorded DNS-based binary audio file on a network, the DNS record associated with the sender name in the “from” header, and

wherein the DNS record includes data specifying that the DNS-based binary audio file is usable for audibly rendering the associated sender name;

determines whether the email contains the data identifying the DNS-based binary audio file; and

when the speech-enabled email client determines that the email contains the data identifying the DNS-based binary audio file for the “from” header and that the sender domain is included in a whitelist, in response to a user request to the speech-enabled email client, retrieves the DNS-based binary audio file via DNS resolution and plays the DNS-based binary audio file instead of converting the “from” header text to speech using TTS.

12. The system of claim 11 , wherein the location of the pre-recorded binary audio file is stored as a URL.

13. The system of claim 11 , wherein the DNS record includes phoneme-based pronunciation data for the sender name in the “from” field.

14. The system of claim 13 , wherein the speech-enabled email client prioritizes retrieval of a pre-recorded binary audio file over phoneme-based pronunciation data when both exist in a DNS record.

15. The system of claim 14 , wherein the DNS record is associated with a domain name of an email address of the sender.

16. The system of claim 11 , wherein the speech-enabled email client retrieves the DNS record for the sender domain only if the email has been authenticated using an authentication scheme.

17. The system of claim 16 wherein the authentication scheme is DomainKeys Identified Mail (DKIM).

18. The system of claim 17 wherein the location of the pre-recorded binary audio file is a TXT record associated with a subdomain of a domain of the sender.

Continuity (2)
Provisional Application 62711469 · Jul 27, 2018
Provisional Application 62701609 · Jul 20, 2018
References Cited (132)
US 6707891B1 · Guedalia · 2004 [cited by examiner]
US 6907578B2 · Wilson et al. · 2005 [cited by applicant]
US 6918082B1 · Gross et al. · 2005 [cited by applicant]
US 7219131B2 · Banister et al. · 2007 [cited by applicant]
US 7305242B2 · Zakharia et al. · 2007 [cited by applicant]
US 7627814B1 · Soldan et al. · 2009 [cited by applicant]
US 7814410B2 · Kothari et al. · 2010 [cited by applicant]
US 7900149B2 · Hatcher et al. · 2011 [cited by applicant]
US 8032597B2 · Khoo · 2011 [cited by applicant]
US 8285813B1 · Colton et al. · 2012 [cited by applicant]
US 8335982B1 · Colton et al. · 2012 [cited by applicant]
US 8527860B1 · Colton et al. · 2013 [cited by applicant]
US 8639680B1 · Ciccolo et al. · 2014 [cited by applicant]
US 8639743B1 · Colton et al. · 2014 [cited by applicant]
US 8719451B1 · Colton et al. · 2014 [cited by applicant]
US 8898296B2 · Zeng et al. · 2014 [cited by applicant]
US 8959426B1 · Thakare et al. · 2015 [cited by applicant]
US 8984048B1 · Maniscalco et al. · 2015 [cited by applicant]
US 9166977B2 · Desai · 2015 [cited by examiner]
US 9218267B1 · Keller · 2015 [cited by applicant]
US 9400774B1 · Kim et al. · 2016 [cited by applicant]
US 9424236B2 · Sullivan et al. · 2016 [cited by applicant]
US 9607332B1 · Nazarov et al. · 2017 [cited by applicant]
US 9817916B2 · Flack · 2017 [cited by applicant]
US 10031549B2 · Costa · 2018 [cited by applicant]
US 20010042098A1 · Gupta et al. · 2001 [cited by applicant]
US 20040034688A1 · Dunn · 2004 [cited by applicant]
US 20040193441A1 · Altieri · 2004 [cited by applicant]
US 20040234169A1 · Tojo · 2004 [cited by applicant]
US 20040268261A1 · Elliott et al. · 2004 [cited by applicant]
US 20050022116A1 · Bowman et al. · 2005 [cited by applicant]
US 20050081159A1 · Gupta et al. · 2005 [cited by applicant]
US 20050100143A1 · Bedingfield · 2005 [cited by examiner]
US 20050246341A1 · Vuattoux et al. · 2005 [cited by applicant]
US 20060023238A1 · Blaszyk et al. · 2006 [cited by applicant]
US 20060174187A1 · White et al. · 2006 [cited by applicant]
US 20060210028A1 · Ng · 2006 [cited by examiner]
US 20070011258A1 · Khoo · 2007 [cited by applicant]
US 20070038718A1 · Khoo et al. · 2007 [cited by applicant]
US 20070073848A1 · Fratti · 2007 [cited by examiner]
US 20070174396A1 · Kumar · 2007 [cited by examiner]
US 20070201752A1 · Gormish et al. · 2007 [cited by applicant]
US 20070211071A1 · Slotznick et al. · 2007 [cited by applicant]
US 20080005247A9 · Khoo · 2008 [cited by applicant]
US 20080178073A1 · Gao et al. · 2008 [cited by applicant]
US 20080307328A1 · Hatcher et al. · 2008 [cited by applicant]
US 20090055187A1 · Leventhal · 2009 [cited by examiner]
US 20090112687A1 · Blair et al. · 2009 [cited by applicant]
US 20090190728A1 · Bushnell · 2009 [cited by examiner]
US 20090240497A1 · Usher · 2009 [cited by examiner]
US 20090254345A1 · Fleizach · 2009 [cited by examiner]
US 20100031366A1 · Knight et al. · 2010 [cited by applicant]
US 20100241488A1 · Jacobson · 2010 [cited by applicant]
US 20110065376A1 · Forutanpour et al. · 2011 [cited by applicant]
US 20110113364A1 · Neil et al. · 2011 [cited by applicant]
US 20110197133A1 · Tarjan et al. · 2011 [cited by applicant]
US 20110202621A1 · Laval · 2011 [cited by applicant]
US 20110258532A1 · Ceze et al. · 2011 [cited by applicant]
US 20110314368A1 · Chevalier · 2011 [cited by applicant]
US 20120110395A1 · Gearhart et al. · 2012 [cited by applicant]
US 20120155292A1 · Zazula et al. · 2012 [cited by applicant]
US 20120173967A1 · Lillesveen · 2012 [cited by applicant]
US 20120212501A1 · Berkland et al. · 2012 [cited by applicant]
US 20120215540A1 · Goktekin · 2012 [cited by examiner]
US 20120245937A1 · Thenthiruperai · 2012 [cited by examiner]
US 20120278695A1 · Ju et al. · 2012 [cited by applicant]
US 20120278700A1 · Sullivan et al. · 2012 [cited by applicant]
US 20130007711A1 · Fryc et al. · 2013 [cited by applicant]
US 20130046616A1 · Williams et al. · 2013 [cited by applicant]
US 20130174010A1 · Le et al. · 2013 [cited by applicant]
US 20130174011A1 · Le et al. · 2013 [cited by applicant]
US 20130176344A1 · Mandic et al. · 2013 [cited by applicant]
US 20130179776A1 · Rossi et al. · 2013 [cited by applicant]
US 20130190021A1 · Vieri · 2013 [cited by examiner]
US 20130197902A1 · Nazarian · 2013 [cited by examiner]
US 20130201107A1 · Rossi et al. · 2013 [cited by applicant]
US 20130205277A1 · Seven et al. · 2013 [cited by applicant]
US 20130219024A1 · Flack · 2013 [cited by applicant]
US 20130290786A1 · Artzi et al. · 2013 [cited by applicant]
US 20130325980A1 · Ohayon · 2013 [cited by applicant]
US 20140123036A1 · Bao et al. · 2014 [cited by applicant]
US 20140136944A1 · Harris et al. · 2014 [cited by applicant]
US 20140282076A1 · Fischer · 2014 [cited by applicant]
US 20140335833A1 · Woloshyn · 2014 [cited by examiner]
US 20150074518A1 · Rumsey et al. · 2015 [cited by applicant]
US 20150082058A1 · Hahm et al. · 2015 [cited by applicant]
US 20150161087A1 · Khoo · 2015 [cited by applicant]
US 20150180811A1 · Cornwell et al. · 2015 [cited by applicant]
US 20150207804A1 · Van et al. · 2015 [cited by applicant]
US 20150242380A1 · Guo et al. · 2015 [cited by applicant]
US 20150309993A1 · Wilde · 2015 [cited by applicant]
US 20160086620A1 · Ostermann · 2016 [cited by examiner]
US 20160132937A1 · Khoo et al. · 2016 [cited by applicant]
US 20160259773A1 · Jadhav et al. · 2016 [cited by applicant]
US 20160284341A1 · Hirakawa · 2016 [cited by examiner]
US 20170018021A1 · Shekhawat et al. · 2017 [cited by applicant]
US 20180090126A1 · Peterson · 2018 [cited by examiner]
US 20180247640A1 · Yassa · 2018 [cited by examiner]
US 20190044902A1 · Teplow et al. · 2019 [cited by applicant]
US 20190073342A1 · Wilson et al. · 2019 [cited by applicant]
US 20200151223A1 · Ly · 2020 [cited by examiner]
US 20200388269A1 · Niehaus · 2020 [cited by examiner]
US 20210209121A1 · Liu · 2021 [cited by examiner]
Campaign Wokhub, “The Only Proofing Solution Built for Email Campaigns”, Available Online at <http://campaignworkhub.com:80/>, Accessed on Oct. 2, 2015, 5 pages. [cited by applicant]
Conceptshare—Tour, “Five Favorite Features Chosen by Customers”, Available Online at <https://www.conceptshare.com/tour>, accessed on Oct. 27, 2018, 4 pages. [cited by applicant]
Dave, Shyamal, “Head to Litmus for Email Testing and Marketing Analytics Needs”, Available Online at <https://yourstory.com/2012/08/litmus>, Aug. 13, 2012, 2 pages. [cited by applicant]
Deltek + Conceptshare, “New Feature List”, Available Online at <http://www.conceptshare.com/new_features>, accessed on Oct. 29, 2018, 7 pages. [cited by applicant]
Email on Acid, “How Can I Run an Email Test?”, Available Online at <https://www.emailonacid.com/help-article/how-can-i-run-an-email-test/>, accessed on Oct. 27, 2018, 9 pages. [cited by applicant]
Garsiel et al., “How Browsers Work: Behind the Scenes of Modern Web Browsers”, Available Online at <https://www.html5rocks.com/en/tutorials/internals/howbrowserswork/>, Aug. 5, 2011, 55 pages. [cited by applicant]
Invision, “Free Design Feedback and Collaboration Tool”, Available Online at <https://www.invisionapp.com/tour/design-feedback-collaboration-tool/>, accessed on Oct. 27, 2018, 5 pages. [cited by applicant]
IPhoneHacks, “How to Use a Mouse on Your iPad & iPhone [Jailbreak Tips]”, Available Online at <https://www.iphonehacks.com/2012/10/how-to-use-a-mouse-on-the-ipad.html>, Oct. 7, 2012, 9 pages. [cited by applicant]
Litmus, “Verify and Check Your Email with Litmus Checklist”, Available Online at <https://litmus.com/email-checklist#0>, accessed on Oct. 27, 2018, 7 pages. [cited by applicant]
Mark Goddard, “Fixing the HOver Event on the iPad/iPhone/iPod”, Available Online at <http://blog.0100.tv/2010/05/fixing-the-hover-event-on-the-ipadiphoneipod/>, 2010, 6 pages. [cited by applicant]
Merriam-Webster, “Definition of Disable”, Available Online at <https://www.merriam-webster.com/dictionary/disable#:˜:text=%3A%20to%20make%20unable%20or%20incapable%20He%20disabled%20the%20computer%20system.>, accessed o… [cited by applicant]
Motto, Todd, “Building an HTML5 Responsive Menu with Media Queries and JavaScript—Ultimate Courses”, Available Online at <https://ultimatecourses.com/blog/building-an-html5-responsive-menu-with-media-queries-javascript>… [cited by applicant]
Notice of Allowance, U.S. Appl. No. 16/361,140, Mar. 30, 2021, 5 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 16/403,322, Mar. 24, 2021, 8 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/696,152, Mar. 25, 2021, 8 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 15/696,152, Dec. 16, 2020, 9 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 16/361,140, May 18, 2021, 9 pages. [cited by applicant]
Pelago, “Sidecar: Emogrifier”, Available Online at <https://web.archive.org/web/20130425165247/http://www.pelagodesign.com/sidecar/emogrifier/>, Apr. 25, 2013, 2 pages. [cited by applicant]
Radford. Edu, “Style Sheet Locations for COMS326”, Available Online at <https://www.radford.edu/˜rstepno/326/csslocations.html>, Mar. 30, 2013, 2 pages. [cited by applicant]
taligarsiel.com, “Behind the Scenes of Modern Web Browsers”, Available Online at <http://taligarsiel.com/Projects/howbrowserswork1.htm>, 27 pages. [cited by applicant]
W3C Working Draft, “Document Object Model (DOM) Level 3 Events Specification”, Available Online at <https://www.w3.org/TR/2013/WD-DOM-Level-3-Events-20131105/>, Nov. 2, 2013, 88 pages. [cited by applicant]
Wilson et al, “Touch and Mouse: Together Again for the First Time”, Available Online at <https://www.html5rocks.com/en/mobile/touchandmouse/>, Mar. 13, 2013, 10 pages. [cited by applicant]
Email Previews, Screen captures of Litmus.com Web Site, https://litmus.com/email-testing, accessed Jun. 13, 2018 (12 pages). [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 17/361,234, Feb. 18, 2022, 11 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 18/045,163, May 15, 2023, 34 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 17/361,234, Jun. 8, 2022, 8 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 17/409,487, Oct. 7, 2022, 13 pages. [cited by applicant]
Rice, Jerry, “LifeFX lives on online”, Magazine, Aug. 9, 2001, 11 pages, Variety, Los Angeles, California, USA, Accessed Aug. 16, 2024, URL: https://variety.com/2001/digital/features/lifefx-lives-on-online-1117851059/. [cited by applicant]
“BIMI Record—What is it? How Does it Add to DMARC?”, Article, 2 pages, MX Toolbox, Austin, Texas, USA, Accessed Aug. 16, 2024, URL: https://mxtoolbox.com/dmarc/details/bimi-record/what-is-a-bimi-record. [cited by applicant]