IP Library › Granted Patent US 12,739,217
Granted Patent B2
US 12,739,217 · App. 18/055,238 · Granted Sep 15, 2026

Generating editable email components utilizing a constraint-based knowledge representation

Inventors: Yeuk-yin Chan (New York, NY); Andrew Thomson (Moraga, CA); Caroline Kim (San Francisco, CA); Cole Connelly (New York, NY); Eunyee Koh (San Jose, CA); Michelle Lee (San Francisco, CA); Shunan Guo (San Jose, CA)
Assignee: Adobe Inc.
H04L51/07G06F3/04842G06F8/38G06F16/3322G06F40/103G06N5/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,739,217
App. No.
18/055,238
Granted
Sep 15, 2026
Kind
B2
Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates editable email components by utilizing an Answer Set Programming (ASP) model with hard and soft constraints. For instance, in one or more embodiments, the disclosed systems generate editable email components from email fragments of an email file utilizing an Answer Set Programming (ASP) model. In particular, the disclosed systems extract facts for the ASP model from the email file. In addition, the disclosed systems determine rows or columns defining cells of the email file utilizing ASP hard constraints defined by a first set of ASP atoms corresponding to the facts. Moreover, the disclosed systems determine editable email component classes for the email fragments utilizing ASP soft constraints defined by ASP classification weights and a second set of ASP atoms corresponding to the facts.

Claims (62)

1 . A method comprising:

receiving a hypertext markup language (HTML) email file comprising HTML fragments generated utilizing a first email editing application; and

generating editable email components from the HTML fragments for editing by a second email editing application incompatible with the first email editing application utilizing an Answer Set Programming (ASP) model by:

extracting facts for the ASP model from the HTML email file;

determining rows or columns defining cells of the HTML email file utilizing ASP hard constraints defined by a first set of ASP atoms corresponding to the facts;

determining editable email component classes for the HTML fragments utilizing ASP soft constraints defined by ASP classification weights and a second set of ASP atoms corresponding to the facts; and

generating the editable email components from the rows or columns defining the cells of the HTML email file and from the editable email component classes for the HTML fragments.

2 . The method of claim 1 , wherein extracting the facts for the ASP model from the HTML email file further comprises at least one of:

determining the facts based on an embedded style sheet of the HTML email file,

determining fragment width, height, or position from a browser rendering of the HTML email file, or

extracting colors or fonts from the HTML email file utilizing a computer vision library.

3 . The method of claim 1 , wherein the HTML email file is generated utilizing the first email editing application and further comprising generating the editable email components from the HTML email file for editing by the second email editing application by:

identifying incompatible labels for the HTML fragments corresponding to the first email editing application from the HTML email file;

upon determining the editable email component classes, generating labels for the HTML fragments compatible with the second email editing application; and

replacing the incompatible labels with labels compatible with the second email editing application to generate the editable email components.

4 . The method of claim 1 , wherein determining the rows or columns defining the cells of the HTML email file utilizing the ASP hard constraints comprises executing a row detection task of the ASP model by:

determining a set of row partition results utilizing the ASP hard constraints;

determining numbers of row partitions in the set of row partition results utilizing a head ASP atom; and

selecting rows defining the cells of the HTML email file utilizing an ASP objective function based on the head ASP atom.

5 . The method of claim 4 , wherein determining the set of row partition results utilizing the ASP hard constraints comprises determining the set of row partition results by utilizing the ASP model to search for possible row partitions subject to the ASP hard constraints, wherein the ASP hard constraints comprise at least one of: a number of rows hard constraint, an email element overlap hard constraint, or an element cross-row duplication hard constraint.

6 . The method of claim 1 , wherein determining editable email component classes utilizing the ASP soft constraints comprises executing an element classification task of the ASP model by, for a first ASP soft constraint, determining a first weight for an editable email component class of an email fragment of the HTML email file utilizing a first bodiless rule defined by a first ASP atom corresponding to a first email component feature and a first ASP classification weight.

7 . The method of claim 6 , wherein executing the element classification task of the ASP model further comprises for a second ASP soft constraint, determining a second weight for the editable email component class of the email fragment of the HTML email file utilizing a second bodiless rule defined by a second ASP atom corresponding to a second email component feature and a second ASP classification weight.

8 . The method of claim 7 , wherein executing the element classification task of the ASP model further comprises selecting the editable email component class for the email fragment utilizing an ASP objective function based on the first weight and the second weight.

9 . The method of claim 1 , further comprising providing the editable email components, for display, via an email editing user interface of a second email editing application, the email editing user interface comprising one or more cell elements based on the cells and one or more editing options based on the editable email component classes.

10 . A non-transitory computer-readable medium storing executable instructions which, when executed by at least one processing device, cause the at least one processing device to perform operations comprising:

receiving an HTML email file comprising HTML fragments generated utilizing a first email editing application; and

generating editable email components from the HTML fragments by a second email editing application incompatible with the first email editing application utilizing an Answer Set Programming (ASP) model by:

extracting facts for the ASP model from the HTML email file;

determining rows or columns defining cells of the HTML email file utilizing ASP hard constraints defined by a first set of ASP atoms corresponding to the facts;

determining editable email component classes for the HTML fragments utilizing ASP soft constraints defined by ASP classification weights and a second set of ASP atoms corresponding to the facts; and

generating the editable email components from the rows or columns defining the cells of the HTML email file and from the editable email component classes for the HTML fragment.

11 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by at least one processing device, cause the at least one processing device to perform operations comprising converting the HTML fragments to editable email components within the second email editing application based on the editable email component classes.

12 . The non-transitory computer-readable medium of claim 10 , wherein determining the rows or columns defining the cells of the HTML email file utilizing the ASP hard constraints comprises:

determining a set of row partition results utilizing the ASP hard constraints; and

selecting rows defining the cells of the HTML email file utilizing an ASP objective function based on the set of row partition results.

13 . The non-transitory computer-readable medium of claim 10 , wherein determining editable email component classes for the HTML fragments comprises:

for a first ASP soft constraint, determining a first weight for a first editable email component class of a first email fragment of the HTML email file utilizing a first bodiless rule defined by a first ASP atom corresponding to a first email component feature and a first ASP classification weight; and

for a second ASP soft constraint, determining a second weight for a second editable email component class of the first email fragment of the HTML email file utilizing a second bodiless rule defined by a second ASP atom corresponding to a second email component feature and a second ASP classification weight.

14 . The non-transitory computer-readable medium of claim 13 , wherein generating editable email components from the HTML email file further comprises selecting the first editable email component class for the first email fragment utilizing an ASP objective function based on the first weight for the first editable email component class and the second weight for the second editable email component class.

15 . A system comprising:

one or more memory components; and

one or more processing devices coupled to the one or more memory components, the one or more processing devices to perform operations comprising:

receiving a hypertext markup language (HTML) email file comprising HTML fragments generated utilizing a first email editing application; and

generating editable email components from the HTML fragments for editing by a second email editing application incompatible with the first email editing application utilizing an Answer Set Programming (ASP) model by:

extracting facts for the ASP model from the HTML email file;

determining rows or columns defining cells of the HTML email file utilizing ASP hard constraints defined by a first set of ASP atoms corresponding to the facts;

determining editable email component classes for the HTML fragments utilizing ASP soft constraints defined by ASP classification weights and a second set of ASP atoms corresponding to the facts; and

generating the editable email components from the rows or columns defining the cells of the HTML email file and from the editable email component classes for the HTML fragments.

16 . The system of claim 15 , wherein extracting the facts for the ASP model from the HTML email file further comprises at least one of:

determining the facts based on an embedded style sheet of the HTML email file,

determining fragment width, height, or position from a browser rendering of the HTML email file, or

extracting colors or fonts from the HTML email file utilizing a computer vision library.

17 . The system of claim 15 , wherein the HTML email file is generated utilizing the first email editing application and further comprising generating the editable email components from the HTML email file for editing by the second email editing application by:

identifying incompatible labels for the HTML fragments corresponding to the first email editing application from the HTML email file;

upon determining the editable email component classes, generating labels for the HTML fragments compatible with the second email editing application; and

replacing the incompatible labels with labels compatible with the second email editing application to generate the editable email components.

18 . The system of claim 15 , wherein determining the rows or columns defining the cells of the HTML email file utilizing the ASP hard constraints comprises executing a row detection task of the ASP model by:

determining a set of row partition results utilizing the ASP hard constraints;

determining numbers of row partitions in the set of row partition results utilizing a head ASP atom; and

selecting rows defining the cells of the HTML email file utilizing an ASP objective function based on the head ASP atom.

19 . The system of claim 18 , wherein determining the set of row partition results utilizing the ASP hard constraints comprises determining the set of row partition results by utilizing the ASP model to search for possible row partitions subject to the ASP hard constraints, wherein the ASP hard constraints comprise at least one of: a number of rows hard constraint, an email element overlap hard constraint, or an element cross-row duplication hard constraint.

20 . The system of claim 15 , wherein determining editable email component classes utilizing the ASP soft constraints comprises executing an element classification task of the ASP model by, for a first ASP soft constraint, determining a first weight for an editable email component class of an email fragment of the HTML email file utilizing a first bodiless rule defined by a first ASP atom corresponding to a first email component feature and a first ASP classification weight.

Assignments (2)
ASSIGNMENT OF INTELLECTUAL PROPERTY Recorded Mar 5, 2024
From: TEAMPAY CORPORATION
To: PAYSTAND INC.
Reel/Frame 066739/0696 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2022
From: CHAN, YEUK-YIN; THOMSON, ANDREW; KIM, CAROLINE; CONNELLY, COLE; KOH, EUNYEE; LEE, MICHELLE; GUO, SHUNAN
To: ADOBE INC.
Reel/Frame 061762/0891 →
Continuity (1)
Related Publication 20240163238A1 · May 16, 2024
References Cited (52)
US 7069497B1 · Desai · 2006 [cited by examiner]
US 8527867B2 · Nair · 2013 [cited by examiner]
US 8620848B1 · Komissarchik · 2013 [cited by examiner]
US 8832205B2 · Nelson · 2014 [cited by examiner]
US 9384187B2 · Verma · 2016 [cited by examiner]
US 10120845B1 · Palm · 2018 [cited by examiner]
US 10657313B2 · Marchsreiter · 2020 [cited by examiner]
US 11144718B2 · Park · 2021 [cited by examiner]
US 11323543B2 · Fedyak · 2022 [cited by examiner]
US 12333240B2 · Becker · 2025 [cited by examiner]
US 20080178073A1 · Gao · 2008 [cited by examiner]
US 20140280134A1 · Horen · 2014 [cited by examiner]
US 20150032829A1 · Barshow · 2015 [cited by examiner]
US 20150212889A1 · Amarendran · 2015 [cited by examiner]
US 20150347368A1 · Carlen · 2015 [cited by examiner]
US 20190155870A1 · Prakash · 2019 [cited by examiner]
M Elgin Akpinar and Yeliz Yeşilada. 2013. Heuristic role detection of visual elements of web pages. In International Conference on Web Engineering. Springer, 123-131. [cited by applicant]
Massimiliano Albanese, Matthias Broecheler, John Grant, Maria Vanina Martinez, and VS Subrahmanian. 2011. PLINI: a probabilistic logic program framework for inconsistent news information. In Logic programming, knowledge… [cited by applicant]
Gerhard Brewka, Thomas Eiter, and Mirosław Truszczyński. 2011. Answer set programming at a glance. Commun. ACM 54, 12 (2011), 92-103. [cited by applicant]
Radek Burget and Ivana Rudolfova. 2009. Web page element classification based on visual features. In 2009 First Asian Conference on Intelligent Information and Database Systems. IEEE, 67-72. [cited by applicant]
Deng Cai, Shipeng Yu, Ji-Rong Wen, and Wei-Ying Ma. 2003. Vips: a visionbased page segmentation algorithm. (2003). [cited by applicant]
Hadrien Cambazard, Tarik Hadzic, and Barry O'Sullivan. 2010. Knowledge compilation for itemset mining. In ECAI 2010. IOS Press, 1109-1110. [cited by applicant]
Sonia Castelo, Thais Almeida, Anas Elghafari, Aécio Santos, Kien Pham, Eduardo Nakamura, and Juliana Freire. 2019. A topic-agnostic approach for identifying fake news pages. In Companion proceedings of the 2019 World Wi… [cited by applicant]
Sandip Debnath, Prasenjit Mitra, and C Lee Giles. 2005. Automatic extraction of informative blocks from webpages. In Proceedings of the 2005 ACM symposium on Applied computing. 1722-1726. [cited by applicant]
Carlos Duarte, Ana Salvado, M Elgin Akpinar, Yeliz Yeşilada, and Luís Carriço. 2018. Automatic role detection of visual elements of web pages for automatic accessibility evaluation. In Proceedings of the 15th Internatio… [cited by applicant]
Julien Dumazert. 2017. Understanding the layout of webpages using automatic zone recognition. Contentsquare Engineering: Stories from the people building Contentsquare (Mar. 6, 2017)(Year: 2017) (2017). [cited by applicant]
Esra Erdem, Erdi Aker, and Volkan Patoglu. 2012. Answer set programming for collaborative housekeeping robotics: representation, reasoning, and execution. Intelligent Service Robotics 5, 4 (2012), 275-291. [cited by applicant]
Esra Erdem, Michael Gelfond, and Nicola Leone. 2016. Applications of answer set programming. AI Magazine 37, 3 (2016), 53-68. [cited by applicant]
Esra Erdem and Umut Oztok. 2015. Generating explanations for biomedical queries. Theory and Practice of Logic Programming 15, 1 (2015), 35-78. [cited by applicant]
Esra Erdem and Ferhan Türe. 2008. Efficient Haplotype Inference with Answer Set Programming.. In AAAI, vol. 8. 436-441. [cited by applicant]
Chia-Hsin Huang, Po-Yi Yen, Yi-Chan Hung, Tyng-Ruey Chuang, and Hahn-Ming Lee. 2006. Enhancing Entropy-based Informative Block Identification Using Block Preclustering Technology. In 2006 IEEE International Conference o… [cited by applicant]
Salvatore Maria Ielpa, Salvatore Iiritano, Nicola Leone, and Francesco Ricca. 2009. An ASP-based system for e-tourism. In International Conference on Logic Programming and Nonmonotonic Reasoning. Springer, 368-381. [cited by applicant]
Said Jabbour, Lakhdar Sais, and Yakoub Salhi. 2015. Decomposition based SAT encodings for itemset mining problems. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 662-674. [cited by applicant]
Matti Järvisalo. 2011. Itemset mining as a challenge application for answer set enumeration. In International Conference on Logic Programming and Nonmonotonic Reasoning. Springer, 304-310. [cited by applicant]
Mohd Nizam Kassim, Mohd Aizaini Maarof, and Majid Bakhtiari. 2020. Fraudulent e-Commerce Website Detection Model Using HTML, Text and Image Features. In Proceedings of the 11th International Conference on Soft Computing… [cited by applicant]
Milos Kovacevic, Michelangelo Diligenti, Marco Gori, and Veljko Milutinovic. 2002. Recognition of common areas in a web page using visual information: a possible application in a page classification. In 2002 IEEE Intern… [cited by applicant]
Yuancheng Li and Jie Yang. 2009. A novel method to extract informative blocks from web pages. In 2009 International Joint Conference on Artificial Intelligence. IEEE, 536-539. [cited by applicant]
Yukun Li, Zhenguo Yang, Xu Chen, Huaping Yuan, and Wenyin Liu. 2019. A stacking model using URL and HTML features for phishing webpage detection. Future Generation Computer Systems 94 (2019), 27-39. [cited by applicant]
Seung-Jin Lim and Yiu-Kai Ng. 2001. An automated change-detection algorithm for HTML documents based on semantic hierarchies. In Proceedings 17th International Conference on Data Engineering. IEEE, 303-312. [cited by applicant]
Sonal Mahajan and William GJ Halfond. 2015. Detection and localization of html presentation failures using computer vision-based techniques. In 2015 IEEE 8th International Conference on Software Testing, Verification an… [cited by applicant]
Marco Manna, Francesco Ricca, and Giorgio Terracina. 2013. Consistent query answering via ASP from different perspectives: Theory and practice. Theory and Practice of Logic Programming 13, 2 (2013), 227-252. [cited by applicant]
Siegfried Nijssen and Tias Guns. 2010. Integrating constraint programming and itemset mining. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 467-482. [cited by applicant]
Francesco Ricca, Antonella Dimasi, Giovanni Grasso, Salvatore Maria Ielpa, Salvatore Iiritano, Marco Manna, and Nicola Leone. 2010. A logic-based system for e-tourism. Fundamenta Informaticae 105, 1-2 (2010), 35-55. [cited by applicant]
Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016). [cited by applicant]
Torsten Schaub and Stefan Woltran. 2018. Special issue on answer set programming. KI—Künstliche Intelligenz 32, 2 (2018), 101-103. [cited by applicant]
Sanjeev Kumar Singh. 2014. Identifying Webpage Regions and Their Roles by Combining Image Processing and Markup Analysis. Ph. D. Dissertation. [cited by applicant]
Erdinç Uzun, Hayri Volkan Agun, and Tarik Yerlikaya. 2013. A hybrid approach for extracting informative content from web pages. Information Processing & Management 49, 4 (2013), 928-944. [cited by applicant]
Yi Wang, Joohyung Lee, and Doo Soon Kim. 2017. A logic based approach to answering questions about alternatives in diy domains. In Twenty-Ninth IAAI Conference. [cited by applicant]
Tim Weninger, William H Hsu, and Jiawei Han. 2010. CETR: content extraction via tag ratios. In Proceedings of the 19th international conference on World wide web. 971-980. [cited by applicant]
Yudong Yang, Yu Chen, and HongJiang Zhang. 2003. HTML page analysis based on visual cues. In Web Document Analysis: Challenges and Opportunities. World Scientific, 113-131. [cited by applicant]
Lan Yi and Bing Liu. 2003. Web page cleaning for web mining through feature weighting. In IJCAI, vol. 3. Citeseer, 43-48. [cited by applicant]
Lan Yi, Bing Liu, and Xiaoli Li. 2003. Eliminating noisy information in web pages for data mining. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. 296-305. [cited by applicant]