IP Library › Granted Patent US 12,282,755
Granted Patent B2
US 12,282,755 · App. 18/460,558 · Granted Apr 22, 2025

Generation of user interfaces from free text

Inventors: Nikolas Louis Ciminelli (Buffalo, NY); Tom Zeev Jacob Palny (Tel Aviv-Yaffo, IL); Ron Zass (Kiryat Tivon, IL)
G06F8/38G06F3/04842G06F3/04845G06F3/04847G06F3/0487G06F8/71G06F9/451G06F40/253G06F40/30G06F40/40G06F40/58G06T11/60G06V30/422
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,282,755
App. No.
18/460,558
Granted
Apr 22, 2025
Kind
B2
Abstract

Systems, methods and non-transitory computer readable media for generation of user interfaces from free text are provided. In some examples, a textual input in a natural language and/or a sketch may be received. The textual input and/or the sketch may be associated with a desire of an individual to create a new user interface. Further, the textual input and/or the sketch may be analyzed to obtain at least two user interface elements. Further, the textual and/or the sketch input may be analyzed to obtain a layout for the at least two user interface elements. Further, a particular draft of the new user interface that includes the at least two user interface elements based on the layout may be generated. Further, the particular draft of the new user interface may be presented to the individual in a form that enables the individual to modify the new user interface.

Claims (65)

1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generation of user interfaces from free text, the operations comprising:

receiving a textual input in a natural language, the textual input is associated with a desire of an individual to create a new user interface;

analyzing at least one other user interface to determine a particular design style;

analyzing the textual input to obtain at least two user interface elements;

analyzing the textual input to obtain a layout for the at least two user interface elements based on the particular design style determined by analyzing the at least one other user interface;

generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and

presenting the particular draft of the new user interface to the individual in a form that enables the individual to modify the new user interface.

2. The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

generating a plurality of alternative drafts of the new user interface;

presenting the plurality of alternative drafts of the new user interface to the individual; and

receiving from the individual a selection of the particular draft of the new user interface of the plurality of alternative drafts of the new user interface.

3. The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

after the presenting the particular draft of the new user interface to the individual, receiving an additional textual input from the individual; and

analyzing the additional textual input to modify the new user interface.

4. The non-transitory computer readable medium of claim 1 , wherein the presentation of the particular draft of the new user interface to the individual includes a visual presentation of the particular draft of the new user interface and a source code associated with the particular draft of the new user interface.

5. The non-transitory computer readable medium of claim 1 , wherein the presentation of the particular draft of the new user interface to the individual enables the individual to select portions of the particular draft of the new user interface, and the operations further comprise:

receiving a pointing input from the individual via a pointing device, the pointing input is indicative of a particular position in the presentation of the particular draft of the user interface;

using the particular position to select a particular portion of the particular draft of the new user interface;

receiving from the individual a second textual input in the natural language, the second textual input is associated with a desire of the individual to modify the particular portion of the particular draft;

analyzing the second textual input to determine a modification to the particular portion of the particular draft;

based on the determined modification, generating a specific draft of the new user interface, the specific draft differs from the particular draft at least in the particular portion and identical to the particular draft at least in one other portion; and

presenting the specific draft of the new user interface to the individual in a form that enables the individual to modify the new user interface.

6. The non-transitory computer readable medium of claim 1 , wherein the at least two user interface elements includes at least one image, and the operations further comprise analyzing the textual input to determine the at least one image.

7. The non-transitory computer readable medium of claim 6 , wherein the operations further comprise:

identifying a first mathematical object in a mathematical space, the first mathematical object corresponds to a first word of the textual input;

identifying a second mathematical object in the mathematical space, the second mathematical object corresponds to a second word of the textual input;

calculating a function of the first mathematical object and the second mathematical object to obtain a third mathematical object in the mathematical space; and

generating the at least one image based on the third mathematical object.

8. The non-transitory computer readable medium of claim 1 , wherein the operations further comprise analyzing the textual input to determine background music for the new user interface, and wherein the generation of the particular draft of the new user interface is further based on the background music.

9. The non-transitory computer readable medium of claim 1 , wherein the textual input includes an adjective, and the operations further comprise determining a characteristic of a particular user interface element in the new user interface based on the adjective.

10. The non-transitory computer readable medium of claim 9 , wherein the adjective is an adjective that describes texture, and the characteristic of the particular user interface element is a texture associated with at least part of the particular user interface element.

11. The non-transitory computer readable medium of claim 1 , wherein the textual input includes an adverb, and the operations further comprise determining a characteristic of a particular user interface element in the new user interface based on the adverb.

12. The non-transitory computer readable medium of claim 11 , wherein the particular user interface element is designed to move, the adverb is an adverb that describes motion, and the characteristic of the particular user interface element is a movement pattern associated with the particular user interface element.

13. The non-transitory computer readable medium of claim 1 , wherein the layout is based on a characteristic of a target audience, and wherein the operations further comprise analyzing historic activities of individuals associated with the target audience to determine the characteristic of the target audience.

14. The non-transitory computer readable medium of claim 1 , wherein the at least one other user interface is at least one user interface associated with a specific brand.

15. The non-transitory computer readable medium of claim 1 , wherein the textual input is indicative of an occasion, and the layout is based on the occasion.

16. The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

receiving a sketch, the sketch is associated with the desire of the individual to create the new user interface; and

analyzing the textual input and the sketch to obtain the layout.

17. The non-transitory computer readable medium of claim 16 , wherein the sketch includes a plurality of pixel values, and the operations further comprise:

calculating a convolution of at least part of the pixel values of the sketch to obtain a result value;

identifying a first mathematical object in the mathematical space, the first mathematical object corresponds to a word of the textual input;

calculating a function of the result value and the first mathematical object to obtain a second mathematical object in the mathematical space; and

obtaining the layout based on the second mathematical object.

18. The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

identifying a first mathematical object in a mathematical space, the first mathematical object corresponds to a first word of the textual input;

identifying a second mathematical object in the mathematical space, the second mathematical object corresponds to a second word of the textual input;

calculating a function of the first mathematical object and the second mathematical object to obtain a third mathematical object in the mathematical space; and

obtaining the layout based on the third mathematical object.

19. The non-transitory computer readable medium of claim 1 , wherein the at least two user interface elements includes at least one input element designed to enable future users of the new user interface to select an option of a plurality of alternatives, and wherein the operations further comprise analyzing the textual input to determine at least one alternative of the plurality of alternatives.

20. The non-transitory computer readable medium of claim 1 , wherein the at least two user interface elements includes at least one textual element in a second natural language different from the natural language, the at least one textual element includes at least one word not included in the textual input, and the operations further comprise analyzing the textual input to generate the at least one textual element.

21. A system for generation of user interfaces from free text, the system comprising at least one processing unit configured to perform operations, the operations comprise:

receiving a textual input in a natural language, the textual input is associated with a desire of an individual to create a new user interface;

analyzing at least one other user interface to determine a particular design style;

analyzing the textual input to obtain at least two user interface elements;

analyzing the textual input to obtain a layout for the at least two user interface elements based on the particular design style determined by analyzing the at least one other user interface;

generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and

presenting the particular draft of the new user interface to the individual in a form that enables the individual to modify the new user interface.

22. A method for generation of user interfaces from free text, the method comprising:

receiving a textual input in a natural language, the textual input is associated with a desire of an individual to create a new user interface;

analyzing at least one other user interface to determine a particular design style;

analyzing the textual input to obtain at least two user interface elements;

analyzing the textual input to obtain a layout for the at least two user interface elements based on the particular design style determined by analyzing the at least one other user interface;

generating a particular draft of the new user interface that includes the at least two user interface elements based on the layout; and

presenting the particular draft of the new user interface to the individual in a form that enables the individual to modify the new user interface.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2023
From: PALNY, TOM ZEEV JACOB; CIMINELLI, NIKOLAS LOUIS; ZASS, RON
To: TWEAK COMMERCE INC.
Reel/Frame 065485/0695 →
Continuity (8)
Provisional Application 63534747 · Aug 25, 2023
Provisional Application 63444841 · Feb 10, 2023
Provisional Application 63441097 · Jan 25, 2023
Provisional Application 63439080 · Jan 14, 2023
Provisional Application 63436639 · Jan 2, 2023
Provisional Application 63426748 · Nov 19, 2022
Provisional Application 63405417 · Sep 10, 2022
Related Publication 20230409298A1 · Dec 21, 2023
References Cited (127)
US 4305131A · Best · 1981 [cited by applicant]
US 6077085A · Parry · 2000 [cited by applicant]
US 6778252B2 · Moulton et al. · 2004 [cited by applicant]
US 8140322B2 · Simonsen et al. · 2012 [cited by applicant]
US 9280973B1 · Soyannwo et al. · 2016 [cited by applicant]
US 9747282B1 · Baker et al. · 2017 [cited by applicant]
US 9864933B1 · Cosic · 2018 [cited by applicant]
US 10360716B1 · van der Meulen et al. · 2019 [cited by applicant]
US 10423999B1 · Doctor · 2019 [cited by applicant]
US 10467792B1 · Roche · 2019 [cited by applicant]
US 10607134B1 · Cosic · 2020 [cited by applicant]
US 10827024B1 · Schissel et al. · 2020 [cited by applicant]
US 11024194B1 · Beigman Klebanov · 2021 [cited by applicant]
US 11140459B2 · Ingel · 2021 [cited by applicant]
US 11159597B2 · Ingel · 2021 [cited by applicant]
US 11202131B2 · Zass · 2021 [cited by applicant]
US 11232645B1 · Roche et al. · 2022 [cited by applicant]
US 11244385B1 · Fraser · 2022 [cited by applicant]
US 11520079B2 · Zass · 2022 [cited by applicant]
US 11966688B1 · Ehrlich · 2024 [cited by applicant]
US 20020087317A1 · Lee et al. · 2002 [cited by applicant]
US 20020161578A1 · Saindon et al. · 2002 [cited by applicant]
US 20020161579A1 · Saindon et al. · 2002 [cited by applicant]
US 20040068410A1 · Mohamed et al. · 2004 [cited by applicant]
US 20040172257A1 · Liqin et al. · 2004 [cited by applicant]
US 20040186712A1 · Coles et al. · 2004 [cited by applicant]
US 20050255431A1 · Baker · 2005 [cited by applicant]
US 20050272013A1 · Knight · 2005 [cited by applicant]
US 20060285654A1 · Nesvadba et al. · 2006 [cited by applicant]
US 20070124166A1 · Van Luchene · 2007 [cited by applicant]
US 20070130529A1 · Shrubsole · 2007 [cited by examiner]
US 20070208569A1 · Subramanian et al. · 2007 [cited by applicant]
US 20070220575A1 · Cooper et al. · 2007 [cited by applicant]
US 20080015968A1 · Van Luchene et al. · 2008 [cited by applicant]
US 20080195386A1 · Proidl · 2008 [cited by applicant]
US 20080295130A1 · Worthen · 2008 [cited by applicant]
US 20090037179A1 · Liu et al. · 2009 [cited by applicant]
US 20090175596A1 · Hirai · 2009 [cited by applicant]
US 20100082326A1 · Bangalore · 2010 [cited by applicant]
US 20100100907A1 · Chang · 2010 [cited by applicant]
US 20100238179A1 · Kelly · 2010 [cited by applicant]
US 20110064388A1 · Brown et al. · 2011 [cited by applicant]
US 20110076992A1 · Chou · 2011 [cited by applicant]
US 20120054619A1 · Spooner et al. · 2012 [cited by applicant]
US 20130038737A1 · Yehezkel · 2013 [cited by examiner]
US 20130110513A1 · Jhunja et al. · 2013 [cited by applicant]
US 20130188862A1 · Lievens · 2013 [cited by applicant]
US 20140142918A1 · Dotterer et al. · 2014 [cited by applicant]
US 20140164507A1 · Tesch et al. · 2014 [cited by applicant]
US 20140303958A1 · Lee et al. · 2014 [cited by applicant]
US 20140358518A1 · Wu et al. · 2014 [cited by applicant]
US 20150092007A1 · Koborita et al. · 2015 [cited by applicant]
US 20150356987A1 · Byron et al. · 2015 [cited by applicant]
US 20150319518A1 · Wilson · 2015 [cited by applicant]
US 20160005436A1 · Axen et al. · 2016 [cited by applicant]
US 20160021334A1 · Rossano et al. · 2016 [cited by applicant]
US 20160042766A1 · Kummer · 2016 [cited by applicant]
US 20160049146A1 · Chang · 2016 [cited by applicant]
US 20160132578A1 · Allen · 2016 [cited by applicant]
US 20160254795A1 · Ballard · 2016 [cited by applicant]
US 20160328391A1 · Choi · 2016 [cited by applicant]
US 20160365087A1 · Freud · 2016 [cited by applicant]
US 20170004820A1 · Lv et al. · 2017 [cited by applicant]
US 20170011745A1 · Navaratnam · 2017 [cited by applicant]
US 20170075877A1 · Lepeltier · 2017 [cited by applicant]
US 20170076749A1 · Kanevsky · 2017 [cited by applicant]
US 20170255616A1 · Yun et al. · 2017 [cited by applicant]
US 20180143809A1 · Zang · 2018 [cited by examiner]
US 20180174577A1 · Jothilingam et al. · 2018 [cited by applicant]
US 20180174595A1 · Dirac et al. · 2018 [cited by applicant]
US 20180253992A1 · Koul et al. · 2018 [cited by applicant]
US 20180260448A1 · Osotio et al. · 2018 [cited by applicant]
US 20180322875A1 · Adachi · 2018 [cited by applicant]
US 20180357215A1 · Austin et al. · 2018 [cited by applicant]
US 20180374461A1 · Serletic · 2018 [cited by applicant]
US 20190065478A1 · Tsujikawa et al. · 2019 [cited by applicant]
US 20190114302A1 · Bequet · 2019 [cited by examiner]
US 20190164533A1 · Lawrenson et al. · 2019 [cited by applicant]
US 20190166176A1 · Jain et al. · 2019 [cited by applicant]
US 20190250891A1 · Kumar · 2019 [cited by examiner]
US 20190317739A1 · Turek · 2019 [cited by examiner]
US 20190354592A1 · Musham et al. · 2019 [cited by applicant]
US 20200005796A1 · Ziv et al. · 2020 [cited by applicant]
US 20200007946A1 · Olkha · 2020 [cited by applicant]
US 20200042601A1 · Doggett · 2020 [cited by applicant]
US 20200043112A1 · Brinkley, II · 2020 [cited by applicant]
US 20200051566A1 · Shin · 2020 [cited by applicant]
US 20200058289A1 · Gabryjelski et al. · 2020 [cited by applicant]
US 20200066304A1 · Chen · 2020 [cited by applicant]
US 20200105245A1 · Gupta · 2020 [cited by applicant]
US 20200111474A1 · Kumar et al. · 2020 [cited by applicant]
US 20200143813A1 · Nakagawa · 2020 [cited by applicant]
US 20200150981A1 · Westberg · 2020 [cited by examiner]
US 20200159862A1 · Kleiner et al. · 2020 [cited by applicant]
US 20200174776A1 · Vinod · 2020 [cited by examiner]
US 20200221176A1 · Hwang et al. · 2020 [cited by applicant]
US 20200296510A1 · Li et al. · 2020 [cited by applicant]
US 20200311120A1 · Zhao et al. · 2020 [cited by applicant]
US 20210019373A1 · Freitag · 2021 [cited by applicant]
US 20210042110A1 · Basyrov · 2021 [cited by examiner]
US 20210063383A1 · Kaminski et al. · 2021 [cited by applicant]
US 20210081101A1 · Speare et al. · 2021 [cited by applicant]
US 20210097976A1 · Chicote et al. · 2021 [cited by applicant]
US 20210264369A1 · Zass · 2021 [cited by applicant]
US 20210279822A1 · Bellaish · 2021 [cited by applicant]
US 20210182468A1 · Co · 2021 [cited by examiner]
US 20210192824A1 · Chen · 2021 [cited by applicant]
US 20210287150A1 · Zass · 2021 [cited by applicant]
US 20210224319A1 · Ingel · 2021 [cited by applicant]
US 20210225365A1 · Sinha et al. · 2021 [cited by applicant]
US 20210232759A1 · Schick et al. · 2021 [cited by applicant]
US 20210271815A1 · Li · 2021 [cited by examiner]
US 20210303318A1 · Raghavan · 2021 [cited by examiner]
US 20210397418A1 · Nikumb · 2021 [cited by examiner]
US 20210400101A1 · Ingel · 2021 [cited by applicant]
US 20220070550A1 · Ingel · 2022 [cited by applicant]
US 20220382524A1 · Ansari · 2022 [cited by examiner]
US 20230048149A1 · Zass · 2023 [cited by applicant]
US 20230049015A1 · Zass · 2023 [cited by applicant]
US 20230052442A1 · Zass · 2023 [cited by applicant]
US 20230057835A1 · Zass · 2023 [cited by applicant]
US 20230069088A1 · Zass · 2023 [cited by applicant]
US 20230095089A1 · Kaliyaperumal · 2023 [cited by examiner]
US 20230115185A1 · Huang · 2023 [cited by examiner]
US 20230252224A1 · Tran · 2023 [cited by applicant]
US 20240220521A1 · Ehrlich · 2024 [cited by applicant]
US 20240220714A1 · Ehrlich · 2024 [cited by applicant]