IP Library › Granted Patent US 12694019

Granted Patent B1

US 12694019 · App. 19/040,650 · Granted Jul 28, 2026

Form conversion using preference alignment training

Inventors: Thanh Tien Vu (Herston, AU); Prashan Madumal (Melbourne, AU); Omid Mohamad Nezami (Sydney, AU); Cong Duy Vu Hoang (Wantirna South, AU); Gioacchino Tangari (Sydney, AU); Duy Vu (Melbourne, AU); Dai Quoc Nguyen (Raceview, AU); Yakupitiyage Don Thanuja Samodhye Dharmasiri (Melbourne, AU); Ying Xu (Melbourne, AU); Thanh Long Duong (Melbourne, AU); Clemence Andree Simone Lanfranchi (Zurich, CH); Damien Alexandre Hilloulin (Zurich, CH); Rhicheek Patra (Zurich, CH); Sungpack Hong (Palo Alto, CA); Hassan Chafi (Zurich, CH)

Assignee: Oracle International Corporation

G06F16/243 G06N3/0475 G06N3/0985

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12694019
App. No.: 19/040,650
Granted: Jul 28, 2026
Kind: B1

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Techniques are disclosed herein for automatically curating preference alignment training (PAT) data and subsequently using the PAT data to train a machine-learning model used to perform natural language-to-query language tasks. A first PAT dataset is generated using a first trained LLM in an execution-based data generation technique. A second PAT dataset is generated using a second trained LLM in a LLM-based data generation technique. The second PAT dataset is populated with synthetic data points generated by the second trained LLM. Each data point of first PAT dataset and the second PAT dataset comprises a natural language query and an executable and non-executable query language statement. The first PAT dataset and the second PAT dataset can be combined and filtered to generate a PAT dataset that is optimized for training a machine-learning model to overcome one or more weaknesses exhibited when generating query language statements responsive to natural language questions.

Claims (88)

1 . A computer-implemented method comprising:

accessing a set of data points of a training dataset, each data point of the set of data points comprising a prompt including a natural language query and a correct query language statement for the natural language query;

generating a first preference alignment training dataset by:

predicting, using a first trained large language model, candidate query language statements for the natural language queries of a first subset of data points of the set of data points,

determining that the candidate query language statements are not executable by submitting the candidate query language statements to a database system,

after determining that the candidate query language statements are not executable, producing an updated version of the first subset of data points by including the candidate query language statements in the first subset of data points as incorrect query language statements for the natural language queries of the first subset of data points, and

including the updated version of the first subset of data points in the first preference alignment training dataset;

generating a second preference alignment training dataset by:

causing a second trained large language model to generate synthetic natural language queries based on database schemas defined in the prompts of a second subset of data points of the set of data points;

causing the second trained large language model to predict, based on the synthetic natural language queries, executable synthetic query language statements for the synthetic natural language queries,

causing the second trained large language model to predict, based on the synthetic natural language queries, non-executable synthetic query language statements for the synthetic natural language queries,

generating a plurality of synthetic data points comprising the synthetic natural language queries, the executable synthetic query language statements, and the non- executable synthetic query language statements, and

including the plurality of synthetic data points in the second preference alignment training dataset; and

generating a preference alignment training dataset by combining the first preference alignment training dataset and the second preference alignment training dataset, wherein combining the first preference alignment training dataset and the second preference alignment training dataset comprises performing at least one filtering operation of a plurality of filtering operations on the first preference alignment training dataset and the second preference alignment training dataset.

2 . The method of claim 1 , further comprising:

accessing a machine-learning model;

using the preference alignment training dataset to generate a fine-tuned version of the machine-learning model; and

deploying the fine-tuned version of the machine-learning model to a system.

3 . The method of claim 1 , wherein performing the at least one filtering operation of the plurality of filtering operations comprises:

for each synthetic data point in the second preference alignment training dataset, determining whether the executable synthetic query language statement is executable and whether the non-executable synthetic query language statement is not executable by submitting the executable synthetic query language statement and the non-executable synthetic query language statement to the database system.

4 . The method of claim 3 , wherein performing the at least one filtering operation of the plurality of filtering operations further comprises:

generating a filtered second preference alignment training dataset by removing from the second preference alignment training dataset any synthetic data points that are determined to erroneously include an executable synthetic query language statement that is not executable or a non-executable synthetic query language statement that is executable.

5 . The method of claim 4 , wherein performing the at least one filtering operation of the plurality of filtering operations further comprises:

scoring, using at least one large language model-as-a-judge trained to grade query language statements, a quality of the correct query language statement and the incorrect query language statement of each data point of the updated version of the first subset of data points in the first preference alignment training dataset and a quality of the executable synthetic query language statement and the non-executable synthetic query language statement of each synthetic data point in the filtered second preference alignment training dataset, given the prompts associated therewith.

6 . The method of claim 5 , wherein performing the at least one filtering operation of the plurality of filtering operations further comprises:

optimizing the preference alignment training dataset to include only a plurality of the data points of the updated version of the first subset of data points in the first preference alignment training dataset having executable query language statements and non-executable query language statements receiving quality scores that meet predetermined threshold values, and a plurality of the synthetic data points of the second preference alignment training dataset having executable synthetic query language statements and non-executable synthetic query language statements receiving quality scores that meet predetermined threshold values.

7 . The method of claim 5 , wherein the at least one large language model-as-a-judge is a plurality of large language model judges and the plurality of large language model judges are used in one or both of two different configurations, comprising:

a panel configuration, where each large language model judge is assigned a different weight based on a strength of the large language model used as the judge; and

a serial configuration, where the plurality of large language model judges are arranged from weakest to strongest, and filtering occurs at each large language model judge in a serial manner.

8 . A system comprising:

one or more processors; and

one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:

generating a first preference alignment training dataset by:

predicting, using a first trained large language model, candidate query language statements for the natural language queries of a first subset of data points of the set of data points,

determining that the candidate query language statements are not executable by submitting the candidate query language statements to a database system,

including the updated version of the first subset of data points in the first preference alignment training dataset;

generating a second preference alignment training dataset by:

causing the second trained large language model to predict, based on the synthetic natural language queries, executable synthetic query language statements for the synthetic natural language queries,

including the plurality of synthetic data points in the second preference alignment training dataset; and

9 . The system of claim 8 , wherein the operations further comprise:

accessing a machine-learning model;

using the preference alignment training dataset to generate a fine-tuned version of the machine-learning model; and

deploying the fine-tuned version of the machine-learning model to a system.

10 . The system of claim 8 , wherein the performing the at least one filtering operation of the plurality of filtering operations comprises:

11 . The system of claim 10 , wherein the performing the at least one filtering operation of the plurality of filtering operations further comprises:

12 . The system of claim 11 , wherein the performing the at least one filtering operation of the plurality of filtering operations further comprises:

13 . The system of claim 12 , wherein the performing the at least one filtering operation of the plurality of filtering operations further comprises:

14 . The system of claim 12 , wherein the at least one large language model-as-a-judge is a plurality of large language model judges and the plurality of large language model judges are usable in one or both of two different configurations, comprising:

a panel configuration, where each large language model judge is assigned a different weight based on a strength of the large language model used as the judge; and

a serial configuration, where the plurality of large language model judges are arranged from weakest to strongest, and filtering occurs at each large language model judge in a serial manner.

15 . One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform operations comprising:

generating a first preference alignment training dataset by:

predicting, using a first trained large language model, candidate query language statements for the natural language queries of a first subset of data points of the set of data points,

determining that the candidate query language statements are not executable by submitting the candidate query language statements to a database system,

including the updated version of the first subset of data points in the first preference alignment training dataset;

generating a second preference alignment training dataset by:

causing the second trained large language model to predict, based on the synthetic natural language queries, executable synthetic query language statements for the synthetic natural language queries,

including the plurality of synthetic data points in the second preference alignment training dataset; and

16 . The one or more non-transitory computer-readable media of claim 15 , wherein the operations further comprise:

accessing a machine-learning model;

using the preference alignment training dataset to generate a fine-tuned version of the machine-learning model; and

deploying the fine-tuned version of the machine-learning model to a system.

17 . The one or more non-transitory computer-readable media of claim 15 , wherein the performing the at least one filtering operation of the plurality of filtering operations comprises:

18 . The one or more non-transitory computer-readable media of claim 17 , wherein the performing the at least one filtering operation of the plurality of filtering operations further comprises:

19 . The one or more non-transitory computer-readable media of claim 18 , wherein the performing the at least one filtering operation of the plurality of filtering operations further comprises:

20 . The one or more non-transitory computer-readable media of claim 18 , wherein the at least one large language model-as-a-judge is a plurality of large language model judges and the plurality of large language model judges are usable in one or both of two different configurations, comprising:

a panel configuration, where each large language model judge is assigned a different weight based on a strength of the large language model used as the judge; and

a serial configuration, where the plurality of large language model judges are arranged from weakest to strongest, and filtering occurs at each large language model judge in a serial manner.