IP Library › Granted Patent US 12,393,504

Granted Patent B2

US 12,393,504 · App. 18/628,773 · Granted Aug 19, 2025

Automated program repair tool

Inventors: Shao Kun Deng (Seattle, WA); Neelakantan Sundaresan (Bellevue, WA); Alexey Svyatkovskiy (Bellevue, WA); Michele Tufano (Bellevue, WA)

Assignee: Microsoft Technology Licensing, LLC

G06F11/362 G06N20/00

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12,393,504
App. No.: 18/628,773
Granted: Aug 19, 2025
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

An automated program repair tool utilizes a neural transformer model with attention to predict the contents of a bug repair in the context of source code having a bug of an identified bug type. The neural transformer model is trained on a large unsupervised corpus of source code using a span-masking denoising optimization objective, and fine-tuned on a large supervised dataset of triplets containing a bug-type annotation, software bug, and repair. The bug-type annotation is derived from an interprocedural static code analyzer. A bug type edit centroid is computed for each bug type and used in the inference decoding phase to generate the bug repair.

Claims (49)

1. A method for automated program repair, the method comprising:

training a neural transformer model with attention with a dataset to predict a repaired source code for a given source code snippet and a given bug type;

prior to training the neural transformer model with attention, pre-training the neural transformer model with attention on unsupervised source code snippets;

identifying a bug type for a source code bug of a code snippet;

obtaining a bug-type edit centroid for the bug type of the code snippet, wherein the bug-type edit centroid represents edits to fix the bug type;

generating a bug repair for the code snippet from the neural transformer model with attention given the bug-type edit centroid, wherein the neural transformer model is trained to predict an encoding for each bug type of the training dataset and transform the encoding of each bug type into the bug-type edit centroid; and

generating a repaired source code for the code snippet based at least in part on the bug repair.

2. The method of claim 1 , further comprising:

wherein the training dataset comprising a plurality of triplets each comprising a source code snippet with a software bug, a repaired source code, and a bug type of the software bug.

3. The method of claim 1 , wherein the neural transformer model with attention includes an encoder block to generate an edit embedding for the bug-type edit centroid.

4. The method of claim 3 , further comprising:

deploying the neural transformer model with attention in an inference system to predict repaired code for a given code snippet having a software bug of a specified bug type, wherein the encoder block receives the bug-type edit centroid of the specified bug type.

5. The method of claim 1 , further comprising:

forming a cluster for each bug type, wherein the cluster includes encodings having a same bug type; and

computing the bug-type edit centroid for each cluster from the encodings of each bug type in a respective cluster.

6. The method of claim 1 , wherein the repaired source code comprises a sequence of source code tokens.

7. A system for automated program repair, the system comprising:

a processor; and

a memory including instructions, which when executed by the processor, cause the processor to perform operations comprising:

training the neural transformer model with attention with a dataset to predict a repaired source code for a given source code snippet and a given bug type;

prior to training the neural transformer model with attention, pre-training the neural transformer model with attention on unsupervised source code snippets;

identifying a bug type for a source code bug of a code snippet;

obtaining a bug-type edit centroid for the bug type of the code snippet, wherein the bug-type edit centroid represents edits to fix the bug type;

generating a repaired source code for the code snippet based at least in part on the bug repair.

8. The system of claim 7 , wherein the operations further comprise:

wherein the training dataset comprising a plurality of triplets each comprising a source code snippet with a software bug, a repaired source code, and a bug type of the software bug.

9. The system of claim 7 , wherein the neural transformer model with attention includes an encoder block to generate an edit embedding for the bug-type edit centroid.

10. The system of claim 9 , wherein the operations further comprise:

11. The system of claim 7 , wherein the operations further comprise:

forming a cluster for each bug type, wherein the cluster includes encodings having a same bug type; and

computing the bug-type edit centroid for each cluster from the encodings of each bug type in a respective cluster.

12. The system of claim 7 , wherein the repaired source code comprises a sequence of source code tokens.

13. A non-transitory computer-readable medium including instructions, which when executed by a processor, cause the processor to perform operations comprising:

training a neural transformer model with attention with a dataset to predict a repaired source code for a given source code snippet and a given bug type;

prior to training the neural transformer model with attention, pre-training the neural transformer model with attention on unsupervised source code snippets;

identifying a bug type for a source code bug of a code snippet;

obtaining a bug-type edit centroid for the bug type of the code snippet, wherein the bug-type edit centroid represents edits to fix the bug type;

generating a repaired source code for the code snippet based at least in part on the bug repair.

14. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:

wherein the training dataset comprising a plurality of triplets each comprising a source code snippet with a software bug, a repaired source code, and a bug type of the software bug.

15. The non-transitory computer-readable medium of claim 13 , wherein the neural transformer model with attention includes an encoder block to generate an edit embedding for the bug-type edit centroid.

16. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:

17. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise:

forming a cluster for each bug type, wherein the cluster includes encodings having a same bug type; and

computing the bug-type edit centroid for each cluster from the encodings of each bug type in a respective cluster.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2024

From: DENG, SHAO KUN; SUNDARESAN, NEELAKANTAN; SVYATKOVSKIY, ALEXEY; TUFANO, MICHELE

To: MICROSOFT TECHNOLOGY LICENSING, LLC

Reel/Frame 069507/0764 →

Continuity (4)

Continuation 17994185 · Nov 25, 2022

Continuation 16897824 · Jun 10, 2020

Provisional Application 63025535 · May 15, 2020

Related Publication 20240370352A1 · Nov 7, 2024

References Cited (6)

US 11593675B1 · Garg · 2023 [cited by examiner]

US 20070022321A1 · Chua · 2007 [cited by examiner]

US 20210182077A1 · Chen · 2021 [cited by examiner]

US 20210311853A1 · Shao · 2021 [cited by examiner]

Md. Mostafizer Rahman et al. “Source Code Assessment and Classification Based on Estimated Error Probability Using Attentive LSTM Language Model and Its Application in Programming Education”, [Online], pp. 1-21, [Retriv… [cited by examiner]

Communication pursuant to Article 94(3) received in European Application No. 21720955.0, mailed on May 13, 2024, 12 pages. [cited by applicant]