IP Library › Granted Patent US 12688423
Granted Patent B2
US 12688423 · App. 18/905,761 · Granted Jul 21, 2026

Machine learning model compression

Inventors: Romain Cosentino (San Francisco, CA); Sarath Shekkizhar (San Francisco, CA); Damjan Kalajdzievski (San Francisco, CA); Adam Earle (San Francisco, CA)
Assignee: Salesforce, Inc.
G06N3/082G06N3/0495
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688423
App. No.
18/905,761
Granted
Jul 21, 2026
Kind
B2
Abstract

Techniques are described herein for a method of machine learning model compression. The method includes receiving a machine learning model comprising a plurality of blocks. The method further includes removing one or more blocks of the plurality of blocks to obtain an intermediate machine learning model comprising a subset of the plurality of blocks. The method further includes adding a block to the intermediate machine learning model to obtain a compressed machine learning model. The block generates an output corresponding to an output of the removed one or more blocks of the plurality of blocks. The method further includes executing the compressed machine learning model on a low resource device.

Claims (44)

1 . A method comprising:

receiving a machine learning model comprising a plurality of blocks that each comprise a respective stack of sequential layers configured to perform a task in the machine learning model;

removing one or more blocks of the plurality of blocks to obtain an intermediate machine learning model comprising a subset of the plurality of blocks;

adding an approximated block to the intermediate machine learning model to obtain a compressed machine learning model, wherein the approximated block is configured to approximate operation of the removed one or more blocks of the plurality of blocks; and

executing the compressed machine learning model on a low resource device, wherein the executing comprises:

predicting, by the approximated block, a domain-specific region.

2 . The method of claim 1 , wherein the removed one or more blocks are sequential blocks of the plurality of blocks.

3 . The method of claim 1 , wherein the executing comprises:

predicting, by the approximated block, a coordinate in an embedding space, wherein the coordinate is based on a sequence of inputs of the machine learning model.

4 . The method of claim 1 , wherein the executing comprises:

providing, to the compressed machine learning model, a domain-specific query; and

generating, by the compressed machine learning model, domain-specific natural language text responsive to the domain-specific query.

5 . The method of claim 1 , wherein the executing comprises:

receiving, by the approximated block, an embedding of a token;

generating, by the approximated block, an approximated output that approximates an output that would have been generated by the removed one or more blocks of the plurality of blocks, wherein the approximated output is an embedding of a next token.

6 . The method of claim 1 , wherein the removed one or more blocks are two or more blocks of the plurality of blocks.

7 . A non-transitory computer-readable medium storing executable instructions, which when executed by a computing device, cause the computing device to perform operations comprising:

receiving a machine learning model comprising a plurality of blocks that each comprise a respective stack of sequential layers configured to perform a task in the machine learning model;

removing one or more blocks of the plurality of blocks to obtain an intermediate machine learning model comprising a subset of the plurality of blocks;

adding an approximated block to the intermediate machine learning model to obtain a compressed machine learning model, wherein the approximated block is configured to approximate operation of the removed one or more blocks of the plurality of blocks, and wherein the approximated block is configured to predict a domain-specific region; and

providing the compressed machine learning model for execution by a low resource device.

8 . The non-transitory computer-readable medium of claim 7 , wherein the removed one or more blocks are sequential blocks of the plurality of blocks.

9 . The non-transitory computer-readable medium of claim 7 , wherein the approximated block is configured to predict a coordinate in an embedding space, wherein the coordinate is based on a sequence of inputs of the machine learning model.

10 . The non-transitory computer-readable medium of claim 7 , wherein execution of the compressed machine learning model on the low resource device further comprises:

providing, to the compressed machine learning model, a domain-specific query; and

generating, by the compressed machine learning model, domain-specific natural language text responsive to the domain-specific query.

11 . The non-transitory computer-readable medium of claim 7 , the operations further comprising:

receiving, by the approximated block, an embedding of a token;

generating, by the approximated block, and approximated output that approximates an output that would have been generated by the removed one or more blocks of the plurality of blocks, wherein the approximated output is an embedding of a next token.

12 . The non-transitory computer-readable medium of claim 7 , wherein the removed one or more blocks are two or more blocks of the plurality of blocks.

13 . A system comprising:

a computing device configured to perform operations comprising:

receiving a machine learning model comprising a plurality of blocks that each comprise a respective stack of sequential layers configured to perform a task in the machine learning model;

removing one or more blocks of the plurality of blocks to obtain an intermediate machine learning model comprising a subset of the plurality of blocks; and

adding an approximated block to the intermediate machine learning model to obtain a compressed machine learning model, wherein the approximated block is configured to approximate operation of the removed one or more blocks of the plurality of blocks, and wherein the approximated block is configured to predict a domain-specific region; and

a low resource device configured to receive and execute the compressed machine learning model.

14 . The system of claim 13 , wherein the removed one or more blocks are sequential blocks of the plurality of blocks.

15 . The system of claim 13 , wherein the approximated block is configured to predict a coordinate of the output in an embedding space, wherein the coordinate is based on a sequence of inputs of the machine learning model.

16 . The system of claim 13 , wherein execution of the compressed machine learning model on the low resource device comprises:

providing, to the compressed machine learning model, a domain-specific query; and

generating, by the compressed machine learning model, domain-specific natural language text responsive to the domain-specific query.

17 . The system of claim 13 , wherein execution of the compressed machine learning model on the low resource device comprises:

receiving, by the approximated block, an embedding of a token;

generating, by the approximated block, an approximated output that approximates an output that would have been generated by the removed one or more blocks of the plurality of blocks, wherein the approximated output is an embedding of a next token.