Granted Patent B2

US 12,579,066 · App. 18/731,851 · Granted Mar 17, 2026

Inventors: Aneesh Dahiya (Zurich, CH); Renata Khasanova (Zurich, CH)

Assignee: Oracle International Corporation

G06F12/0802

Abstract

Claims (50)

1 . A computer-implemented method comprising:

receiving an input for a deterministic model, the input comprising data from a data stream, the input having an index value;

in accordance with the input not being stored in a cache:

computing an output from the model based on the input,

storing the input and the output in the cache as an element of the cache, and

in accordance with the index value being at a limit value:

using an optimization algorithm to determine a set of parameters associated with the cache, wherein the algorithm optimizes a function including a time taken by the model to generate outputs from a set of inputs sampled from the data stream;

calculating a caching score associated with each element of the cache, wherein the caching score for an element comprises a sum of a first product of a first score related to the element and a first parameter of the set of parameters, a second product of a second score related to the element and a second parameter of the set of parameters, and a third product of a third score related to the element and a third parameter of the set of parameters; and

subsequently, in accordance with receiving an additional input not stored in the cache, removing from the cache the element having the lowest caching score.

2 . The computer-implemented method of claim 1 , wherein the algorithm comprises a Bayesian optimization algorithm.

3 . The computer-implemented method of claim 1 , wherein elements of the cache have a priority order according to the time taken by the model to generate the output from the input.

4 . The computer-implemented method of claim 1 , wherein the cache comprises a plurality of elements each comprising an input-output pair, wherein each input-output pair in the plurality of elements is stored in a hashmap.

5 . The computer-implemented method of claim 4 , wherein each input-output pair comprises a key value pair stored in the hashmap.

6 . The computer-implemented method of claim 1 , further comprising calculating a first normalized score, a second normalized score and a third normalized score based on the first score, the second score and the third score respectively.

7 . The computer-implemented method of claim 6 , wherein the calculating comprises a min-max normalization procedure.

8 . The computer-implemented method of claim 1 , wherein the first, second, third scores respectively correspond to the time taken by the model to generate the output, a usage of the element expressed as a number of iterations over which the element has been retained in the cache, and a frequency of usage of the element.

9 . A non-transitory computer-readable medium comprising instructions executable by a processor to:

receive an input for a deterministic model, the input comprising data from a data stream, the input having an index value;

determining whether the input is stored in a cache;

in accordance with the input not being stored in the cache:

compute an output from the model based on the input,

store the input and the output in the cache as an element of the cache, and

determine whether the index value is at a limit value;

in accordance with the index value being at the limit value:

use an algorithm to determine a set of parameters associated with the cache, wherein the algorithm optimizes a function including a time taken by the model to generate outputs from a set of inputs sampled from the data stream;

calculate a caching score associated with each element of the cache, wherein the caching score for an element comprises a sum of a first product of a first score related to the element and a first parameter of the set of parameters, a second product of a second score related to the element and a second parameter of the set of parameters, and a third product of a third score related to the element and a third parameter of the set of parameters; and

subsequently, in accordance with receiving an additional input not stored in the cache, remove from the cache the element having the lowest caching score.

10 . The non-transitory computer-readable medium of claim 9 , wherein the algorithm comprises a Bayesian optimization algorithm.

11 . The non-transitory computer-readable medium of claim 9 , further comprising instructions executable by the processor to:

store the input and the output in the cache, in accordance with the input not being stored in the cache and the index value being less than the limit value, the elements of the cache having a priority order according to the time taken by the model to generate the output from the input.

12 . The non-transitory computer-readable medium of claim 9 , wherein the cache comprises a plurality of elements each comprising an input-output pair, wherein each input-output pair in the plurality of elements is stored in a hashmap.

13 . The non-transitory computer-readable medium of claim 9 , further comprising instructions executable by the processor to calculate a first normalized score, a second normalized score and a third normalized score based on the first score, the second score and the third score respectively.

14 . The non-transitory computer-readable medium of claim 9 , wherein the first, second, third scores respectively correspond to the time taken by the model to generate the output, a usage of the element expressed as a number of iterations over which the element has been retained in the cache, and a frequency of usage of the element.

15 . A system comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:

receiving an input for a model, the input comprising data from a data stream, the input having an index value;

determining whether the input is stored in a cache;

in accordance with the input not being stored in the cache:

computing an output from the model based on the input,

storing the input and the output in the cache as an element of the cache, and

determining whether the index value is at a limit value;

in accordance with the index value being at the limit value:

using an algorithm to determine a set of parameters associated with the cache, wherein the algorithm optimizes a function including a time taken by the model to generate outputs from a set of inputs sampled from the data stream;

calculating a caching score associated with each element of the cache, wherein the caching score for an element comprises a sum of a first product of a first score related to the element and a first parameter of the set of parameters, a second product of a second score related to the element and a second parameter of the set of parameters, and a third product of a third score related to the element and a third parameter of the set of parameters; and

subsequently, in accordance with receiving an additional input not stored in the cache, removing from the cache the element having the lowest caching score.

16 . The system of claim 15 , wherein the algorithm comprises a Bayesian optimization algorithm.

17 . The system of claim 15 , wherein the cache comprises a plurality of elements each comprising an input-output pair, wherein each input-output pair in the plurality of elements is stored in a hashmap.

18 . The system of claim 15 , further comprising calculating a first normalized score, a second normalized score and a third normalized score based on the first score, the second score and the third score respectively.

19 . The system of claim 15 , wherein the first, second, third scores respectively correspond to the time taken by the model to generate the output, a usage of the element expressed as a number of iterations over which the element has been retained in the cache, and a frequency of usage of the element.

Assignments (1)

Continuity (1)

References Cited (23)