Thread-shared software code caches
View Patent ↗A runtime system using thread-shared code caches is provided which avoids brute-force all-thread-suspension and monolithic global locks. In one embodiment, medium-grained runtime system synchronization reduces lock contention. The system includes trace building that combines efficient private construction with shared results, in-cache lock-free lookup table access in the presence of entry invalidations, and a delayed deletion algorithm based on timestamps and reference counts. These enable reductions in memory usage and performance overhead.
1. A computer implemented method for using a code cache, comprising:
caching code; and
sharing the cached code among multiple threads while avoiding brute-force all-thread suspension;
unlinking a block of code from the cached code to make it unreachable;
recording a first timestamp at a time of unlinking of the block of code;
using a plurality of second timestamps to determine whether the multiple threads have exited the cache since making the block of code unreachable;
determining that each thread in the cache has exited the cache since making the block of code unreachable; and
freeing the block of code based on the determination that each thread in the cache has exited the cache since making the block of code unreachable.
2. The computer implemented method of claim 1 ,
wherein unlinking the block of code to make it unreachable is executed prior to removing the block of code.
3. The computer implemented method of claim 1 ,
wherein deleting a block of code from the code cache comprises deleting the block of code from the code cache when all threads that were in the cache at the time of making the block of code unreachable have exited the code cache at least once.
4. The computer implemented method of claim 1 , further comprising, at the time of the unlinking of the block of code, recording a number of threads in the block of code as a reference count.
5. The computer implemented method of claim 4 , wherein using the plurality of second timestamps to determine whether a thread has exited the cache since making the block of code unreachable comprises decreasing the reference count by one each time a thread exits the cache at a time later than the first timestamp.
6. The computer implemented method of claim 5 , wherein determining that each thread in the cache has exited the cache since making the block of code unreachable comprises determining that the reference count is zero.
7. A computer, comprising:
a storage device;
a communication interface; and
one or more processors in communication with the storage device and the communication interface, the one or more processors being programmed to:
implement a code cache that is shared among different threads while avoiding brute-force all-thread suspension;
unlink a block of code from the cached code to make it unreachable;
record a first timestamp at a time of unlinking of the block of code;
use a plurality of second timestamps to determine whether the multiple threads have exited the cache since making the block of code unreachable;
determine that each thread in the cache has exited the cache since making the block of code unreachable; and
free the block of code based on the determination that each thread in the cache has exited the cache since making the block of code unreachable.
8. The computer of claim 7 , wherein at the time of the unlinking of the block of code the one or more processors are further programmed to record a number of threads in the block of code as a reference count.
9. The computer of claim 8 , wherein using the plurality of second timestamps to determine whether a thread has exited the cache since making the block of code unreachable comprises decreasing the reference count by one each time a thread exits the cache at a time later than the first timestamp.
10. The computer implemented method of claim 9 , wherein determining that each thread in the cache has exited the cache since making the block of code unreachable comprises determining that the reference count is zero.
11. One or more non-transitory processor readable storage devices storing processor readable code, the processor readable code for programming one or more processors to perform a method comprising:
implementing a code cache;
sharing the code cache among threads while avoiding brute-force all-thread-suspension;
unlinking a block of code from the cached code to make it unreadable;
recording a first timestamp at a time of unlinking of the block of code;
using a plurality of second timestamps to determine whether the multiple threads have exited the cache since making the block of code unreachable;
determining that each thread in the cache has exited the cache since making the block of code unreachable; and
freeing the block of code based on the determination that each thread in the cache has exited the cache since making the block of code unreachable.
12. The one or more non-transitory processor readable storage devices of claim 11 , wherein the processor readable code for programming one or more processors to perform a method further comprises, at the time of the unlinking of the block of code, recording a number of threads in the block of code as a reference count.
13. The one or more non-transitory processor readable storage devices of claim 12 , wherein using the plurality of second timestamps to determine whether a thread has exited the cache since making the block of code unreachable comprises decreasing the reference count by one each time a thread exits the cache at a time later than the first timestamp.
14. The one or more non-transitory processor readable storage devices of claim 13 , wherein determining that each thread in the cache has exited the cache since making the block of code unreachable comprises determining that the reference count is zero.