Method and system for performing a memory safety check of a program written in an unmanaged programming language

1. A method for performing a memory safety check of a program, comprising:

receiving an intermediate representation (IR) of the program, wherein the program is coded in an unmanaged programming language;

performing a first static analysis pass of the IR to generate a plurality of annotations comprising a safe pointer and an unsafe pointer;

removing, during a second static analysis pass of the IR, the safe pointer from the plurality of annotations;

inserting into the IR, before compiling and using the plurality of annotations, a sandbox function call at a preceding step to the unsafe pointer to generate a modified IR;

compiling the modified IR, including the sandbox function call, to generate an executable version of the program;

executing, inside a sandbox framework, the executable version of the program;

generating, during runtime and upon reaching the sandbox function call, a metadata entry and an enhanced pointer;

comparing, during runtime and upon reaching a use of the unsafe pointer, the metadata entry with the enhanced pointer;

compressing the unsafe pointer and the metadata entry into a compressed pointer; and

storing the compressed pointer in memory in place of the unsafe pointer.

2. The method of claim 1 , further comprising:

generating, in response to the metadata entry not validating the enhanced pointer, a memory safety error;

halting execution of the executable version; and

displaying the memory safety error.

3. The method of claim 1 , wherein compressing the unsafe pointer enables an atomic store of the unsafe pointer and the metadata entry.

4. The method of claim 1 , further comprising:

receiving a request to load the unsafe pointer from memory; and

decompressing the compressed pointer from memory into the unsafe pointer and the metadata entry.

5. The method of claim 4 , wherein decompressing enables the unsafe pointer to be loaded from memory atomically.

6. The method of claim 5 , wherein the program is multi-threaded.

7. The method of claim 1 , wherein the metadata entry comprises an unsafe object address, an unsafe object size, and an unsafe object signature and the enhanced pointer comprises an index, a signature, and an offset.

8. The method of claim 7 , wherein the offset and the unsafe object address enable pointer arithmetic to be performed.

9. The method of claim 1 , further comprising:

identifying, during the first static analysis pass of the IR, a pointer escape instruction that stores a pointer to a memory location;

annotating the pointer escape instruction with an escape annotation; and

adding the pointer escape instruction to a plurality of locations.

10. A system for performing a memory safety check of a program, comprising:

a processor;

a compiler, executing on the processor, and configured to:

receive intermediate representation (IR) of the program, wherein the program is coded in an unmanaged programming language;

perform a first static analysis pass of the IR to generate a plurality of annotations comprising a safe pointer and an unsafe pointer;

remove, during a second static analysis pass of the IR, the safe pointer from the plurality of annotations;

insert into the IR, before compilation and using the plurality of annotations, a sandbox function call at a preceding step to the unsafe pointer to generate a modified IR; and

compile the modified IR, including the sandbox function call, to generate an executable version of the program;

a sandbox framework executing on the processor and configured to:

execute the executable version of the program;

generate, during runtime and upon reaching the sandbox function call, a metadata entry and an enhanced pointer;

compare, during runtime and upon reaching a use of the unsafe pointer, the metadata entry with the enhanced pointer;

compress the unsafe pointer and the metadata entry into a compressed pointer; and

store the compressed pointer in memory in place of the unsafe pointer.

11. The system of claim 10 , the sandbox framework further configured to:

generate, in response to the metadata entry not validating the enhanced pointer, a memory safety error;

halt execution of the executable version; and

display the memory safety error.

12. The system of claim 10 , wherein compressing the unsafe pointer enables an atomic store of the unsafe pointer and the metadata entry.

13. The system of claim 10 , the sandbox framework further configured to:

receive a request to load the unsafe pointer from memory; and

decompress the compressed pointer from memory into the unsafe pointer and the metadata entry.

14. The system of claim 13 , wherein decompressing enables the unsafe pointer to be loaded from memory atomically.

15. The system of claim 14 , wherein the program is multi-threaded.

16. The system of claim 10 , wherein the metadata entry comprises an unsafe object address, an unsafe object size, and an unsafe object signature and the enhanced pointer comprises an index, a signature, and an offset.

17. The system of claim 16 , wherein the offset and the unsafe object address enable pointer arithmetic to be performed.

18. The system of claim 10 , further comprising:

identifying, during the first static analysis pass of the IR, a pointer escape instruction that stores a pointer to a memory location;

annotating the pointer escape instruction with an escape annotation; and

adding the pointer escape instruction to a plurality of locations.

19. A non-transitory computer readable storage medium (CRM) storing instructions for performing a memory safety check of a program, the instructions executable on a processor and comprising functionality to:

receive an intermediate representation (IR) of the program, wherein the program is coded in an unmanaged programming language;

perform a first static analysis pass of the IR to generate a plurality of annotations comprising a safe pointer and an unsafe pointer;

remove, during a second static analysis pass of the IR, the safe pointer from the plurality of annotations;

insert into the IR, before compilation and using the plurality of annotations, a sandbox function call at a preceding step to the unsafe pointer to generate a modified IR;

compile the modified IR, including the sandbox function call, to generate an executable version of the program;

execute, inside a sandbox framework, the executable version of the program;

generate, during runtime and upon reaching the sandbox function call, a metadata entry and an enhanced pointer;

compare, during runtime and upon reaching a use of the unsafe pointer, the metadata entry with the enhanced pointer;

compress the unsafe pointer and the metadata entry into a compressed pointer; and

store the compressed pointer in memory in place of the unsafe pointer.

20. The system of claim 19 , wherein compressing the unsafe pointer enables an atomic store of the unsafe pointer and the metadata entry.

21. The non-transitory CRM of claim 19 , the instructions further comprising functionality to:

receive a request to load the unsafe pointer from memory; and

decompress the compressed pointer from memory into the unsafe pointer and the metadata entry.

22. The non-transitory CRM of claim 21 , wherein decompressing enables the unsafe pointer to be loaded from memory atomically.

23. The non-transitory CRM of claim 22 , wherein the program is multi-threaded.