Reducing HTTP malware scanner latency using HTTP range queries for random access
View Patent ↗A system, method, and computer program product for scanning downloaded files providing improved performance relative to conventional download scanning techniques by performing the download and scanning functions in parallel and random order, thus improving overall performance. A method of scanning a requested file for a computer malware comprising the steps of: receiving a request from a user system to transfer a file from a server. Starting to transfer said file, and at the same time, handing a partial copy of the file to computer malware scanning software, receiving requests from the computer malware scanning software for data comprising additional arbitrary portions of the requested file, transferring only the requested portion of the file and supplying the requested data to the computer malware scanning software until the malware scanning software can produce a clean/infected indication. Then completing the transfer of the entire file from the server and handing it off to the user system in parallel.
1. A method of transferring a requested file for a computer malware scanning software comprising the steps of:
receiving a request to transfer a file from a computer malware scanning software;
receiving a request from the computer malware scanning software for data comprising a randomly accessed portion of the requested file selected in a random order by the computer malware scanning software from among portions of the requested file and based on a portion of the requested file to be scanned by the computer malware scanning software, wherein the randomly accessed portion of the requested file is requested utilizing a byte range technique;
in response to a determination that the requested portion of the requested file cannot be transferred, transferring an entirety of the requested file and supplying the requested data to the computer malware scanning software to fulfill the request for data comprising a portion of the requested file, wherein it is determined that the requested portion of the requested file cannot be transferred if the requested portion of the requested file cannot be randomly accessed;
in response to a determination that the requested portion of the requested file can be transferred, transferring the requested portion of the requested file and supplying the requested data to the computer malware scanning software to fulfill the request for data comprising a portion of the requested file;
tracking information associated with each transfer of a requested portion of the requested file; and
determining whether information associated with the requested file has changed;
wherein the byte range technique turns a serial download mechanism into a random access file mechanism;
wherein the data associated with the request from the computer malware scanning software comprises a plurality of randomly accessed portions of the requested file.
2. The method of claim 1 , wherein the request to transfer the requested file from the computer malware scanning software comprises a request to transfer the requested file from an external system.
3. The method of claim 2 , wherein the external system is communicatively connected via a network.
4. The method of claim 3 , wherein the network comprises the Internet.
5. The method of claim 4 , wherein the step of transferring the requested portion of the requested file comprises the step of:
initiating a session with the external system to obtain the requested portion of the requested file.
6. The method of claim 5 , wherein the session is a hypertext transfer protocol session.
7. The method of claim 6 , wherein the hypertext transfer protocol session uses the byte range technique.
8. The method of claim 1 , wherein an indication that the requested portion of the requested file cannot be randomly accessed comprises an error indication or a transfer of the entire requested file.
9. The method of claim 1 , wherein the information comprises hypertext transfer protocol entity tags or last modified timestamp information.
10. The method of claim 9 , further comprising the step of:
restarting the requests from the computer malware scanning software for data.
11. The method of claim 10 , further comprising the step of:
transferring the entirety of the requested file.
12. The method of claim 1 , further comprising the step of:
performing the steps of claim 1 in response to a request from a user system for the requested file.
13. The method of claim 12 , further comprising the steps of:
scanning at the computer malware scanning software the data comprising a portion of the requested file to determine if the requested file includes a computer malware; and
delivering the requested file to the user system in response to determining that the requested file does not include a computer malware.
14. The method of claim 13 , wherein the step of delivering the requested file to the user system comprises the steps of:
determining whether the entirety of the requested file has been transferred;
starting delivery of the requested file to the user system even if the entire file has not been transferred; and
transferring those portions of the requested file that have not been transferred and delivering those portions of the requested file once they have been transferred.
15. The method of claim 14 , wherein the step of transferring those portions of the requested file that have not been transferred comprises the step of:
initiating a session with the external system to obtain those portions of the requested file that have not been transferred.
16. The method of claim 15 , wherein the session is a hypertext transfer protocol session.
17. The method of claim 16 , wherein the hypertext transfer protocol session uses the byte range technique.
18. The method of claim 1 , wherein the plurality of randomly accessed portions of the requested file are read in a random order.
19. The method of claim 1 , wherein a system call handler intercepts system level calls made by the computer malware scanning software and simulates system level function calls utilized by the computer malware scanning software to determine whether the requested file includes the computer malware.
20. The method of claim 1 , wherein scanning by the computer malware scanning software is performed in parallel with transfers of requested portions of the requested file to the malware scanning software.
21. A system comprising:
a processor operable to execute computer program instructions;
a memory operable to store computer program instructions executable by the processor; and
computer program instructions stored in the memory and executable to perform the steps of:
receiving a request to transfer a file from a computer malware scanning software;
receiving a request from the computer malware scanning software for data comprising a randomly accessed portion of the requested file selected in a random order by the computer malware scanning software from among portions of the requested file and based on a portion of the requested file to be scanned by the computer malware scanning software, wherein the system is operable such that the randomly accessed portion of the requested file is requested utilizing a byte range technique;
in response to a determination that the requested portion of the requested file cannot be transferred, transferring an entirety of the requested file and supplying the requested data to the computer malware scanning software to fulfill the request for data comprising a portion of the requested file, wherein the system is operable such that it is determined that the requested portion of the requested file cannot be transferred if the requested portion of the requested file cannot be randomly accessed;
in response to a determination that the requested portion of the requested file can be transferred, transferring the requested portion of the requested file and supplying the requested data to the computer malware scanning software to fulfill the request for data comprising a portion of the requested file;
tracking information associated with each transfer of a requested portion of the requested file; and
determining whether information associated with the requested file has changed;
wherein the system is operable such that the byte range technique turns a serial download mechanism into a random access file mechanism;
wherein the system is operable such that the data associated with the request from the computer malware scanning software comprises a plurality of randomly accessed portions of the requested file.
22. The system of claim 21 , wherein the request to transfer the requested file from the computer malware scanning software comprises a request to transfer the requested file from an external system.
23. The system of claim 22 , wherein the external system is communicatively connected via a network.
24. The system of claim 23 , wherein the network comprises the Internet.
25. The system of claim 24 , wherein the step of transferring the requested portion of the requested file comprises the step of:
initiating a session with the external system to obtain the requested portion of the requested file.
26. The system of claim 25 , wherein the session is a hypertext transfer protocol session.
27. The system of claim 26 , wherein the hypertext transfer protocol session uses the byte range technique.
28. The system of claim 21 , wherein an indication that the requested portion of the requested file cannot be randomly accessed comprises an error indication or a transfer of the entire requested file.
29. The system of claim 21 , wherein the information comprises hypertext transfer protocol entity tags or last modified timestamp information.
30. The system of claim 29 , further comprising the step of:
restarting the requests from the computer malware scanning software for data.
31. The system of claim 30 , further comprising the step of:
transferring the entirety of the requested file.
32. The system of claim 21 , further comprising the step of:
performing the steps of claim 1 in response to a request from a user system for the requested file.
33. The system of claim 32 , further comprising the steps of:
scanning at the computer malware scanning software the data comprising a portion of the requested file to determine if the requested file includes a computer malware; and
delivering the requested file to the user system in response to determining that the requested file does not include a computer malware.
34. The system of claim 33 , wherein the step of delivering the requested file to the user system comprises the steps of:
determining whether the entirety of the requested file has been transferred;
starting delivery of the requested file to the user system even if the entire requested file has not been transferred; and
transferring those portions of the requested file that have not been transferred and delivering those portions of the requested file once they have been transferred.
35. The system of claim 34 , wherein the step of transferring those portions of the requested file that have not been transferred comprises the step of:
initiating a session with the external system to obtain those portions of the requested file that have not been transferred.
36. The system of claim 35 , wherein the session is a hypertext transfer protocol session.
37. The system of claim 36 , wherein the hypertext transfer protocol session uses the byte range technique.
38. A computer program product comprising:
a tangible computer readable storage medium;
computer program instructions, recorded on the computer readable storage medium, executable by a processor, for performing the steps of:
receiving a request to transfer a file from a computer malware scanning software;
receiving a request from the computer malware scanning software for data comprising a randomly accessed portion of the requested file selected in a random order by the computer malware scanning software from among portions of the requested file and based on a portion of the requested file to be scanned by the computer malware scanning software, wherein the computer program product is operable such that the randomly accessed portion of the requested file is requested utilizing a byte range technique;
in response to a determination that the requested portion of the requested file cannot be transferred, transferring an entirety of the requested file and supplying the requested data to the computer malware scanning software to fulfill the request for data comprising a portion of the requested file, wherein the computer program product is operable such that it is determined that the requested portion of the requested file cannot be transferred if the requested portion of the requested file cannot be randomly accessed,
in response to a determination that the requested portion of the requested file can be transferred, transferring the requested portion of the requested file and supplying the requested data to the computer malware scanning software to fulfill the request for data comprising a portion of the requested file;
tracking information associated with each transfer of a requested portion of the requested file; and
determining whether information associated with the requested file has changed;
wherein the computer program product is operable such that the byte range technique turns a serial download mechanism into a random access file mechanism;
wherein the computer program product is operable such that the data associated with the request from the computer malware scanning software comprises a plurality of randomly accessed portions of the requested file.
39. The computer program product of claim 38 , wherein the request to transfer the requested file from the computer malware scanning software comprises a request to transfer the requested file from an external system.
40. The computer program product of claim 39 , wherein the external system is communicatively connected via a network.
41. The computer program product of claim 40 , wherein the network comprises the Internet.
42. The computer program product of claim 41 , wherein the step of transferring the requested portion of the requested file comprises the step of:
initiating a session with the external system to obtain the requested portion of the requested file.
43. The computer program product of claim 42 , wherein the session is a hypertext transfer protocol session.
44. The computer program product of claim 43 , wherein the hypertext transfer protocol session uses the byte range technique.
45. The computer program product of claim 38 , wherein an indication that the requested portion of the requested file cannot be randomly accessed comprises an error indication or a transfer of the entire requested file.
46. The computer program product of claim 38 , wherein the information comprises hypertext transfer protocol entity tags or last modified timestamp information.
47. The computer program product of claim 46 , further comprising the step of:
restarting the requests from the computer malware scanning software for data.
48. The computer program product of claim 47 , further comprising the step of:
transferring the entirety of the requested file.
49. The computer program product of claim 38 , further comprising the step of:
performing the steps of claim 1 in response to a request from a user system for the requested file.
50. The computer program product of claim 49 , further comprising the steps of:
scanning at the computer malware scanning software the data comprising a portion of the requested file to determine if the requested file includes a computer malware; and
delivering the requested file to the user system in response to determining that the requested file does not include a computer malware.
51. The computer program product of claim 50 , wherein the step of delivering the requested file to the user system comprises the steps of:
determining whether the entirety of the requested file has been transferred;
starting delivery of the requested file to the user system even if the entire requested file has not been transferred; and
transferring those portions of the requested file that have not been transferred and delivering those portions of the requested file once they have been transferred.
52. The computer program product of claim 51 , wherein the step of transferring those portions of the requested file that have not been transferred comprises the step of:
initiating a session with the external system to obtain those portions of the requested file that have not been transferred.
53. The computer program product of claim 52 , wherein the session is a hypertext transfer protocol session.
54. The computer program product of claim 53 , wherein the hypertext transfer protocol session uses the byte range technique.