Systems for performing an external (disk-based) sort of a large data file which take advantage of “presorted” data already present in the input
View Patent ↗Systems and programs for improving the efficiency of a sorting process in a computer system are disclosed. Data is provided in an input file external to the central processing unit of the computer system. In one embodiment, the implemented process involves investigating the contents of the input file in order to identify presorted portions thereof; incorporating the identified presorted portions of the input file into a second file external to the central processing unit, performing this step by rearranging directory information, without physically transferring the presorted portions from the input file. In sort processes involving both a string generation phase and a merge phase, the techniques described may be used in either or both phases, as well as in any output phase. Rearranging directory information rather than physically transferring data provides for greater efficiency in disk I/O.
1. A system for ordering data, comprising:
(a) a central processing unit;
(b) memory accessible to the central processing unit;
(c) at least one external storage device whose contents are made accessible to processes running on the central processing unit through a directory structure of a file system;
(d) processing logic to investigate the contents of an input file stored in the at least one data storage device and to identify presorted portions thereof; and
(e) processing logic to incorporate the identified presorted portions of the input file into a second file in the at least one data storage device, by rearranging information in the directory structure so as to incorporate in the second file the presorted portions of the input file without physically relocating the presorted portions in the at least one data storage device.
2. The system of claim 1 , wherein the second file is regarded as a sortwork file in a sort merge process.
3. The new system of claim 1 , wherein the file system is NTSF.
4. The new system of claim 1 , wherein the directory structure comprises:
a plurality of pointers, the plurality of pointers corresponding to a plurality of blocks, the plurality of blocks constituting the second file.
5. The system of claim 4 , wherein rearranging information in the directory structure comprises:
updating the plurality of pointers.
6. A computer program product having a computer readable storage medium bearing thereon a non-transitory recording of instructions for performing, on a processor, a process for ordering data provided in an input file stored in at least one external data storage device made accessible through a directory structure of a file system to processes running on the processor, comprising:
(a) instructions for investigating the contents of the input file in order to identify presorted portions thereof; and
(b) instructions for incorporating the identified presorted portions of the input file into a second file in the at least one data storage device, by rearranging information in the directory structure so as to incorporate in the second file the presorted portions of the input file without physically relocating the presorted portions in the at least one data storage device.
7. The computer program product of claim 6 , wherein said second file is regarded as a sortwork file in a sort merge process.