Granted Patent B2

US 12,619,576 · App. 16/880,571 · Granted May 5, 2026

Inventors: David Tobin (Atherton, CA); Pawel Adamowicz (London, GB); Steven Fackler (Menlo Park, CA); Sri Krishna Vempati (Jersey City, NJ); Wilson Wong (Menlo Park, CA); Orcun Simsek (New York City, NY)

Assignee: Palantir Technologies Inc.

G06F16/1727G06F16/2428G06F16/24568G06F16/248

Abstract

Claims (50)

1 . A system comprising:

a computer readable storage medium storing program instructions; and

a computer hardware processor in communication with the computer readable storage medium, wherein the program instructions, when executed by the computer hardware processor, cause the computer hardware processor to:

process data associated with a first time-series stored in a log, the first time-series comprising a subset of time-series data from a data source;

merge, into a first data file, time-series data having overlapping timestamp values in a third data file and a fourth data file, wherein the merging comprises including a single data value in place of multiple data values associated with overlapping timestamp values in the third and fourth data files;

determine that an incremental backup is triggered;

write the first time-series, including the merged time-series data, to a disk in the first data file in association with the first time-series;

record via a separate log that the first data file is written to the disk;

perform the incremental backup of a second data file associated with the first time-series and received prior to the incremental backup being triggered;

determine that the first data file is a merged data file formed from merging time-series data of the third data file and the fourth data file; and

in response to determining that:

(1) the first data file is a merged data file, and

(2) incremental backup of the first data file, which includes merged data of the third data file and the fourth data file, is complete, delete the third data file and the fourth data file, including the overlapping timestamp values in each of the third and fourth data files.

2 . The system of claim 1 , wherein the log is a global write ahead log.

3 . The system of claim 2 , wherein the program instructions, when executed by the computer hardware processor, further cause the computer hardware processor to write the first time-series to the disk after the global write ahead log is flushed.

4 . The system of claim 1 , wherein the log is a local write ahead log.

5 . The system of claim 4 , wherein the program instructions, when executed by the computer hardware processor, further cause the computer hardware processor to write the first time-series to the disk after the local write ahead log is flushed.

6 . The system of claim 1 , wherein the log is an in-memory buffer.

7 . The system of claim 1 , wherein the program instructions, when executed by the computer hardware processor, further cause the computer hardware processor to determine that the incremental backup is triggered based on a global state of a time-series data store.

8 . The system of claim 1 , wherein the separate log is configured to track changes that occur after the incremental backup is triggered.

9 . The system of claim 1 , wherein the incremental backup does not include new data received after the backup is triggered.

10 . A computer-implemented method comprising:

processing data associated with a first time-series stored in a log, the first time-series comprising a subset of time-series data from a data source;

merging, into a first data file, time-series data having overlapping timestamp values in a third data file and a fourth data file, wherein the merging comprises including a single data value in place of multiple data values associated with overlapping timestamp values in the third and fourth data files;

determining that an incremental backup is triggered;

writing the first time-series, including the merged time-series data, to a disk in the first data file in association with the first time-series;

recording via a separate log that the first data file is written to the disk;

performing the incremental backup of a second data file associated with the first time-series and received prior to the incremental backup being triggered;

determining that the first data file is a merged data file formed from merging time-series data of the third data file and the fourth data file; and

in response to determining that:

(1) the first data file is a merged data file; and

(2) incremental backup of the first data file, which includes merged data of the third data file and the fourth data file, is complete,

deleting the third data file and the fourth data file, including the overlapping timestamp values in each of the third and fourth data files.

11 . The computer-implemented method of claim 10 , wherein the log is a global write ahead log.

12 . The computer-implemented method of claim 11 , wherein writing the first time-series to the disk further comprises writing the data to the disk after the global write ahead log is flushed.

13 . The computer-implemented method of claim 10 , wherein the log is a local write ahead log.

14 . The computer-implemented method of claim 13 , wherein writing the first time-series data to the disk further is performed after the local write ahead log is flushed.

15 . The computer-implemented method of claim 10 , wherein the log is an in-memory buffer.

16 . A non-transitory computer-readable medium comprising one or more program instructions recorded thereon, the instructions configured for execution by a computer hardware processor in communication with the computer-readable medium in order to cause a system to:

process data associated with a first time-series stored in a log, the first time-series comprising a subset of time-series data from a data source;

merge, into a first data file, time-series data having overlapping timestamp values in a third data file and a fourth data file, wherein the merging comprises including a single data value in place of multiple data values associated with overlapping timestamp values in the third and fourth data files;

determine that an incremental backup is triggered;

write the first time-series, including the merged time-series data, to a disk in the first data file in association with the first time-series;

record via a separate log that the first data file is written to the disk;

perform the incremental backup of a second data file associated with the first time-series and received prior to the incremental backup being triggered;

determine that the first data file is a merged data file formed from compaction of the third data file and the fourth data file; and

in response to determining that:

(1) the first data file is a merged data file; and

(2) incremental backup of the first data file, which includes merged data of the third data file and the fourth data file, is complete,

delete the third data file and the fourth data file, including the overlapping timestamp values in each of the third and fourth data files.

Assignments (2)

Continuity (3)

References Cited (264)