Generating stub objects with metadata referencing upload parts of multi-part upload object
The technologies described herein are generally directed toward generating a stub object with metadata referencing uploaded parts of a multipart upload object. In an embodiment, a method can include receiving a multipart upload object comprising references to upload parts. Further, the method can include, based on the multipart upload object being predicted to be subject to spillover during creation of the multipart upload object according to the upload parts, identifying the upload parts referenced by the multipart upload object, resulting in identified upload parts. The method can further include generating a stub object with metadata referencing the identified upload parts of the multipart upload object.
1 . A method, comprising
receiving, by a system comprising at least one processor, a multipart upload object comprising references to upload parts;
based on the multipart upload object being predicted to be subject to spillover during creation of the multipart upload object according to the upload parts, identifying, by the system, the upload parts referenced by the multipart upload object, resulting in identified upload parts;
generating, by the system, a stub object with metadata referencing the identified upload parts of the multipart upload object, wherein the metadata references the identified upload parts based on respective upload identifier values of the identified upload parts;
generating, by the system, an index structure for the stub object based on an aggregation of the respective key values of the identified upload parts;
receiving, by the system, a first read object request corresponding to a first requested range of the stub object; and
based on the index structure, responding, by the system, to the first read object request with a first portion of the identified upload parts corresponding to the first requested range.
2 . The method of claim 1 , further comprising, after generating the stub object, removing, by the system, the multipart upload object.
3 . The method of claim 1 , further comprising, based on the multipart upload object being predicted to be subject to the spillover, postponing, by the system, garbage collection of the identified upload parts.
4 . The method of claim 1 , wherein the metadata comprises an upload identifier of the multipart upload object.
5 . The method of claim 1 , further comprising, analyzing, by the system, the multipart upload object, wherein the multipart upload object was predicted to be subject to the spillover based on the analyzing.
6 . The method of claim 5 , wherein the multipart upload object was predicted to be subject to the spillover based on the analyzing indicating that a size of the multipart upload object implicates a size threshold.
7 . The method of claim 1 , wherein the aggregation of the respective key values comprises shifting data ranges to be relative to the stub object.
8 . The method of claim 1 , further comprising:
receiving, by the system, a second read object request corresponding to a second requested range of the stub object;
based on the metadata, identifying, by the system, the respective key values of the identified upload parts that correspond to the second requested range, resulting in identified key values; and
responding, by the system, to the second read object request with a second portion of the identified upload parts corresponding to the identified key values.
9 . The method of claim 1 , further comprising:
after the generating of the stub object, deleting, by the system, the multipart upload object.
10 . Controller equipment, comprising:
at least one processing unit; and
at least one memory coupled to the at least one processing unit and storing instructions configured to be executed by the at least one processing unit, wherein the instructions, when executed by the at least one processing unit, cause the controller equipment to perform actions comprising:
identifying a multipart upload object received by cloud storage equipment,
analyzing the multipart upload object, resulting in a prediction that the multipart upload object is threshold likely to be subject to spillover during creation of the multipart upload object,
based on the prediction and in response to a completion command being executed that implicates the multipart upload object, generating a stub object with metadata referencing received upload parts of the multipart upload object,
generating an index structure for the stub object based on an aggregation of respective key values of the received upload parts of the multipart upload object,
receiving a read object request corresponding to a requested range of the stub object, and
based on the index structure, responding to the read object request with a portion of the received upload parts corresponding to the requested range.
11 . The controller equipment of claim 10 , wherein the aggregation of the respective key values of the received upload parts of the multipart upload object comprises shifting data ranges to be relative to the stub object.
12 . The controller equipment of claim 10 , wherein the multipart upload object was predicted to be subject to the spillover based on the analyzing indicating that a size of the multipart upload object implicates a size threshold.
13 . The controller equipment of claim 10 , wherein the operations further comprise, based on the multipart upload object being predicted to be subject to the spillover, postponing garbage collection of the received upload parts.
14 . The controller equipment of claim 10 , further comprising:
after the generating of the stub object, deleting the multipart upload object.
15 . A non-transitory machine-readable medium comprising executable instructions that, when executed by at least one processor, facilitate performance of operations, the operations, comprising:
receiving a multipart upload object;
based on the multipart upload object being predicted to be subject to spillover in connection with completion of the receiving of the multipart upload object, identifying received upload parts referenced by the multipart upload object;
generating a stub object with metadata referencing the received upload parts of the multipart upload object, wherein the metadata references the received upload parts based on respective upload identifier values of the received upload parts;
generating an index structure for the stub object based on an aggregation of the respective key values of the received upload parts;
receiving a read object request corresponding to a requested range of the stub object; and
based on the index structure, responding to the read object request with a portion of the received upload parts corresponding to the requested range.
16 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise, after the generating of the stub object, deleting the multipart upload object.
17 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise, based on the multipart upload object being predicted to be subject to the spillover, postponing garbage collection of the received upload parts.
18 . The non-transitory machine-readable medium of claim 15 , wherein the metadata comprises an upload identifier of the multipart upload object.
19 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:
analyzing the multipart upload object, wherein the multipart upload object was predicted to be subject to the spillover based on the analyzing, and
wherein the multipart upload object was predicted to be subject to the spillover based on the analyzing indicating that a size of the multipart upload object implicates a first size threshold.
20 . The non-transitory machine-readable medium of claim 15 , wherein the aggregation of the respective key values of the received upload parts of the multipart upload object comprises shifting data ranges to be relative to the stub object.