IP Library › Granted Patent US 11,263,220
Granted Patent B2
US 11,263,220 · App. 16/586,562 · Granted Mar 1, 2022

On-demand execution of object transformation code in output path of object storage service

Inventors: Timothy Lawrence Harris (Cambridge, GB); Kevin C. Miller (Bainbridge Island, WA); Ramyanshu Datta (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G06F16/24575G06F16/2329G06F16/24524G06F16/24552G06F16/9035
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,263,220
App. No.
16/586,562
Granted
Mar 1, 2022
Kind
B2
Abstract

Systems and methods are described for modifying input and output (I/O) to an object storage service by implementing one or more owner-specified functions to I/O requests. A function can implement a data manipulation, such as filtering out sensitive data before reading or writing the data. The functions can be applied prior to implementing a request method (e.g., GET or PUT) specified within the I/O request, such that the data to which the method is applied my not match the object specified within the request. For example, a user may request to obtain (e.g., GET) a data set. The data set may be passed to a function that filters sensitive data to the data set, and the GET request method may then be applied to the output of the function. In this manner, owners of objects on an object storage service are provided with greater control of objects stored or retrieved from the service.

Claims (49)

1. A computer-implemented method comprising:

under control of a computing system comprising one or more computer processors configured to execute specific instructions,

receiving a request to obtain a data object stored in a data store;

determining; based at least partly on the request, to execute a function using the data object prior to providing a response to the request;

configuring a code execution system to execute the function, wherein the code execution system provides on-demand execution of functions in an input/output (I/O) pipeline of the data store;

executing the function using the code execution system prior to providing the response, wherein executing the function comprises:

obtaining the data object, wherein obtaining the data object comprises obtaining a collection of records, wherein a first portion of the data object comprises at least a portion of a record of the collection of records;

determining that the first portion is to be obfuscated in the response to the request;

generating a first obfuscated portion comprising the first portion in obfuscated form, wherein the first obfuscated portion comprises a masked version of the portion of the record; and

generating a function output comprising the first obfuscated portion, wherein the function output does not include the first portion in un-obfuscated form; and

providing the response to a computing device based at least partly on the function output.

2. The computer-implemented method of claim 1 , wherein executing the function further comprises obtaining context data associated with the request, wherein determining to obfuscate the first portion is based at least partly on the context data.

3. The computer-implemented method of claim 2 , wherein executing the function further comprises determining that the context data satisfies one or more criteria, wherein the context data represents one of: a source of the request, a location associated with the request, or an access right associated with the request, and wherein the one or more criteria relate to at least one of: a permitted source of the request, a prohibited source of the request, a permitted location of the request, a prohibited location of the request, a permitted access right associated with the request, or a prohibited access right associated with the request.

4. The computer-implemented method of claim 1 , wherein executing the function further comprises:

obtaining context data associated with the request; and

determining an obfuscation type, of a plurality of obfuscation types, to apply to the first portion based at least partly on the context data.

5. The computer-implemented method of claim 4 , wherein determining the obfuscation type comprises determining one of: a customized obfuscation associated with the data object, or a default obfuscation.

6. The computer-implemented method of claim 1 , further comprising:

receiving a second request for the data object, wherein the request is associated with first context data and wherein the second request is associated with second context data different than the first content data; and

executing the function using the code execution system to generate a second function output based at least partly on the second context data, wherein the second function output is different than the function output.

7. The computer-implemented method of claim 1 , further comprising:

storing the function output in a cache;

receiving a second request for the data object; and

providing a second response to the second request based at least partly on the function output stored in the cache.

8. The computer-implemented method of claim 1 , wherein executing the function using the code execution system prior to providing the response comprises executing a serverless function on serverless code execution system.

9. A system comprising:

a data store storing plurality of data objects; and

one or more computing devices in communication with the data store and configured to at least:

receive a request to obtain a data object of the plurality of data objects;

determine, based at least partly on the request, to execute a function using the data object prior to providing a response to the request;

configure a code execution system to execute the function, wherein the code execution system provides on-demand execution of serverless functions in an input/output (I/O) pipeline of the data store;

wherein the code execution system is thereby configured to:

obtain the data object, wherein the data object comprises a collection of records, wherein a first portion of the data object comprises at least a portion of a record of the collection of records;

determine to transform at least the first portion, wherein the response is not to include the first portion in an un-transformed state;

transform the first portion of the data object to create a transformed first portion;

store the data object comprising the transformed first portion in the data store; and

generate function output comprising the transformed first portion; and

provide the response to a computing device based at least partly on the function output.

10. The system of claim 9 , wherein the code execution system being configured to transform the first portion comprises the code execution system being configured to obfuscate the first portion.

11. The system of claim 9 , wherein the code execution system being configured to transform the first portion comprises the code execution system being configured to apply a transformation comprising at least one of: addition of a watermark, alteration of a resolution, alteration of a bitrate, or application of encryption.

12. The system of claim 9 , wherein the code execution system being configured to transform the first portion comprises the code execution system being configured to remove at least a portion of the first portion.

13. The system of claim 9 , wherein the code execution system being configured to transform the first portion comprises the code execution system being configured to modify a format of the first portion.

14. The system of claim 9 , wherein the code execution system is further configured to obtain context data associated with the request, wherein determining to transform the first portion is based at least partly on the context data satisfying one or more criteria.

15. The system of claim 14 , wherein the code execution system is further configured to determine that the context data satisfies one or more criteria, wherein the context data represents one of: a source of the request, a location associated with the request, or an access right associated with the request, and wherein the one or more criteria relate to at least one of: a permitted source of the request, a prohibited source of the request, a permitted location of the request, a prohibited location of the request, a permitted access right associated with the request, or a prohibited access right associated with the request.

16. The system of claim 9 , wherein at least a subset of the plurality of data objects are owned by an owner, and wherein the code execution system initiates execution of owner-defined code prior to providing responses to requests to perform input/output (I/O) operations on the subset of the plurality data objects.

17. The system of claim 9 , wherein the request is a hypertext transport protocol (HTTP) GET request.

18. The system of claim 9 , wherein the one or more computing devices are further configured to at least:

generate a requested file handle, wherein the data object is readable from the data store using the requested file handle; and

send the request and the requested file handle to the code execution system, wherein the code execution system obtains the data object using the requested file handle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2020
From: HARRIS, TIMOTHY LAWRENCE; MILLER, KEVIN C; DATTA, RAMYANSHU
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 053832/0017 →
Continuity (1)
Related Publication 20210097083A1 · Apr 1, 2021