IP Library › Granted Patent US 10,419,537
Granted Patent B1
US 10,419,537 · App. 15/080,554 · Granted Sep 17, 2019

Architecture for a converged compute and file system within network-attached storage clusters

Inventors: Jaumir Valença da Silveira Junior (Rio de Janeiro, BR); Rômulo Teixeira de Abreu Pinho (Niterói, BR); Angelo Ernani Maia Ciarlini (Rio de Janeiro, BR); Karin Koogan Breitman (Rio de Janeiro, BR)
Assignee: EMC IP HOLDING COMPANY LLC
H04L67/1097G06F16/183
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,419,537
App. No.
15/080,554
Filed
Mar 24, 2016
Granted
Sep 17, 2019
Kind
B1
Examiner
WOO, ISAAC M
Art Unit
2163
USPC
707/827
Abstract

Scale-out network attached storage (NAS) file systems can employ an Ingest, Transform, Store (ITS) framework for data processing. In one aspect, the ITS-NAS file systems comprise NAS nodes and high performance computing (HPC) nodes that operate under a common operating system and that are coupled to each other via a common high-bandwidth, low-latency private network infrastructure. The NAS nodes can present data to the HPC nodes as well as dispatch the execution of transform services to the HPC nodes. The ITS-NAS file systems enable massive parallelization of operations on files, for example, complex distributed operations on large files and/or simple parallel operations on large collections of small files, all within the same hardware and software architecture.

Claims (36)

1. A file system, comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:

receiving, from a client device, request data that is related to accessing data that is stored within a network attached storage server device, wherein the network attached storage server device is part of a front-end of the file system;

in response to determining that the data is stored within a first folder of a plurality of folders, of the network attached storage server device, that has been assigned a group of rules that are employable to initiate a transform service that transforms at least a portion of the data, directing at least a portion of the data from the network attached storage server device to a high-performance computing server device of the file system via a private back-end network link, the high-performance computing server device being inaccessible to the client device, wherein the group of rules is related to an availability of system resources of the file system; and

instructing the high-performance computing server device to implement the transform service to generate transformed data that is to be transferred to the client device.

2. The file system of claim 1 , wherein the operations further comprise:

receiving the transformed data from the high-performance computing server device via the private back-end network link.

3. The file system of claim 1 , wherein the operations further comprise:

selecting at least the portion of the data based on configuration data received via a user device.

4. The file system of claim 3 , wherein the high-performance computing server device has been assigned a coordinator role for a defined time period.

5. The file system of claim 1 , wherein the directing comprises asynchronously directing at least the portion of the data to the high-performance computing server device.

6. The file system of claim 1 , wherein the instructing comprises instructing the high-performance computing server device to process at least the portion of the data in response to determining that a resource utilization criterion associated with the file system has been satisfied.

7. The file system of claim 6 , wherein the resource utilization criterion comprises a threshold associated with available computational resources associated with the high-performance computing server device.

8. The file system of claim 6 , wherein the resource utilization criterion comprises a threshold associated with available storage resources associated with the network attached storage server device.

9. The file system of claim 1 , wherein the file system comprises a single operating system instance that has a single workload queue.

10. The file system of claim 1 , wherein the high-performance computing server device is a first high-performance computing server device of a group of high-performance computing server devices, and wherein the operations further comprise:

directing at least a second portion of the data from the network attached storage server device to a second high-performance computing server device of the group of high-performance computing server devices of the file system via the private back-end network link, the second high-performance computing server device being inaccessible to the client device.

11. A method, comprising:

receiving, by a network attached storage server device comprising a processor, request data from a client device via a front-end network, wherein the request data represents a request for content stored within the network attached storage server device; and

in response to determining the content is stored within a first folder of a plurality of folders, of the network attached storage server device, that has been assigned rules to process the content, communicating, by the network attached storage server device, with a high-performance computing server device via a backplane link of a private communication network, the high-performance computing server device being inaccessible to the client device, wherein the rules are related to an availability of system resources of the file system, and wherein the communicating comprises instructing the high-performance computing server device to implement a service that transforms at least a portion of the data to generate result data that is to be provided to the client device.

12. The method of claim 11 , wherein the communicating comprises directing at least a portion of the content to the high-performance computing server device.

13. The method of claim 12 , wherein the communicating comprises communicating via an InfiniBand link.

14. The method of claim 13 , further comprising:

receiving, by the network attached storage server device, the result data from the high-performance computing server device via the backplane link.

15. The method of claim 14 , further comprising:

storing, by the network attached storage server device, the result data within a defined output folder of the plurality of folders of the network attached storage server device, wherein the output folder is specified within the rules.

16. The method of claim 12 , further comprising:

based on defined configuration data, selecting, by the network attached storage server device, at least the portion of the content.

17. The method of claim 12 , wherein the instructing comprises instructing the high-performance computing server device to implement a transcoding service that modifies a format of at least the portion of the content.

18. A non-transitory computer-readable storage medium comprising instructions that, in response to execution, cause a high-performance computing server device comprising a processor to perform operations, comprising:

receiving content from a network attached storage server device via a back-end network link of a private communication network that couples the high-performance computing server device with the network attached storage server device, wherein the network attached storage server device is deployed in a front-end of a file system and the high-performance computing server device is deployed in a back-end of the file system, wherein the instruction data is received in response to determining that a request for accessing the content has been received from a client device and that the content is stored within a first folder of a plurality of folders of the network attached storage server device that has been assigned a set of rules that are that are employable to initiate a transform service that transforms the content, wherein the set of rules is related to an availability of system resources of the file system, and wherein the high-performance computing server device is inaccessible to the client device; and

in response to receiving instruction data to initiate the transform service, facilitating a transformation of the content to generate result data that is to be transferred to the client device.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

facilitating a transmission of the result data to the network attached storage server device via the back-end network link.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the back-end network link comprises a backplane bus.

Assignments (14)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (051302/0528) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO WYSE TECHNOLOGY L.L.C.); SECUREWORKS CORP.
Reel/Frame 060438/0593 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
RELEASE OF SECURITY INTEREST AT REEL 051449 FRAME 0728 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.; SECUREWORKS CORP.; EMC CORPORATION
Reel/Frame 058002/0010 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Dec 31, 2019
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.; SECUREWORKS CORP.; EMC CORPORATION
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 051449/0728 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Dec 16, 2019
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.; SECUREWORKS CORP.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 051302/0528 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2016
From: DA SILVEIRA JUNIOR, JAUMIR VALENÇA; PINHO, RÔMULO TEIXEIRA DE ABREU; CIARLINI, ANGELO ERNANI MAIA; BREITMAN, KARIN KOOGAN
To: EMC CORPORATION
Reel/Frame 038097/0725 →
Cited By (3)
US 12,287,709 US 12,348,580 US 12,367,151