IP Library › Granted Patent US 10,078,639
Granted Patent B1
US 10,078,639 · App. 14/499,429 · Granted Sep 18, 2018

Cluster file system comprising data mover modules having associated quota manager for managing back-end user quotas

Inventors: Sorin Faibish (Newton, MA); Sassan Teymouri (Saratoga, CA); John M. Bent (Los Alamos, NM); James M. Pedone, Jr. (West Boylston, MA)
Assignee: EMC IP Holding Company LLC
G06F17/30073G06F17/30082G06F17/30221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,078,639
App. No.
14/499,429
Granted
Sep 18, 2018
Kind
B1
Abstract

A cluster file system comprises a front-end file system, a back-end file system, data mover modules arranged between the front-end and back-end file systems, and a quota manager associated with at least a given one of data mover modules. The data mover modules are configured to control archiving of data between the front-end file system and the back-end file system for respective users based at least in part on respective user quotas established by the quota manager and identifying corresponding portions of the back-end file system available to the users. The front-end file system may comprise archive directories configured for respective ones of the users, with the data mover modules being configured to archive a given file from the front-end file system in the back-end file system responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system.

Claims (47)

1. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

the at least one processing device implementing a front-end file system comprising a plurality of archive directories configured for respective ones of a plurality of users;

the at least one processing device implementing a back-end file system;

the at least one processing device implementing a plurality of data mover modules arranged between the front-end and back-end file systems; and

the at least one processing device implementing a quota manager associated with at least a given one of the plurality of data mover modules;

wherein the data mover modules are configured to control archiving of data between the front-end file system and the back-end file system for respective ones of the users based at least in part on respective user quotas established by the quota manager and identifying corresponding portions of the back-end file system available to said users; and

wherein the data mover modules are configured to archive a given file from the front-end file system in the back-end file system responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system;

the data mover modules thereby being configured to support selective archiving of files from the front-end file system in the back-end file system responsive to respective links to those files being stored in corresponding ones of the archive directories, while automatically preventing archiving of other files for which such links are not stored in corresponding ones of the archive directories, so as to conserve storage space in the back-end file system and facilitate conformance with the user quotas;

the respective links to the selectively archived files thereby being stored in at least one of the archive directories;

wherein the quota manager is further configured to establish archive directory quotas for respective ones of the archive directories and to proactively recommend changes to the archive directory quotas based on current sizes of the archive directories relative to the established archive directory quotas.

2. The apparatus of claim 1 wherein archiving of temporary files from the front-end file system in the back-end file system is prevented by preventing storage of links to the temporary files in the archive directories.

3. The apparatus of claim 1 wherein said archive directories include the respective archive directory quotas.

4. The apparatus of claim 1 wherein the quota manager has an associated policy engine providing one or more policies for use by the quota manager in establishing the user quotas identifying corresponding portions of the back-end file system available to said users.

5. The apparatus of claim 1 wherein the quota manager is implemented internally to a given one of the data mover modules.

6. The apparatus of claim 1 wherein the data mover modules enforce the respective user quotas for their corresponding users.

7. The apparatus of claim 1 further comprising a plurality of back-end file systems implemented by the at least one processing device with the data mover modules being configured to control archiving of data between the front-end file system and each of the back-end file systems responsive to user quotas established by the quota manager for each of the back-end file systems.

8. The apparatus of claim 1 wherein the front-end file system and back-end file system are associated with respective different sets of one or more hierarchical storage tiers for use in hierarchical storage management.

9. The apparatus of claim 1 further comprising a plurality of compute nodes coupled to the front-end file system and wherein the data mover modules are implemented at least in part utilizing software running on respective ones of the compute nodes.

10. The apparatus of claim 1 wherein a given one of the data mover modules comprises a plurality of server nodes that are synchronized with one another utilizing a message passing interface (MPI) protocol.

11. The apparatus of claim 1 wherein the data mover modules are implemented at least in part utilizing a virtual layer of a parallel log-structured file system (PLFS) and wherein the virtual layer comprises PLFS middleware implemented by the at least one processing device and configured to migrate files between the front-end and back-end file systems.

12. The apparatus of claim 1 wherein the front-end file system comprises a hierarchical storage management application programming interface and wherein the data mover modules are configured to control movement of data between the front-end file system and the back-end file system responsive to information received via the hierarchical storage management application programming interface.

13. A processing platform comprising the apparatus of claim 1 .

14. A method comprising:

configuring a plurality of data mover modules for arrangement between a front-end file system and a back-end file system;

configuring a quota manager associated with at least a given one of the plurality of data mover modules;

establishing via the quota manager a plurality of user quotas identifying corresponding portions of the back-end file system available to respective users; and

controlling archiving of data between the front-end file system and the back-end file system via the data mover modules based at least in part on the established user quotas;

wherein the front-end file system comprises a plurality of archive directories configured for respective ones of said users and controlling archiving further comprises archiving a given file from the front-end file system in the back-end file system via one or more of the data mover modules responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system;

the data mover modules thereby being configured to support selective archiving of files from the front-end file system in the back-end file system responsive to respective links to those files being stored in corresponding ones of the archive directories, while automatically preventing archiving of other files for which such links are not stored in corresponding ones of the archive directories, so as to conserve storage space in the back-end file system and facilitate conformance with the user quotas;

the respective links to the selectively archived files thereby being stored in at least one of the archive directories;

wherein the quota manager is further configured to establish archive directory quotas for respective ones of the archive directories and to proactively recommend changes to the archive directory quotas based on current sizes of the archive directories relative to the established archive directory quotas; and

wherein the configuring, establishing and controlling are implemented using at least one processing device comprising a processor coupled to a memory.

15. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device comprising a processor coupled to a memory causes the at least one processing device:

to configure a plurality of data mover modules for arrangement between a front-end file system and a back-end file system;

to configure a quota manager associated with at least a given one of the plurality of data mover modules;

to establish via the quota manager a plurality of user quotas identifying corresponding portions of the back-end file system available to respective users; and

to control archiving of data between the front-end file system and the back-end file system via the data mover modules based at least in part on the established user quotas;

wherein the front-end file system comprises a plurality of archive directories configured for respective ones of said users and controlling archiving further comprises archiving a given file from the front-end file system in the back-end file system via one or more of the data mover modules responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system;

the data mover modules thereby being configured to support selective archiving of files from the front-end file system in the back-end file system responsive to respective links to those files being stored in corresponding ones of the archive directories, while automatically preventing archiving of other files for which such links are not stored in corresponding ones of the archive directories, so as to conserve storage space in the back-end file system and facilitate conformance with the user quotas;

the respective links to the selectively archived files thereby being stored in at least one of the archive directories;

wherein the quota manager is further configured to establish archive directory quotas for respective ones of the archive directories and to proactively recommend changes to the archive directory quotas based on current sizes of the archive directories relative to the established archive directory quotas.

16. The apparatus of claim 1 wherein the apparatus is implemented in a cluster file system coupled to a computer system in an information processing system, the computer system comprising a plurality of compute nodes.

17. The method of claim 14 wherein a given one of the data mover modules comprises a plurality of server nodes that are synchronized with one another utilizing a message passing interface (MPI) protocol.

18. The method of claim 14 wherein the data mover modules are implemented at least in part utilizing a virtual layer of a parallel log-structured file system (PLFS) and wherein the virtual layer comprises PLFS middleware configured to migrate files between the front-end and back-end file systems.

19. The method of claim 14 wherein the front-end file system comprises a hierarchical storage management application programming interface and wherein the data mover modules are configured to control movement of data between the front-end file system and the back-end file system responsive to information received via the hierarchical storage management application programming interface.

20. The article of manufacture of claim 15 wherein the front-end file system comprises a hierarchical storage management application programming interface and wherein the data mover modules are configured to control movement of data between the front-end file system and the back-end file system responsive to information received via the hierarchical storage management application programming interface.

Assignments (10)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2015
From: FAIBISH, SORIN; TEYMOURI, SASSAN; BENT, JOHN M.; PEDONE, JAMES M., JR.
To: EMC CORPORATION
Reel/Frame 034735/0601 →
Cited By (5)
US 12,342,007 US 12,382,148 US 12,436,916 US 12,499,085 US 12,585,621