Virtualized file servers and methods to persistently store file system event data
Examples of a virtualized file server that persistently stores file system event data records are described. The virtualized file server may include an audit framework with a connector publisher that is configured to publish the event data records and other information for consumption by other services using a message system. The event data records may include data related to various operations on the file system executed by the VFS. The VFS may persistently store event data records according to a data retention policy and/or until an event data record is consumed by one or more particular services.
1. At least one non-transitory computer-readable storage medium including instructions that, when executed by a computing node, cause the computing node to:
create, at a virtualized file server, an event data record based on event data corresponding to an operation performed within the virtualized file server;
temporarily store, by an audit framework associated with a file server virtual machine (FSVM) and comprising an event logger, the event data record associated with the event data in an audit queue;
persistently store, by the event logger, the event data record from the audit queue and a write index in an event log of a volume group associated with the file server virtual machine;
migrate the file server virtual machine to a different computing node of the virtualized file server;
continue updating the event log after migration based on the write index; and
maintain the event data record in the event log until the event data record has been transmitted to a requesting service.
2. The at least one computer-readable storage medium of claim 1 , wherein the instructions further cause the computing node to stop maintenance of the event data record in the event log in response to receipt of an acknowledgment corresponding to the event data record from the requesting service.
3. The at least one computer-readable storage medium of claim 1 , wherein the instructions further cause the computing node to stop maintenance of the event data record in response to receipt of an acknowledgment corresponding to the event data record from an analytics system hosted on a second computing node of the virtualized file server.
4. The at least one computer-readable storage medium of claim 3 , wherein the instructions further cause the second computing node to process the event record, via the analytics system, to provide an entry in an analytics datastore.
5. The at least one computer-readable storage medium of claim 1 , wherein the instructions, when executed, cause the computing node to create the event data record in response to receipt of information pertaining to the event data record via a network file server communication or a server message block server communication.
6. The at least one computer-readable storage medium of claim 1 , wherein the instructions, when executed, cause the computing node to store the event data record in the event log with a corresponding index value based on the write index.
7. The at least one computer-readable storage medium of claim 6 , wherein the instructions, when executed, cause the computing node to adjust the write index in response to storing the event data record in the event log with the corresponding index value.
8. The at least one computer-readable storage medium of claim 6 , wherein the instructions, when executed, cause the computing node to store the write index to a control block for persistent storage.
9. The at least one computer-readable storage medium of claim 8 , wherein the instructions, when executed, cause the computing node to, after a restart, retrieve the write index from the control block to use for storing subsequent event data records.
10. The at least one computer-readable storage medium of claim 1 , wherein the instructions, when executed, cause the computing node to provide the event data record corresponding to a read index value from the event log in response to receipt of a request identifying the read index value.
11. The at least one computer-readable storage medium of claim 10 , wherein the instructions, when executed, cause the computing node to update the read index value to a new value in response to receipt of an acknowledgment of receipt of the event data record corresponding to the read index value from the requesting service.
12. A system comprising:
a distributed file server comprising a plurality of file server virtual machines (FSVMs), wherein a first FSVM of the plurality of FSVMs is configured to receive event data corresponding to an operation performed by the first FSVM;
a first audit framework associated with the first FSVM configured to:
temporarily store an event data record associated with the event data in an audit queue, and
persistently store, by an event logger, the event data record from the audit queue and a write index in an event log of a volume group associated with the first FSVM;
wherein the distributed file server is further configured to:
migrate the first FSVM to another computing node of the distributed file server, and
continue updating the event log after migration based on the write index; and
an analytics system configured to request the event data record from the event logger, wherein the analytics system is further configured to process the event record to provide an entry in an analytics datastore.
13. The system of claim 12 , wherein the analytics system is configured to provide an acknowledgment to the first audit framework in response to receipt of the event data record.
14. The system of claim 13 , wherein the first audit framework is configured to stop maintenance of the event data record in the event log in response to receipt of the acknowledgment from the analytics system.
15. The system of claim 12 , wherein the first audit framework is configured to create the event data record in response to receipt of information pertaining to the event data record via a network file server communication or a server message block server communication.
16. The system of claim 12 , wherein a second audit framework associated with a second FSVM of the plurality of FSVMs is configured to create a second event data record of second event data corresponding to an operation performed by the second FSVM for storage in a second event log and to persistently maintain the second event data record in the second event log.
17. The system of claim 12 , wherein, in response to failure of the first FSVM, the distributed file server is configured to migrate the first FSVM and the first audit framework to the another computing node and to resume updating the event log after the migration based on the write index retrieved from a control block.
18. The system of claim 17 , wherein the first audit framework is configured to store the write index to the control block for persistent storage.
19. A method, comprising:
receiving event data corresponding to an operation performed by a virtualized file server;
temporarily storing an event data record associated with the event data in an audit queue of an audit framework associated with a file server virtual machine and associated with the virtualized file server;
retrieving, by an event logger associated with the audit framework, the event data record from the audit queue;
persistently storing, by the event logger, the event data record of the event data from the audit queue, a write index, and an index value in an event log of a volume group associated with the file server virtual machine;
migrating the file server virtual machine to a different computing node of the virtualized file server;
continue updating the event log after migration based on the write index; and
maintaining the event data record in the event log according to a retention policy.
20. The method of claim 19 , further comprising selecting the index value based on the write index associated with a previously-stored event data record.
21. The method of claim 20 , further comprising adjusting the write index after storing the event data record.
22. The method of claim 19 , further comprising stopping maintenance of the event data record in response to receipt of an acknowledgment of successful receipt of the event data record by a requesting service.
23. The method of claim 22 , further comprising, processing, via an analytics system of the requesting service, to provide an entry in an analytics datastore.
24. The method of claim 19 , further comprising receiving the event data via a network file server communication or a server message block server communication.
25. The method of claim 19 , further comprising, after a restart, retrieving the write index from a control block to use for storing subsequent event data records.
26. The method of claim 25 , further comprising storing the write index to the control block for persistent storage.
27. The at least one computer-readable storage medium of claim 1 , wherein the instructions further cause the computing node to, responsive to transmission of the event data record to the requesting service, stop persistent storage of the requested event data record.
28. The at least one computer-readable storage medium of claim 1 , wherein the event logger is separate from the audit queue.
29. The at least one computer-readable storage medium of claim 1 , wherein the event log of the volume group associated with the FSVM is accessible to a plurality of FSVMs.
30. The system of claim 12 , wherein the first audit framework further configured to, responsive to receipt of the request, stop persistent storage of the requested event data record.
31. The system of claim 12 , wherein the event logger is separate from the audit queue.
32. The system of claim 12 , wherein a virtualized file server comprises the plurality of FSVMs including at least the first FSVM, and wherein the event log of the volume group associated with the first FSVM is accessible to the plurality of FSVMs.
33. The method of claim 19 , further comprising responsive to transmission of the event data record by the audit framework to a requesting service, stopping persistent storage of the requested event data record.
34. The method of claim 19 , wherein the event logger is separate from the audit queue.
35. The method of claim 19 , wherein the virtualized file server comprises a plurality of FSVMs including at least the FSVM, and wherein the event log of the volume group associated with the FSVM is accessible to the plurality of FSVMs.