Telemetry capture system for storage systems
A method of operating a telemetry capture system within a data storage system comprising storage devices is provided. The method includes generating a telemetry packet, and providing the telemetry packet to one or more taps via a telemetry path independent of data and control paths within the storage devices. The method also includes capturing the telemetry packet in one or more of the taps, and generating real-time telemetry data based at least on the telemetry packet.
1. A method of operating a telemetry capture system within a data storage system comprising storage devices, the method comprising:
generating a telemetry packet;
providing the telemetry packet to one or more taps via a telemetry path independent of data and control paths within the storage devices;
capturing the telemetry packet in one or more of the taps; and
generating real-time telemetry data based at least on the telemetry packet.
2. The method of claim 1 , wherein the generation of real-time telemetry data occurs during normal operation of the data storage system.
3. The method of claim 1 , wherein a telemetry packet is generated for each IO command executed by the data storage system.
4. The method of claim 1 , further comprising:
adjusting a configuration of at least one of the storage devices based at least on the real-time telemetry data.
5. The method of claim 4 , wherein the configuration of the at least one storage devices is adjusted during normal operation of the data storage system.
6. The method of claim 4 , wherein the configuration of the at least one of the storage devices affects storage device performance, storage device life, or storage device reliability.
7. The method of claim 1 , wherein the telemetry packet comprises:
identifying information for a type of event;
identifying information for a source of the telemetry packet;
event specific payload information;
a timestamp; and
a sequence number.
8. The method of claim 7 , wherein the event specific payload information comprises at least one element from the group of: IO command type, IO command size, IO command start/end times, latency, IO location, media bytes written, media bytes read, wear leveling event data, garbage collection event data, data integrity event data, environmental condition data, flash operation performance, and internal hardware operation performance.
9. The method of claim 1 , further comprising:
generating a plurality of telemetry packets; and
providing the plurality of telemetry packets as a telemetry stream on the telemetry path.
10. The method of claim 9 , wherein the one or more taps are configured to capture a subset of the plurality of telemetry packets.
11. The method of claim 1 , further comprising:
storing the real-time telemetry data on one or more of the storage devices; and
generating the real-time telemetry data during a target period of time.
12. A storage controller for controlling storage devices within a data storage system, comprising:
a host interface, configured to receive data from, and transmit data to, one or more host systems;
a media interface, configured to transmit data to, and receive data from, the storage devices; and
a processing system coupled with the host interface and the drive interface, configured to:
generate a telemetry packet;
provide the telemetry packet to one or more taps via a telemetry path independent of data and control paths within the storage devices;
capture the telemetry packet in one or more of the taps; and
generate real-time telemetry data based at least on the telemetry packet.
13. The storage controller of claim 12 , wherein the processing system is further configured to:
adjust a configuration of at least one of the storage devices based at least on the real-time telemetry data.
14. The storage controller of claim 13 , wherein the configuration of the at least one storage device is adjusted during normal operation of the data storage system.
15. The storage controller of claim 13 , wherein the configuration of the at least one of the storage devices affects storage device performance, storage device life, or storage device reliability.
16. The storage controller of claim 12 , wherein the processing system is further configured to:
generate a plurality of telemetry packets; and
provide the plurality of telemetry packets as a telemetry stream on the telemetry path.
17. The storage controller of claim 16 , wherein the one or more taps are configured to capture a subset of the plurality of telemetry packets.
18. The storage controller of claim 12 , wherein the generation of real-time telemetry data occurs during normal operation of the data storage system.
19. The storage controller of claim 12 , wherein a telemetry packet is generated for each IO command executed by the data storage system.
20. A storage device, comprising:
a data storage medium; and
a storage controller, coupled to the data storage medium and configured to store data onto the data storage medium;
the storage controller further configured to:
generate a plurality of telemetry packets, wherein a telemetry packet is generated for each IO command executed by the storage device, and wherein each telemetry packet comprises:
identifying information for a type of event;
identifying information for a source of the telemetry packet;
event specific payload information;
a timestamp; and
a sequence number;
provide the plurality of telemetry packets as a telemetry stream to one or more taps via a telemetry path independent of data and control paths within the storage device;
capture at least one telemetry packet in one or more of the taps; and
generate real-time telemetry data based at least on the at least one telemetry packet during normal operation of the storage device.