Data access method and storage subsystem therefor
The present invention discloses a data access method and a storage subsystem implementing the method. The storage subsystem includes two storage virtualization controllers (SVC 1 , SVC 2 ), a backplane, a physical storage device (PSD), and a clock generation circuit. The clock generation circuit is mounted on the backplane and provides a dedicated clock signal to the PSD. The PSD has a single port, and the lanes of the single port are divided into two half portions, each of which is coupled to one of the two storage virtualization controllers (SVC 1 , SVC 2 ). Therefore, the present invention eliminates the need for additional intermediate components to couple the single-port PSD to the two storage virtualization controllers (SVC 1 , SVC 2 ). This leads to simplified wiring and reduced costs, and the PSD will not be affected by failures of the intermediate components, thereby enhancing the redundancy architecture of the storage subsystem.
1 . A data access method applied to a RAID subsystem ( 4 D, 4 E), wherein the RAID subsystem ( 4 D, 4 E) comprises a first storage virtualization controller (SVC 1 ) ( 10 ), a second storage virtualization controller (SVC 2 ) ( 20 ), a first switch (switch 1 ) ( 109 ), a second switch (switch 2 ) ( 209 ), a first physical storage device (PSD 1 ) ( 9 , 9 a ), a backplane ( 401 ), and a first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ), the method comprising:
installing the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) on the backplane ( 401 ), the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) being configured to provide a first dedicated clock signal to the first physical storage device (PSD 1 ) ( 9 , 9 a );
coupling a first half (half portion 1 ) ( 901 ) of a single port of the first physical storage device (PSD 1 ) ( 9 , 9 a ) to the first storage virtualization controller (SVC 1 ) ( 10 ) via the first switch (switch 1 ) ( 109 );
coupling a second half (half portion 2 ) ( 902 ) of the single port of the first physical storage device (PSD 1 ) ( 9 , 9 a ) to the second storage virtualization controller (SVC 2 ) ( 20 ) via the second switch (switch 2 ) ( 209 );
configuring the first storage virtualization controller (SVC 1 ) ( 10 ) as a user of the first physical storage device (PSD 1 ) ( 9 , 9 a );
receiving, by the first physical storage device (PSD 1 ) ( 9 , 9 a ), at least one first input/output (I/O) request from the first storage virtualization controller (SVC 1 ) ( 10 ) and processing the at least one first I/O request based on the first dedicated clock signal provided by the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a );
configuring the second storage virtualization controller (SVC 2 ) ( 20 ) as the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ); and
receiving, by the first physical storage device (PSD 1 ) ( 9 , 9 a ), at least one second I/O request from the second storage virtualization controller (SVC 2 ) ( 20 ) via the second half (half portion 2 ) ( 902 ) of the single port and processing the at least one second I/O request based on the first dedicated clock signal provided by the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ).
2 . The data access method of claim 1 , wherein the step of configuring the first storage virtualization controller (SVC 1 ) ( 10 ) as a user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) further comprises:
enabling, by the first storage virtualization controller (SVC 1 ) ( 10 ), at least one first lane coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ), wherein the at least one first lane corresponds to a lane of the first half (half portion 1 ) ( 901 ) of the single port of the first physical storage device (PSD 1 ) ( 9 , 9 a );
sending, by the first storage virtualization controller (SVC 1 ) ( 10 ), a first reset signal to the first physical storage device (PSD 1 ) ( 9 , 9 a ) to request the first physical storage device (PSD 1 ) ( 9 , 9 a ) to configure the first storage virtualization controller (SVC 1 ) ( 10 ) as its user;
verifying, by the first storage virtualization controller (SVC 1 ) ( 10 ), the at least one first lane;
sequencing, by the first storage virtualization controller (SVC 1 ) ( 10 ), the at least one first lane; and
utilizing, by the first storage virtualization controller (SVC 1 ) ( 10 ), the at least one first lane.
3 . The data access method of claim 2 , wherein the step of configuring the first storage virtualization controller (SVC 1 ) ( 10 ) as a user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) further comprises:
determining mutually, by the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ), via a redundant controller communicating interconnect ( 7 ), that the first storage virtualization controller (SVC 1 ) ( 10 ) is a primary controller of the first physical storage device (PSD 1 ) ( 9 , 9 a ).
4 . The data access method of claim 3 , wherein the redundant controller communicating interconnect ( 7 ) is selected from one of the following communication interconnect combinations, comprising: Peripheral Component Interconnect Express (PCI Express, PCI-E), Ethernet, Remote Direct Memory Access (RDMA), Fiber Channel-Arbitrated Loop (FC-AL), parallel Small Computer System Interface (parallel SCSI) operating in a target mode, and Serial Advanced Technology Attachment (SATA) operating in a target mode.
5 . The data access method of claim 1 , wherein the step of configuring the second storage virtualization controller (SVC 2 ) ( 20 ) as a user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) further comprises:
enabling, by the second storage virtualization controller (SVC 2 ) ( 20 ), at least one second lane coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ), wherein the at least one second lane corresponds to a lane of the second half (half portion 2 ) ( 902 ) of the single port of the first physical storage device (PSD 1 ) ( 9 , 9 a );
sending, by the second storage virtualization controller (SVC 2 ) ( 20 ), a second reset signal to the first physical storage device (PSD 1 ) ( 9 , 9 a ) for requesting the first physical storage device (PSD 1 ) ( 9 , 9 a ) to set the second storage virtualization controller (SVC 2 ) ( 20 ) as its user;
verifying, by the second storage virtualization controller (SVC 2 ) ( 20 ), the at least one second lane;
sequencing, by the second storage virtualization controller (SVC 2 ) ( 20 ), the at least one second lane; and
utilizing, by the second storage virtualization controller (SVC 2 ) ( 20 ), the at least one second lane.
6 . The data access method of claim 1 , wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) provides the first dedicated clock signal to the first physical storage device (PSD 1 ) ( 9 , 9 a ) in a one-to-one manner.
7 . The data access method of claim 1 , wherein the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ) do not require additional circuit components to assist in routing their clock signals to the first physical storage device (PSD 1 ) ( 9 , 9 a ).
8 . The data access method of claim 1 , further comprising that the second storage virtualization controller (SVC 2 ) ( 20 ) determines, through a monitoring mechanism, that the first storage virtualization controller (SVC 2 ) ( 10 ) is not functioning properly.
9 . The data access method of claim 8 , wherein the monitoring mechanism comprises a step of sending, by the first storage virtualization controller (SVC 1 ) ( 10 ), a confirmation signal to the second storage virtualization controller (SVC 2 ) ( 20 ) at each first time interval.
10 . The data access method of claim 8 , wherein the monitoring mechanism comprises a step of sending, by the second storage virtualization controller (SVC 2 ) ( 20 ), a query signal to the first storage virtualization controller (SVC 1 ) ( 10 ) at each second time interval and waiting for a response signal from the first storage virtualization controller (SVC 1 ) ( 10 ).
11 . The data access method of claim 1 , wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) is a component directly mounted on the backplane ( 401 ).
12 . The data access method of claim 1 , wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) is coupled to a first socket on the backplane ( 401 ).
13 . The data access method of claim 12 , wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) is coupled to the first socket on the backplane ( 401 ) via a first interposer.
14 . The data access method of claim 1 , wherein the first storage virtualization controller (SVC 1 ) ( 10 ), the second storage virtualization controller (SVC 2 ) ( 20 ), the first switch (switch 1 ) ( 109 ), and the second switch (switch 2 ) ( 209 ) are separate and independent circuitries, respectively.
15 . The data access method of claim 1 , wherein the first switch (switch 1 ) ( 109 ) and the second switch (switch 2 ) ( 209 ) are respectively integrated into the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ).
16 . The data access method of claim 1 , wherein the first switch (switch 1 ) ( 109 ) and the second switch (switch 2 ) ( 209 ) are integrated into a single switch, through which the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ) are respectively coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ).
17 . The data access method of claim 1 , wherein, during a process of switching the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) from the first storage virtualization controller (SVC 1 ) ( 10 ) to the second storage virtualization controller (SVC 2 ) ( 20 ), the first physical storage device (PSD 1 ) ( 9 , 9 a ) continues to process the at least one first I/O request issued by the first storage virtualization controller (SVC 1 ) ( 10 ) based on the first dedicated clock signal provided by the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ).
18 . The data access method of claim 1 , wherein, during a process of switching the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) from the first storage virtualization controller (SVC 1 ) ( 10 ) to the second storage virtualization controller (SVC 2 ) ( 20 ), the first physical storage device (PSD 1 ) ( 9 , 9 a ) stops executing the at least one first I/O request issued by the first storage virtualization controller (SVC 1 ) ( 10 ).
19 . The data access method of claim 1 , wherein the RAID subsystem ( 4 D, 4 E) further comprises a second physical storage device (PSD 2 ) ( 9 , 9 b ) and a second clock generation circuit (clock generation circuit 2 ) ( 85 , 85 b ), the method further comprising:
installing the second clock generation circuit (clock generation circuit 2 ) ( 85 , 85 b ) on the backplane ( 401 ), the second clock generation circuit (clock generation circuit 2 ) ( 85 , 85 b ) being configured to provide a second dedicated clock signal to the second physical storage device (PSD 2 ) ( 9 , 9 b );
coupling a first half (half portion 1 ) ( 901 ) of a single port of the second physical storage device (PSD 2 ) ( 9 , 9 b ) to the second storage virtualization controller (SVC 2 ) ( 20 ) via the second switch (switch 2 ) ( 209 );
coupling a second half (half portion 2 ) ( 902 ) of the single port of the second physical storage device (PSD 2 ) ( 9 , 9 b ) to the first storage virtualization controller (SVC 1 ) ( 10 ) via the first switch (switch 1 ) ( 109 );
configuring the second storage virtualization controller (SVC 2 ) ( 20 ) as a user of the second physical storage device (PSD 2 ) ( 9 , 9 b );
receiving, by the second physical storage device (PSD 2 ) ( 9 , 9 b ), at least one second I/O request from the second storage virtualization controller (SVC 2 ) ( 20 ) via the first half (half portion 1 ) ( 901 ) of the single port, and processing the at least one second I/O request based on the second dedicated clock signal provided by the second clock generation circuit (clock generation circuit 2 ) ( 85 , 85 b );
sending, by the first storage virtualization controller (SVC 1 ) ( 10 ), a reset signal to the first switch (switch 1 ) ( 109 ), thereby configuring the user of the second physical storage device (PSD 2 ) ( 9 , 9 b ) as the first storage virtualization controller (SVC 1 ) ( 10 ); and
processing, by the second physical storage device (PSD 2 ) ( 9 , 9 b ), at least one third I/O request issued by the first storage virtualization controller (SVC 1 ) ( 10 ) via lanes of the second half (half portion 2 ) ( 902 ), based on the second dedicated clock signal provided by the second clock generation circuit (clock generation circuit 2 ) ( 85 , 85 b ).
20 . The data access method of claim 19 , wherein the second clock generation circuit (clock generation circuit 2 ) ( 85 , 85 b ) provides the second dedicated clock signal to the second physical storage device (PSD 2 ) ( 9 , 9 b ) in a one-to-one manner.
21 . A RAID (redundant array of independent disks) subsystem ( 4 D, 4 E), which is coupled to a host ( 15 ) and is configured to receive and process at least one access request from the host ( 15 ), the RAID subsystem ( 4 D, 4 E) comprising:
a first storage virtualization controller (SVC 1 ) ( 10 ) comprising a first switch (switch 1 ) ( 109 );
a second storage virtualization controller (SVC 2 ) ( 20 ) comprising a second switch (switch 2 ) ( 209 ) and communicating with the first storage virtualization controller (SVC 1 ) ( 10 ) via a redundant controller communicating interconnect ( 7 );
a first physical storage device (PSD 1 ) ( 9 , 9 a ) comprising a single port, wherein lanes of the single port are divided into a first half (half portion 1 ) ( 901 ) and a second half (half portion 2 ) ( 902 ), wherein the first half (half portion 1 ) ( 901 ) is coupled to the first switch (switch 1 ) ( 109 ) of the first storage virtualization controller (SVC 1 ) ( 10 ), and the second half (half portion 2 ) ( 902 ) is coupled to the second switch (switch 2 ) ( 209 ) of the second storage virtualization controller (SVC 2 ) ( 20 ); and
a first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) being coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ) for providing a first dedicated clock signal to the first physical storage device (PSD 1 ) ( 9 , 9 a );
wherein when the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ) determine, via the redundant controller communicating interconnect ( 7 ), that the first storage virtualization controller (SVC 1 ) ( 10 ) is a user of the first physical storage device (PSD 1 ) ( 9 , 9 a ), the first storage virtualization controller (SVC 1 ) ( 10 ) configures itself as the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) and sends at least one first input/output (I/O) request to the first physical storage device (PSD 1 ) ( 9 , 9 a ) through the first switch (switch 1 ) ( 109 ), wherein the first physical storage device (PSD 1 ) ( 9 , 9 a ) processes the at least one first I/O request based on the first dedicated clock signal provided by the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ); and
wherein when the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) switches to the second storage virtualization controller (SVC 2 ) ( 20 ), the second storage virtualization controller (SVC 2 ) ( 20 ) configures itself as the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) and sends at least one second input/output (I/O) request to the first physical storage device (PSD 1 ) ( 9 , 9 a ) through the second switch (switch 2 ) ( 209 ), wherein the first physical storage device (PSD 1 ) ( 9 , 9 a ) processes the at least one second I/O request based on the first dedicated clock signal provided by the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ).
22 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first storage virtualization controller (SVC 1 ) ( 10 ) further comprises:
a first central processing circuitry (CPC 1 ) ( 103 ) for processing the at least one access request from the host ( 15 ); and
a first device-side input/output (I/O) device interconnect controller (device-side I/O device interconnect controller 1 ) ( 107 ) for receiving messages and commands from the first central processing circuitry (CPC 1 ) ( 103 ), and being coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ) via the first switch (switch 1 ) ( 109 ) to serve as an interface and a buffer between the first physical storage device (PSD 1 ) ( 9 , 9 a ) and the first central processing circuit (CPC 1 ) ( 103 ).
23 . The RAID subsystem ( 4 D, 4 E) of claim 22 , wherein the first switch (switch 1 ) ( 109 ) is integrated into the first device-side I/O device interconnect controller (device-side I/O device interconnect controller 1 ) ( 107 ).
24 . The RAID subsystem ( 4 D, 4 E) of claim 22 , wherein the first switch (switch 1 ) ( 109 ) and the first device-side I/O device interconnect controller (device-side I/O device interconnect controller 1 ) ( 107 ) are integrated into the first central processing circuitry (CPC 1 ) ( 103 ).
25 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the second storage virtualization controller (SVC 2 ) ( 20 ) further comprises:
a second central processing circuitry (CPC 2 ) ( 203 ) for processing the at least one access request from the host ( 15 ); and
a second device-side input/output (I/O) device interconnect controller (device-side I/O device interconnect controller 2 ) ( 207 ) for receiving messages and commands from the second central processing circuitry (CPC 2 ) ( 203 ), and being coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ) via the second switch (switch 2 ) ( 209 ) to serve as an interface and a buffer between the first physical storage device (PSD 1 ) ( 9 , 9 a ) and the second central processing circuitry (CPC 2 ) ( 203 ).
26 . The RAID subsystem ( 4 D, 4 E) of claim 25 , wherein the second switch (switch 2 ) ( 209 ) is integrated into the second device-side I/O device interconnect controller (device-side I/O device interconnect controller 2 ) ( 207 ).
27 . The RAID subsystem ( 4 D, 4 E) of claim 25 , wherein the second switch (switch 2 ) ( 209 ) and the second device-side I/O device interconnect controller (device-side I/O device interconnect controller 2 ) ( 207 ) are integrated into the second central processing circuitry (CPC 2 ) ( 203 ).
28 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the redundant controller communicating interconnect ( 7 ) is selected from one of the following communication interconnect combinations, comprising: Peripheral Component Interconnect Express (PCI Express, PCI-E), Ethernet, Remote Direct Memory Access (RDMA), Fiber Channel-Arbitrated Loop (FC-AL), parallel Small Computer System Interface (parallel SCSI) operating in a target mode, and Serial Advanced Technology Attachment (SATA) operating in a target mode.
29 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) provides the first dedicated clock signal to the first physical storage device (PSD 1 ) ( 9 , 9 a ) in a one-to-one manner.
30 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ) do not require additional circuit components to assist in routing their clock signals to the first physical storage device (PSD 1 ) ( 9 , 9 a ).
31 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first storage virtualization controller (SVC 1 ) ( 10 ), the second storage virtualization controller (SVC 2 ) ( 20 ), the first switch (switch 1 ) ( 109 ), and the second switch (switch 2 ) ( 209 ) are separate and independent circuitries, respectively.
32 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first switch (switch 1 ) ( 109 ) and the second switch (switch 2 ) ( 209 ) are respectively integrated into the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ).
33 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first switch (switch 1 ) ( 109 ) and the second switch (switch 2 ) ( 209 ) are integrated into a single switch, through which the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ) are respectively coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ).
34 . The RAID subsystem ( 4 D, 4 E) of claim 21 , further comprising a backplane ( 401 ), wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) is directly mounted on the backplane ( 401 ) and coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ) via the backplane ( 401 ).
35 . The RAID subsystem ( 4 D, 4 E) of claim 21 , further comprising a backplane ( 401 ), wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) is coupled to a first socket on the backplane ( 401 ) and coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ) via the backplane ( 401 ).
36 . The RAID subsystem ( 4 D, 4 E) of claim 35 , wherein the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) is coupled to the first socket on the backplane ( 401 ) via a first interposer.
37 . The RAID subsystem ( 4 D, 4 E) of claim 21 , further comprising a first interposer, the first interposer comprising the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ) and a slot, the slot being configured to accommodate the first physical storage device (PSD 1 ) ( 9 , 9 a ), thereby enabling the first physical storage device (PSD 1 ) ( 9 , 9 a ) to be coupled to the first storage virtualization controller (SVC 1 ) ( 10 ) and the second storage virtualization controller (SVC 2 ) ( 20 ).
38 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein, when the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) is switched from the first storage virtualization controller (SVC 1 ) ( 10 ) to the second storage virtualization controller (SVC 2 ) ( 20 ), the first physical storage device (PSD 1 ) ( 9 , 9 a ) continues to process the at least one first I/O request from the first storage virtualization controller (SVC 1 ) ( 10 ) based on the first dedicated clock signal provided by the first clock generation circuit (clock generation circuit 1 ) ( 85 , 85 a ).
39 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein, when the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) is switched from the first storage virtualization controller (SVC 1 ) ( 10 ) to the second storage virtualization controller (SVC 2 ) ( 20 ), the first physical storage device (PSD 1 ) ( 9 , 9 a ) stops executing the at least one first I/O request from the first storage virtualization controller (SVC 1 ) ( 10 ).
40 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the first storage virtualization controller (SVC 1 ) ( 10 ) configures itself as the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) by performing the following steps: enabling at least one first lane coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ); sending a first reset signal to the first physical storage device (PSD 1 ) ( 9 , 9 a ) to request the first physical storage device (PSD 1 ) ( 9 , 9 a ) to configure the first storage virtualization controller (SVC 1 ) ( 10 ) as its user; verifying the at least one first lane; sequencing the at least one first lane; and utilizing the at least one first lane, wherein the at least one first lane corresponds to a lane of the first half (half portion 1 ) ( 901 ) of the single port of the first physical storage device (PSD 1 ) ( 9 , 9 a ).
41 . The RAID subsystem ( 4 D, 4 E) of claim 21 , wherein the second storage virtualization controller (SVC 2 ) ( 20 ) configures itself as the user of the first physical storage device (PSD 1 ) ( 9 , 9 a ) by performing the following steps: enabling at least one second lane coupled to the first physical storage device (PSD 1 ) ( 9 , 9 a ); sending a second reset signal to the first physical storage device (PSD 1 ) ( 9 , 9 a ) to request the first physical storage device (PSD 1 ) ( 9 , 9 a ) to configure the second storage virtualization controller (SVC 2 ) ( 20 ) as its user; verifying the at least one second lane; sequencing the at least one second lane; and utilizing the at least one second lane, wherein the at least one second lane corresponds to a lane of the second half (half portion 2 ) ( 902 ) of the single port of the first physical storage device (PSD 1 ) ( 9 , 9 a ).