SCSI-3 Persistent Group Reservation

In this blog series, I plan to discuss What,When and Why of interesting technologies as I learn about them. Also it is an effort to share information and any constructive comment is welcome.


To start with, let us discuss about SCSI-3 Persistent Group Reservation. One reason I choose to start with it is because I couldn't find much information about it on the internet. So, lets start.


Definitions/Abbreviations

• SCSI : Small Computer System Interface.
• SCSI-3 : SCSI standard 3.
• I_T Nexus : In SCSI, I_T nexus stands for relationship between two SCSI endpoints namely, SCSI initiator and target ports.
• PGR : Persistent Group Reservation.
• Device Server : A logical unit processing SCSI commands.

Why SCSI-3 Persistent Group Reservation ?

1. Like any other reservation mechanism, SCSI-3 PGR is used for managing the access of I_T nexuses to a SCSI-3 device servers. It helps us create set of I_T nexuses with specific device server access credentials. I_T nexuses within this set are allowed to perform specific set of operations on the device server while others are not permitted to perform those operations.
2. SCSI-3 PGR extends the SCSI-2 reservations by allowing the set of permitted I_T nexuses to have more than one element in it.For e.g. consider a device server accessible through 4 I_T nexuses i.e. (I1,T1), (I2,T2), (I3,T3) and (I4,T4).The SCSI-2 reservation mechanism allows the device server to be accessible through only one nexus at a time. For instance,Permitted Nexus = { (I1,T1) };Restricted Nexuses = { (I2,T2), (I3,T3), (I4,T4) );
   However, SCSI-3 PGR allows the device server to be accessible through a group of nexuses (the G in PGR). So, following is a possible configuration:Permitted Nexuses = { (I1,T1), (I2,T2) };Restricted Nexuses = { (I3,T3), (I4,T4) );
3. The PGR takes SCSI reservation one level further by keeping the reservation activities persistent across disruptive events like SCSI initiator device failure, I_T nexus loss, logical unit reset and optionally across power failure at SCSI target device.

What is SCSI-3 Persistent Group Reservation ?

SCSI-3 PGR is a logical mechanism for controlling the access to a device server. It is implemented using PERSISTENT RESERVE IN and PERSISTENT RESERVE OUT SCSI commands. These SCSI commands provide multiple sub-options like register, reserve, release, etc for achieving various access permissions.

Let us understand the basic and commonly used sub-options, We will use sg3_utils library for understanding SCSI-3 PGR with sg_persist utility program. In following examples, we would be using /dev/sdai and /dev/sdbn as two SCSI paths (nexuses) connecting to a SCSI LUN (device server).

• Read Keys: It is a sub-option for PERSISTENT RESERVE IN command and is used reading the keys registered with the device server.
• Read Reservations: It is another sub-option of PERSISTENT RESERVE IN command and is used for reading the current reservation, along with its type and scope, on the device.
  
• Register: It is a sub-option for PERSISTENT RESERVE OUT command and used for establishing the information on an I_T nexus with the device server. The registration process establishes a key with the device server. This key is used as a reference for further operations by the nexus on device server.
  

• Reservation: Having registered a key with device, the next possible action is to reserve it with required access permissions. We will be talking about different reservation types later. Reservation is once again a sub-option to PERSISTENT RESERVE OUT command and below is the required sg_persist command for reserving a device in Exclusive Access-Registrants Only type.
  

• Release: This sub-option of PERSISTENT RESERVE OUT command is used to release an existing reservation. Only the I_T nexus holding the reservation is allowed to execute this command.
  

• Unregister: This sub-option is used with PERSISTENT RESERVE OUT command to remove an existing registration with a device. Only a registered I_T nexus is allowed to unregister itself.
  

When to use SCSI-3 Persistent Group Reservation ?

SCSI-3 persistent group reservations are most commonly employed as fencing mechanisms in High-availability clusters to safegaurd against split-brain conditions.

Some interesting and useful links

1. SCSI-3 Primary commands standard documentation at http://www.t10.org/drafts.htm
2. SCSI-3 Persistent reservations at http://sfdoccentral.symantec.com/sf/5.0/linux/html/vcs_install/ch_vcs_install_iofence4.html
3. Kristien's blog on SCSI reservation on SUN cluster 3.x at http://blogs.sun.com/kristien/entry/scsi_reservations_in_sun_cluster

Coming up next

Discussion on types of SCSI-3 Persistent Group Reservations.