Tuesday, November 9, 2010

SCSI-3 Persistent Group Reservation

In this blog series, I plan to discuss What,When and Why of interesting technologies as I learn about them. Also it is an effort to share information and any constructive comment is welcome.


To start with, let us discuss about SCSI-3 Persistent Group Reservation. One reason I choose to start with it is because I couldn't find much information about it on the internet. So, lets start.


Definitions/Abbreviations
  • SCSI : Small Computer System Interface.
  • SCSI-3 : SCSI standard 3.
  • I_T Nexus : In SCSI, I_T nexus stands for relationship between two SCSI endpoints namely, SCSI initiator and target ports.
  • PGR : Persistent Group Reservation.
  • Device Server : A logical unit processing SCSI commands.

Why SCSI-3 Persistent Group Reservation ?
  1. Like any other reservation mechanism, SCSI-3 PGR is used for managing the access of I_T nexuses to a SCSI-3 device servers. It helps us create set of I_T nexuses with specific device server access credentials. I_T nexuses within this set are allowed to perform specific set of operations on the device server while others are not permitted to perform those operations.
  2. SCSI-3 PGR extends the SCSI-2 reservations by allowing the set of permitted I_T nexuses to have more than one element in it.For e.g. consider a device server accessible through 4 I_T nexuses i.e. (I1,T1), (I2,T2), (I3,T3) and (I4,T4).The SCSI-2 reservation mechanism allows the device server to be accessible through only one nexus at a time. For instance,Permitted Nexus = { (I1,T1) };Restricted Nexuses = { (I2,T2), (I3,T3), (I4,T4) );
    However, SCSI-3 PGR allows the device server to be accessible through a group of nexuses (the G in PGR). So, following is a possible configuration:
    Permitted Nexuses = { (I1,T1), (I2,T2) };Restricted Nexuses = { (I3,T3), (I4,T4) );
  3. The PGR takes SCSI reservation one level further by keeping the reservation activities persistent across disruptive events like SCSI initiator device failure, I_T nexus loss, logical unit reset and optionally across power failure at SCSI target device.

What is SCSI-3 Persistent Group Reservation ?

SCSI-3 PGR is a logical mechanism for controlling the access to a device server. It is implemented using PERSISTENT RESERVE IN and PERSISTENT RESERVE OUT SCSI commands. These SCSI commands provide multiple sub-options like register, reserve, release, etc for achieving various access permissions.
Let us understand the basic and commonly used sub-options, We will use sg3_utils library for understanding SCSI-3 PGR with sg_persist utility program. In following examples, we would be using /dev/sdai and /dev/sdbn as two SCSI paths (nexuses) connecting to a SCSI LUN (device server).
  • Read Keys: It is a sub-option for PERSISTENT RESERVE IN command and is used reading the keys registered with the device server.
  • Read Reservations: It is another sub-option of PERSISTENT RESERVE IN command and is used for reading the current reservation, along with its type and scope, on the device.
  • Register: It is a sub-option for PERSISTENT RESERVE OUT command and used for establishing the information on an I_T nexus with the device server. The registration process establishes a key with the device server. This key is used as a reference for further operations by the nexus on device server.
  • Reservation: Having registered a key with device, the next possible action is to reserve it with required access permissions. We will be talking about different reservation types later. Reservation is once again a sub-option to PERSISTENT RESERVE OUT command and below is the required sg_persist command for reserving a device in Exclusive Access-Registrants Only type.

  • Release: This sub-option of PERSISTENT RESERVE OUT command is used to release an existing reservation. Only the I_T nexus holding the reservation is allowed to execute this command.
  • Unregister: This sub-option is used with PERSISTENT RESERVE OUT command to remove an existing registration with a device. Only a registered I_T nexus is allowed to unregister itself.
When to use SCSI-3 Persistent Group Reservation ?

SCSI-3 persistent group reservations are most commonly employed as fencing mechanisms in High-availability clusters to safegaurd against
split-brain conditions.

Some interesting and useful links

  1. SCSI-3 Primary commands standard documentation at http://www.t10.org/drafts.htm
  2. SCSI-3 Persistent reservations at http://sfdoccentral.symantec.com/sf/5.0/linux/html/vcs_install/ch_vcs_install_iofence4.html
  3. Kristien's blog on SCSI reservation on SUN cluster 3.x at http://blogs.sun.com/kristien/entry/scsi_reservations_in_sun_cluster

Coming up next

Discussion on types of SCSI-3 Persistent Group Reservations.

1 comment:

  1. Hello Praveen, can you clarify how SCSI-3 PGR works, if for example I have nine nodes that can see a shared lun, one set of three nodes forms one cluster, another set of three nodes forms another cluster and the remaining three nodes are not clustered, how does SCSI-3 PGR ensure exclusive/fenced access to say one of the three clustered nodes?

    ReplyDelete