summaryrefslogtreecommitdiff
path: root/docs/persistent_naming/testing_scsi_notes.txt
diff options
context:
space:
mode:
Diffstat (limited to 'docs/persistent_naming/testing_scsi_notes.txt')
-rw-r--r--docs/persistent_naming/testing_scsi_notes.txt203
1 files changed, 203 insertions, 0 deletions
diff --git a/docs/persistent_naming/testing_scsi_notes.txt b/docs/persistent_naming/testing_scsi_notes.txt
new file mode 100644
index 0000000000..8d94c842b4
--- /dev/null
+++ b/docs/persistent_naming/testing_scsi_notes.txt
@@ -0,0 +1,203 @@
+Using UDEV to do Persistent storage device naming
+for large numbers of storage devices
+3/16/2004
+
+Here are some lessons we learned at OSDL recently on how to use UDEV
+(version 021) to do persistent device naming for lots of storage devices.
+We used what was available in udev for scsi devices. Here is an outline of
+this report:
+
+Background information
+ a list of resources we needed to get started.
+Setup
+ what we needed to create the right enviroment (kernel, patches,
+ drivers)
+How udev works to assign persistent storage device names
+ what the documentation didn't tell us.
+Performance
+ A sanity test we ran to compare with and without persistent naming.
+
+
+BACKGROUND INFORMATION
+To get started, here are some references. Review the overview articles so
+that the rest of the information makes sense.
+
+Download the latest udev stuff from:
+ http://www.kernel.org/pub/linux/utils/kernel/hotplug/
+
+mailing list:
+ linux-hotplug-devel@lists.sourceforge.net
+
+Here is a nice overview article to get started (warning, this is from
+summer 2003 so many items indicated as "todo" have been done and
+configuration file name references have sometime changed):
+http://www.kroah.com/linux/talks/ols_2003_udev_paper/Reprint-Kroah-Hartman-OLS2003.pdf
+(also included when you download udev)
+
+More general info (also included in the udev package):
+ http://kernel.org/pub/linux/utils/kernel/hotplug/udev-FAQ
+UDEV version 021 Announcement:
+ http://marc.theaimsgroup.com/?l=linux-hotplug-devel&m=107827264803336&w=2
+
+"Managing Dynamic Naming":
+ http://lwn.net/Articles/28897/
+
+If you are a fan of devfs, whatever you do, don't complain until you read
+everything you possibly can about udev. This for example:
+ http://kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs
+
+You will need to create udev.rules to supply consistent names. (See
+etc/udev/udev.rules in the download). This article gives you some
+background about udev.rules, but avoids describing the "PROGRAM" key which
+is needed for our work. Read it for background: writing udev rules
+(current as of udev 018)
+ http://www.reactivated.net/udevrules.php
+
+bitkeeper tree:
+ bk://kernel.bkbits.net/gregkh/udev
+
+Libsysfs used to get sysfs information):
+ http://www-124.ibm.com/linux/papers/libsysfs/libsysfs-linuxconfau2004.pdf
+
+UDEV works using the way hotplug events are handled by the kernel.
+Several overview articles about hotplug include:
+Hotplug events
+http://lwn.net/Articles/52621/
+Overview of Hotplug
+http://linux-hotplug.sourceforge.net/
+
+Gentoo centric install info:
+http://webpages.charter.net/decibelshelp/LinuxHelp_UDEVPrimer.html
+
+rpms built against Red Hat FC2-test1 may be available at:
+http://kernel.org/pub/linux/utils/kernel/hotplug/udev-021-1.i386.rpm
+
+with the source rpm at:
+http://kernel.org/pub/linux/utils/kernel/hotplug/udev-021-1.src.rpm
+
+
+
+SETUP
+
+Here is a brief checklist of what you need on your system for this to
+work:
+
+Kernel must be a 2.6 kernel
+
+Must use CONFIG_HOTPLUG kernel config option, since the solution is based
+on hotplug capabilities.
+
+To test more than 256 scsi devices you need a patch to the scsi driver to
+support that many (available from IBM or SuSE). To see the patch we used,
+see this link:
+http://developer.osdl.org/maryedie/DCL/PSDN/lotsofdisks.patch
+
+Your storage device must support (via the driver) a unique identifier for
+persistent device naming. (Adaptec RAID device does not, for example.)
+
+Your device driver must support sysfs (new in 2.6 kernel). This is already
+done for scsi devices and most if not all block devices.
+
+A program (scsi_id) exists in the udev download (extras/scsi_id/scsi_id.c)
+for scsi devices. It can read the identifier and is needed for persistent
+naming.
+
+
+HOW UDEV WORKS TO ASSIGN PERSISTENT NAMES:
+
+There are three places where device information is stored that udev
+uses:
+(1) /sys maintained by sysfs
+(2) /etc/udev/udev.rules - where you can store the identifier to NAME
+ mapping information.
+(3) The tdb (udev-021/tdb/tdb.c), trivial data base, that is held in
+ memory and holds the valid system configuration. It is not saved
+ between one boot to the next. It is constructed at boot time and
+ updated with configuration changes.
+
+The persistent names are kept (at least this is one way to do it) in
+udev.rules (uuid and NAME), one entry per device. If you want to initially
+give your 1000 disk devices a default name and then make sure those names
+are preserved, here is how :
+
+Start with no special entry in udev.rules when do you an initial boot of
+your system with disks in place. Udev will assign default names (there
+are ways to control what you want for default too).
+
+Once the names are assigned, use a script supplied for scsi devices -
+udev-021/extras/scsi_id/gen_scsi_id_udev_rules.sh to generate the lines
+needed for udev.rules, one per device. Each line indicates the identifier
+and the NAME it was assigned. You could optionally create this manually if
+you prefer other names .
+
+[example entries in udev.rules for scsi disks]
+BUS="scsi", PROGRAM="scsi_id", RESULT="<uuid1>",NAME="<name1>"
+BUS="scsi", RESULT="<uuid2>",NAME="<name2>"
+...
+BUS="scsi", RESULT="<uuid1000>",NAME="<name1000>"
+
+(The actual file we used is the file udev.rules_1000_scsi_debug in this
+directory )
+
+Upon reboot, for each device a hotplug event occurs. The udev.rules file
+is scanned looking for the device type (BUS) in this case for "scsi". The
+first entry generated by the above program references a PROGRAM in the key
+field (scsi_id) which is called to probe the device and determine the
+unique identifier. sysfs is used to determine the major/minor number for
+the device. The result of the program execution (the uuid) is compared
+with the RESULT entry in the same udev.rules line.
+
+- If it matches, then the NAME entered on this line is used. The uuid and
+ major/minor number is saved in tdb (newly recreated upon boot). That
+ device is created in /udev (the target directory name is configurable)
+ with the assigned NAME.
+
+- If it doesn't match, the RESULT (uuid) is preserved for use on the next
+ udev.rules line as long as the bus type (scsi) is the same. So the
+ result (the uuid) is compared on the next line, and the next until a
+ match occurs.
+
+- If no match occurs, the device will be assigned a default name.
+
+- Tdb is updated with the resulting name assignment.
+
+
+Thus if the uuid and names are enumerated, they will be found, assigned,
+and are therefore permanent.
+
+If the device is removed from a live system, a hotplug event occurs, and it
+is removed from tdb and the /udev entry disappears.
+
+If it is re-inserted at a new location, the udev.rules file is scanned as
+above. The new major/minor number goes in tdb with the uuid , the name in
+udev.rules is found again, and the /udev name re-appears.
+
+
+
+PERFORMANCE
+
+Now the question becomes, how much longer does it take to scan the
+udev.rules table once there are 1000 entries?
+
+To test this, we created 1000 "scsi " devices using the scsi debug device
+driver supplied in the kernel. When this device driver is loaded you can
+specify how many fake scsi devices to create. There is no real I/O
+involved but it does respond to some scsi commands. It simulates the uuid
+by using the device number assigned when the device is created.
+
+Then we auto-generated entries into udev.rules with
+gen_scsi_id_udev_rules.sh. We then removed the devices and reassigned them
+to simulate a reboot. The delta between assigning defaults and assigning
+the names enumerated in the udev.rules file was 7 seconds (that's for 1000
+drives).
+
+Scripts utilized the feature (described above) that saves the "RESULT" key
+after one scsi-id program call for later reference with other udev.rules
+entries (so only have one PROGRAM key is the moral of the story). If you
+repeated the PROGRAM key, you would unnecessarily call the program up to
+999 times!
+
+The script that creates udev.rules did not work for 1000 drives (the input
+line is too long). We determined that a patch for this already existed but
+had not yet been checked in.
+