Free Software Linux

Customers are always looking for ways to get their cost of Linux deployments down lower, and make management easier on their staff. One of, at least in my opinion, the best options they have is to get rid of 3^rd party multi path IO solutions for your SAN and disk management.

I was at one of my customers the other day helping them set up MPIO that is built into SLES 10. While I was there I took a few notes for what we did to get things working for their environment. These same instructions should work with other SAN’s that can handle multi path IO.

SLES 10 supports a lot of SAN’s right out of the box and automatically detects them so you don’t really need an /etc/multipath.conf. My customer likes to be able to change the black list for various types of hardware they use and wanted user-friendly names. To do this I created a multipath.conf for them that looked like the following…

## /etc/multipath.conf file for SLES 10
## You may find a full copy of this file, with comments, here..
## /usr/share/doc/packages/multipath-tools/multipath.conf

# Setup user friendly names

# name : user_friendly_names
# scope : multipath
# desc : If set to “yes”, using the bindings file
# /var/lib/multipath/bindings to assign a persistent and
# unique alias to the multipath, in the form of mpath<n>.
# If set to “no” use the WWID as the alias. In either case
# this be will be overriden by any specific aliases in this
# file.
# values : yes|no
# default : no

defaults {
user_friendly_names yes

}

# Setup the blacklisted devices….

# name : blacklist
# scope : multipath & multipathd
# desc : list of device names that are not multipath candidates
# default : cciss, fd, hd, md, dm, sr, scd, st, ram, raw, loop
#

blacklist {
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
devnode “^hd[a-z][[0-9]*]”
devnode “^cciss!c[0-9]d[0-9]*[p[0-9]*]”

}

If your curious about what platforms that SLES 10 supports out of the box a list is in the SLES documentation.

Assuming you already have your LUNs assigned to you. And once you have the /etc/multipath.conf file there are a few services that need to be started to make all this work.

# service boot.multipth start
# service multipathd start

That should start the demons and load kernel modules that you need. To check that do an lsmod to see if you see dm_multipath and multipath. Once that is done you can check your setup to see if it is correct…

# multipath –v2 -d

create: mpath10 (360080480000290100601544032363831) EMC,SYMMETRIX

[=200G][features=0][hwhandler=0]

\_ round-robin 0 [prio=4][undef]

\_ 11:0:0:39 sdbr 68:80 [undef][ready]

\_ 11:0:1:39 sdcc 69:0 [undef][ready]

\_ 10:0:0:39 sdl 8:176 [undef][ready]

\_ 10:0:1:39 sdw 65:96 [undef][ready]

create: mpath11 (360080480000290100601544032363832) EMC,SYMMETRIX

[=400G][features=0][hwhandler=0]

\_ round-robin 0 [prio=4][undef]

\_ 11:0:0:40 sdbs 68:96 [undef][ready]

\_ 11:0:1:40 sdcd 69:16 [undef][ready]

\_ 10:0:0:40 sdm 8:192 [undef][ready]

\_ 10:0:1:40 sdx 65:112 [undef][ready]

That is what it looks like for the EMC Symmetrix I was working with so your mileage may vary.

Once you have the devices showing up correctly you need to make sure the multi path modules load on reboot. To do that run the following commands…

# chkconfig multipathd on
# chkconfig boot.multipath on

The next thing is to configure LVM to scan these devices so you can use them in your volume groups. To do this you will need to edit /etc/lvm/lvm.conf in the following places…

filter = [ “a|/dev/disk/by-id/.*|”, “r|.*|” ]types = [ “device-mapper”, 253 ]

Above limits the devices that LVM will scan to only devices that show up by-id. If your using LVM to manage other disks that are not in that directory, think local scsi drives, you will need to make sure those are still available by adjusting your filter more like this…

filter = [ “a|/dev/disk/by-id/.*|”, “a|/dev/sda1$/”, “r|.*|” ]

Once that is done do a lvmdiskscan to get LVM to see the new drives.

A few other things that customers often ask for is how to have SLES scan for new LUNs on the san without rebooting. With SLES10 it’s s simple as passing a few parameters to the sys file system.

# echo 1 > /sys/class/fc_host/host<number>/issue_lip

That will make the kernel aware of the new devices at a very low level, but the devices are not yet usable. To make them usable do the following…

# echo “- - -” > /sys/class/scsi_host/host<number>/scan

That will scan all devices and add the new ones for you. All of this information is in the SLES 10 Storage Administration Guide, including various ways to recover from issues.

Also since SP1 SLES10 has been able to boot a mpio device from the SAN. The doc for doing that in SP1 is located here.

Have fun and enjoy..

You have already tagged this post. Your tags: