[rescue] Re: [SunHELP] Help with RAID

[Robert Novak]

On Thu, 5 Jun 2003, Curtis H. Wilbar Jr. wrote:

[13 lines of quoted headers removed]
[13 lines of unnecessary body removed]

> >Fortunately, what you're trying to do is something in which Solaris
> >Disksuite knows to Do The Right Thing.  You see, the way most people
> >think of doing this is to create two stripes and then mirror them, but
> >the correct way is to create N mirrors and stripe them.  Disksuite knows
> >this, so when you ask for the former, it's bright enough to do the
> >latter.  (This, at least, is my understanding as explained to me by a
> >technical wizard in Sun's High Performance Computing group.)
>
> Ummm... everywhere I've ever done this, you create either two stripes
> which you then mirror, or two concats and then mirror.... I've never
> heard of someone concatenating a bunch of little mirrors...

Logically it makes more sense to concatenate a "bunch of little
mirrors"... think about it.

<theory mode, not SDS applied mode here>

Two 10-component stripes are mirrored. A component in one stripe dies,
that stripe is offline. A component in the other stripe dies, your
filesystem is offline. You can lose exactly one component and stay
running.

Ten 2-component mirrors are striped. Lose a component, its associated
submirror keeps going. You can lose one half of your components and keep
running (in this case, 10 components).

Sure, odds are you won't lose ten disks in the time it takes to replace
one. Unless your stripes are on their own controllers and one controller
dies.

</theory mode>

> >Here's an example of the metainit commands to do it:
> >
> >metainit d11 1 10 c1t{0,1,2,3,4,5,6,7,8,9}d0s0 -i 128k
> >metainit d12 1 10 c2t{0,1,2,3,4,5,6,7,8,9}d0s0 -i 128k
> >metainit d10 -m d11
> >metattach d10 d12
>
> This is creating a mirror of two stripes !

Yeah, it is. But Solstice Disksuite treats it as a stripe of mirrors
under certain configuration conditions.

Go read http://www.psychofx.com/chris/unix/sun/dsraid10.txt
and http://sunsolve.sun.com/pub-cgi/retrieve.pl?doc=finfodoc/45222 (you
may have to google for the cache version of the latter--just go to
google.com and put this url in the search bar to find the cached copy).

Summary:

	"If the RAID 0+1 metadevice is created such that the number of
	columns in each of the mirrored stripes is the same, and the sizes
	of the physical disk slices in each column is the same, DiskSuite
	will internally converts that metadevice to a RAID 1+0 device.
	However, no difference in it's outward appearance (for example,
	using the metastat command) will occur."

> >Although this looks like two stripes mirrored to each other, internally
> >Disksuite implements this as a stripe of ten individual two-disk mirrors.
>
> Say what ?  Is this in the docs somewhere.... I've never heard of this.
> It also runs contradictory to what you are setting up.... you are first
> creating the ODS stripes, then creating an ODS mirror.  Internally if
> it treats it as something other than this would be quite odd to handle.

You should read Brian Wong's book. Configuration and Capacity Planning for
Solaris Servers http://www.amazon.com/exec/obidos/tg/detail/-/0133499529

This is where I first saw the concept, and at one point I had an indirect
argument with Brian through a friend about it. Took me a couple of years
to find a better answer than "it just does"... but, well, it just does.

> Are you sure about this ?  I'd like to know more about it and the
> source, becasue I worked for Sun, have friends that work for Sun, have
> used Disk Suite since the SunOS 4.X days and I have never heard of this.

I worked for Sun, have friends that work for Sun, have used Disksuite
since the Solaris 2.5.1 days, and I've heard of it. None of the former
details really matter with regard to the latter.  The topic has come up
here and/or on sunhelp@ before too.

> OK... this makes sense... but I'd still think you need to create the
> 10 submirrors which you would then stripe.  I doubt ODS is going to do
> it automagically as such hiding the fact that it has done so...

That's exactly what it does. You can't add a metadevice to a stripe. You
can only add a slice to a stripe. You can add a stripe to a mirror though.
SDS hides this from you and just does it in the conditions mentioned
above.

> I question the chances of either happening before the first disk is
> replaced (at least in a production environment).

I've had three disks die in the time it took to get the fourth replaced,
and I was using a disk array that used four-drive magazines. That was not
pleasant. So while it's unlikely, it happens. And given the opportunity to
prepare for it happening, I would take that opportunity if possible. I
felt that way before I had a big bad batch of 73gb disks, but now that
I've run into it, I feel even more sure of that stance.

--Rob