[geeks] ZFS sanity check

[Shannon Hendrix]

On May 12, 2008, at 11:30 , Phil Stracchino wrote:

> Hey folks,
> I have a new server to bring up, and no prior experience with ZFS.   
> I'd like to pick your brains a little.
>
> The new box has twelve 300GB SATA disks.  I'm going to need a non- 
> ZFS boot volume for Solaris 10.  So for starters, I'm going to need  
> a slice of each of the first two disks for mirrored boot.  How much  
> disk space, realistically, am I going to need for Solaris 10?  Does  
> this have to be UFS, or are there other options?  If so, are any of  
> them better?

UFS or ZFS, unless you buy something else.

I don't believe Solaris offers a VFS layer like BSD and Linux.

My Solaris 10 box:

Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c0d0s0         17G   1.1G    16G     7%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   4.7G   756K   4.7G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/usr/lib/libc/libc_hwcap2.so.1    17G   1.1G    16G     7%    /lib/ 
libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
/dev/dsk/c0d0s3        3.9G   802M   3.1G    21%    /var
swap                   4.7G     4K   4.7G     1%    /tmp
swap                   4.7G    28K   4.7G     1%    /var/run
sys/home                50G   1.9G    44G     5%    /home
sys/local               50G   2.3G    44G     5%    /local
sys/opt                 50G   1.6G    44G     4%    /opt
sys                     50G    24K    44G     1%    /sys

That's just the first drive, to give you an idea how much each mount  
point is using.

The first partition of the root drive is UFS and that's where / and / 
var live.  I read in several places it was a bad idea to /, /usr, and / 
var on separate drives or ZFS because of the links between them.

I have not installed all of Solaris, just most of the console level  
tools, and I put the bulk of my software in /opt which lives on ZFS.

I think a full install of Solaris uses around 6.5GB, with under 1GB  
in /opt.

> That's where the questions come in.  Can I throw all the remaining  
> space on the first few disks, plus the whole of the untouched disks,  
> into one big zpool and RAIDZ2 across the lot?

Yes, but it is always better to match the chunks if you can.

> Or do I need to group physical volumes of the same sizes and add the  
> groups to the zpool?

It's much better, yes.

> I know my boot volume can't be ZFS yet.

I believe the OpenSolaris release does do root and boot on ZFS, if you  
want to try that instead of U5.

> Can my swap be on the zpool, and is it a good idea to do so?  Am I  
> better off to put the swap on separate spindles from the boot disks  
> to spread accesses, or put it on the same spindles to minimize the  
> number of disks ZFS is only partially managing?

If you think your system will be swapping a lot, then it does help to  
spread the I/O out a little.

However, when a system is paging to disk so much that drive speed  
becomes a factor, you are already running at a fraction of your  
system's full speed, so the real solution is to not need so many  
active pages on disk.

> What else should I know?  What other factors should I be aware of  
> that can affect my filesystem planning?  Is there a better way of  
> approaching the whole problem?

The best method, if you have time, is to do a full install of  
everything you want and partition the system so every base dir has its  
own mount point.

Then look at how much space is taken up and make a plan.

It sucks in a way, but it's a sure way to know what is taken.

One thing about Solaris, your base install won't every change much  
because most of the extra tools go in /opt and /local.  Put those on  
ZFS.


-- 
Shannon Hendrix
shannon at widomaker.com