[geeks] One, two, three, panic!

[Jonathan C. Patschke]

On Sun, 15 Mar 2009, Phil Stracchino wrote:

> But this time I saw it.  Reinstalled 7.1R-i386, remirrored boot, set up
> a zpool on the 9500, put /usr/ports onto it, tried to do an updatedb for
> locate, and FreeBSD fell over instantly:

Did you tune your kernel prior to setting up the zpool?  ZFS on
FreeBSD/i386 is extremely panic-prone if you don't tune first because of
the massive amount of kernel memory it can ask for on short notice.
FreeBSD/i386 tends to keep the kernel footprint slim because of the
architectural limits of the i386 memory map.

Specifically, you'll want at least these tunables in /boot/loader.conf:
   vfs.zfs.arc_min="32M"
   vfs.zfs.arc_max="256M"
   vfs.zfs.prefetch_disable=1

I also have:
   vm.kmem_size_max="1536M"
   vm.kmem_size="1536M"

These values are from an 8GB system with lots of ZFS attached, your needs
may vary.  I also don't particularly care about the performance of the ZFS
volumes (since the databases are on UFS), so I can get away with a
relatively small ARC.

vfs.zfs.prefetch_disable is an absolute must for running on i386.

> I looked around for this a bit, and it appears to be a known and fairly
> common problem.  Apparently the FreeBSD kernel auto-calculates several
> kernel parameters based on the amount of memory it finds in the box, but
> if the box has more than 1G of RAM the algorithm apparently often
> calculates values bigger than the kernel can handle.

This information sounds uselessly out-of-date.  Where did you find it?  I
can't remember the last time I installed FreeBSD on a system with -less-
than 1GB memory.

> According to the documentation I found, the alleged fix is to recompile
> the kernel with VM_KMEM_SIZE_MAX set to 400MB.

In 7.x, this doesn't need to be compiled-in, it can be set in
/boot/loader.conf.

> # sysctl -h vm.kmem_size_max
> vm.kmem_size_max: 335544320
>
> So much for that fix ... I'm already 65MB into "safe" territory.

I think you misunderstand the fix.  The FreeBSD kernel reserves a certain
amount range of physmem specifically for its use.  vm.kmem_size_max
regulates the maximum amount of memory the kernel will claim.  The panic
you got indicates that the kernel needed to allocate more than the limit.
Raising the limit from 320MB to 400MB would probably make that go away,
provided that the need is genuine or you've managed to cap whatever is
leaking.

If you were wondering, I doubt this is related to your problem running
FreeBSD/amd64, in any case.

-- 
Jonathan Patschke ( "They don't have the right to read a book out loud."
Elgin, TX         (                  --Paul Aiken
USA               (                    Executive Director, Authors Guild