[rescue] FreeBSD and ZFS, was Re: 3ware raid

[JP Hindin]

On Thu, 4 Aug 2011, N. Miller wrote:
> You need to be very careful with dedup.  IMO, I would never implement it
> without first doing extensive tests with a copy of the real data I expected to
> see in production.  You need lots of RAM on top of the normal "lots" of RAM
> required by ZFS, because the entire dedup table is kept in resident memory.
>
> Also, you can't revert without doing a dataset (or maybe pool, not sure if
> this is a pool-wide or dataset level config) rebuild.  Example: you can turn
> on compression, anything written to the dataset while compression is on will
> be compressed.  You can then turn off compression and anything written to disk
> will not be compressed, but compressed data can still be read, etc.  With
> dedup, it's ON, and you can only turn it off by sending all the data to a
> dataset with dedup OFF.


We "accidentally" ended up having de-dupe turned on in our various Nexenta
(ZFS) arrays and it caused no end of troubles with false disk failures,
data loss and all sorts of crazy stuff.
As soon as we verified the issue and turned de-dupe off the issues all
vanished and we've had a set of solid Nexentas ever since (12mos and
counting on three Nexentas with about 120T of storage between them).

I know it's anecdotal, but it was a total nightmare for us.

 - JP