[rescue] ZFS file disaster

[Jonathan Patschke]

On Mon, 17 Oct 2016, Christopher Robinson wrote:

> Recently, after a bad shutdown of opensolaris, the boot archive became
> corrupted and would not bring up the graphical login.  In text mode, I
> could see all of my files and I was able to rescue the most important
> pdf files using the photorec tool. However,of around 300text articles
> only about 1/6 kept their original author/title labelling.

Is this x86 or SPARC?  Do have you snapshots of the affected filesystems?
When you say the text files are mislabeled, do you mean they're in the
wrong directory or that the contents are corrupt?

> It was suggested to me that I take out theoriginal drive, install
> opensolaris on a new drive and then reinstall the original drive. When I
> did this none of the files on the original drive showed up.

Right.  You'd need to import the old zpool.  Since it's a bootable zpool,
you need to import it at an alternate root, or it'll mount files in the
same spot as your new boot pool.

> However, immediately after doing this the system would not boot at all.

And this will happen.  If you disconnect your old disk, the system will
probably come up in single-user mode, and you can then export the
disconnected pool, but you might find a better selection of recovery tools
on FreeBSD, in the packages tree.

> I do not know if there had been a mechanical failure of the drive or
> some other issue caused by trying to mount it.

Could be that.  Could be bad RAM.  Apart from bugs in really early[0] ZFS
code, I've only had ZFS disasters on machines with dodgy RAM.  Before you
do anything else read-write on that system, let memtest86+ run on it for a
few hours.

> I am exploring data recovery options as a last resort, but these are a
> really awful rip-off.

Try this:  If your system is x86 (or amd64), install FreeBSD 10.3 or 11.0
on a scratch disk.  Let it automatically set up ZFS for you, but give your
root pool a non-default name (I usually use rootvg because I miss AIX).
You'll probably want to boot the system once and install bash and your
editor of choice (pkg install bash ; chsh -s /usr/local/bin/bash root ;
pkg install vim) before you set to work.

Once you have the base system happy, power down, connect your data disk,
boot to FreeBSD, and try this:

         zpool import -o readonly=on -R /data rpool

Replace "rpool" with whatever your pool was called under OpenSolaris.  If.
that doesn't work, try a base "zpool import" to get a list of pools
available to the system.  Give the zpool command above with the GUID of
your pool.

Once your pool is imported (read-only), use your scratch disk as a
lifeboat to carry your files off.

If you run "zpool status," you'll get a message saying you should upgrade
the pool.  Do NOT do that; the resulting pool won't mount on OpenSolaris,
and if you think the filesystem structure is dodgy, you should do as
little to it as possible.

If your system is SPARC, doing the same thing with a late release of
Solaris Express might work.


[0] OpenSolaris might count, but my corruption happened under Solaris 10u6
     or so.
-- 
Jonathan Patschke
Austin, TX
USA