[geeks] The Idea(tm)... Homebrew Terabytes Cheap

[Chris Byrne]

Jody,

First of all you work for tattered cover? Kick ass, that is the coolset book
store. I always go every time I'm in Denver, even though the parking is a
pain (or at least it was last time I was there about a year ago). I think we
had to park in the safeway down the street and walk over.

Anyway, managing a huge amount of storage isn't as simple as it seems. While
I admit they cost far more than they should, there really is a reason that
large scale storage arrays cost mucho bucks. The biggest problems here are
bus bandwidth and filesystem management.

There's just no way you can split up the limited badwidth of a. the system
bus and b. the drive controllers over that many drives efficiently. I've
tried sticking 16 drives on a single general purpose system and have it max
out at less than 5% of the sustained throughput capabilites of each drive.

And that assumes your bus functions properly with that many devices attached
to it, and that you don't exceed your cable length limit. I've never tried
putting that many devices on a single bus before, but my gut feeling is that
90+ devices (80 drives, 10 plexes, plus all the SCSI cards and any other
ancillary cards) isn't going to work.

Also you are gonna need a HONKING power supply in each box. Up until
recently I would have said that'd be a problem, but enermax just came out
with a truly kick ass 650 watt PS with two fans and 16 power taps.


The other issue is filesystem/volume management (actually add scheduling of
backups to maximize performance to that as well)
That's mostly an administrative issue on the order of "do I want to take the
admin bandwidth to handle volume and backup management manually, or do I
want to buy expensive volume management software ala veritas, or do I want
to hack an opensource tool to work in this somewhat exceptional
environment".

I went through this last year with StorageWay, and we came to the conclusion
that with all of the factors weighd in, it's actually cheaper to buy a
serious storage array than to try and build and maintain one yourself.

Now if they woyuld jsut make one that didnt cost $250k

Actually the good news is, you can often find mid range storage arrays on
EBay for pretty reasonable prices. Add a veritas license, and a machine to
control the thing, and for the same $20k you were talking about you can have
a hitachi or EMC (probably a 1 TB array rather than a 3 TB)


Chris Byrne


-----Original Message-----
From: geeks-admin at sunhelp.org [mailto:geeks-admin at sunhelp.org]On Behalf
Of Jody Stephens
Sent: Friday, April 13, 2001 13:32
To: geeks at sunhelp.org
Subject: [geeks] The Idea(tm)... Homebrew Terabytes Cheap


Here is the deal. Currently we have about 5 DDS-2 tape
drives, media we get for ~$15 a pop. Problem, we have
to keep all of the tapes for 1 year, after that we
keep month beginnings and month ends. Each machine does
about 2GB uncompressed.

So we've been trying to think of way to cut costs (we
seem to spend >$5000 yearly for tapes including about a
%10 failure rate). Some of the things I've looked at are

1. Buying cheap IDE drives, writing to them, pulling them
storing them. Actually this works out to about the same
price, but we get better reliabilty (in theory), quicker
backups and restores. But we have to deal with HD's, which
are not exactlly durable. So that is out.

2. CD-R. Alright except each machine would span more than
1 disc, creating problems if one fails. And dealing with
all those CD's doesn't sound like fun.

3. Optical. Too bloody expensive.

Sooo... My thought.
10 IDE plex at ($460 each)
80 WD 40GB drives ($130 each)
put these in a group of rackmount cases.
Buy a decent machine with a bunch of memory.
Buy a couple of good SCSI cards.

Put these together with Linux (or OS of choice) and
a couple of big slices. Bingo. For less than $20,000
I just got 3.2TB max (probably would be less as
I would want to make some hot spares and such). That
gets me ~3yrs data. If I'm feeling crazy, I put together
another one at the other store and let them mirror
(safe from fire).

This wouldn't really need to be a high performance beast,
as data would basically be dribbling in. But I guess with
more busess and some striping it could be a decent performer.
If there is a way to get some drives to sleep, then power
consumption would drop dramatically. It seems that the
same company also makes a IDE backplane which makes some
sort of hot swapping availble. WD gives a 5 year life span
to the drive, but by then I'm sure you could just buy the
new 300GB drives and stick them in (what is that... 240TB,
good lord).

For comparison Sun's A1000 gives 436GB for $22,220.00 or
a nickel a MB compared to $20,000 for this system at
6/10 of a penny per MB

Am I on crack? It just seems wrong.  Do I have some order
of magnitude wrong here?
What do you guys think of this?
--
                Jody Stephens -- (303)322-1965 x2608
                      jodys at tatteredcover.com
                     Tattered Cover Book Store
                          1628 16th Street
                         Denver, Co. 80202
                       (303)322-1965 Offices
(303)436-1070 LoDo Store         (303)629-1704 Office and LoDo Fax
(303)322-7727 Cherry Creek Store (303)399-2279 Cherry Creek Fax
_______________________________________________
GEEKS:  http://www.sunhelp.org/mailman/listinfo/geeks