[SunRescue] SPARCserver+SSA Questions (lots :)

[sparcy]

Hi Bjorn,

First of all, thanx for this excellent reply ! It's just the info I've been
looking for !

> > Now I've been trying to find some info, both on the Server as
> > on the Array but still haven't been able to find much. The only thing that came
> > close was some info on Sparc Clusters (which seem to be just the same just multiple
> > servers with multiple array's hooked up together) on http://docs.sun.com.
> 
> You are practicly referring to the SPARCcluster 1000PDB, which indeed
> are 2 SPARCserver 1000(E) with each SPARCstorage Array 100(/110),
> connected together to form a kind cluster.
> I have some broschures covering this setup somewhere... Aah, now when I
> think of it;
> 
> check
> http://www.eagle.y.se/~ramq/sunpics/SPARCcluster_1000PDB_1.jpg
> ...to
> http://www.eagle.y.se/~ramq/sunpics/SPARCcluster_1000PDB_4.jpg
> 
> and
> http://www.eagle.y.se/~ramq/sunpics/SPARCserver_1000E_1.jpg
> ...to
> http://www.eagle.y.se/~ramq/sunpics/SPARCserver_1000E_4.jpg
> 
> And sorry for the huge filesizes. These are just raw (unedited) scans
> which eventually will show up at www.sunhelp.org hopefully someday.

Excellent pictures, I love 'm ! I'm gonna print these out and hang 'm above
my bed :)
 
> > 1. I'd love to get a manual for this thing (either online or printed), is there a
> > manual online at all ? (I can't even find the specs of eiter on www.sun.com),
> > does sun deliver just a manual for a particular (old) system, and how much would
> > this approx. cost ? - or even I only need a couple of pages I guess, anybody has
> > a manual and would be kind enough to photocopy or scan some pages (although
> > I'm not sure if this is okay, copyright wise) - or sell it to me ?
> 
> I guess someone internally @ Sun could jack the document of their
> Intranet, but the chance is almost nonexcistent.
> You could always try calling up Sun and they could probably sell you a
> hardcopy of the manual, but that would sure cost you some $$$.
> Nobody even want the User Manuals anyway, since they only try to explain
> how to plug in the power cable and where you find the 'power on' button,
> all along in 300+ pages.
> 
> If it's technical/service specs your after, look for the FEH (Field
> Engineer Handbook).

Yeah, I was afraid this was gonna be hard, I guess the best shot is to check
out some book stores which I know once in a while have some old books on
various computer systems.

> > 2. The sparc server has 4 'drawers' (basically like your clothing drawer, but
> > smaller, at the back, you can take them out by pulling the plastic levers)
> > 
> > - The upper drawer has the monitor ouput and the fiber output (which connects
> >   to the array), a standard A and B port, a SCSI port, a keyboard port and a UTP (TP) port.
> 
> This is the 1st system board. Probably equipped with 2 CPUs and some
> memory.
> (The UTP is 10Mbit Ethernet, and the first serial-port makes the
> console-port)
> 
> > - The 2nd drawer from above has another A and B port, another keyboard port, 2 UTP ports,
> >   and a scsi port
> 
> This is the 2nd system board, which probably makes this a 4-way system.
> Sounds like a fine machine.
> What I don't know, is that second UTP port. Could be 100Mbit interface,
> I don't know.
> Carefully slide the board out and check for and part-number on that
> Sbus-board.
> (xxx-xxxx digits)

This might indeed be 100Mbit, I don't have the hardware here to check
it (only have a simple 10Mbit subnet) but I'll try to take fluke lanmeter
from work and see if it can tell me anything. I could't find a digit
in the xxx-xxxx form (is there a specific place where I should look ?)
I looked at the chipset and this seems te be a "Player DP83257VF", an
additional sticker says 650-0310-01 rev 1.0u1. A search on that sticker number
didn't bring anything but a search on the DP83257VF got me to the
National Semiconductor site which had a datasheet on this chip, saying
it's a "Player & Device Enhanced FDDI Physical Layer Controller"
right ..... err .... so what the heck is that then ?
(http://207.82.57.10/pf/DP/DP83257.html#General Description)
It doesn't tell me much in first instance so I'm gonna read up on
this chip and try to figure out what it does.

Another thing that struck me was that both system board 1 and 2 have both
4 cpu's on it, so there should be 8 cpu's in total! (I know this arch
can support 8, but when linux boots it only tells me it has 4), I know
that if I boot my SparcStation20 with "boot -v" it tells me which
cpu it sees (e.g. counting them) - but booting this baby with the -v
switch tells me a lot - just not much about the cpu's. Is there another
way to get this info from the prom ? (I guess I need to look at the
raw device node's ... right ?)

> > - The 3rd drawer has 3 scsi ports, IN, OUT and one without a label.
> 
> That's the 'disc expansion board'. Simply a dumb board with connectors
> to the backplane XDBus, providing the necessary +5V and +12V (and GND)
> to the drives. The 'IN'-port sure is SCSI-IN, where you route a cable
> from the 1st (or 2nd) systemboards SCSI-connector. Put a SCSI-terminator
> at the SCSI-OUT port. Or, if you choose to route the cable even further,
> put the unused SCSI-terminator at the unlabeled port. :-)

Okay, that was what I thought, so I just need to buy a external scsi cable
and plug it between board 1/2 and 3 right ?
 
> > - The 4th drawer is just a platic front.
> 
> And therefore; nothing. :-)

Well, the online manual says (great - and this is just *not* the info I was looking
for :) that it's important to put this in to keep a good airflow inside ...
(but I guess it's more there just that you don't have a gaping hole in the
back of your server :_)
 
> > 3. As mentioned, the first and second drawer have 'knight-rider' (you know,
> > the silly series with that black car 'kit' and that what-do-you-mean-I-can't-act?
> > david hasselhoff) these are green and yellow and go up and down. This looks awfully
> > cool but I guess they must have a purpose (I can't imagine sun constructor's going:
> > yeah - that looks cool! - we don't need a purpose for this! - It just look cool! :)
> > so what are they ?
> 
> The first 2 green lights is sitting there just telling you if either CPU
> is working or not. (Green LED off = nonfunctional or non-excistent CPU)
> The other ones just have the same function as on the 4/300, 4/400 and
> 4/600 CPU's (and probably some Sun3s); activity-LEDS - in a very cool
> way. :-)
> I haven't found any practical use for these, other than they tend to
> slow down when the system load goes up, and provides necessary
> diagnostic information.

Cool! So they actually *do* have a meaning ! It's shame they are at the back
tho, I've caught myself many times this week hanging my head over the server
and looking at those neat lights :-)

Excellent tho, I'm gonna have a whack at running 12 compilers and cranking
up the load av. just to see how they react.
 
> > 4. On to the sparc array, I think this thing can hold up to 30 disks, there are currently
> > (what seems 12 disks installed in it, I guess 2.1Gig each) - The problem is that I can't
> > completely pull out any of the slots (there are 3) which hold the harddisks, I can only
> > pull it out halfways and then it's stuck (the same for all slots), So I can't verify
> > if there are more harddisks in the back of the slot (it's really hard to see). There
> > must be a trick to pull out these slots completely (given that in an emergency one
> > would need to replace a disk at the very back of a slot) - how does on do this ?
> 
> Pull the handle out and slide the tray out fully. Carefully push it back
> just a liiiittle bit, and lift it straight up. (not OUT) Check under the
> tray. There should be a some sort of "clamp" there, which you can push.
> Push that little metallic clamp, and meanwhile slide the tray out
> completely. Viola.

Indeed. That was the trick, there's indeed a rather large metal plate at the
bottom of the drawers and one can pull it in for about .5 cm and then you
can just pop it out.
 
Thanx for this tip! I would never have figured this out.

> > The ------ line means solid, and -_-_-_ line means blinking, so the 2nd harddisk from
> > the back (and I'm guessing there's a harddisk there - since I can see it, see
> > question 4) seems to have a problem ? yes ? Anybody know what 6229 means and
> > what blinking or not blinking means ?
> > 
> > Also there's a LED at the back of the array ('sys ok') which is blinking (?)
> 
> This is normal functionality.
> Although, that blinking activity should mean that it's spinning up the
> drives?
> Either way, I haven't got enough experience with these arrays to totally
> answer that question. I guess you'll have to find the "Using the
> SPARCstorage Array 2.3 AnswerBook" on docs.sun.com, under "Hardware" ->
> 2Storage", at the bottom.

This is kind of wierd ; indeed they all blink when you power-on, then
each drive spins up (you can actually hear this, just like a tiny
jet engine powering up) and after a couple of minutes all lines are
solid except those 3 ones (they keep blinking). At least that was
until yesterday, now only 2 lines keep blinking (the most left and
right) ... maybe there's just something wrong with these drives
although it's strange that they are all in one line ...

That number, 6229, means (at least that's what I gathered) from the
online docs just the array ID, so nothing scary about that number,
as far as finding out what the blinking lines and the blinking
led at the back of the array mean, still no luck, I looked at
the document you pointed out but there's nothing about either
the LCD (well some info but not much) or about the LED at the
back ... I'm gonna keep looking for this info.
 
> These SSA's weren't designed to scroll the text "there seems to be some
> trouble with my 29th drive, please check.", but instead just yell
> something "1234!!!" into your face, leaving you there with a funny
> questionmark above your head. :-)
> Infact, all the error-handling goes through the FC-interface, to some
> intelligent software on the host-side.

Now you mention it ... wouldn't it be cool, scrolling text ?

"warning ... harddisk 4 is on fire, possible meltdown, please shutdown now"

But I get your point, I don't mind getting a number, but having
a table with the corresponding explanation would be even nicer :)
 
> > 6. I've been trying to find some info on the commmunication protocol for talking between
> > the server and the array (the fiber optic cable) - this seems to be Solaris specific,
> > it seems that only solaris is able to communicate with the array through specific
> > software (which is probably still installed on the system) - is this true ? Or is
> > there some way to access the array as a 'virtual' scsi bay ? (this is probably a
> > silly question, but I clueless to how this works)
> 
> To keep things short; this is no hardware-RAID box.
> 
> Some facts:
> The array consists of either a 40MHz MicroSPARC (the 100/102 model), or
> a 110MHz MicroSPARC-II (the 110/112/114 model) which cares for the
> connection between the FC-AL interface, and the SCSI-buses.
> Yes. Buses. Six Fast/Wide SCSI-buses divided into 5 slots each,
> providing upto 30 drives. (8GB capacity is maximum for these arrays) The
> last number just mention the capacity of the drives, in gigabytes.
> 
> All RAID/striping/mirroring is handled in software. Either by the
> "Solstice Disksuite" (crap) or the "SPARCstorage Volume Manager"
> (better). Either way, find one of these and you have a perfect setup,
> and you're able to push the limits of the box through spreading data
> along all SCSI-channels.
> 
> The communications protocol is Fibre Channel. And no, that's not exactly
> the same FC as today, just a predecessor of the same standard, running
> at 1/4th the speed of todays; 25MB/s. Probably enough by that time of
> manufacture, but sluggish by comparing with todays Fibre Channel. (And
> probably missing some features)
> 
> The nice thing about the FC-interface, is that you have a maximum of 500
> meters of cable to deal with, which means you could place all your
> arrays in an entirely different room, isolated from the rest in case of
> fire and such.
> That probably doesn't mind you, but a nice fact. :-)
> Another nice feature is that you can have dual GBICs (fiber modules) on
> both the host communication card, and the SSA, providing redundancy. A
> nice thing that you can run two independent arrays on one host card.
> Perfect setup for mass storage, if you're running out of Sbus expansion
> slots.
> (Imagine 30x8GB, on two arrays, in just one Sbus slot)
> Ofcourse, that shouldn't be a big problem in a SS1000 with 4 system
> boards, but for smaller configurated servers like SS 5/10/20, or Ultra
> 1.

Wow. Thanks for the explanation, great to get some insight on the technical
details. I looked at the processor in the array and it's a MicroSPARC-II
(STP 1012PGA, MB 86904 to be precise) - so this must be a 110/112/114 model
then right ? (btw is there a mark/sticker somewhere which tells me what this
is exactly (e.g. 110, 112 or 114) - I've looked in various places but
still haven't been able to find it. I've never played around with FC, but
it certainly looks good, I gathered there's even an ANSI standard for
it (and there aren't that many ANSI compliant standards for hardware
communication protocols) so I'm gonna look a little closer at this,
I find it quite remarkable that Sun already had this kind of technolage
in the early nineties actually.

As far as software support, yeah I read something about Solstice,
I haven't come to much of the software side of things yet, still
figuring out most of the hardware.
 
> > 7. The last question (finally ;-) - I gathered that support from other OS's for sun4d
> > arch's is pretty much lacking (Netbsd, OpenBSD, Linux don't support it) are there
> > any OS's out (apart from solaris) that do support sun4d ?
> 
> I don't know, but NetBSD seems to be the most platform-wide OS I've ever
> come by, ever. Although, I have no idea about eventual support for SSA's
> and other equipment, in either OS.
> Try Solaris. It's free (now), it's fast, it's standard, it's Sun (on Sun
> equipment).

I'm certainly gonna play a little more with Solaris (it's still on one
of the disks in the server) - it seems though that there's actually
array support in the linux kernel though! It's called pluto
(for the kernel hackers among us :
/usr/src/linux-2.2.*/drivers/scsi/pluto.c, and /usr/src/drivers/fc4/*.c)
which calls a soc module to set up initial communication with the array
(sending&recieving packets over the FC) and then tries to communicate
with the array itself. I haven't had luck with actually using it, 
it keeps complaining about "too many continuations for port A", I'm
compiling it in debug mode as we speak to see what's going on.

Anyway, please forgive me if some of these question are quite stupid,
I've only been playing with Sun hardware for 2 weeks now so I'm still
a bit of a newbie as far as Sun Hardware.

Thanx again for the excellent info & help!

Cheers,
sparcy.
 
> > *Any* help on any of these questions would be greatly appreciated!
> 
> Hope this helps.
> 
> 	/Regards, Bjorn
> 
> _______________________________________________
> Rescue maillist  -  Rescue at sunhelp.org
> http://www.sunhelp.org/mailman/listinfo/rescue
>