[rescue] J90 on epay

[Dave McGuire]

On Saturday, March 1, 2003, at 02:24 PM, Joshua D. Boyd wrote:
>>    IOS stands for Input/Output System.  The J90 IOS is based on a 
>> VME64
>> cardcage containing an IO}B interface (a high-speed channel directly 
>> to
>> a CPU), a VME SS5 (which does RAID, caching, and error handling) and
>> disk/network interfaces.  The VME SS5 sets up high-speed DMA
>> transactions into the Cray processors' memory, which run over the IOBB
>> bus.
>
> First, aren't there more than 1 IOB and VME SS5s in the machine you
> showed me down the street?

   In mine you mean?  My J90 has three IOSs...two in one VME64 chassis 
and one in a second.  subsystems.  Each IOS has an IOBB interface going 
into the mainframe.  See below for more details on this.

> Second, does the SS5 just set up the DMA, or does the SS5 actually do
> the DMA?  It seems to me that if the SS5 doesn't do the DMA, then 
> either
> the IO card (ethernet, scsi, hippi, whatever) needs to understand cray
> memory, or else the cray memory needs to accept VME style DMA, where as
> having the SS5 take DMA from the IO card, then stuff it into cray 
> memory
> by a second DMA transaction would be better for accomidating 3rd party
> cards.  But, I don't really know.  Just inquiring further into the 
> inner
> workings of Cray IO.

   First of all, for anyone else reading this, we're talking about the 
IOS SS5s, which are 6U VME64 cards.  The "regular" Sun SS5 is called 
the OWS...Operators Work Station.  It only handles diags, OS 
installation, booting, and shutdown.

   As far as I'm aware, the IOS SS5s set up DMA transactions between the 
peripheral controllers and the IOBB board.  What happens then is 
unclear...I'm not sure if the IOBB board buffers the data then DMAs it 
into the J90's memory during a second transaction, or if it's more of a 
"passthrough" interface.

   For booting, the IOS SS5 (the "master" IOS if there's more than one) 
keeps the J90 processor halted, squirts a kernel into their memory 
along with a configuration file which contains details device 
configuration and such, then starts the processor at the entry point of 
the kernel.  It's a fascinating process.  It took James Sharp and I 
days of work to reverse-engineer how the boot process works.

   Each IOS VME64 chassis' backplane is broken into four segments and 
can hold four separate IOS board sets...each with its own SS5, IOBB, 
and disk/network interfaces.  An IOBB interface connects via a big fat 
cable to a mezzanine card mounted in one of the J90 CPU modules.  Each 
CPU module has four mezzanine slots which can hold either an IOBB 
interface, a HIPPI source board, or a HIPPI destination board.  With 
four mezzanine slots on each CPU module, and a maximum of four CPU 
modules in a J916, that means up to sixteen IOSs.  Most have one or two 
at most.  A minimally-configured J90 consists of two cabinets, and a 
big one can be up to five.

   As you can imagine, this multiple-IOS design allows great flexibility 
in distributing I/O and tuning it for the machine's intended 
applications and I/O patterns.

   The EL family boots in the same way, except the IOS (which is a VME 
68020 CPU board) boots from a locally-attached SCSI disk instead of 
over the ethernet from the OWS.  It uses a regular ASCII console 
terminal connected to the "master" (if there's more than one) IOS's 68K 
CPU board.

           -Dave

--
Dave McGuire             "I've grown hair again, just
St. Petersburg, FL           for the occasion."       -Doc Shipley