[SunRescue] OT: dual PPro mb's...

[James Lockwood]

On Fri, 30 Mar 2001, Greg A. Woods wrote:

> Rumour has it (I'm not a Linux user) even the 2.4 kernel has far
> "grainier" data structure locking for SMP than would be desirable for
> really good CPU utilisation.  I'm not sure how many CPUs it's designed
> to handle well either.  IIRC Linus said something about the locking
> being a little more coarse than he'd like in his rushed announcement of
> the 2.4 kernel.

I wouldn't be surprised.  High end scalability has never been a primary
focus for Linux, though many people have been helping it in that
direction.

> (I really don't know what the problem is -- the Bell Labs folks did a
> dual-CPU implementation of V7 way back in the early 80's and there have

This was a jumbo lock single kernel thread approach, if you're referring
to the 11/74 experiment.

SunOS 4 took the same approach.  It's very easy to do and offers
reasonable gains if you have a very small number of CPU's and you spend
very little time in the kernel.

Good fine-grained SMP (especially when combined with RT scheduling) is
far, far more difficult to implement.  Decisions on where to place locks,
what type and on what (data vs code) are difficult.  There are no hard and
fast rules, everything that is optimal for one situation is a trade-off
for another.

> been lots of people doing it even better ever since and many of them
> have documented their methods and the various tricks and traps they've
> encountered along the way.  I.e. the knowledge is out there and it
> should be trivial enough to apply it, especially in Linux where the main
> target still seems to be the Intel processor family.  It should merely
> be a matter of engineering at this point.  The NetBSD folks have been

Not to disparage your comments, but I have a feeling that you've never
worked on a large SMP kernel.  Kernel lock contention analysis is _hard_
and will always be suboptimal for different hardware and syscall usage
patterns.  Solaris (on both SPARC and x86) is actually a fairly good
example of how to do it right and it's taken them a long time to get there
(with high end customers pushing all the way).  Some other commercial
Unices do a reasonable job (Tru64 and AIX, IMHO) while HP-UX was lagging
in kernel parallelization last I checked.

-James