[SunRescue] I could run a small country...

[James Lockwood]

On Wed, 27 Dec 2000, Dave McGuire wrote:

> On December 26, James Lockwood wrote:
> > The beauty of a net is that you just have to give it input, expected
> > output, and a staggering amount of CPU time.  :)
> 
>   Hey James...do you think that code will vectorize well?

Yes, extremely.  Neural net training is an iterative process, but within
each iteration there is room for massive SIMD vectorization.  The same
operations are being applied to thousands of nodes.  Each "layer" depends
on the previous (or next, depending on if you're using the net or training
it) but within a layer everything can be vectorized.

It doesn't parallelize in the multithreading sense as well because
overhead starts to kill you, but you can parallelize the entire training
process with slightly different perturbations (known as annealing).

-James