XOR Exercise

With default settings, all initial squares are grey. Let us explain why.
Sum of random small weights (positive and negative) times small numbers (random sigmoid outputs) tends to be close to 0.
Sigmoid output close to 1/2.
All squares are grey (around 1/2).

See what happens when you:
- Change the learning rate to 0.
- Change the learning rate to 0.01.
Change the learning rate to 10. Might have to think about this.

See what happens when you:
- Change randomWeight to return constant 0.
- Change randomWeight to return constant small 0.5.
- Change randomWeight to return constant large 5.
As explained in the notes, the weights cannot start the same. If they do, the machine now has in effect only one hidden node, and can only do what such a simple machine can do. Which is not nothing. But not enough.

All other things being equal, large weights lead to slow learning.
Change weights to large randomised (like -10 to 10) and see what happens. Might have to think about this.
Change weights to large randomised all positive (like 10 to 20) and see what happens. Might have to think about this.

Change the exemplars to represent a different function (across 0,1).
Get it to successfully learn that function from exemplars.
Constant output function is learnt fast:
- If you change it to constant output 1 for all exemplars, it learns that function incredibly quickly. Why?
- Same if you change it to constant output 0. It learns super quickly. Why?

Extra exercises

I wrote some more exercises, if you want a challenge.

My code has a bug. I divide the 2D area into squares. I take one point to represent each square.
But my code actually has an issue:
- To colour the square starting at (0.9,0.5) and extending for 0.1 along x and 0.1 along y:
  We fill the square with a colour representing the point f(0.9,0.5)
- This means the origin square is filled with the value of f(0,0) which is good.
- But the other corners are filled with the values of f(0, 0.9) and f(0.9, 0) and f(0.9, 0.9) which is not perfect.
Fix my code:
- The 4 corner squares should be filled with the colours for the exact values:
  f(0,0) and f(0,1) and f(1,0) and f(1,1)
- The other squares can be filled with the colours for their centre points.

To see how it is learning, each time round the "draw" function, calculate the values for the exact 4 corner points, f(0,0) and f(0,1) and f(1,0) and f(1,1).
Write these to console. Watch it converge.
It is a bit hard to see this in a scrolling window like console.
So you could repeatedly write them to the "run header" window using AB.msg. Then you can write the data to the same place on screen.
AB.msg can write HTML formatting.
Or you could keep the console, and repeatedly clear it with console.clear().

It is hard to slow it down to see what is happening, since it trains a few times before it even draws the squares.
Get debug info outputted before train.
Or move train to after drawsquare in the loop.

Output all the weights to watch the network changing.
As above, you will probably want to use AB.msg (fixed in place) not console.log (scrolling).