XOR Exercise

Click to run World: XOR multi-layer network at Ancient Brain.

• Clone and edit the world

console.log

• Uncomment the console.log of x1, x2 and y in "drawsquare".
• y has too many decimal places. Let us reduce it.
• But y is an array. (Why?) So this does not work:

   y.toFixed(2) 

• Get the first element of y and round to 2 decimal places.

Initial weights and initial output

• With default settings, all initial squares are grey. Let us explain why.
  Sum of random small weights (positive and negative) tends to be close to 0.
  Sigmoid output close to 1/2.
  All squares are grey (around 1/2).

Learning rate

• See what happens when you:
  • Change the learning rate to 0.
  • Change the learning rate to 0.01.

• Change the learning rate to 10. Might have to think about this.

Weights constant

• See what happens when you:
  • Change randomWeight to return constant 0.
  • Change randomWeight to return constant small 0.5.
  • Change randomWeight to return constant large 5.
• As explained in the notes, the weights cannot start the same. If they do, the machine now has in effect only one hidden node, and can only do what such a simple machine can do. Which is not nothing. But not enough.

Weights large

• All other things being equal, large weights lead to slow learning.

• Change weights to large randomised (like -10 to 10) and see what happens. Might have to think about this.

• Change weights to large randomised all positive (like 10 to 20) and see what happens. Might have to think about this.

Hidden nodes

• See what happens when you:
  • Reduce hidden nodes to 1.
  • Hidden nodes 2.
  • Hidden nodes 3.
  • Hidden nodes 50.
  • Hidden nodes 500.

• Starts to get very strange:
  • Hidden nodes 10000.
  • Hidden nodes 50000.

• Q. Why is huge number of hidden units so weird?

Change the function

• Change the exemplars to represent a different function (across 0,1).
• Get it to successfully learn that function from exemplars.

• Constant output function is learnt fast:
  • If you change it to constant output 1 for all exemplars, it learns that function incredibly quickly. Why?
  • Same if you change it to constant output 0. It learns super quickly. Why?

---

Extra exercises

I wrote some more exercises, if you want a challenge.

Fix my code

• My code has a bug. I divide the 2D area into squares. I take one point to represent each square.
• But my code actually has an issue:
  • To colour the square starting at (0.9,0.5) and extending for 0.1 along x and 0.1 along y:
    We fill the square with a colour representing the point f(0.9,0.5)
  • This means the origin square is filled with the value of f(0,0) which is good.
  • But the other corners are filled with the values of f(0, 0.9) and f(0.9, 0) and f(0.9, 0.9) which is not perfect.

• Fix my code:
  • The 4 corner squares should be filled with the colours for the exact values:
    f(0,0) and f(0,1) and f(1,0) and f(1,1)
  • The other squares can be filled with the colours for their centre points.

Output the 4 points

• To see how it is learning, each time round the "draw" function, calculate the values for the exact 4 corner points, f(0,0) and f(0,1) and f(1,0) and f(1,1).
• Write these to console. Watch it converge.

• It is a bit hard to see this in a scrolling window like console.
• So you could repeatedly write them to the "run header" window using AB.msg. Then you can write the data to the same place on screen.
• AB.msg can write HTML formatting.
• Or you could keep the console, and repeatedly clear it with console.clear().

Slow it down

• It is hard to slow it down to see what is happening, since it trains a few times before it even draws the squares.
  Get debug info outputted before train.
  Or move train to after drawsquare in the loop.

Output all the weights

• Output all the weights to watch the network changing.
• As above, you will probably want to use AB.msg (fixed in place) not console.log (scrolling).