Character recognition neural network on Ancient Brain

This is a neural network that learns from a database of images paired with what character is written in the image. After a while, it can recognise new images, that it never saw before, with 95 percent accuracy.

Then it tries to use that knowledge to recognise human handwriting. You can write characters in the browser and the network will recognise them with accuracy of maybe 60 percent, no matter who you are, and despite the fact that this network knows literally nothing at the start of the run.

Click to run World: Character recognition neural network at Ancient Brain.

• Credits:
  • This is a heavily modified port of the MNIST character recognition program by Daniel Shiffman at The Coding Train.
  • See long livestream video.

• It uses various support files (links to Coding Train account uploads):
  • JS files
  • Other files (includes the MNIST dataset)

• This uses the MNIST database of exemplars (handwritten digits and correct answer) for training.
• For testing, the MNIST test set is used. These are different exemplars to the training exemplars.

• The images are tiny: 28 x 28 pixels, greyscale integer values 0 to 255.
• We convert these pixels to (28 squared = 784) neural network input nodes, taking real number values 0 to 1.
• Note this is not ideal way of inputting image because it loses the spatial 2D layout. Better to use a Convolutional neural network. But this still works well as a backprop demo.
• There are 10 output nodes. They generate output for each character 0 .. 9. The highest value is our "guess" as to which of the 10 this is.

• As before, I have edited Matrix.randomize() in matrix.js so it calls a function randomWeight() that we define in the World.

---

Run the World

• There are three sections:
  1. Doodle: Draw your own image. You can draw a "doodle" of a digit in this area and the network will try to classify it.
  2. Training: This is where training takes place. The network shows the exemplars it is training on. They flash by. It also runs an ongoing test of how well it is doing out of 100 percent accurate classification. (It does not display the test exemplars.)
  3. Demo: A random test demo. Picks a random image and tests if the network can classify it. These images are not from the training set. They are from a separate test set. The network has not been trained on these images. So these are new, unseen before images for the network.

---

Results

• Test set: 95 percent accuracy
  • The neural network is incredibly accurate in a very short time. At least, over the test set.
  • You can see this illustrated by trying lots of samples in the "Demo" section.
  • After a very short time, it gets over 90 percent accuracy in classifying new images from the test set. After a long time, it gets to about 95 percent.

• Doodle: 60 percent accuracy
  • The doodle recognition is harder, though.
  • I recommend that you wait a while to try a doodle. Wait until the program has got up to at least 90 percent on the test data.
  • After a long time, it gets to about 60 percent accuracy on the doodles.
  • Considering random is 10 percent, this is a lot better than random. But not at all near the accuracy for the test set. We will consider this again in the exercise.
  • Still, consider what it is doing. It is recognising, at a level much better than chance, handwriting, and even fairly random doodling, done at run-time by a person it has never met before. Not bad.

---

Fetching the data

The first 1 hr 15 min of the Daniel Shiffman video is about just fetching the (huge) data set at run-time and including it in the JS. There are issues here both for Daniel Shiffman's version and the Ancient Brain version.

• First, JS on a website cannot read local files on the client. (Thankfully!) So the data must be on the Ancient Brain server.

• Binary file on Ancient Brain:
  • Ancient Brain allows the upload of lots of file formats, such as JSON.
  • The MNIST data is in its own binary format, explained here. It does not use any file extension.
  • For this port, I made an exception to allow the MNIST data on Ancient Brain without any file extension.

• Fetching the data from JS
  • The next issue is JS reading this binary file.
  • Daniel Shiffman writes a file mnist.js to do this.
  • This uses fetch to get the files from local server.
  • I have edited it to point to the Ancient Brain files

Other ways to upload the image database

After all the above, I had some new thoughts and discoveries:

1. You can actually upload arbitrary binary data to Ancient Brain.
   • I realised that I allow file extensions for 3D models that could be re-used to upload any binary data.
   • An example would be .bin files used for 3D models.
   • You can rename MNIST or other binary data to .bin and it should upload.

2. You can get the MNIST database in JSON/JS format.
   • mnist - Database of MNIST digits as JS files and JSON files. All plain text.
   • You can upload these files to Ancient Brain no problem.