"Not Enough Stuff" arguments in AI

AI has a history of people saying that the systems and algorithms simply need more of some property and there will be a breakthrough.

To a large extent this has been true.

The main "Not Enough Stuff" arguments would be:

1. Not enough speed.
   "If we just get faster machines we will have full AI".
   • This has been happening for the whole history of AI. Faster machines make older algorithms suddenly more useful.
   • The recent use of GPUs as simple parallel computers has massively helped the speed of certain algorithms.

2. Not enough neurons/memory. Not big enough networks.
   "Neural networks need to be much bigger, and with a much higher number of layers."
   • Theoretical breakthroughs have made this happen.

3. Not enough data.
   "The machine needs to read the entire Internet."
   • The scaling up of AI models and the rise of massive AI companies spending billions on training has made this happen.
   • When you try to train a model on basically all human words ever written you do start to get something a lot more profound.
   • Modern AI feels like we are trying to extract parts of millions of human brains into one machine.

4. Not enough time.
   "We get 20 years to grow our intelligence. Maybe a machine needs a 20 year childhood too."
   • Many people have noted the gap between a human childhood, switched on and interacting with the physical world 15 hours a day, 365 days a year, for 20 years, and robot "learning", where the robot is typically switched on and learning only for very short time periods.
   • While software-only experiments can run for months, bringing a 20 year human childhood to a robot with a physical body is quite a challenge.
   • Cloud robotics is an idea that multiple robots could share their learning and have a collective brain, something that does not exist in nature.

5. Not enough authors. Not enough diversity in the mind.
   "Real brains will consist of a huge, diverse society of sub-brains of different types of intelligence and different forms of learning and memory."
   • This is the AI model behind Ancient Brain.
   • Ensemble learning is the field of constructing systems of multiple learners of different types, and trying to combine their strengths in a hybrid system. For example in classification taking the average over their outputs.
   • See also Mixture of experts.
   • Many of the large AI models have "Mixture of experts" structure.

---

Some of my old thoughts on this, before the AI boom:

• "Not enough stuff" in AI. In Constructing complex minds through multiple authors, Proceedings of the 7th International Conference on the Simulation of Adaptive Behavior (SAB-02), 2002.

• AI is too big a problem. In "The World-Wide-Mind: Draft Proposal", Dublin City University, School of Computing, Technical Report no. CA-0301, 2001.

ancientbrain.com      w2mind.org      humphrysfamilytree.com

On the Internet since 1987.      New 250 G VPS server.

Note: Links on this site to user-generated content like Wikipedia are highlighted in red as possibly unreliable. My view is that such links are highly useful but flawed.