Dr. Mark Humphrys

School of Computing. Dublin City University.

Home
Blog
Research
Publications
Contact

Online coding site: Ancient Brain

 

Search:

Teaching
Project ideas
1st yr
3rd yr
MCM

CA425 - Artificial Intelligence

Notes

Reinforcement Learning

State-space control
  1. Continuum of Autonomy
  2. State-space control
  3. RL as Pattern Classification
  4. Reinforcement Learning - Reference


Reinforcement Learning - Intro

  1. RL - The world
  2. RL - The task
  3. Exercise - long-term reward
  4. Q-learning
  5. Building up a running average
  6. How Q-learning works


Movie demo

  1. Movie demo of W-learning contains within it a demo of basic Q-learning.


Program code in C++

  1. Coding the state-space as a lookup-table
  2. Sample code for lookup-table Q-learning (Includes Boltzmann "soft max" option)


Reinforcement Learning - More

  1. Convergence

  2. The control policy
  3. Boltzmann "soft max" distribution
  4. How to make a decision probabilistically

  5. Building a model of Pxa(r)
  6. Building a model of Pxa(y)
  7. Learning rate that does not start at 1

Reinforcement Learning with Neural Networks

Reinforcement Learning with Neural Networks
  1. Neural Networks course

  2. Using a Neural Network as a generalisation in RL
  3. Q-learning with a Neural Network
  4. Using a Neural Network with RL

Multiple Minds

Multiple Minds
  1. Multi-Module Reinforcement Learning
  2. Multiple Minds in the same body - Test of Hierarchical Q-learning
  3. The general form of a Society of Mind based on Reinforcement Learning
  4. Open Issues in AI
  5. Architectures of Autonomous Agents
  6. The World-Wide-Mind

Practical

Practical - Play "X's and O's" with RL

Reading

Experiments in Adaptive State-Space Robotics, Clocksin and Moore, 1989. A simple introduction to the very idea of state-space robotic or agent control.

How to Make Software Agents Do the Right Thing: An Introduction to Reinforcement Learning, Singh et al, 1996. A simple introduction to the idea of RL.

"Reinforcement Learning: A Survey", Kaelbling et al, Journal of Artificial Intelligence Research, 4:237-285, 1996. A survey.

Action Selection methods using Reinforcement Learning. My PhD thesis, 1997, has an intro to RL.

Books

Reinforcement Learning: An Introduction, Sutton and Barto, 1998. Also here.

Reinforcement Learning: State-of-the-Art, Marco Wiering and Martijn Otterlo (Editors), 2012.

Library categories
ancientbrain.com      w2mind.org      humphrysfamilytree.com

On the Internet since 1987.      New 250 G VPS server.

Note: Links on this site to user-generated content like Wikipedia are highlighted in red as possibly unreliable. My view is that such links are highly useful but flawed.