Theano-based implementation of Deep Q-learning is available on github. During the DeepHack hackathon our team «’drop table table users;» (which consists of me, Alexey Ozerin, Alexander Notchenko, Mikhail and Artur Kuzin) managed to get this code up and running on Ubuntu in a couple of minutes because of the provided dep_script.sh. But running this code on Mac resulted in some problems, which we were able to solve succeffully.
First of all to run test script with ./run_nips –rom breakout you need to make sure you have corresponding roms:
$ cd ./roms
$ wget https://atariage.com/2600/roms/Breakout.zip
$ unzip -e Breakout.zip && rm Breakout.zip
You might need to alter ~/.theanorc:
$ cat ~/.theanorc
floatX = float32
device = gpu0
fastmath = True
Error with unavailable numpy/multiarray.h can be solved with exporting proper PATH variables:
and / or altering a ale_data_set.py file replacing coressponding lines:
import numpy as np
Annoying dnn error
File "/lib/python2.7/site-packages/Lasagne-0.1.dev0-py2.7.egg/lasagne/layers/dnn.py", line 13, in
raise ImportError("dnn not available") # pragma: no cover
ImportError: dnn not available
can be solved with copying cudnn header and lib files to /usr/local/lib and /usr/local/include:
$ cd /cudnn-6.5-osx-v2/
CUDNN_License.pdf cudnn.h libcudnn.dylib
INSTALL.txt libcudnn.6.5.dylib libcudnn_static.a
$ cp cudnn.h /usr/local/include/
$ cp *.dylib /usr/local/lib/
$ cp libcudnn_static.a /usr/local/lib/
$ cd /deep_q_rl/deep_q_rl
#and the running deep_q_rl looks like this:
$ ./run_nips.py --rom breakout
RL-Glue Version 3.04, Build 909
RL-Glue is listening for connections on port=4096
RL-Glue Python Experiment Codec Version: 2.1 (Build 738)
Connecting to 127.0.0.1 on port 4096...
RL-Glue :: Experiment connected.
A.L.E: Arcade Learning Environment (version 0.5.0)
[Powered by Stella]
Use -help for help screen.
Warning: couldn't load settings file: ./stellarc
Game console created:
ROM file: ../roms/breakout.bin
Cart Name: Breakout - Breakaway IV (1978) (Atari)
Cart MD5: f34f08e5eb96e500e851a80be3277a56
Display Format: AUTO-DETECT ==> NTSC
ROM Size: 2048
Bankswitch Type: AUTO-DETECT ==> 2K
Running ROM file…
Random Seed: Time
Game will be controlled through RL-Glue.
Initializing ALE RL-Glue …
RL-Glue :: Environment connected.
Using gpu device 0: GeForce GT 750M
RL-Glue Python Agent Codec Version: 2.1 (Build 738)
Connecting to 127.0.0.1 on port 4096…
Agent Codec Connected
RL-Glue :: Agent connected.
/Lasagne-0.1.dev0-py2.7.egg/lasagne/layers/helper.py:69: UserWarning: get_all_layers() has been changed to return layers in topological order. The former implementation is still available as get_all_layers_old(), but will be removed before the first release of Lasagne. To ignore this warning, use `warnings.filterwarnings(‘ignore’, ‘.*topo.*’)`.
warnings.warn(“get_all_layers() has been changed to return layers in ”
INFO:root:training epoch: 1 steps_left: 50000
Hope that this notes have made you able to train and successfully run an atari game agent.
P.S. If you were unable to run code and you still get ImportError: dnn not available error, you might want to debug theano/sandbox/cuda/dnn.py dnn_available() function.