Running Theano-based DeepMind DQN on Mac

Theano-based implementation of Deep Q-learning is available on github. During the DeepHack hackathon our team «’drop table table users;» (which consists of me, Alexey Ozerin, Alexander Notchenko, Mikhail and Artur Kuzin) managed to get this code up and running on Ubuntu in a couple of minutes because of the provided But running this code on Mac resulted in some problems, which we were able to solve succeffully.

First of all to run test script with ./run_nips –rom breakout you need to make sure you have corresponding roms:

$ cd ./roms
$ wget
$ unzip -e && rm

You might need to alter ~/.theanorc:

$ cat ~/.theanorc
floatX = float32
device = gpu0

fastmath = True

Error with unavailable numpy/multiarray.h can be solved with exporting proper PATH variables:

export PATH=/Developer/NVIDIA/CUDA-7.0/bin:$PATH

and / or altering a file replacing coressponding lines:

import pyximport
import numpy as np
pyximport.install(setup_args={'include_dirs': np.get_include()})

Annoying dnn error

File "/lib/python2.7/site-packages/Lasagne-0.1.dev0-py2.7.egg/lasagne/layers/", line 13, in
raise ImportError("dnn not available") # pragma: no cover
ImportError: dnn not available

can be solved with copying cudnn header and lib files to /usr/local/lib and /usr/local/include:

$ cd /cudnn-6.5-osx-v2/
$ ls
CUDNN_License.pdf cudnn.h libcudnn.dylib
INSTALL.txt libcudnn.6.5.dylib libcudnn_static.a
$ cp cudnn.h /usr/local/include/
$ cp *.dylib /usr/local/lib/
$ cp libcudnn_static.a /usr/local/lib/
$ cd /deep_q_rl/deep_q_rl
#and the running deep_q_rl looks like this:
$ ./ --rom breakout
RL-Glue Version 3.04, Build 909
RL-Glue is listening for connections on port=4096
RL-Glue Python Experiment Codec Version: 2.1 (Build 738)
Connecting to on port 4096...
RL-Glue :: Experiment connected.
A.L.E: Arcade Learning Environment (version 0.5.0)
[Powered by Stella]
Use -help for help screen.
Warning: couldn't load settings file: ./stellarc
Game console created:
ROM file: ../roms/breakout.bin
Cart Name: Breakout - Breakaway IV (1978) (Atari)
Cart MD5: f34f08e5eb96e500e851a80be3277a56
Display Format: AUTO-DETECT ==> NTSC
ROM Size: 2048
Bankswitch Type: AUTO-DETECT ==> 2K

Running ROM file…
Random Seed: Time
Game will be controlled through RL-Glue.
Initializing ALE RL-Glue …
RL-Glue :: Environment connected.
Using gpu device 0: GeForce GT 750M
RL-Glue Python Agent Codec Version: 2.1 (Build 738)
Connecting to on port 4096…
Agent Codec Connected
RL-Glue :: Agent connected.
/Lasagne-0.1.dev0-py2.7.egg/lasagne/layers/ UserWarning: get_all_layers() has been changed to return layers in topological order. The former implementation is still available as get_all_layers_old(), but will be removed before the first release of Lasagne. To ignore this warning, use `warnings.filterwarnings(‘ignore’, ‘.*topo.*’)`.
warnings.warn(“get_all_layers() has been changed to return layers in ”
INFO:root:OPENING breakout_07-20-18-18_0p0002_0p95/results.csv
INFO:root:training epoch: 1 steps_left: 50000
INFO:root:steps/second: 100.20

Hope that this notes have made you able to train and successfully run an atari game agent.

P.S. If you were unable to run code and you still get ImportError: dnn not available  error, you might want to debug theano/sandbox/cuda/ dnn_available() function.