Homework 4

5 minute read

Submission date: July 2nd, 2020

Topics

Deep reinforcement learning based on policy gradients.

Downloading

The assignment code is no longer available.

Technical Notes for Part 1

Part 1 does not require a GPU. We won’t need large models, and the computation bottleneck will be the generation of episodes to train on.
In order to run this notebook on the server, you must prepend the xvfb-run command to create a virtual screen. For example,
- to run the jupyter lab script with srun do
```
  srun -c2 --gres=gpu:1 xvfb-run -a -s "-screen 0 1440x900x24" ./jupyter-lab.sh
```
- To run the submission script, do e.g.
```
  srun -c2 xvfb-run -a -s "-screen 0 1440x900x24" python main.py prepare-submission ...
```
  and so on.
The OpenAI gym library is not officially supported on windows. However it should be possible to install and run the necessary environment for this exercise. However, we cannot provide you with technical support for this. If you have trouble installing locally, we suggest running on the course server.

FAQ

Make sure to read the getting started page, the guide for using course servers and our collaboration policy before starting the assignment.

Q: What is the _final checkpoint file?
A: You must use this to create your final submission with result video from your best-trained model. When you get results that your happy with, rename the checkpoint file by appending _final. You don’t need to submit the checkpoints/ folder (the main.py script will ignore them).

Q: Should the results/ direcory be part of the submission?
A: Yes. The submission script will include it for you. This is OK. Do not put any unnecessary files in this directory apart from the results files generated by the notebooks.