Homework 4

5 minute read

Submission date: ~January 28th~ March 1st, 2020

Topics

  • Deep reinforcement learning based on policy gradients.

Downloading

The assignment code is available here.

Technical Notes

  • This part does not require a GPU. We won’t need large models, and the computation bottleneck will be the generation of episodes to train on.
  • In order to run this notebook on the server, you must prepend the xvfb-run command to create a virtual screen. For example,
    • to run the jupyter lab script with srun do 
        srun -c2 --gres=gpu:1 xvfb-run -a -s "-screen 0 1440x900x24" ./jupyter-lab.sh
    • To run the submission script, do e.g. 
        srun -c2 xvfb-run -a -s "-screen 0 1440x900x24" python main.py prepare-submission ...

      and so on.

  • The OpenAI gym library is not officially supported on windows. However it should be possible to install and run the necessary environment for this exercise. However, we cannot provide you with technical support for this. If you have trouble installing locally, we suggest running on the course server.

Updates

  1. 2020-01-19
    • Update technical notes: added -a argument to xvfb-run examples.
    • In TrainBatch, update the type of the q_vals tensor to LongTensor.

FAQ

Make sure to read the getting started page and the guide for using course servers.

Q: What is the checkpoint_file_final for?
A: You can use this to create your final submission with result images from your best-trained model. Just train with checkpoints enabled, and when you get results that your happy with rename the checkpoint file with _final. You don’t need to submit the checkpoint files (the main.py script will ignore them).

Q: Should the results/ direcory be part of the submission?
A: Yes. The submission script will include it for you. This is OK. Do not put any unnecessary files in this directory apart from the results files generated by the notebooks.

Updated: