YouTube Description

The video is an edited live coding session that shows step-by-step how to implement a DQN from scratch with TensorFlow 2 to play the Cart Pole game:

For Part 1 (delete in Part 2):

0:00 - Let's get started

0:05 - Stub for the reply buffer class

1:12 - Stub for the DQN agent class

2:44 - Implement the training loop

4:26 - Implement helper function for collecting gameplay experiences

5:44 - Implement the DQN agent class

For Part 2 (delete in Part 1):

0:00 - Let's continue

0:06 - Continue implementing the DQN agent class

1:46 - Implement the replay buffer class

4:58 - Implement the training API in the DQN agent class

8:40 - Helper function for performance evaluation

10:57 - API for updating the target Q network

12:04 - Initial end-to-end training

12:24 - Add visualization for training

12:38 - Train again with visualization

12:55 - Add randomness to encourage DQN agent exploration

14:14 - Final training

14:29 - Training results

The video goes hand-in-hand with a tutorial Medium blog post: {insert when published}

The code implemented in the video can be found in https://github.com/ultronify/dqn-from-scratch-with-tf2

A more sophisticated version which is also a little bit more complicated can be found in https://github.com/ultronify/cartpole-tf-dqn

Since my video editing software crashed on exporting the entire video, I separated it into 2 parts. In case it's hard to find, the link for Part 1 is https://youtu.be/W-iQIwrdNik and the link for Part 2 is https://youtu.be/ltC_lVCxgi4.

If you find it helpful, please make sure to like and subscribe!

Follow me on Medium @ https://medium.com/@jacksonz666

Follow me on GitHub @ https://github.com/tianhaoz95

For my latest posts and videos, subscribe to the newsletter @ https://jacksonz.substack.com