Environment seeding doesn't allow reproducibility

[quanvuong]

Hi Hartikainen,

Thank you for maintaining the repo. Sorry if this is done elsewhere in the code, but shouldn't we set the seed of the environments (both training and eval) after creating them here for reproducibility purposes?

softlearning/examples/development/main.py

Line 49 in 1f6686d

Gym maintains its own internal copy of numpy and use that internal copy to sample the initial state. Setting the seed of the global numpy module does not affect this internal copy of numpy.

Thanks!

[quanvuong]

I create a self-contained snippet that illustrates the issue. Thank you for your attention!

[hartikainen]

Thanks for opening the issue, and sorry for the delayed reply. I think you're right, the way we set the seeds in our code does not guarantee full reproducibility but only that the randomizations are different across the runs.

Without looking too deep into this, I think the env seeding is the only one that's currently making things not reproducible. I would solve this by setting the seeds in the ExperimentRunner._build:

softlearning/examples/development/main.py

Line 39 in 34ee921

def _build(self):

I can try looking into this at some point, but feel free to submit out a PR if you want to take a stab at it in the meanwhile.

[quanvuong]

Sorry about closing and opening the issue (misclick).

I'm happy to send a PR in. It's literally 2 lines : ).