Store cluster.yaml in logs

[zigaLuksic]

We decided for a simple procedure:

1. eogrow-ray transfers cluster.yaml onto the head instance (similar to configs). This does not need to be named in a 'execution-specific-way' because if you switch to a different cluster configuration you'll (almost surely) also switch the head node
2. Each pipeline checks if there is a cluster.yaml in the expected location, if there is, it copies it over.

Tested it out on a simple case to make sure it works

[veseln]

Just one question. Does this work if you spin a cluster, attach to it and then run a pipeline? Not a problem if it doesn't, just want to understand.

ray up cluster.yaml
ray attach cluster.yaml 
eogrow something

[zigaLuksic]

Just one question. Does this work if you spin a cluster, attach to it and then run a pipeline? Not a problem if it doesn't, just want to understand.

ray up cluster.yaml
ray attach cluster.yaml 
eogrow something

If you do a manual ray rsync_up cluster.yaml cluster.yaml "~/.synced_configs/cluster.yaml" then it will (or if you first run a pipeline with eogrow-raywhich copies it)

[veseln]

Thanks, looks good.