Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(To be discussed) (Idea) feature: multi dimensional reward #225

Open
wants to merge 13 commits into
base: develop
Choose a base branch
from
Open

(To be discussed) (Idea) feature: multi dimensional reward #225

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
ce6709f
experiment: multi-dimensional reward
YuriCat Nov 18, 2021
4a2ab4b
fix: train value loss
YuriCat Nov 23, 2021
ab35544
Merge branch 'develop' into feature/multi_dimensional_reward
YuriCat Dec 2, 2021
09bddbd
Merge develop
YuriCat Feb 13, 2022
380d4e2
fix: environment sample tictactoe
YuriCat Feb 13, 2022
074cdf0
fix: remove outcome() from test
YuriCat Feb 13, 2022
b3ae663
fix: parallel tic-tac-toe environment sample for md-reward style
YuriCat Feb 13, 2022
c4500b1
chore: stop using calling terminal_rewards 'outcome' in Geister envir…
YuriCat Feb 13, 2022
3866c74
fix: geister environment sample for md-reward setting
YuriCat Feb 13, 2022
461993d
fix: gamma outside of batches
YuriCat Feb 13, 2022
d970061
Merge branch 'develop' into feature/multi_dimensional_reward
YuriCat Feb 15, 2022
dc5cc11
Merge develop
YuriCat Feb 15, 2022
493f8a8
Merge develop
YuriCat Nov 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion handyrl/train.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ def compose_losses(outputs, log_selected_policies, total_advantages, targets, ba
entropy = dist.Categorical(logits=outputs['policy']).entropy().mul(tmasks.sum(-1))
losses['ent'] = entropy.sum()

base_loss = losses['p'] + losses.get('r', 0)
base_loss = losses['p'] + losses.get('v', 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow, thank you

entropy_loss = entropy.mul(1 - batch['progress'] * (1 - args['entropy_regularization_decay'])).sum() * -args['entropy_regularization']
losses['total'] = base_loss + entropy_loss

Expand Down