[amp] dygraph amp support param_group

[zhiqiu]

PR types

New features

PR changes

Others

Describe

• dygraph amp support param_group

linear_1 = paddle.nn.Linear(10, 10)
linear_2 = paddle.nn.Linear(10, 10)
inp = paddle.uniform(shape=[10, 10], min=-0.1, max=0.1)
with paddle.amp.auto_cast():
    out = linear_1(inp)
    out = linear_2(out)
loss = paddle.mean(out)
sgd = paddle.optimizer.SGD(
    learning_rate=0.1,
    parameters=[{
        'params': linear_1.parameters()
    }, {
        'params': linear_2.parameters(),
        'weight_decay': 0.001,
        'learning_rate': 0.1
    }],
    weight_decay=0.01)              
scaler=paddle.amp.GradScaler()     
scaler.scale(out).backward()
scaler.step(sgd)
sgd.clear_grad()

• add step method for class GradScaler, to be consistent with class Optimizer, since Optimizer.minimize() does not support param_group

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[jerrywgz]

LGTM

[TCChenlong]

import 前要加一个空行

[zhiqiu]

done, thx

[TCChenlong]

Args 前要加空行

[zhiqiu]

done, thx

[TCChenlong]

Examples 前要加空行

[zhiqiu]

done, thx

[TCChenlong]

LGTM for API docs