Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support BF16 training for sharding and dp #46846

Merged
merged 18 commits into from
Oct 17, 2022
Merged

Support BF16 training for sharding and dp #46846

merged 18 commits into from
Oct 17, 2022

Commits on Sep 14, 2022

  1. Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result 

    is wrong.
    @GhostScreaming
    GhostScreaming committed Sep 14, 2022
    Configuration menu
    Copy the full SHA
    1e70140 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'reduce_sum' of https://github.com/GhostScreaming/Paddle … 

    …into develop
    @GhostScreaming
    GhostScreaming committed Sep 14, 2022
    Configuration menu
    Copy the full SHA
    e1f08a2 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2022

  1. Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into… 

    … develop
    @GhostScreaming
    GhostScreaming committed Sep 17, 2022
    Configuration menu
    Copy the full SHA
    ff1bfbc View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2022

  1. support pure bfloat16

    @sneaxiy
    sneaxiy committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    f4fe24f View commit details
    Browse the repository at this point in the history

  2. support bf16 linear

    @sneaxiy
    sneaxiy committed Sep 21, 2022
    Configuration menu
    Copy the full SHA
    b420a32 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2022

  1. Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into… 

    … develop
    @GhostScreaming
    GhostScreaming committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    5b7bc39 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2022

  1. update PR to pass CI

    @sneaxiy
    sneaxiy committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    7ff1388 View commit details
    Browse the repository at this point in the history

  2. tiny fix where_grad_kernel.cu

    @sneaxiy
    sneaxiy committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    b9a7c14 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into… 

    … fix_bfloat16
    @sneaxiy
    sneaxiy committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    46662c4 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'fix_bfloat16' of https://github.com/sneaxiy/Paddle into… 

    … fix_bfloat16
    @GhostScreaming
    GhostScreaming committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    9e18791 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into… 

    … fix_bfloat16
    @GhostScreaming
    GhostScreaming committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    29a9e77 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2022

  1. Support bfloat16 type for reducer and sharding.

    @GhostScreaming
    GhostScreaming committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    817d7ee View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into… 

    … fix_bfloat16
    @GhostScreaming
    GhostScreaming committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    6e15126 View commit details
    Browse the repository at this point in the history

  3. Fix some bug.

    @GhostScreaming
    GhostScreaming committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    44abf06 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2022

  1. Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into… 

    … fix_bfloat16
    @GhostScreaming
    GhostScreaming committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    7fe04f2 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2022

  1. Polish code.

    @GhostScreaming
    GhostScreaming committed Oct 14, 2022
    Configuration menu
    Copy the full SHA
    384d497 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2022

  1. Polise code.

    @GhostScreaming
    GhostScreaming committed Oct 16, 2022
    Configuration menu
    Copy the full SHA
    d012390 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2022

  1. Add bfloat16 datatype in fill_grad kernels.

    @GhostScreaming
    GhostScreaming committed Oct 17, 2022
    Configuration menu
    Copy the full SHA
    480c732 View commit details
    Browse the repository at this point in the history