New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

V0.4 Release Plan #123

Open

5 of 9 tasks

cp5555 opened this issue Nov 6, 2023 · 1 comment

Labels

cp5555 commented Nov 6, 2023 •

edited by Mr-Philo

Loading

Release Manager

Endgame

Code freeze: Feb. 9th, 2024
Bug Bash date: Feb. 12th, 2024
Release date: Feb. 23rd, 2024

Main Features

MS-AMP O3 Optimization

1. Support auto scaling factor tuning (ASFT) for FP8 collective communication (Related to Auto scaling factor tuning for FP8 collective communication #41, [Feature] Auto scaling factor tuning for FP8 collective communication #140)
2. Support PyTorch FSDP (Related to Support for MS-AMP in FSDP #122)

MS-AMP Improvement

1. Move extension installation from post install to setup.py (Related to Moving extension installation from post install to setup.py under project root folder #43)
2. Improve FP8 kernel performance in MS-AMP (Optimize performance by fuse adding high precision tensor to fp8 tensor #132)
3. MS-AMP support on different devices (Nvidia A100 and AMD MI300X)

MS-AMP Examples

1. Release the datapoints (Related to Training curve datapoints or smoothing #115)

Backlog

FP8 on Activation

cp5555 added the iteration plan label

Contributor

tocean commented Feb 29, 2024

Will move all the non-started work items to next release v0.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment