New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move embedding to phi #39901

Merged

phlrain merged 14 commits into develop from move_embedding_to_phi

Mar 22, 2022

Collaborator

phlrain commented Feb 24, 2022

PR types

Breaking changes

PR changes

OPs

Describe

move embedding to phi

phlrain added 6 commits

February 23, 2022 14:14


          move embeding to phi;

e037504


          update sig; test=develop

b6c8ef9


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ec0e839

… move_embedding_to_phi


          move reset impl to phi; test=develop

e81773c


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

… move_embedding_to_phi


          remove old register; test=develop

ab1042f

paddle-bot-old bot commented Mar 8, 2022

Sorry to inform you that ab1042f's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

phlrain added 3 commits

March 11, 2022 14:22


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

95c71a5

… move_embedding_to_phi


          fix cpu bf16 bug; test=develop

61dccb0


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

945f20a

… move_embedding_to_phi

zyfncg reviewed

View reviewed changes

paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated

+              namespace phi {
+              template <typename T, typename Context>
+              struct LookupTableV2GradCPUFunctor {

Contributor

zyfncg Mar 18, 2022

在phi下面LookupTableV2要不要都统一成Embedding？

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated

+                  // paddings makes no sense and we don't deal with it in backward.
+                  {
+                    auto* d_output = &out_grad_;
+                    // auto d_table = weight_grad_;

Contributor

zyfncg Mar 18, 2022

注释可以删除

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated

+                            ids_data[i],
+                            N,
+                            phi::errors::InvalidArgument(
+                                "Variable value (input) of OP(fluid.layers.embedding) "

Contributor

zyfncg Mar 18, 2022

fluid.layers.embedding->paddle.nn.functional.embedding

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/embedding_grad_kernel.cc Outdated

+                            ids_data[i],
+,
+                            phi::errors::InvalidArgument(
+                                "Variable value (input) of OP(fluid.layers.embedding) "

Contributor

zyfncg Mar 18, 2022

fluid.layers.embedding->paddle.nn.functional.embedding

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/embedding_grad_kernel.cc

+              };
+              template <typename T, typename Context>
+              void EmbeddingSparseGradKernel(const Context& ctx,

Contributor

zyfncg Mar 18, 2022

这个Kernel可以放到selected_rows下

Collaborator Author

phlrain Mar 20, 2022

selected rows的拆分，单独用一个pr来做

paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc


		namespace phi {

		template <typename T, typename Context>

Contributor

zyfncg Mar 18, 2022

这个文件可以放到selected_rows目录下

Collaborator Author

phlrain Mar 20, 2022

selected rows的拆分，单独用一个pr来做

paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated

+              #include "paddle/phi/kernels/funcs/embedding_util.h"
+              #include "paddle/fluid/framework/convert_utils.h"
+              #include "paddle/fluid/framework/data_type.h"

Contributor

zyfncg Mar 18, 2022

引用phi下的data_type.h

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated

+                            ids_data[i],
+,
+                            phi::errors::InvalidArgument(
+                                "Variable value (input) of OP(fluid.layers.embedding) "

Contributor

zyfncg Mar 18, 2022

fluid.layers.embedding->paddle.nn.functional.embedding

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/sparse_weight_embedding_kernel.cc Outdated

+                  auto output_t = out_;
+                  int64_t row_width = table_t.value().dims()[1];
+                  const auto* table = table_t.value().template data<T>();
+                  auto* output = output_t->template mutable_data<T>(dev_ctx_.GetPlace());

Contributor

zyfncg Mar 18, 2022

使用dec_ctx.Alloc

Collaborator Author

phlrain Mar 20, 2022

done

paddle/phi/kernels/cpu/sparse_weight_embedding_kernel.cc

Comment on lines +49 to +50

		auto input_data_type =
		paddle::framework::TransToProtoVarType(table_t.value().dtype());

Contributor

zyfncg Mar 18, 2022

这里可以不转ProtoVarType，直接用dtype

Collaborator Author

phlrain Mar 20, 2022

done

phlrain added 3 commits

March 19, 2022 13:01


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

755ad25

… move_embedding_to_phi


          fix lookup speed error

7ba14d7


          polish code

867fc05

zyfncg previously approved these changes

View reviewed changes

chenwhql reviewed

View reviewed changes

paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated

+                } else if (input.dtype() == phi::DataType::INT64) {
+                  functor.template apply<int64_t>();
+                } else {
+                  PADDLE_THROW("emebdding input only support int32 and int64");

Contributor

chenwhql Mar 21, 2022

这不符合报错规范，需要加报错类型

Collaborator Author

phlrain Mar 21, 2022

已修改

paddle/phi/kernels/cpu/sparse_weight_embedding_grad_kernel.cc Outdated

+                } else if (input.dtype() == phi::DataType::INT64) {
+                  functor.template apply<int64_t>();
+                } else {
+                  PADDLE_THROW("emebdding input only support int32 and int64");

Contributor

chenwhql Mar 21, 2022

同上

Collaborator Author

phlrain Mar 21, 2022

已修改

paddle/phi/kernels/gpu/embedding_grad_kernel.cu Outdated

+                } else if (input.dtype() == phi::DataType::INT64) {
+                  functor.template apply<int64_t>();
+                } else {
+                  PADDLE_THROW("emebdding input only support int32 and int64");

Contributor

chenwhql Mar 21, 2022

同上，共性问题，麻烦都check一下

paddle/phi/kernels/sparse_weight_embedding_grad_kernel.h

+              namespace phi {
+              template <typename T, typename Context>
+              void SparseWeightEmbeddingGradKernel(const Context& ctx,

Contributor

chenwhql Mar 21, 2022

SelectedRows相关的尽快移到selected_rows目录

phlrain added 2 commits

March 21, 2022 04:11


          fix paddle throw type

7ca48f9


          Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

1a7399b

… move_embedding_to_phi

phlrain dismissed zyfncg’s stale review via

1a7399b

March 21, 2022 04:17

phlrain closed this

phlrain reopened this

chenwhql approved these changes

View reviewed changes

Contributor

chenwhql left a comment

如评论，selected rows kernel后续再调整下目录和形式

Collaborator Author

phlrain commented Mar 22, 2022

如评论，selected rows kernel后续再调整下目录和形式

TODO已记录

XiaoguangHu01 approved these changes

View reviewed changes

Contributor

XiaoguangHu01 left a comment

LGTM

phlrain merged commit 0331cfd into develop

chenwhql deleted the move_embedding_to_phi branch

March 22, 2022 14:30

Liangwei-0521 commented Jul 26, 2023

Traceback (most recent call last):
File "D:\rank_v1.2\quant_predictor_2.py", line 191, in
results.extend(predictor.predict(batch_data, tokenizer))
File "D:\rank_v1.2\quant_predictor_2.py", line 134, in predict
self.predictor.run()
NotImplementedError: (Unimplemented) There are no kernels which are registered in the lookup_table_v2 operator.
[Hint: Expected kernels_iter != all_op_kernels.end(), but received kernels_iter == all_op_kernels.end().] (at C:\home\workspace\Paddle\paddle\fluid\framework\operator.cc:1895)
[operator < lookup_table_v2 > error

embedding量化后，导入静态模型报错。

Liangwei-0521 commented Jul 26, 2023

embedding量化代码
def _quant_embeddings(input_prefix):

# input_prefix: 静态模型路径
# 最终量化的模型路径：
output_filename_prefix = "rank_quant_emb"
paddle.enable_static()
place = paddle.set_device('cpu')
exe = paddle.static.Executor(place)
main_program, feed_target_names, fetch_targets = paddle.static.load_inference_model(input_prefix, exe)

config = {
        "quantize_op_types": ["lookup_table_v2"], 
        "lookup_table_v2": {
            "quantize_type": "log",
            'quantize_bits': 8,
            'dtype': 'int8'
            }}

quant_emb_program = quant.quant_embedding(main_program, place, config)

input_dir = os.path.dirname(input_prefix)

paddle.fluid.io.save_inference_model(
    input_dir,
    feed_target_names,
    fetch_targets,
    exe,
    quant_emb_program,
    model_filename=output_filename_prefix + ".pdmodel",
    params_filename=output_filename_prefix + ".pdiparams",
    export_for_deployment=True,
    program_only=False,
)

if name == 'main':
_quant_embeddings(input_prefix='./checkpoint/quant')

Contributor

zyfncg commented Jul 27, 2023

Traceback (most recent call last): File "D:\rank_v1.2\quant_predictor_2.py", line 191, in results.extend(predictor.predict(batch_data, tokenizer)) File "D:\rank_v1.2\quant_predictor_2.py", line 134, in predict self.predictor.run() NotImplementedError: (Unimplemented) There are no kernels which are registered in the lookup_table_v2 operator. [Hint: Expected kernels_iter != all_op_kernels.end(), but received kernels_iter == all_op_kernels.end().] (at C:\home\workspace\Paddle\paddle\fluid\framework\operator.cc:1895) [operator < lookup_table_v2 > error

embedding量化后，导入静态模型报错。

@1998-Chen 这个问题在issue里提下吧，在PR下面不太好跟踪

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet