Update Embedding doc #6806
Open
Update Embedding doc #6806
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
感谢你贡献飞桨文档，文档预览构建中，Docs-New 跑完后即可预览，预览链接：http://preview-pr-6806.paddle-docs-preview.paddlepaddle.org.cn/documentation/docs/zh/api/index_cn.html |
zhwesky2010
reviewed
Aug 5, 2024
| @@ -38,6 +38,8 @@ Embedding | |||
| - **num_embeddings** (int) - 嵌入字典的大小，input 中的 id 必须满足 ``0 <= id < num_embeddings`` 。 | |||
| - **embedding_dim** (int) - 每个嵌入向量的维度。 | |||
| - **padding_idx** (int|long|None，可选) - padding_idx 的配置区间为 ``[-weight.shape[0], weight.shape[0]]``，如果配置了 padding_idx，那么在训练过程中遇到此 id 时，其参数及对应的梯度将会以 0 进行填充。 | |||
| - **max_norm** (float，可选) - 如果给定，则每个范数大于 max_norm 的嵌入向量都会被重新规范化为具有范数 max_norm。注意，在动态图模式下，将会原位修改权重。默认值：None。 | |||
There was a problem hiding this comment.
有一个其他Bug，不影响本文档合入。
在静态图下，是否会导致梯度无法计算，因为返回了一个新的weight。这个应该写为：
if max_norm:
embedding_renorm_(
x, weight, max_norm=max_norm, norm_type=norm_type
)
保证是原来的weight参与组网计算与反向传播，返回一个新的weight，当前weight就从网络中分离了。这个静态图的问题需要修一下 @AndPuQing
There was a problem hiding this comment.
这样的话，静态图的renorm 是不是就失效了
| @@ -9,7 +9,7 @@ torch.nn.functional.embedding(input, weight, padding_idx=None, max_norm=None, no | |||
| ### [paddle.nn.functional.embedding](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/nn/functional/embedding_cn.html#embedding) | |||
|
|
|||
| ```python | |||
| paddle.nn.functional.embedding(x, weight, padding_idx=None, sparse=False, name=None) | |||
| paddle.nn.functional.embedding(x, weight, padding_idx=None, max_norm=None, norm_type=2.0, sparse=False, name=None) | |||
There was a problem hiding this comment.
分类类别：torch参数更多->仅参数名不一致
There was a problem hiding this comment.
这个的话，scale_grad_by_freq参数暂时还不支持，应该不用修改吧
| @@ -29,8 +31,8 @@ PyTorch 相比 Paddle 支持更多其他参数，具体如下： | |||
| | num_embeddings | num_embeddings | 表示嵌入字典的大小。 | | |||
| | embedding_dim | embedding_dim | 表示每个嵌入向量的维度。 | | |||
| | padding_idx | padding_idx | 在此区间内的参数及对应的梯度将会以 0 进行填充 | | |||
| | max_norm | - | 如果给定，Embeddding 向量的范数（范数的计算方式由 norm_type 决定）超过了 max_norm 这个界限，就要再进行归一化，Paddle 无此参数，暂无转写方式。 | | |||
| | norm_type | - | 为 maxnorm 选项计算 p-范数的 p。默认值 2，Paddle 无此参数，暂无转写方式。 | | |||
| | max_norm | max_norm | 如果给定，Embeddding 向量的范数（范数的计算方式由 norm_type 决定）超过了 max_norm 这个界限，就要再进行归一化。 | | |||
There was a problem hiding this comment.
分类类别：torch参数更多->仅参数名不一致
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
相关链接