-
Notifications
You must be signed in to change notification settings - Fork 714
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Docathon][Add CN Doc No.26] #6390
Closed
Closed
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
e0e3924
[Docathon][Fix COPY-FROM No.6-10]
Kaedeharai 094b07a
[Docathon][Fix COPY-FROM No.6-10]
Kaedeharai bf9a460
Merge branch 'develop' of https://github.com/PaddlePaddle/docs into m…
Kaedeharai 02cdb30
Merge branch 'develop' of https://github.com/PaddlePaddle/docs into m…
Kaedeharai 5884bd4
文档修复80
Kaedeharai 536fc6f
Merge branch 'develop' of https://github.com/PaddlePaddle/docs into kk
Kaedeharai 21ecfe7
Merge branch 'develop' of https://github.com/PaddlePaddle/docs into kk
Kaedeharai 2370751
[Docathon][Add CN Doc No.26]
Kaedeharai 6123dbd
Apply suggestions from code review
Kaedeharai cce66c4
Merge branch 'PaddlePaddle:develop' into kk
Kaedeharai b1eb856
Merge branch 'PaddlePaddle:develop' into kk
Kaedeharai 0d197b1
Update docs/api/paddle/nn/Overview_cn.rst
Kaedeharai File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
42 changes: 42 additions & 0 deletions
42
docs/api/paddle/nn/functional/scaled_dot_product_attention_cn.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
.. _cn_api_paddle_nn_functional_scaled_dot_product_attention: | ||
|
||
scaled_dot_product_attention | ||
------------------------------- | ||
|
||
.. py:function:: paddle.nn.functional.scaled_dot_product_attention(query, key, value, attn_mask=None, dropout_p=0.0, is_causal=False, training=True, name=None) | ||
|
||
计算公式为: | ||
|
||
.. math:: | ||
result=softmax(\frac{ Q * K^T }{\sqrt{d}}) * V | ||
|
||
其中, ``Q``、``K`` 和 ``V`` 表示注意力模块的三个输入参数。这三个参数的尺寸相同。``d`` 表示三个参数中最后一个维度的大小。 | ||
|
||
.. warning:: | ||
此 API 仅支持数据类型为 float16 和 bfloat16 的输入。 | ||
|
||
|
||
参数 | ||
:::::::::: | ||
|
||
- **query** (Tensor) - 注意力模块中的查询张量。具有以下形状的四维张量:[batch_size, seq_len, num_heads, head_dim]。数据类型可以是 float61 或 bfloat16。 | ||
- **key** (Tensor) - 注意力模块中的关键张量。具有以下形状的四维张量:[batch_size, seq_len, num_heads, head_dim]。数据类型可以是 float61 或 bfloat16。 | ||
- **value** (Tensor) - 注意力模块中的值张量。具有以下形状的四维张量: [batch_size, seq_len, num_heads, head_dim]。数据类型可以是 float61 或 bfloat16。 | ||
- **attn_mask** (Tensor, 可选) - 与添加到注意力分数的 ``query``、 ``key``、 ``value`` 类型相同的浮点掩码, 默认值为空。 | ||
- **dropout_p** (float) - ``dropout`` 的比例, 默认值为 0.00 即不进行正则化。 | ||
- **is_causal** (bool) - 是否启用因果关系, 默认值为 False 即不启用。 | ||
- **training** (bool): - 是否处于训练阶段, 默认值为 True 即处于训练阶段。 | ||
- **name** (str, 可选) - 默认值为 None。通常不需要用户设置此属性。欲了解更多信息, 请参阅:ref:`api_guide_Name`。 | ||
|
||
|
||
返回 | ||
:::::::::: | ||
|
||
- ``out`` (Tensor): 形状为 ``[batch_size, seq_len, num_heads, head_dim]`` 的 4 维张量。数据类型可以是 float16 或 bfloat16。 | ||
- ``softmax`` (Tensor): 如果 return_softmax 为 False,则为 None。 | ||
|
||
|
||
代码示例 | ||
:::::::::: | ||
|
||
COPY-FROM: paddle.nn.functional.scaled_dot_product_attention |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
该API并没有
return_softmax
参数,也没有返回softmax
。There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
更新在