Paddlebox #26
Merged
Paddlebox #26
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tiancaitzp
reviewed
Nov 22, 2023
| if(flags.empty()) { | ||
| offset += outputs[i]->dims()[0]; | ||
| } else { | ||
| if(is_expand_slot_small==true){ |
| std::vector<int> slot_inner_offset(total_length); | ||
| int out_count = 0; | ||
| for(int i=0;i<slot_num;i++) { | ||
| for(int j=0;j<slot_lengths[i];j++) { |
There was a problem hiding this comment.
这里的slot_length[i]是int64的值，用int j去迭代循环存在死循环风险
| push_offset, total_length, slot_vector, slot_lens, slot_num, | ||
| hidden_size, batch_size, total_dims, skip_offset, key2slot); | ||
| void *d_slot_inner_offset = nullptr; | ||
| xpu_malloc((void **)&d_slot_inner_offset, total_length * sizeof(int)); |
| std::vector<int> slot_inner_offset(total_length); | ||
| int out_count = 0; | ||
| for(int i=0;i<slot_num;i++) { | ||
| for(int j=0;j<slot_lengths[i];j++) { |
There was a problem hiding this comment.
< * 这些双目运算符的两边空格，后面建议都修改以下
| if(outputs[embedx_offset]->numel()==0) { | ||
| outputs[embedx_offset]->set_layout(paddle::framework::DataLayout::UNDEFINED); | ||
| } else { | ||
| int offset = slot_dims0_offset[i]*dims1* sizeof(T); |
There was a problem hiding this comment.
这里的offset类型建议用size_t, 后面set_offset传进去的就是size_t类型。
相应的，slot_dims0_offset也建议改成size_t类型
HuangShiqing
force-pushed
the
paddlebox
branch
from
48b7466
to
e85e2a4
Compare
HuangShiqing
force-pushed
the
paddlebox
branch
from
e85e2a4
to
da37757
Compare
tiancaitzp
approved these changes
Nov 22, 2023
| @@ -3,6 +3,7 @@ | |||
| #include "xpu/kernel/xtdk.h" | |||
| // #include "xpu/kernel/debug.h" | |||
| #include "xpu/kernel/xtdk_math.h" | |||
| #include "xpu/kernel/xtdk_io.h" | |||
| @@ -2114,8 +2114,11 @@ void PadBoxSlotDataset::PreLoadIntoDisk(const std::string& path, | |||
| } | |||
| CHECK(slot_pool_ != nullptr) << "slotrecord pool nullptr"; | |||
| read_ins_ref_ = thread_num_; | |||
| if (disable_shuffle_) { | |||
| read_ins_ref_ = 1; | |||
| } | |||
There was a problem hiding this comment.
此处会有问题，参考PadBoxSlotDataset::PreLoadIntoMemory()中修改方式
| @@ -30,6 +30,9 @@ limitations under the License. */ | |||
| #include "xpu/kernel/xtdk_simd.h" | |||
|
|
|||
| #ifdef TRACE_PROFILE | |||
| #include "xpu/kernel/xtdk_io.h" | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
PR changes
Describe