【PaddlePaddle Hackathon 2】77、基于ERNIE_DOC完成长文本分类任务

[TCChenlong]

（此 ISSUE 为 PaddlePaddle Hackathon 第二期活动的任务 ISSUE，更多详见 【PaddlePaddle Hackathon 第二期】任务总览）

【任务说明】

• 任务标题：基于ERNIE_DOC完成长文本分类任务

• 技术标签：Python, PaddleNLP，文本分类

• 任务难度：简单

详细描述：参考论文以及PaddleNLP，完成在IFK数据集上的长文本分类任务。

【提交流程】

• 请将相关代码提交到 PaddleNLP repo

【提交内容】

• 完整代码：包括数据、训练、预测、动转静
• 指标
• readme文档：参考文档

【合入标准】

• ERNIE_DOC模型在IFK上ACC=62.4 (见论文 Table 6)

【技术要求】

• 模型使用

【参考内容】

• PaddleNLP repo

【答疑交流】

• 如果在开发中对于上述任务有任何问题，欢迎在本 ISSUE 下留言交流。
• 对于开发中的共性问题，在活动过程中，会定期组织答疑，请大家关注官网&QQ群的通知，及时参与。

[renmada]

ernie-doc的实现好像不支持动转静，原因是paddle.ones的shape要为非负
`
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlenlp/transformers/ernie_doc/modeling.py", line 765, in forward
input_ids, memories, token_type_ids, position_ids, attn_mask)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlenlp/transformers/ernie_doc/modeling.py", line 664, in forward
n_head_self_attn_mask = self._create_n_head_attn_mask(attn_mask,
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlenlp/transformers/ernie_doc/modeling.py", line 555, in _create_n_head_attn_mask
paddle.ones(
shape=[batch_size, self.memory_len, 1],
dtype=attn_mask.dtype), attn_mask
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
],
axis=1)

File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py", line 269, in ones
    return fill_constant(value=1.0, shape=shape, dtype=dtype, name=name)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/tensor.py", line 749, in fill_constant
    check_shape(shape)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py", line 384, in check_shape
    "All elements in ``shape`` must be positive when it's a list or tuple"

ValueError: All elements in ``shape`` must be positive when it's a list or tuple`

[yingyibiao]

必要时可以修改模型代码

[renmada]

改了一些，往下调试还会出现其他因为shape导致的问题

    File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlenlp-2.2.0-py3.7.egg/paddlenlp/transformers/ernie_doc/modeling.py", line 124, in _rel_shift
        """
        # input shape: [B, N, T, 2 * T + M]
        x_shape = x.shape
        ~~~~~~~~~~~~~~~~~ <--- HERE
        x = x.reshape([x_shape[0], x_shape[1], x_shape[3], x_shape[2]])
        x = x[:, :, 1:, :]

    File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/manipulation.py", line 2001, in reshape
        return paddle.fluid.layers.reshape(x=x, shape=shape, name=name)
    File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/nn.py", line 6258, in reshape
        attrs["shape"] = get_attr_shape(shape)
    File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/nn.py", line 6234, in get_attr_shape
        "be -1. But received shape[%d] is also -1." % dim_idx)

    AssertionError: Only one dimension value of 'shape' in reshape can be -1. But received shape[2] is also -1.

[renmada]

双下划线卡头的类方法在静态图是，提示找到不方法。改成单下划线后正常
` File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlenlp-2.2.0-py3.7.egg/paddlenlp/transformers/ernie_doc/modeling.py", line 177, in forward
q, k, v, r, t = self._compute_qkv(queries, keys, values, rel_pos,
rel_task)
q_w, q_r, q_t = list(
~~~~~~~~~~~~~~~~~~~~~ <--- HERE
map(lambda x: q + x.unsqueeze([0, 1]),
[self.r_w_bias, self.r_r_bias, self.r_t_bias]))

File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 1110, in __getattr__
    return object.__getattribute__(self, name)

AttributeError: 'MultiHeadAttention' object has no attribute '__split_heads'`

[zche4846]

请问测试结果62.4是要在CLUEbenchmark上提交才有效是吗

[github-actions]

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天，即将关闭。