Add a c-api interface to initialize the thread environment of Paddle #5773

Xreki · 2017-11-20T05:33:56Z

Fix C-API库进行gpu多线程inference时，出现cublas status: not initialized错误 #5669
- paddle_init_thread() need to be called in a newly launched thread to initialize the thread environment of Paddle
Add a GPU c-api example

… develop

…nd add a GPU example.

… develop

QingshuChen · 2017-11-20T06:14:59Z

paddle/capi/Main.cpp

+  if (isInit) return kPD_NO_ERROR;
+
+  if (FLAGS_use_gpu) {
+    hl_init(FLAGS_gpu_id);


hl_init will set the t_resource.device to -1, is it a bug?

hedaoyuan · 2017-11-20T06:53:23Z

paddle/capi/Main.cpp

@@ -43,4 +43,16 @@ paddle_error paddle_init(int argc, char** argv) {
  isInit = true;
  return kPD_NO_ERROR;
 }
+
+paddle_error paddle_init_thread() {
+  static __thread bool isInit = false;


Do not need 48 and 49 lines.
In the hl_init interface will determine whether it has been initialized.

hedaoyuan · 2017-11-20T06:56:04Z

paddle/capi/Main.cpp

@@ -43,4 +43,16 @@ paddle_error paddle_init(int argc, char** argv) {
  isInit = true;
  return kPD_NO_ERROR;
 }
+
+paddle_error paddle_init_thread() {


Need to add an argument of device_id.
Users may want to initialize the thread to a different device environment.

If adding a device_id, another question is when user invokes the paddle_matrix_create API, which gpu is the matrix on? There is no device id in the paddle_matrix_create API.

Maybe need to wrap the hl_get_device interface.

I tried to add device_id and set it to different values. The binary failed:

F1120 08:05:09.430253 1282 hl_cuda_device.cc:565] Check failed: cudaSuccess == cudaStat (0 vs. 77) Cuda Error: an illegal memory access was encountered

This problem is due to the thread share the same parameter model with the main thread. In this case, you need to pass in the same device id as the main thread.

I think I know what your means. In the current mode, the Parameters are shared among multi-threads, but each thread has its own network. If each thread wants to run on different GPU, it should have its own network + Parameters. Then Parameters cannot be shared anymore, and we should not use the interface paddle_gradient_machine_create_shared_param but directly using paddle_gradient_machine_create_for_inference and paddle_gradient_machine_load_parameter_from_disk instead.

我认为，这个多线程的例子，是希望使用paddle_gradient_machine_create_shared_param共享参数，不能支持跑在不同的GPU上。

如果希望多个线程跑在不同的GPU上，则应该使用trainer_count>1或者，在每个线程里面单独地paddle_gradient_machine_create_for_inference以及paddle_gradient_machine_load_parameter_from_disk。

hedaoyuan · 2017-11-20T07:01:37Z

paddle/capi/error.h

@@ -27,4 +27,21 @@ typedef enum {
  kPD_UNDEFINED_ERROR = -1,
 } paddle_error;

+static const char* paddle_error_string(paddle_error err) {


Move the function implementation into a .cc file and avoid generating duplicate copies of code in the inference library.

hedaoyuan · 2017-11-20T07:07:50Z

paddle/capi/main.h

+/**
+ * Initialize the thread environment of Paddle.
+ */
+PD_API paddle_error paddle_init_thread();


Since some previous CPU users did not use this interface. Here need to indicate only GPU need this interface.

…addle_error_string into a .cpp file.

hedaoyuan · 2017-12-08T02:33:02Z

paddle/capi/examples/model_inference/multi_thread/main_gpu.c

+#include <pthread.h>
+#include <time.h>
+#include "../common/common.h"
+


Add some comments about this example. For example, this is an inference implementation where multiple threads share a GPU.

Xreki added 4 commits November 13, 2017 10:26

Fix bug in MergeModel.cpp.

f1996bc

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

5c829af

… develop

Add a c-api inferface to initilize the thread environment of Paddle a…

0cc1b6c

…nd add a GPU example.

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

64418e3

… develop

Xreki mentioned this pull request Nov 20, 2017

C-API库进行gpu多线程inference时，出现cublas status: not initialized错误 #5669

Closed

Xreki requested a review from hedaoyuan November 20, 2017 05:43

Merge branch 'develop' into fix_capi_multi_thread

3948801

Xreki force-pushed the fix_capi_multi_thread branch from 862b3b6 to 3948801 Compare November 20, 2017 05:55

QingshuChen reviewed Nov 20, 2017

View reviewed changes

hedaoyuan requested changes Nov 20, 2017

View reviewed changes

Add some note for paddle_init_thread and move the inplementation of p…

ee5df62

…addle_error_string into a .cpp file.

Xreki mentioned this pull request Dec 7, 2017

[C-API]需要一个GPU的运行示例 #6389

Closed

hedaoyuan previously approved these changes Dec 8, 2017

View reviewed changes

Xreki dismissed hedaoyuan’s stale review via 1d78cf9 December 8, 2017 02:45

hedaoyuan previously approved these changes Dec 8, 2017

View reviewed changes

Add some comments.

68f6b80

Xreki dismissed hedaoyuan’s stale review via 68f6b80 December 8, 2017 02:52

Xreki force-pushed the fix_capi_multi_thread branch from 1d78cf9 to 68f6b80 Compare December 8, 2017 02:52

hedaoyuan approved these changes Dec 8, 2017

View reviewed changes

Xreki merged commit 00b64f6 into PaddlePaddle:develop Dec 8, 2017

Xreki deleted the fix_capi_multi_thread branch November 14, 2018 02:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a c-api interface to initialize the thread environment of Paddle #5773

Add a c-api interface to initialize the thread environment of Paddle #5773

Xreki commented Nov 20, 2017 •

edited

Loading

QingshuChen Nov 20, 2017

hedaoyuan Nov 20, 2017

Xreki Nov 20, 2017

hedaoyuan Nov 20, 2017

QingshuChen Nov 20, 2017 •

edited

Loading

hedaoyuan Nov 20, 2017

Xreki Nov 20, 2017

hedaoyuan Nov 20, 2017

Xreki Nov 27, 2017

Xreki Dec 5, 2017

hedaoyuan Nov 20, 2017

Xreki Nov 20, 2017

hedaoyuan Nov 20, 2017

Xreki Nov 20, 2017

hedaoyuan Dec 8, 2017

Xreki Dec 8, 2017

Add a c-api interface to initialize the thread environment of Paddle #5773

Add a c-api interface to initialize the thread environment of Paddle #5773

Conversation

Xreki commented Nov 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

QingshuChen Nov 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Xreki commented Nov 20, 2017 •

edited

Loading

QingshuChen Nov 20, 2017 •

edited

Loading