Skip to content

Commit

Permalink
[Doc]fix serving doc link
Browse files Browse the repository at this point in the history
  • Loading branch information
ChaoII committed Jul 10, 2024
1 parent cd0ee79 commit 7e1af8f
Show file tree
Hide file tree
Showing 4 changed files with 28 additions and 18 deletions.
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ python/build_cpu.sh
python/fastdeploy/ThirdPartyNotices*
*.so*
fpython/astdeploy/libs/third_libs
fastdeploy/core/config.h
fastdeploy/pybind/main.cc
python/fastdeploy/libs/lib*
python/fastdeploy/libs/third_libs
Expand All @@ -29,7 +28,6 @@ python/scripts/process_libraries.py
.DS_Store
miniprogram_npm
node_modules
.DS_Store
dist
etc
lib
Expand All @@ -40,6 +38,7 @@ yalc.*
.yalc
examples/vision/collect_quantize_cc.sh
examples/vision/tests_quantize
ppocr_keys_v1.txt
fastdeploy/LICENSE
fastdeploy/ThirdPartyNotices.txt
FastDeployCSharp.cmake
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/doc/imgs/12.jpg

# x.y.z为镜像版本号,需参照serving文档替换为数字
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10
docker run -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 bash
docker run --gpus all -dit --net=host --name fastdeploy --shm-size="1g" -v $PWD:/ocr_serving registry.baidubce.com/paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.5-21.10 bash
docker exec -it -u root fastdeploy bash
```

Expand All @@ -81,7 +81,7 @@ fastdeployserver --model-repository=/ocr_serving/models
```

参数:
- `model-repository`(required): 整套模型streaming_pp_tts存放的路径.
- `model-repository`(required): 整套模型PP-OCRv3存放的路径.
- `http-port`(optional): HTTP服务的端口号. 默认: `8000`. 本示例中未使用该端口.
- `grpc-port`(optional): GRPC服务的端口号. 默认: `8001`.
- `metrics-port`(optional): 服务端指标的端口号. 默认: `8002`. 本示例中未使用该端口.
Expand Down
2 changes: 1 addition & 1 deletion serving/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

FastDeploy builds an end-to-end serving deployment based on [Triton Inference Server](https://github.com/triton-inference-server/server). The underlying backend uses the FastDeploy high-performance Runtime module and integrates the FastDeploy pre- and post-processing modules to achieve end-to-end serving deployment. It can achieve fast deployment with easy-to-use process and excellent performance.

> FastDeploy also provides an easy-to-use Python service deployment method, refer [PaddleSeg deployment example](../examples/vision/segmentation/paddleseg/serving/simple_serving) for its usage.
> FastDeploy also provides an easy-to-use Python service deployment method, refer [PaddleSeg deployment example](../examples/vision/segmentation/paddleseg/semantic_segmentation/serving/simple_serving/README.md) for its usage.
## Prepare the environment

Expand Down
37 changes: 24 additions & 13 deletions serving/README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,48 +4,59 @@

## 简介

FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-server/server)搭建了端到端的服务化部署。底层后端使用FastDeploy高性能Runtime模块,并串联FastDeploy前后处理模块实现端到端的服务化部署。具有快速部署、使用简单、性能卓越的特性。
FastDeploy基于[Triton Inference Server](https://github.com/triton-inference-server/server)
搭建了端到端的服务化部署。底层后端使用FastDeploy高性能Runtime模块,并串联FastDeploy前后处理模块实现端到端的服务化部署。具有快速部署、使用简单、性能卓越的特性。

>
FastDeploy同时还提供了基于Python搭建的服务化部署能力,只需要通过Python即可启动服务,可参考[PaddleSeg部署示例](../examples/vision/segmentation/paddleseg/semantic_segmentation/serving/simple_serving/README_CN.md)
了解其用法。

> FastDeploy同时还提供了基于Python搭建的服务化部署能力,只需要通过Python即可启动服务,可参考[PaddleSeg部署示例](../examples/vision/segmentation/paddleseg/serving/simple_serving)了解其用法。
## 准备环境

### 环境要求

- Linux
- 如果使用GPU镜像, 要求NVIDIA Driver >= 470(如果是旧的Tesla架构GPU,如T4使用的NVIDIA Driver可以是418.40+、440.33+、450.51+、460.27+)
- 如果使用GPU镜像, 要求NVIDIA Driver >= 470(如果是旧的Tesla架构GPU,如T4使用的NVIDIA
Driver可以是418.40+、440.33+、450.51+、460.27+)

### 获取镜像

#### CPU镜像

CPU镜像仅支持Paddle/ONNX模型在CPU上进行服务化部署,支持的推理后端包括OpenVINO、Paddle Inference和ONNX Runtime

``` shell
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.7-cpu-only-21.10
```

#### GPU镜像

GPU镜像支持Paddle/ONNX模型在GPU/CPU上进行服务化部署,支持的推理后端包括OpenVINO、TensorRT、Paddle Inference和ONNX Runtime

```
docker pull registry.baidubce.com/paddlepaddle/fastdeploy:1.0.7-gpu-cuda11.4-trt8.5-21.10
```

用户也可根据自身需求,参考如下文档自行编译镜像

- [FastDeploy服务化部署镜像编译说明](docs/zh_CN/compile.md)

## 其它文档

- [模型仓库目录说明](docs/zh_CN/model_repository.md) (说明如何准备模型仓库目录)
- [模型配置说明](docs/zh_CN/model_configuration.md) (说明runtime的配置选项)
- [服务化部署示例](docs/zh_CN/demo.md) (服务化部署示例)
- [客户端访问说明](docs/zh_CN/client.md) (客户端访问说明)
- [Serving可视化部署](docs/zh_CN/vdl_management.md) (Serving可视化部署)


### 服务化部署示例

| 任务场景 | 模型 |
|---|-----------------------------------------------------------------------------------|
| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) |
| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) |
| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) |
| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md) |
| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md) |
| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md) |
| OCR | [PaddleOCR/PP-OCRv3](../examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md) |
| 任务场景 | 模型 |
|----------------|-------------------------------------------------------------------------------------------|
| Classification | [PaddleClas](../examples/vision/classification/paddleclas/serving/README.md) |
| Detection | [PaddleDetection](../examples/vision/detection/paddledetection/serving/README.md) |
| Detection | [ultralytics/YOLOv5](../examples/vision/detection/yolov5/serving/README.md) |
| NLP | [PaddleNLP/ERNIE-3.0](../examples/text/ernie-3.0/serving/README.md) |
| NLP | [PaddleNLP/UIE](../examples/text/uie/serving/README.md) |
| Speech | [PaddleSpeech/PP-TTS](../examples/audio/pp-tts/serving/README.md) |
| OCR | [PaddleOCR/PP-OCRv3](../examples/vision/ocr/PP-OCR/serving/fastdeploy_serving/README.md) |

0 comments on commit 7e1af8f

Please sign in to comment.