Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(response-rewrite): support filters #6750

Merged
merged 23 commits into from
Apr 11, 2022
Merged

feat(response-rewrite): support filters #6750

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
5bcf2f5
feat(response-rewrite): support filters
kwanhur Mar 29, 2022
89b67f4
test(plugin): split response-rewrite test cases into two parts
kwanhur Apr 1, 2022
0f70eeb
feat(response-rewrite): remove filters field type checking
kwanhur Apr 1, 2022
e505e6d
feat(response-rewrite): return at the end of conf.filters
kwanhur Apr 1, 2022
9177cd4
chore(annotation): remove improper fixme
kwanhur Apr 1, 2022
562adb1
feat(response-rewrite): remove conf.filters field type and value chec…
kwanhur Apr 1, 2022
d3e82b7
feat(response-rewrite): check filter.regex invalid when empty
kwanhur Apr 1, 2022
5d2978a
feat(response-rewrite): add prefix message at filter.regex and option…
kwanhur Apr 1, 2022
54a472e
test(plugin): reindex response-rewrite test cases
kwanhur Apr 1, 2022
cce127d
feat(response-rewrite): add minItems 1 limitation on conf.filters
kwanhur Apr 1, 2022
9c87396
Merge branch 'feat-resp-filter' of github.com:kwanhur/apisix into fea…
kwanhur Apr 1, 2022
656124d
docs(response-rewrite): merge filters nested fields
kwanhur Apr 1, 2022
dc809e1
feat(response-rewrite): filters description with string concat
kwanhur Apr 3, 2022
93e6e23
feat(response-rewrite): revert to if on conf.body
kwanhur Apr 3, 2022
f42d160
feat(response-rewrite): fix filters.scope one to once
kwanhur Apr 3, 2022
f46ca71
feat(response-rewrite): add required on filter field regex and replace
kwanhur Apr 6, 2022
4d6abb8
feat(response-rewrite): pcall compile invalid regex options
kwanhur Apr 6, 2022
451627b
feat(response-rewrite): remove default value and update docs on requi…
kwanhur Apr 7, 2022
2767308
chore: place one space before `filter.regex`
kwanhur Apr 7, 2022
3ec79ac
feat(response-rewrite): filter.regex minLength 1
kwanhur Apr 8, 2022
df3ede3
docs(response-rewrite): update filter scope option once and global usage
kwanhur Apr 9, 2022
c42d70c
docs(response-rewrite): fix regex options referece link
kwanhur Apr 9, 2022
cd3f883
feat(response-rewrite): add re_sub/gsub err log
kwanhur Apr 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 74 additions & 1 deletion apisix/plugins/response-rewrite.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,15 @@
--
local core = require("apisix.core")
local expr = require("resty.expr.v1")
local re_compile = require("resty.core.regex").re_match_compile
local plugin_name = "response-rewrite"
local ngx = ngx
local re_sub = ngx.re.sub
local re_gsub = ngx.re.gsub
local pairs = pairs
local ipairs = ipairs
local type = type
local pcall = pcall


local schema = {
Expand Down Expand Up @@ -48,6 +53,41 @@ local schema = {
vars = {
type = "array",
},
filters = {
description = "a group of filters that modify response body" ..
"by replacing one specified string by another",
type = "array",
minItems = 1,
items = {
description = "filter that modifies response body",
type = "object",
required = {"regex", "replace"},
properties = {
regex = {
description = "match pattern on response body",
type = "string",
default = "",
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
},
scope = {
description = "regex substitution range",
type = "string",
enum = {"once", "global"},
default = "once",
},
replace = {
description = "regex substitution content",
type = "string",
default = "",
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
},
options = {
description = "regex options",
type = "string",
default = "jo",
}
},
},
},
oneOf = {"body", "filters"},
},
minProperties = 1,
}
Expand Down Expand Up @@ -115,6 +155,19 @@ function _M.check_schema(conf)
end
end

if conf.filters then
for _, filter in ipairs(conf.filters) do
if filter.regex == "" then
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
return false, "invalid value as filter field regex"
end
local ok, err = pcall(re_compile,filter.regex, filter.options)
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
if not ok then
return false, "regex \"" .. filter.regex ..
"\" validation failed: " .. err
end
end
end

return true
end

Expand All @@ -126,6 +179,25 @@ function _M.body_filter(conf, ctx)
return
end

if conf.filters then

local body = core.response.hold_body_chunk(ctx)
if not body then
return
end

for _, filter in ipairs(conf.filters) do
if filter.scope == "once" then
body = re_sub(body, filter.regex, filter.replace, filter.options)
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
else
body = re_gsub(body, filter.regex, filter.replace, filter.options)
end
end

ngx.arg[1] = body
return
end

kwanhur marked this conversation as resolved.
Show resolved Hide resolved
if conf.body then

if conf.body_base64 then
Expand All @@ -148,7 +220,8 @@ function _M.header_filter(conf, ctx)
ngx.status = conf.status_code
end

if conf.body then
-- if filters have no any match, response body won't be modified.
if conf.filters or conf.body then
core.response.clear_header_as_body_modified()
end

Expand Down
21 changes: 14 additions & 7 deletions docs/en/latest/plugins/response-rewrite.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,20 @@ response rewrite plugin, rewrite the content returned by the upstream as well as

## Attributes

| Name | Type | Requirement | Default | Valid | Description |
| ----------- | ------- | ----------- | ------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| status_code | integer | optional | | [200, 598] | New `status code` to client, keep the original response code by default. |
| body | string | optional | | | New `body` to client, and the content-length will be reset too. |
| body_base64 | boolean | optional | false | | Identify if `body` in configuration need base64 decoded before rewrite to client. |
| headers | object | optional | | | Set the new `headers` for client, can set up multiple. If it exists already from upstream, will rewrite the header, otherwise will add the header. You can set the corresponding value to an empty string to remove a header. The value can contain Nginx variables in `$var` format, like `$remote_addr $balancer_ip` |
| vars | array[] | optional | | | A DSL to evaluate with the given ngx.var. See `vars` [lua-resty-expr](https://github.com/api7/lua-resty-expr#operator-list). if the `vars` is empty, then all rewrite operations will be executed unconditionally |
| Name | Type | Requirement | Default | Valid | Description |
|-----------------|---------|-------------|---------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| status_code | integer | optional | | [200, 598] | New `status code` to client, keep the original response code by default. |
| body | string | optional | | | New `body` to client, and the content-length will be reset too. |
| body_base64 | boolean | optional | false | | Identify if `body` in configuration need base64 decoded before rewrite to client. |
| headers | object | optional | | | Set the new `headers` for client, can set up multiple. If it exists already from upstream, will rewrite the header, otherwise will add the header. You can set the corresponding value to an empty string to remove a header. The value can contain Nginx variables in `$var` format, like `$remote_addr $balancer_ip`. |
| vars | array[] | optional | | | A DSL to evaluate with the given ngx.var. See `vars` [lua-resty-expr](https://github.com/api7/lua-resty-expr#operator-list). if the `vars` is empty, then all rewrite operations will be executed unconditionally. |
| filters | array[] | optional | | | A group of filters that modify response body by replacing one specified string by another. |
| filters.regex | string | optional | | | match pattern on response body. |
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
| filters.scope | string | optional | "once" | "once","global" | substitution range. |
kwanhur marked this conversation as resolved.
Show resolved Hide resolved
| filters.replace | string | optional | | | substitution content. |
| filters.options | string | optional | "jo" | | regex options, See `[ngx.re.match](https://github.com/openresty/lua-nginx-module#ngxrematch)`. |

Only one of `body`, `filters` can be specified.

## How To Enable

Expand Down
21 changes: 14 additions & 7 deletions docs/zh/latest/plugins/response-rewrite.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,20 @@ title: response-rewrite

## 属性

| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
| ----------- | ------- | ------ | ------ | ---------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| status_code | integer | 可选 | | [200, 598] | 修改上游返回状态码，默认保留原始响应代码。 |
| body | string | 可选 | | | 修改上游返回的 `body` 内容，如果设置了新内容，header 里面的 content-length 字段也会被去掉 |
| body_base64 | boolean | 可选 | false | | 描述 `body` 字段是否需要 base64 解码之后再返回给客户端，用在某些图片和 Protobuffer 场景 |
| headers | object | 可选 | | | 返回给客户端的 `headers`，这里可以设置多个。头信息如果存在将重写，不存在则添加。想要删除某个 header 的话，把对应的值设置为空字符串即可。这个值能够以 `$var` 的格式包含 Nginx 变量，比如 `$remote_addr $balancer_ip` |
| vars | array[] | 可选 | | | `vars` 是一个表达式列表，只有满足条件的请求和响应才会修改 body 和 header 信息，来自 [lua-resty-expr](https://github.com/api7/lua-resty-expr#operator-list)。如果 `vars` 字段为空，那么所有的重写动作都会被无条件的执行。 |
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
|-----------------|---------|-----|--------|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| status_code | integer | 可选 | | [200, 598] | 修改上游返回状态码，默认保留原始响应代码。 |
| body | string | 可选 | | | 修改上游返回的 `body` 内容，如果设置了新内容，header 里面的 content-length 字段也会被去掉。 |
| body_base64 | boolean | 可选 | false | | 描述 `body` 字段是否需要 base64 解码之后再返回给客户端，用在某些图片和 Protobuffer 场景。 |
| headers | object | 可选 | | | 返回给客户端的 `headers`，这里可以设置多个。头信息如果存在将重写，不存在则添加。想要删除某个 header 的话，把对应的值设置为空字符串即可。这个值能够以 `$var` 的格式包含 Nginx 变量，比如 `$remote_addr $balancer_ip`。 |
| vars | array[] | 可选 | | | `vars` 是一个表达式列表，只有满足条件的请求和响应才会修改 body 和 header 信息，来自 [lua-resty-expr](https://github.com/api7/lua-resty-expr#operator-list)。如果 `vars` 字段为空，那么所有的重写动作都会被无条件的执行。 |
| filters | array[] | 可选 | | | 一组过滤器，采用指定字符串表达式修改响应体。 |
| filters.regex | string | 可选 | | | 用于匹配响应体正则表达式。 |
| filters.scope | string | 可选 | "once" | "once","global" | 替换范围。 |
| filters.replace | string | 可选 | | | 替换后的内容。 |
| filters.options | string | 可选 | "jo" | | 正则匹配有效参数，可选项见 `[ngx.re.match](https://github.com/openresty/lua-nginx-module#ngxrematch)`。 |

`body` 和 `filters`，两个只能配置其中一个。

## 示例

Expand Down
Loading