Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(limit-count): support multiple variables as key #5378

Merged
merged 6 commits into from
Nov 4, 2021
Merged

feat(limit-count): support multiple variables as key #5378

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
555405b
feat(limit-count): support multiple variables as key
Xunzhuo Oct 30, 2021
506244e
test: add cases when key_type is var
Xunzhuo Nov 4, 2021
c70c8ed
fix: remove $ symbol when key_type is var
Xunzhuo Nov 4, 2021
e1f5bda
fix: change http_x_real_ip to remote_addr
Xunzhuo Nov 4, 2021
5b112eb
test: add more details
Xunzhuo Nov 4, 2021
b329fb8
fix: change relative http_x_real_ip to remote_addr
Xunzhuo Nov 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 27 additions & 6 deletions apisix/plugins/limit-count.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,10 @@ local schema = {
properties = {
count = {type = "integer", exclusiveMinimum = 0},
time_window = {type = "integer", exclusiveMinimum = 0},
key = {
type = "string",
enum = {"remote_addr", "server_addr", "http_x_real_ip",
"http_x_forwarded_for", "consumer_name", "service_id"},
default = "remote_addr",
key = {type = "string", default = "remote_addr"},
key_type = {type = "string",
enum = {"var", "var_combination"},
tokers marked this conversation as resolved.
Show resolved Hide resolved
default = "var",
},
rejected_code = {
type = "integer", minimum = 200, maximum = 599, default = 503
Expand Down Expand Up @@ -171,7 +170,29 @@ function _M.access(conf, ctx)
return 500
end

local key = (ctx.var[conf.key] or "") .. ctx.conf_type .. ctx.conf_version
local conf_key = conf.key
local key
if conf.key_type == "var_combination" then
local err, n_resolved
key, err, n_resolved = core.utils.resolve_var(conf_key, ctx.var);
if err then
core.log.error("could not resolve vars in ", conf_key, " error: ", err)
end

if n_resolved == 0 then
key = nil
end
else
key = (ctx.var[conf_key] or "")
Xunzhuo marked this conversation as resolved.
Show resolved Hide resolved
end

if key == nil then
core.log.info("bypass the limit count as the key is empty")
-- Bypass the limit count when the key is empty.
-- This behavior is the same as Nginx
return
end
key = key .. ctx.conf_type .. ctx.conf_version
core.log.info("limit key: ", key)

local delay, remaining = lim:incoming(key, true)
Expand Down
34 changes: 29 additions & 5 deletions docs/en/latest/plugins/limit-count.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ Limit request rate by a fixed number of requests in a given time window.
| ------------------- | ------- | --------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| count | integer | required | | count > 0 | the specified number of requests threshold. |
| time_window | integer | required | | time_window > 0 | the time window in seconds before the request count is reset. |
| key | string | optional | "remote_addr" | ["remote_addr", "server_addr", "http_x_real_ip", "http_x_forwarded_for", "consumer_name", "service_id"] | The user specified key to limit the count. <br /> Now accept those as key: "remote_addr"(client's IP), "server_addr"(server's IP), "X-Forwarded-For/X-Real-IP" in request header, "consumer_name"(consumer's username) and "service_id". |
| key_type | string | optional | "var" | ["var", "var_combination"] | the type of key. |
| key | string | optional | "remote_addr" | | the user specified key to limit the rate. If the `key_type` is "var", the key will be treated as a name of variable. If the `key_type` is "var_combination", the key will be a combination of variables. For example, if we use "$remote_addr $consumer_name" as keys, plugin will be restricted by two keys which are "remote_addr" and "consumer_name". |
| rejected_code | integer | optional | 503 | [200,...,599] | The HTTP status code returned when the request exceeds the threshold is rejected, default 503. |
| rejected_msg | string | optional | | non-empty | The response body returned when the request exceeds the threshold is rejected. |
| policy | string | optional | "local" | ["local", "redis", "redis-cluster"] | The rate-limiting policies to use for retrieving and incrementing the limits. Available values are `local`(the counters will be stored locally in-memory on the node), `redis`(counters are stored on a Redis server and will be shared across the nodes, usually use it to do the global speed limit), and `redis-cluster` which works the same as `redis` but with redis cluster. |
Expand All @@ -53,11 +54,9 @@ Limit request rate by a fixed number of requests in a given time window.
| redis_cluster_nodes | array | required when policy is `redis-cluster` | | | When using `redis-cluster` policy，This property is a list of addresses of Redis cluster service nodes (at least two). |
| redis_cluster_name | string | required when policy is `redis-cluster` | | | When using `redis-cluster` policy, this property is the name of Redis cluster service nodes. |

**Key can be customized by the user, only need to modify a line of code of the plug-in to complete. It is a security consideration that is not open in the plugin.**

## How To Enable

Here's an example, enable the `limit count` plugin on the specified route:
Here's an example, enable the `limit count` plugin on the specified route when setting `key_type` to `var` :

```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
Expand All @@ -68,13 +67,38 @@ curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key_type": "var",
"key": "remote_addr"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"39.97.63.215:80": 1
"127.0.0.1:9001": 1
}
}
}'
```

Here's an example, enable the `limit count` plugin on the specified route when setting `key_type` to `var_combination` :

```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/index.html",
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key_type": "var_combination",
"key": "$consumer_name $remote_addr"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"127.0.0.1:9001": 1
}
}
}'
Expand Down
31 changes: 27 additions & 4 deletions docs/zh/latest/plugins/limit-count.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ title: limit-count
| ------------------- | ------- | --------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| count | integer | 必须 | | count > 0 | 指定时间窗口内的请求数量阈值 |
| time_window | integer | 必须 | | time_window > 0 | 时间窗口的大小（以秒为单位），超过这个时间就会重置 |
| key | string | 可选 | "remote_addr" | ["remote_addr", "server_addr", "http_x_real_ip", "http_x_forwarded_for", "consumer_name", "service_id"] | 用来做请求计数的有效值。<br />例如，可以使用主机名（或服务器区域）作为关键字，以便限制每个主机名规定时间内的请求次数。我们也可以使用客户端地址作为关键字，这样我们就可以避免单个客户端规定时间内多次的连接我们的服务。<br />当前接受的 key 有："remote_addr"（客户端 IP 地址）, "server_addr"（服务端 IP 地址）, 请求头中的"X-Forwarded-For" 或 "X-Real-IP", "consumer_name"（consumer 的 username）, "service_id" 。 |
| key_type | string | 可选 | "var" | ["var", "var_combination"] | key 的类型 |
| key | string | 可选 | "remote_addr" | | 用来做请求计数的依据。如果 `key_type` 为 "var"，那么 key 会被当作变量名称。如果 `key_type` 为 "var_combination"，那么 key 会当作变量组。比如如果设置 "$remote_addr $consumer_name" 作为 keys，那么插件会同时受 remote_addr 和 consumer_name 两个 key 的约束。 |
| rejected_code | integer | 可选 | 503 | [200,...,599] | 当请求超过阈值被拒绝时，返回的 HTTP 状态码 |
| rejected_msg | string | 可选 | | 非空 | 当请求超过阈值被拒绝时，返回的响应体。 |
| policy | string | 可选 | "local" | ["local", "redis", "redis-cluster"] | 用于检索和增加限制的速率限制策略。可选的值有：`local`(计数器被以内存方式保存在节点本地，默认选项) 和 `redis`(计数器保存在 Redis 服务节点上，从而可以跨节点共享结果，通常用它来完成全局限速)；以及`redis-cluster`，跟 redis 功能一样，只是使用 redis 集群方式。 |
Expand All @@ -56,13 +57,11 @@ title: limit-count
| redis_cluster_nodes | array | 当 policy 为 `redis-cluster` 时必填| | | 当使用 `redis-cluster` 限速策略时，该属性是 Redis 集群服务节点的地址列表（至少需要两个地址）。 |
| redis_cluster_name | string | 当 policy 为 `redis-cluster` 时必填 | | | 当使用 `redis-cluster` 限速策略时，该属性是 Redis 集群服务节点的名称。 |

**key 是可以被用户自定义的，只需要修改插件的一行代码即可完成。并没有在插件中放开是处于安全的考虑。**

## 如何使用

### 开启插件

下面是一个示例，在指定的 `route` 上开启了 `limit count` 插件:
下面是一个示例，在指定的 `route` 上开启了 `limit count` 插件，并设置 `key_type` 为 `var`：

```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
Expand All @@ -85,6 +84,30 @@ curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335
}'
```

下面是一个示例，在指定的 `route` 上开启了 `limit count` 插件，并设置 `key_type` 为 `var_combination`：

```shell
curl -i http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/index.html",
"plugins": {
"limit-count": {
"count": 2,
"time_window": 60,
"rejected_code": 503,
"key_type": "var_combination",
"key": "$consumer_name $remote_addr"
}
},
"upstream": {
"type": "roundrobin",
"nodes": {
"127.0.0.1:9001": 1
}
}
}'
```

你也可以通过 web 界面来完成上面的操作，先增加一个 route，然后在插件页面中添加 limit-count 插件：
![添加插件](../../../assets/images/plugin/limit-count-1.png)

Expand Down
2 changes: 1 addition & 1 deletion t/control/services.t
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ services:
}
}
--- response_body
{"id":"5","plugins":{"limit-count":{"allow_degradation":false,"count":2,"key":"remote_addr","policy":"local","rejected_code":503,"show_limit_quota_header":true,"time_window":60}},"upstream":{"hash_on":"vars","nodes":[{"host":"127.0.0.1","port":1980,"weight":1}],"pass_host":"pass","scheme":"http","type":"roundrobin"}}
{"id":"5","plugins":{"limit-count":{"allow_degradation":false,"count":2,"key":"remote_addr","key_type":"var","policy":"local","rejected_code":503,"show_limit_quota_header":true,"time_window":60}},"upstream":{"hash_on":"vars","nodes":[{"host":"127.0.0.1","port":1980,"weight":1}],"pass_host":"pass","scheme":"http","type":"roundrobin"}}



Expand Down
5 changes: 2 additions & 3 deletions t/plugin/limit-count.t
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,12 @@ done



=== TEST 2: wrong value of key
=== TEST 2: set key empty
--- config
location /t {
content_by_lua_block {
local plugin = require("apisix.plugins.limit-count")
local ok, err = plugin.check_schema({count = 2, time_window = 60, rejected_code = 503, key = 'host'})
local ok, err = plugin.check_schema({count = 2, time_window = 60, rejected_code = 503})
if not ok then
ngx.say(err)
end
Expand All @@ -72,7 +72,6 @@ done
--- request
GET /t
--- response_body
property "key" validation failed: matches none of the enum values
done
--- no_error_log
[error]
Expand Down
Loading