Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Openvpn 2.6 causes high CPU usage #2313

Open
engageub opened this issue Jun 7, 2024 · 6 comments
Open

Bug: Openvpn 2.6 causes high CPU usage #2313

engageub opened this issue Jun 7, 2024 · 6 comments
Labels
Category: OpenVPN 🔐 Priority: 🚨 Urgent Status: 🗯️ Waiting for feedback

Comments

@engageub
Copy link

engageub commented Jun 7, 2024 •
edited

Is this urgent?

None

Host OS

Ubuntu 22

CPU arch

x86_64

VPN service provider

OPENVPN

What are you using to run the container

docker run

What is the version of Gluetun

Running version latest built on 2024-05-18T18:08:57.405Z (commit 4218dba)

What's the problem 🤔

When the docker image qmcgaw/gluetun is used. The CPU utilization of the container goes to about 100% of 1 CORE where as the following image qmcgaw/gluetun:v3.37.0 uses less than 1% of 1 CORE.
Could you please look into the latest version and compare it with v3.37.0.

Share your logs (at least 10 lines)

root@vmi1921324:~/InternetIncome-test# sudo docker container logs gluetuntest
========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version latest built on 2024-05-18T18:08:57.405Z (commit 4218dba)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2024-06-07T07:37:19Z INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.8 and family v4
2024-06-07T07:37:19Z INFO [routing] local ethernet link found: eth0
2024-06-07T07:37:19Z INFO [routing] local ipnet found: 172.17.0.0/16
2024-06-07T07:37:19Z INFO [firewall] enabling...
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --policy INPUT DROP
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --policy OUTPUT DROP
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --policy FORWARD DROP
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --policy INPUT DROP
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --policy OUTPUT DROP
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --policy FORWARD DROP
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --append INPUT -i lo -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --append INPUT -i lo -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --append OUTPUT -o lo -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --append OUTPUT -o lo -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --append OUTPUT -o eth0 -s 172.17.0.8 -d 172.17.0.0/16 -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] ip6tables --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2024-06-07T07:37:19Z DEBUG [firewall] iptables-legacy --append INPUT -i eth0 -d 172.17.0.0/16 -j ACCEPT
2024-06-07T07:37:19Z INFO [firewall] enabled successfully
2024-06-07T07:37:20Z INFO [storage] creating /gluetun/servers.json with 19425 hardcoded servers
2024-06-07T07:37:20Z DEBUG [netlink] IPv6 is not supported after searching 0 routes
2024-06-07T07:37:20Z INFO Alpine version: 3.19.1
2024-06-07T07:37:20Z INFO OpenVPN 2.5 version: 2.5.8
2024-06-07T07:37:20Z INFO OpenVPN 2.6 version: 2.6.8
2024-06-07T07:37:20Z INFO Unbound version: 1.20.0
2024-06-07T07:37:20Z INFO IPtables version: v1.8.10
2024-06-07T07:37:20Z INFO Settings summary:
├── VPN settings:
|   ├── VPN provider settings:
|   |   ├── Name: custom
|   |   └── Server selection settings:
|   |       ├── VPN type: openvpn
|   |       └── OpenVPN server selection settings:
|   |           ├── Protocol: UDP
|   |           └── Custom configuration file: /gluetun/custom.conf
|   └── OpenVPN settings:
|       ├── OpenVPN version: 2.6
|       ├── User: [set]
|       ├── Password: [set]
|       ├── Custom configuration file: /gluetun/custom.conf
|       ├── Network interface: tun0
|       ├── Run OpenVPN as: root
|       └── Verbosity level: 1
├── DNS settings:
|   ├── Keep existing nameserver(s): no
|   ├── DNS server address to use: 127.0.0.1
|   └── DNS over TLS settings:
|       └── Enabled: no
├── Firewall settings:
|   └── Enabled: yes
├── Log settings:
|   └── Log level: debug
├── Health settings:
|   ├── Server listening address: 127.0.0.1:9999
|   ├── Target address: cloudflare.com:443
|   ├── Duration to wait after success: 5s
|   ├── Read header timeout: 100ms
|   ├── Read timeout: 500ms
|   └── VPN wait durations:
|       ├── Initial duration: 6s
|       └── Additional duration: 5s
├── Shadowsocks server settings:
|   └── Enabled: no
├── HTTP proxy settings:
|   └── Enabled: no
├── Control server settings:
|   ├── Listening address: :8000
|   └── Logging: yes
├── OS Alpine settings:
|   ├── Process UID: 1000
|   └── Process GID: 1000
├── Public IP settings:
|   ├── Fetching: every 12h0m0s
|   ├── IP file path: /tmp/gluetun/ip
|   └── Public IP data API: ipinfo
└── Version settings:
    └── Enabled: yes
2024-06-07T07:37:20Z INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.8 and family v4
2024-06-07T07:37:20Z DEBUG [routing] ip rule add from 172.17.0.8/32 lookup 200 pref 100
2024-06-07T07:37:20Z INFO [routing] adding route for 0.0.0.0/0
2024-06-07T07:37:20Z DEBUG [routing] ip route replace 0.0.0.0/0 via 172.17.0.1 dev eth0 table 200
2024-06-07T07:37:20Z INFO [firewall] setting allowed subnets...
2024-06-07T07:37:20Z INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.8 and family v4
2024-06-07T07:37:20Z DEBUG [routing] ip rule add to 172.17.0.0/16 lookup 254 pref 98
2024-06-07T07:37:20Z INFO [dns] using plaintext DNS at address 8.8.8.8
2024-06-07T07:37:20Z INFO [http server] http server listening on [::]:8000
2024-06-07T07:37:20Z INFO [firewall] allowing VPN connection...
2024-06-07T07:37:20Z DEBUG [firewall] iptables-legacy --append OUTPUT -d 211.104.231.58 -o eth0 -p tcp -m tcp --dport 1489 -j ACCEPT
2024-06-07T07:37:20Z INFO [healthcheck] listening on 127.0.0.1:9999
2024-06-07T07:37:20Z DEBUG [firewall] iptables-legacy --append OUTPUT -o tun0 -j ACCEPT
2024-06-07T07:37:20Z DEBUG [firewall] ip6tables --append OUTPUT -o tun0 -j ACCEPT
2024-06-07T07:37:20Z INFO [openvpn] DEPRECATED OPTION: --cipher set to 'AES-128-CBC' but missing in --data-ciphers (AES-256-GCM:AES-1        28-GCM:CHACHA20-POLY1305). OpenVPN ignores --cipher for cipher negotiations.
2024-06-07T07:37:20Z INFO [openvpn] OpenVPN 2.6.8 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
2024-06-07T07:37:20Z INFO [openvpn] library versions: OpenSSL 3.1.4 24 Oct 2023, LZO 2.10
2024-06-07T07:37:20Z WARN [openvpn] No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mi        tm for more info.
2024-06-07T07:37:20Z INFO [openvpn] TCP/UDP: Preserving recently used remote address: [AF_INET]211.104.231.58:1489
2024-06-07T07:37:20Z INFO [openvpn] Attempting to establish TCP connection with [AF_INET]211.104.231.58:1489
2024-06-07T07:37:20Z INFO [openvpn] TCP connection established with [AF_INET]211.104.231.58:1489
2024-06-07T07:37:20Z INFO [openvpn] TCPv4_CLIENT link local: (not bound)
2024-06-07T07:37:20Z INFO [openvpn] TCPv4_CLIENT link remote: [AF_INET]211.104.231.58:1489
2024-06-07T07:37:21Z INFO [openvpn] [opengw.net] Peer Connection Initiated with [AF_INET]211.104.231.58:1489
2024-06-07T07:37:22Z INFO [openvpn] OPTIONS ERROR: failed to negotiate cipher with server.  Add the server's cipher ('AES-128-CBC') t        o --data-ciphers (currently 'AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305') if you want to connect to this server.
2024-06-07T07:37:22Z ERROR [openvpn] Failed to apply push options
2024-06-07T07:37:22Z INFO [openvpn] Failed to open tun/tap interface
2024-06-07T07:37:22Z INFO [openvpn] SIGUSR1[soft,process-push-msg-failed] received, process restarting
2024-06-07T07:37:26Z INFO [healthcheck] program has been unhealthy for 6s: restarting VPN
2024-06-07T07:37:26Z INFO [healthcheck] 👉 See https://github.com/qdm12/gluetun-wiki/blob/main/faq/healthcheck.md
2024-06-07T07:37:26Z INFO [healthcheck] DO NOT OPEN AN ISSUE UNLESS YOU READ AND TRIED EACH POSSIBLE SOLUTION
2024-06-07T07:37:26Z INFO [vpn] stopping
2024-06-07T07:37:26Z INFO [vpn] starting
2024-06-07T07:37:26Z INFO [firewall] allowing VPN connection...
2024-06-07T07:37:26Z INFO [openvpn] DEPRECATED OPTION: --cipher set to 'AES-128-CBC' but missing in --data-ciphers (AES-256-GCM:AES-1        28-GCM:CHACHA20-POLY1305). OpenVPN ignores --cipher for cipher negotiations.
2024-06-07T07:37:26Z INFO [openvpn] OpenVPN 2.6.8 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
2024-06-07T07:37:26Z INFO [openvpn] library versions: OpenSSL 3.1.4 24 Oct 2023, LZO 2.10
2024-06-07T07:37:26Z WARN [openvpn] No server certificate verification method has been enabled.  See http://openvpn.net/howto.html#mi        tm for more info.
2024-06-07T07:37:26Z INFO [openvpn] TCP/UDP: Preserving recently used remote address: [AF_INET]211.104.231.58:1489
2024-06-07T07:37:26Z INFO [openvpn] Attempting to establish TCP connection with [AF_INET]211.104.231.58:1489
2024-06-07T07:37:26Z INFO [openvpn] TCP connection established with [AF_INET]211.104.231.58:1489
2024-06-07T07:37:26Z INFO [openvpn] TCPv4_CLIENT link local: (not bound)
2024-06-07T07:37:26Z INFO [openvpn] TCPv4_CLIENT link remote: [AF_INET]211.104.231.58:1489
2024-06-07T07:37:27Z INFO [openvpn] [opengw.net] Peer Connection Initiated with [AF_INET]211.104.231.58:1489
2024-06-07T07:37:28Z INFO [openvpn] OPTIONS ERROR: failed to negotiate cipher with server.  Add the server's cipher ('AES-128-CBC') t        o --data-ciphers (currently 'AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305') if you want to connect to this server.
2024-06-07T07:37:28Z ERROR [openvpn] Failed to apply push options
2024-06-07T07:37:28Z INFO [openvpn] Failed to open tun/tap interface
2024-06-07T07:37:28Z INFO [openvpn] SIGUSR1[soft,process-push-msg-failed] received, process restarting

Share your configuration

-e LOG_LEVEL=debug -e VPN_SERVICE_PROVIDER=custom -e VPN_TYPE=openvpn -e OPENVPN_USER=vpn -e OPENVPN_PASSWORD=vpn -v $volume -e OPENVPN_CUSTOM_CONFIG=/gluetun/custom.conf -v '/dev/net/tun:/dev/net/tun' --cap-add=NET_ADMIN -e DOT=off -e DOT_PROVIDERS=google,cloudflare -e DOT_CACHING=off -e BLOCK_MALICIOUS=off qmcgaw/gluetun
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 7, 2024

@qdm12 is more or less the only maintainer of this project and works on it in his free time.
Please:

@elmagow
Copy link

elmagow commented Jun 8, 2024

Same for me on latest Fedora.
Not as high usage, but still on top of my cpu usage with my server doing nothing

@qdm12
Copy link
Owner

qdm12 commented Jun 12, 2024

OpenVPN version: 2.6 is the difference. Try using OPENVPN_VERSION=2.5?

@qdm12 qdm12 changed the title Bug: High CPU when using the latest version Bug: Openvpn 2.6 causes high CPU usage Jun 12, 2024
@engageub
Copy link
Author

Hi,
Thank you for the response. Yes, OPENVPN_VERSION=2.5 reduces the CPU similar to v3.37.0. However, there is a problem with consistency in the latest version.
When I run the command sudo docker stats <gluetun_container_name> for about a minute to get the stats, the CPU utilization suddenly spikes to 100% and comes back to normal, whereas with v3.37.0 this is not the case.

Thank you

@qdm12
Copy link
Owner

qdm12 commented Jun 16, 2024

reduces the CPU similar to v3.37.0.

Do you also have the problem with v3.38.0?

Anyway if Openvpn 2.6 is at fault, there isn't much I can do as far as I know, nothing changed except the openvpn version. Still a strange issue... It might be worth reporting it to the OpenVPN dev team? 🤔

the CPU utilization suddenly spikes to 100% and comes back to normal, whereas with v3.37.0 this is not the case.

I'm not sure I understand this fully, the CPU spikes to 100% for Gluetun only, or for the entire machine, and for how long? Does it happen only when querying docker stats?

@engageub
Copy link
Author

Do you also have the problem with v3.38.0?

v3.38.0 is slightly better than v3.37.0 when compared to memory usage. v3.38.0 was consuming about 56 MB where as v3.37.0 was consuming about 64 MB.
CPU is normal in this version.

Anyway if Openvpn 2.6 is at fault, there isn't much I can do as far as I know, nothing changed except the openvpn version. Still a strange issue... It might be worth reporting it to the OpenVPN dev team? 🤔

If OpenVPN version is the only problem, then it is supposed to be informed to them to resolve the issue.

I'm not sure I understand this fully, the CPU spikes to 100% for Gluetun only, or for the entire machine, and for how long? Does it happen only when querying docker stats?

I started the container with --cpus=1 options in 4 core machine. The CPU is 100% only for gluetun container displayed by docker stats command.
This can also be tested on Play with Docker website directly without using --cpus option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Category: OpenVPN 🔐 Priority: 🚨 Urgent Status: 🗯️ Waiting for feedback
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@elmagow @qdm12 @engageub