Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Firewall fails to start on IPv6 rule #1583

Closed
blbecker-tn opened this issue May 21, 2023 · 16 comments
Closed

Bug: Firewall fails to start on IPv6 rule #1583

blbecker-tn opened this issue May 21, 2023 · 16 comments
Labels
Priority: 🚨 Urgent

Comments

@blbecker-tn
Copy link

Is this urgent?

None

Host OS

Debian Buster

CPU arch

x86_64

VPN service provider

Private Internet Access

What are you using to run the container

docker run

What is the version of Gluetun

qmcgaw/gluetun:v3.34.0

What's the problem 🤔

I'm running gluetun in a kubernetes cluster. The container fails to start with the firewall enabled when creating an ipv6 output rule. It's attempting to configure the rule with an IPv4 source address, which fails.

Share your logs

========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version v3.34.0 built on 2023-05-21T13:07:49.386Z (commit 63303bc)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2023-05-21T11:36:58-04:00 INFO [routing] default route found: interface eth0, gateway 10.42.0.1, assigned IP 10.42.0.198 and family v4
2023-05-21T11:36:58-04:00 INFO [routing] local ethernet link found: eth0
2023-05-21T11:36:58-04:00 INFO [routing] local ipnet found: 10.42.0.0/24
2023-05-21T11:36:58-04:00 INFO [routing] local ipnet found: fe80::/64
2023-05-21T11:36:58-04:00 INFO [firewall] enabling...
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --policy INPUT DROP
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --policy OUTPUT DROP
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --policy FORWARD DROP
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --policy INPUT DROP
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --policy OUTPUT DROP
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --policy FORWARD DROP
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --append INPUT -i lo -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --append INPUT -i lo -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --append OUTPUT -o lo -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o lo -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --append OUTPUT -o eth0 -s 10.42.0.198 -d 10.42.0.0/24 -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -s 10.42.0.198 -d fe80::/64 -j ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --flush
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --flush
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --delete-chain
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --delete-chain
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --policy INPUT ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --policy OUTPUT ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] iptables --policy FORWARD ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --policy INPUT ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --policy OUTPUT ACCEPT
2023-05-21T11:36:58-04:00 DEBUG [firewall] ip6tables-nft --policy FORWARD ACCEPT
2023-05-21T11:36:58-04:00 ERROR enabling firewall: command failed: "ip6tables-nft --append OUTPUT -o eth0 -s 10.42.0.198 -d fe80::/64 -j ACCEPT": ip6tables v1.8.8 (nf_tables): host/network `10.42.0.198' not found
Try `ip6tables -h' or 'ip6tables --help' for more information.: exit status 2
2023-05-21T11:36:58-04:00 INFO Shutdown successful

Share your configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gluetun 
  namespace: gluetun
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: gluetun 
  template:
    metadata:
      labels:
        app.kubernetes.io/name: gluetun
    spec:
      containers:
      - name: gluetun
        image: qmcgaw/gluetun:v3.34.0
        securityContext:
          capabilities:
            add:
              - NET_ADMIN
        env:
          - name: VPC_SERVICE_PROVIDER
            value: private internet access
          - name: SERVER_REGIONS
            value: US Chicago
          - name: FIREWALL_DEBUG
            value: on # Remove in prod
          - name: TZ
            value: "America/New_York"
          - name: OPENVPN_USER
            valueFrom:
              secretKeyRef: 
                name: pia-credentials
                key: pia-user
          - name: OPENVPN_PASSWORD
            valueFrom:
              secretKeyRef: 
                name: pia-credentials
                key: pia-pass
        ports:
          - name: httpproxy
            containerPort: 8888
            protocol: TCP
          - name: shadowsocks-tcp
            containerPort: 8388
            protocol: TCP
          - name: shadowsocks-udp
            containerPort: 8388
            protocol: UDP
          
        livenessProbe:
          httpGet:
            path: /
            port: 9999
          initialDelaySeconds: 10
          timeoutSeconds: 10
        readinessProbe:
          httpGet:
            path: /
            port: 9999
          initialDelaySeconds: 10
          timeoutSeconds: 5
@cohenchris
Copy link

cohenchris commented May 21, 2023 •
edited
Loading

I have this exact same issue, except I'm using docker instead of kubernetes:

Is this urgent?
None

Host OS
Arch Linux

CPU arch
x86_64

VPN service provider
Mullvad

What are you using to run the container
docker-compose up -d

What is the version of Gluetun
qmcgaw/gluetun:latest

What's the problem thinking
Same as the original post

Share your logs

Running version latest built on 2023-05-20T20:42:16.268Z (commit 5200ee5)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2023-05-21T08:39:34-07:00 INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.2 and family v4
2023-05-21T08:39:34-07:00 INFO [routing] local ethernet link found: eth0
2023-05-21T08:39:34-07:00 INFO [routing] local ipnet found: 172.17.0.0/16
2023-05-21T08:39:34-07:00 INFO [routing] local ipnet found: fe80::/64
2023-05-21T08:39:34-07:00 INFO [firewall] enabling...
2023-05-21T08:39:34-07:00 ERROR enabling firewall: command failed: "ip6tables-nft --append OUTPUT -o eth0 -s 172.17.0.2 -d fe80::/64 -j ACCEPT": ip6tables v1.8.8 (nf_tables): host/network `172.17.0.2' not found
Try `ip6tables -h' or 'ip6tables --help' for more information.: exit status 2
2023-05-21T08:39:34-07:00 INFO Shutdown successful
========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version latest built on 2023-05-20T20:42:16.268Z (commit 5200ee5)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2023-05-21T08:40:00-07:00 INFO [routing] default route found: interface eth0, gateway 172.17.0.1, assigned IP 172.17.0.2 and family v4
2023-05-21T08:40:00-07:00 INFO [routing] local ethernet link found: eth0
2023-05-21T08:40:00-07:00 INFO [routing] local ipnet found: 172.17.0.0/16
2023-05-21T08:40:00-07:00 INFO [routing] local ipnet found: fe80::/64
2023-05-21T08:40:00-07:00 INFO [firewall] enabling...
2023-05-21T08:40:01-07:00 ERROR enabling firewall: command failed: "ip6tables-nft --append OUTPUT -o eth0 -s 172.17.0.2 -d fe80::/64 -j ACCEPT": ip6tables v1.8.8 (nf_tables): host/network `172.17.0.2' not found
Try `ip6tables -h' or 'ip6tables --help' for more information.: exit status 2
2023-05-21T08:40:01-07:00 INFO Shutdown successful

  vpn:
    image: qmcgaw/gluetun
    container_name: vpn
    cap_add:
      - NET_ADMIN
    network_mode: bridge
    ports:
      - 6789:6789/tcp   # NZBGet
      - 6881:6881       # qBittorrent
      - 6881:6881/udp   # qBittorrent
      - 7878:7878       # Radarr
      - 8080:8080       # qBittorrent
      - 8686:8686       # Lidarr
      - 8787:8787       # Readarr Ebooks
      - 8788:8788       # Readarr Audiobooks
      - 8989:8989       # Sonarr
      - 9696:9696       # Prowlarr
    volumes:
      - ${CONFIG}/vpn:/gluetun
    environment:
      - VPN_SERVICE_PROVIDER=mullvad
      - VPN_TYPE=wireguard
      - WIREGUARD_PRIVATE_KEY=<priv-key>
      - WIREGUARD_ADDRESSES=<addresses>
      - FIREWALL_VPN_INPUT_PORTS=<ports>
      - SERVER_CITIES=<cities>
      - OWNED_ONLY=yes
      - TZ
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0
    restart: always
    labels:
      com.centurylinklabs.watchtower.enable: "true"

@Northcode
Copy link

Northcode commented May 21, 2023 •
edited
Loading

Just tested with this on v3.34.0 and v3.33.0.
It works on v3.33.0, so it seems to be a regression introduced in v3.34.0

apiVersion: v1
kind: Pod
metadata:
  name: gluetun-test-vpn
spec:
  containers:
    - name: shell
      image: debian
      stdin: true
      tty: true
    - name: gluetun
      image: docker.io/qmcgaw/gluetun:v3.34.0 # this doesn't work
      # image: docker.io/qmcgaw/gluetun:v3.33.0 # this works
      env: # real values obscured
        - name: 'VPN_SERVICE_PROVIDER'
          value: 'custom'
        - name: 'VPN_TYPE'
          value: 'wireguard'
        - name: 'VPN_ENDPOINT_IP'
          value: '###.###.###.###'
        - name: 'VPN_ENDPOINT_PORT'
          value: '51820'
        - name: 'WIREGUARD_PRIVATE_KEY'
          value: '####'
        - name: 'WIREGUARD_PUBLIC_KEY'
          value: '####'
        - name: 'WIREGUARD_ADDRESSES'
          value: '###.###.###.###/32,####:####:####:###::#:####/128' # Specifying only ipv4 address also results in the bug
        - name: 'FIREWALL'
          value: 'on' # turning this off on v3.34.0 also works, but then you don't get a firewall
        - name: 'FIREWALL_DEBUG'
          value: 'on'
      securityContext:
        capabilities:
          add: ["NET_ADMIN"]

@blbecker-tn
Copy link
Author

Aha, can confirm that this issue is not present in 3.33.0.

@qdm12
Copy link
Owner

qdm12 commented May 21, 2023

Probably due to 0a29337 I'll investigate

@qdm12
Copy link
Owner

qdm12 commented May 21, 2023 •
edited
Loading

Can someone post debug logs of the v3.33.0 image to compare please? I see the problem (mixing ipv4 and ipv6), but I'm trying to nail down why the heck this is happening now to ideally avoid an ugly obscure if block to resolve this 😄

EDIT: also once this finishes building, can someone try image qmcgaw/gluetun:pr-1585 to see if it resolves it? That's the ugly if block

@blbecker-tn
Copy link
Author

Firewall debug logs on v3.33.0

========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version v3.33.0 built on 2023-04-11T16:03:22.528Z (commit 16ecf48)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2023-05-21T13:45:07-04:00 INFO [routing] default route found: interface eth0, gateway 10.42.0.1 and assigned IP 10.42.0.218
2023-05-21T13:45:07-04:00 INFO [routing] local ethernet link found: eth0
2023-05-21T13:45:07-04:00 INFO [routing] local ipnet found: 10.42.0.0/24
2023-05-21T13:45:07-04:00 INFO [routing] local ipnet found: fe80::/64
2023-05-21T13:45:07-04:00 INFO [firewall] enabling...
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --policy INPUT DROP
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --policy OUTPUT DROP
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --policy FORWARD DROP
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --policy INPUT DROP
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --policy OUTPUT DROP
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --policy FORWARD DROP
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --append INPUT -i lo -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append INPUT -i lo -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --append OUTPUT -o lo -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o lo -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --append OUTPUT -o eth0 -s 10.42.0.218 -d 10.42.0.0/24 -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -s fe80::10e1:deff:fe9e:3224 -d fe80::/64 -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] iptables --append INPUT -i eth0 -d 10.42.0.0/24 -j ACCEPT
2023-05-21T13:45:07-04:00 DEBUG [firewall] ip6tables-nft --append INPUT -i eth0 -d fe80::/64 -j ACCEPT
2023-05-21T13:45:07-04:00 INFO [firewall] enabled successfully
2023-05-21T13:45:08-04:00 INFO [storage] creating /gluetun/servers.json with 13064 hardcoded servers
2023-05-21T13:45:08-04:00 INFO Alpine version: 3.17.3
2023-05-21T13:45:08-04:00 INFO OpenVPN 2.4 version: 2.4.12
2023-05-21T13:45:08-04:00 INFO OpenVPN 2.5 version: 2.5.8
2023-05-21T13:45:08-04:00 INFO Unbound version: 1.17.1
2023-05-21T13:45:08-04:00 INFO IPtables version: v1.8.8
2023-05-21T13:45:08-04:00 INFO Settings summary:
├── VPN settings:
...
[snip]
...
├── Firewall settings:
|   ├── Enabled: yes
|   └── Debug mode: on
├── Log settings:
|   └── Log level: INFO
├── Health settings:
|   ├── Server listening address: 0.0.0.0:9999
|   ├── Target address: cloudflare.com:443
|   ├── Read header timeout: 100ms
|   ├── Read timeout: 500ms
|   └── VPN wait durations:
|       ├── Initial duration: 6s
|       └── Additional duration: 5s
├── Shadowsocks server settings:
|   └── Enabled: no
├── HTTP proxy settings:
|   └── Enabled: no
├── Control server settings:
|   ├── Listening address: :8000
|   └── Logging: yes
├── OS Alpine settings:
|   ├── Process UID: 1000
|   ├── Process GID: 1000
|   └── Timezone: America/New_York
├── Public IP settings:
|   ├── Fetching: every 12h0m0s
|   └── IP file path: /tmp/gluetun/ip
└── Version settings:
    └── Enabled: yes
2023-05-21T13:45:08-04:00 INFO [routing] default route found: interface eth0, gateway 10.42.0.1 and assigned IP 10.42.0.218
2023-05-21T13:45:08-04:00 DEBUG [routing] ip rule add from 10.42.0.218/32 lookup 200 pref 100
2023-05-21T13:45:08-04:00 INFO [routing] adding route for 0.0.0.0/0
2023-05-21T13:45:08-04:00 DEBUG [routing] ip route replace 0.0.0.0/0 via 10.42.0.1 dev eth0 table 200
2023-05-21T13:45:08-04:00 INFO [firewall] setting allowed subnets...
2023-05-21T13:45:08-04:00 INFO [routing] default route found: interface eth0, gateway 10.42.0.1 and assigned IP 10.42.0.218
2023-05-21T13:45:08-04:00 INFO TUN device is not available: open /dev/net/tun: no such file or directory; creating it...
2023-05-21T13:45:08-04:00 INFO [dns over tls] using plaintext DNS at address 1.1.1.1
2023-05-21T13:45:08-04:00 INFO [http server] http server listening on [::]:8000
2023-05-21T13:45:08-04:00 INFO [healthcheck] listening on 0.0.0.0:9999
2023-05-21T13:45:08-04:00 INFO [firewall] allowing VPN connection...
2023-05-21T13:45:08-04:00 DEBUG [firewall] iptables --append OUTPUT -d 181.214.164.10 -o eth0 -p udp -m udp --dport 1197 -j ACCEPT
2023-05-21T13:45:08-04:00 DEBUG [firewall] iptables --append OUTPUT -o tun0 -j ACCEPT
2023-05-21T13:45:08-04:00 DEBUG [firewall] ip6tables-nft --append OUTPUT -o tun0 -j ACCEPT
2023-05-21T13:45:08-04:00 INFO [openvpn] OpenVPN 2.5.8 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Nov  2 2022
2023-05-21T13:45:08-04:00 INFO [openvpn] library versions: OpenSSL 3.0.8 7 Feb 2023, LZO 2.10
2023-05-21T13:45:08-04:00 INFO [openvpn] CRL: loaded 1 CRLs from file -----BEGIN X509 CRL-----
2023-05-21T13:45:08-04:00 INFO [openvpn] MIIDWDCCAUAwDQYJKoZIhvcNAQENBQAwgegxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJDQTETMBEGA1UEBxMKTG9zQW5nZWxlczEgMB4GA1UEChMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxIDAeBgNVBAsTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAwHgYDVQQDExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UEKRMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxLzAtBgkqhkiG9w0BCQEWIHNlY3VyZUBwcml2YXRlaW50ZXJuZXRhY2Nlc3MuY29tFw0xNjA3MDgxOTAwNDZaFw0zNjA3MDMxOTAwNDZaMCYwEQIBARcMMTYwNzA4MTkwMDQ2MBECAQYXDDE2MDcwODE5MDA0NjANBgkqhkiG9w0BAQ0FAAOCAgEAppFfEpGsasjB1QgJcosGpzbf2kfRhM84o2TlqY1ua+Gi5TMdKydA3LJcNTjlI9a0TYAJfeRX5IkpoglSUuHuJgXhP3nEvX10mjXDpcu/YvM8TdE5JV2+EGqZ80kFtBeOq94WcpiVKFTR4fO+VkOK9zwspFfb1cNs9rHvgJ1QMkRUF8PpLN6AkntHY0+6DnigtSaKqldqjKTDTv2OeH3nPoh80SGrt0oCOmYKfWTJGpggMGKvIdvU3vH9+EuILZKKIskt+1dwdfA5Bkz1GLmiQG7+9ZZBQUjBG9Dos4hfX/rwJ3eU8oUIm4WoTz9rb71SOEuUUjP5NPy9HNx2vx+cVvLsTF4ZDZaUztW9o9JmIURDtbeyqxuHN3prlPWB6aj73IIm2dsDQvs3XXwRIxs8NwLbJ6CyEuvEOVCskdM8rdADWx1J0lRNlOJ0Z8ieLLEmYAA834VN1SboB6wJIAPxQU3rcBhXqO9y8aa2oRMg8NxZ5gr+PnKVMqag1x0IxbIgLxtkXQvxXxQHEMSODzvcOfK/nBRBsqTj30P+R87sU8titOoxNeRnBDRNhdEy/QGAqGh62ShPpQUCJdnKRiRTjnil9hMQHevoSuFKeEMO30FQL7BZyo37GFU+q1WPCplVZgCP9hC8Rn5K2+f6KLFo5bhtowSmu+GY1yZtg+RTtsA=
2023-05-21T13:45:08-04:00 INFO [openvpn] -----END X509 CRL-----
2023-05-21T13:45:08-04:00 INFO [openvpn] TCP/UDP: Preserving recently used remote address: [AF_INET]181.214.164.10:1197
2023-05-21T13:45:08-04:00 INFO [openvpn] UDP link local: (not bound)
2023-05-21T13:45:08-04:00 INFO [openvpn] UDP link remote: [AF_INET]181.214.164.10:1197
2023-05-21T13:45:08-04:00 WARN [openvpn] 'link-mtu' is used inconsistently, local='link-mtu 1569', remote='link-mtu 1554'
2023-05-21T13:45:08-04:00 WARN [openvpn] 'keysize' is used inconsistently, local='keysize 256', remote='keysize 128'
2023-05-21T13:45:08-04:00 WARN [openvpn] 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo'
2023-05-21T13:45:08-04:00 INFO [openvpn] [chicago411] Peer Connection Initiated with [AF_INET]181.214.164.10:1197
2023-05-21T13:45:08-04:00 INFO [openvpn] sitnl_send: rtnl: generic error (-101): Network unreachable
2023-05-21T13:45:08-04:00 INFO [openvpn] TUN/TAP device tun0 opened
2023-05-21T13:45:08-04:00 INFO [openvpn] /sbin/ip link set dev tun0 up mtu 1500
2023-05-21T13:45:08-04:00 INFO [openvpn] /sbin/ip link set dev tun0 up
2023-05-21T13:45:08-04:00 INFO [openvpn] /sbin/ip addr add dev tun0 10.6.110.10/24
2023-05-21T13:45:08-04:00 WARN [openvpn] OpenVPN was configured to add an IPv6 route. However, no IPv6 has been configured for tun0, therefore the route installation may fail or may not work as expected.
2023-05-21T13:45:08-04:00 INFO [openvpn] add_route_ipv6(2000::/3 -> :: metric -1) dev tun0
2023-05-21T13:45:08-04:00 INFO [openvpn] UID set to nonrootuser
2023-05-21T13:45:08-04:00 INFO [openvpn] Initialization Sequence Completed
2023-05-21T13:45:08-04:00 INFO [dns over tls] downloading DNS over TLS cryptographic files
2023-05-21T13:45:09-04:00 INFO [dns over tls] downloading hostnames and IP block lists
2023-05-21T13:45:09-04:00 INFO [healthcheck] healthy!
2023-05-21T13:45:15-04:00 INFO [dns over tls] init module 0: validator
2023-05-21T13:45:15-04:00 INFO [dns over tls] init module 1: iterator
2023-05-21T13:45:15-04:00 INFO [dns over tls] start of service (unbound 1.17.1).
2023-05-21T13:45:15-04:00 INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
2023-05-21T13:45:15-04:00 INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
2023-05-21T13:45:16-04:00 INFO [dns over tls] ready
2023-05-21T13:45:16-04:00 INFO [ip getter] Public IP address is 181.214.164.10 (United States, Illinois, Chicago)
2023-05-21T13:45:16-04:00 INFO [vpn] There is a new release v3.34.0 (v3.34.0) created 4 hours ago

@Northcode
Copy link

Logs for 3.34.0:

========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version v3.34.0 built on 2023-05-21T13:07:49.386Z (commit 63303bc)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2023-05-21T18:11:12Z INFO [routing] default route found: interface eth0, gateway 10.42.0.1, assigned IP 10.42.0.27 and family v4
2023-05-21T18:11:12Z INFO [routing] local ethernet link found: eth0
2023-05-21T18:11:12Z INFO [routing] local ipnet found: 10.42.0.0/24
2023-05-21T18:11:12Z INFO [routing] local ipnet found: fe80::/64
2023-05-21T18:11:12Z INFO [firewall] enabling...
2023-05-21T18:11:12Z DEBUG [firewall] iptables --policy INPUT DROP
2023-05-21T18:11:12Z DEBUG [firewall] iptables --policy OUTPUT DROP
2023-05-21T18:11:12Z DEBUG [firewall] iptables --policy FORWARD DROP
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --policy INPUT DROP
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --policy OUTPUT DROP
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --policy FORWARD DROP
2023-05-21T18:11:12Z DEBUG [firewall] iptables --append INPUT -i lo -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --append INPUT -i lo -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --append OUTPUT -o lo -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o lo -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --append OUTPUT -o eth0 -s 10.42.0.27 -d 10.42.0.0/24 -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -s 10.42.0.27 -d fe80::/64 -j ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --flush
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --flush
2023-05-21T18:11:12Z DEBUG [firewall] iptables --delete-chain
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --delete-chain
2023-05-21T18:11:12Z DEBUG [firewall] iptables --policy INPUT ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --policy OUTPUT ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] iptables --policy FORWARD ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --policy INPUT ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --policy OUTPUT ACCEPT
2023-05-21T18:11:12Z DEBUG [firewall] ip6tables-nft --policy FORWARD ACCEPT
2023-05-21T18:11:12Z ERROR enabling firewall: command failed: "ip6tables-nft --append OUTPUT -o eth0 -s 10.42.0.27 -d fe80::/64 -j ACCEPT": ip6tables v1.8.8 (nf_tables): host/network `10.42.0.27' not found
Try `ip6tables -h' or 'ip6tables --help' for more information.: exit status 2
2023-05-21T18:11:12Z INFO Shutdown successful

Logs for 3.33.0 (connection info redacted):

========================================
========================================
=============== gluetun ================
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version v3.33.0 built on 2023-04-11T16:03:22.528Z (commit 16ecf48)

🔧 Need help? https://github.com/qdm12/gluetun/discussions/new
🐛 Bug? https://github.com/qdm12/gluetun/issues/new
✨ New feature? https://github.com/qdm12/gluetun/issues/new
☕ Discussion? https://github.com/qdm12/gluetun/discussions/new
💻 Email? [email protected]
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2023-05-21T18:10:07Z INFO [routing] default route found: interface eth0, gateway 10.42.0.1 and assigned IP 10.42.0.26
2023-05-21T18:10:07Z INFO [routing] local ethernet link found: eth0
2023-05-21T18:10:07Z INFO [routing] local ipnet found: 10.42.0.0/24
2023-05-21T18:10:07Z INFO [routing] local ipnet found: fe80::/64
2023-05-21T18:10:07Z INFO [firewall] enabling...
2023-05-21T18:10:07Z DEBUG [firewall] iptables --policy INPUT DROP
2023-05-21T18:10:07Z DEBUG [firewall] iptables --policy OUTPUT DROP
2023-05-21T18:10:07Z DEBUG [firewall] iptables --policy FORWARD DROP
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --policy INPUT DROP
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --policy OUTPUT DROP
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --policy FORWARD DROP
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append INPUT -i lo -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append INPUT -i lo -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append OUTPUT -o lo -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o lo -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append OUTPUT -o eth0 -s 10.42.0.26 -d 10.42.0.0/24 -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -s fe80::45f:c4ff:fe5a:5deb -d fe80::/64 -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o eth0 -d ff02::1:ff/104 -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append INPUT -i eth0 -d 10.42.0.0/24 -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append INPUT -i eth0 -d fe80::/64 -j ACCEPT
2023-05-21T18:10:07Z INFO [firewall] enabled successfully
2023-05-21T18:10:07Z INFO [storage] creating /gluetun/servers.json with 13064 hardcoded servers
2023-05-21T18:10:07Z INFO Alpine version: 3.17.3
2023-05-21T18:10:07Z INFO OpenVPN 2.4 version: 2.4.12
2023-05-21T18:10:07Z INFO OpenVPN 2.5 version: 2.5.8
2023-05-21T18:10:07Z INFO Unbound version: 1.17.1
2023-05-21T18:10:07Z INFO IPtables version: v1.8.8
2023-05-21T18:10:07Z INFO Settings summary:
├── VPN settings:
|   ├── VPN provider settings:
|   |   ├── Name: custom
|   |   └── Server selection settings:
|   |       ├── VPN type: wireguard
|   |       ├── Target IP address: #VPN_ENDPOINT_IP#
|   |       └── Wireguard selection settings:
|   |           ├── Endpoint IP address: #VPN_ENDPOINT_IP#
|   |           ├── Endpoint port: #VPN_ENDPOINT_PORT#
|   |           └── Server public key: #WIREGUARD_PUBLIC_KEY#
|   └── Wireguard settings:
|       ├── Private key: ####
|       ├── Interface addresses:
|       |   ├── #WIREGUARD_ADDRESSES_IPV4#/32
|       |   └── #WIREGUARD_ADDRESSES_IPV6#/128
|       └── Network interface: tun0
├── DNS settings:
|   ├── DNS server address to use: 127.0.0.1
|   ├── Keep existing nameserver(s): no
|   └── DNS over TLS settings:
|       ├── Enabled: yes
|       ├── Update period: every 24h0m0s
|       ├── Unbound settings:
|       |   ├── Authoritative servers:
|       |   |   └── cloudflare
|       |   ├── Caching: yes
|       |   ├── IPv6: no
|       |   ├── Verbosity level: 1
|       |   ├── Verbosity details level: 0
|       |   ├── Validation log level: 0
|       |   ├── System user: root
|       |   └── Allowed networks:
|       |       ├── 0.0.0.0/0
|       |       └── ::/0
|       └── DNS filtering settings:
|           ├── Block malicious: yes
|           ├── Block ads: no
|           ├── Block surveillance: no
|           └── Blocked IP networks:
|               ├── 127.0.0.1/8
|               ├── 10.0.0.0/8
|               ├── 172.16.0.0/12
|               ├── 192.168.0.0/16
|               ├── 169.254.0.0/16
|               ├── ::1/128
|               ├── fc00::/7
|               ├── fe80::/10
|               ├── ::ffff:7f00:1/104
|               ├── ::ffff:a00:0/104
|               ├── ::ffff:a9fe:0/112
|               ├── ::ffff:ac10:0/108
|               └── ::ffff:c0a8:0/112
├── Firewall settings:
|   ├── Enabled: yes
|   └── Debug mode: on
├── Log settings:
|   └── Log level: INFO
├── Health settings:
|   ├── Server listening address: 127.0.0.1:9999
|   ├── Target address: cloudflare.com:443
|   ├── Read header timeout: 100ms
|   ├── Read timeout: 500ms
|   └── VPN wait durations:
|       ├── Initial duration: 6s
|       └── Additional duration: 5s
├── Shadowsocks server settings:
|   └── Enabled: no
├── HTTP proxy settings:
|   └── Enabled: no
├── Control server settings:
|   ├── Listening address: :8000
|   └── Logging: yes
├── OS Alpine settings:
|   ├── Process UID: 1000
|   └── Process GID: 1000
├── Public IP settings:
|   ├── Fetching: every 12h0m0s
|   └── IP file path: /tmp/gluetun/ip
└── Version settings:
    └── Enabled: yes
2023-05-21T18:10:07Z INFO [routing] default route found: interface eth0, gateway 10.42.0.1 and assigned IP 10.42.0.26
2023-05-21T18:10:07Z DEBUG [routing] ip rule add from 10.42.0.26/32 lookup 200 pref 100
2023-05-21T18:10:07Z INFO [routing] adding route for 0.0.0.0/0
2023-05-21T18:10:07Z DEBUG [routing] ip route replace 0.0.0.0/0 via 10.42.0.1 dev eth0 table 200
2023-05-21T18:10:07Z INFO [firewall] setting allowed subnets...
2023-05-21T18:10:07Z INFO [routing] default route found: interface eth0, gateway 10.42.0.1 and assigned IP 10.42.0.26
2023-05-21T18:10:07Z INFO TUN device is not available: open /dev/net/tun: no such file or directory; creating it...
2023-05-21T18:10:07Z INFO [dns over tls] using plaintext DNS at address 1.1.1.1
2023-05-21T18:10:07Z INFO [http server] http server listening on [::]:8000
2023-05-21T18:10:07Z INFO [firewall] allowing VPN connection...
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append OUTPUT -d #VPN_ENDPOINT_IP# -o eth0 -p udp -m udp --dport #VPN_ENDPOINT_PORT# -j ACCEPT
2023-05-21T18:10:07Z INFO [healthcheck] listening on 127.0.0.1:9999
2023-05-21T18:10:07Z DEBUG [firewall] iptables --append OUTPUT -o tun0 -j ACCEPT
2023-05-21T18:10:07Z DEBUG [firewall] ip6tables-nft --append OUTPUT -o tun0 -j ACCEPT
2023-05-21T18:10:07Z INFO [wireguard] Using available kernelspace implementation
2023-05-21T18:10:07Z INFO [wireguard] Connecting to #VPN_ENDPOINT_IP#:#VPN_ENDPOINT_PORT#
2023-05-21T18:10:07Z INFO [wireguard] Wireguard is up
2023-05-21T18:10:08Z INFO [dns over tls] downloading DNS over TLS cryptographic files
2023-05-21T18:10:09Z INFO [healthcheck] healthy!
2023-05-21T18:10:09Z INFO [dns over tls] downloading hostnames and IP block lists
2023-05-21T18:10:15Z INFO [dns over tls] init module 0: validator
2023-05-21T18:10:15Z INFO [dns over tls] init module 1: iterator
2023-05-21T18:10:15Z INFO [dns over tls] start of service (unbound 1.17.1).
2023-05-21T18:10:15Z INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
2023-05-21T18:10:16Z INFO [dns over tls] ready
2023-05-21T18:10:16Z INFO [ip getter] Public IP address is #######
2023-05-21T18:10:16Z INFO [vpn] There is a new release v3.34.0 (v3.34.0) created 5 hours ago

@qdm12
Copy link
Owner

qdm12 commented May 21, 2023 •
edited
Loading

Can you try image qmcgaw/gluetun:pr-1585? (Pushed locally on the rush, only amd64)
That should hopefully fix it with 558263e essentially I believe the conversion from the 'older' net.IPNet to the newer netip.Prefix form was faulty. Maybe not, but let's find out!

PS:

  • Sorry my network still doesn't have IPv6 and I have no time to configure it lol
  • If this fixes it, I might be a bit annoying to really nail down the minimal changes for a fix (i'll bundle the rest of changes in the master branch/latest image)

@owine
Copy link

owine commented May 21, 2023

Can you try image qmcgaw/gluetun:pr-1585? (Pushed locally on the rush, only amd64) That should hopefully fix it with 558263e essentially I believe the conversion from the 'older' net.IPNet to the newer netip.Prefix form was faulty. Maybe not, but let's find out!

PS:

  • Sorry my network still doesn't have IPv6 and I have no time to configure it lol
  • If this fixes it, I might be a bit annoying to really nail down the minimal changes for a fix (i'll bundle the rest of changes in the master branch/latest image)

This image tag works for me now

@blbecker-tn
Copy link
Author

Same here. The PR image is working for me.

@qdm12
Copy link
Owner

qdm12 commented May 21, 2023

Does it log something like source IP address %s and destination subnet %s are not of the same IP family, skipping... or not? Thanks!

@owine
Copy link

owine commented May 21, 2023

Does it log something like source IP address %s and destination subnet %s are not of the same IP family, skipping... or not? Thanks!

I don't see this, but I do not have debug logging enabled at this time - I can try enabling it in a bit.

@cohenchris
Copy link

The PR image works for me as well. I can give logs, but after a little Googling, still not sure how to change the log level to DEBUG, any pointers? I am using docker-compose.

@AlyoshaVasilieva
Copy link

AlyoshaVasilieva commented May 22, 2023 •
edited
Loading

@cohenchris Add LOG_LEVEL=debug to environment in the docker compose

@cohenchris
Copy link

cohenchris commented May 22, 2023 •
edited
Loading

Does it log something like source IP address %s and destination subnet %s are not of the same IP family, skipping... or not? Thanks!

@qdm12 I do not see this message in my container.

Thanks for this fantastic project! Have been using it daily for a few years.

@qdm12
Copy link
Owner

qdm12 commented May 22, 2023

Thanks everyone for the confirmation! I made the minimal changes to fix v3.34.0 in 6d48f9c and released v3.34.1.

The latest image has that fix as well, but the if check + debug log to prevent mixing IP families in the routing is removed, please feel free to try it 😉

@qdm12 qdm12 closed this as completed May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Priority: 🚨 Urgent
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@AlyoshaVasilieva @Northcode @owine @qdm12 @cohenchris @blbecker-tn