Linux 安装 OCserv：搭建 OpenConnect VPN 服务器

OCserv is the reference server implementation of the OpenConnect protocol, which is itself a reverse-engineered, open replacement for Cisco’s AnyConnect SSL VPN. Because clients are just the stock AnyConnect or OpenConnect app, you get a VPN that works out of the box on iOS, Android, macOS, Windows and Linux, without installing anything unusual on the client side, and without trusting a third-party VPN provider — the server is a Linux box you control.

This guide covers: getting a server, installing ocserv both the quick way (package manager) and the flexible way (from source), the configuration file and certificates, and — the part most write-ups skip — fronting ocserv with Nginx so it shares port 443 with a normal website via SNI-based TCP routing, instead of needing its own exposed port.

1. Get a Remote Server

A remote server here means a box that runs 24/7, almost always Linux, reachable from the client network you actually want to bypass restrictions from.

1.1 Where to Get One

Virtual Private Server:

Vultr

Cloud compute with a free tier:

Oracle Cloud, which offers an Always Free tier with 2 compute instances.

Both require a Visa or Mastercard to sign up.

1.2 Baseline Hardening

Before installing anything internet-facing, do the basic hardening pass first: SSH key auth, a non-root sudo user, Fail2Ban, and a firewall. See Linux 云服务器初始化与安全加固 for the full walkthrough.

2. Install OCserv

2.1 Quick Install (Package Manager)

The fastest path, and enough if you don’t need a specific version or custom build flags.

# RHEL family (Rocky Linux, AlmaLinux, ...)
sudo dnf install epel-release -y
sudo dnf install ocserv -y

# Debian/Ubuntu
sudo apt update && sudo apt install ocserv -y

This gets you a working binary and a systemd unit immediately; skip to 2.3 The Configuration File if this is all you need.

2.2 Building From Source

Build from source when you want the latest release ahead of your distro’s packages, or need compile-time options your package doesn’t ship with (seccomp sandboxing, a specific GnuTLS version, etc.).

Install build dependencies.

# Debian/Ubuntu
sudo apt update
sudo apt install -y build-essential autoconf automake libtool pkg-config gperf \
  libgnutls28-dev libnl-route-3-dev libseccomp-dev libpam0g-dev \
  libprotobuf-c-dev protobuf-c-compiler libreadline-dev liblz4-dev

# RHEL family
sudo dnf install -y gcc make autoconf automake libtool pkgconfig gperf \
  gnutls-devel libnl3-devel libseccomp-devel pam-devel \
  protobuf-c-devel readline-devel lz4-devel

If ./configure later complains about a missing package, install the matching -dev/-devel package for whatever it names and re-run it — the list above covers the common case (GnuTLS for TLS/DTLS, libnl3 for tun device management, libseccomp for the worker sandbox, PAM for authentication, protobuf-c for the internal main/worker IPC, readline for occtl’s interactive shell).

Download and build. Check the release list for the current version instead of hardcoding an old one:

OCSERV_VERSION=1.3.0   # replace with the current release
wget "https://ocserv.gitlab.io/www/download/ocserv-${OCSERV_VERSION}.tar.xz"
tar -xf "ocserv-${OCSERV_VERSION}.tar.xz"
cd "ocserv-${OCSERV_VERSION}"

./configure \
  --prefix=/usr \
  --sysconfdir=/etc \
  --localstatedir=/var \
  --with-systemd-unitdir=/usr/lib/systemd/system

make -j"$(nproc)"
sudo make install

Using --prefix=/usr --sysconfdir=/etc --localstatedir=/var keeps the install laid out like the distro package (config under /etc/ocserv, unit installed via --with-systemd-unitdir), so the rest of this guide applies unchanged regardless of which install path you took.

Create the service account and directories (the package install does this for you; from source, you do it once):

sudo groupadd -r ocserv 2>/dev/null || true
sudo useradd -r -M -s /usr/sbin/nologin -g ocserv ocserv 2>/dev/null || true
sudo mkdir -p /etc/ocserv/ssl /var/lib/ocserv
sudo chown -R ocserv:ocserv /var/lib/ocserv
sudo systemctl daemon-reload

/var/lib/ocserv is the chroot-dir the worker processes are jailed into (see the config below) — it must exist and be writable by the ocserv user before the service starts.

Sanity check:

ocserv --version
occtl --version

2.3 The Configuration File

Save this as /etc/ocserv/ocserv.conf. It’s the same shape whether you installed from a package or from source:

ocserv.conf

# --- Listener ---
tcp-port = 9001
# udp-port defaults to tcp-port when unset. Only set it separately if you're
# fronting the TCP port with Nginx (see section 3) — in that setup this
# becomes udp-port = 443 while tcp-port stays an internal-only port.

# --- Process / sandboxing ---
run-as-user = ocserv
run-as-group = ocserv
socket-file = ocserv.sock
chroot-dir = /var/lib/ocserv
isolate-workers = true
pid-file = /var/run/ocserv.pid

# --- Limits & rate limiting ---
max-clients = 16
max-same-clients = 4
rate-limit-ms = 100
server-stats-reset-time = 604800
max-ban-score = 80
ban-reset-time = 1200

# --- Session / keepalive timers ---
keepalive = 32400
dpd = 90
mobile-dpd = 1800
switch-to-tcp-timeout = 25
try-mtu-discovery = false
auth-timeout = 240
min-reauth-time = 300
cookie-timeout = 300
rekey-time = 172800
rekey-method = ssl
deny-roaming = false

# --- TLS / compatibility ---
tls-priorities = "NORMAL:%SERVER_PRECEDENCE"
compression = true
cisco-client-compat = true
dtls-legacy = true

# --- Authentication ---
# Password-based auth against a htpasswd-style file managed by ocpasswd.
# Switch to auth = "certificate" if you want client-cert auth instead, in
# which case cert-user-oid/cert-group-oid below start to matter.
auth = "plain[passwd=/etc/ocserv/ocpasswd]"
cert-user-oid = 0.9.2342.19200300.100.1.1
cert-group-oid = 2.5.4.11

# --- Server certificate ---
ca-cert = /etc/ocserv/ssl/ca-cert.pem
server-cert = /etc/ocserv/ssl/server-cert.pem
server-key = /etc/ocserv/ssl/server-key.pem

# --- Network ---
device = vpns
predictable-ips = true
default-domain = example.com
ipv4-network = 10.10.11.0/24
dns = 1.1.1.1
dns = 8.8.4.4
ping-leases = false
use-occtl = true
# Routes carved out of the tunnel, e.g. to avoid looping the VPN's own
# subnet or a local network back through itself:
no-route = 10.10.10.0/24
no-route = 192.168.192.0/24

This is the same directive set as the original example, with the duplicate/contradictory cert-user-oid line from earlier revisions removed and a few directives grouped so it’s readable top to bottom instead of one flat list. See the official manual for the full directive reference.

2.4 TLS Certificates

ocserv needs its own CA and server certificate — this is separate from any certificate your web server uses, since ocserv terminates its own TLS/DTLS.

Self-signed (fine for personal use; clients will need to accept/pin it once):

sudo mkdir -p /etc/ocserv/ssl && cd /etc/ocserv/ssl

# CA
sudo certtool --generate-privkey --outfile ca-key.pem
sudo certtool --generate-self-signed --load-privkey ca-key.pem \
  --template <(printf 'cn = "ocserv-ca"\norganization = "homelab"\nserial = 1\nexpiration_days = 3650\nca\nsigning_key\ncert_signing_key\ncrl_signing_key\n') \
  --outfile ca-cert.pem

# Server certificate, signed by the CA above
sudo certtool --generate-privkey --outfile server-key.pem
sudo certtool --generate-certificate --load-privkey server-key.pem \
  --load-ca-certificate ca-cert.pem --load-ca-privkey ca-key.pem \
  --template <(printf 'cn = "vpn.example.com"\norganization = "homelab"\nexpiration_days = 3650\nsigning_key\nencryption_key\ntls_www_server\n') \
  --outfile server-cert.pem

Publicly trusted certificate — recommended if you’re fronting ocserv with Nginx on a real domain (section 3), so clients don’t have to accept a self-signed cert at all. Issue one the normal way (e.g. via Certbot/ACME) for vpn.example.com, then point server-cert/server-key in ocserv.conf at that certificate’s fullchain.pem/privkey.pem instead of the self-signed pair, and drop ca-cert (or point it at the public CA bundle) since clients already trust the issuing CA.

2.5 Creating a VPN User & Starting the Service

sudo ocpasswd -c /etc/ocserv/ocpasswd    # prompts for username + password

sudo systemctl enable --now ocserv
sudo systemctl status ocserv

Verify it’s actually listening and check for errors:

sudo ss -lntup | grep -E ':9001'
sudo journalctl -u ocserv -f
sudo occtl show status

2.6 Firewall & Network Tuning

Enable IP forwarding and turn on BBR congestion control, since a VPN’s throughput is bottlenecked by both:

echo "net.ipv4.ip_forward = 1" | sudo tee /etc/sysctl.d/60-ocserv.conf
echo "net.core.default_qdisc=fq" | sudo tee -a /etc/sysctl.d/60-ocserv.conf
echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.d/60-ocserv.conf
sudo sysctl -p /etc/sysctl.d/60-ocserv.conf

Open the port and NAT traffic leaving the VPN subnet:

sudo firewall-cmd --add-port=9001/tcp --zone=public --permanent
sudo firewall-cmd --add-port=9001/udp --zone=public --permanent
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="10.10.11.0/24" masquerade'
sudo firewall-cmd --reload

(ufw/iptables equivalents if you’re not on a firewalld distro: ufw allow 9001/tcp, ufw allow 9001/udp, plus a MASQUERADE rule for 10.10.11.0/24 on your egress interface.)

2.7 Provider-Level Firewall

Some VPS providers (Vultr included) have a separate firewall layer on top of the OS’s own. Open the same port (9001 in the example above) in the provider’s control panel too, or none of the OS-level rules matter.

At this point you have a working VPN reachable at your_server_ip:9001. The next section is optional but worth doing if the server also runs a normal website: it lets the VPN share port 443 instead of needing its own exposed port.

3. Fronting OCserv With Nginx: TCP-Level SNI Routing

3.1 Why

Running ocserv on its own port (9001 above) works, but it’s an obvious, fingerprintable VPN endpoint, and it’s one more open port to firewall. If the same server already runs Nginx for a website on port 443, you can have Nginx look at the TLS ClientHello’s SNI field — without decrypting anything — and route vpn.example.com connections straight to ocserv while everything else goes to the website, all on the same port 443.

This works because Nginx’s stream module can peek at the SNI hostname before any TLS handshake actually happens (ssl_preread), then forward the raw TCP bytes to whichever backend matches — Nginx never terminates ocserv’s TLS, ocserv still does that itself.

3.2 Requirement: `ssl_preread`

Check your Nginx has the module compiled in:

nginx -V 2>&1 | grep -o with-stream_ssl_preread_module

Most distro packages (Debian/Ubuntu, Fedora, the official nginx.org repo) already include it. If you built Nginx from source yourself, add --with-stream_ssl_preread to ./configure and rebuild.

3.3 Split ocserv’s TCP and UDP Ports

This is the key trick that makes the setup actually work for a VPN and not just a plain HTTPS site: TCP and UDP are independent port spaces at the OS level, so Nginx binding TCP/443 does not conflict with ocserv binding UDP/443 directly. Set, in ocserv.conf:

tcp-port = 9001   # internal only; Nginx forwards matched SNI here
udp-port = 443    # ocserv binds this directly, bypassing Nginx entirely

Binding port 443 requires root or CAP_NET_BIND_SERVICE — the systemd unit installed by both the package and the build in section 2.2 already runs ocserv with the capability it needs, so no extra step there. Restart ocserv after changing the config:

sudo systemctl restart ocserv

3.4 The Nginx `stream {}` Block

stream {} is a top-level context, a sibling of http {}, not nested inside it — add this in /etc/nginx/nginx.conf or an included file under it:

stream {
    map $ssl_preread_server_name $vpn_upstream {
        vpn.example.com   ocserv_backend;
        default           website_backend;
    }

    upstream ocserv_backend {
        server 127.0.0.1:9001;
    }

    upstream website_backend {
        server 127.0.0.1:8443;   # your website's real TLS listener, moved off 443
    }

    server {
        listen 443;
        listen [::]:443;
        proxy_pass $vpn_upstream;
        ssl_preread on;
    }
}

Adding this listener means the website’s own server { listen 443 ssl; ... } blocks in http {} must move to an internal-only port (8443 above) so stream {} is the only thing actually bound to the public 443 TCP socket; the website_backend branch of the map then forwards everything that isn’t vpn.example.com to that internal port unmodified — the website’s own TLS handling doesn’t change at all, only which port it listens on.

Validate and reload:

sudo nginx -t
sudo systemctl reload nginx

3.5 Updating the Firewall

With Nginx fronting the TCP side, ocserv’s TCP port no longer needs to be reachable from outside — only Nginx (on the same host) talks to it. The UDP port, by contrast, still needs to be open directly since Nginx never touches it:

sudo firewall-cmd --remove-port=9001/tcp --zone=public --permanent   # no longer needed publicly
sudo firewall-cmd --add-port=443/udp --zone=public --permanent       # ocserv's direct UDP path
sudo firewall-cmd --reload

443/tcp should already be open for the website; nothing changes there.

3.6 The Catch: This Only Covers TCP

ssl_preread/SNI routing is a TCP-only trick — it works by reading the plaintext ClientHello at the start of a TCP connection. OpenConnect’s data channel prefers DTLS over UDP for throughput, and there’s no equivalent “peek at the hostname” mechanism for UDP in Nginx, so that traffic can’t be SNI-routed the same way. The 3.3 port split works around this rather than solving it: TCP (443) goes through Nginx to ocserv on 9001, UDP (443) goes to ocserv directly and never touches Nginx at all. If the direct UDP path is ever blocked (a stricter upstream firewall, a client network that only allows outbound 443/TCP), ocserv’s switch-to-tcp-timeout in section 2.3 makes clients fall back to TCP-only automatically — slower, but through the same Nginx-fronted path, so connectivity doesn’t break.

4. Client Connection

If you’re using the Nginx setup from section 3, connect to vpn.example.com with no port (defaults to 443). Otherwise, connect to your_server_ip:9001.

4.1 iOS/Android

AnyConnect, available in the App Store even in mainland China.

Server address: vpn.example.com (or your_server_ip:9001 without the Nginx setup).
Username and password: whatever you set with ocpasswd.

4.2 Mac/Linux/Windows

AnyConnect exists for macOS too, though the installer takes a bit more digging to find. Connecting works the same as on iOS.

The alternative, and the simplest on Linux, is OpenConnect — install via brew install openconnect on macOS or your distro’s package manager on Linux, then connect with:

sudo openconnect vpn.example.com

Now you have your own VPN, reachable indistinguishably from a normal HTTPS site if you set up section 3.