Let's Encrypt and ACME: From TLS Fundamentals to Automated Certificate Operations

An HTTPS deployment must solve two problems: the client must authenticate the server, and both sides must establish encryption keys across an untrusted network. TLS certificates provide the authenticated public keys used during this process, while ACME automates domain validation, certificate issuance, and renewal.

This guide first explains the trust model, then builds practical workflows with Certbot, acme.sh, Cloudflare DNS, Nginx, and Docker. For broader Nginx structure and reverse-proxy configuration, see Nginx Installation and Configuration in Practice.

1. TLS Certificates and ACME Validation

1.1 What a certificate proves

A TLS certificate is an X.509 document signed by a certificate authority (CA). It contains the subject names covered by the certificate, the server’s public key, validity dates, issuer information, and the CA’s signature. A certificate authenticates a public key for one or more names; it does not contain the server’s private key.

During a TLS 1.3 connection, the client and server exchange supported parameters, the server sends its certificate chain and proves possession of the corresponding private key, and both sides derive temporary symmetric session keys. A simplified flow is:

Client                                      Server
  | -------- ClientHello --------------------> |
  | <------- ServerHello, Certificate, -------- |
  |          CertificateVerify, Finished       |
  | -- validate names, chain, signature, time --|
  | -------- Finished ------------------------> |
  | <====== encrypted application data ======> |

The client checks that the requested hostname appears in the certificate, the certificate is within its validity period, the signatures form a chain to a trusted root, and the server proves possession of the private key. ECDHE key exchange provides ephemeral shared secrets, while efficient symmetric encryption protects subsequent HTTP traffic. The CA signature is verified with public-key cryptography; a client does not decrypt the signature to recover a certificate hash.

1.2 What ACME automates

Traditional certificate operations required an administrator to create a private key and certificate signing request, prove control of a domain, download the issued chain, install it, and repeat the process before expiry. The Automatic Certificate Management Environment protocol, standardized in RFC 8555, turns account creation, domain validation, ordering, issuance, and renewal into an API workflow.

An ACME client such as Certbot or acme.sh creates an account key, requests an order, completes a challenge that proves control of each identifier, finalizes a certificate signing request, and downloads the certificate chain. Automation must cover the final deployment step as well: a renewed file on disk is not useful until the server reloads it.

1.3 HTTP-01 and DNS-01

With HTTP-01, the CA gives the client a token. The client publishes the response at http://example.com/.well-known/acme-challenge/<TOKEN>, and the CA retrieves it through public port 80. HTTP-01 is simple for a public web server but cannot validate wildcard identifiers such as *.example.com.

With DNS-01, the client publishes a TXT record under _acme-challenge.example.com. The CA queries authoritative DNS and verifies the response. DNS-01 can issue wildcard certificates and does not require the origin server to expose port 80, but unattended renewal requires narrowly scoped credentials for the DNS provider’s API.

Let’s Encrypt is a nonprofit public CA and a major ACME operator. Other public CAs can also expose ACME endpoints. Cloudflare plays a different role when it proxies a site: browsers receive an edge certificate from Cloudflare, while the Cloudflare-to-origin connection uses a separate origin-side certificate. The Certbot and acme.sh workflows below request publicly trusted certificates through ACME; the Cloudflare token is used only to create DNS-01 records and is unrelated to a Cloudflare Origin CA certificate.

2. Certbot with HTTP-01 and Nginx

2.1 Prepare a webroot

Install Certbot by following the packaging instructions for the operating system, then reserve a directory for challenge files. Nginx must serve the challenge path through HTTP before redirecting other requests to HTTPS:

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;

    location /.well-known/acme-challenge/ {
        alias /var/www/certbot/.well-known/acme-challenge/;
        try_files $uri =404;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

Create the directory and reload a valid configuration:

sudo mkdir -p /var/www/certbot/.well-known/acme-challenge
sudo chown -R www-data:www-data /var/www/certbot
sudo nginx -t
sudo systemctl reload nginx

The service account varies by distribution. Confirm that Nginx can read the webroot and that an ordinary test file under .well-known/acme-challenge/ is reachable from the public internet before requesting a certificate.

2.2 Request and install the certificate

The certonly --webroot mode obtains files without rewriting the rest of the Nginx configuration:

sudo certbot certonly \
  --webroot \
  -w /var/www/certbot \
  -d example.com \
  -d www.example.com \
  --email [email protected] \
  --agree-tos \
  --no-eff-email

-w identifies the webroot, each -d adds a Subject Alternative Name, and --email supplies an operational contact. Successful issuance creates managed links under /etc/letsencrypt/live/example.com/. Nginx should use fullchain.pem for the leaf and intermediate chain, and privkey.pem for the private key. Never commit or copy the private key into a public location.

A current Nginx HTTPS server can use:

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name example.com www.example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;

    location / {
        root /var/www/html;
        index index.html;
    }
}

Run sudo nginx -t before sudo systemctl reload nginx. Cipher selection depends on the Nginx and TLS-library versions; current defaults are usually safer than copying an old static cipher list.

2.3 Test renewal and reload only after deployment

certbot renew reads every renewal profile under /etc/letsencrypt/renewal/ and renews only certificates that are due. Test the entire challenge path with sudo certbot renew --dry-run. A deploy hook should reload Nginx only after a certificate has actually renewed: sudo certbot renew --quiet --deploy-hook "systemctl reload nginx".

Many Certbot packages install a systemd timer automatically. Check systemctl list-timers before adding a duplicate cron job. If the package provides no timer, schedule the renew command at least daily. Frequent checks are harmless because Certbot decides whether issuance is necessary.

3. Wildcard Certificates with Cloudflare DNS-01

3.1 Create a least-privilege API token

DNS-01 is required for *.example.com and is useful when port 80 cannot reach the origin. In Cloudflare, create an API token with Zone / DNS / Edit permission limited to the specific zone. Do not use the Global API Key, because it grants broad account access. IP filtering can narrow the credential further when the ACME client has a stable egress address.

Cloudflare displays a new token once. Store it in a root-readable credential file, never in source control. On Debian or Ubuntu, install the DNS plugin with sudo apt install python3-certbot-dns-cloudflare; use the appropriate supported package on other distributions.

Create the credential file:

sudo install -d -m 700 /root/.secrets/certbot
sudo tee /root/.secrets/certbot/cloudflare.ini >/dev/null <<'EOF'
dns_cloudflare_api_token = YOUR_CLOUDFLARE_API_TOKEN
EOF
sudo chmod 600 /root/.secrets/certbot/cloudflare.ini

Permissions matter because the file contains a plaintext credential that can modify public DNS records.

3.2 Issue and renew through DNS

Request the apex and wildcard identifiers together:

sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /root/.secrets/certbot/cloudflare.ini \
  --dns-cloudflare-propagation-seconds 30 \
  -d example.com \
  -d '*.example.com' \
  --email [email protected] \
  --agree-tos \
  --no-eff-email

The propagation delay gives authoritative DNS enough time to expose the TXT record before validation. Choose the smallest reliable delay for the provider and environment. Certbot stores the authenticator and credential path in the renewal profile, so later certbot renew runs the same DNS workflow automatically. Test renewal before relying on it.

4. acme.sh as a Lightweight Client

4.1 Issue certificates

acme.sh is a shell-based ACME client with built-in DNS-provider hooks. Review the downloaded installer before piping remote code into a shell. A conventional installation is curl https://get.acme.sh | sh -s [email protected]; it installs under ~/.acme.sh/ and normally creates a scheduled renewal job.

For HTTP-01 with an existing webroot, run acme.sh --issue -d example.com -w /var/www/certbot. For Cloudflare DNS-01, export a least-privilege token and account identifier, then issue the apex and wildcard names:

export CF_Token="YOUR_CLOUDFLARE_API_TOKEN"
export CF_Account_ID="YOUR_CLOUDFLARE_ACCOUNT_ID"
acme.sh --issue --dns dns_cf -d example.com -d '*.example.com'

acme.sh stores the DNS credentials in its protected account configuration so scheduled renewals can reuse them. Protect that directory as carefully as the Certbot credential file.

4.2 Deploy to stable Nginx paths

Do not point Nginx directly at acme.sh’s internal working directory. Use --install-cert to copy renewed files to stable service paths and bind the reload action:

sudo mkdir -p /etc/nginx/ssl/example.com
acme.sh --install-cert -d example.com \
  --key-file /etc/nginx/ssl/example.com/privkey.pem \
  --fullchain-file /etc/nginx/ssl/example.com/fullchain.pem \
  --reloadcmd "systemctl reload nginx"

Check the installed schedule with crontab -l | grep acme.sh, and run acme.sh --cron once to verify the account, renewal state, deployment paths, and reload command.

5. Nginx and Certbot in Docker Compose

5.1 Share only the required state

Container deployments need persistent certificate state and a shared HTTP-01 webroot. Keep business files, challenge files, and ACME account state in separate directories:

/opt/ssl-stack/
├── compose.yaml
├── nginx/
│   └── conf.d/
│       └── default.conf
└── data/
    ├── certbot/
    │   ├── conf/
    │   └── www/
    └── webroot/

A compact Compose definition can keep Nginx long-running and invoke Certbot as a scheduled one-shot command from the host or another scheduler:

services:
  nginx:
    image: nginx:1.27-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
      - ./data/certbot/conf:/etc/letsencrypt:ro
      - ./data/certbot/www:/var/www/certbot:ro
      - ./data/webroot:/usr/share/nginx/html:ro

  certbot:
    image: certbot/certbot:latest
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt:rw
      - ./data/certbot/www:/var/www/certbot:rw

Pin image versions according to the deployment’s update policy. A host systemd timer or cron job can run docker compose run --rm certbot renew and reload Nginx after successful renewal. Avoid a container loop that renews files but never reliably signals the Nginx process.

5.2 Bootstrap HTTP before enabling HTTPS

Nginx cannot start with certificate paths that do not exist, while HTTP-01 needs Nginx to serve the challenge. Break the cycle with an initial HTTP-only configuration:

server {
    listen 80;
    server_name example.com;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 200 'Initializing TLS...';
        add_header Content-Type text/plain;
    }
}

Start Nginx and run a one-shot issuance command:

docker compose up -d nginx
docker compose run --rm certbot certonly \
  --webroot --webroot-path=/var/www/certbot \
  -d example.com \
  --email [email protected] \
  --agree-tos --no-eff-email

After issuance, replace the bootstrap server with the final HTTP redirect and HTTPS server:

server {
    listen 80;
    server_name example.com;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    http2 on;
    server_name example.com;
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;

    location / {
        root /usr/share/nginx/html;
        index index.html;
    }
}

Validate and reload inside the container with docker compose exec nginx nginx -t followed by docker compose exec nginx nginx -s reload. The Docker architecture guide explains images, containers, volumes, networks, and Compose in more detail.

6. Choose a Workflow and Verify the Whole Lifecycle

Use Certbot webroot when a public Nginx server can expose port 80 and explicit control of the service configuration matters. Use DNS-01 with a narrowly scoped provider token for wildcard names or private origins. Use acme.sh when a shell-based client and its DNS hooks fit the operating environment. Containerization changes process isolation and file mounting, but it does not remove the need for persistent account state, renewal scheduling, secure credentials, and a post-renewal reload.

Before treating the deployment as complete, verify the public hostname, full certificate chain, expiry date, HTTP-to-HTTPS redirect, scheduled renewal, deploy hook, private-key permissions, DNS-token scope, and recovery procedure. Run the renewal dry-run after every material change. Certificate issuance and service configuration are separate stages; reliable HTTPS requires both stages to remain automated and observable.