Mail Server (Stalwart)

larakube mail:init deploys Stalwart — an all-in-one, Rust-based mail server (SMTP, IMAP, JMAP, POP3, plus CalDAV/CardDAV) — into the larakube-shared namespace. larakube mail:wire then points your other tools (n8n, Vaultwarden…) at it so their invite and notification emails are sent through your own mail server.

How it differs from the other shared tools

• Self-contained storage. Unlike most shared tools, Stalwart does not use Plex Commons. It ships with an embedded store on its own PersistentVolumeClaim, so there is no tenant database to allocate and nothing to back up in Postgres — Stalwart owns its own data.
• It's a StatefulSet, not a Deployment. A mail server has durable identity and storage (mailboxes, config, DKIM keys), so it runs as a StatefulSet (stalwart-0) with a volumeClaimTemplate, rather than as an interchangeable stateless pod.
• Two network paths. The admin/JMAP web UI is fronted by Traefik at mail.<domain> (TLS terminated at the ingress). The mail protocol ports — 25, 465, 587, 993, 4190 — are exposed by a LoadBalancer Service; on single-node k3s the built-in ServiceLB (klipper) binds them straight to the node's IP, so there is no paid cloud load balancer and no clash with Traefik's 80/443.
• Firewall ports are opened automatically (cloud VPS). Klipper binds the mail ports on the node, but two firewalls still default-deny them: the DigitalOcean cloud firewall and the host UFW. On a cloud deploy, mail:init opens all five (25/465/587/993/4190) on both layers — a dedicated larakube-mail-fw-<droplet> cloud firewall (so your Terraform-managed base firewall is never touched) plus UFW over SSH — and --remove closes them again. On a managed cluster (DOKS) the LoadBalancer Service provisions a real cloud load balancer that exposes the ports itself, so no firewall poking is needed.

Webmail is a separate tool

Stalwart's web UI is for administration and account self-service — it is not a webmail client. Your team can read and send mail with a standard IMAP/JMAP client (Apple Mail, Thunderbird, the Gmail app, etc.), or with a browser at larakube webmail:init, which deploys the Bulwark JMAP webmail UI on top of Stalwart.

larakube mail:init

larakube mail:init [environment] [options]

Option	Description
environment	local (default) or a cloud env such as production.
--domain=	Raw domain override → mail.<domain> (skips the host prompt).
--context=	Target a specific kube-context.
--vpn-only	Restrict the admin UI behind NetBird VPN IP whitelisting.
--remove	Tear down the Stalwart stack (StatefulSet, Services, ingress, PVC, secret).

On success it prints the admin console URL, the generated admin password, and the setup runbook below.

larakube mail:check

Rather than eyeball the runbook below, run the health-check any time to see exactly what's done and what's left:

larakube mail:check [environment]

It verifies the pod is running, the admin console is reachable, all five mail ports are reachable from your machine, and the DNS/deliverability records (A, MX, SPF, DMARC) — querying a public resolver (1.1.1.1) so a stale local cache can't give a false result — and prints a one-line fix hint for anything that isn't green. Exit code is non-zero while any check fails, so it also works in CI. (DKIM's selector is per-domain, so it points you at the admin UI rather than guessing.)

Post-install runbook

Stalwart boots into a first-run setup wizard — a few steps in the web UI finish the install. Prefer larakube mail:check over ticking these off by hand. On Stalwart v0.16 the wizard writes its answers into Stalwart's own store (there is no config file to pre-seed), so the install is a one-time manual step per environment.

Run the wizard once, on a clean store

The wizard is reliable on a clean store but loops if it finds half-written state from a previous attempt (An item with the same key already exists). To avoid that:

1. Start clean — larakube mail:init <env> --remove wipes the PVC, then larakube mail:init <env>.
2. Complete the wizard in one pass — do not press Back or refresh mid-way.
3. On the final "Setup complete" screen, restart Stalwart once so it loads the config from its store: larakube mail:restart <env>. After it restarts, /admin shows a normal login instead of the wizard. (The config lives in Stalwart's store, not a file, so a restart is required — skipping it leaves /admin looping back to the wizard.)

note

Why Stalwart Restarts Are Required (larakube mail:restart)

Stalwart reads its configuration store (PostgreSQL/RocksDB) into memory at boot time. When you make changes in the Stalwart Web UI or wizard:

1. You must click Save in the bottom-right corner of the Web UI to commit the changes to PostgreSQL.
2. You must run larakube mail:restart <env> so Stalwart's pod reloads the updated store configuration into memory.

1. Complete the setup wizard & configure Plex Commons stores

Open https://mail.<domain> and follow the setup wizard. You can configure Stalwart to use your Plex Commons infrastructure for database and blob storage:

Data (Main) Store & Search Store (PostgreSQL)

• Host: postgres.larakube-plex.svc.cluster.local
• Port: 5432
• Database: stalwart (auto-created)
• Username: stalwart
• Password: Select Secret read from environment variable and enter STALWART_STORE_PASSWORD.

Blob Store (SeaweedFS S3)

• Store Type: S3-compatible
• Region: Select Custom in the dropdown (this dynamically reveals the URL input box)
• URL: http://seaweedfs.larakube-plex.svc.cluster.local:8333
• Region Name: us-east-1
• Bucket: stalwart
• Access Key: larakube (or select env var STALWART_S3_KEY_ID)
• Secret Key: Select env var STALWART_S3_SECRET_KEY (or literal key from larakube mail:show)

Cache Store (Valkey / Redis)

• Redis URL: redis://redis.larakube-plex.svc.cluster.local:6379/0

On the final "Setup complete" screen, click Save, then apply the changes with:

larakube mail:restart <env>

After restarting, /admin presents the standard login screen. Sign in with admin and the password mail:init printed.

2. Valid TLS on the mail ports (DNS-01)

The ingress covers the web UI, but the mail ports (465/993) are not behind Traefik, so Apple Mail/Thunderbird would hit Stalwart's self-signed cert and warn. Give Stalwart its own certificate via a DNS-01 ACME provider (no port conflict with Traefik):

In Settings → Server → TLS → ACME Providers → Create:

• Challenge = DNS-01
• DNS provider = Cloudflare (configure the DNS provider under Settings → Network → DNS with an API token that can edit the zone)
• Subject names = mail.<domain>

Click Save, then run larakube mail:restart <env> to issue and load the certificates.

3. Add your domain + DKIM

In Directory → Domains, add <domain>. Open its DKIM tab and copy the generated selector record — you'll add it to DNS in step 5.

4. Create accounts

In Directory → Accounts:

• one account per workmate (this issues their email address + login);
• a dedicated noreply@<domain> account with an application password — this is what mail:wire uses.

5. DNS records

Add these for <domain> (the A record for mail.<domain> is created automatically by ExternalDNS):

Type	Name	Value
MX	<domain>	mail.<domain> (priority 10)
TXT	<domain>	v=spf1 mx ~all
TXT	(DKIM selector)	paste from step 3
TXT	_dmarc.<domain>	v=DMARC1; p=quarantine; rua=mailto:postmaster@<domain>
PTR	reverse DNS on the droplet	mail.<domain> (set at your provider)

6. Configure clients

Setting	Value
IMAP	mail.<domain> · port 993 · SSL/TLS
SMTP	mail.<domain> · port 465 · SSL/TLS
Username	the full email address
Password	the account (or application) password

Submission runs on 465, not 587

Stalwart's default setup binds submission on 465 (implicit TLS), so a 587 test shows Connection refused (the port is reachable but nothing listens on it). Use 465 / SSL in your client. In Apple Mail, set the User Name to the full email address — leaving it "Automatic" sends only the local part and fails with "unable to verify account name or password".

Multiple domains (larakube mail:domain)

A single Stalwart instance is multi-domain. You can onboard additional domains with fully automatic DNS, TLS, and DKIM management:

larakube mail:domain production --zone=partner.example --cloudflare-token=<token>

• Cloudflare API Token Scope: The partner provides an API token with Zone · DNS · Edit permissions for their specific domain.
• Stalwart Automation: Registers an x:DnsServer in Stalwart, provisions a Let's Encrypt TLS certificate via DNS-01, configures single RSA-only DKIM, and creates autoconfig / autodiscover / SRV / MTA-STS records directly on Cloudflare.

Inbound MX & Cloudflare Email Routing

• Inbound Delivery: Ensure the domain's root MX record is pointed to your Stalwart server (send.luchtech.dev or mail.<domain>) with priority 10.
• Email Routing Conflict: If the partner previously configured Cloudflare Email Routing / email forwarding, make sure it is disabled or does not override the primary MX records.

Outbound Relay DNS (larakube mail:dns)

When outbound mail is relayed through Amazon SES (or Brevo), relay domain authentication is required to pass DMARC and prevent delivery failures (bounces to Gmail):

larakube mail:dns production --zone=partner.example --provider=ses \
  --ses-tokens=token1,token2,token3 \
  --mail-from=bounce

This automatically:

1. Auto-discovers the partner's Cloudflare token from Stalwart's x:DnsServer.
2. Publishes the 3 AWS SES Easy DKIM CNAME records (<token>._domainkey.partner.example ➔ <token>.dkim.amazonses.com).
3. Updates the root SPF TXT record to include Amazon SES: v=spf1 mx include:amazonses.com ~all.
4. Publishes Custom MAIL FROM DNS records (bounce.partner.example MX + SPF) to guarantee 100% SPF/DKIM alignment under strict DMARC policies (p=reject / p=quarantine).

Existing mail providers conflict on MX

MX records are domain-wide. If a domain already delivers mail elsewhere (e.g. Namecheap PrivateEmail, Google Workspace), you cannot split some addresses to that provider and others to Stalwart via MX. Use a separate domain (or subdomain) for Stalwart to avoid disrupting an existing mailbox.

larakube mail:wire

Wires a tool's outbound email through Stalwart, with no per-tool glue code.

larakube mail:wire [tool] [--all] [--env=production]

• With no tool argument it shows a menu of installed, SMTP-capable tools.
• --all wires every installed SMTP-capable tool at once.
• On first run it prompts for the sender (noreply@<domain>) and its Stalwart application password, then caches them (in a mail-sender secret) so later wires don't re-prompt.

It writes a per-tool secret whose keys are the tool's own SMTP env var names, sets them on the tool's Deployment (kubectl set env --from=secret), and restarts it.

Supported tools

Any tool becomes wireable by declaring a smtpEnv() schema (deployment, namespace, secret, and env-var map). Shipped today:

Tool	Namespace	Env vars
Teable (sheets)	larakube-shared	BACKEND_MAIL_*
n8n (flow)	larakube-shared	N8N_SMTP_* + N8N_EMAIL_MODE=smtp
Vaultwarden (passwords)	larakube-vault	SMTP_*

Everything uses port 465 (implicit TLS)

Both your mail clients and mail:wire send through mail.<domain>:465 (implicit TLS / SSL) — Stalwart's default submission port, and the modern-recommended one (RFC 8314). So it all works out of the box with no extra listener. (Stalwart's default setup does not bind 587/STARTTLS; only add a 587 listener if a specific legacy client requires it.)

Auto-offer on install

When you install an SMTP-capable tool with larakube tool:add and Stalwart is present, it offers to wire the tool right away.

Certificate timing

On a brand-new host, Traefik may attempt Let's Encrypt (HTTP-01) a moment before ExternalDNS has created the DNS record, logging NXDOMAIN looking up A for mail.<domain> and temporarily serving its fallback cert. Because .dev/.app are HSTS-preloaded, the browser won't let you click through. It resolves on Traefik's next retry once DNS has propagated (restarting the Traefik pod forces an immediate retry). Once you configure the DNS-01 provider in step 2, Stalwart renews the mail-port certificate independently.

"Site can't be reached" right after a (re)deploy? It's your DNS cache

Whenever you tear down and redeploy, ExternalDNS re-creates the A record — but your machine may still have the old NXDOMAIN negatively cached, so the browser shows ERR_NAME_NOT_RESOLVED even though the server is fine. This isn't mail-specific — it applies to any ExternalDNS-managed host you've just recreated. Confirm the server is actually up (dig mail.<domain> returns the node IP), then flush:

# macOS
sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
# Chrome also caches separately: chrome://net-internals/#dns → Clear host cache

Or just open it in an Incognito window to test immediately.

Deliverability

Three separate things gate real-world mail; internal mail (teammate ↔ teammate on the same domain) works without any of them.

• Inbound needs MX. External mail only reaches you once <domain>'s MX points at mail.<domain> (step 5).
• Outbound needs a relay on DigitalOcean. DigitalOcean (like most clouds) blocks outbound port 25 by default, so Stalwart cannot deliver directly to recipients' servers — mail to Gmail and other external addresses will queue but never deliver. Configure an outbound relay with larakube mail:relay (Brevo today; it drives Stalwart's API to add the relay route + point outbound at it, and prints the provider's SPF/DKIM steps). A relay also solves cold-IP reputation, so it's the right choice even where 25 is open. --remove reverts to direct MX.
• Reputation. Set PTR/rDNS and publish SPF/DKIM/DMARC (steps 3 & 5).

tip

Send a real test email — larakube mail:test

larakube mail:test [email protected] --env=production authenticates as a Stalwart account and submits a message (from inside the pod, so your own network can't block it). It reports whether AUTH succeeded and whether the message was accepted, and — for an external recipient — warns if no relay is configured (so a "queued" result isn't mistaken for "delivered"). Send to a local @<domain> address to confirm the internal path with no relay; send to your Gmail to confirm the relay once it's set up.

Verify without the Enterprise "History"

The Emails → History (inbound/outbound delivery log) is a Stalwart Enterprise feature and shows a padlock on the community edition — it does not affect mail flow. To verify delivery on community: use Emails → Delivery tests (unlocked) for outbound tracing, Emails → Queued for in-flight messages, and a connected IMAP client's inbox to confirm receipt.