Deploying DANE and TLSA for SMTP Without Losing Mail

Why DANE, and why the caution

SMTP between mail servers still runs on opportunistic TLS. A sending server offers STARTTLS, the receiving server accepts, and everyone is encrypted — unless someone in the network path strips the STARTTLS capability from the conversation, in which case both ends happily fall back to plaintext. Nothing alerts, nothing fails. The sender has no independent way to know that your server should have offered TLS with a particular key.

DANE closes that gap by publishing the expected certificate association in DNS, signed with DNSSEC. A validating sender looks up the TLSA record for your MX host, and from that moment TLS with a matching key is mandatory: no record match, no delivery. This is the strength and the risk in one sentence. Publish a wrong record and DANE-validating senders — a group that includes major European providers and a growing share of security-conscious operators — will start bouncing mail that would otherwise have arrived.

Prerequisites you should not negotiate away

First, DNSSEC on the zone that carries your MX hostnames — not the recipient domain, the MX host's zone. An unsigned TLSA record is decorative: the whole design assumes the record itself cannot be spoofed, and only a validated DNSSEC chain provides that. Verify with the DNSSEC Validator that the zone reports secure, not merely that records exist.

Second, stable key management. Decide before publishing anything whether your certificate automation preserves the private key across renewals. With a stable key pair, the common TLSA profile 3 1 1 — DANE-EE, SubjectPublicKeyInfo, SHA-256 — survives every routine renewal untouched. With key rotation on every renewal, each renewal becomes a coordinated DNS change, and you have signed yourself up for the exact operational hazard that breaks most DANE deployments.

Third, monitoring that actually watches the right thing: the digest of the key your MX currently presents versus the digest in DNS. Certificate expiry monitoring does not cover this. The dangerous state is a valid, freshly renewed certificate whose key no longer matches the published record.

Designing the records

TLSA records live at an owner name derived from the service, not the domain: _25._tcp.mail.example.com for SMTP on mail.example.com. Every MX host needs its own record set; the recipient domain itself carries nothing. Build the digests from the actual certificates with the TLSA Record Checker/Generator — it derives the association data from a pasted certificate or from the live certificate the host presents, and diffs the result against what DNS already answers.

For parameter choice, the ecosystem consensus is clear: 3 1 1 as the primary record. DANE-EE pins your end-entity key directly, needs no chain building on the sender side, and combined with the SPKI selector it is renewal-proof as long as the key persists. The 2 1 1 trust-anchor variant has legitimate uses — pinning your CA instead of each host key — but it requires the full chain to be served correctly and adds moving parts. Start with 3 1 1; add complexity only when you have a reason.

The rollout sequence

Publish the TLSA records and then wait — deliberately — before you consider the job done. The records propagate on their TTL; validating senders begin enforcing as soon as they see them. This is the window where a typo shows up as a trickle of bounces from exactly the best-run sender infrastructures, so watch your inbound mail flow and your MX logs for TLS failures during the first days.

Rollovers follow one iron rule: DNS first, service second. Publish the record for the next key alongside the current one, wait out the record TTL, switch the certificate or key on the MX, then remove the old record. TLSA validation passes when any published record matches, which is precisely what makes the overlap safe. Skipping the wait is the classic self-inflicted outage: the new key goes live while caches still serve only the old digest.

Verify the end state from the outside. The DANE Validator judges the deployment the way a validating sender does — owner name, parameter sanity, DNSSEC context, and live certificate match in one pass — and the SMTP TLS Handshake / Mail Domain Delivery Readiness shows the surrounding transport picture including MTA-STS and TLS-RPT.

Failure modes seen in the wild

The renewal that changed the key: automation rotated the key pair, the certificate is valid, browsers are happy, and DANE-validating senders bounce everything. From outside this is indistinguishable from an attack, which is why senders treat it as one. Prevention is key persistence or disciplined rollovers; detection is digest monitoring, not certificate monitoring.

The forgotten secondary MX: the primary is perfect, the backup MX has no TLSA record or a stale one, and mail that fails over during maintenance starts bouncing at the worst possible time. Every host in the MX set needs the same care — check them individually with the MX Record Check first.

The DNSSEC expiry that takes DANE down with it: RRSIG signatures lapse because the re-signing automation died, the zone goes bogus, and with it every TLSA lookup fails. DANE inherits every DNSSEC failure mode, so the signing pipeline deserves the same monitoring attention as the mail stack itself.