Automated Let’s Encrypt DNS Validation Using Certbot and AWS Route53

Let’s Encrypt is a free, automated, and open certificate authority (CA). By February 27, 2020 it has issued one billion certificates.

Let’s Encrypt lets websites to obtain SSL certificates to ascertain the server’s identity and to encrypt the client-server communication, free of charge. It allows the SSL certificates to be issued, renewed and revoked automatically without human inputs or interventions.

Let’s Encrypt uses the ACME protocol to automatically verify the ownership of a domain and issue SSL certificates for that domain. For details on how Let’s Encrypt works, see here. However, in a nutshell, here are the basic steps for acquiring an SSL certificate from Let’s Encrypt:

1. The web server initiates a request for an SSL certificate for a domain name (e.g., www.example.com).
2. Let’s Encrypt asks the web server to prove that it owns the domain name (Domain Validation). This is done by putting a special token, provided by Let’s Encrypt, to a place that only the owner of the domain name has access to. The fact that Let’s Encrypt can access the token from these specified places is the proof that the SSL certificate request originated from the owner of the domain name. Domain Validation can be done in several ways, including:
   HTTP Validation: Let’s Encrypt asks the web server to serve a secret token under a pre-defined URI under the domain name (e.g., http://www.example.com/.well-known/acme-challenge/<TOKEN>);
   DNS Validation: Let’s Encrypt asks the owner of the domain name adds a secret string to the domain’s name server (DNS).
3. Let’s Encrypt verifies that the token is present at the specified location and matches the value it is expecting. This completes the domain validation.
4. Let’s Encrypt issues a new SSL certificate for that domain.

HTTP Validation is the most popular method for getting Let’s Encrypt certificates. It can be conveniently done by installing a ACME client onto the web server and let it handle all tasks related to SSL certificate management: certificate renewal can be set up as an automated job under the system task scheduler; domain validation is as simple as saving the token under the web server’s file system; updating certificate for the web server is just replacing the old certificate files with new ones.

However, there are times that HTTP validation is not possible. For example, you may not have permission to install the ACME client onto the web server, or your web server is an internal server that couldn’t be accessed from outside your network. Under these circumstances, DNS validation will need to be used.

This article will show you how to set up an automated structure for managing Let’s Encrypt SSL certificates using DNS validation with AWS Route53.

Install Certbot

Certbot is a free, open-source ACME client. You need to install it to the machine used for managing the SSL certificates. For HTTP validation it needs to be installed on the web server. However, for DNS validation, it can be installed to any computer that has Internet access.

• To install Certbot on Ubuntu:

sudo apt-get update
sudo apt-get install certbot

• To install Certbot on macOS:

brew install certbot

• To install Certbot on Windows, please follow the detailed instructions listed here. Unfortunately, however, the following instructions only work on Linux and Mac systems…

Prepare Manual DNS Validation Scripts

To automate DNS validation process, a “manual authentication hook” script need to be created. This scripts takes care of adding required DNS entries to the domain name server which are queried later by Let’s Encrypt to verify domain ownership.

This script will be called by Certbot when it needs to conduct DNS validation. Certbot will pass the following values to the script as environment variables.

• CERTBO_DOMAIN: The domain name that is being authenticated
• CERTBOT_VALIDATION: The validation string (token)

The validation string needs to be saved as a TXT record under the _acme-challenge sub-domain. For example, if the CERTBOT_DOMAIN is www.example.com, the validation string must be saved as a TXT record for acme-challenge.www.example.com.

The manual authentication hook script below uses AWS command-line tool to manage records in the AWS Route53 service. It can be adapted for use with other DNS service providers (if they provide command-line tools for updating DNS records).

Make sure you save this script to ./etc/letsencrypt/renewal-hooks/auth/manual-auth-hook.sh.

#!/bin/bash

ROUTE53_ZONE=xxxxxxxxxxxxxx
ACME_HOSTNAME="_acme-challenge.${CERTBOT_DOMAIN}"

echo "Adding TXT record \"${CERTBOT_VALIDATION}\" for ${ACME_HOSTNAME} ..."
aws route53 change-resource-record-sets \
    --hosted-zone-id ${ROUTE53_ZONE} \
    --change-batch "{\"Changes\":[{\"Action\":\"UPSERT\",\"ResourceRecordSet\":{\"Name\":\"${ACME_HOSTNAME}\",\"Type\":\"TXT\",\"TTL\":30,\"ResourceRecords\":[{\"Value\": \"\\\"${CERTBOT_VALIDATION}\\\"\"}]}}]}"
sleep 15

TXT_VALUE="$(dig -t txt "${ACME_HOSTNAME}" | sed -n "s/^${ACME_HOSTNAME}.*\"\(.*\)\"/\1/p")"
while [[ "${TXT_VALUE}" != "${CERTBOT_VALIDATION}" ]]; do
    echo "Current TXT value '${TXT_VALUE}' does not match '${CERTBOT_VALIDATION}'. Wait 5 seconds before retry ..."
    sleep 5
    TXT_VALUE="$(dig -t txt "${ACME_HOSTNAME}" | sed -n "s/^${ACME_HOSTNAME}.*\"\(.*\)\"/\1/p")"
done

Prepare Certbot Commands

Most of the online tutorials about Certbot uses it with sudo. This is not necessary. That said, as by default Certbot writes to system folders such as /etc and /var, using it directly without sudo will give you errors.

To avoid using sudo while circumventing the permission issue, you can specify the folders to use by providing several command-line options, such as --config-dir, --work-dir and --logs-dir. Please see examples below.

• Here is the new-cert.sh script for issuing a new SSL certificate.

#!/bin/bash

certbot certonly \
    --config-dir `pwd`/etc/letsencrypt \
    --work-dir `pwd`/var/lib/letsencrypt \
    --logs-dir `pwd`/var/log \
    --manual --preferred-challenges dns \
    --manual-auth-hook `pwd`/etc/letsencrypt/renewal-hooks/auth/manual-auth-hook.sh

• Here is the list-certs.sh for listing issued certificates.

#!/bin/bash

certbot certificates \
    --config-dir `pwd`/etc/letsencrypt \
    --work-dir `pwd`/var/lib/letsencrypt \
    --logs-dir `pwd`/var/log

• Finally, here is the renew-certs.sh for renewing certificates.

#!/bin/bash

certbot renew \
    --config-dir `pwd`/etc/letsencrypt \
    --work-dir `pwd`/var/lib/letsencrypt \
    --logs-dir `pwd`/var/log \
    --manual-auth-hook `pwd`/etc/letsencrypt/renewal-hooks/auth/manual-auth-hook.sh

I documented above how I set up automated SSL certificate renewal for one of our internal web server (no outside access). Hope this will inspire you on finding your solutions to your specific issues.