Feature: Add ability to respond to dns-01 challenges from CA

[rilstrats]

Many CAs require the solving of an ACME challenge even when using EAB credentials. This pull request enables acme-proxy to solve dns-01 challenges between itself and the upstream CA.

Changes

• Added DNS-01 solver support
• Integrated lego DNS providers
• Added environment file to systemd service
• Added basic logging/error handling for DNS challenge setup
• Added documentation on how to use this feature

[netops2devops]

Hi. Can you please elaborate more on what the use case for this feature request is? Can you also please share which CA requires solving challenges while using EAB?

[rilstrats]

Hello. My organization is also an InCommon member. I wanted to say thanks for your contributions to making ACME possible for my organization, other InCommon participants, and anyone who uses this software.

InCommon's current CA, Sectigo, supports DCV inheritance. You can manually validate the TLD (example.edu) and all subdomains are validated as well. This is what enabled certs to be issued without completing an ACME challenge. I should point out that if the manual DCV of the TLD is not completed, acme-proxy would be required to complete an ACME challenge with this CA as well.

InCommon is currently migrating to Certinext, which does not support inherited DCV. This means that every domain has to validated before a cert is issued. This is too much for our organization to do manually, which is why we want ACME to complete the DCV. To do so, it means that every ACME request between acme-proxy and Certinext must complete an ACME challenge.

This change also supports the shortening lifespan of cert lifetimes. By March 2029, certs will have a lifetime of 47 days. Even more drastic, the DCV lifetime will shorten to 10 days. Even if the CA supports DCV inheritance, I don't think anyone wants to complete manual DCV of a single TLD every 10 days.

As for your question of which CAs require solving ACME challenges even when EAB credentials are passed, any CA that hasn't already completed DCV for the requested domain will require solving an ACME challenge. That could be:

• Sectigo without manually completing DCV for TLD
• ZeroSSL (this is what I used to test my changes)
• Certinext

Why DNS-01

I chose to use a DNS-01 challenge between acme-proxy and the upstream CA, because HTTP-01 and TLSALPN-01 have technical limitations in this case. To complete a HTTP-01 or TLSALPN-01 challenge requires that a text file be placed on a web server with the IP resolved from the requested domain. The acme-proxy server and ACME client (webserver on campus) are unlikely to have the same IP, so it can't be completed without doing some sort of crazy DNS setup, reverse proxying, or file transferring. These approaches would add unnecessary complexity to acme-proxy.

Alternatives Considered

I also looked into using https://github.com/RIT-ITS/CertifiCat. This is another server software that acts as an ACME proxy. The difference is that it uses the Rest API to get the certificate from the upstream CA. The software currently has two big downsides.

1. Certinext does not have an API endpoint for completing DCV. This leaves the software unable to function until that endpoint is added by Certinext.
2. EAB credentials are required from each ACME client (webserver on campus).

These two reasons make acme-proxy a much more appealing solution.

[netops2devops]

Thanks for your detailed response @rilstrats.

InCommon is currently migrating to Certinext, which does not support inherited DCV. This means that every domain has to validated before a cert is issued. This is too much for our organization to do manually, which is why we want ACME to complete the DCV. To do so, it means that every ACME request between acme-proxy and Certinext must complete an ACME challenge.

While I was aware of the ongoing CertiNext migration, I genuinely did not know that they require you to complete ACME challenges along with EAB. Thanks a lot for bringing that up. That said, I may need to discuss this with my fellow teammates and think this through a bit more from implementation side. I will get back to you soon because this is a problem we'd need to solve anyway. Have you tested your proposed changes in your environment?

But a while back when I was still developing this project, I wrote a separate plugin (outside of externalcas) which used any DNS provider plugin(s) supported by Lego to solve the TXT challenge and it worked just fine with LetsEncrypt! I did not take EAB into account because LetsEncrypt does not support EAB.

Thanks,
-Kapil

[rilstrats]

You're welcome. Sorry the first post was sparse, I was rushing to submit it before my workday ended.

I know that inherited DCV is a feature that Certinext is planning to add, but the timeframe is unknown. Even then, I still prefer a solution that doesn't require manual DCV. I haven't had the chance to test this in our environment yet, but I am working with @ry-bar to do so. We aren't in Certinext yet and are still waiting on DNS API keys. I did get a working example with a personal domain, cloud flare DNS provider, and ZeroSSL though.

I am curious, would there be a reason we couldn't have this software combine it's functionality with your other plugin/this pull request? EAB credentials could be made optional, which would make this CA agnostic, and could work with any CA that requires just EAB, just dns-01 challenges, or both. The challenge_* fields are optional in this pull request, which could be an example for making the EAB credentials optional. I think that would be the most robust approach, which could add support for Let's Encrypt along with the other CAs this software currently supports. In that case it would even become software I would enjoy using in my home lab.