Skip to content

Take over certbot-managed endpoints

What you'll have at the end: an endpoint that certbot used to renew, now renewed by Cyphers end to end — monitored, scored, delivered by its Scout — with certbot retired and the old /etc/letsencrypt/live/... paths still working (as symlinks into the Scout's managed storage, so the web server config never changed).

Prerequisites:

Time: about 30 minutes, spread around one Cyphers renewal.

The example endpoint is shop.example.com on Scout web-01, with certbot serving from /etc/letsencrypt/live/shop.example.com/.

The two failure modes this guide exists to avoid

A window where nothing renews the endpoint — certbot disabled before Cyphers has proven it can renew — and two renewers fighting — certbot still firing after Cyphers owns the files, rotating keys and certificates underneath it. The step order below is what prevents both: Cyphers is brought fully up first, proven with a closed renewal, and only then is certbot switched off. Never invert it.

Step 1 — Track the endpoint

Follow Track a new endpoint Steps 1–3: add shop.example.com:443 to the Scout's targets file and confirm the row on Manage.

You should now see the endpoint on Manage as Untracked, with a served certificate and a score.

Step 2 — Make the certbot directory visible to the Scout

The Scout can only adopt keys from — and later re-point files under — directories it is configured to watch. /etc/letsencrypt/live is the Scout's built-in default watch directory, so on most installs this step is a check, not a change:

$ systemctl cat cyphers-scout | grep -o -- '--cert-dir [^ ]*'

(The bare --cert-dir flag is the one that matters here; every standard install also passes --cert-dir-file, which is additive and does not affect the default.)

  • No output — no explicit --cert-dir flag, so the built-in /etc/letsencrypt/live default applies; nothing to do.
  • One or more --cert-dir flags listed — explicit flags replace the default, so add the certbot directory to the cert-dirs file and reload:
$ echo "/etc/letsencrypt/live" >> /opt/cyphers/cert-dirs.conf
$ sudo systemctl reload cyphers-scout

You should now see (within a scan cycle) the on-disk certificate in the endpoint's drawer — the Scout has discovered the certbot-managed files. Details: Scout → Certificate discovery.

Step 3 — Adopt the endpoint with a profile

On Manage, click Adopt on the shop.example.com row, pick the profile in the Renewal profile select, and click Apply.

You should now see the row managed. Nothing has been renewed yet, and certbot is still active — that is correct at this point. (Details: Adopt an endpoint.)

Step 4 — If keeping the key: adopt it now, while certbot still runs

Skip this step if your profile rotates the key on renewal (the default) — go to Step 5.

If the profile has Reuse private key across renewals enabled (published pins, DANE, clients that pin the key), the served key must be brought under management before the first renewal: follow Adopt a private key. The Scout copies the key certbot has been maintaining into its own managed storage.

You should now see the managed copy with the same modulus digest as the served key (that guide's Step 4).

certbot can invalidate the adopted key

By default certbot generates a new key every renewal. If certbot fires between this adoption and Step 6, the adopted copy is no longer the served key, and the next Cyphers renewal refuses with:

pre-flight refusal: key_reuse_diverged

"The managed private key is no longer the key this endpoint serves — something else has rotated it (another renewer, e.g. certbot, still managing this host)" — with the remedy: stop the other renewer, then re-adopt the key it left in place. The refusal is deliberate: reuse is refused rather than silently rotating what the endpoint serves. To shrink the window, do Steps 4–6 in one sitting, or check systemctl list-timers certbot.timer first and work between firings.

Step 5 — Renew through Cyphers and watch it close

Click Renew Now on the certificate. Wait for the renewal to finish: drawer → Renewals tab, newest entry Closed.

Then verify the file takeover on the host. The deploy transaction re-points the certificate and key paths the endpoint serves from — certbot's live paths included — as symlinks into the Scout's managed directory:

$ ls -l /etc/letsencrypt/live/shop.example.com/fullchain.pem
lrwxrwxrwx ... fullchain.pem -> /opt/cyphers/data/certs/shop.example.com/fullchain.pem
$ ls -l /etc/letsencrypt/live/shop.example.com/privkey.pem
lrwxrwxrwx ... privkey.pem -> /opt/cyphers/data/certs/shop.example.com/privkey.pem

You should now see both symlinks pointing into /opt/cyphers/data/certs/shop.example.com/, and the endpoint serving the new certificate:

$ echo | openssl s_client -connect shop.example.com:443 2>/dev/null \
    | openssl x509 -noout -serial -enddate

The web server's configuration still names the /etc/letsencrypt/live/... paths it always named; from now on those paths follow every Cyphers renewal automatically. The files certbot had in place were quarantined, not deleted (see the rollback note at the end).

Cyphers now demonstrably renews this endpoint. Only now does certbot lose its job.

Step 6 — Disable certbot

On the host:

$ sudo systemctl disable --now certbot.timer

Then check for the other ways certbot gets scheduled — any one of them left active is a second renewer:

$ systemctl list-timers | grep -i certbot        # incl. snap.certbot.renew.timer
$ sudo crontab -l | grep -i certbot
$ grep -ri certbot /etc/cron.d /etc/cron.daily 2>/dev/null

Disable or remove whatever you find (for a snap install: sudo systemctl disable --now snap.certbot.renew.timer).

You should now see no certbot timer in systemctl list-timers and no certbot cron entries. Exactly one thing renews this endpoint, and it is the Hub.

Step 7 — Final check

Back on Manage: the row is managed and healthy, the drawer's Renewals tab shows the closed Cyphers renewal, and the certificate expiry reflects the new certificate. From here, renewals run on Renew Now — or enable autopilot once you're happy with the lane.

Rollback

The takeover destroys nothing. The pre-takeover certificate and key files were quarantined by the deploy transaction rather than deleted, and a failure mid-deploy restores the original pair rather than leaving a mismatched certificate/key on disk. If you need to hand the endpoint back to certbot, re-enable the timer and re-issue with certbot — it will write fresh files over the symlinks' targets or its own layout as usual. Just apply this guide's rule in reverse: never leave both renewers active at once.

Where to next