Fixes https://github.com/certbot/certbot/issues/10342
When doing ARI checks in acme.renewal_time, we catch RequestException
and return a default value. That's so an unavailable ARI server doesn't
cause issues.
Before we get to acme.renewal_time, we have to create an ACME client,
and in the process fetch a directory. We should make the directory fetch
similarly resilient.
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
my desire to do this came from the discussion at
https://github.com/certbot/certbot/pull/10358#discussion_r2198273164
the code i'm deleting here came from
https://github.com/certbot/certbot/pull/4733
i think doing string parsing on the exception like this is convoluted
and overkill. i also agree with erica from the linked thread above that
we shouldn't be raising a ValueError here, especially when the docstring
for this function says `:raises requests.exceptions.RequestException: in
case of any problems` and doesn't mention ValueError
i prefer we do the simple thing and just delete the code. in the my
opinion unlikely event we decide polishing this important, i think we
can reconsider more complex approaches here
When making an ARI request, use the server listed in the cert's renewal
conf rather than the one passed in certbot's config.
Fixes#10339
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
https://github.com/certbot/certbot/pull/10297 modified
`letsencrypt-auto-source/letsencrypt-auto`. It should never be modified
except during a release, and we have a test to make sure of it. Old
scripts pull directly from that file on github, so let's put that back
asap.
Another token of gratitude for a super useful tool and service.
More about codespell: https://github.com/codespell-project/codespell .
I personally introduced it to dozens if not hundreds of projects already
and so far only positive feedback.
CI workflow has 'permissions' set only to 'read' so also should be safe.
---------
Signed-off-by: Yaroslav O. Halchenko <debian@onerussian.com>
Fixes https://github.com/certbot/certbot/issues/10259
This PR moves post-hook execution from `main.renew` to
`renewal.handle_renewal_request` so that failed and renewed domains
actually get passed into post-hook execution as promised, even when
failures happened.
I suspect the original PR was being overly cautious by putting the whole
thing into a try/finally so that post-hooks definitely happen, but
`handle_renewal_request` is already full of exception catching. I
understand the worry about executing a pre-hook and then failing to
execute its matching post-hook, but the code really is already
structured to make sure that that won't happen. And then when we added
`FAILED_DOMAINS` and `RENEWED_DOMAINS`, we both kept that
overly-cautious hooks execution location, but also kept the error so we
have a summary at the end...which meant that if failures happened, the
env vars were never set.
If we really want to keep the `hooks.run_saved_post_hooks` call on the
outside of everything in main, we can, but then we will have to do one
of the following:
- pass in the output lists to be filled out during execution. not my
favorite pattern
- throw the output lists in the error object or make a wrapper error,
not sure, haven't looked at `errors.py` too closely
- stop raising that final error where we report failures at the very
bottom. it's a little outdated maybe but I do like it and I think people
are used to it
- raise that error in main, returning the number of parse and renewal
failures. this is my favorite of the options, but I still like it less
than what I've implemented here.
Here's the integration/regression test failing on main:
https://dev.azure.com/certbot/certbot/_build/results?buildId=9237&view=logs&j=fca58cec-e7ce-563a-f36f-5c233894d750
You can see here that that branch just has the integration test without
the fix (and removing other tests for efficiency):
https://github.com/certbot/certbot/compare/main...test-fail-env-on-main
It's the default, but just to be clear, this should definitely have two
reviewers.
I added the exact same service hook we use for nightly failures for
release failures.
<img width="1347" alt="Screenshot 2025-06-11 at 10 32 18 AM"
src="https://github.com/user-attachments/assets/b4728d0b-212b-4ecb-84c6-0ed62715f0ff"
/>
Service hooks can be viewed here:
https://dev.azure.com/certbot/certbot/_settings/serviceHooks
Now there's no reason to keep around the manual notification stage, it
wasn't working in either case anyway. Since it's literally the same as
the nightly hook, I don't personally feel the need to test the release
branch but I can if the reviewer would like.
certbot's standalone code contains confusing references to things like
`SSLSocket` which we were hoping to deprecate in
https://github.com/certbot/certbot/issues/10284. are they relevant?
they're sure not!
certbot's standalone plugin only supports HTTP-01 so comments about
things like `ACMETLSServer` and the completely unused `certs` variable
can be deleted
furthermore, the type of the different variables named things like
`http_01_resources` were wrong in multiple places. as can be seen in
certbot's standalone code, the type is
`Set[acme_standalone.HTTP01RequestHandler.HTTP01Resource]`. this is also
[the type used in acme.standalone's
tests](https://github.com/certbot/certbot/blob/723fe64d4dbc778f1e8fd0b93320c1a30dc6fa95/acme/src/acme/_internal/tests/standalone_test.py#L78-L81)
despite the file's type annotations saying it takes a different type. i
think the incorrect type annotations were never caught because mypy
can't fully make sense of our overly complex server classes here
finally, `from __future__ import annotations` was added to make [forward
references in type
annotations](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#forward-references)
easier
This is a feature people didn't have before and won't miss if it fails.
We can always raise it later, but let's reduce it for now to stop people
worrying about the big red warning.
This is one solution to https://github.com/certbot/certbot/issues/10327.
It won't test an ARI check during a dry run, since it will just avoid
the mismatch problem by checking for dry run first and returning before
checking ARI. This PR will make the big error (actually a warning, but
red and scary) go away though.
I thought https://github.com/certbot/certbot/pull/9804/ was abandoned
but the author just missed my comment. I would like to accept that PR to
get it in, but in the process of updating the PR I wrote a nicer
changelog entry, so I would like to add that.
When a CA fails to issue a certificate after finalisation Certbot
currently prints the following unhelpful message:
```
An unexpected error occurred:
acme.errors.IssuanceError
```
This PR makes Certbot print the ACME error object from the order, as
such
```
An unexpected error occurred:
CAA error :: Invalid CAA: CAA prohibits issuance
```
## Pull Request Checklist
- [ ] The Certbot team has recently expressed interest in reviewing a PR
for this. If not, this PR may be closed due our limited resources and
need to prioritize how we spend them.
- [x] If the change being made is to a [distributed
component](https://certbot.eff.org/docs/contributing.html#code-components-and-layout),
edit the `master` section of `certbot/CHANGELOG.md` to include a
description of the change being made.
- [x] Add or update any documentation as needed to support the changes
in this PR.
- [x] Include your name in `AUTHORS.md` if you like.
Fixes#10312. This is perhaps overly detailed, but I was hoping that by
giving a viable path forward it would forestall requests to change it
back, add a flag to ignore ari, or otherwise change the behavior. Very
open to suggestions on wording/content/length/etc.
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Previously, we were constructing an ACME client for ARI checking that
used the global value for `server`, not the one recorded in a lineage's
renewal file.
This resulted in errors in the logs and failure to observe ARI for
lineages that used a non-default `--server` (e.g. staging or non-Let's
Encrypt CAs).
---------
Co-authored-by: ohemorange <ebportnoy@gmail.com>
This depends on a pending Pebble pull request and so will fail
integration tests until/unless that lands:
https://github.com/letsencrypt/pebble/pull/501
However, I'd appreciate some eyes on this PR in this regard: is the
interface we're using in Pebble useful and appropriate? If not, we can
adjust the Pebble PR.
Inspired based on conversation on
https://github.com/certbot/certbot/pull/10307, but note that this just
tests the general case; it does not test the "default server differs
from lineage server" case yet; when I try adding that I get some bugs
that may reflect a problem in #10307 I need to fix (or may reflect that
I need to inhibit the `--server` flag rather than trying to override it
late in the command line).
fixes https://github.com/certbot/certbot/issues/10308
my thinking here was if the spec forbids checking ARI for expired certs,
this check should happen directly in the renewal_time function. if we do
that, what's its most useful response? error? return None? return a
datetime in the past?
i feel the latter is most helpful. tell the caller to renew now rather
than erroring out or giving it no suggestion about when it should renew
it probably doesn't matter much, but i think this would be nice to have
for 4.1.0 as it fixes a (minor) spec compliance issue in our ARI
implementation that is being released
[Recent changes](https://github.com/certbot/certbot/pull/10272/) to
`renewal.should_autorenew` assumed that if
`RenewableCert.configuration.renew_before_expiry` was set, that means
the user set it. That's wasn't true; we were throwing in a default value
if the user didn't set it. But there's no reason for that, especially
since we now set the default renewal time dynamically. Also, we were
writing out a commented `# renew_before_expiry = 30 days` without any
further documentation, in a file that we tell users they [shouldn't
really be
editing](https://eff-certbot.readthedocs.io/en/latest/using.html#modifying-the-renewal-configuration-file).
We now do neither of those things.
this is a follow up from https://github.com/certbot/certbot/pull/10286
and related to https://github.com/certbot/certbot/issues/10302
sorry i initially missed this! in #10286 our tests were just yelling at
me about the different augeas package needed, but python headers and a
compiler are also needed for things to work with an updated version of
python-augeas
i don't believe we need this change in our macOS instructions because:
1. homebrew doesn't split up python packages the way many linux distros
do. there is no equivalent python-dev package
2. if you're using homebrew, you already have a compiler because
[homebrew requires command line tools for
xcode](https://docs.brew.sh/Installation#macos-requirements)
3. "it works on my machine"
Follow-up to #10241. The acme module code is mostly the same, except the
switch to return a tuple containing Retry-After.
This includes the CLI-side work to call out to the new `renewal_time`
method when checking for renewal.
I moved `should_autorenew` from `storage.py` into `renewal.py`, where it
fits better (and also this solves an import cycle problem). To make the
edits more visible I split this into one commit for the move and [one
commit for the subsequent
edits](https://github.com/certbot/certbot/pull/10272/commits/4e137d9b00fb0a299723978d7a289b881716d36b#diff-fad906e31304c767d620bfd243f4c7adf1e63a3420fd634ee57a0f6651c182cf).
This does not yet attempt to store the Retry-After info, or failure
retries, in renewal configs. I figured since that's a pretty big chunk
of work and design on its own, I wanted to get interim feedback as is. I
think this PR would be okay to land with the current default crons /
systemd timers that run twice a day. I think we should implement storage
of retry information before increasing the frequency of runs. And if the
team would like to hold off on landing any ARI until that storage is
done, I'm good with that too. 👍🏻
it looks like https://github.com/certbot/certbot/pull/10098 introduced a
couple bugs into this file:
1.
[RSAPrivateKeys](https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa/#cryptography.hazmat.primitives.asymmetric.rsa.RSAPrivateKey)
don't have a `public_bytes` method
2. `cryptography.x509` wasn't imported and
[load_pem_x509_certificate](https://cryptography.io/en/latest/x509/reference/#cryptography.x509.load_pem_x509_certificate)
takes bytes, not a string
i think avoiding this is unfortunately difficult as this file has no
tests, but it was useful for me just now when testing
https://github.com/certbot/certbot/pull/10283 so i wanted to fix it up
i also changed the script to initially create the account without an
email address as the fake@example.com email causes registration with
LE's staging server to fail early in execution
with the changes in this PR changes, if you:
1. change the value of
[DOMAIN](https://github.com/certbot/certbot/blob/0075104805e2e4fd9cf0e6ec69d80ab203e03655/acme/examples/http01_example.py#L57)
to a domain pointing at your machine
2. as root, activate your certbot dev environment, and run `python
acme/examples/http01_example.py `
it will fail late in the script with:
```
Traceback (most recent call last):
File "/home/brad/certbot/acme/examples/http01_example.py", line 237, in <module>
example_http()
~~~~~~~~~~~~^^
File "/home/brad/certbot/acme/examples/http01_example.py", line 223, in example_http
regr = client_acme.update_registration(
regr.update(
...<3 lines>...
)
)
File "/home/brad/certbot/acme/src/acme/client.py", line 101, in update_registration
updated_regr = self._send_recv_regr(regr, body=body)
File "/home/brad/certbot/acme/src/acme/client.py", line 373, in _send_recv_regr
response = self._post(regr.uri, body)
File "/home/brad/certbot/acme/src/acme/client.py", line 392, in _post
return self.net.post(*args, **kwargs)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/brad/certbot/acme/src/acme/client.py", line 766, in post
return self._post_once(*args, **kwargs)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/brad/certbot/acme/src/acme/client.py", line 781, in _post_once
response = self._check_response(response, content_type=content_type)
File "/home/brad/certbot/acme/src/acme/client.py", line 630, in _check_response
raise messages.Error.from_json(jobj)
acme.messages.Error: urn:ietf:params:acme:error:invalidContact :: The provided contact URI was invalid :: Unable to update account :: invalid contact: contact email has forbidden domain "example.org"
```
if you also change [this email
variable](https://github.com/certbot/certbot/blob/0075104805e2e4fd9cf0e6ec69d80ab203e03655/acme/examples/http01_example.py#L223)
to a valid email address, the script will run successfully
