Previously, we were constructing an ACME client for ARI checking that
used the global value for `server`, not the one recorded in a lineage's
renewal file.
This resulted in errors in the logs and failure to observe ARI for
lineages that used a non-default `--server` (e.g. staging or non-Let's
Encrypt CAs).
---------
Co-authored-by: ohemorange <ebportnoy@gmail.com>
This depends on a pending Pebble pull request and so will fail
integration tests until/unless that lands:
https://github.com/letsencrypt/pebble/pull/501
However, I'd appreciate some eyes on this PR in this regard: is the
interface we're using in Pebble useful and appropriate? If not, we can
adjust the Pebble PR.
Inspired based on conversation on
https://github.com/certbot/certbot/pull/10307, but note that this just
tests the general case; it does not test the "default server differs
from lineage server" case yet; when I try adding that I get some bugs
that may reflect a problem in #10307 I need to fix (or may reflect that
I need to inhibit the `--server` flag rather than trying to override it
late in the command line).
Follow-up to #10241. The acme module code is mostly the same, except the
switch to return a tuple containing Retry-After.
This includes the CLI-side work to call out to the new `renewal_time`
method when checking for renewal.
I moved `should_autorenew` from `storage.py` into `renewal.py`, where it
fits better (and also this solves an import cycle problem). To make the
edits more visible I split this into one commit for the move and [one
commit for the subsequent
edits](https://github.com/certbot/certbot/pull/10272/commits/4e137d9b00fb0a299723978d7a289b881716d36b#diff-fad906e31304c767d620bfd243f4c7adf1e63a3420fd634ee57a0f6651c182cf).
This does not yet attempt to store the Retry-After info, or failure
retries, in renewal configs. I figured since that's a pretty big chunk
of work and design on its own, I wanted to get interim feedback as is. I
think this PR would be okay to land with the current default crons /
systemd timers that run twice a day. I think we should implement storage
of retry information before increasing the frequency of runs. And if the
team would like to hold off on landing any ARI until that storage is
done, I'm good with that too. 👍🏻
Fixes#10252.
See further discussion here: https://github.com/pypa/pip/issues/11457
We are doing option:
> Alternatively, enable the --use-pep517 pip option, possibly with
--no-build-isolation. The --use-pip517 flag will force pip to use the
modern mechanism for editable installs. --no-build-isolation may be
needed if your project has build-time requirements beyond setuptools and
wheel. By passing this flag, you are responsible for making sure your
environment already has the required dependencies to build your package.
Once the legacy mechanism is removed, --use-pep517 will have no effect
and will essentially be enabled by default in this context.
Major changes made here include:
- Add `--use-pep517` to use the modern mechanism, which will be the only
mechanism in future pip releases
- Change to `/src` layout to appease mypy, and because for editable
installs that really is the normal way these days.
- `cd acme && mkdir src && mv acme src/` etc.
- add `where='src'` argument to `find_packages` and add
`package_dir={'': 'src'},` in `setup.py`s
- update `MANIFEST.in` files with new path locations
- Update our many hardcoded filepaths
- Update `importlib-metadata` requirement to fix
double-plugin-entry-point problem in oldest tests
i wanted this for testing
https://github.com/certbot/certbot/issues/10190
alex started working on this in
https://github.com/certbot/certbot/pull/9207 years ago, but pebble
didn't end up doing a release containing his work while he was still
regularly contributing to certbot. this has now changed though
before this PR, our integration tests only worked on amd64 linux
systems. with this PR, i've successfully run our integration tests on
all combinations of the architectures amd64 and arm64 and the OSes linux
and macos
---------
Co-authored-by: ohemorange <erica@eff.org>
Because of the change from using setuptools.pkg_resources to using
importlib, we no longer need a runtime dependency on setuptools. It is
still required, however, for running setup.py.
Pebble 2.5.1 supports OCSP stapling, so we can finally replace all boulder tests/harnesses with the much simpler pebble setup.
Closes#9898
* Remove unused `--acme-server` argument
Since this argument is never set and always defaults to 'pebble', just
remove it to simplify assumptions about which test server's being used.
* Remove boulder option from integration tests
Now that pebble supports all of our test cases, we can move off of
the much more complicated boulder test harness.
* pebble_artifacts: bump to latest pebble release
* pebble_artifacts: fix download path
* certbot-ci: unzip pebble assets
* CI: rip out windows tests/jobs
* tox.ini: rm outdated Windows comment
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* ci: rm redundant integration test
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* acme_server: raise error if proxy and http-01 port are both set
* acme_server: rm vestigial preterimate commands stuff
---------
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Azure recently dropped the `docker-compose` standalone executable (aka
docker-compose v1), and since it's not receiving updates anymore, let's
get with the times and update to v2 as well.
* Drop Python 3.7 support
* Fix lint and test
* Check for venv generation
* Update requirements
* Update oldest constaints and compatibility tests runtime
* Migrate pkg_resources API related to resources to importlib_resources
* Fix lint and mypy + pin lexicon
* Update filterwarnings
* Update oldest tests requirements
* Update pinned dependencies
* Fix for modern versions of python
* Fix assets load in nginx integration tests
* Fix a warning
* Isolate static generation from importlib.resource into a private function
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
* Do not call deprecated datetime.utcnow() and datetime.utcfromtimestamp()
* Ignore DeprecationWarnings from importing dependencies
$ python3 -Wdefault
Python 3.12.0b4 (main, Jul 12 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
/usr/lib/python3.12/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
>>> import pytz
/usr/lib/python3.12/site-packages/pytz/tzinfo.py:27: DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).
_epoch = datetime.utcfromtimestamp(0)
* Used pytz.UTC consistently for clarity
* repin current
* repin oldest
* csr must have version set to zero
* only set PIP_USE_PEP517 for macOS
* experiment with brew update git failure workaround
* add some missing types
* install pkg-config
* install pkg-config for docker too
* add pkg-config to plugins
* pkg-config when cryptography may need to be built
* deps cleanup
* more comments
* more tweaks
Now that we're using pytest more aggressively, I think we should start transitioning our tests to that style rather than continuing to use unittest. This PR removes some unnecessary uses of unittest I found.
I kept the test classes (while removing the inheritance from unittest.TestCase) where I felt like it added structure or logical grouping of tests.
I verified that pytest still finds all the tests in both this branch and master by running commands like:
```
pytest $(git diff --name-only master | grep -v windows_installer_integration_tests)
```
I want to use isort as part of https://github.com/certbot/certbot/issues/9572 because I want to do it programmatically, however, I felt like the config needed to be tweaked a bit due to it not understanding what is and is not our own code.
This PR updates the isort config so it recognizes our own modules and runs `isort .` from the root of the repo to update everything.
* update isort config
* run "isort ."
* Add command to update config files without issuing/renewing cert
* toss up a vague untested skeleton
* remove duplicated code
* set certname in config
* consistent name, no zope
* import copy
* reconsitute is in renewal
* import renewal
* import cli
* fix lint errors
* call choose_configurator_plugins for its side effect of writing to config
* Set certonly in choose config plugins as we do for renew
* rewrite by piggybacking on existing side effects of a dry run instead
* do not allow domains to be set while reconfiguring
* remove unused cert_manager.reconfigure
* remove unused imports
* Add comments and messages
* add cli information
* start adding tests
* remove test code
* get certname before setting up plugins
* get plugin from lineage if not set on cli
* import copy
* always reconstitute
* only load cert once
* add error message
* improve comment
* mock everything out for tests
* test functionality is working!
* add tests for adding and modifying hooks
* test that we don't modify the config if the dry run fails
* improve documentation
* add webroot to reconfigure common options
* lint and clean up intermediate artifacts
* mock validate_hooks for windows
* print success message with updated parameters
* Improve success message
* add message for no changes have been made
* improve changed message to show before as well
* syntax
* Add changes will apply at the next renewal message
* lint
* lint really likes dict.items() for some reason
* run the deploy hook
* turn off dry run to test deploy hook
* patch list_hooks call for tests
* factor out reporting results code
* Remove reporting of which values were changed
* add flag to run deploy hook despite doing a dry run, and recommend setting that to yes when running reconfigure and modifying the deploy hook
* missing () around multi-line string
* test if the two dicts are equal instead of finding the actual changes, thus avoiding having to deal with webroot_map being a list
* refer to --deploy-hook instead of deploy hook
* use renewal configuration instead of configuration information
* mention that the deploy hook will use the active cert not the test one
* disable lint and remove new from language asking about running a deploy hook
* pluralize run deploy hook(s)
* Add test for reporting results when there is a webroot map
* update changelog
* Update error message about modifying domains on the certificate
* update changelog
* Add basic integration tests
* Just set -a rather than redoing the whole testing infrastructure
* used webroot in integration test since it's already installed
* file contents are accessed twice now
---------
Co-authored-by: Alex Zorin <alex@zorin.au>
* certbot-ci: boulder will now only supports port 80 for http-01
* forgot to actually use the http_01_port argument
* print the port the proxy listens on
* try allow binding to privileged ports
Based on my design [here](https://docs.google.com/document/d/1jGh_bZPnrhi96KzuIcyCJfnudl4m3pRPGkiK4fTo8e4/edit?usp=sharing).
Fixes https://github.com/certbot/certbot/issues/4634 and https://github.com/certbot/certbot/issues/4635.
- [x] Deprecate `NamespaceConfig.csr_dir`,`NamespaceConfig.key_dir`, ~~`constants.CSR_DIR` and `constants.KEY_DIR`~~. (`constants` is `_internal` so we can just delete it eventually).
- [x] Update `certbot.crypto_util.generate_csr` and `.generate_key` to make `csr_dir` and `key_dir` optional, respectively.
- [x] Change `certbot._internal.client.Client.obtain_certificate` to no longer include `csr_dir` and `key_dir` to the `.generate_csr` and `.generate_key` calls, respectively.
- Automatically delete unwanted lineage items:
- [x] In `certbot._internal.storage.RenewableCert`, add a function to truncate the lineage history according to the criteria (keep the current and the 5 prior certificates).
- [x] Add a test suite for `truncate`
- [x] In `certbot._internal.renewal.renew_cert`, call the lineage truncation function after the symlinks have been updated for the renewal.
* Stop writing new files to /csr and /keys
* storage: add lineage truncation
* remove unused code
* deprecate keys_dir and csr_dir
* update CHANGELOG
* just keep 5 prior certificates, dont be clever with expiry
* docs: remove reference to /archive and /keys
* filter {csr,key}_dir deprecations directly in tests
