Adrien and I added this is in https://github.com/certbot/certbot/pull/6590 in response to https://github.com/certbot/certbot/issues/6582 which I wrote. I now personally think these tests are way more trouble than they're worth.
In almost all cases, the versions pinned in `tools/requirements.txt` are used. The two exceptions to this that come to mind are users using OS packages and pip. In the former, the version of our dependencies is picked by the OS and do not change much on most systems. As for pip, [we only "support it on a best effort basis"](https://eff-certbot.readthedocs.io/en/stable/install.html#alternative-2-pip).
Even for pip users, I'm not convinced this buys us much other than frequent test failures. We have our tests configured to error on all Python warnings and [we regularly update `tools/requirements.txt`](https://github.com/certbot/certbot/commits/master/tools/requirements.txt). Due to that, assuming our dependencies follow normal conventions, we should have a chance to fix things in response to planned API changes long before they make their way to our users. I do not think it is necessary for our tests to break immediately after an API is deprecated.
I think almost all other failures due to these tests are caused by upstream bugs. In my experience, almost all of them will sort themselves out pretty quickly. I think that responding to those that are not or planned API changes we somehow missed can be addressed when `tools/requirements.txt` is updated or when someone opens an issue. I personally don't think blocking releases or causing our nightly tests to fail is at all worth it here. I think removing this frequent cause of test failures makes things just a little bit easier for Certbot devs without costing us much of anything.
* Add async interface for finalization to acme.client.ClientV2
Add `begin_order_finalization()`/`poll_finalization()` to
`acme.client.ClientV2`, which are directly analogous to
`answer_challenge()`/`poll_authorizations()`. This allows us to
finalize an order and then later poll for its completion as separate
steps.
* Address code review feedback
Rename `begin_order_finalization` -> `begin_finalization` and tweak
wording of changelog entry
Right now if you to_json() an `OrderResource` and later deserialize
it, the `AuthorizationResource` objects don't come back through the
round-trip (they just get de-jsonified as frozendicts and worse, they
can't even be passed to `AuthorizationResource.from_json` because
frozendicts aren't dicts). In addition, the `csr_pem` field gets
encoded as an array of integers, which definitely does not get
de-jsonified into what we want.
Fix these by adding an encoder to `authorizations` and encoder and
decoder to `csr_pem`.
`lock_test.py` is a weird, heavily customized, standalone testing relic that's giving me trouble because the name currently conflicts with `certbot/tests/lock_test.py`. Moving `certbot/tests` inside the Certbot package as discussed at https://github.com/certbot/certbot/issues/7909#issuecomment-1448675456 would avoid this, however, this is at least somewhat blocked on getting that test code passing lint and mypy checks again because we run those checks on the entirety of the Certbot package 🙃 Since `lock_test.py` could probably stand to be rewritten/refactored anyway, I took this approach.
What I did is I rewrote something largely equivalent to `lock_test.py` inside Certbot's unit tests. I chose not to do this in `certbot-ci` because its not necessary to have an ACME server available. We're no longer explicitly testing things with the nginx plugin here like we were in `lock_test.py`, however, we are checking that `prepare` is called on the plugin at the right time and I added comments about the importance of checking that we lock the directory during the call to `prepare` in the Apache and nginx test code.
As a bonus, this fixes https://github.com/certbot/certbot/issues/8121.
Now that we're using pytest more aggressively, I think we should start transitioning our tests to that style rather than continuing to use unittest. This PR removes some unnecessary uses of unittest I found.
I kept the test classes (while removing the inheritance from unittest.TestCase) where I felt like it added structure or logical grouping of tests.
I verified that pytest still finds all the tests in both this branch and master by running commands like:
```
pytest $(git diff --name-only master | grep -v windows_installer_integration_tests)
```
* generate multiarch images for non-architecture tags
* lock docker build to legacy docker buider, and bugfix
* rename deploy.sh to deploy_by_arch.sh
* Update documentation related to multiarch Docker
* Consistent IFS value with respect to other scripts
Co-authored-by: humanoid2050 <humanoid2050@monolith>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Add deprecation warning when update_symlinks is run
* Remove information about update_symlinks from help
* ignore our own warning and remove unused imports
* Update changelog
* run unittest2pytest
The command used here was `unittest2pytest -nw acme/tests certbot*/tests`.
* fix with pytest.raises
* add parens to fix refactoring
* <= not <
I want to use isort as part of https://github.com/certbot/certbot/issues/9572 because I want to do it programmatically, however, I felt like the config needed to be tweaked a bit due to it not understanding what is and is not our own code.
This PR updates the isort config so it recognizes our own modules and runs `isort .` from the root of the repo to update everything.
* update isort config
* run "isort ."
* Add command to update config files without issuing/renewing cert
* toss up a vague untested skeleton
* remove duplicated code
* set certname in config
* consistent name, no zope
* import copy
* reconsitute is in renewal
* import renewal
* import cli
* fix lint errors
* call choose_configurator_plugins for its side effect of writing to config
* Set certonly in choose config plugins as we do for renew
* rewrite by piggybacking on existing side effects of a dry run instead
* do not allow domains to be set while reconfiguring
* remove unused cert_manager.reconfigure
* remove unused imports
* Add comments and messages
* add cli information
* start adding tests
* remove test code
* get certname before setting up plugins
* get plugin from lineage if not set on cli
* import copy
* always reconstitute
* only load cert once
* add error message
* improve comment
* mock everything out for tests
* test functionality is working!
* add tests for adding and modifying hooks
* test that we don't modify the config if the dry run fails
* improve documentation
* add webroot to reconfigure common options
* lint and clean up intermediate artifacts
* mock validate_hooks for windows
* print success message with updated parameters
* Improve success message
* add message for no changes have been made
* improve changed message to show before as well
* syntax
* Add changes will apply at the next renewal message
* lint
* lint really likes dict.items() for some reason
* run the deploy hook
* turn off dry run to test deploy hook
* patch list_hooks call for tests
* factor out reporting results code
* Remove reporting of which values were changed
* add flag to run deploy hook despite doing a dry run, and recommend setting that to yes when running reconfigure and modifying the deploy hook
* missing () around multi-line string
* test if the two dicts are equal instead of finding the actual changes, thus avoiding having to deal with webroot_map being a list
* refer to --deploy-hook instead of deploy hook
* use renewal configuration instead of configuration information
* mention that the deploy hook will use the active cert not the test one
* disable lint and remove new from language asking about running a deploy hook
* pluralize run deploy hook(s)
* Add test for reporting results when there is a webroot map
* update changelog
* Update error message about modifying domains on the certificate
* update changelog
* Add basic integration tests
* Just set -a rather than redoing the whole testing infrastructure
* used webroot in integration test since it's already installed
* file contents are accessed twice now
---------
Co-authored-by: Alex Zorin <alex@zorin.au>
In #9127, where @osirisinferi added the `show_account` verb, I made a call not to include the thumbprint in the output of `certbot show_account`.
In hindsight, and after a community member asked for this feature, I think it's better to include it.
It is useful on occasion and `show_account` is fairly specialized anyway. It's only really good for getting your account URL for rate limit increases, checking your contacts, and (now) and doing *magic* with the thumbprint for stateless/distributed HTTP-01 responders.
Without this feature, a clever user might figure out their thumbprint by doing a `certonly --manual --preferred-challenges http` request, but most users would probably be lost.
* show_account: display account thumbprint
* use local key for display
* fix cover tox envs
* make test work on all Pythons
* Remove unused import
Co-authored-by: alexzorin <alex@zorin.id.au>
Co-authored-by: alexzorin <alex@zorin.id.au>
* acme.messages.Error: add mutability
As of Python 3.11, an exception caught within a `with` statement will
update the __traceback__ attribute. Because acme.messages.Error was
immutable, this was causing a knock-on exception, causing certbot to
exit abnormally. This commit hacks in mutability for acme.messages.Error
Fixes#9539
* Add CHANGELOG entry
Based on my design [here](https://docs.google.com/document/d/1jGh_bZPnrhi96KzuIcyCJfnudl4m3pRPGkiK4fTo8e4/edit?usp=sharing).
Fixes https://github.com/certbot/certbot/issues/4634 and https://github.com/certbot/certbot/issues/4635.
- [x] Deprecate `NamespaceConfig.csr_dir`,`NamespaceConfig.key_dir`, ~~`constants.CSR_DIR` and `constants.KEY_DIR`~~. (`constants` is `_internal` so we can just delete it eventually).
- [x] Update `certbot.crypto_util.generate_csr` and `.generate_key` to make `csr_dir` and `key_dir` optional, respectively.
- [x] Change `certbot._internal.client.Client.obtain_certificate` to no longer include `csr_dir` and `key_dir` to the `.generate_csr` and `.generate_key` calls, respectively.
- Automatically delete unwanted lineage items:
- [x] In `certbot._internal.storage.RenewableCert`, add a function to truncate the lineage history according to the criteria (keep the current and the 5 prior certificates).
- [x] Add a test suite for `truncate`
- [x] In `certbot._internal.renewal.renew_cert`, call the lineage truncation function after the symlinks have been updated for the renewal.
* Stop writing new files to /csr and /keys
* storage: add lineage truncation
* remove unused code
* deprecate keys_dir and csr_dir
* update CHANGELOG
* just keep 5 prior certificates, dont be clever with expiry
* docs: remove reference to /archive and /keys
* filter {csr,key}_dir deprecations directly in tests
Brad Warren