* Handle CAA failure on finalize_order during renewal (#9251)
* Fix CAA error on renewal test
* Attempt to fix failing test in CI
* Retry errors with subproblems in obtain_certificate_from_csr with allow_subset_of_names
Only retry if not all domains succeeded
* Back out renewal changes
* Fix linting error line too long
* Update log message for more general case and only log on retry
* Changelog entry
* Add retry logic to order creation
* Changelog entry wording
* Fix acme error handling when no subproblems provided
* Fix test name
* Use summarize domain list to display list of failed domains
* Tidy up incorrect client tests
* Remove unused var and output all failed domains
* Add logging to failed authorization case
* use _retry_obtain_certificate for failed authorizations
* Fix typo failing in CI
* Retry logic comments
* Preserve original error
* Move changelog entry to latest version
* If an installer is provided to certonly, restart after cert issuance
* Add myself to AUTHORS.md
* Handle certonly's "installer" error case
* Handle interactive case, use lazy interpolation
* fix trailing whitespace
* fix whitespace in error message, re-raise exception
* Handle cases where user specified an authenticator but no installer
* make tox happy
* Clarify comment in selection.py
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* Add tests for the certonly installer changes
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* Add message to account reg. error
* Changelog
* Remove forced lowercase first char
* Catch errors raised by acme library
* Fix mypy and add some comments
* Add some tests
* Move changelog entry to current version
* Address comments
* Address additional comments
Put everything in this commit instead of using the "Commit suggestion"
feat on Github, which would resolve in 4 different tiny commits.
* improve handling and ux of unexpected key type migration
* update unit tests
* update integration tests
* if --cert-name and --key-type are set, dont prompt
* Add challenge info to `--debug-challenges`
* Expand/add tests
* Add changelog entry
* Make tests Python 3.6 and 3.7 compatible
* Don't use `config.namespace`
* And don't use `config.namespace` in tests too
* Expand tests to check for token/thumbprint
* Add test for the DNS-01 challenge
Changed the Apache authenticator to the manual authenticator. Doesn't
seem to make a difference to the tests, but makes more sense if the
DNS-01 challenge is being used.
* Reword changelog entry
* Mention feature in --help output
* Better variable assignment in test
Co-authored-by: alexzorin <alex@zor.io>
* Better variable assignment in test
Co-authored-by: alexzorin <alex@zor.io>
* Remove unnecessary `verbose_count` assignment
Co-authored-by: alexzorin <alex@zor.io>
* Use terminology from RFC 8555
* Compress the two new tests into one
* s/world wide web/internet
* Move new code into separate function
* Remove superfluous newline with mixed challs
Co-authored-by: alexzorin <alex@zor.io>
* Add support for revoking ecdsa keys without --cert-name.
Co-Authored-By: commonism <commonism@users.noreply.github.com>
* Move alg to acme_client.ClientNetwork instantiating in acme_from_config_key
* Fix argument for RS256/ES256
* Support also ES384 and ES512 signing algorithms.
* Fetch and print account contacts from ACME server
* Add tests
* Add changelog entryAdd changelog entry
* Add account URI and thumbprint output
Only show these items when verbosity > 0
* Add test case for account URI and thumbprint
* Move changelog entry to new placeholder
* Add test for `cb_client.acme` (coverage)
* Address comments
* Update changelog
* Few small word changes
* Add server to error messages
* Remove phone contact parts
* Added --issuance-timeout command line option
* clarification of command line option name,docstring and add tests
* fix test case for python36
* improved the command line options
* Certificate issuing on Window while having web.confing and more then one domain in request
* add a test
* update changelog
Co-authored-by: Serghei Trufkin <Serghei.Trufkin@Technosoft.md>
Fixes https://github.com/certbot/certbot/issues/9058.
The changes to the CI config are equivalent to the ones made in https://github.com/certbot/certbot/pull/8460.
Other than ignoring some warnings raised by botocore, the main additional work that had to be done here was switching away from using `distutils.version.LooseVersion` since the entire `distutils` module was deprecated in Python 3.10. To do that, I took a few different approaches:
* If the version strings being parsed are from Python packages such as Certbot or setuptools, I switched to using [pkg_resources.parse_version](https://setuptools.pypa.io/en/latest/pkg_resources.html#parsing-utilities) from `setuptools`. This functionality has been available since [setuptools 8.0 from 2014](https://setuptools.pypa.io/en/latest/history.html#id865).
* If the version strings being parsed are not from Python packages, I added code equivalent to `distutils.version.LooseVersion` in `certbot.util.parse_loose_version`.
* The code for `CERTBOT_PIP_NO_BINARY` can be completely removed since that variable isn't used or referenced anywhere in this repo.
* add python 3.10 support
* make some version changes
* don't use looseversion in setup.py
* switch to pkg_resources
* deprecate get_strict_version
* fix route53 tests
* remove unused CERTBOT_PIP_NO_BINARY code
* stop using distutils in letstest
* add unit tests
* more changelog entries
* Generate a web.config file to serve properly challenge files with IIS
* Fix cleanup, add test
* FIx lint
* Do not overwrite existing web.config. Delete only web.config when it has been created by Certbot and is unmodified.
* Fix lint
* Update certbot/certbot/_internal/plugins/webroot.py
Co-authored-by: alexzorin <alex@zor.io>
* Add log
* Check for POSIX_MODE before web.config deletion attempt.
* Add documentation
* Update certbot/CHANGELOG.md
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/docs/using.rst
Co-authored-by: alexzorin <alex@zor.io>
@osirisinferi noticed [in chat](https://opensource.eff.org/eff-open-source/pl/sa85u4n71tywfpc15c1wu59wae) that "NEXT STEPS:" was ignoring `--quiet` and was being printed unconditionally.
I think it ended up being written this way in #8860 because I was trying not to avoid dumping ANSI escapes and newlines into the log file and confused myself in the process.
This change makes things a bit more explicit in separating presentation/message.
* fix 'NEXT STEPS' being printed to stdout during -q
* fix tests
* Fix some typos (found by codespell)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
* Remove typo fixes for some files which should not be modified
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Fixes https://github.com/certbot/certbot/issues/6844.
This PR does two things:
1. Changes ACMEv1 deprecation warnings from `PendingDeprecationWarning` to `DeprecationWarning`.
2. Changes the ACMEv1 deprecation warnings to be on references to the class themselves. This is the approach taken in https://github.com/certbot/certbot/pull/8989, the PRs linked there, and the `cryptography` code in the code comment. I think this approach warns in more cases and I updated our unit tests to avoid hitting these warnings.
* add ip address support to acme saving
* remove client-site check for ip address
* using right prefix for san parsing
* remove type hint for backward compatibility
* remove bare ip blocking check from main_test
* upppercase
* lint tix
* add additional tests for new IP support
* support for ipv6 bare address
* make apache and nginx plugin raise error for certs with ip address
* linting
* add pem file's last newline char
* gen_ss_cert ip support and comment fixup
* fix test coverage
* indent fix and assetTrue to assetIN
* indent mistake, made a note where class end
* acme lib now receive IPs as separate list
* fix typos
* type 2
* fix tests
* Deny IP address on certbot/certbot side as LE don't support it
* remove excess empty line to rerun tox
* comment indent and typo fix
Apply suggestions from code review
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* trim unused functions
* trim unused import
* make raw san list extraction as separate function
* Apply suggestions from code review
mostly comment suggestions here
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* apply patches suggested on review.
* remove excessive empty lines
* update CHANGELOG.md
* added acme lib update about ipaddress support in CHANGELOG.md
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
As a follow-up to #8971, this PR removes all references to the old Zope interfaces, except the ones used to deprecate them and prepare for their removal.
In the process, some documentation and tests about the `Display` objects are simply removed since they are not relevant anymore given that they are removed from the public API.
* Cleanup some interfaces.IInstaller
* Cleanup IConfig doc
* Allmost complete removal
* Remove useless tests
* Fixes
* More cleanup
* More cleanup
* More cleanup
* Remove a non existent reference
* Better type
* Fix lint
`distro.linux_distribution` was deprecated (https://github.com/python-distro/distro/pull/296) in the release of `distro` at the end of last week. The deprecation is causing the `nopin` nightly tests to fail.
This change migrates Certbot off that function.
As far as I can tell, the Arch Linux edge case described in the code comments no longer happens, but better to be safe than sorry I think.
* stop using deprecated distro.linux_distribution
* update comment
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* cli: vary renewal advice for hookless manual certs
1. Don't print that the certificate will be automatically renewed,
because it won't be.
2. Add a "NEXT STEP" telling the user that they will need to manually
re-issue the certificate in order to renew it.
* kill superfluous comma
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* clarify wording of the next step
* fix the test
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Also, update `dev-cli.ini` example to use new flag.
Although https://github.com/bw2/ConfigArgParse/pull/216 allowed setting a `count` action value in a config file, our default detection system won't let us use that functionality. While we should eventually fix that, for now, let developers have a cli.ini with a higher logging level by adding this flag.
Note that this flag is intended to work the same way adding `-vvv`s does; that is, as a modifier to the pre-set level, rather than setting the absolute level. The number it is set to is equivalent to the number of `v`s that would otherwise have been passed, with "2" as the current maximum effective number of levels (warning --> info --> debug).
* Add --verbose-level flag for devs to set in cli.ini
* Update dev-cli.ini to use new flag
* Deprecate usage of IConfig as a singleton in Certbot
* Fix local oldest requirements
* Add changelog
* Add tests for certbot.crypto_util.init_save_* functions
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* later printing of renewal and install retry advice
Move printing of advice for automated renewal, and retrying installation
in case of failure, towards the end of `run` and `certonly`.
Also adds some renewal advice for the --csr case (no autorenewal).
* update renewal advice for preconfigured-renewal
* rewrite in terms of "NEXT STEPS" for run/certonly
* fix lint
* re-add "Could not install certificate"
* update --csr renewal advice
* rewrite non-preconfigured-renewal renewal advice
* Remove no names found in configuration files because it sounds like an error but actually it is fine
* fix test
* Pose question more grammatically and specifically, and remove extra space
* fix lint
Co-authored-by: Alex Zorin <alex@zorin.id.au>
Streamline and reorganize Certbot's CLI output.
This change is a substantial command-line UX overhaul,
based on previous user research. The main goal was to streamline
and clarify output. To see more verbose output, use the -v or -vv flags.
---
* nginx,apache: CLI logging changes
- Add "Successfully deployed ..." message using display_util
- Remove IReporter usage and replace with display_util
- Standardize "... could not find a VirtualHost ..." error
This changes also bumps the version of certbot required by certbot-nginx
and certbot-apache to take use of the new display_util function.
* fix certbot_compatibility_test
since the http plugins now require IDisplay, we need to inject it
* fix dependency version on certbot
* use better asserts
* try fix oldest deps
because certbot 1.10.0 depends on acme>=1.8.0, we need to use
acme==1.8.0 in the -oldest tests
* cli: redesign output of new certificate reporting
Changes the output of run, certonly and certonly --csr. No longer uses
IReporter.
* cli: redesign output of failed authz reporting
* fix problem sorting to be stable between py2 & 3
* add some catch-all error text
* cli: dont use IReporter for EFF donation prompt
* add per-authenticator hints
* pass achalls to auth_hint, write some tests
* exclude static auth hints from coverage
* dont call auth_hint unless derived from .Plugin
* dns fallback hint: dont assume --dns-blah works
--dns-blah won't work for third-party plugins, they need to be specified
using --authenticator dns-blah.
* add code comments about the auth_hint interface
* renew: don't restart the installer for dry-runs
Prevents Certbot from superfluously invoking the installer restart
during dry-run renewals. (This does not affect authenticator restarts).
Additionally removes some CLI output that was reporting the fullchain
path of the renewed certificate.
* update CHANGELOG.md
* cli: redesign output when cert installation failed
- Display a message when certificate installation begins.
- Don't use IReporter, just log errors immediately if restart/rollback
fails.
- Prompt the user with a command to retry the installation process once
they have fixed any underlying problems.
* vary by preconfigured_renewal
and move expiry date to be above the renewal advice
* update code comment
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* update code comment
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* fix lint
* derve cert name from cert_path, if possible
* fix type annotation
* text change in nginx hint
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* print message when restarting server after renewal
* log: print "advice" when exiting with an error
When running in non-quiet mode.
* try fix -oldest lock_test.py
* fix docstring
* s/Restarting/Reloading/ when notifying the user
* fix test name
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* type annotations
* s/using the {} plugin/installer: {}/
* copy: avoid "plugin" where possible
* link to user guide#automated-renewals
when not running with --preconfigured-renewal
* cli: reduce default logging verbosity
* fix lock_test: -vv is needed to see logger.debug
* Change comment in log.py to match the change to default verbosity
* Audit and adjust logging levels in apache module
* Audit and adjust logging levels in nginx module
* Audit, adjust logging levels, and improve logging calls in certbot module
* Fix tests to mock correct methods and classes
* typo in non-preconfigured-renewal message
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* fix test
* revert acme version bump
* catch up to python3 changes
* Revert "revert acme version bump"
This reverts commit fa83d6a51c.
* Change ocsp check error to warning since it's non-fatal
* Update storage_test in parallel with last change
* get rid of leading newline on "Deploying [...]"
* shrink renewal and installation success messages
* print logfile rather than logdir in exit handler
* Decrease logging level to info for idempotent operation where enhancement is already set
* Display cert not yet due for renewal message when renewing and no other action will be taken, and change cert to certificate
* also write to logger so it goes in the log file
* Don't double write to log file; fix main test
* cli: remove trailing newline on new cert reporting
* ignore type error
* revert accidental changes to dependencies
* Pass tests in any timezone by using utcfromtimestamp
* Add changelog entry
* fix nits
* Improve wording of try again message
* minor wording change to changelog
* hooks: send hook stdout to CLI stdout
includes both --manual and --{pre,post,renew} hooks
* update docstrings and remove TODO
* add a pending deprecation on execute_command
* add test coverage for both
* update deprecation text
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Co-authored-by: Alex Zorin <alex@zorin.id.au>
Co-authored-by: alexzorin <alex@zor.io>
This is part of https://github.com/certbot/certbot/issues/8782. I took it on now because the currently pinned version of `pylint` doesn't work with newer versions of `poetry` which I wanted to upgrade as part of https://github.com/certbot/certbot/issues/8787.
To say a bit more about the specific changes in this PR:
* Newer versions of `pylint` complain if `Popen` isn't used as a context manager. Instead of making this change, I switched to using `subprocess.run` which is simpler and [recommended in the Python docs](https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module). I also disabled this check in a few places where no longer using `Popen` would require significant refactoring.
* The deleted code in `certbot/certbot/_internal/renewal.py` is cruft since https://github.com/certbot/certbot/pull/8685.
* The unused argument to `enable_mod` in the Apache plugin is used in some over the override classes that subclass that class.
* unpin pylint and repin dependencies
* disable raise-missing-from
* disable wrong-input-order
* remove unused code
* misc lint fixes
* remove unused import
* various lint fixes
* Update assertTrue/False to Python 3 precise asserts
* Fix test failures
* Fix test failures
* More replacements
* Update to Python 3 asserts in acme-module
* Fix Windows test failure
* Fix failures
* Fix test failure
* More replacements
* Don't include the semgrep rules
* Fix test failure
* Move version.py to tests/letstest since it's used by test_sdists.sh
* Delete unused components of certbot-auto
* Remove test_leauto_upgrades.sh and references to it
* Remove test_letsencrypt_auto_certonly_standalone.sh and references to it
* Remove outstanding references to certbot-auto
* Remove references to letsencrypt-auto
* find certbot in the correct directory
* delete letsencrypt-auto-source line from .isort.cfg since that directory no longer contains any python code
* remove (-auto) from certbot(-auto)
* delete line from test
* Improve style for version.py
This is one of the things that newer versions of `pylint` complains about.
* git grep -l super\( | xargs sed -i 's/super([^)]*)/super()/g'
* fix spacing
Built on top of #8748, this PR reenables mypy strict mode and adds the appropriate corrections to pass the types checks.
* Upgrade mypy
* First step for acme
* Cast for the rescue
* Fixing types for certbot
* Fix typing for certbot-nginx
* Finalize type fixes, configure no optional strict check for mypy in tox
* Align requirements
* Isort
* Pylint
* Protocol for python 3.6
* Use Python 3.9 for mypy, make code compatible with Python 3.8<
* Pylint and mypy
* Pragma no cover
* Pythonic NotImplemented constant
* More type definitions
* Add comments
* Simplify typing logic
* Use vararg tuple
* Relax constraints on mypy
* Add more type
* Do not silence error if target is not defined
* Conditionally import Protocol for type checking only
* Clean up imports
* Add comments
* Align python version linting with mypy and coverage
* Just ignore types in an unused module
* Add comments
* Fix lint
* Work in progress
* Finish type control
* Isort
* Fix pylint
* Fix imports
* Fix cli subparser
* Some fixes
* Coverage
* Remove --no-strict-optional (obviously...)
* Update certbot-apache/certbot_apache/_internal/configurator.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/certbot/_internal/display/completer.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Cleanup dns_google
* Improve lock controls and fix subparser
* Use the expected interfaces
* Fix code
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8661.
As mentioned in https://github.com/certbot/certbot/issues/8661#issuecomment-806168214, there are quite a few remaining references, but until we modify the release script, we still need those. The changes here and the list there were created by grepping for the following terms:
```
certbot-auto
cb-auto
cbauto
certbotauto
letsencrypt-auto
le-auto
leauto
letsencryptauto
LEAUTO
LE_AUTO
LETSENCRYPT_AUTO
LETSENCRYPTAUTO
CB_AUTO
CERTBOT_AUTO
CBAUTO
CERTBOTAUTO
```
* Remove references to certbot-auto from certbot code
* Remove references to LEAUTO
* Remove references to CERTBOT_AUTO
* Remove references to letsencrypt-auto
* Remove references to certbot-auto from docs and tools
* remove cli constants header files
* Remove Python virtual environment section
