* Fix some typos (found by codespell)
Signed-off-by: Stefan Weil <sw@weilnetz.de>
* Remove typo fixes for some files which should not be modified
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Fixes https://github.com/certbot/certbot/issues/6844.
This PR does two things:
1. Changes ACMEv1 deprecation warnings from `PendingDeprecationWarning` to `DeprecationWarning`.
2. Changes the ACMEv1 deprecation warnings to be on references to the class themselves. This is the approach taken in https://github.com/certbot/certbot/pull/8989, the PRs linked there, and the `cryptography` code in the code comment. I think this approach warns in more cases and I updated our unit tests to avoid hitting these warnings.
* add ip address support to acme saving
* remove client-site check for ip address
* using right prefix for san parsing
* remove type hint for backward compatibility
* remove bare ip blocking check from main_test
* upppercase
* lint tix
* add additional tests for new IP support
* support for ipv6 bare address
* make apache and nginx plugin raise error for certs with ip address
* linting
* add pem file's last newline char
* gen_ss_cert ip support and comment fixup
* fix test coverage
* indent fix and assetTrue to assetIN
* indent mistake, made a note where class end
* acme lib now receive IPs as separate list
* fix typos
* type 2
* fix tests
* Deny IP address on certbot/certbot side as LE don't support it
* remove excess empty line to rerun tox
* comment indent and typo fix
Apply suggestions from code review
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* trim unused functions
* trim unused import
* make raw san list extraction as separate function
* Apply suggestions from code review
mostly comment suggestions here
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* apply patches suggested on review.
* remove excessive empty lines
* update CHANGELOG.md
* added acme lib update about ipaddress support in CHANGELOG.md
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
As a follow-up to #8971, this PR removes all references to the old Zope interfaces, except the ones used to deprecate them and prepare for their removal.
In the process, some documentation and tests about the `Display` objects are simply removed since they are not relevant anymore given that they are removed from the public API.
* Cleanup some interfaces.IInstaller
* Cleanup IConfig doc
* Allmost complete removal
* Remove useless tests
* Fixes
* More cleanup
* More cleanup
* More cleanup
* Remove a non existent reference
* Better type
* Fix lint
`distro.linux_distribution` was deprecated (https://github.com/python-distro/distro/pull/296) in the release of `distro` at the end of last week. The deprecation is causing the `nopin` nightly tests to fail.
This change migrates Certbot off that function.
As far as I can tell, the Arch Linux edge case described in the code comments no longer happens, but better to be safe than sorry I think.
* stop using deprecated distro.linux_distribution
* update comment
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* cli: vary renewal advice for hookless manual certs
1. Don't print that the certificate will be automatically renewed,
because it won't be.
2. Add a "NEXT STEP" telling the user that they will need to manually
re-issue the certificate in order to renew it.
* kill superfluous comma
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* clarify wording of the next step
* fix the test
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Also, update `dev-cli.ini` example to use new flag.
Although https://github.com/bw2/ConfigArgParse/pull/216 allowed setting a `count` action value in a config file, our default detection system won't let us use that functionality. While we should eventually fix that, for now, let developers have a cli.ini with a higher logging level by adding this flag.
Note that this flag is intended to work the same way adding `-vvv`s does; that is, as a modifier to the pre-set level, rather than setting the absolute level. The number it is set to is equivalent to the number of `v`s that would otherwise have been passed, with "2" as the current maximum effective number of levels (warning --> info --> debug).
* Add --verbose-level flag for devs to set in cli.ini
* Update dev-cli.ini to use new flag
* Deprecate usage of IConfig as a singleton in Certbot
* Fix local oldest requirements
* Add changelog
* Add tests for certbot.crypto_util.init_save_* functions
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* later printing of renewal and install retry advice
Move printing of advice for automated renewal, and retrying installation
in case of failure, towards the end of `run` and `certonly`.
Also adds some renewal advice for the --csr case (no autorenewal).
* update renewal advice for preconfigured-renewal
* rewrite in terms of "NEXT STEPS" for run/certonly
* fix lint
* re-add "Could not install certificate"
* update --csr renewal advice
* rewrite non-preconfigured-renewal renewal advice
* Remove no names found in configuration files because it sounds like an error but actually it is fine
* fix test
* Pose question more grammatically and specifically, and remove extra space
* fix lint
Co-authored-by: Alex Zorin <alex@zorin.id.au>
Streamline and reorganize Certbot's CLI output.
This change is a substantial command-line UX overhaul,
based on previous user research. The main goal was to streamline
and clarify output. To see more verbose output, use the -v or -vv flags.
---
* nginx,apache: CLI logging changes
- Add "Successfully deployed ..." message using display_util
- Remove IReporter usage and replace with display_util
- Standardize "... could not find a VirtualHost ..." error
This changes also bumps the version of certbot required by certbot-nginx
and certbot-apache to take use of the new display_util function.
* fix certbot_compatibility_test
since the http plugins now require IDisplay, we need to inject it
* fix dependency version on certbot
* use better asserts
* try fix oldest deps
because certbot 1.10.0 depends on acme>=1.8.0, we need to use
acme==1.8.0 in the -oldest tests
* cli: redesign output of new certificate reporting
Changes the output of run, certonly and certonly --csr. No longer uses
IReporter.
* cli: redesign output of failed authz reporting
* fix problem sorting to be stable between py2 & 3
* add some catch-all error text
* cli: dont use IReporter for EFF donation prompt
* add per-authenticator hints
* pass achalls to auth_hint, write some tests
* exclude static auth hints from coverage
* dont call auth_hint unless derived from .Plugin
* dns fallback hint: dont assume --dns-blah works
--dns-blah won't work for third-party plugins, they need to be specified
using --authenticator dns-blah.
* add code comments about the auth_hint interface
* renew: don't restart the installer for dry-runs
Prevents Certbot from superfluously invoking the installer restart
during dry-run renewals. (This does not affect authenticator restarts).
Additionally removes some CLI output that was reporting the fullchain
path of the renewed certificate.
* update CHANGELOG.md
* cli: redesign output when cert installation failed
- Display a message when certificate installation begins.
- Don't use IReporter, just log errors immediately if restart/rollback
fails.
- Prompt the user with a command to retry the installation process once
they have fixed any underlying problems.
* vary by preconfigured_renewal
and move expiry date to be above the renewal advice
* update code comment
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* update code comment
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* fix lint
* derve cert name from cert_path, if possible
* fix type annotation
* text change in nginx hint
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* print message when restarting server after renewal
* log: print "advice" when exiting with an error
When running in non-quiet mode.
* try fix -oldest lock_test.py
* fix docstring
* s/Restarting/Reloading/ when notifying the user
* fix test name
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* type annotations
* s/using the {} plugin/installer: {}/
* copy: avoid "plugin" where possible
* link to user guide#automated-renewals
when not running with --preconfigured-renewal
* cli: reduce default logging verbosity
* fix lock_test: -vv is needed to see logger.debug
* Change comment in log.py to match the change to default verbosity
* Audit and adjust logging levels in apache module
* Audit and adjust logging levels in nginx module
* Audit, adjust logging levels, and improve logging calls in certbot module
* Fix tests to mock correct methods and classes
* typo in non-preconfigured-renewal message
Co-authored-by: ohemorange <ebportnoy@gmail.com>
* fix test
* revert acme version bump
* catch up to python3 changes
* Revert "revert acme version bump"
This reverts commit fa83d6a51c.
* Change ocsp check error to warning since it's non-fatal
* Update storage_test in parallel with last change
* get rid of leading newline on "Deploying [...]"
* shrink renewal and installation success messages
* print logfile rather than logdir in exit handler
* Decrease logging level to info for idempotent operation where enhancement is already set
* Display cert not yet due for renewal message when renewing and no other action will be taken, and change cert to certificate
* also write to logger so it goes in the log file
* Don't double write to log file; fix main test
* cli: remove trailing newline on new cert reporting
* ignore type error
* revert accidental changes to dependencies
* Pass tests in any timezone by using utcfromtimestamp
* Add changelog entry
* fix nits
* Improve wording of try again message
* minor wording change to changelog
* hooks: send hook stdout to CLI stdout
includes both --manual and --{pre,post,renew} hooks
* update docstrings and remove TODO
* add a pending deprecation on execute_command
* add test coverage for both
* update deprecation text
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Co-authored-by: Alex Zorin <alex@zorin.id.au>
Co-authored-by: alexzorin <alex@zor.io>
This is part of https://github.com/certbot/certbot/issues/8782. I took it on now because the currently pinned version of `pylint` doesn't work with newer versions of `poetry` which I wanted to upgrade as part of https://github.com/certbot/certbot/issues/8787.
To say a bit more about the specific changes in this PR:
* Newer versions of `pylint` complain if `Popen` isn't used as a context manager. Instead of making this change, I switched to using `subprocess.run` which is simpler and [recommended in the Python docs](https://docs.python.org/3/library/subprocess.html#using-the-subprocess-module). I also disabled this check in a few places where no longer using `Popen` would require significant refactoring.
* The deleted code in `certbot/certbot/_internal/renewal.py` is cruft since https://github.com/certbot/certbot/pull/8685.
* The unused argument to `enable_mod` in the Apache plugin is used in some over the override classes that subclass that class.
* unpin pylint and repin dependencies
* disable raise-missing-from
* disable wrong-input-order
* remove unused code
* misc lint fixes
* remove unused import
* various lint fixes
* Update assertTrue/False to Python 3 precise asserts
* Fix test failures
* Fix test failures
* More replacements
* Update to Python 3 asserts in acme-module
* Fix Windows test failure
* Fix failures
* Fix test failure
* More replacements
* Don't include the semgrep rules
* Fix test failure
* Move version.py to tests/letstest since it's used by test_sdists.sh
* Delete unused components of certbot-auto
* Remove test_leauto_upgrades.sh and references to it
* Remove test_letsencrypt_auto_certonly_standalone.sh and references to it
* Remove outstanding references to certbot-auto
* Remove references to letsencrypt-auto
* find certbot in the correct directory
* delete letsencrypt-auto-source line from .isort.cfg since that directory no longer contains any python code
* remove (-auto) from certbot(-auto)
* delete line from test
* Improve style for version.py
This is one of the things that newer versions of `pylint` complains about.
* git grep -l super\( | xargs sed -i 's/super([^)]*)/super()/g'
* fix spacing
Built on top of #8748, this PR reenables mypy strict mode and adds the appropriate corrections to pass the types checks.
* Upgrade mypy
* First step for acme
* Cast for the rescue
* Fixing types for certbot
* Fix typing for certbot-nginx
* Finalize type fixes, configure no optional strict check for mypy in tox
* Align requirements
* Isort
* Pylint
* Protocol for python 3.6
* Use Python 3.9 for mypy, make code compatible with Python 3.8<
* Pylint and mypy
* Pragma no cover
* Pythonic NotImplemented constant
* More type definitions
* Add comments
* Simplify typing logic
* Use vararg tuple
* Relax constraints on mypy
* Add more type
* Do not silence error if target is not defined
* Conditionally import Protocol for type checking only
* Clean up imports
* Add comments
* Align python version linting with mypy and coverage
* Just ignore types in an unused module
* Add comments
* Fix lint
* Work in progress
* Finish type control
* Isort
* Fix pylint
* Fix imports
* Fix cli subparser
* Some fixes
* Coverage
* Remove --no-strict-optional (obviously...)
* Update certbot-apache/certbot_apache/_internal/configurator.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/certbot/_internal/display/completer.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Cleanup dns_google
* Improve lock controls and fix subparser
* Use the expected interfaces
* Fix code
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8661.
As mentioned in https://github.com/certbot/certbot/issues/8661#issuecomment-806168214, there are quite a few remaining references, but until we modify the release script, we still need those. The changes here and the list there were created by grepping for the following terms:
```
certbot-auto
cb-auto
cbauto
certbotauto
letsencrypt-auto
le-auto
leauto
letsencryptauto
LEAUTO
LE_AUTO
LETSENCRYPT_AUTO
LETSENCRYPTAUTO
CB_AUTO
CERTBOT_AUTO
CBAUTO
CERTBOTAUTO
```
* Remove references to certbot-auto from certbot code
* Remove references to LEAUTO
* Remove references to CERTBOT_AUTO
* Remove references to letsencrypt-auto
* Remove references to certbot-auto from docs and tools
* remove cli constants header files
* Remove Python virtual environment section
Fixes#8427
This PR converts the Python 2 types hints into Python 3 types annotations. I have used the project https://github.com/ilevkivskyi/com2ann which has been designed for that specific purpose and did that very well.
The only remaining things to do were to fix broken type hints that became wrong code after migration, and to fix lines too long with the new syntax.
* Raw execution of com2ann
* Fixing broken type annotations
* Cleanup imports
There are still some left, but the `modification_check` test fails. Some are still in `tools`, and they can probably be removed as well. `with_statement` was introduced officially in Python 2.5, so there's really old stuff in the code base.
* revoke: try determine the server automatically
When revoking via --cert-name, use the server from the lineage (unless
overriden by the CLI).
* RenewableCert.storage might be None
* guard against an empty lineage server
There is some code in [`_paths_parser`](https://github.com/certbot/certbot/blob/ae3ed200c09f2d1913ec69c33a2604ae1fe32eb5/certbot/certbot/_internal/cli/paths_parser.py#L17-L34) which has the effect of varying the value type of `config.cert_path` and `config.key_path` based on the CLI verb. When the verb is `revoke`, the type is a tuple `(path: str, contents: bytes)`, otherwise it is a single `str` representing the file path. (I wasn't able to find a written reason as to why it works this way).
This commit removes that special `revoke` case and ensures it is always a `str`.
Why change it now?
I am trying to write some changes and there's some code in `cert_manager` which only works if the verb is `revoke`, you hack `config.cert_path` to be a tuple beforehand, or you [(not actually in `master`) try sniff for the value type](https://github.com/certbot/certbot/blob/49911afaa62ade1fca4c4ec407f817720cb354e3/certbot/certbot/_internal/cert_manager.py#L224-L227). I have a bad feeling about such workarounds. I would prefer to just make these variables simpler to use, but I'm open to opinions.
In addition to the test suites, I've manually tested `revoke` (including by `--key-path`) and `install`. Are there other places I may have missed?
Unblocks #8636 and #8671.
Fixes#8680.
We seem to have no existing testing code anywhere in this vicinity, so figured I'd get this up quickly then work on that. Manual tests (renew staging certificate, should allow it; renew non-staging cert as staging, should error) passed.
* Remove check for 'fake' in issuer name when renewing certs
* Change fake issuer name to make sure we're not relying on it anywhere
This PR deprecates the certbot-auto specific CLI flags, in the perspective of removing them in a future release as said in #8483.
* Deprecate certbot-auto specific flags
* Update changelog
* Clean tests
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes https://github.com/certbot/certbot/issues/8494.
I left the `six` dependency pinned in `tests/letstest/requirements.txt` and `tools/oldest_constraints.txt` because `six` is still a transitive dependency with our current pinnings.
The extra moving around of imports is due to me using `isort` to help me keep dependencies in sorted order after replacing imports of `six`.
* remove some six usage in acme
* remove six from acme
* remove six.add_metaclass usage
* fix six.moves.zip
* fix six.moves.builtins.open
* six.moves server fixes
* 's/six\.moves\.range/range/g'
* stop using six.moves.xrange
* fix urllib imports
* s/six\.binary_type/bytes/g
* s/six\.string_types/str/g
* 's/six\.text_type/str/g'
* fix six.iteritems usage
* fix itervalues usage
* switch from six.StringIO to io.StringIO
* remove six imports
* misc fixes
* stop using six.reload_module
* no six.PY2
* rip out six
* keep six pinned in oldest constraints
* fix log_test.py
* update changelog
* Remove deprecated options as early as possible using an explicit list
* add deprecated options to cli init import list
* use correct dict comprehension syntax for py3
* lint
* add test for renewal reconstitution code
* add test to ensure we're not saving deprecated values
* comment code
* --preferred-chain: only match root name
Currently, when certbot is given the `--preferred-chain='Some Name'`
flag, it iterates through all alternate chains offered by the ACME
server until it finds any certificate which has `'Some Name'` as its
Issuer Common Name. Unfortunately, this means that if the desired
alternate chain is a strict subset of any earlier chain (e.g. the
default chain is 'EE <-- Int <-- Root1 <-- Root2', but the desired
chain is 'EE <-- Int <-- Root1'), there is no name which can be
provided by the user which will allow the client to select the desired
chain.
This change makes it so that the `find_chain_with_issuer` logic only
cares about the Issuer Common Name found in the last certificate in
each chain. In the example above, the user would then be able to get
their desired chain by specifying `--preferred-chain='Root1'`: although
that name appears in the default chain, it does not appear in the
highest certificate of that chain.
This change is technically backwards-incompatible. However, the only
advice that has been given to users of certbot (and the only usecase
that we believe has existed so far) involved setting the flag to a
value that is the name of a root, not an intermediate, so we don't
expect any real-world configurations or use-cases to be broken.
Fixes#8577
* Update interfaces.py
* test: certbot-ci crash due to no p521 on boulder
The bugfix in #8598 added an integration test to request a certificate
for an EC P-521 key, which is unsupported when ACME_SERVER=boulder,
failing our nightly integration tests.
* add an integration test for all EC curves
The method `os.readlink()` has a significant behavior change with Python 3.8+ on Windows.
Starting with this version, it will return the resolved path in its "extended-style" form unconditionally, a form which allows to use more than 259 characters in a Windows path, and its string representation is prepended with "\\\\?\\".
See https://docs.microsoft.com/fr-fr/windows/win32/fileio/naming-a-file?redirectedfrom=MSDN#maximum-path-length-limitation
Problem is that `os.readlink()` does it for any path, including paths that could be represented with the normal form. As a consequence, any string comparison with a path provided in the normal form will fail even if it represents the same path. This makes Certbot partially break on Windows with Python 3.8.
My proposition in this PR is to forbid `os.readlink()`, and provide `certbot.compat.filesystem.readlink()` which serves the same purpose at resolving the pointed path of a link, and has a consistent behavior over supported Python versions.
* Forbid os.readlink()
* Use readlink
* Raise error with long paths on Windows
* Add unit tests
* Update certbot/certbot/compat/filesystem.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Edit certs -> certificates in user-facing text.
To reduce confusion, we should consistently use the full term.
* Edit certs->certificates in more user-facing text.
* fix failing lint (line too long)
* fix typo
Co-authored-by: Jacob Hoffman-Andrews <github@hoffman-andrews.com>
Co-authored-by: Alex Zorin <alex@zorin.id.au>
* update_account: print correct message for -m ""
When -m "" was passed on the CLI, Certbot would print that it updated
the email to '' (an empty string) rather than printing that it removed
the contact details.
This commit also refactors the update_account tests to be a bit more
modern.
* use addCleanup instead of tearDown in tests
* cli: clean up `certbot renew` summary
- Unduplicate output which was being sent to both stdout and stderr
- Don't use IDisplay.notification to buffer output
- Remove big "DRY RUN" guards above and below, instead change language
to "renewal" or "simulated renewal"
- Reword "Attempting to renew cert ... produced an unexpected error"
to be more concise.
* add newline to docstring
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Co-authored-by: ohemorange <ebportnoy@gmail.com>
Fixes https://github.com/certbot/certbot/issues/8495.
To further explain the problem here, `modify_kwargs_for_default_detection` as called in `add` is simplistic and doesn't always work. See https://github.com/certbot/certbot/issues/6164 for one other example.
In this case, were bitten by the code https://github.com/certbot/certbot/blob/d1e7404358c05734aaf436ef3c9d709029d62b09/certbot/certbot/_internal/cli/helpful.py#L393-L395
The action used for deprecated arguments isn't in `ZERO_ARG_ACTIONS` so it assumes that all deprecated flags take one parameter.
Rather than trying to fix this function (which I think can only realistically be fixed by https://github.com/certbot/certbot/issues/4493), I took the approach that was previously used in `HelpfulArgumentParser.add_deprecated_argument` of bypassing this extra logic entirely. I adapted that function to now call `HelpfulArgumentParser.add` as well for consistency and to make testing easier.
* Rename deprecated arg action class
* Skip extra parsing for deprecated arguments
* Add back test of --manual-public-ip-logging-ok
* Add changelog entry
* cli: improve Obtaining/Renewing wording
* dont use logger, and use new phrasing
* .display_util.notify: dont wrap
As this function is supposed to be an analogue for print, we do not want
it to wrap by default.
