The Lexicon-based DNS plugins use a mechanism to determine which actual segment of the input domain is actually the DNS zone in which the DNS-01 challenge has to be initiated (eg. `subdomain.domain.com` or `domain.com` for input `subdomain.domain.com`): they tries recursively to configure Lexicon and initiate authentication from the most specific to most generic domain segment, and select the first segment where Lexicon stop erroring out.
This mechanism broke with #9746 because now the plugins call Lexicon client instead of the underlying providers, and the client makes guess on the actual domain requested. Typically for `subdomain.domain.com` it will actually try to authenticate against `domain.com`, and so the mechanism above does not work anymore.
This PR fixes the issue by using the `delegated` field in Lexicon config each time the plugin needs it. This field is designed for this kind of purpose: it will instruct Lexicon what is the actual DNS zone domain instead of guessing it.
I tested the change with one of my OVH account. The expected behavior is re-established and the plugin is able to test `subdomain.domain.com` then `domain.com` as before.
Fixes#9791Fixes#9818
* Drop Python 3.7 support
* Fix lint and test
* Check for venv generation
* Update requirements
* Update oldest constaints and compatibility tests runtime
* Refactor Lexicon-based DNS plugins and upgrade minimal version of Lexicon
* Relax filterwarning to comply with envs where boto3 is not installed
* Update pinned dependencies
* Use our previous method to deprecate part of modules
* Safe import internally
* Add changelog
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Migrate entrypoint logic from pkg_resources to importlib.metadata
* Usage of importlib_metadata up to Python 3.9 to align API behavior to Python 3.10
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
Co-authored-by: Adrien Ferrand <adrien.ferrand@arteris.com>
* Migrate pkg_resources API related to resources to importlib_resources
* Fix lint and mypy + pin lexicon
* Update filterwarnings
* Update oldest tests requirements
* Update pinned dependencies
* Fix for modern versions of python
* Fix assets load in nginx integration tests
* Fix a warning
* Isolate static generation from importlib.resource into a private function
---------
Co-authored-by: Adrien Ferrand <adrien.ferrand@amadeus.com>
* Work in progress
* Work in progress
* Work in progress
* Work in progress
* Fix issues around nullability of VirtualHost.path, may discuss that during review
* Work in progress
* Fix remaining types
* Various lint fixes
* Reconfigure tox and mypy to disallow untyped defs globally
* Cleanup compatibility tests
* Use cast for unused v2 logic
* Improve types
* Remove unused comment
* Fix coverage
* Better types
* Fix another type
* Update certbot-apache/certbot_apache/_internal/apacheparser.py
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot-apache/certbot_apache/_internal/assertions.py
Co-authored-by: alexzorin <alex@zor.io>
* Fix type
* Various fixes
* Refactor imports
* Keep naming convention consistent on TypeVars
* Improve types
* Improve types
* Remove remaining Sequence[str] in the project
Co-authored-by: alexzorin <alex@zor.io>
* Add generic methods to save some casts, and fix lint
* Update current and oldest pinning
* Fix classes
* Remove some todos thanks to josepy 1.11.0
* Cleanup some useless pylint disable
* Finish complete typing
* Better TypeVar names
* Upgrade pinning and fix some typing errors
* Use protocol
* Fix types in apache
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes https://github.com/certbot/certbot/issues/8983
Python 3.6 is now EOL: https://endoflife.date/python
This is normally a good time to create warnings about Python 3.6 deprecation the Certbot upcoming release 1.23.0 so that its support is removed in 1.24.0.
We have to say here that EPEL maintainers asked us to keep maintaining support of Python 3.6 because Python 3.7 will never be shipped to CentOS 7. This support would be needed in theory up to 2 more years, basically until CentOS 7 EOL in 2024-06-30. It has been said that we could support as a best effort until a reasonable need on Certbot side requires to drop Python 3.6. See https://github.com/certbot/certbot/issues/8983 for more information.
However some of us (including me) consider that there is already a reasonable need right now. Indeed, keeping the support on Python 3.6 while the Python community globally moves away from it will pin implicitly some Certbot dependencies to the last version of these dependencies supporting Python 3.6 as the upstream maintainers decide to make the move. At any point in a future time, one of these dependencies could require an urgent upgrade (typically a critical uncovered vulnerability): then we would require to drop Python 3.6 immediately without further notice instead of following an organized deprecation path.
This reason motivates to proactively deprecate then drop the Python versions once they are EOL. You can see the discussion in Mattermost starting from [this post](https://opensource.eff.org/eff-open-source/pl/ntzs9zy1fprjmkso3xrqspnoce) to get more elements about the reasoning.
* Deprecate Python 3.6 support.
* Ignore our own PendingDeprecationWarning
* Add types in all DNS plugins
* Order imports
* Fix type
* Update certbot-dns-route53/certbot_dns_route53/_internal/dns_route53.py
Co-authored-by: alexzorin <alex@zor.io>
* Clean up imports
Co-authored-by: alexzorin <alex@zor.io>
* Start more types
* Second run
* Work in progress
* Types in all acme module
* Various fixes
* Various fixes
* Final fixes
* Disallow untyped defs for acme project
* Fix coverage
* Remote unecessary type ignore
* Use Mapping instead of Dict as input whenever it is possible
* Update acme/acme/client.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update acme/acme/client.py
Co-authored-by: alexzorin <alex@zor.io>
* Various fixes
* Fix code
* Fix code
* Update acme/acme/client.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update acme/acme/challenges.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update acme/acme/client.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Fix deactivate_registration and simplify signature of update_registration
* Do not leak personal data during account deactivation
* Clean more Dicts
* New fix to not leak contact field in the account deactivation payload.
* Add ignore for python 3.6 type check
* Revert "Add ignore for python 3.6 type check"
This reverts commit da7338137b.
* Let's find a smarter way than "type: ignore"
* Update certbot/certbot/_internal/account.py
Co-authored-by: alexzorin <alex@zor.io>
* Fix an annotation
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: alexzorin <alex@zor.io>
* Generate a web.config file to serve properly challenge files with IIS
* Fix cleanup, add test
* FIx lint
* Do not overwrite existing web.config. Delete only web.config when it has been created by Certbot and is unmodified.
* Fix lint
* Update certbot/certbot/_internal/plugins/webroot.py
Co-authored-by: alexzorin <alex@zor.io>
* Add log
* Check for POSIX_MODE before web.config deletion attempt.
* Add documentation
* Update certbot/CHANGELOG.md
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/docs/using.rst
Co-authored-by: alexzorin <alex@zor.io>
It seems that all required pre-compiled wheels to install Certbot on Python 3.9 on Windows are present.
This PR upgrades Windows tests on Python 3.9 and repackages the installer on this version of Python.
This PR upgrades the pinned version of the dependencies. Version `1.9.0` of josepy is used so errors related to JWK serialization with EC keys (see https://github.com/certbot/josepy/issues/109) are fixed for Certbot.
This PR removes all zope dependencies from plugins configuration.
It also lets Sphinx upgrade to the next major version by removing the plugin dedicated to zope interfaces documentation. As a consequence, the deprecated zope interfaces are not documented anymore.
* Cleanup zope dependencies in plugins and upgrade sphinx
* Update pinnings
As a follow-up to #8971, this PR removes all references to the old Zope interfaces, except the ones used to deprecate them and prepare for their removal.
In the process, some documentation and tests about the `Display` objects are simply removed since they are not relevant anymore given that they are removed from the public API.
* Cleanup some interfaces.IInstaller
* Cleanup IConfig doc
* Allmost complete removal
* Remove useless tests
* Fixes
* More cleanup
* More cleanup
* More cleanup
* Remove a non existent reference
* Better type
* Fix lint
Fixes#8899
This PR removes the pinning upper limit of mypy currently set to <0.900 and adds the required types-* stub packages to make recent versions of mypy work.
* Unpin mypy
* Improve type in TempHandler
* Add types
This PR is a new approach for fixing #8732 based on the discussions occurred in the first PR #8877.
This PR upgrades python-augeas to the latest version, and avoids tests failure of Windows because of this upgrade. To do so it leverages the [tox multi-platform feature](https://tox.readthedocs.io/en/latest/example/platform.html) and modifications to `tools/venv.py` in order to not install and not test `certbot-apache` on Windows.
* Unpin python-augeas and upgrade current pinnings
* Do not install certbot-apache in Windows dev environments
* Introduce tox specific win packages + remove certbot compatibility on windows
* Add libaugeas to sphinx build
* Redefine lint and mypy targets
* Keep the lint and mypy environments
* Deprecate usage of IConfig as a singleton in Certbot
* Fix local oldest requirements
* Add changelog
* Add tests for certbot.crypto_util.init_save_* functions
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8824
This PR makes the installer first delete (if exist) the previous `pkg` directory in the Certbot installation in order to avoid dependencies conflicts when a new version of Certbot (with new versions of dependencies) is intaller other an existing one.
I took the simplest approach here, which is to delete specifically the directories known to create conflicts, instead of more complex approaches that involve to factor in some way the complete uninstaller logic. This is because the complexity added without a clear improvement does not worth it in my opinion. More specifically:
* factorizing in some way the uninstaller section in the NSIS template make the installer use any potential new logic of a new installation of Certbot instead of the one applying for the current installation, and may create unexpected errors during installation or at runtime
* calling the existing `uninstaller.exe` would be better, but I could not find a proper way to let NSIS wait for the actual end of the uninstall logic, and again may create unexpected errors during installation or at runtime
* Cleanup Certbot pkg dir before installing to avoid dependencies conflicts
* Add a changelog
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Since Saturday the CI pipeline is failing due to several Sphinx errors. See https://dev.azure.com/certbot/certbot/_build/results?buildId=3928&view=logs&j=d74e04fe-9740-597d-e9fa-1d0400037dfd&t=dde413a4-f24c-59a0-9684-e33d79f9aa02
First, the build of certbot-dns-google is failing because of a particular configuration. It seems that this configuration has been written here to activate the support of the RST instruction `.. code-block:: json` in documentation. However, it does not seem to be necessary for a similar situation in certbot-dns-route53 documentation. So let's try to remove it and fix the Sphinx builds.
Second, Sphinx builds were not pinning dependencies, so Sphinx 4.x (that has been released yesterday) started to be used in the pipeline. Sadly this new version is not compatible with the plugin `repoze.sphinx.autointerface`, used to extract documentation from `zope.interface`. So I fixed the pinning and also explicitly pin Sphinx to 3.5.x for now.
Technically speaking the second action is sufficient to fix the first error, but I keep the dedicated solution because it improves the documentation in my opinion.
This situation could be fixed by not requiring `repoze.sphinx.autointerface`, but this is possible only if we remove `zope.interface` from Certbot. Luckily I started the work few days ago ;).
* Remove explicit lexer call in certbot-dns-google doc builds.
* Write a valid JSON file in the documentation
* Apply constraints to sphinx build environments
* Pin Sphinx to 3.5.4
* Update dependencies
* Pin traitlets
In https://github.com/certbot/certbot/pull/8748#discussion_r605457670 we discussed about changing the dict used to set OS options for Apache configurators into a dedicated object.
* Create _OsOptions class to configure the os specific options of the Apache configurators
* Fix tests
* Clean imports
* Fix naming
* Fix compatibility tests
* Rename a class
* Ensure restart_cmd_alt is set for specific OSes.
* Add docstring
* Fix override
* Fix coverage
Built on top of #8748, this PR reenables mypy strict mode and adds the appropriate corrections to pass the types checks.
* Upgrade mypy
* First step for acme
* Cast for the rescue
* Fixing types for certbot
* Fix typing for certbot-nginx
* Finalize type fixes, configure no optional strict check for mypy in tox
* Align requirements
* Isort
* Pylint
* Protocol for python 3.6
* Use Python 3.9 for mypy, make code compatible with Python 3.8<
* Pylint and mypy
* Pragma no cover
* Pythonic NotImplemented constant
* More type definitions
* Add comments
* Simplify typing logic
* Use vararg tuple
* Relax constraints on mypy
* Add more type
* Do not silence error if target is not defined
* Conditionally import Protocol for type checking only
* Clean up imports
* Add comments
* Align python version linting with mypy and coverage
* Just ignore types in an unused module
* Add comments
* Fix lint
* Work in progress
* Finish type control
* Isort
* Fix pylint
* Fix imports
* Fix cli subparser
* Some fixes
* Coverage
* Remove --no-strict-optional (obviously...)
* Update certbot-apache/certbot_apache/_internal/configurator.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/certbot/_internal/display/completer.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Cleanup dns_google
* Improve lock controls and fix subparser
* Use the expected interfaces
* Fix code
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8773
I took option 2 from the issue mentionned above (importing `typing-extensions` on dev dependencies) to avoid modifying certbot runtime requirements given that what needs to be added is useful for mypy only.
I did not change the Python version used to execute the linting and mypy on the standard tests, given that the tox `docker_dev` target already checks if the development environment is working for Python < 3.8.
Fixes#8425
This PR upgrades mypy to the latest version available, 0.812.
Given the advanced type inference capabilities provided by this newer version, this PRs also fixes various type inconsistencies that are now detected. Here are the non obvious changes done to fix types:
* typing in mixins has been solved using `Protocol` classes, as recommended by mypy (https://mypy.readthedocs.io/en/latest/more_types.html#mixin-classes, https://mypy.readthedocs.io/en/stable/protocols.html)
* `cast` when we are playing with `Union` types
This PR also disables the strict optional checks that have been enable by default in recent versions of mypy. Once this PR is merged, I will create an issue to study how these checks can be enabled.
`typing.Protocol` is available only since Python 3.8. To keep compatibility with Python 3.6, I try to import the class `Protocol` from `typing`, and fallback to assign `object` to `Protocol` if that fails. This way the code is working with all versions of Python, but the mypy check can be run only with Python 3.8+ because it needs the protocol feature. As a consequence, tox runs mypy under Python 3.8.
Alternatives are:
* importing `typing_extensions`, that proposes backport of newest typing features to Python 3.6, but this implies to add a dependency to Certbot just to run mypy
* redesign the concerned classes to not use mixins, or use them differently, but this implies to modify the code itself even if there is nothing wrong with it and it is just a matter of instructing mypy to understand in which context the mixins can be used
* ignoring type for these classes with `# type: ignore` but we loose the benefit of mypy for them
* Upgrade mypy
* First step for acme
* Cast for the rescue
* Fixing types for certbot
* Fix typing for certbot-nginx
* Finalize type fixes, configure no optional strict check for mypy in tox
* Align requirements
* Isort
* Pylint
* Protocol for python 3.6
* Use Python 3.9 for mypy, make code compatible with Python 3.8<
* Pylint and mypy
* Pragma no cover
* Pythonic NotImplemented constant
* More type definitions
* Add comments
* Simplify typing logic
* Use vararg tuple
* Relax constraints on mypy
* Add more type
* Do not silence error if target is not defined
* Conditionally import Protocol for type checking only
* Clean up imports
* Add comments
* Align python version linting with mypy and coverage
* Just ignore types in an unused module
* Add comments
* Fix lint
In #8649 we added some code to trick pynsist and make it understand that `abi3` wheels for Windows are forward compatible, meaning that the cryptography wheel tagged `cp36-abi3` is in fact compatible with Python 3.6+, and not only Python 3.6.
Since pynsist 2.7 the tool now understand `abi3` wheels properly, and this trick is not needed anymore.
Please note that despite modifying the pynsist pinning in `dev_constraints.txt`, it will have no effect since pynsist currently escape the pinning system. This is handled in https://github.com/certbot/certbot/pull/8749.
* Pin pynsist
* Update dependencies
* Set windows installer a proper python project
* Optimize usage of the venvs
* Add windows-installer when venv is set up
* Fix call
* Remove env marker
Fixes#8700
Now that `snapcraft remote-build` truly uses new builds for each call, we can split the builds to have a dedicated Azure job for each target architecture. This PR does that.
* Split snap_build job on each architecture
* Also parallelize the publish_snap jobs over each architecture
We observed recently several unexpected behavior during the execution of snap jobs in Azure. In particular it seems that `snapcraft remote-build` is tending to reattach to the latest builds on Launchpad triggered by the nightly builds on master, independently from the actual branch, status of the code, or targeted architectures.
Primarily if the builds on Launchpad are stalled for some reason, it blocks effectively any other Azure snap jobs until someone manually cancel the builds on Launchpad. Secondarily it means that the outcome of the builds may be inconsistent, because they can be the result of a build for the master source even if you are on a PR that modifieds these sources (including `snapcraft.yaml`).
After digging in `snapcraft` source code, I realized that the signature computed to understand if a build should be resumed, is not based one some hashes against the snapcraft working directory content, but is simply a hash of the working directory absolute path *itself*. It means that every builds triggered from the working directory `/my/path/certbot` for instance, are recognized as the same unique build on Launchpad side, and may be resumed if they already exist, and so independently from the source code, `snapcraft.yaml` or targeted archs.
For the record, relevant parts in `snapcraft` source code:
https://github.com/snapcore/snapcraft/blob/82024d3748f96fc63077de8188c5752b58d151be/snapcraft/project/_project.py#L44https://github.com/snapcore/snapcraft/blob/82024d3748f96fc63077de8188c5752b58d151be/snapcraft/project/_project.py#L86-L89https://github.com/snapcore/snapcraft/blob/82024d3748f96fc63077de8188c5752b58d151be/snapcraft/cli/remote.py#L128-L132
This PR makes effectively the resume build mechanism effectively a noop by moving the source code first in a temporary directory with random name before running `snapcraft remote-build`. This way the signature is never the same and builds are always recognized as brand new builds.
* Invalidate snapcraft remote-build cache by using a temporary workspace.
* Capture one more state in the build
* Precise the certificate naming convention mechanism in a note.
* Add certificate name convention in user guide, refer to it in compatibility page.
* Update certbot/docs/compatibility.rst
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/docs/using.rst
Co-authored-by: alexzorin <alex@zor.io>
* Update certbot/docs/using.rst
Co-authored-by: alexzorin <alex@zor.io>
* Improve the note about naming conventions
Co-authored-by: alexzorin <alex@zor.io>
While working on #8640, I realized that there were some hidden circular dependencies in certbot._internal.cli package. Then cerbot could break if the order of these imports changes.
This PR fixes that and apply isort on top of the result.
* Kill snapcraft build when a "Chroot problem" is encountered
* Display specific helper for "Chroot problem" status and cancel retry mechanism in this case.
* Isolate build tmp directories
* Configure XDG_CACHE_HOME
* Kill snapcraftctl with chroot problem is encountered
Fixes#8427
This PR converts the Python 2 types hints into Python 3 types annotations. I have used the project https://github.com/ilevkivskyi/com2ann which has been designed for that specific purpose and did that very well.
The only remaining things to do were to fix broken type hints that became wrong code after migration, and to fix lines too long with the new syntax.
* Raw execution of com2ann
* Fixing broken type annotations
* Cleanup imports
This PR deprecates the certbot-auto specific CLI flags, in the perspective of removing them in a future release as said in #8483.
* Deprecate certbot-auto specific flags
* Update changelog
* Clean tests
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8389#8584.
This PR makes the necessary modifications to officially drop Python 2 support in the Certbot project.
I did not remove the specific Python 2 compatibility branches that has been added in various places in the codebase, to reduce the size of this PR and this will be done in a future one
* Update classifiers and python_requires in setup.py
* Remove warnings about Python 2 deprecation
* Remove Azure jobs on Python 2.7
* Remove references to python 2 in documentation
* Pin dnspython to 2.1.0
* Update changelog
* Remove warning ignore
Now that we have a new pipstrap script with recent version of pip, dependencies for Windows can be resolved correctly on Python 3.8.
This PR enables tests on Python 3.8, and package Certbot for Windows on Python 3.8 also. I do not move up to Python 3.9 since some dependencies (`cryptography`, `pynacl`) do not provide wheels for Python 3.9 yet on Windows, which would require a complete C++ build system to compile them.
* Enable windows tests on Python 3.8 and package it on Python 3.8 also.
* Upgrade pynsist, nsis and pywin32, remove old workarounds
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
The method `os.readlink()` has a significant behavior change with Python 3.8+ on Windows.
Starting with this version, it will return the resolved path in its "extended-style" form unconditionally, a form which allows to use more than 259 characters in a Windows path, and its string representation is prepended with "\\\\?\\".
See https://docs.microsoft.com/fr-fr/windows/win32/fileio/naming-a-file?redirectedfrom=MSDN#maximum-path-length-limitation
Problem is that `os.readlink()` does it for any path, including paths that could be represented with the normal form. As a consequence, any string comparison with a path provided in the normal form will fail even if it represents the same path. This makes Certbot partially break on Windows with Python 3.8.
My proposition in this PR is to forbid `os.readlink()`, and provide `certbot.compat.filesystem.readlink()` which serves the same purpose at resolving the pointed path of a link, and has a consistent behavior over supported Python versions.
* Forbid os.readlink()
* Use readlink
* Raise error with long paths on Windows
* Add unit tests
* Update certbot/certbot/compat/filesystem.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8256
First let's sum up the problem to solve. We disabled the build isolation available in pip>=19 because it could potential break certbot build without a control on our side. Basically builds are not reproductible. Indeed the build isolation triggers build of PEP-517 enabled transitive dependencies (like `cryptography`) with the build dependencies defined in their `pyproject.toml`. For `cryptography` in particular these requirements include `setuptools>=40.6.0`, and quite logically pip will install the latest version of `setuptools` for the build. And when `setuptools` broke with the version 50, our build did the same.
But disabling the build isolation is not a long term solution, as more and more project will migrate on this approach and it basically provides a lot of benefit in how dependencies are built.
The ideal solution would be to be able to apply version constraints on our side on the build dependencies, in order to pin `setuptools` for instance, and decide precisely when we upgrade to a newer version. However for now pip does not provide a mechanism for that (like a `--build-constraint` flag or propagation of existing `--constraint` flag).
Until I saw https://github.com/pypa/pip/issues/9081 and https://github.com/pypa/pip/issues/8439.
Apart the fact that https://github.com/pypa/pip/issues/9081 shows that pip maintainers are working on this issue, it explains how pip works regarding PEP-517 and infers which workaround can be used to still pin the build dependencies. It turns out that pip invokes itself in each build isolation to install the build dependencies. It means that even if some flags (like `--constraint`) are not explicitly passed to the pip sub call, the global environment remains, in particular the environment variables.
Thus it is known that every pip flag can alternatively be set by environment variable using the following pattern for the variable name: `PIP_[FLAG_NAME_UPPERCASE]`. So for `--constraint`, it is `PIP_CONSTRAINT`. And so you can pass a constraint file to the pip sub call through that mechanism.
I made some tests with a constraint file containing pinning for `setuptools`: indeed under isolation zone, the constraint file has been honored and the provided pinned version has been used to build the dependencies (I tested it with `cryptography`).
Finally this PR takes advantage of this mechanism, by setting `PIP_CONSTRAINT` to `pip_install`, the snap building process, the Dockerfiles and the windows installer building process.
I also extracted out the requirements of the new `pipstrap.py` to be reusable in these various build processes.
* Use workaround to fix build requirements in build isolation, and renable build isolation
* Clean imports in pipstrap
* Externalize pipstrap reqs to be reusable
* Inject pipstrap constraints during pip_install
* Update docker build
* Update snapcraft build
* Prepare installer build
* Fix pipstrap constraints in snap build
* Add back --no-build-cache option in Docker images build
* Update snap/snapcraft.yaml
* Use proper flags with pip
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
This PR adds a `--timeout` flag to `tools/snap/build_remote.py` in order to fail the process if the time execution reaches the provided timeout. It is set to 5h30 on the relevant Azure job, while the job itself has a timeout of 6h managed on Azure side. This allows a slightly better output for these jobs when the snapcraft build stales for any reason.
This PR proposes an alternative configuration for the snap build that avoid the need to use `--system-site-package` when constructing the virtual environment in the snap.
The rationale of `--system-site-package` was that by default, snapcraft creates a virtual environment without `wheel` installed in it. However we need it to build the wheels like `cryptography` on ARM architectures. Sadly there is not way to instruct snapcraft to install some build dependencies in the virtual environment before it kicks in the build phase itself, without overriding that entire phase (which is possible with `parts.override-build`).
The alternative proposed here is to not override the entire build part, but just add some preparatory steps that will be done before the main actions handled by the `python` snap plugin. To do so, I take advantage of the `--upgrade` flag available for the `venv` module in Python 3. This allows to reuse a preexisting virtual environment, and upgrade its component. Adding a flag to the `venv` call is possible in snapcraft, thanks to the `SNAPCRAFT_PYTHON_VENV_ARGS` environment variable (and it is already used to set the `--system-site-package`).
Given `SNAPCRAFT_PYTHON_VENV_ARGS` set to `--upgrade` , we configure the build phase as follows:
* create the virtual environment ourselves in the expected place (`SNAPCRAFT_PART_INSTALL`)
* leverage `tools/pipstrap.py` to install `setuptools`, `pip`, and of course, `wheel`
* let the standard build operations kick in with a call to `snapcraftctl build`: at that point the `--upgrade` flag will be appended to the standard virtual environment creation, reusing our crafted venv instead of creating a new one.
This approach has also the advantage to invoke `pipstrap.py` as it is done for the other deployable artifacts, and for the PR validations, reducing risks of shifts between the various deployment methods.
Fixes#7717
This PR adds a `--dns-server` option to the `run_acme_server` test tool, in order to provide an arbitrary DNS server to Pebble or Boulder for the integration tests.
I also take this occasion to make `run_acme_server` a real CLI tool using argparse, and set the `--server-type` (default `pebble`) option as well.
* Set --dns-server flag in run_acme_server
* Default to pebble
* Add documentation
* Configure also Boulder
Do we have any specific reason to run the standard Linux integration tests on Python 2.7?
If not, we should move to a more recent version of Python. This PR does it for Python 3.8.
Fixes#8365
This PR adds a control when `certbot certonly` or `certbot run` are called for a certificate that already exists and would eventually be replaced. As described in #8365, this control is here to ensure that the user will not modify the key type of their certificate (eg. ECDSA to RSA) without an explicit approval (set explicitly `--cert-name` and `--key-type`), since RSA is the default if not specified.
* Handle unexpected key type migration.
* Update certbot-ci/certbot_integration_tests/certbot_tests/test_main.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8355
During the troubleshooting of #8355, I came to the conclusion that using buildkit was creating the problem. Without it all docker images are built correctly. Initially buildkit was enabled to avoid a building problem in Azure Pipeline, but I also found in my recent tests that this problem was not there anymore.
You can find more details about the troubleshooting and reasoning in #8355.
As a consequence, I disable the usage of buildkit in this PR which will solve the issue.
Fixes#8202
This PR adds an Azure Pipeline job to execute certbot plugins --prepare for each Docker image created during the CI on amd64.
* Prepare basic integration tests for certbot dockers
* Add a displayName for the integration tests task
Partial fix for #8280
This PR refactors the bash script wrapper for snap (`/certbot.wrapper`) into certbot python codebase. Here are the keypoints of this refactoring:
* the wrapping is applied when `main` function from `certbot._internal.main` is called if environment variable `CERTBOT_SNAPPED` is `True`, which is set during the snap build
* the initial bash script wrapper is removed, simplifying `snap/snapcraft.yaml` by removing the `certbot.wrapper` part
* the dependency to `curl` and `jq` binaries are removed
* the failure during requesting the snapd socket is correctly handled, and displays an informative message in order to correct the situation, as required by #8280
One side note about the modifications done to `app.certbot.command` in `snapcraft.yaml`. Normally calling `bin/certbot` should be sufficient and it is effectively under a normal situation (`core` snap up-to-date). However in the same situation than when the problem occurs in #8280, using `bin/certbot` makes the snap raise an exception about `certbot.main` module that cannot be found.
It seems that when `core` snap is not up-to-date (in Debian for instance with default `snapd` installation), the shebang `/usr/bin/env python3` in the `bin/certbot` wrapper is wrongly resolved to the host Python, instead of the snap Python. It is working as expected if `core` snap is up-to-date. One way to fix that is to keep a bash script wrapper, because in this case, it is the `PATH` value that matters to resolve the Python interpreter, and `PATH` is correctly set up to resolve it from the snap first.
However to keep the simplification provided by the wrapper removal, I prefered to use `bin/python3 $SNAP/bin/certbot` as `command` to explicitly target the correct Python interpreter. Again normally it is not needed because everything is working correctly with a `core` snap up-to-date, but since the root purpose of all of this is to target bad situations, well, it is better to have a snap that is effectively able to start to display the informative message...
* Refactor the bash wrapper for snap execution as Python code into certbot
* Remove wrapper, finalize the python logic
* Organize code
* Improve error handling
* Update command
* Setup basic certbot logging before running the snap prepare logic
* Improve instructions
* Use logging facility
* Handle properly an exception in snap_config
* Use the python script call approach
* Update instructions to keep sync with https://github.com/certbot/website/pull/650
Partial fix for #8256
This PR makes tox calls pipstrap before any commands is executed, and Azure Pipelines calls pipstrap when appropriate (when an actual call to pip is done).
* Invoke pipstrap in tox and during the CI
* Set default value for PYTHON_VERSION and always set python interpreter
* Set Python for snaps_build also
* Fix the build for Windows installer
* Add a warning comment for pinned versions in pipstrap
* Rebuild letsencrypt-auto
* Same version than the installer build
* Let's update to latest pip for installer tests
Fixes#7585
This PR removes the specific configuration to configure the test runner included in `setuptools` to use pytest, the deprecated parameters related to setuptools testing in `setup.py`, and update the packaging guide to use `python -m pytest` instead of `python setup.py test`.
The farm test `test_sdist.sh` is also updated to use directly pytest. This test is designed to reproduce the steps used by OS integrators when they package `certbot`, and ensure that we are not breaking something that will impact their work. We discussed with integrators from RHEL/CentOS and Debian, and they are fine with us testing sdist directly with pytest.
One execution of the `test_sdist.sh` farm test with the modifications made by this PR can be seen here: https://dev.azure.com/certbot/certbot/_build/results?buildId=2606&view=results
* Remove setuptools deprecated features about testing
* Updating packaging guide
* Add changelog entry
Fixes#8252
With @bmw we digged quite a lot on why the failure happens on ARM snap, and here we what we understood:
* the failure occurs since the version 50 of setuptools is available
* normally, we should not be impacted because the setuptools version used in the snap build should be the one installed by the `core20` base snap, because the build occurs in a `venv` created with `--system-site-packages`
* BUT associated with the build isolation provided by recent versions of pip (to implement PEP 517), a bad interaction happens: following the definition of the build system provided by `cryptography`, pip installs the most recent version of setuptools on a separate path for the build (because `cryptography` just asks for a minimal version of `setuptools`), then features of this version conflict with the old version of `setuptools` initially present
* the exact interaction is described here: https://github.com/pypa/pip/issues/6264#issuecomment-685230919. Basically the new version of `setuptools` triggers some hacks, that are then applied at runtime on the old version of `setuptools` that is also still available in `sys.path` at this point, and breaks the build.
To fix that, one can disable the isolation build on cryptography, by passing `PIP_NO_ISOLATION_BUILD=no` to pip. It is the purpose of this PR.
This will have the consequence to not be PEP 517 compliant: if needed the `cryptography` library will be built using the `setuptools` available in the system. In general I think it makes sense for the snap build purpose, since we control precisely the build environment, and makes consistent build that will not be broken by a new version of a build system if library maintainers did not provide a strict version of it in their build requirements. However we need now to take care about having a compatible build system for all libraries that may have specific requirements in their build system using the PEP 517 definition in `pyproject.toml`.
I think as of now that it is a safe move if we keep using the most recent version of `setuptools` available in Ubuntu 20.04, and it is the case here for snap builds. It may however be problematic if some libraries require another build system than `setuptools` and do not provide a fallback to a `setuptools` build. For the record, `dns-lexicon`, that I maintain, uses `poetry` and so a PEP 517 compliant definition of a build system, but provides also this fallback (https://github.com/AnalogJ/lexicon/blob/master/setup.py).
Full test suite compiling the snaps for the 3 architectures using this PR is available here: https://dev.azure.com/certbot/certbot/_build/results?buildId=2596&view=results
Fixes#8169
This PR improves snaps remote builds script by dumping the output of `snapcraft remote-build` when unexpected behavior is detected:
* when all builds for a project finish with a zero status code, and none of them are marked as failed, we expect to have all the associated snap files available locally.
* when some builds are marked as failed, we expect to have a build output for each of them available locally.
In these two situations, if the expectation are not matched, then the script will display the output of `snapcraft remote-build` itself. I added also a control error to handle nicely the absence of an expected build output on the local machine.
* Improve log dump in snaps remote builds when an unexpected behavior is detected
* Use the manager
* Update tools/snap/build_remote.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Snapcraft has a feature name `remote-build`. It allows to compile snaps using the Canonical dedicated build architecture for several architectures. Compared to the QEMU-enabled Docker approach used currently, the remote build has several advantages:
* the builds are done on the native architecture, making them basically faster than what can be achieved on QEMU
* it avoids to depend on `adferrand/snapcraft` (which could be otherwise be fixed with the merge of https://github.com/snapcore/snapcraft/pull/3144, but this will not happen in the short term)
* when everything is good, all snaps build can be run in parallel and then can be orchestrated by one single Azure Pipeline job, since the heavy tasks are done remotely.
This PR makes the necessary ajustements to use the remote build feature instead of the QEMU-enabled docker approach.
One complex task was to be able to compile the `certbot` snap on `arm64` and `armhf`. Indeed on these architectures the pre-compiled wheel for `cffi` is not available. So it needs to be compiled during the snap build. Sadly, the current version of the python plugin in snapcraft is limited by the fact that `wheels` is not installed in the virtual environment set up to build the python packages, and there is no easy way to change that except by overridding the whole build process.
In the long term, I think I will open a PR on `snapcraft` Git repository to provide a consistent solution. But for the short term, I used the possibility to provide arguments to the `venv` module, to add the flag `--system-site-packages`. With it, the virtual environment can use the system site package, where `wheel` is available.
The other significant additions are in `tools/snap/build_remote.py` script. If invoking the remote build on a local machine is quite straight-forward, it is another story on the CI because we need build auditability and resiliency during these non-interactive actions. In particular we should avoid as possible inconsistent results on the nightly pipeline and the release pipeline.
So this script wraps the `snapcraft` call into a retry logic, and improves its logs in the context of parallel builds.
For the minor modifications, it is mainly about ensuring that plugins can be built (some of them also need `cffi` for instance), and simplify the Azure Pipeline since all snaps are retrieved in one go.
Please note that the `test-` branches still run only the `amd64` architecture. Indeed I noticed that builds on `arm64` and `armhf` are tending to be very slow to start (up to 40 min) while the `amd64` ones wait at max 10 mins, and usually 30 seconds only when the overall load on Canonical side is low.
To work on `certbot/certbot` repository, one secured file needs to be added, because `snapcraft` needs to be authenticated against Launchpad with credentials allowing remote builds. To do so, from a local machine that have this capability, one can extract the existing file at `$HOME/.local/share/snapcraft/provider/launchpad/credentials`, and register it as a secured file in Azure Pipeline with the name `snapcraftRemoteBuildCredentials`.
* Define scripts
* Setup pipeline to use remote builds
* Focus on packaging builds
* Set credentials
* Setup git
* Launch all builds in parallel
* Add dev dependencies to build cffi and cryptography
* Convert to a python logic
* Reorganize the pipeline
* Handle the fact that snap builds may be taken from cache
* Generate constraints
* Exit code
* Check existence
* Try to handle better non zero exit code
* Add --system-site-packages to get wheel in the venv
* Add executable permissions
* Troubleshoot
* Dynamic display, take the maximum timeout for snap build job
* Allow retries if the remote build does not start
* Trigger only amd64 builds for test branches
* Exit properly
* Update snapcraft.yaml
* Fix snap run
* Set secured file name
* Update .azure-pipelines/templates/jobs/packaging-jobs.yml
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update .azure-pipelines/templates/jobs/packaging-jobs.yml
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update .azure-pipelines/templates/jobs/packaging-jobs.yml
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Move order in deps
* Reactivate all builds
* Use Manager() as a context manager
* Use Pool as a context manager
* Some nice refactorings
* Check snapcraft execution interruption with exit codes
* Use f-string and format expressions
* Start log
* Consistent use of single/double quotes
* Better loop to extract lines
* Retry on build failures
* Few optimizations
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#8149
This PR adds warnings to warn about the incoming deprecation of Python 3.5 in Certbot.
* Add warnings about Python 3.5 deprecation in Certbot
* Update certbot/certbot/__init__.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Fixes#4351
This PR proposes a solution to use the third party plugins with the prefix `pip_package_name:` in the plugin name, plugin specific flags and keys in dns plugin credential files.
A first solution has been proposed in #6372, and a more advanced one in #7026. In #7026 was also added a deprecation warning when the old plugin name `pip_package_name:plugin_name` was used.
However there were some limitations with #7026, in particular the fact that existing flags of type `pip_package_name:dns_plugin_option` or keys like `pip_package_name:key` in dns plugin credential files were not read anymore. This would have led to silent failures during renewals if the configuration was not explicitly updated by the user.
I tried to fix that based on #7026, but the changes needed are complex, and create new problems on their own, like unexpected erasure of values in the renewal configurations.
Instead I try in this PR a new approach: the `PluginsRegistry` in `certbot._internal.plugins.disco` module register two plugins for a given entrypoint refering to a third party plugin when `find_all()` is called:
* one plugin with the name `plugin_name`
* one plugin with the name `pip_package_name:plugin_name` (like before)
This way, every existing configuration continues to work without any change (credentials, renewal configuration, CLI flags). And new configurations can refer to the new plugin name without prefix, and use the approriate CLI flags, credentials without this prefix.
On top of it I added the deprecation path given in #7026 (thanks @coldfix!):
* the plugin named `pip_package_name:plugin_name` is hidden from `certbot plugins` output
* the help for this plugin is still displayed, and a deprecation warning is displayed in the description
* when invoked, the same deprecation warning is displayed in the terminal
* Support both prefixed and not prefix third party plugins
* Adapt tests
* Add deprecation path
* Named parameters
* Add deprecation warning in CLI
* Add a changelog
Fixes#8041
This PR makes Azure Pipeline build the DNS plugins snaps for the 3 architectures during the CI.
It leverages the existing logic for building the Certbot snap in order to deploy a QEMU environment with Docker, and leverages the local PyPI index to speed up the build when installing `cffi` and `cryptography`.
All DNS plugins snaps are constructed in one unique docker container, in order to save the time required to install the system dependencies upon first start of `snapcraft`, and so speed up significantly the build.
Finally, all `amd64` DNS plugins snaps are built within 6 minutes. For `arm64` and `armhf`, it is around 40 mins: this is quite fast in fact, considering that 14 DNS plugins snaps are built.
However, this is still an extremely heavy task to make the full 3 architectures builds, even for Azure Pipelines and its 10 parallel jobs capability. That is why I make the `arm64` and `armhf` builds be skipped for the `full-test-suite`, and let them run only for `nightly` and `release`. This means however that these builds will not be done for the release branches. If this is a problem, I can put a more elaborate suspend condition to triggers the builds in this case.
All snaps are stored in the pipeline artifacts storage, making them available for publication during a `release` pipeline.
The PR is set as Draft for now, because I use temporarily `pr_test-suite` to validate the packaging jobs when commits are pushed. Once the PR is ready, I will revert it back to the normal configuration (run the standard tests).
* Configure a script to build DNS snaps
* Focus on packaging
* Trigger all architectures
* Add extra index
* Prepare conditional suspend
* Set final suspend logic
* Set final suspend value
* Loop for publication
* Use python3
* Clean before build
* Add a test
* Add test job in Azure
* Preserve env
* Apply normal config for pipelines
* Skip QEMU jobs only for test branches
* Makes snap run tests depends also on the Certbot snap build
* Update .azure-pipelines/templates/jobs/packaging-jobs.yml
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update .azure-pipelines/templates/stages/deploy-stage.yml
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* More accurate way to get the plugin snap name
* Integrate DNS snap tests into certbot-ci
* Fixes
* Update certbot-ci/snap_integration_tests/conftest.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update certbot-ci/snap_integration_tests/conftest.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Clean an _init_.py file
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Short PR to improve some things during snap builds:
* cleanup snapcraft assets before a build, in order to avoid some weird errors when two builds are executed consecutively without cleanup
* use python3 explicitly in `tools/simple_http_server.py` because on several recent distributions, `python` binary is not exposed anymore, only `python2` or `python3`.
Fixes#8071 and fixes https://github.com/certbot/certbot/issues/8110.
This PR migrates every job from Travis in Azure Pipeline.
This PR essentially converts the Travis jobs into Azure Pipeline with a complete iso-fonctionality (or I made a mistake). The jobs are added in the relevant existing pipelines (`main`, `nightly`, `advanced-test`, `release`). A global refactoring thanks to the templating system is done to reduce greatly the verbosity of the pipeline descriptions.
A specific feature (not present in Travis) is added: the stage `On_Failure`. Using directly the Mattermost API, it allows to notify pipeline failure in a Mattermost channel with a link to the failed pipelines without the need to authenticate to Microsoft.
See https://github.com/certbot/certbot/pull/8098#issuecomment-649873641 for the post merge actions to do at the end of this work.
Fixes#7957
This PR makes snapcraft generate itself the dependencies constraints file during the snap build, instead of having an external script that does it before calling snapcraft.
* Base logic
* Various controls when email is None
* Adapt eff tests
* Forward compatibility
* Also for csr
* Explicit regr or meta updates in account objects
* Adapt logic to ask for eff subscription during registering
* Adapt tests
* Move dry-run control
* Add some relevant controls on handle_subscription call checks
This PR proposes a way to build the certbot snap for several architectures, using a QEMU-base emulation approach, and several optimizations to keep the build time of each snap below 20 minutes.
Most of the reasoning for the approach proposed here is described in the original PR: https://github.com/basak/certbot-snap-build/pull/27
On top of it, I added a docker pull to a pre-compiled snapcraft docker, instead of compiling it during the Travis pipeline, in order to save 5 to 7 minutes more on each snap build. The snap images are compiled and stored here: https://hub.docker.com/repository/docker/adferrand/snapcraft. Depending on the time the PR will be reviewed, we can:
* continue to use `adferrand/snapcraft`
* move its logic to certbot scope and use something like `certbot/snapcraft`
* wait for https://github.com/snapcore/snapcraft/pull/3144 to be merged, and use `snapcore/snapcraft`.
* Backport https://github.com/basak/certbot-snap-build/pull/27 into Certbot project
* Fix build deps
* Integrate proactively #8012 to fix builds on non-amd64 archs
* Configure jobs on Travis
* Focus on snap builds. Disable temporarily some jobs. Disable deploy actions by security.
* Specify TARGET_ARCH during snap build
* Do not do anything if TOXENV is not set
* Various optimizations
* Use recent version of ubuntu for get correct features on snap out of the box
* Add up to date wheels
* Organizing scripts
* Set dest dir
* Get back original configuration for Travis
* Add comments
* Update common_libs.sh
* Use adferrand/snapcraft
* Test build
* Stable snapcraft
* Update build_and_install.sh
* Move back snap builds to the cron/release pipeline
* Update snap/local/compile_native_wheels.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update snap/local/compile_native_wheels.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update snap/local/compile_native_wheels.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Update snap/local/build_and_install.sh
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Enable i386 builds, various optimizations
* Update dependencies
* Configure a simple http server to serve the pre compiled wheels
* Fix wheels compilation
* Relax permissions
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
This PR gets its root from an observation I did on current version of Certbot (1.3.0): the `renewal-hooks` directory in Certbot configuration directory is created on Windows with write permissions to everybody.
I thought it was a critical bug since this directory contains hooks that are executed by Certbot, and you certainly do not want this folder to be open to any malicious hook that could be inserted by everyone, then executed with administrator privileges by Certbot.
Turns out for this specific problem that the bug is not critical for the hooks, because the scripts are expected to be in subdirectories of `renewal-hooks` (namely `pre`, `post` and `deploy`), and these subdirectories have proper permissions because we set them explicitly when Certbot is starting.
Still, there is a divergence here between Linux and Windows: on Linux all Certbot directories without explicit permissions have at maximum `0o755` permissions by default, while on Windows it is a `0o777` equivalent. It is not an immediate security risk, but it is definitly error-prone, not expected, and so a potential breach in the future if we forget about it.
Root cause is that umask is not existing in Windows. Indeed under Linux the umask defines the default permissions when you create a file or a directory. Python takes that into account, with an API for `os.open` and `os.mkdir` that expose a `mode` parameter with default value of `0o777`. In practice it is never `0o777` (either you the the `mode` explictly or left the default one) because the effective mode is masked by the current umask value in the system: on Linux it is `0o022`, so files/directories have a maximum mode of `0o755` if you did not set the umask explicitly, and it is what it is observed for Certbot.
However on Windows, the `mode` value passed (and got from default) to the `open` and `mkdir` of `certbot.compat.filesystem` module is taken verbatim, since umask does not exit, and then is used to calculate the DACL of the newly created file/directory. So if the mode is not set explicitly, we end up with files and directories with `0o777` permissions.
This PR fixes this problem by implementing a umask behavior in the `certbot.compat.filesystem` module, that will be applied to any file or directory created by Certbot since we forbid to use the `os` module directly.
The implementation is quite straight-forward. For Linux the behavior is not changed. On Windows a `mask` parameter is added to the function that calculates the DACL, to be invoked appropriately when file or directory are created. The actual value of the mask is taken from an internal class of the `filesystem` module: its default value is `0o755` to match default umasks on Linux, and can be changed with the new method `umask` that have the same behavior than the original `os.umask`. Of course `os.umask` becomes a forbidden function and `filesystem.umask` must be used instead.
Existing code that is impacted have been updated, and new unit tests are created for this new function.
* Implement umask for Windows
* Set umask at the beginning of tests
* Fix lint, update local oldest requirements
* Update certbot-apache/setup.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Improve tests
* Adapt filesystem.makedirs for Windows
* Fix
* Update certbot-apache/setup.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Changelog entries
* Fix lint
* Update certbot/CHANGELOG.md
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
This PR upgrades Certbot pinned dependencies through `letsencrypt-auto-source/rebuild_dependencies.py` while taking into account the problems detected in https://github.com/certbot/certbot/pull/8035:
* `cryptography` is pinned to `2.8` to continue to support OpenSSL 1.0.1 on non-x86 ancient Linux distributions (RHEL 6 + Debian 8)
* `parsedatetime` is pinned to `2.5` because of an incompatibility with Python 2.7 (see https://github.com/bear/parsedatetime/issues/246)
* `letsencrypt-auto-source/rebuild_dependencies.py` now takes into account the environment markers that are aded to `AUTHORITATIVE_CONSTRAINTS`: this is used for the `enum34` dependency, to not install it on Python 3.6+ and not break the distribution by swapping the built-in `enum` module during the setup of Certbot venv.
Fixes#8030
* Pin cryptography and parsedatetime
* Upgrade dependencies
* Remove authoritative constraint
* Upgrade dependencies
* Rebuild certbot-auto
* Update letsencrypt-auto-source/rebuild_dependencies.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
* Honor specific requirements in the AUTHORITATIVE_CONSTRAINTS
* Fix injection
* Update dependencies
* Update rebuild_dependencies.py
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
