i wanted this for testing
https://github.com/certbot/certbot/issues/10190
alex started working on this in
https://github.com/certbot/certbot/pull/9207 years ago, but pebble
didn't end up doing a release containing his work while he was still
regularly contributing to certbot. this has now changed though
before this PR, our integration tests only worked on amd64 linux
systems. with this PR, i've successfully run our integration tests on
all combinations of the architectures amd64 and arm64 and the OSes linux
and macos
---------
Co-authored-by: ohemorange <erica@eff.org>
in https://github.com/certbot/certbot/pull/9124 we had the problem of
certbot-nginx's `Addr.fromstring` method possibly returning None which
is not possible in the `Addr` method in the certbot base class or in
certbot-apache. we fixed this by telling mypy the common
`Addr.fromstring` method returns an `Optional[Addr]` (despite it
actually always returning an `Addr`) and then unnecessarily complicating
certbot-apache's code a bit. the need for extra complexity with this
approach is going even further in
https://github.com/certbot/certbot/pull/10151 where we have to use
`cast` to assure mypy that the type isn't actually `Optional`. i
personally don't like all this
fixes https://github.com/certbot/certbot/issues/10131
this seems simple enough, but i also requested alex's review as a quick
sanity check if he doesn't mind providing one
i've verified this fixes the problem and that PKCS#8 was used in certbot
3.0.1
fixes#10105
this PR updates our minimally required cryptography and pyopenssl
versions as well as updating our policy for choosing minimum dependency
versions generally
before this PR, we were trying to keep compatibility with packages
available in EPEL 9 using the python 3 version available in RHEL 9.
after the discussion in #10105 we decided not to do this anymore
because:
* EPEL 9 may not want to update to certbot 3.0+ anyway because of our
backwards incompatible changes from certbot 2.x
* RHEL 9 appstream repos now contain newer versions of many of our
dependencies for newer versions of python
* alternate installation methods for RHEL 9 based users including our
snaps and pip are available
on a call we then discussed what distro repositories we should track
instead of EPEL 9. our docs previously said Debian sid/unstable, but we
felt this as unnecessary because Debian sid can and does change very
quickly. if we wanted a new dependency there, Debian could probably
accommodate it
we also considered RHEL 10 + EPEL 10, however, these repos are not even
stable yet and certbot and many of its dependencies are not yet packaged
there at all
for these reasons, plus many of the reasons we decided to upgrade past
EPEL 9 with the default python 3 version there, we decided that at least
for now, we will remove any linux distro considerations when choosing
minimal dependency versions of certbot
as i wrote in the contributing docs, we may choose to reconsider this
plan if there are requests for us to do so, but based on the information
above, we are not sure this will ever even happen and removing this
constraint significantly simplifies development of certbot
this hopefully at least helps the problem hit at
https://github.com/certbot/certbot/pull/10126#discussion_r1909714276
i took this approach because in my experience, linux specific shell
commands have crept into our scripts repeatedly over the years so i
think just having macOS devs use the linux versions is much more
reliable. it's what i've personally been doing for years now
when reviewing https://github.com/certbot/certbot/pull/10126 and running
`tools/pinning/oldest/repin.sh` using a freshly created dev environment,
i was repeatedly given the message
> The "poetry.dev-dependencies" section is deprecated and will be
removed in a future version. Use "poetry.group.dev.dependencies"
instead.
i believe this section was generated automatically by poetry's tooling
when it created the initial boilerplate file for us, but we don't use
it, so i just deleted the section which makes the warnings disappear
this PR hopefully improves two things that i hit while working on #10035
1) i found that repinning our dependencies took ~6 minutes!
digging into it a bit, the biggest culprit i found was the inclusion of
`--no-cache-dir` here which seemed to cause poetry to redownload the
same packages over and over in a single run. this comes from
https://github.com/certbot/certbot/pull/9453 which fixed a problem i
(but not alex) was having with a major performance penalty. i removed
the flag here and instead included instructions on clearing poetry's
caches in case anyone ever hits this in the future. with this change,
the script now takes about 40 seconds on my laptop
2) every run of this script ended with the output:
```
Warning: poetry-plugin-export will not be installed by default in a
future version of Poetry.
In order to avoid a breaking change and make your automation
forward-compatible, please install poetry-plugin-export.
explicitly. See https://python-poetry.org/docs/plugins/#using-plugins
for details on how to install a plugin.
To disable this warning run 'poetry config warnings.export false'.
```
setting `POETRY_WARNINGS_EXPORT=false` fixes this which i believe is
safe to do because of
https://github.com/certbot/certbot/blob/2c8609464c3e371053f78e515fc7beb26446f5a3/certbot/setup.py#L53-L56
i hit this when working on https://github.com/certbot/certbot/pull/10076
where i found that updating all our dependencies no longer worked
because of new deprecations in pyopenssl. this pr fixes that
This is another and very minor piece of https://github.com/certbot/certbot/issues/9988.
We've done nothing to warn/migrate installations using the old `certbot-route53:auth` plugin name and installations like that still exist according to https://gist.github.com/bmw/aceb69020dceee50ba827ec17b22e08a. We could try to warn/migrate these users for a future release or decide it's niche enough that we'll just let it break, but I think it's easy enough to keep the simple shim around.
This PR just moves the code raising a deprecation warning into `_internal` as part of cleaning up all deprecation warnings I found in https://github.com/certbot/certbot/issues/9988. I manually tested this with a Certbot config using the `certbot-route53:auth` plugin name and renewal worked just fine.
The `gnupg` package from Homebrew only installs a `gpg` binary, not a `gpg2` binary. I had previously worked around this by manually creating an alias, but I think we can do better.
GPG version 1 is ancient and [hasn't seen a release since 2006](https://gnupg.org/download/release_notes.html). Additionally, `gpg` has referred to GPG 2 in Ubuntu since at least 20.04 which is the oldest non-EOL'd version as of writing this so I think this change is safe to make.
while working on https://github.com/certbot/certbot/issues/9938, i updated our dependencies which updated mypy introducing new errors that mypy wanted me to fix. i think this makes the regularly necessary process of updating our dependencies too tedious and we should instead pin our linters that do this to a specific version and update them manually as desired. we already do this with pylint in the lines above my changes in this PR for the same reason
* Set the delegated field in Lexicon config to bypass subdomain resolution (#9821)
The Lexicon-based DNS plugins use a mechanism to determine which actual segment of the input domain is actually the DNS zone in which the DNS-01 challenge has to be initiated (eg. `subdomain.domain.com` or `domain.com` for input `subdomain.domain.com`): they tries recursively to configure Lexicon and initiate authentication from the most specific to most generic domain segment, and select the first segment where Lexicon stop erroring out.
This mechanism broke with #9746 because now the plugins call Lexicon client instead of the underlying providers, and the client makes guess on the actual domain requested. Typically for `subdomain.domain.com` it will actually try to authenticate against `domain.com`, and so the mechanism above does not work anymore.
This PR fixes the issue by using the `delegated` field in Lexicon config each time the plugin needs it. This field is designed for this kind of purpose: it will instruct Lexicon what is the actual DNS zone domain instead of guessing it.
I tested the change with one of my OVH account. The expected behavior is re-established and the plugin is able to test `subdomain.domain.com` then `domain.com` as before.
Fixes#9791Fixes#9818
(cherry picked from commit cf4f07d17e)
* add changelog entry for 9821 (#9822)
(cherry picked from commit 7bb85f8440)
---------
Co-authored-by: Adrien Ferrand <adferrand@users.noreply.github.com>
* helpful: fix handling of abbreviated ConfigArgparse arguments (#9796)
* helpful: fix handling of abbreviated ConfigArgparse arguments
ConfigArgparse allows for "abbreviated" arguments, i.e. just the prefix
of an argument, but it doesn't set the argument sources in these cases.
This commit checks for those cases and sets the sources appropriately.
* failing to find an action raises an error instead of logging
* Update changelog
* Add handling for short arguments, fix equals sign handling
These were silently being dropped before, possibly leading to instances
of `NamespaceConfig.set_by_user()` returning false negatives.
(cherry picked from commit 11e17ef77b)
* Fix finish_release.py (#9800)
* response is value
* rename vars
(cherry picked from commit a96fb4b6ce)
* Merge pull request #9762 from certbot/docs/yaml-config
Add YAML files for Readthedocs requirements
(cherry picked from commit 44046c70c3)
* Update Lexicon requirements to stabilize certbot-dns-ovh behavior (#9802)
* Update minimum Lexicon version required for certbot-dns-ovh
* Add types
* FIx mypy
* Fix lint
* Fix BOTH lint and mypy
(cherry picked from commit 5cf5f36f19)
* simplify code (#9807)
(cherry picked from commit 6f7b5ab1cd)
* Include linting fixes from 8a95c03
---------
Co-authored-by: Will Greenberg <willg@eff.org>
Co-authored-by: Alexis <alexis@eff.org>
Co-authored-by: Adrien Ferrand <adferrand@users.noreply.github.com>
* update quickstart and remove os import
* simplify theme use
* list sphinx_rtd_theme as extension
Our docs builds failed last night, presumably because #9754 updated `sphinx_rtd_theme` which changed some unknown thing.
Looking into it, our usage of this project was very unconventional. Following the code comment I deleted in this PR to https://docs.readthedocs.io/en/stable/faq.html#i-want-to-use-the-read-the-docs-theme-locally, simple instructions are given to put the following in your `conf.py` file:
```
extensions = [
...
'sphinx_rtd_theme',
]
html_theme = "sphinx_rtd_theme"
```
I did this instead of the more complicated logic we were using and all builds passed locally. I also triggered a build on readthedocs with these changes which also passed.
This takes care of the dependabot alerts those with access can see at https://github.com/certbot/certbot/security/dependabot.
Pinning back `cython` is needed because without it, our full test suite will fail when trying to build `pyyaml` on ARM systems.
In addition to the speed improvements in CI, the speed improvements locally with both this https://github.com/certbot/certbot/pull/9666 which this builds on is even more significant. After it's been run once so it's had a chance to set up the different virtual environments, `tox` locally now takes 39 seconds on my laptop when it used to take 137 seconds.
* add some missing types
* install pkg-config
* install pkg-config for docker too
* add pkg-config to plugins
* pkg-config when cryptography may need to be built
* deps cleanup
* more comments
* more tweaks
Fixes https://github.com/certbot/certbot/issues/7921.
In all cases when we run `pip_install.py`, we first run `pipstrap.py`. This PR combines these two steps for convenience and to make always doing that less error prone. This will also help me with some of the `tox.ini` refactoring I'm planning to do.
I ran the full test suite on everything and tested the release script changes locally.
This change shouldn't have any effect on cryptography's setup because they install `certbot[test]` which depends on pip, setuptools, and wheel.
* always pipstrap
* use pip_install.py during releases
