remove CERTBOT_NO_PIN (#9634)

Adrien and I added this is in https://github.com/certbot/certbot/pull/6590 in response to https://github.com/certbot/certbot/issues/6582 which I wrote. I now personally think these tests are way more trouble than they're worth.

In almost all cases, the versions pinned in `tools/requirements.txt` are used. The two exceptions to this that come to mind are users using OS packages and pip. In the former, the version of our dependencies is picked by the OS and do not change much on most systems. As for pip, [we only "support it on a best effort basis"](https://eff-certbot.readthedocs.io/en/stable/install.html#alternative-2-pip).

Even for pip users, I'm not convinced this buys us much other than frequent test failures. We have our tests configured to error on all Python warnings and [we regularly update `tools/requirements.txt`](https://github.com/certbot/certbot/commits/master/tools/requirements.txt). Due to that, assuming our dependencies follow normal conventions, we should have a chance to fix things in response to planned API changes long before they make their way to our users. I do not think it is necessary for our tests to break immediately after an API is deprecated.

I think almost all other failures due to these tests are caused by upstream bugs. In my experience, almost all of them will sort themselves out pretty quickly. I think that responding to those that are not or planned API changes we somehow missed can be addressed when `tools/requirements.txt` is updated or when someone opens an issue. I personally don't think blocking releases or causing our nightly tests to fail is at all worth it here. I think removing this frequent cause of test failures makes things just a little bit easier for Certbot devs without costing us much of anything.
This commit is contained in:
Brad Warren
2023-03-27 17:01:27 -07:00
committed by GitHub
parent f004383582
commit 208ef4eb94
6 changed files with 22 additions and 49 deletions
+9 -16
View File
@@ -585,27 +585,20 @@ include our snaps, Docker images, Windows installer, CI, and our development
environments.
In most cases, the file where dependency versions are specified is
``tools/requirements.txt``. There are two exceptions to this. The first is our
"oldest" tests where ``tools/oldest_constraints.txt`` is used instead. The
purpose of the "oldest" tests is to ensure Certbot continues to work with the
oldest versions of our dependencies which we claim to support. The oldest
versions of the dependencies we support should also be declared in our setup.py
files to communicate this information to our users.
The second exception to using ``tools/requirements.txt`` is in our unpinned
tests. As of writing this, there is one test we run nightly in CI where we
leave Certbot's dependencies unpinned. The thinking behind this test is to help
us learn about breaking changes in our dependencies so that we can respond
accordingly.
``tools/requirements.txt``. The one exception to this is our "oldest" tests
where ``tools/oldest_constraints.txt`` is used instead. The purpose of the
"oldest" tests is to ensure Certbot continues to work with the oldest versions
of our dependencies which we claim to support. The oldest versions of the
dependencies we support should also be declared in our setup.py files to
communicate this information to our users.
The choices of whether Certbot's dependencies are pinned and what file is used
if they are should be automatically handled for you most of the time by
Certbot's tooling. The way it works though is ``tools/pip_install.py`` (which
many of our other tools build on) checks for the presence of environment
variables. If ``CERTBOT_NO_PIN`` is set to 1, Certbot's dependencies will not
be pinned. If that variable is not set and ``CERTBOT_OLDEST`` is set to 1,
``tools/oldest_constraints.txt`` will be used as constraints for ``pip``.
Otherwise, ``tools/requirements.txt`` is used as constraints.
variables. If ``CERTBOT_OLDEST`` is set to 1, ``tools/oldest_constraints.txt``
will be used as constraints for ``pip``, otherwise, ``tools/requirements.txt``
is used as constraints.
Updating dependency versions
----------------------------