This continues from the work of @sydneyli in PR #6671
I didn't do much here. Basically added support for reading data from sys.stdin to both tools/merge_requirements.py and tools/strip_hashes.py as well as support for reading files from paths passed as cli parameters to strip_hashes.py.
Reading the filepaths was not strictly required, but I thought would be a good thing to do in order to keep the tooling usage options consistent.
Fixes#6581
* Generate constraints file to pin deps in Docker images
Dockerfiles pin versions using constraints file
Pulling out strip_hashes and add --no-deps flag
* Add stdin option for merge_requirements
Add stdin and file path support to strip_hashes
* Address review comments
* Mock set_by_cli in _test_relevant_values_common.
* Empty commit
* Revert "Mock set_by_cli in _test_relevant_values_common."
This reverts commit 9dfec8dfa9.
* mock less
* Use plugin_common code instead of reimplementing.
* Revert 2nd implementation.
* Simplify certbot.storage.relevant_values() tests.
In addition to cleaning up the code a bit, it also removes the problems we've
seen in these tests with the global state used in cli.py.
I think this is causing failures in some of our tests so this PR reverts the change until we can fix the problem.
The 2nd commit is to keep the change using more idiomatic wording in the changelog for another change that got included in this PR.
* Revert "Use built-in support for OCSP in cryptography >= 2.5 (#6603)"
This reverts commit 2ddaf3db04.
* keep changelog correction
We always run a full set of CI tests before beginning the release process. The way this would work previously is we would either trigger tests on the `test-everything` branch to run through Travis' web UI or if it was a point release, create a new branch based on `test-everything` but modify `.travis.yml` so the branch that was pulled in to be tested was the point release branch instead of `master`.
This no longer works because the former `test-everything` tests are now only run when Travis automatically runs our tests nightly.
We could create and maintain a separate branch for the purpose of manually running all tests or remove the conditionals from the latest `.travis.yml` file every time before we want to run these tests, but there must be A Better Way™.
This PR makes the change that in addition to running all tests nightly, they would also run on pushes to tested branches other than master. These changes do not affect the tests run on PRs or on commits to `master`.
What is affected is commits to point release branches and branches named `test-*`. (See [.travis.yml](https://github.com/certbot/certbot/blob/2ddaf3db043ea8526ae3f9ab2ef120b194b2e506/.travis.yml#L177) for what branches we run tests on.) Running all tests on point release branches automates the step of running our full test suite before doing a point release.
The changes to `test-*` could be a mixed bag, however, since we switched to travis-ci.com over 3 weeks ago, I'm the only one who has used this functionality and I personally prefer things this way. At the very least, since these branches don't seem to be widely used, I think we can make this change and reevaluate if it becomes a problem.
* Test all on push events to non-master branches.
* Move branches section up.
* expand comment
It was pointed out to me that you can no longer run tox.cover.py directly to run coverage tests on a subset of the packages in this repo.
This happened after we did both of:
1. Factored out --pyargs from the different test files and put it in pytest.ini.
2. Moved the options we added to pytest.ini to tox.ini meaning that --pyargs is not set unless you run the file through tox.
I think the fact that we factored out --pyargs from the files that needed it was a mistake. --pytest is needed by tox.cover.py and install_and_test.py in order to work correctly.
I think CLI options like this which are needed for the file to function should be left in the file directly. Doing anything else in my opinion unnecessarily couples these scripts to other files making them more brittle and harder to maintain.
With that said, I also think CLI options which are not needed (such as --numprocesses) can be left to be optionally added through PYTEST_ADDOPTS.
* Add --pyargs to tox.cover.py.
* Add --pyargs to install_and_test.py.
* Remove --pyargs from tox.ini.
Fixes#6106.
AMIs were taken from https://wiki.debian.org/Cloud/AmazonEC2Image/Stretch and https://cloud-images.ubuntu.com/locator/ec2/.
I didn't update the AMI for Fedora due to #6698.
These new AMIs pass on all test farm tests we run during the release process except Ubuntu 18.04 and 18.10 fail on test_apache2.sh. This is tracked at #6706. If this PR lands before this issue is resolved, we should list these systems as expected failures in the release notes.
Adding these AMIs slows down our tests significantly. I didn't measure it, but it feels 50-100% slower at least on my setup. I think it's worth it though.
* Update test farm targets.
* use different ubuntu ami
* Fix test_leauto_upgrades.sh on newer OSes.
Fixes#6697.
This PR updates the version of setuptools pinned in pipstrap which works around the problems we have seen on recent OSes.
Why did I pick this version of setuptools? Because it's the latest and greatest, [supports all versions of Python that we do](https://github.com/pypa/setuptools/blob/v40.6.3/setup.py#L173), [has been out for a month and a half without the need for a point release](https://setuptools.readthedocs.io/en/latest/history.html), and has no non-optional dependencies.
For the last point about dependencies, I don't think we have too much to worry about. setuptools did have a period between versions 34.0.0 and 36.0.0 where they tried to have normal dependencies on other packages, but reverted these changes. See their [changelog for 36.0.0](https://setuptools.readthedocs.io/en/latest/history.html#v36-0-0).
You can also compare their [current setup.py file](https://github.com/pypa/setuptools/blob/v40.6.3/setup.py) to the [setup.py file for the currently pinned version](https://github.com/pypa/setuptools/blob/v29.0.1/setup.py) and you'll see [not much has changed](https://pastebin.com/nQj6d7D8).
Not only that, but I have successfully tested these changes on:
* ubuntu18.10
* ubuntu18.04LTS
* ubuntu16.04LTS
* ubuntu14.04LTS
* ubuntu14.04LTS_32bit
* debian9
* debian8.1
* amazonlinux-2015.09.1
* amazonlinux-2015.03.1
* RHEL7
* fedora23
* fedora29
* centos7
* centos6
* freebsd11
* macOS
* Update setuptools to 40.6.3.
* Build letsencrypt-auto.
* update changelog
* Don't use pipstrap in Dockerfile.centos6.
(cherry picked from commit b7211c3f39)
Fixes#6697.
This PR updates the version of setuptools pinned in pipstrap which works around the problems we have seen on recent OSes.
Why did I pick this version of setuptools? Because it's the latest and greatest, [supports all versions of Python that we do](https://github.com/pypa/setuptools/blob/v40.6.3/setup.py#L173), [has been out for a month and a half without the need for a point release](https://setuptools.readthedocs.io/en/latest/history.html), and has no non-optional dependencies.
For the last point about dependencies, I don't think we have too much to worry about. setuptools did have a period between versions 34.0.0 and 36.0.0 where they tried to have normal dependencies on other packages, but reverted these changes. See their [changelog for 36.0.0](https://setuptools.readthedocs.io/en/latest/history.html#v36-0-0).
You can also compare their [current setup.py file](https://github.com/pypa/setuptools/blob/v40.6.3/setup.py) to the [setup.py file for the currently pinned version](https://github.com/pypa/setuptools/blob/v29.0.1/setup.py) and you'll see [not much has changed](https://pastebin.com/nQj6d7D8).
Not only that, but I have successfully tested these changes on:
* ubuntu18.10
* ubuntu18.04LTS
* ubuntu16.04LTS
* ubuntu14.04LTS
* ubuntu14.04LTS_32bit
* debian9
* debian8.1
* amazonlinux-2015.09.1
* amazonlinux-2015.03.1
* RHEL7
* fedora23
* fedora29
* centos7
* centos6
* freebsd11
* macOS
* Update setuptools to 40.6.3.
* Build letsencrypt-auto.
* update changelog
* Don't use pipstrap in Dockerfile.centos6.
These tests were running on Ubuntu Precise and Debian Wheezy which have reached their end of life and are no longer maintained by the respective distros. This updates the tests to a newer version of Debian and Ubuntu.
* Remove tests on the deprecated precise.
* Add tests for Xenial.
* update Jessie tests to use Wheezy
* update .travis.yml
Fixes#6585.
I wrote up three suggestions for fixing this at https://github.com/certbot/certbot/issues/6585#issuecomment-448054502. I took the middle approach of requiring the user to provide an ACME server to use. I like this better than the other approaches which were:
> Resolve#5938 instead of this issue.
There is value in these tests as is over the compatibility tests in that they don't use Docker and run on different OSes.
> Spin up a local Python server to return the directory object.
Trying to set up a dummy ACME server seemed hacky and error prone.
Other notes about this PR are:
* I put the Pebble setup in `tox.ini` rather than `.travis.yml` as this seems much cleaner and more natural.
* I created a new `tox` environment called `apacheconftest-with-pebble` that reuses the code from `testenv:apacheconftest` so `apacheconftest` can continue to be used with servers other than Pebble like is done in our test farm tests.
* I chose the environment variable `SERVER` for consistency with our integration tests. I chose to not give this environment variable a default but to fail fast when it is not set.
* I ran test farm tests on this PR and they passed.
#6636 broke [test-everything tests](https://travis-ci.org/certbot/certbot/builds/475173804) because `_common.sh` is a common file shared between Certbot and Nginx integration tests and `--no-random-sleep-on-renew` isn't defined for the version of Certbot used in the "oldest" integration tests.
This PR adds code to `_common.sh` to check the Certbot version and if it's new enough, add `--no-random-sleep-on-renew` to the command line. I repurposed `$store_flags` and stopped exporting it because it's not used anywhere outside of this file.
Other approaches I considered and decided against were:
1. Adding this flag in `certbot-boulder-integration.sh`. I decided against it because it's setting us up for the same problem in the future if the oldest version of Certbot is upgraded in the Nginx tests and we call `certbot renew`.
2. Just upgrading the oldest version of Certbot required by Nginx to avoid these issues. While this would work (with perhaps some unnecessary burden for our packagers), I think it's avoiding the real problem here which should now be able to addressed easily with the addition of `$other_flags` and `version_at_least`.
* Add version_at_least().
* Conditionally disable sleep.
* Consolidate store_flags and other_flags.
* update comments
Fixes the problem at https://github.com/certbot/certbot/pull/6592#discussion_r245106383.
The tests use `eval` which neither myself or `pylint` like very much. I started to change this by splitting the path we wanted to test and repeatedly calling `getattr`, but it didn't seem worth the effort to me.
* Add missing acme.jose attribute.
* update changelog
When working on an update to our packages in Ubuntu Xenial, @NCommander noticed that importing code through acme.jose no longer works since josepy became a separate package and remembers having to fix up some code that was using acme.jose himself.
This PR should fix that problem by making all of josepy accessible through acme.jose. This is primarily beneficial to our OS package maintainers who want to avoid subtle API changes when updating packages in stable repositories. They will likely backport this change, but I figure we might as well add it ourselves to minimize divergences in our OS packages in the future and avoid problems on the off chance someone hasn't upgraded acme and was relying on this feature.
GitHub notified us about a security vulnerability in our pinned version of `urllib3` earlier this week. It doesn't affect us, but we might as well upgrade anyway. I checked:
* There are no backwards incompatible features we care about listed at https://github.com/urllib3/urllib3/blob/master/CHANGES.rst.
* urllib3's dependencies don't also need to be updated according to https://github.com/urllib3/urllib3/blob/1.24.1/setup.py.
* The hashes match when obtained from different network vantage points.
While reducing noise in test output is valuable, this flag has made a couple aspects of Certbot's development difficult:
1. We test with different sets of dependencies and running pip in quiet mode removes all output about the packages being installed which has made reviewing changes to these tests more difficult.
2. When pip fails, it provides significantly less output about the failure in quiet mode than it does normally. The output is reduced so much that in the two times I've hit this issue in the last month, I was only able to see that installing package X failed rather than what the cause of that failure was which could be seen with `--quiet` removed.
Also, since running pip without `--quiet` is the tox default, I expect Python developers to be familiar with what they see here.
Currently the release script in master fails for a few reasons. First, it's trying to use --numprocesses which comes from a pytest plugin that we're not installing in the release script. Second, many new warnings are raised when we're not using pinned versions of our dependencies.
I'm not sure I agree, but one could argue that we should fix these issues and use the file during the release. I'm particularly hesitant for us to do this when it comes to warnings. We currently do not pin our dependencies in the release script. Do we really want to stop the release because a new package was released and is warning about something? One could argue we do because these warnings may be visible to the user, but they very rarely are and I think this makes the release process much too painful.
I especially do not think we should block the release on this now as we are not up to date on the warnings raised by the latest versions of our packages so there is a lot to work through.
* Don't use pytest.ini during the release.
* State that pytest.ini isn't used in release script.
This PR has the value of VENV_NAME override any value set in the tools/venv* scripts.
I also removed the use of VENV_ARGS. This was used in _venv_common.sh as a means of passing arguments for virtualenv between the scripts, however, there is no other use of the variable in this repository and passing the arguments through a function call is much more natural in Python.
* Respect VENV_NAME in tools/venv*.
* Stop using VENV_ARGS
* Remove VENV_NAME_ENV_VAR and add docstrings.
Main piece of #5810.
* Rename Certbot integration tests
* Remove nginx from certbot tests
* allow for running individual integration tests
* fail under 65
* Add set -e
* Track Nginx coverage and omit it from report later.
* Use INTEGRATION_TEST in script
* add INTEGRATION_TEST=all
* update min certbot percentage
Fixes#4686.
In Sphinx 1.6, they changed how they handle images in latex and PDF files. You can learn more about this by reading the linked issue (or I can answer any questions), but the shortish version is we now need to use the extension sphinx.ext.imgconverter. This is only available in Sphinx 1.6+.
I also updated our pinned versions to use the latest Sphinx and a new dependency it pulled in called sphinxcontrib-websupport. To build the latex and PDF docs, you must first run:
apt-get install imagemagick latexmk texlive texlive-latex-extra
Afterwards, if you create the normal Certbot dev environment using this branch, activate the virtual environment, and from the root of the repo run make -C docs clean latex latexpdf, you'll successfully build the PDF docs.
* fix#4686
* bump minimum Sphinx req
Fixes#6207.
As noted by Erica:
- we no longer need to check if it exists before linking to it, because we delete properly.
- the previously excisting check on if server is in `LE_REUSE_SERVERS` before unlinking is nice, but probably not necessary, especially since we don't officially support people doing weird things with symlinks in our directories, and because we rmdir which will fail if it's not empty anyway.
* Create single account symlink.
* refactor _delete_accounts_dir_for_server_path
* add symlinked account dir deletion
* add tests
(cherry picked from commit 9b0d2714c1)
Fixes#6207.
As noted by Erica:
- we no longer need to check if it exists before linking to it, because we delete properly.
- the previously excisting check on if server is in `LE_REUSE_SERVERS` before unlinking is nice, but probably not necessary, especially since we don't officially support people doing weird things with symlinks in our directories, and because we rmdir which will fail if it's not empty anyway.
* Create single account symlink.
* refactor _delete_accounts_dir_for_server_path
* add symlinked account dir deletion
* add tests
This stops us from printing messages like:
"Could not choose appropriate plugin for updaters: Could not select or initialize the requested installer None."
when certbot renew --force-renewal is run with a lineage that doesn't have an installer.
* unquote None
* Test None values aren't saved in config file.
Now that yaml/pyyaml#126 is resolved, #6170 can be reverted by bumping the pinned version of PyYAML.
You can see this code passing with full macOS and integration tests at https://travis-ci.org/certbot/certbot/builds/400957729.
* Revert "Allow py37 testing (#6170)"
This reverts commit cad95466b0.
* Bump pyyaml pinning to work on Python 3.7.
We released josepy 1.1.0 a while ago to work around newer versions of cryptography deprecating some of the functionality we were using. We haven't yet upgraded our pinned josepy version though and since #6169 has landed, we're now seeing these deprecation warnings in our tests. This would be shown to certbot-auto users as well.
This PR removes these warnings by upgrading our pinned version of josepy.
* update pinned josepy version
* build leauto
* update pinned dev version of josepy
* Remove apacheconftest packages.
The apacheconftests handle installing Apache dependencies, so let's remove it from the general case.
* We don't need to run dpkg -s in before_install.
* Remove augeas sources.
We only needed it for Ubuntu Precise which is dead and it doesn't work in Ubuntu Xenial.
* Upgrade Python 3.6 tests to 3.7.
Let's continue the approach of testing on the oldest and newest versions of Python 3. We will continue testing on Python 3.6 in the nightly tests.
* Revert "We don't need to run dpkg -s in before_install."
This reverts commit e5d35099a7.
* let apacheconftest handle deps
This allows us to depend on packages like acme>=0.26.0.dev0 during development
and automatically change it to acme>=0.26.0 during the release. We use `git add
-p` to be safe, but if .dev0 is used at all in our released setup.py files,
we're probably doing something wrong.
* Reorganize packages in tox to allow for py37 tests
certbot-dns-cloudflare doesn't currently work in Python 3.7 because it transitively depends on pyYAML which doesn't yet support Python 3.7. See https://github.com/yaml/pyyaml/issues/126 for more info.
* add py37 tox environment
* Use greater than or equal to in requirements.
This changes the existing requirements using strictly greater than to greater
than or equal to so that they're more conventional.
* Use >= for certbot-postfix.
Despite it previously saying 'certbot>0.23.0', certbot-postfix/local-oldest-requirements.txt was pinned to 0.23.0 so let's just use certbot>=0.23.0.
This partially reverts commit 15f1405fff.
A basic tls-alpn-01 implementation is left so we can successfully parse the
challenge so it can be used in boulder's tests.
* Revert "Revert "switch signature verification to use pure cryptography (#6000)" (#6074)"
This reverts commit 3cffe1449c.
* Fixes#6073.
This silences the deprecation warnings from cryptography. I looked into only
silencing the cryptography warning specifically in the function, however,
CryptographyDeprecationWarning doesn't seem to be publicly documented, so we
probably shouldn't depend on it.
For the past couple of releases, twine has errored while trying to upload
packages and this is fixed by upgrading to a newer version of twine. This
commit updates our pinned version installed when using tools/venv.sh to the
latest available version. pkginfo had to be upgraded as well to support the
latest version of twine.
Festival isn't available via Homebrew and is only needed to read the hash
aloud, so let's not make it a strict requirement that it's installed. You can
simply read the hash from the terminal instead.
