Commit Graph
100 Commits
Author SHA1 Message Date
Erica Portnoy 8194e8faef Bump version to 1.19.0 2021-08-03 13:23:45 -07:00
Erica Portnoy 06698ad95f Add contents to certbot/CHANGELOG.md for next version 2021-08-03 13:23:45 -07:00
Erica Portnoy 0d76d1f219 Release 1.18.0 2021-08-03 13:23:13 -07:00
Erica Portnoy 5c3c682b6e Update changelog for 1.18.0 release 2021-08-03 13:12:59 -07:00
ohemorangeandGitHub c8255dded5 Add --verbose-level flag and fix logging level calculations (#8900)
Also, update `dev-cli.ini` example to use new flag.

Although https://github.com/bw2/ConfigArgParse/pull/216 allowed setting a `count` action value in a config file, our default detection system won't let us use that functionality. While we should eventually fix that, for now, let developers have a cli.ini with a higher logging level by adding this flag.

Note that this flag is intended to work the same way adding `-vvv`s does; that is, as a modifier to the pre-set level, rather than setting the absolute level. The number it is set to is equivalent to the number of `v`s that would otherwise have been passed, with "2" as the current maximum effective number of levels (warning --> info --> debug).

* Add --verbose-level flag for devs to set in cli.ini

* Update dev-cli.ini to use new flag
2021-06-10 16:45:07 -07:00
ohemorangeandGitHub b48e336554 Allow nginx parser to handle empty file (#8895)
* Allow parsing empty files

* add unit test

* lint

* update parser_test

* Update configurator_test

* update changelog
2021-06-11 09:21:52 +10:00
Erica Portnoy 814d8d1aba Bump version to 1.17.0 2021-06-01 10:52:31 -07:00
Erica Portnoy a190480517 Add contents to certbot/CHANGELOG.md for next version 2021-06-01 10:52:31 -07:00
Erica Portnoy 7e8f22e136 Release 1.16.0 2021-06-01 10:52:23 -07:00
Erica Portnoy 965a403699 Update changelog for 1.16.0 release 2021-06-01 10:49:17 -07:00
ohemorangeandGitHub e946479b9f Use shortlink for renewal setup instructions (#8874) 2021-05-28 14:50:59 -07:00
18631b99ef Add instructions for setting up a cronjob in the docs (#8870)
* Add instructions for setting up a cronjob in the docs

* Be more specific about where the cron entry will be created

Co-authored-by: alexzorin <alex@zorin.id.au>

* Correct &amp;s to &s

Co-authored-by: alexzorin <alex@zorin.id.au>

* Correct other &amp; to &

Co-authored-by: alexzorin <alex@zorin.id.au>

* De-weasel the double-scheduled-task comment

Co-authored-by: alexzorin <alex@zorin.id.au>

* Have users create directory hooks instead of command line hooks

* Use sudo in command

Co-authored-by: alexzorin <alex@zorin.id.au>

* tell windows users to ignore these instructions instead of telling them they won't work

* Use the same commands that we have in the general instructions

Co-authored-by: alexzorin <alex@zorin.id.au>
2021-05-28 16:27:56 +10:00
2e31b1ca41 Remove no names found in configuration files because it sounds like an error but actually it is fine (#8866)
* Remove no names found in configuration files because it sounds like an error but actually it is fine

* fix test

* Pose question more grammatically and specifically, and remove extra space

* fix lint

Co-authored-by: Alex Zorin <alex@zorin.id.au>
2021-05-27 07:54:04 +10:00
6f27c32db1 Command-line UX overhaul (#8852)
Streamline and reorganize Certbot's CLI output.

This change is a substantial command-line UX overhaul,
based on previous user research. The main goal was to streamline
and clarify output. To see more verbose output, use the -v or -vv flags.

---

* nginx,apache: CLI logging changes

- Add "Successfully deployed ..." message using display_util
- Remove IReporter usage and replace with display_util
- Standardize "... could not find a VirtualHost ..." error

This changes also bumps the version of certbot required by certbot-nginx
and certbot-apache to take use of the new display_util function.

* fix certbot_compatibility_test

since the http plugins now require IDisplay, we need to inject it

* fix dependency version on certbot

* use better asserts

* try fix oldest deps

because certbot 1.10.0 depends on acme>=1.8.0, we need to use
acme==1.8.0 in the -oldest tests

* cli: redesign output of new certificate reporting

Changes the output of run, certonly and certonly --csr. No longer uses
IReporter.

* cli: redesign output of failed authz reporting

* fix problem sorting to be stable between py2 & 3

* add some catch-all error text

* cli: dont use IReporter for EFF donation prompt

* add per-authenticator hints

* pass achalls to auth_hint, write some tests

* exclude static auth hints from coverage

* dont call auth_hint unless derived from .Plugin

* dns fallback hint: dont assume --dns-blah works

--dns-blah won't work for third-party plugins, they need to be specified
using --authenticator dns-blah.

* add code comments about the auth_hint interface

* renew: don't restart the installer for dry-runs

Prevents Certbot from superfluously invoking the installer restart
during dry-run renewals. (This does not affect authenticator restarts).

Additionally removes some CLI output that was reporting the fullchain
path of the renewed certificate.

* update CHANGELOG.md

* cli: redesign output when cert installation failed

- Display a message when certificate installation begins.
- Don't use IReporter, just log errors immediately if restart/rollback
  fails.
- Prompt the user with a command to retry the installation process once
  they have fixed any underlying problems.

* vary by preconfigured_renewal

and move expiry date to be above the renewal advice

* update code comment

Co-authored-by: ohemorange <ebportnoy@gmail.com>

* update code comment

Co-authored-by: ohemorange <ebportnoy@gmail.com>

* fix lint

* derve cert name from cert_path, if possible

* fix type annotation

* text change in nginx hint

Co-authored-by: ohemorange <ebportnoy@gmail.com>

* print message when restarting server after renewal

* log: print "advice" when exiting with an error

When running in non-quiet mode.

* try fix -oldest lock_test.py

* fix docstring

* s/Restarting/Reloading/ when notifying the user

* fix test name

Co-authored-by: ohemorange <ebportnoy@gmail.com>

* type annotations

* s/using the {} plugin/installer: {}/

* copy: avoid "plugin" where possible

* link to user guide#automated-renewals

when not running with --preconfigured-renewal

* cli: reduce default logging verbosity

* fix lock_test: -vv is needed to see logger.debug

* Change comment in log.py to match the change to default verbosity

* Audit and adjust logging levels in apache module

* Audit and adjust logging levels in nginx module

* Audit, adjust logging levels, and improve logging calls in certbot module

* Fix tests to mock correct methods and classes

* typo in non-preconfigured-renewal message

Co-authored-by: ohemorange <ebportnoy@gmail.com>

* fix test

* revert acme version bump

* catch up to python3 changes

* Revert "revert acme version bump"

This reverts commit fa83d6a51c.

* Change ocsp check error to warning since it's non-fatal

* Update storage_test in parallel with last change

* get rid of leading newline on "Deploying [...]"

* shrink renewal and installation success messages

* print logfile rather than logdir in exit handler

* Decrease logging level to info for idempotent operation where enhancement is already set

* Display cert not yet due for renewal message when renewing and no other action will be taken, and change cert to certificate

* also write to logger so it goes in the log file

* Don't double write to log file; fix main test

* cli: remove trailing newline on new cert reporting

* ignore type error

* revert accidental changes to dependencies

* Pass tests in any timezone by using utcfromtimestamp

* Add changelog entry

* fix nits

* Improve wording of try again message

* minor wording change to changelog

* hooks: send hook stdout to CLI stdout

includes both --manual and --{pre,post,renew} hooks

* update docstrings and remove TODO

* add a pending deprecation on execute_command

* add test coverage for both

* update deprecation text

Co-authored-by: ohemorange <ebportnoy@gmail.com>

Co-authored-by: Alex Zorin <alex@zorin.id.au>
Co-authored-by: alexzorin <alex@zor.io>
2021-05-25 10:47:39 +10:00
ohemorangeandGitHub f339d23e54 Remove further references to certbot-auto in the repo (#8814)
* Move version.py to tests/letstest since it's used by test_sdists.sh

* Delete unused components of certbot-auto

* Remove test_leauto_upgrades.sh and references to it

* Remove test_letsencrypt_auto_certonly_standalone.sh and references to it

* Remove outstanding references to certbot-auto

* Remove references to letsencrypt-auto

* find certbot in the correct directory

* delete letsencrypt-auto-source line from .isort.cfg since that directory no longer contains any python code

* remove (-auto) from certbot(-auto)

* delete line from test

* Improve style for version.py
2021-04-27 15:27:21 -07:00
ohemorangeandGitHub ba912018f8 Remove pytest run from release script (#8810)
Fixes #8802.

Also removed the unused `kgs` cruft while I was here, since it's leftover from the [initial release commit](https://github.com/certbot/certbot/commit/3c08b512c3b546528880a1353f6b8607307533a3) and I'm pretty sure we don't use that anymore.
2021-04-26 15:18:05 -07:00
32247b3c89 Remove modifications to certbot-auto from the release script (#8797)
Fixes #8707.

* Remove modifications to certbot-auto from the release script

* Update tools/_release.sh

* Delete tools/eff-pubkey.pem

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2021-04-22 13:37:46 -07:00
ohemorangeandGitHub c21f277248 Merge pull request #8779 from certbot/candidate-1.14.0
Update files from 1.14.0 release
2021-04-06 16:09:19 -07:00
ohemorangeandGitHub 8759ccaecb Update issue template to list snap as an installation option (#8754) 2021-03-29 13:56:08 -07:00
ohemorangeandGitHub fa25d8356d Remove references to certbot-auto and letsencrypt-auto that we don't need for the final release (#8738)
Fixes #8661.

As mentioned in https://github.com/certbot/certbot/issues/8661#issuecomment-806168214, there are quite a few remaining references, but until we modify the release script, we still need those. The changes here and the list there were created by grepping for the following terms:

```
certbot-auto
cb-auto
cbauto
certbotauto
letsencrypt-auto
le-auto
leauto
letsencryptauto
LEAUTO
LE_AUTO
LETSENCRYPT_AUTO
LETSENCRYPTAUTO
CB_AUTO
CERTBOT_AUTO
CBAUTO
CERTBOTAUTO
```

* Remove references to certbot-auto from certbot code

* Remove references to LEAUTO

* Remove references to CERTBOT_AUTO

* Remove references to letsencrypt-auto

* Remove references to certbot-auto from docs and tools

* remove cli constants header files

* Remove Python virtual environment section
2021-03-24 16:58:15 -07:00
ohemorangeandGitHub fd62a09197 dump test farm failure logs (#8740) 2021-03-24 16:19:54 -07:00
ohemorangeandGitHub f90e93134c Upgrade cryptography to 3.4.6 (#8730)
* Upgrade cryptography to 3.4.6

* Fix comment with instructions for how to use hashin

* run tools/rebuild_certbot_constraints.py

* add deps for building cryptography in snaps

* Update cryptography build dependencies for docker

* Update sources for test farm tests

* Remove rust if it's installed for test farm tests

* source bootstrap script and call sudo as needed
2021-03-24 10:29:12 -07:00
ohemorangeandGitHub 40ae5d939e Fix linux-py39-cover test (#8720)
* update setuptools

* upgrade Markupsafe
2021-03-19 14:27:29 -07:00
Erica Portnoy 9d97be3a84 Bump version to 1.14.0 2021-03-02 13:50:04 -08:00
Erica Portnoy 4d6db0eb71 Add contents to certbot/CHANGELOG.md for next version 2021-03-02 13:50:03 -08:00
Erica Portnoy 92a66454b6 Release 1.13.0 2021-03-02 13:49:58 -08:00
Erica Portnoy 976068b5a0 Update changelog for 1.13.0 release 2021-03-02 13:37:04 -08:00
ohemorangeandGitHub ae3ed200c0 Remove check for 'fake' in issuer name when renewing certs (#8685)
Fixes #8680.

We seem to have no existing testing code anywhere in this vicinity, so figured I'd get this up quickly then work on that. Manual tests (renew staging certificate, should allow it; renew non-staging cert as staging, should error) passed.

* Remove check for 'fake' in issuer name when renewing certs

* Change fake issuer name to make sure we're not relying on it anywhere
2021-02-24 14:51:57 -08:00
Erica Portnoy 2bcd8c59db Bump version to 1.13.0 2021-02-02 11:06:48 -08:00
Erica Portnoy 57cba3690d Add contents to certbot/CHANGELOG.md for next version 2021-02-02 11:06:47 -08:00
Erica Portnoy 786a130b7d Release 1.12.0 2021-02-02 11:06:40 -08:00
Erica Portnoy df866b907b Update changelog for 1.12.0 release 2021-02-02 10:58:41 -08:00
ohemorangeandGitHub bdfb9f19c4 Remove deprecated options as early as possible using an explicit list (#8617)
* Remove deprecated options as early as possible using an explicit list

* add deprecated options to cli init import list

* use correct dict comprehension syntax for py3

* lint

* add test for renewal reconstitution code

* add test to ensure we're not saving deprecated values

* comment code
2021-01-28 12:34:50 -08:00
ohemorangeandGitHub 5c3fd7d9ee Merge pull request #8573 from certbot/candidate-1.11.0
Update files from 1.11.0 release
2021-01-05 13:25:11 -08:00
ohemorangeandGitHub d3166d7072 Merge pull request #8505 from certbot/candidate-1.10.1
Candidate 1.10.1
2020-12-03 12:29:26 -08:00
ohemorangeandGitHub faa8d230c7 Merge pull request #8487 from certbot/candidate-1.10.0
Update files from 1.10.0 release
2020-12-01 12:25:10 -08:00
ohemorangeandGitHub 3673ca77a5 Fix LXD setup in snap README (#8416)
Fixes #8409.

Change the line in the README to allow `sudo /snap/bin/lxd.migrate -yes` to fail (for example, if there's nothing to migrate), but the whole command to succeed.

I tested this on a clean Focal install and confirmed it works.
2020-10-28 15:51:16 -07:00
ohemorangeandGitHub fc864543a7 Simplify/document snap creation (#8404)
This PR adds the following documentation improvements to fix https://github.com/certbot/certbot/issues/7958:

- Simplify building external plugins
- Separate out certbot snap instructions from plugin instructions
- Mention that dnsimple is just an example for the plugin instructions
- Mention remote build for other architectures
- Mention snap doc exists elsewhere in developer guide (`contributing.rst`)

* Set up generate_dnsplugins_all.sh for all files and parametrize snapcraft and postrefreshhook files

* Create constraints file in the generate_dnsplugins_all script

* Separate out plugin and certbot snaps and update instructions

* Add remote build instructions

* Add pointers to the README to contributing.rst
2020-10-27 10:22:40 -07:00
ohemorangeandGitHub 8a3aed0476 add status messages to create_github_release script (#8353)
It took long enough to do all the downloading and uploading that I found myself wishing I could be sure things were happening.
2020-10-07 08:31:37 -07:00
Erica Portnoy eff761ab1e Bump version to 1.10.0 2020-10-06 12:15:29 -07:00
Erica Portnoy 5f040a8e32 Add contents to certbot/CHANGELOG.md for next version 2020-10-06 12:15:29 -07:00
Erica Portnoy 5173ab6b90 Release 1.9.0 2020-10-06 12:15:27 -07:00
Erica Portnoy 448fd9145a Update changelog for 1.9.0 release 2020-10-06 11:39:49 -07:00
ohemorangeandGitHub fca7ec896a Improve error message for prepare-plug-plugin hook when certbot isn't installed (#8338)
Provides a partial fix for #8182 by improving the error message.
2020-09-30 12:43:24 -07:00
ohemorangeandGitHub be6c890874 Retry Snap upload in pipeline (#8300)
* add set -e to all bash instances in deploy-stage.yml

* retry uploading snap if we fail

* Add the rest of the set -e calls for bash in azure while we're here

* use retry based on travis_retry

* add set -e to the script: sections that run on macOS/Linux

* actually don't fail on result

* reset result before running command because bash short circuits or conditionals

* remove inapplicable comment
2020-09-25 15:31:13 -07:00
ohemorangeandGitHub acb6d34c5f Update test farm tests to stop using certbot-auto (#8207)
* Create bootstrap script

* Delete a whole bunch of the bootstrap script

* modify test_tests to use new script

* put python version checking in back in

* add x

* call the venv creation from inside the bootstrap

* add targets back

* modify test_apache2 to use new format

* shouldn't need virtualenv on rhel

* readd targets

* Update test_sdists to use new script

* move setting up venv back out of script so it's not run with sudo

* take venv3.py call out of bootstrap in all scripts

* add additional python3-devel pkg name

* fix test_sdists

* enable additional rhel7 repos

* clean up code and comments

* Update tests and instructions to use auto_targets.yaml with test_leauto_upgrades.sh and test_letsencrypt_auto_certonly_standalone.sh

* only install python3-devel.x86_64 for rhel7

* Upgrade python version for debian in test_apache2.sh

* don't run test_tests or test_sdists on debian 9 or ubuntu 16.04

* Add 20.04 and 20.04 arm images to targets.yaml

* use pyenv to upgrade to python3.5

* remove arm64 instance because it's having auth trouble

* correct pyenv usage on ubuntu

* add arm64 target to targets.yaml

* replace debian 9 arm64 with ubuntu 20

* don't try to upgrade a perfectly good python version

* let's just add ubuntu20 to apache2_targets while we're here

* uncomment test_apache2

* move adding python3-devel.x86_64 to bootstrap_os_packages to avoid potential race condition

* no need to specify the arch once extra rhel7 repos enabled

* explicitly specify python3

* don't fail if we can't enable rhel7 extras

* capture python36-devel as well
2020-08-18 10:07:27 -07:00
ohemorangeandGitHub 22730dc0ac Merge pull request #8192 from certbot/docker-base
Add certbot-docker files to this repository preserving history
2020-08-06 16:46:17 -07:00
086e6c46b6 Improve github release creation process (#8189)
* Improve github release creation process

* Comment file

* Update tools/create_github_release.py

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* run chmod +x on tools/create_github_release.py

* Add description of create github release method

* remove references to unnecessary azure credential

* remove unnecessary import

* Add reminders to update other file to definitions in .azure-pipelines

* Raise an error if we fail to fetch the artifact from azure

* Create github release as a draft, upload artifacts, then un-draft, for hooks to be run at the right point

* get the version number from the release

* add new packages to dev3_extras so they're installed by tools/venv3.py

* remove unnecessary import

* fun fact: tempdirs behave differently when used as a context manager

* Move comment to construct.py

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2020-08-06 16:32:57 -07:00
ohemorangeandGitHub 48a0cc0c42 Merge pull request #8188 from certbot/candidate-1.7.0
Release 1.7.0
2020-08-04 13:09:34 -07:00
Erica Portnoy 5415fc201c Release version v1.7.0 2020-08-04 12:33:20 -07:00
Erica Portnoy b08fdc7dfb Bump version to 1.8.0 2020-08-04 11:33:04 -07:00
Erica Portnoy 6eb5954f0e Add contents to certbot/CHANGELOG.md for next version 2020-08-04 11:33:04 -07:00
Erica Portnoy 6ec83d52b5 Release 1.7.0 2020-08-04 11:33:03 -07:00
Erica Portnoy 403ded5c58 Update changelog for 1.7.0 release 2020-08-04 11:20:15 -07:00
ohemorangeandGitHub 086c8b1b3e Mention the availability of DNS plugin snaps in our docs under certbot/docs. (#8176)
Part of #8142.

* Mention that DNS plugins are available as snaps

* Mention snaps in guide to writing third-party plugins
2020-07-27 13:21:31 -07:00
9ee4831f78 Make externally snapped plugin updates more stable (#8145)
Fixes #7863.

Connect command is `sudo snap connect certbot-dns-dnsimple:certbot-metadata certbot:certbot-metadata`
Logs are `cat /var/snap/certbot-dns-dnsimple/current/debuglog`
Echos in hook are only printed to terminal when it exits 0; otherwise, check logs in `debuglog` mentioned above.

Manual tests include all iterations of connected, unconnected, installed for the first, second time, etc, with passing and failing version checks.

* Make dnsimple not update if certbot is too old

* create an interface to read cb version

* add missing newline

* fix syntax

* trying to figure out the consumer syntax

* trying to figure out the consumer syntax, again

* only check post first install

* valid setting name

* test for first install differently

* snapctl doesn't error if it fails I guess

* time to do some print debugging

* continue playing with syntax

* once again, fooled by bash int vs string comparisons!

* debugging

* if we use post and pre together we can do this

* is this how content interface syntax works

* it's a directory?

* more debug

* what's that error message again?

* try other syntax

* if it's not documented just guess at syntax

* actually, I think this is the syntax

* oops didn't set for new hook

* test passing information along connection

* interface attributes can only be set during the execution of prepare hooks

* just do it with main connection

* undo last few test changes

* Add some printing to make sure we understand what's going on

* create empty directory to bind to

* put mkdir in the correct part

* let's inspect the environment

* it can't run bash directly.

* perhaps only directories can be shared via the contente interface

* update name of folder

* echo to debug log to understand what's going on exactly. we have file access though!

* update grep for new file

* more printing

* echo to the debug log

* ok NOW all print statements are going to the log

* why does echo need two >s

* remove unnecessary extra check, just check if the init file is available

* check if certbot version will be available post-refresh after all

* pre-refresh hook is not necessary to get certbot version

* update mkdir so we don't have to clean each time

* try comparing version numbers in python

* it's python3

* we need different prints for if we succeed or if we fail.

* improve bash syntax

* remove some debugging code

* Remove debug script

* remove spaces for clarity

* consolidate parts and remove more test code

* s/certbot-version/certbot-metadata/g

* use sys.exit instead of exit

* find and save certbot version on the certbot side

* change presence test to new file

* switch to using packaging.version.parse instead of LooseVersion

* switch to requiring certbot version >= plugin version

* add plugin snap changes to generate script

* Add comment to generation file saying not to edit generated files manually

* Create post-refresh hook for all plugins with script

* generate files using new script

* update snapcraft.yaml files for plugins

* bin/sh comes first

* Add packaging to install_requires

* Check that refresh is allowed in integration test

* switch plug and slot names in integration test

* Update tools/generate_dnsplugins_postrefreshhook.sh

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* small bash fixes

* Update snap readme with new instructions

* Run tools/generate_dnsplugins_postrefreshhook.sh

* Update tools/snap/generate_dnsplugins_postrefreshhook.sh

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2020-07-22 17:07:07 -07:00
ohemorangeandGitHub cb3ff9ef18 Set up CentOS 8 test farm tests (#8122)
Fixes #7420.

* Set up CentOS 8 test farm tests

* Don't add to apache2_targets until 7273 is resolved

* Start upgrade test from a version that works on centos 8

* remove when possible from targets
2020-07-01 17:07:41 -07:00
ohemorangeandGitHub 2af297d72f Make each DNS plugin respect EXCLUDE_CERTBOT_DEPS (#8117)
* Don't include certbot deps when EXCLUDE_CERTBOT_DEPS is set

* import os
2020-06-29 16:58:26 -07:00
ohemorangeandGitHub 713b91495b Fix paths when calling out to programs outside of snap (#8108)
Fixes #8093.

This PR modifies and audits all uses of `subprocess` and `Popen` outside of tests, `certbot-ci/`, `certbot-compatibility-test/`, `letsencrypt-auto-source/`, `tools/`, and `windows-installer/`. Calls to outside programs have their `env` modified to remove the `SNAP` components of paths, if they exist. This includes any calls made from hooks, calls to `apachectl` and `nginx`, and to `openssl` from `ocsp.py`.

For testing manually, rsync flags will look something like:

```
rsync -avzhe ssh root@focal.domain:/home/certbot/certbot/certbot_*_amd64.snap .
rsync -avzhe ssh certbot_*_amd64.snap root@centos7.domain:/root/certbot/
```

With these modifications, `certbot plugins --prepare` now passes on Centos 7.

If I'm wrong and we package the `openssl` binary, the modifications should be removed from `ocsp.py`, and `env` should be passed into `run_script` rather than set internally in its calls from nginx and apache.

One caveat with this approach is the disconnect between why it's a problem (packaging) and where it's solved (internal to Certbot). I considered a wrapping approach, but we'd still have to audit specific calls. I think the best way to address this is robust testing; specifically, running the snap on other systems.

For hooks, all calls will remove the snap paths if they exist. This is probably fine, because even if the hook intends to call back into certbot, it can do that, it'll just create a new snap.

I'm not sure if we need these modifications for the Mac OS X/ Darwin calls, but they can't hurt.

* Add method to plugins util to get env without snap paths

* Use modified environment in Nginx plugin

* Pass through env to certbot.util.run_script

* Use modified environment in Apache plugin

* move env_no_snap_for_external_calls to certbot.util

* Set env internally to run_script, since we use that only to call out

* Add env to mac subprocess calls in certbot.util

* Add env to openssl call in ocsp.py

* Add env for hooks calls in certbot.compat.misc.

* Pass env into execute_command to avoid circular dependency

* Update hook test to assert called with env

* Fix mypy type hint to account for new param

* Change signature to include Optional

* go back to using CERTBOT_PLUGIN_PATH

* no need to modify PYTHONPATH in env

* robustly detect when we're in a snap

* Improve env util fxn docstring

* Update changelog

* Add unit tests for env_no_snap_for_external_calls

* Import compat.os
2020-06-25 15:36:29 -07:00
ohemorangeandGitHub bce14ae65f Make DNS plugin snaps use core20 (#8106)
Fixes #8103.

* Update the DNS plugin generator script to core20 syntax

* Generate new snapcraft.yamls for the DNS plugins

* Update certbot.wrapper to search for python3.8 paths
2020-06-23 09:31:08 -07:00
ohemorangeandGitHub 70c8481fd8 Don't include certbot deps when EXCLUDE_CERTBOT_DEPS is set in plugins (#8091)
This will allow DNS plugin snaps to build if they rely on unreleased acme/certbot, and remove other copy of Certbot from externally snapped plugins. Fixes #8064 and fixes #7946. Implementation is based on the design [here](https://github.com/certbot/certbot/issues/8064#issuecomment-645513120).

To test, see reverted commit 8632064. Steps taken:

- added changes to setup.py and snapcraft.yaml
- successfully snapped, connected, ran `sudo certbot plugins --prepare`
- added temporary changes to have both certbot and certbot-dns-dnsimple use DNSAuthenticator2
- snapped and installed certbot, `certbot plugins` failed as expected.
- snapped and installed certbot-dns-dnsimple, `sudo certbot plugins --prepare` succeeded
- Inspected dns plugin's `bin` and `lib`; no `certbot` or `acme`, as expected.
```
$ ls /snap/certbot-dns-dnsimple/current/lib/python3.6/site-packages/
OpenSSL                                         future-0.18.2.dist-info        requests_file.py
PyYAML-5.3.1.dist-info                          idna                           setuptools
_cffi_backend.cpython-36m-x86_64-linux-gnu.so   idna-2.9.dist-info             setuptools-47.3.1.dist-info
certbot_dns_dnsimple                            lexicon                        six-1.15.0.dist-info
certbot_dns_dnsimple-1.6.0.dev0-py3.6.egg-info  libfuturize                    six.py
certifi                                         libpasteurize                  tldextract
certifi-2020.4.5.1.dist-info                    past                           tldextract-2.2.2.dist-info
cffi                                            pip                            urllib3
cffi-1.14.0.dist-info                           pip-20.1.1.dist-info           urllib3-1.25.9.dist-info
chardet                                         pkg_resources                  wheel
chardet-3.0.4.dist-info                         pyOpenSSL-19.1.0.dist-info     wheel-0.34.2.dist-info
cryptography                                    pycparser                      yaml
cryptography-2.8.dist-info                      pycparser-2.20.dist-info       zope
dns_lexicon-3.3.26.dist-info                    requests                       zope.interface-5.1.0-py3.6-nspkg.pth
easy_install.py                                 requests-2.23.0.dist-info      zope.interface-5.1.0.dist-info
future                                          requests_file-1.5.1.dist-info
$ ls /snap/certbot-dns-dnsimple/current/bin/
chardetect  futurize  lexicon  pasteurize  tldextract
```
- reset to HEAD^
- snapped and installed certbot to not have the DNSAuthenticator2 changes, `certbot plugins` failed as expected.

* Don't include certbot deps when EXCLUDE_CERTBOT_DEPS is set

* Set EXCLUDE_CERTBOT_DEPS in certbot-dns-dnsimple/snap/snapcraft.yaml
2020-06-18 15:24:10 -07:00
ohemorangeandGitHub c5e5594ac3 Switch to using snap-constraints in dns plugin so the .gitignore catches it. (#8092) 2020-06-18 15:15:41 -07:00
ohemorangeandGitHub ad8ffc1bf0 Merge pull request #8089 from certbot/squashed-snap-plugin
Merge the snap-plugin branch into master
2020-06-18 12:59:04 -07:00
Erica Portnoy 29a23d3148 Update README instructions for merged master
Use Focal when following README instructions

Discard changes to disco.py

bring README up to date with current knowledge

Move README to snap/local/
2020-06-18 12:49:58 -07:00
ohemorangeandGitHub bcb3554836 Merge pull request #8086 from certbot/core20-squashed
Upgrade snap to be based on core20

This PR makes several changes to be built on top of the core20 base snap. Fixes #7854.

The main changes are to `snapcraft.yaml`. With Snapcraft 4.0/core20 base, the python plugin is a thin wrapper, basically creating a `venv` and installing the packages from the source. The trouble with this is that it runs pycache, creating caches that conflict from the different parts. So to solve that, we put everything in a single part. Other changes include:

- We use classic confinement, so we need to specify a bunch of python packages to `stage-packages`, as mentioned [here](https://forum.snapcraft.io/t/trouble-bundling-python-with-classic-confinement-in-core20-4-0-4/18234/2).
- The certbot executable now lives in `bin`, so specify running `certbot/bin`.
- Since `python-augeas` is now being pulled into the single part, remove the pinning from constraints so we can use the latest version directly from github.
- Precompile our `cryptography` and `cffi` wheels to be based on python3.8.

Separately, we had to upgrade the snapcraft docker image to be based on focal, due to the thin wrapper situation. This was accomplished [here](https://github.com/adferrand/snapcraft/pull/1).
2020-06-17 17:20:46 -07:00
Erica PortnoyandBrad Warren 0f53e8ad4e Upgrade snap to be based on core20
* Get rid of a whole bunch of error message
* Remove some more overlaps
* don't use certbot from nginx and apache
* use python3 from bin
* certbot needs to be in bin
* try to exclude just the certbot folder
* try a couple things to use the python from the venv bin
* play around with which versions of things we want from each package
* ok, certbot-nginx does need to stage bin
* certbot needs to not stage bin. why does certbot not put certbot in bin?
* fail to inspect more versions of things in the container shell
* take cffi backend from python-augeas
* if we use certbot from bin things should work?
* why is bin not in path? no idea, but let's get it compiled then inspect things in the snap shell
* use snap.certbot instead of bin/certbot
* it does require bin/certbot. I don't know why.
* let's see if we can stick it all in one step
* try installing local subdirectories
* move python-augeas into the single part
* remove after
* put back python-augeas part for now; ERROR: Could not satisfy constraints for 'python-augeas': installation from path or url cannot be constrained to a version
* how was this previously working without git installed? install git.
* maybe it needs to already have python3-wheel installed
* maybe wheel will install first if I change it to -e
* no -e
* maybe try a different python3 package to stage
* this last change wasn't necessary
* remove the bin/ from renew
* nope, it does need bin/certbot
* back to wget
* stage a bare python3
* add all necessary python packages to stage-packages
* pretty sure we don't actually need wheel. let's try removing it!
* remove python-augeas, since we have it pinned to an older version in cb-auto that might work
* stage augeas
* still need libaugeas-dev
* ok let's try building
* combining into one part works! just make sure to unpin python-augeas when generating snap-constraints.txt
* change our scripts to unpin python-augeas
* Use ubuntu 20 in compile_native_wheels.sh
* .travis.yml should use python3-dev instead of python-dev
* jk! we don't need python3-dev in travis
* Update cffi and cryptography wheels for ubuntu20 version of python
* looks like we need python3-dev to build things
* Remove deprecated i386 wheels
2020-06-17 16:57:51 -07:00
8d776fb7ac Make Certbot find externally snapped plugins (#8054)
* Add snap plugin support

Switch to a PoC branch of certbot that supports the new
CERTBOT_PLUGIN_PATH and wrap the snap to set this variable correctly
based on the content interfaces connected.

(cherry picked from commit 7076a55fd82116d068e2aca7239209b7203917d2)

* Modify certbot.wrapper to append to PYTHONPATH instead of separate CERTBOT_PLUGIN_PATH variable

* Update certbot-wrapper to python3.6 version

* add source field

* Update changelog

* Use bash instead of sh

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* Exit if something goes wrong

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>

* No leading : when modifying empty PYTHONPATH

* Improve bash handling of PYTHONPATH manipulation

Co-authored-by: Robie Basak <robie.basak@canonical.com>
Co-authored-by: Brad Warren <bmw@eff.org>
Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
2020-06-10 13:52:56 -07:00
Erica Portnoy bebcad0588 update changelog 2020-06-04 15:57:34 -07:00
Erica Portnoy 92f26367eb Merge remote-tracking branch 'alexzorin/7979_restore_v1_as_v2' into upgrade-acmev1 2020-06-04 14:29:56 -07:00
ohemorangeandGitHub 8c8d3fab91 Merge pull request #8040 from certbot/candidate-1.5.0
Release 1.5.0
2020-06-02 12:19:39 -07:00
ohemorangeandGitHub 2acc1dcc89 Fix TLS-ALPN tests with newer versions of OpenSSL (#8026)
Fixes #7988. As described there, the steps involved are:

1. Update our tests so they fail due to this problem.
2. Update the keys used in the tests so they pass with the new changes.

For 1, see a [failing travis run](https://travis-ci.com/github/certbot/certbot/jobs/340710511) with the included change. And for the full output to confirm that this is what is failing, see a [run on debian 10](https://github.com/certbot/certbot/files/4692350/debian_run_log.txt).

This PR adds `rsa4096_key.pem` and `rsa4096_cert.pem`, updates the `TLS-ALPN` test to use those keys in place of the 1024-bit versions, and fixes the README in that `testdata` folder with correct instructions to generate these files.

* export PIP_NO_BINARY in pip install subshell in test_sdists.sh

* set environment variable on the line that installs most packages

* Generate 4096-bit rsa key and cert, and fix README instructions to do so.

* Update TLS_ALPN test to use 4096-bit key instead of 1024-bit key.

* Update changelog

* Older versions of Python have an error when both VIRTUAL_NO_DOWNLOAD and PIP_NO_BINARY are set, so only apply the latter at the install phase.

* Add enum34 constraint manually, since rebuild_dependencies.py seems to be broken.

* only delete key if it exists

* Check OpenSSL version before trying to set PIP_NO_BINARY

* Add comment explaining why we only set PIP_NO_BINARY at the install step
2020-06-01 15:18:38 -07:00
ohemorangeandGitHub 332def46da Require explicit confirmation of snap plugin permissions before connecting (#8013)
Fixes #7667.

Implements the plan described in #7667.

Here's a terminal log showing that it does so:

```
# sudo snap connect certbot:plugin certbot-dns-dnsimple
error: cannot perform the following tasks:
- Run hook prepare-plug-plugin of snap "certbot" (run hook "prepare-plug-plugin": 
-----
Only connect this interface if you trust the plugin author to have root on the system
Run `snap set certbot trust-plugin-with-root=ok` to acknowledge this and then run this command again to perform the connection
-----)
# snap set certbot trust-plugin-with-root=ok
# sudo snap connect certbot:plugin certbot-dns-dnsimple
# sudo snap disconnect certbot:plugin certbot-dns-dnsimple:certbot
# sudo snap connect certbot:plugin certbot-dns-dnsimple
error: cannot perform the following tasks:
- Run hook prepare-plug-plugin of snap "certbot" (run hook "prepare-plug-plugin": 
-----
Only connect this interface if you trust the plugin author to have root on the system
Run `snap set certbot trust-plugin-with-root=ok` to acknowledge this and then run this command again to perform the connection
-----)
```

* Add plugin connection hook to accept root trust

* snapctl requires a configure hook to set options

* Add sh notice

* Update changelog
2020-05-26 12:02:33 -07:00
ohemorangeandGitHub 3ea5170647 Error out earlier in apache installer when mod_ssl is not available (#7984)
* Error out in apache installer when mod_ssl is not available

* Update to MisconfigurationError and add/fix tests

* Remove error cases we no longer hit and associated test

* mock out function to have consistent error across machines

* improve changelog message

* only check key in modules list, not value
2020-05-19 15:34:21 -07:00
Erica Portnoy 361d1f732e Release version v1.4.0 2020-05-05 14:45:57 -07:00
Erica Portnoy b224b49986 Bump version to 1.5.0 2020-05-05 13:44:23 -07:00
Erica Portnoy 1a72cdecf2 Add contents to certbot/CHANGELOG.md for next version 2020-05-05 13:44:23 -07:00
Erica Portnoy 5586ae071a Release 1.4.0 2020-05-05 13:44:21 -07:00
Erica Portnoy ca60ad52b9 Update changelog for 1.4.0 release 2020-05-05 12:37:33 -07:00
ohemorangeandGitHub a06d5ac7a1 Deprecate certbot-auto on Gentoo, macOS, and FreeBSD (#7926)
* Deprecate certbot-auto on Gentoo

* Deprecate certbot-auto on macOS

* Deprecate certbot-auto on FreeBSD

* build le-auto

* Update Changelog

* Deprecate certbot-auto on Gentoo, FreeBSD, and macOS
2020-04-23 12:49:35 -07:00
ohemorangeandGitHub eed45827ad Remove references to the apache-parser-v2 branch (#7925)
Fixes #7786.
2020-04-21 13:06:30 -07:00
ohemorangeandGitHub 751d836746 Update using.rst to accurately describe acmev2 behavior (#7924)
Fixes #7268

I removed the reference to automatically selecting which ACME protocol we use, since at some point we'll want to rip out the non-spec-compliant ACMEv1 code.
2020-04-21 12:47:59 -07:00
ohemorangeandGitHub 9c345ac301 Do not require mock in Python 3 in certbot-dns modules (#7900)
Part of #7886.

This PR conditionally installs `mock` in `certbot-dns-*/setup.py` based on setuptools version and python version, when possible. It then updates the tests to use `unittest.mock` when `mock` isn't available.

* Do not require mock in Python 3 in certbot-dns modules

* update changelog

* error when trying to build wheels with old setuptools

* add type: ignores
2020-04-15 11:54:44 -07:00
ohemorangeandGitHub af21d1d56e Do not require mock in Python 3 in certbot-compatibility-test module (#7899)
* Do not require mock in Python 3 in certbot-compatibility-test module

* error when trying to build wheels with old setuptools

* add type: ignores
2020-04-15 11:40:03 -07:00
ohemorangeandGitHub 49912732ac Do not require mock in Python 3 in nginx module (#7898)
* Do not require mock in Python 3 in nginx module

* error when trying to build wheels with old setuptools

* add type: ignores
2020-04-15 11:39:44 -07:00
ohemorangeandGitHub 8fb9a395ab Do not require mock in Python 3 in apache module (#7896)
Part of #7886.

This PR conditionally installs mock in `apache/setup.py` based on setuptools version and python version, when possible. It then updates `apache` tests to use `unittest.mock` when `mock` isn't available.

* Conditionally install mock in apache

* error out on newer python and older setuptools

* error when trying to build wheels with old setuptools

* use unittest.mock when third-party mock isn't available in apache, with no cover and type ignore
2020-04-15 11:30:08 -07:00
ohemorangeandGitHub 35fb99b86f Do not require mock in Python 3 in certbot module (#7911)
This PR is exactly the same as #7895, but know we know a little bit more about what was going on with `mypy`.

Part of #7886.

This PR conditionally installs mock in `certbot/setup.py` based on setuptools version and python version, when possible. It then updates `certbot` tests to use `unittest.mock` when `mock` isn't available.

* Conditionally install mock in certbot

* use unittest.mock when third-party mock isn't available in certbot

* Add type:ignores because of https://github.com/python/mypy/issues/1153

* error out on newer python and older setuptools

* error when trying to build wheels with old setuptools
2020-04-15 11:28:47 -07:00
ohemorangeandGitHub 127d2dc307 Do not require mock in Python 3 in acme module (#7910)
Part of #7886.

This PR conditionally installs mock in `acme/setup.py` based on setuptools version and python version, when possible. It then updates `acme` tests to use `unittest.mock` when `mock` isn't available.

Now with `type: ignore` as appropriate. Once the "future steps" of #7886 are finished, and mypy is on Python 3, the `pragma no cover`s and `type ignore`s will be gone.

* Conditionally install mock in acme

* error out on newer python and older setuptools

* error when trying to build wheels with old setuptools

* use unittest.mock when third-party mock isn't available in acme, with no cover and type ignore
2020-04-15 11:27:55 -07:00
ohemorangeandGitHub ff732bf975 Revert the last two mock PRs (#7903)
* Revert "Do not require mock in Python 3 in certbot module (#7895)"

This reverts commit 77871ba71c.

* Revert "Do not require mock in Python 3 in acme module (#7894)"

This reverts commit cd0acf5dcc.
2020-04-13 17:09:24 -07:00
ohemorangeandGitHub 77871ba71c Do not require mock in Python 3 in certbot module (#7895)
Part of #7886.

This PR conditionally installs mock in `certbot/setup.py` based on setuptools version and python version, when possible. It then updates `certbot` tests to use `unittest.mock` when `mock` isn't available.

* Conditionally install mock in certbot

* use unittest.mock when third-party mock isn't available in certbot

* Add type:ignores because of https://github.com/python/mypy/issues/1153

* error when trying to build wheels with old setuptools
2020-04-13 14:33:23 -07:00
ohemorangeandGitHub cd0acf5dcc Do not require mock in Python 3 in acme module (#7894)
Part of #7886.

This PR conditionally installs mock in acme/setup.py based on setuptools version and python version, when possible. It then updates acme tests to use unittest.mock when mock isn't available.

* Conditionally install mock in acme

* use unittest.mock when third-party mock isn't available in acme

* error when trying to build wheels with old setuptools
2020-04-13 14:32:22 -07:00
ohemorangeandGitHub 316e4640f8 Upgrade the test farm tests to use Python 3 (#7876)
Fixes #7857.

* stop using urllib2 in test farm tests

* use six for urllib instead

* remove fabric lcd usage

* correct lcd removal

* remove fabric cd

* convert some remote calls to v2

* move more cxns to v2

* get run working with prefix

* get sudo commands working

* remove final fabric v1 references including local

* update requirements and README

* add new venv to gitignore

* update version used in travis

* remove deploy_script unused kwargs

* fix killboulder implementation so I can test creating a new boulder server

* hardcode the gopath due to broken env manamagement in fabric2

* Update letstest readme

* move the comment about hardcoding the ggopath

* catch BaseException instead of Exception

* work around fabric #2007

* use connections as context managers to ensure they're closed

* remove reference to virtualenv
2020-04-09 14:35:47 -07:00
ohemorangeandGitHub dbda499b08 Remove interactive redirect ask (#7861)
Fixes #7594.

Removes the code asking interactively if the user would like to add a redirect.

* Remove interactive redirect ask

* display.enhancements is no longer used, so remove it.

* update changelog

* remove references to removed display.enhancements

* add redirect_default flag to enhance_config to conditionally set default for redirect value

* Update default in help text.
2020-03-31 12:12:14 -07:00
ohemorangeandGitHub 9e3c348dff Disable TLS session tickets in Apache (#7771)
Fixes #7350.

This PR changes the parsed modules from a `set` to a `dict`, with the filepath argument as the value. Accordingly, after calling `enable_mod` to enable `ssl_module`, modules now need to be re-parsed, so call `reset_modules`.

* Add mechanism for selecting apache config file, based on work done in #7191.

* Check OpenSSL version

* Remove os imports

* debian override still needs os

* Reformat remaining apache tests with modules dict syntax

* Clean up more apache tests

* Switch from property to method for openssl and add tests for coverage.

* Sometimes the dict location will be None in which case we should in fact return None

* warn thoroughly and consistently in openssl_version function

* update tests for new warnings

* read file as bytes, and factor out the open for testing

* normalize ssl_module_location path to account for being relative to server root

* Use byte literals in a python 2 and 3 compatible way

* string does need to be a literal

* patch builtins open

* add debug, remove space

* Add test to check if OpenSSL detection is working on different systems

* fix relative test location for cwd

* put </IfModule> on its own line in test case

* Revert test file to status in master.

* Call augeas load before reparsing modules to pick up the changes

* fix grep, tail, and mod_ssl location on centos

* strip the trailing whitespace from fedora

* just use LooseVersion in test

* call apache2ctl on debian systems

* Use sudo for apache2ctl command

* add check to make sure we're getting a version

* Add boolean so we don't warn on debian/ubuntu before trying to enable mod_ssl

* Reduce warnings while testing by setting mock _openssl_version.

* Make sure we're not throwing away any unwritten changes to the config

* test last warning case for coverage

* text changes for clarity
2020-03-23 16:49:52 -07:00
ohemorangeandGitHub 68f4ae12be Merge pull request #7831 from certbot/candidate-1.3.0
Update files from 1.3.0 release
2020-03-03 17:34:31 -08:00
ohemorangeandGitHub c5a2ba03da Merge pull request #7735 from certbot/apache-parser-v2
[Apache v2] Merge apache-parser-v2 feature branch back to master
2020-02-06 15:29:28 -08:00
ohemorangeandGitHub bca73f9932 Grammar improvements (#18)
Update the README with improved grammar.
2020-02-05 16:08:57 -08:00
Erica Portnoy a180d5d5c9 Release version v1.2.0 2020-02-04 15:34:00 -08:00
Erica Portnoy 6a4b610269 Bump version to 1.3.0 2020-02-04 14:01:04 -08:00
Erica Portnoy 97ae63efa6 Add contents to certbot/CHANGELOG.md for next version 2020-02-04 14:01:03 -08:00
Erica Portnoy 3907b53b4b Release 1.2.0 2020-02-04 14:01:02 -08:00