interfaces.GenericUpdater and new enhancement interface updater functions get run on every invocation of Certbot with "renew" verb for every lineage. This causes performance problems for users with large configurations, because of plugin plumbing and preparsing happening in prepare() method of installer plugins. This PR moves the responsibility to call prepare() to the plugin (possibly) implementing a new style enhancement interface.
Fixes: #6153
* Do not call IPlugin.prepare() for updaters when running renew
* Check prepare called in tests
* Refine pydoc and make the function name more informative
* Verify the plugin type
If either --cert-name or both --key-path and --cert-path (in which case the user requests installation for a certificate not managed by Certbot) are not provided, prompt the user with managed certificates and let them choose.
Fixes: #5824
* Reuse accounts made with ACMEv1 when using an ACMEv2 Let's Encrypt server. This commit turns the feature on for the production server; the bulk of the work was done in 8e4303a.
* add upgrade test for production server
This PR adds the functionality to enhance Apache configuration to include HTTP Strict Transport Security header with a low initial max-age value.
The max-age value will get increased on every (scheduled) run of certbot renew regardless of the certificate actually getting renewed, if the last increase took place longer than ten hours ago. The increase steps are visible in constants.AUTOHSTS_STEPS.
Upon the first actual renewal after reaching the maximum increase step, the max-age value will be made "permanent" and will get value of one year.
To achieve accurate VirtualHost discovery on subsequent runs, a comment with unique id string will be added to each enhanced VirtualHost.
* AutoHSTS code rebased on master
* Fixes to match the changes in master
* Make linter happy with metaclass registration
* Address small review comments
* Use new enhancement interfaces
* New style enhancement changes
* Do not allow --hsts and --auto-hsts simultaneuously
* MyPy annotation fixes and added test
* Change oldest requrements to point to local certbot core version
* Enable new style enhancements for run and install verbs
* Test refactor
* New test class for main.install tests
* Move a test to a correct test class
- Finishing refactor of postconf/postfix command-line utilities
- Plugin uses starttls_policy plugin to specify per-domain policies
Cleaning up TLS policy code.
Print warning when setting configuration parameter that is overridden by master.
Update client to use new policy API
Cleanup and test fixes
Documentation fix
smaller fixes
Policy is now an enhancement and reverting works
Added a README, and small documentation fixes throughout
Moving testing infra from starttls repo to certbot-postfix
fixing tests and lint
Changes against new policy API
starttls-everywhere => starttls-policy
testing(postfix): Added more varieties of certificates to test against.
Moar fixes against policy API.
Address comments on README and setup.py
Address small comments on postconf and util
Address comments in installer
Python 3 fixes and Postconf tester extends TempDir test class
Mock out postconf calls from tests and test coverage for master overrides
More various fixes. Everything minus testing done
Remove STARTTLS policy enhancement from this branch.
sphinx quickstart
99% test coverage
some cleanup and testfixing
cleanup leftover files
Remove print statement
testfix for python 3.4
Revert dockerfile change
mypy fix
fix(postfix): brad's comments
test(postfix): coverage to 100
test(postfix): mypy
import mypy types
fix(postfix docs): add .rst files and fix build
fix(postfix): tls_only and server_only params behave nicely together
some cleanup
lint
fix more comments
bump version number
* Revert "Revert "switch signature verification to use pure cryptography (#6000)" (#6074)"
This reverts commit 3cffe1449c.
* Fixes#6073.
This silences the deprecation warnings from cryptography. I looked into only
silencing the cryptography warning specifically in the function, however,
CryptographyDeprecationWarning doesn't seem to be publicly documented, so we
probably shouldn't depend on it.
* Reuse ACMEv1 accounts for ACMEv2
* Correct behavior
* add unit tests
* add _find_all_inner to comply with interface
* acme-staging-v01 --> acme-staging
* only create symlink to previous account if there is one there
* recurse on server path
* update tests and change internal use of load to use server_path
* fail gracefully on corrupted account file by returning [] when rmdir fails
* only reuse accounts in staging for now
* Initial work on new version of --reuse-key
* Test for reuse_key
* Make lint happier
* Also test a non-dry-run reuse_key renewal
* Test --reuse-key in boulder integration test
* Better reuse-key integration testing
* Log fact that key was reused
* Test that the certificates themselves are different
* Change "oldkeypath" to "old_keypath"
* Simply appearance of new-key generation logic
* Reorganize new-key logic
* Move awk logic into TotalAndDistinctLines function
* After refactor, there's now explicit None rather than missing param
* Indicate for MyPy that key can be None
* Actually import the Optional type
* magic_typing is too magical for pylint
* Remove --no-reuse-key option
* Correct pylint test disable
When Certbot is run with --dry-run, skip running GenericUpdater and RenewDeployer interface methods.
This PR also makes the parameter order of updater.run_generic_updaters and updater.run_renewal_deployer consistent.
Fixes#5927
* Do not call updaters and deployers when run with --dry-run
* Use ConfigTestCase instead of mocking config objects manually
In order to give more flexibility for plugins using interfaces.GenericUpdater interface, lineage needs to be passed to the updater method instead of individual domains. All of the (present and potential) installers do not work on per domain basis, while the lineage does contain a list of them for installers which do.
This also means that we don't unnecessarily run the updater method multiple times, potentially invoking expensive tooling up to $max_san_amount times.
* Make GenericUpdater use lineage as parameter and get invoked only once per lineage
This change will allow registering/updating account with multi emails.
Detail is enclosed in #4242
* support multi emails register
* add more test cases
* update test to unregister before register
* update create path to support multi emaill
* refactor payload updating
* fix typo
* move command line doc to another place
* revert the change for updating account registration info, added unit test
* rearrange text for consistency
* switch signature verification to use pure cryptography
On systems that prevent write/execute pages this prevents a segfault
that is caused by pyopenssl creating a dynamic callback in the
verification helper.
* switch to using a verifier for older cryptography releases
also add ec support, test vectors, and a test
* If --dry-run is used and there exists no staging account, create account with no email
* added unit testing of dry-run to ensure certbot does not ask the user to create an email, and that certbot creates an account with no email
* Remove unsupported pylint disable options
* star-args removed in Pylint 1.4.3
* abstract-class-little-used removed in Pylint 1.4.3
* Fixes new lint errors
* Copy dummy-variable-rgx expression to new ignored-argument-names expression to ignore unused funtion arguments
* Notable changes
* Refactor to satisfy Pylint no-else-return warning
* Fix Pylint inconsistent-return-statements warning
* Refactor to satisfy consider-iterating-dictionary
* Remove methods with only super call to satisfy useless-super-delegation
* Refactor too-many-nested-statements where possible
* Suppress type checked errors where member is dynamically added (notably derived from josepy.JSONObjectWithFields)
* Remove None default of func parameter for ExitHandler and ErrorHandler
Resolves#5973
* Fixes#5570.
The issue is calls to atexit aren't mocked out. During the tests there are many
repeated calls registering functions to be called when the process exits so
when the tests finishes, it prints a ton of output from running those
registered functions. This suppresses that by mocking out atexit.
* Mock at a lower level.
This ensures we don't mess with any other mocks in this test class by mocking
at the lowest level we can. Other tests shouldn't be mocking out specific
internals of functions in other modules, so this should work just fine.
* Fix crash when email submission endpoint unavailable
Handle KeyError and ValueError so that if the email submission endpoint
goes down, Certbot can still run.
Add tests to eff_test.py:
- simulate non-JSON response as described in issue #5858
- simulate JSON response without 'status' element
Non-JSON response throws an uncaught ValueError when attempting to
decode as JSON. A JSON response missing the 'status' element throws an
uncaught KeyError when checking whether status is True or False.
Teach _check_response to handle ValueError and KeyError and report an
issue to the user.
Rewrite if statement as assertion with try-except block to make error
handling consistent within the function. Update test_not_ok to make
mocked raise_for_status function raise a requests.exceptions.HTTPError.
Resolves#5858
* Update PR with requested changes
- Use `if` instead of `assert` to check `status` element of response JSON
- Handle KeyError and ValueError in the same way
- Import requests at the beginning of eff_test.py
- Clear JSON in test case in a more idiomatic way
Include a line break before "(None)" to maintain consistency with output
for lists that are not empty.
Previous result as expected for non-empty lists:
>>> _format_list('+', ['one', 'two', 'three'])
'\n+ one\n+ two\n+ three'
Previous unexpected result for empty lists:
>>> _format_list('+', [])
'\n+ '
New result as expected (unchanged) for non-empty lists:
>>> _format_list('+', ['one', 'two', 'three'])
'\n+ one\n+ two\n+ three'
New behavior more explicit for empty lists:
>>> _format_list('+', [])
'\n(None)'
Resolves#5886
* ServerTLSUpdater and InstallerSpecificUpdater implementation
* Fixed tests and added disables for linter :/
* Added error logging for misconfigurationerror from plugin check
* Remove redundant parameter from interfaces
* Renaming the interfaces
* Finalize interface renaming and move tests to own file
* Refactored the runners
* Refactor the cli params
* Fix the interface args
* Fixed documentation
* Documentation and naming fixes
* Remove ServerTLSConfigurationUpdater
* Remove unnecessary linter disable
* Rename run_renewal_updaters to run_generic_updaters
* Do not raise exception, but make log message more informative and visible for the user
* Run renewal deployer before installer restart
The base class for Installer plugins `certbot.plugins.common.Installer` now provides functionality of `PluginStorage` to all installer plugins. This allows a plugin to save and retrieve variables in between of invocations.
The on disk storage is basically a JSON file at `config_dir`/`.pluginstorage.json`, usually `/etc/letsencrypt/.pluginstorage.json`. The JSON structure is automatically namespaced using the internal plugin name as a namespace key. Because the actual storage is JSON, the supported data types are: dict, list, tuple, str, unicode, int, long, float, boolean and nonetype.
To add a variable from inside the plugin class:
`self.storage.put("my_variable_name", my_var)`
To fetch a variable from inside the plugin class:
`my_var = self.storage.fetch("my_variable_key")`
The storage state isn't written on disk automatically, but needs to be called:
`self.storage.save()`
* Plugin storage implementation
* Added config_dir to existing test mocks
* PluginStorage test cases
* Saner handling of bad config_dir paths
* Storage moved to Installer and not initialized on plugin __init__
* Finetuning and renaming
* Revert "document default is ACMEv2 (#5818)"
This reverts commit 2c502e6f8b.
* Revert "Update default to ACMEv2 server (#5722)"
This reverts commit 4d706ac77e.
