Second part of #4443. Built on #4444. Fixes#3148.
This fixes an old problem with code logging messages before logging has been set up. How this works is explained in the docstring of certbot.log.pre_arg_setup.
* add memory handler
* Add exit_with_log_path
* add new_except_hook
* pre_arg_parse_setup++ and remove old except_hook
* Rewrite post_arg_setup
* test restricted permissions
* move changes to main
* Use .name of NamedTemporaryFile
* use better assertions
* set exc_info in except_hook
* Make post_arg_setup more robust
* final cleanup
* Add TempHandler
* undo main_test changes
* improve documentation
* use decorators instead of with for mock.patch
* add inline comment about logging.shutdown
It's relatively finicky to make a CSR appropriate to pass to poll_and_request_issuance. I think most users want to be able to give a list of domains and a private key, and get back a CSR. This branch adds that functionality to crypto_util.
Note that the two new functions take arguments, and return values, as PEM-encoded buffers. This is a departure from some existing ACME interfaces that take PyOpenSSL types. I've discussed with the Certbot team, and we agree that this is broadly the direction the ACME API should take, so that users of the module don't need to import PyOpenSSL themselves, or use its primitives.
* Add make_csr.
* accept privkey
* Tweak API.
* Remove make_csr from certbot package.
* Skip test in older Pythons.
* Move get_Extensions call under protection.
* Remove assertIn because not backwards-compatible.
* Fix encoding, and use PEM.
* Fix test
* Fix tests on py35.
* Fix error in test.
* Make import_csr_file always return PEM.
Also delete get_sans_from_csr (unused) and get_names_from_csr (newly unused).
* Fix function doc.
* Fix indent
* Fix call of obtain_certificate_from_Csr
* lint
* Handle review feedback.
* Fix test.
In some sense, certbot.client.Client is an ACME client, but it's the not the client in the ACME library and this leads to confusion. Let's make what this is clear.
* call certbot client cb_client rather than acme_client
* update tests
* rename more files in rename command
* Revert "Hide rename command (#4007)"
This reverts commit 8c14de13a5.
* Rename files in configuration files
* Delete new files if we fail during the renaming process
* update tests and error catching
* More expressive error message
* add fasteners as a dependency
* add LOCK_FILE constant
* Add lock file to Certbot
* Move code to _run_subcommand
* move lock file path into CLI_CONSTANTS
* add --lock-path flag
* move locking code to separate function
* Add TestAcquireFileLock
* assert we log
* test lock contention
* add fasteners to certbot-auto
* Use a different lock file for each test in MainTest
An early version of the spec indicated that clients should process issuance
sequentially, following Link rel=next from an account URL to an authz URL, to a
new-cert URL. However, the spec has long since moved to putting these URLs in
the directory.
Certbot nominally supports either; This change consolidates on always using the
directory, simplifying things and making the transition to the latest ACME spec
easier.
* Revert "Revert "Remove Link rel=next for authzs and new-certs." (#4277)"
This reverts commit 11ec1eb911.
* Save new_authzr_uri with account for older clients.
* Add test that new_authzr_uri exists in regr.
* Restore backwards compatibility for new_authzr_uri.
* Fix account_test.
* Add test for deprecated URI argument to request_challenges.
* Review feedback.
* Fix test
* Add omitempty to new_cert_uri.
* Refactor main to simplify logic
* Update tests and comments
* Correct main test
* increase timeout limit
* reset timeout limit
* call renew_cert in appropriate main test
* Update docstrings and revert signatures of _report_new_cert and _suggest_donation_of_appropriate
* replace renew_cert logic
* update tests
* rename _csr_obtain_cert and add a check to _report_new_cert
This includes two new tests in the integration test script to check that
boulder gets the correct code. The encoding is specified in RFC5280
5.3.1. The codes that boulder will accept are a subset of that,
specified in `boulder.revocation.reasons.go`.
* CLI flag for forcing interactivity
* add --force-interactive
* Add force_interactive error checking and tests
* Add force_interactive parameter to FileDisplay
* add _can_interact
* Add _return_default
* Add **unused_kwargs to NoninteractiveDisplay
* improve _return_default assertion
* Change IDisplay calls and write tests
* Document force_interactive in interfaces.py
* Don't force_interactive with a new prompt
* Warn when skipping an interaction for the first time
* add specific logger.debug message
* organize cert_manager.py
* add delete files to cert manager and storage
* add tests
* add to main and cli
* Clean up all related files we can find, even if some are missing.
* error messages, debug logs, and remove RenewerConfiguration
* add logs for failure to remove
* remove renewer_config_file
* Start reorganising -h output
* Fix the --debug flag
- Currently exceptions are often caught and burried in log files, even
if this flag is provided!
* Explain the insanity
* Parallalelise nosetests from tox (#3836)
* Parallalelise nosetests from tox
* Parallelise even more things, break even more things
* Now unbreak all the tests that aren't ready for ||ism
* Try to pass tests!
- Remove non-working hack in reporter_test
- also be selective about ||ism in the cover environment
* Try again
* certbot-apache tests also work, given enough time
* Nginx may need more time in Travis's cloud
* Unbreak reporter_test under ||ism
* More timeout
* Working again?
* This goes way faster
* Another big win
* Split a couple more large test suites
* A last improvement
* More ||ism!
* ||ise lint too
* Allow nosetests to figure out how many cores to use
* simplify merge
* Mark the new CLI tests as ||izable
* Simplify reporter_test changes
* Rationalise ||ism flags
* Re-up coverage
* Clean up reporter tests
* Stop modifying testdata during tests
* remove unused os
* Improve the "certbot certificates" output (#3846)
* Begin making "certbot certificates" future safe
* Handle the case where a renewal conf file has no "server" entry
* Improvements, tweaks
* Capitalise on things
* Print the command summary for -h and -h all, but not otherwise
Also, update nginx not installed CLI hint
* Add a "certificates" help section
* Clean up usage string construction
* Greatly improve "certbot -h TOPIC"
- subcommands now get their own usage headings if they want them
- added "certbot -h commands"
* A few more cli formatting tests
* Auto-populate the verb subgroups from the docs
* Show the new help output
* Lint, tweak
* More lint, and cleanup
* Infinite lint
* Add rename to command summary; sort "-h commands" output
* Use fancy string formatting
* More space
* Implement --help manage
Also, implement a general mechanism for documenting subcommands within
topics
* Remove one comma
* Only create weird parser structures if -h is provided :)
* Update sample cli out
* Lint
* Revert cli-help.txt to previous release version
* Grammar & style
* Fix the --debug flag
- Currently exceptions are often caught and burried in log files, even
if this flag is provided!
* Explain the insanity
* Make things slightly nicer
* Rename and simplify main functions
* pass certname to auth method
* find cert by certname flag
* Implement --cert-name command
* don't ask to confirm new cert when we have domains and no existing certs with the lineage name
* Refactor and add --new-cert-name flag
* add interactivity to rename verb
* allow noninteractive and more descriptive function names
* Begin breaking out cli_test.py
* simplify main
* refactor porse tests
* move determine account tests to main_test.py
* move duplicate cert test to main_test.py
* move cli stuff out of the way
* add test_renewal.py
* move error test into error_handler_test.py
* move test_read_file
* move test_no_gui out of MainTest
* move test_install_abspath to parsetest
* Move main tests into main_test.py
* move cli tests back into cli_test.py
* clean up cli_test.py
* move punycode test to util_test.py
* Fix NameError from missing plugins_disco
* Fix linting errors
* test_renewal.py -> renewal_test.py
* rm not_cli_test.py
* Move main._handle_exception test to main_test.py
* Move renewal import in renewal_test.py
from @ohemorange comments
* certbot.tests.test_util -> certbot.tests.util
* Fix issues from rebasing.
* Fix testing issue with option_was_set
* fix linting issue
* Stop passing around config and refactor tests
* Refactor and warn during enhance_config
* Use mock.ANY to make new Pythons happy
* Remove verbose enhance_config from test names
* Fix spacing in warning
* Output status for `revoke` operation. Fixes#2819.
- Added method to `certbot.display.ops` to output confirmation of `revoke`.
- Wrapped call to `acme.client.Client.revoke` in a try to statement to
handle possible error.
- Added test for `main.revoke`.
* Added test for failure of certificate revocation.
Moved creation of mocks into RevokeTest setup function.
Stopped mocks in RevokeTest teardown function.
* Fixed lint errors.
* Do not call `unittest.TestCase.assertRaises` as a context manager (to work with py26).
* Fixed spelling error in successful revocation notification.
Added test for the notification.
* Changed informational messages because of confusing message on reinstallation.
Certbot prompts the user when it detects that an appropriately fresh certificate
is already available:
You have an existing certificate that contains exactly the same domains you requested and isn't close to expiry.
(ref: <path>)
What would you like to do?
-------------------------------------------------------------------------------
1: Attempt to reinstall this existing certificate
2: Renew & replace the cert (limit ~5 per 7 days)
-------------------------------------------------------------------------------
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1
On selecting '1' (reinstall), the resulting message is:
-------------------------------------------------------------------------------
Your existing certificate has been successfully reinstalled, and the new
certificate has been installed.
The new certificate covers the following domains: https://<whatever>
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=<whatever>
-------------------------------------------------------------------------------
"Your existing certificate has been successfully reinstalled" <-- Okay
"and the new certificate has been installed." <-- Wait, what?
The issue appears to come from assumptions in certbot/certbot/main.py
It uses `len(lineage.available_versions("cert"))` to determine if this was a
fresh install or renewal, and then calls either `display_ops.success_renewal()`
(which produces the "existing certificate ... and the new certificate" language)
or `display_ops.success_installation()` (which has no messaging about existing
vs. new certificates).
The len(lineage) test isn't the right way to make this choice. The certificate's
lineage length doesn't imply anything about whether we've just obtained a new
certificate, because there is no new certificate in the case of a "reinstall"
action.
The new logic calls `display_ops.success_installation()` on all "reinstall"
actions, and otherwise employs the existing `len(lineage)` test.
Additionally the `display_ops.success_installation()` has been enhanced to
accept an action parameter, and has the message reworded slightly to make
sense regardless of the action passed. The messaging is mostly unchanged if it's
called without the action parameter:
Original message:
-------------------------------------------------------------------------------
Congratulations! You have successfully enabled https://<whatever>
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=<whatever>
-------------------------------------------------------------------------------
New message on initial install:
-------------------------------------------------------------------------------
Congratulations! You have successfully installed a certificate for
https://<whatever>
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=<whatever>
-------------------------------------------------------------------------------
New message on re-install:
-------------------------------------------------------------------------------
Congratulations! You have successfully reinstalled a certificate for
https://<whatever>
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=<whatever>
-------------------------------------------------------------------------------
* Typo in display message.
* Typo, characters transposed.
* undo changes to certbot/display/ops.py
* remove invalid todos
* Test success_installation() called for reinstall
* Simplify display_ops.success* functions
* refactor and expand run() tests
* Switch to using absolute path in symlink
* save archive_dir to config and read it back
* cli_config.archive_dir --> cli_config.default_archive_dir
* Use archive_dir specified in renewal config file
* add helpful broken symlink info
* add docstring to method
* Add tests
* remove extraneous test imports
* fix tests
* py2.6 syntax fix
* git problems
* Add list-certs command
* no dict comprehension in python2.6
* add test coverage
* More py26 wrangling
* update tests for py3 and lint
* remove extra dep from test
* test coverage
* test shouldn't be based on dict representation order
* Redo report UX and add tests to cover
* remove storage str test
* lint and use mock properly
* mock properly
* address code review comments
* lineage --> certificate name and print fullchain and privkey paths
* make py26 happy
* actually make py26 happy
* don't wrap text
* Switch to using absolute path in symlink
* save archive_dir to config and read it back
* cli_config.archive_dir --> cli_config.default_archive_dir
* Use archive_dir specified in renewal config file
* add helpful broken symlink info
* add docstring to method
* Add tests
* remove extraneous test imports
* fix tests
* py2.6 syntax fix
* git problems
* no dict comprehension in python2.6
* add test coverage
* More py26 wrangling
* reduce logging level and ignore verbose flags in quiet mode
* Simplify setup_logging parameters
The extra parameters were there in the past when the letsencrypt-renewer was a
separate executable that also used this function. This is cruft that can be
removed.
* Add basic tests for setup_logging
Re-do the fix for #3497 to ensure it works in all cases.
* If lineages are in an inconsistent (non-deployed) state, deploy them
* Test new _handle_identical_cert case
* Move lineage.has_pending_deployment() check up to _auth_from_domains
Less conceptually nice, but in the "renew" verb case it wasn't being called :(
* Swap _auth_from_domains return type
* It now matches _treat_as_renewal & _handle_identical_cert_request etc
* Revert "Move lineage.has_pending_deployment() check up to _auth_from_domains"
This reverts commit a7fe734d73.
* Move test back to handle_identical_cert_request
* We need to check for non-deployment on two separate code paths
- Once high up in "renew" (because failure to be deployed stops us from
divind down the stack)
- Once way down in _handle_identical_cert_request (because that's where it
makes the most sense for run / certonly)
- So refactor that work into storage.py
* We don't necessarily reinstall
This is a quick fix to https://github.com/certbot/certbot/issues/3382, so that
Python log rotation is less likely to delete files before logrotate can get to
them. Specifically, if logrotate's config says to preserve fewer than 1000 logs,
it will do the right thing.
