16 Commits
Author SHA1 Message Date
Adrien FerrandandBrad Warren d5de24d9fc [Windows] Security model for files permissions - STEP 2 (#6895)
This PR is the second part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)

This PR creates the module certbot.compat.os, that delegates everything to os, and that will be the safeguard against problematic methods of the standard module. On top of that, a quality check wrapper is called in the lint tox environment. This wrapper calls pylint and ensures that standard os module is no used directly in the certbot codebase.

Finally local oldest requirements are updated to ensure that tests will take the new logic when running.

* Add executable permissions

* Add the delegate certbot.compat.os module, add check coding style to enforce usage of certbot.compat.os instead of standard os

* Load certbot.compat.os instead of os

* Move existing compat test

* Update local oldest requirements

* Import sys

* Update account_test.py

* Update os.py

* Update os.py

* Update local oldest requirements

* Implement the new linter_plugin

* Fix local oldest for nginx

* Remove check coding style

* Update linter_plugin.py

* Add several comments

* Update the setup.py

* Add documentation

* Update acme dependencies

* Update certbot/compat/os.py

* Update docs/contributing.rst

* Update linter_plugin.py

* Handle os.path. Simplify checker.

* Add a comment to a reference implementation

* Update changelog

* Fix module registering

* Update docs/contributing.rst

* Update config and changelog
2019-04-12 13:32:51 -07:00
Adrien Ferrand 4515a52d3f Merge branch 'master' into pylint
# Conflicts:
#	acme/acme/client.py
#	acme/acme/crypto_util.py
#	acme/acme/standalone.py
#	certbot-apache/certbot_apache/configurator.py
#	certbot-apache/certbot_apache/parser.py
#	certbot-apache/certbot_apache/tests/tls_sni_01_test.py
#	certbot-apache/certbot_apache/tests/util.py
#	certbot-apache/certbot_apache/tls_sni_01.py
#	certbot-nginx/certbot_nginx/configurator.py
#	certbot-nginx/certbot_nginx/parser.py
#	certbot-nginx/certbot_nginx/tests/util.py
#	certbot/account.py
#	certbot/cert_manager.py
#	certbot/cli.py
#	certbot/configuration.py
#	certbot/main.py
#	certbot/ocsp.py
#	certbot/plugins/dns_common_lexicon.py
#	certbot/plugins/standalone.py
#	certbot/plugins/util.py
#	certbot/plugins/webroot.py
#	certbot/tests/auth_handler_test.py
#	certbot/tests/cert_manager_test.py
#	certbot/tests/display/util_test.py
#	certbot/tests/main_test.py
#	certbot/tests/util.py
#	certbot/util.py
#	tox.ini
2019-04-02 22:32:01 +02:00
Adrien FerrandandBrad Warren 6ce6c67932 [Windows] Security model for files permissions - STEP 1 (#6893)
This PR is the first part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)

This step 1 refactor existing certbot.compat module into certbot.compat.misc, without any logic changed. Package certbot.compat will host the new modules that constitute the security model for Windows.

* Create the certbot.compat package. Move logic in certbot.compat.misc

* Add doc

* Fix lint

* Correct mypy

* Update client.py
2019-03-28 15:51:48 -07:00
Adrien FerrandandBrad Warren 85a859d63f Make Certbot runnable on Windows (#6296)
* Add and use a compatibility layer to allow certbot to be run on windows.

* Fix path comparison

* Corrections on compat and util for tests

* Less intrusive way to parse prefix in webroot plugin working for both linux and windows.

* Disable pylint import-error for some optional imports in compat.py

* Ensure path is normalized before prefixes are generated in webroot plugin

* Same prefixes in linux and windows, in fact root path is not needed in webroot plugin

* Check that user has administrative rights before continuing on windows (necessary for symlink creation)

* More straightforward way to test administrative rights on windows

* Try to resolve import error in travis ci

* OK. We go for full introspection to trick the ci.

* Move the administrative rights control to the certbot entrypoint

* Add comment for a really non trivial code.

* Allow some commands to be run on a shell without admin rights

* Avoid races conditions on windows for lock files

* Add sphinx doc to the compat functions.

* Remove irrelevant Windows error in the lock mechanism.

* Some corrections on compat
2018-09-08 07:34:27 -07:00
Dmitry FigolandBrad Warren 36dfd06503 Prepare certbot module for mypy check untyped defs (#6005)
* Prepare certbot module for mypy check untyped defs

* Fix #5952

* Bump mypy to version 0.600 and fix associated bugs

* Fix pylint bugs after introducing mypy

* Implement Brad's suggestions

* Reenabling pylint and adding nginx mypy back
2018-05-18 06:28:17 -07:00
James Payne 5300d7d71f Fix Pylint upgrade issues
* Remove unsupported pylint disable options
    * star-args removed in Pylint 1.4.3
    * abstract-class-little-used removed in Pylint 1.4.3

* Fixes new lint errors

* Copy dummy-variable-rgx expression to new ignored-argument-names expression to ignore unused funtion arguments

* Notable changes
    * Refactor to satisfy Pylint no-else-return warning
    * Fix Pylint inconsistent-return-statements warning
    * Refactor to satisfy consider-iterating-dictionary
    * Remove methods with only super call to satisfy useless-super-delegation
    * Refactor too-many-nested-statements where possible
    * Suppress type checked errors where member is dynamically added (notably derived from josepy.JSONObjectWithFields)
    * Remove None default of func parameter for ExitHandler and ErrorHandler

Resolves #5973
2018-05-16 20:37:39 +00:00
sydneyliandBrad Warren bca0aa48c2 logging: log timestamps as local timezone instead of UTC (#5607)
* logging: log timestamps as local timezone instead of UTC

* test(logging): expect localtime instead of gmtime

* linter fix in logging
2018-03-21 15:41:33 -07:00
HugoandBrad Warren 42638afc75 Drop support for EOL Python 2.6 and 3.3
* Drop support for EOL Python 2.6

* Use more helpful assertIn/NotIn instead of assertTrue/False

* Drop support for EOL Python 3.3

* Remove redundant Python 3.3 code

* Restore code for RHEL 6 and virtualenv for Py2.7

* Revert pipstrap.py to upstream

* Merge py26_packages and non_py26_packages into all_packages

* Revert changes to *-auto in root

* Update by calling letsencrypt-auto-source/build.py

* Revert permissions for pipstrap.py
2018-02-16 16:14:01 -08:00
r5dandBrad Warren b9d129bd43 certbot: Stop using print in log module. (#5160)
* Update certbot.log.post_arg_parse_except_hook function.
* Update certbot.tests.log_test._test_common method.

See discussion #3720.
2017-10-03 12:52:41 -07:00
Brad Warren a7a8e060e3 Finish adding configurable log rotation
* Update log backupCount name and description.

* Add additional error handling to --log-backups

* test --log-backups flag

* Pass log_backups value to RotatingFileHandler

* Test that log_backups is properly used

* add _test_success_common

* Add test_success_with_rollover

* Add test_success_without_rollover

* mock stderr in cli tests

* Set log_backups in PostArgParseSetupTest

* Rename "log backups" to "max log backups"
2017-07-11 21:14:18 -05:00
Brad WarrenandGitHub 93310fe67c Fixes #4719 (#4737)
* Automatically delete temp log file when not used.

This allows close() calls in logging.shutdown() to cause the file to be
deleted when no logging output has been written to the file.

* Make certbot.log.MemoryHandler.flush() a noop.

This causes MemoryHandler.flush() calls in logging.shutdown to be a noop,
allowing us to control when the handler is actually flushed. This prevents log
records from being sent to a temporary file handler for things like
`certbot --version`.

* Keep reference to certbot.log.MemoryHandler.target

In Python 2.7+, the logging module only keeps weak references to created
logging handlers. Because of this, the MemoryHandler's target will not be
properly flushed and closed when logging.shutdown() is called on program exit
unless we keep a reference to it in the MemoryHandler.

* Fixes #4719.

This completes the changes necessary to fix #4719. Now temporary log files are
not created if sys.exit() is called before logging is fully set up. These files
are still created if Certbot crashes for any other reason.

* Document pre_arg_parse_except_hook args.
2017-05-25 12:16:05 -07:00
Brad WarrenandPeter Eckersley 5ca8f7c5b9 Add lockfile (#4449)
* add lock_file

* cleanup lock file

* Add LockFile tests

* add lock_dir

* add lock_dir_until_exit

* add set_up_core_dir and move lock_dir_until_exit

* Move lock_and_call to certbot.test.util

* Add lock to Apache

* Add lock to the Nginx plugin

* Improve permissions error message

* sort plugins

* add test_prepare_order

* provide more actionable permissions error

* Document and catch use of OSError

* don't lock a directory twice

* add conditional dependency on ordereddict

* Add lock_test

* expand sorted plugins comment

* Add lock_test to lint

* make make_lineage more conventional and flexible

* enhance lock_test.py

* add lock_test to tox

* Readd success message

* make py26 happy

* add test_acquire_without_deletion
2017-05-01 14:49:12 -07:00
Brad WarrenandGitHub bd8c31021a Logging setup before argument parsing (#4446)
Second part of #4443. Built on #4444. Fixes #3148.

This fixes an old problem with code logging messages before logging has been set up. How this works is explained in the docstring of certbot.log.pre_arg_setup.

* add memory handler

* Add exit_with_log_path

* add new_except_hook

* pre_arg_parse_setup++ and remove old except_hook

* Rewrite post_arg_setup

* test restricted permissions

* move changes to main

* Use .name of NamedTemporaryFile

* use better assertions

* set exc_info in except_hook

* Make post_arg_setup more robust

* final cleanup

* Add TempHandler

* undo main_test changes

* improve documentation

* use decorators instead of with for mock.patch

* add inline comment about logging.shutdown
2017-04-13 13:42:01 -07:00
Brad WarrenandGitHub e194e0dd5f Refactoring for better logging (#4444)
* Move colored_logging.py to log.py

* Add atexit.register code to util

* Add tests for atexit_register

* Copy except_hook to log

* Add pre_arg_setup

* move setup_log_file_handler to log.py

* Add post_arg_setup

* move changes to main

* Undo changes to MainTest

* s/pre_arg_setup/pre_arg_parse_setup

* s/post_arg_setup/post_arg_parse_setup
2017-03-30 16:17:57 -07:00
Erica PortnoyandBrad Warren d54cb76432 Remove the curses dialog, thereby deprecating the --help and --dialog command line options (#3665)
* Remove the curses dialog, thereby deprecating the --help and --dialog command line options

* Deprecate --dialog and suppress --text
2016-10-21 15:45:57 -07:00
Brad Warren 3961b70deb s/letsencrypt/certbot 2016-04-13 16:03:59 -07:00