Fixes#7212
This PR forbid os.stat and os.fstat, and fix or provide alternatives to avoid its usage in certbot outside of certbot.compat.filesystem.
* Reimplement private key mode propagation
* Remove other os.stat
* Remove last call of os.stat in certbot package
* Forbid stat and fstat
* Implement mode comparison checks
* Add unit tests
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Handle case where multiple ace concerns a given SID in has_min_permissions
* Add a new test scenario
* Add a simple test for has_same_ownership
* Fix name function
* Add a comment explaining an ACE structure
* Move a test in its dedicated class
* Improve a message error
* Calculate has_min_permission result using effective permission rights to be more generic.
* Change an exception message
* Add comments, avoid to skip a test.
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
This PR implements the filesystem.copy_ownership_and_apply_mode method from #6497.
This method is used in two places in Certbot, replacing os.chown, to copy the owner and group owner from a file to another one, and apply to the latter the given POSIX mode.
* Implement copy_ownership_and_apply_mode
* Update certbot/compat/os.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Remove default values
* Rewrite a comment.
* Relaunch CI
* Pass as keyword arguments
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Make the private key permissions transfer platform specific
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Rename variable
* Fix comment0
* Add unit test for copy_ownership_and_apply_mode
* Adapt coverage
* Execute unconditionally chmod with copy_ownership_and_apply_mode. Improve doc.
This PR implements the filesystem.chmod method from #6497.
* Implement filesystem.chmod
* Conditionally add pywin32 on setuptools versions that support environment markers.
* Update apache plugin requirements
* Use a try/except import approach similar to lock
* Add comments about well-known SIDs
* Add main command
* Call filesystem.chmod in tests, remove one test
* Add test for os module
* Update environment marker
* Ensure we are not building wheels using an old version of setuptools
* Added a link to list of NTFS rights
* Simplify sid comparison
* Enable coverage
* Sometimes, double-quote is the solution
* Add entrypoint
* Add unit tests to filesystem
* Resolve recursively the link, add doc
* Move imports to the top of the file
* Remove string conversion of the ACL, fix setup
* Ensure admins have all permissions
* Simplify dacl comparison
* Conditionally raise for windows temporary workaround
* Add a test to check filesystem.chown is protected against symlink loops
* Modifications for misc
* Add some types
* Use os_rename
* Move rename into filesystem
* Use our os package
* Rename filesystem.rename to filesystem.replace
* Disable globally function redefined lint in os module
This PR is the second part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)
This PR creates the module certbot.compat.os, that delegates everything to os, and that will be the safeguard against problematic methods of the standard module. On top of that, a quality check wrapper is called in the lint tox environment. This wrapper calls pylint and ensures that standard os module is no used directly in the certbot codebase.
Finally local oldest requirements are updated to ensure that tests will take the new logic when running.
* Add executable permissions
* Add the delegate certbot.compat.os module, add check coding style to enforce usage of certbot.compat.os instead of standard os
* Load certbot.compat.os instead of os
* Move existing compat test
* Update local oldest requirements
* Import sys
* Update account_test.py
* Update os.py
* Update os.py
* Update local oldest requirements
* Implement the new linter_plugin
* Fix local oldest for nginx
* Remove check coding style
* Update linter_plugin.py
* Add several comments
* Update the setup.py
* Add documentation
* Update acme dependencies
* Update certbot/compat/os.py
* Update docs/contributing.rst
* Update linter_plugin.py
* Handle os.path. Simplify checker.
* Add a comment to a reference implementation
* Update changelog
* Fix module registering
* Update docs/contributing.rst
* Update config and changelog
This PR is the first part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)
This step 1 refactor existing certbot.compat module into certbot.compat.misc, without any logic changed. Package certbot.compat will host the new modules that constitute the security model for Windows.
* Create the certbot.compat package. Move logic in certbot.compat.misc
* Add doc
* Fix lint
* Correct mypy
* Update client.py
Not updating the guidelines in using.rst-- I don't want to encourage people to use this. now that Certbot preserves gid, the better way to set permissions is to change the group permisisons!
* Preserve other-read bit on private keys too
* fix integration test
* fix and rename permission routines in integration tests
Fixes#1473.
writes privkey.pem to 0600 by default for new lineages
on renewals where a new privkey is generated, preserves group mode and gid
Things this PR does not do:
we talked about forcing 0600 on privkeys when a Certbot upgrade is detected. Instead, this PR only creates new lineages with the more restrictive permission to prevent renewal breakages.
this doesn't solve many of the problems mentioned in #1473 that are not directly related to the title issue!
* safe_open on archive keyfiles
* keep group from current lineage
* clean up integration test
* safe_open can follow symlinks
* fix tests on windows, maybe
* Address Brad's comments
* Revert changes to safe_open
* Test chown is called when saving new key
* Reorder chown operation
* Changelog and documentation
* Fix documentation style
On Linux, you can invoke os.rename(src, dst) even if dst already exists. In this case, destination file will be atomically replaced by the source file.
On Windows, this will lead to an OSError because changes are not atomic. This cause certbot renew to fail in particular, because the old certificate configuration needs to be replace by the new when a certificate is effectively renewed.
One could use the cross-platform function os.replace, but it is available only on Python >= 3.3.
This PR add a function in compat to handle correctly this case on Windows, and delegating everything else to os.rename.
* Cross platform compatible os.rename (we can use os.replace if its python 3)
* Use os.replace instead of custom non-atomic code.
* Avoid errors for lint and mypy. Add a test.
We want to discourage people from moving things around in `/etc/letsencrypt/live`! So we dropped an extra README in the `/etc/` directory when it's first created.
* Remove unsupported pylint disable options
* star-args removed in Pylint 1.4.3
* abstract-class-little-used removed in Pylint 1.4.3
* Fixes new lint errors
* Copy dummy-variable-rgx expression to new ignored-argument-names expression to ignore unused funtion arguments
* Notable changes
* Refactor to satisfy Pylint no-else-return warning
* Fix Pylint inconsistent-return-statements warning
* Refactor to satisfy consider-iterating-dictionary
* Remove methods with only super call to satisfy useless-super-delegation
* Refactor too-many-nested-statements where possible
* Suppress type checked errors where member is dynamically added (notably derived from josepy.JSONObjectWithFields)
* Remove None default of func parameter for ExitHandler and ErrorHandler
Resolves#5973
* Change "Attempting to parse" warning to info.
This message shows up on every renewal run when the config was updated
by a newer version of Certbot than the one being run. For instance, if a
user has the certbot packages installed from PPA (currently 0.18.2), but
runs certbot-auto once to try out the latest version (0.21.1), they will
start getting this message via email every 12 hours.
* Switching from old branch (issue-4109) and addressing changes requested
in last iteration of review:
https://github.com/certbot/certbot/pull/4202/files/80aa857fd21d7a6be8e7334777dac19456261b02
Requested changes that were addressed:
- fixed outdated docstring for `cert_path_to_lineage`
- removed `full_archive_dir_from_renewal_conf` amd replaced with `full_archive_path` (and `_full_archive_path` -> `full_archive_path`)
- matching on `cert` instead of `chain` in `cert_manager.cert_path_to_lineage`
- fixed the two coding wrongs make a right issue
Requested changes which were not addressed:
- moving `cert_path_to_lineage` from `cert_manager` to `storage`,
as it would introduce a hard to resolve circular dependency.
* Update integration tests to handle default deletion after revoke.
* Swapping test domains.
* Addressing PR feedback:
- calling storage.full_archive_path with a ConfigObj instead of None
- Removing lambda x: x.chain_path as an option to match against
* Addressing PR feedback: it's expected that len(pattern) is 0, so handle that case properly.
* Testing of conflicting values of --cert-name and --cert-path non-interactive mode.
* Silly test for when neither certname nor cert-path were specified.
* Changing archive_files to a private function, because mocking nested functions seems impossible.
* Tests for storage.cert_path_for_cert_name
* Splitting out _acceptable_matches
* Some tests for cert_manager.cert_path_to_lineage
* Offerings to the Lint God
* Cleaner way of dealing with files in archive dirs
* Handling the two different use cases of match_and_check_overlaps a bit better
* late night syntax errors
* Test for when multiple lineages share an archive dir
* Tests for certbot.cert_manager.match_and_check_overlaps
* Removing unneeded nesting
* Lint errors that Travis caught that didn't show up locally
* Adding two integration tests (matching & mismatched --cert-path, --cert-name) based on feedback.
* Asking the user if they want to delete in interactive mode.
* certbot: Update storage.get_link_target (#4750)
* The `get_link_target` function raises `errors.CertStorageError` when
link does not exists.
* certbot: Fix typo in storage.get_link_target.
Ensure that permissions are preserved when renewal data is written to
conf files. This allows users to limit access to the file, if they wish.
Testing done:
* `tox -e py27`
* `tox -e lint`
* Manual Testing
* Got a new certificate. Restricted the permissions on the renewal
conf. Renewed the certificate. Verified that the new renewal conf
permissions matched.
* Use binary flag when writing cert and key files
Add binary flag to mode argument when opening files for writing key and
certificate files.
On Python 3 the data buffers use for writing are bytes objects not
strings, and the write fails accordingly.
As far as I understand, it the "b" flag will not hurt things in Python 2
either.
* Update the tests for RenewableCert::save_successor
Update the tests for RenewableCert::save_successor after changing
three parameters to be called with bytes objects instead of strings.
Also, update the doc comment of the function.
* rename more files in rename command
* Revert "Hide rename command (#4007)"
This reverts commit 8c14de13a5.
* Rename files in configuration files
* Delete new files if we fail during the renaming process
* update tests and error catching
* More expressive error message
* Refactor main to simplify logic
* Update tests and comments
* Correct main test
* increase timeout limit
* reset timeout limit
* call renew_cert in appropriate main test
* Update docstrings and revert signatures of _report_new_cert and _suggest_donation_of_appropriate
* replace renew_cert logic
* update tests
* rename _csr_obtain_cert and add a check to _report_new_cert
* organize cert_manager.py
* add delete files to cert manager and storage
* add tests
* add to main and cli
* Clean up all related files we can find, even if some are missing.
* error messages, debug logs, and remove RenewerConfiguration
* add logs for failure to remove
* remove renewer_config_file
* Rename and simplify main functions
* pass certname to auth method
* find cert by certname flag
* Implement --cert-name command
* don't ask to confirm new cert when we have domains and no existing certs with the lineage name
* Refactor and add --new-cert-name flag
* add interactivity to rename verb
* allow noninteractive and more descriptive function names
* Switch to using absolute path in symlink
* save archive_dir to config and read it back
* cli_config.archive_dir --> cli_config.default_archive_dir
* Use archive_dir specified in renewal config file
* add helpful broken symlink info
* add docstring to method
* Add tests
* remove extraneous test imports
* fix tests
* py2.6 syntax fix
* git problems
* Add list-certs command
* no dict comprehension in python2.6
* add test coverage
* More py26 wrangling
* update tests for py3 and lint
* remove extra dep from test
* test coverage
* test shouldn't be based on dict representation order
* Redo report UX and add tests to cover
* remove storage str test
* lint and use mock properly
* mock properly
* address code review comments
* lineage --> certificate name and print fullchain and privkey paths
* make py26 happy
* actually make py26 happy
* don't wrap text
* Standardize arguments name for mode and chmod in the util API
* Handle OpenSSL pem as bytes objects only for Python3 compatibility
* Handle OpenSSL pem as bytes objects only (remaining bits)
* Manipulate bytes objects only when testing PEM-related functions
* Fix argument order when calling util.unique_file
* Switch to using absolute path in symlink
* save archive_dir to config and read it back
* cli_config.archive_dir --> cli_config.default_archive_dir
* Use archive_dir specified in renewal config file
* add helpful broken symlink info
* add docstring to method
* Add tests
* remove extraneous test imports
* fix tests
* py2.6 syntax fix
* git problems
* no dict comprehension in python2.6
* add test coverage
* More py26 wrangling
Re-do the fix for #3497 to ensure it works in all cases.
* If lineages are in an inconsistent (non-deployed) state, deploy them
* Test new _handle_identical_cert case
* Move lineage.has_pending_deployment() check up to _auth_from_domains
Less conceptually nice, but in the "renew" verb case it wasn't being called :(
* Swap _auth_from_domains return type
* It now matches _treat_as_renewal & _handle_identical_cert_request etc
* Revert "Move lineage.has_pending_deployment() check up to _auth_from_domains"
This reverts commit a7fe734d73.
* Move test back to handle_identical_cert_request
* We need to check for non-deployment on two separate code paths
- Once high up in "renew" (because failure to be deployed stops us from
divind down the stack)
- Once way down in _handle_identical_cert_request (because that's where it
makes the most sense for run / certonly)
- So refactor that work into storage.py
* We don't necessarily reinstall
