This PR implements the filesystem.copy_ownership_and_apply_mode method from #6497.
This method is used in two places in Certbot, replacing os.chown, to copy the owner and group owner from a file to another one, and apply to the latter the given POSIX mode.
* Implement copy_ownership_and_apply_mode
* Update certbot/compat/os.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Remove default values
* Rewrite a comment.
* Relaunch CI
* Pass as keyword arguments
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Make the private key permissions transfer platform specific
* Update certbot/compat/filesystem.py
Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
* Rename variable
* Fix comment0
* Add unit test for copy_ownership_and_apply_mode
* Adapt coverage
* Execute unconditionally chmod with copy_ownership_and_apply_mode. Improve doc.
os.linesep isn't supposed to be used when writing to files opened in
text mode, where '\n' is escaped to the platform-specific ASCII
sequence. For example, on Windows, os.linesep is '\r\n' and in text
mode is escaped to ASCII sequence CR CR LF rather than just CR LF.
This is also true for the default logger and IDisplay notifications.
Replacing os.linesep with '\n' ensures the right sequence is escaped.
Resolves: 6899
This PR implements the filesystem.chmod method from #6497.
* Implement filesystem.chmod
* Conditionally add pywin32 on setuptools versions that support environment markers.
* Update apache plugin requirements
* Use a try/except import approach similar to lock
* Add comments about well-known SIDs
* Add main command
* Call filesystem.chmod in tests, remove one test
* Add test for os module
* Update environment marker
* Ensure we are not building wheels using an old version of setuptools
* Added a link to list of NTFS rights
* Simplify sid comparison
* Enable coverage
* Sometimes, double-quote is the solution
* Add entrypoint
* Add unit tests to filesystem
* Resolve recursively the link, add doc
* Move imports to the top of the file
* Remove string conversion of the ACL, fix setup
* Ensure admins have all permissions
* Simplify dacl comparison
* Conditionally raise for windows temporary workaround
* Add a test to check filesystem.chown is protected against symlink loops
* Modifications for misc
* Add some types
* Use os_rename
* Move rename into filesystem
* Use our os package
* Rename filesystem.rename to filesystem.replace
* Disable globally function redefined lint in os module
* Validate OCSP response for responders that are not the certificate's issuer.
* Improve OCSP tests using a issuer/responder pair for OCSP responses
* Clean code
* Update ocsp_test.py
* Add various comments
* Add several cases of ocsp responder. More factories for the resilience tests.
* Update ocsp_test.py
* Ignore editor backups when running hooks.
When processing hooks, certbot also runs editor backups even though
such files are outdated, clearly warranted correction and may quite
possibly be defective.
That behavior could lead to unexpected breakage, and perhaps even pose
security risks---for example, if a previous script was careless with
file permissions. As an aggravating factor, the backup runs after the
corrected version and could unintentionally override a fix the user
thought was properly implemented.
This commit causes editor backup files ending in tilde (~) to be
excluded when running hooks.
Additional information can be found here:
https://github.com/certbot/certbot/issues/7107https://community.letsencrypt.org/t/editor-backup-files-executed-as-renewal-hooks/94750
* Add unit test for hook scripts with filenames ending in tilde.
* Provide changelog entry for not running hook scripts ending in tilde.
* Add Felix Lechner to the list of contributors.
Revert #6702
After some discussions, we realized that changing the path for FreeBSD users, if it corresponds to the path used when Certbot is installed using ports, will break for users that installed it through certbot-auto.
Indeed in this case, the path used was the one for Linux. After #6702, Certbot would not find anymore the existing config path by default.
It would require, to be integrated, a proper documentation and a migration path. For now, it is preferable to revert it.
This reverts commit 7fe82cf1ac.
This PR attempts to better inform people about the problem identified at https://community.letsencrypt.org/t/certbot-auto-deployment-best-practices/91979/.
I was hesitant to add the flag --no-permissions-check, however, if there's some obscure distro out there (or custom user setup) that has a strange users and groups, I didn't want us to either:
Have to put out a bug fix release
Refuse to fix the problem and let them deal with warnings on every run
* add check_permissions.py
* Update letsencrypt-auto.template.
* build letsencrypt-auto
* Add test_permissions_warnings to auto_test
* Allow uid/gid < 1000.
* Add --no-permissions-check to Certbot.
* Add --no-permissions-check to certbot-auto.
* Add test farm test that letsencrypt-auto is quiet.
As a bonus, this new test will catch problems like the one that the caused
0.33.1 point release.
* Update CHANGELOG about permissions check.
* Update permissions comment.
* Fix symlink handling.
* Use a better default in auto_test.py.
First step of #6960.
* Warn install users that future versions of certbot will automatically redirect
* Only warn when the user declines or auto-declines redirect
* Unit tests
* Update changelog
This PR is the second part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)
This PR creates the module certbot.compat.os, that delegates everything to os, and that will be the safeguard against problematic methods of the standard module. On top of that, a quality check wrapper is called in the lint tox environment. This wrapper calls pylint and ensures that standard os module is no used directly in the certbot codebase.
Finally local oldest requirements are updated to ensure that tests will take the new logic when running.
* Add executable permissions
* Add the delegate certbot.compat.os module, add check coding style to enforce usage of certbot.compat.os instead of standard os
* Load certbot.compat.os instead of os
* Move existing compat test
* Update local oldest requirements
* Import sys
* Update account_test.py
* Update os.py
* Update os.py
* Update local oldest requirements
* Implement the new linter_plugin
* Fix local oldest for nginx
* Remove check coding style
* Update linter_plugin.py
* Add several comments
* Update the setup.py
* Add documentation
* Update acme dependencies
* Update certbot/compat/os.py
* Update docs/contributing.rst
* Update linter_plugin.py
* Handle os.path. Simplify checker.
* Add a comment to a reference implementation
* Update changelog
* Fix module registering
* Update docs/contributing.rst
* Update config and changelog
This PR is a part of the effort to remove the last broken unit tests in certbot codebase for Windows, as described in #6850.
It solves the problems associated to ErrorHandler in Windows (enlighted by tests errors) by ... wipping out the problem: no signal is handled by ErrorHandler on Windows. See the relevant inline comment in certbot.error_handler for explanation and sources.
This PR is the first part of #6497 to ease the integration, following the new plan propose by @bmw here: #6497 (comment)
This step 1 refactor existing certbot.compat module into certbot.compat.misc, without any logic changed. Package certbot.compat will host the new modules that constitute the security model for Windows.
* Create the certbot.compat package. Move logic in certbot.compat.misc
* Add doc
* Fix lint
* Correct mypy
* Update client.py
This PR is a part of the tls-sni-01 removal plan described in #6849.
This PR removes --tls-sni-01-port, --tls-sni-01-address and tls-sni-01/tls-sni options from --preferred-challenges. They are replace by deprecation warning, indicating that these options will be removed soon.
This deprecation, instead of complete removal, is done to avoid certbot instances to hard fail if some automated scripts still use these flags for some users.
Once this PR lands, we can remove completely theses flags in one or two release.
* Remove tls-sni related flags in cli. Add a deprecation warning instead.
* Adapt tests to cli and renewal towards tls-sni flags deprecation
* Add https_port option. Make tls_sni_01_port show a deprecation warning, but silently modify https_port if set
* Migrate last items
* Fix lint
* Update certbot/cli.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Ensure to remove all occurences of tls-sni-01
* Remove unused parameter
* Revert modifications on cli-help.txt
* Use logger.warning instead of sys.stderr
* Update the logger warning message
* Remove standalone_supported_challenges option.
* Fix order of preferred-challenges
* Remove supported_challenges property
* Fix some tests
* Fix lint
* Fix tests
* Add a changelog
* Clean code, fix test
* Update CI
* Reload
* No hard date for tls-sni removal
* Remove useless cast to list
* Update certbot/tests/renewal_test.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Add entry to the changelog
* Add entry to the changelog
Explicit is better than implicit
When calling raise without an argument, Python will raise the last error occured from the caller except block. This makes my PyCharm very sad however. So this PR makes the function handling the error raising explicitly the error received as an argument.
This PR is a part of the effort to remove the last broken unit tests in certbot codebase for Windows, as described in #6850.
This PR fixes various unit tests on Windows, whose resolution was only to modify some logic in the tests, or minor changes in certbot codecase impacting Windows only (like handling correctly paths with DOS-style).
* Correct several tests
* Skip test definitively
* Test to be reactivated with #6497
* Mock log system to avoid errors due to multiple calls to main in main_test
* Simplify mock
* Update cli_test.py
* One test to be repaired when windows file permissions PR is merged
Fixes#6836.
OCSP responses contains a thisUpdate and nextUpdate that allow to calculate its validity. Certbot currently uses datetime.now() to get the current time when OCSP check is done through cryptography. But datetime.now() expresses the date in the machine local time, and comparison operators on datetime do not take into account the offset between two datetime objects expressed in difference timezones.
As a consequence, a given thisUpdate may be seen as a future date depending on the local time, failing the OCSP check process.
The error is not critical for certbot, as it will just make some valid OCSP responses giving an EXPIRED status been ignored.
This PR fixes this comparison by taking the current time in UTC using datetime.utctime().
Fixes#5789
This PR is about allowing Certbot to respect the Retry-After HTTP header that an ACME CA server can return to a client POSTing to a challenge, to instruct him and retry the request later.
However, this feature was not easily implementable in the current code of certbot.auth_handler, because the code became really hard to read. In fact, @bmw was thinking that the code was really deceiving, and a lot of supposed functionalities declared in the comments were in fact not implemented or not functional.
So I took the time to understand what was going on, and effectively, most of the code is in fact not usable or not used. Then I did a refactoring against the bare ACME spec about what to do to prepare challenges, instruct the ACME CA server to perform them, then polling regularly the authorization resources until they are decided (valid or invalid).
And of course this implementation takes care of Retry-After ^^
I added a lot of comments in the new implementation, to explain what is going on for a future developer. The workflow I used is relying on the relationships between authorizations and challenges states as described in section 7.1.6 of the ACME spec draft: https://datatracker.ietf.org/doc/draft-ietf-acme-acme/
* Clean auth_handler a bit, and implement retry-after.
* Remove a debug logger
* Correct tests
* Fix mypy and lint. Setup max retries and default retry after accordingly.
* Ease a comparison in tests
* Update documentation
* Add tests
* Adapt windows coverage threshold to the global LOC reduction
* Update certbot/auth_handler.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Corrections under review
* Correction under review
* Update certbot/auth_handler.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Corrections under review
* Update auth_handler_test.py
* Reimplementing user readable report for failed authorizations
* Fixes two tests
* Fix another test + lint + mypy
* Update auth_handler.py
* Update auth_handler_test.py
* Fix tests
* Update certbot/auth_handler.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Raise directly the exception on polling timout
* Improve interface documentation
* Move the wait on top of the loop, to be used initially or after a new loop iteration. Do not wait for negative values.
* Always display the report about failed authorizations.
* Clarify an exception.
* Return, instead of break
* Use setdefault
* Remove useless assertion
* Adapt tests
* Improve a test about retry after value.
* Update certbot/auth_handler.py
Co-Authored-By: adferrand <adferrand@users.noreply.github.com>
* Add a complete test on best_effort
* Add entry to the changelog
* Gather all failed authzrs to be reported in one unique report in case of best_effort
* Build complete warn/report/raise process about failed authzrs
* Mock set_by_cli in _test_relevant_values_common.
* Empty commit
* Revert "Mock set_by_cli in _test_relevant_values_common."
This reverts commit 9dfec8dfa9.
* mock less
* Use plugin_common code instead of reimplementing.
* Revert 2nd implementation.
* Simplify certbot.storage.relevant_values() tests.
In addition to cleaning up the code a bit, it also removes the problems we've
seen in these tests with the global state used in cli.py.
Erica Portnoy