So here we are: after #6361 has been merged, time is to provide an environment to execute the automated testing on Windows.
Here are the assertions used to build the CI on Windows:
every test running on Linux should ultimately be runnable on Windows, in a cross-platform compatible manner (there is one or two exception, when a test does not have any meaning for Windows),
currently some tests are not runnable on Windows: theses tests are ignored by default when the environment is Windows using a custom decorator: @broken_on_windows,
test environment should have functionalities similar to Travis, in particular an execution test matrix against various versions of Python and Windows,
so test execution is done through AppVeyor, as it supports the requirements: it add a CI step along Travis and Codecov for each PR, all of this ensuring that Certbot is entirely functional on both Linux and Windows,
code in tests can be changed, but code in Certbot should be changed as little as possible, to avoid regression risks.
So far in this PR, I focused on the tests on Certbot core and ACME library. Concerning the plugins, it will be done later, for plugins which have an interest on Windows. Test are executed against Python 3.4, 3.5, 3.6 and 3.7, for Windows Server 2012 R2 and Windows Server 2016.
I succeeded at making 258/259 of acme tests to work, and 828/868 of certbot core tests to work. Most of the errors where not because of Certbot itself, but because of how the tests are written. After redesigning some test utilitaries, and things like file path handling, or CRLF/LF, a lot of the errors vanished.
I needed also to ignore a lot of IO errors typically occurring when a tearDown test process tries to delete a file before it has been closed: this kind of behavior is acceptable for Linux, but not for Windows. As a consequence, and until the tearDown process is improved, a lot of temporary files are not cleared on Windows after a test campaign.
Remaining broken tests requires a more subtile approach to solve the errors, I will correct them progressively in future PR.
Last words about tox. I did not used the existing tox.ini for now. It is just to far from what is supported on Windows: lot of bash scripts that should be rewritten completely, and that contain test logic not ready/relevant for Windows (plugin tests, Docker compilation/test, GNU distribution versatility handling and so on). So I use an independent file tox-win.ini for now, with the goal to merge it ultimately with the existing logic.
* Define a tox configuration for windows, to execute tests against Python 3.4, 3.5, 3.6 and 3.7 + code coverage on Codecov.io
* Correct windows compatibility on certbot codebase
* Correct windows compatibility on certbot display functionalities
* Correct windows compatibility on certbot plugins
* Correct test utils to run tests on windows. Add decorator to skip (permanently) or mark broken (temporarily) tests on windows
* Correct tests on certbot core to run them both on windows and linux. Mark some of them as broken on windows for now.
* Lock tests are completely skipped on windows. Planned to be replace in next PR.
* Correct tests on certbot display to run them both on windows and linux. Mark some of them as broken on windows for now.
* Correct test utils for acme on windows. Add decorator to skip (permanently) or mark broken (temporarily) tests on windows.
* Correct acme tests to run them both on windows and linux. Allow a reduction of code coverage of 1% on acme code base.
* Create AppVeyor CI for Certbot on Windows, to run the test matrix (py34,35,36,37+coverage) on Windows Server 2012 R2 and Windows Server 2016.
* Update changelog with Windows compatibility of Certbot.
* Corrections about tox, pyreadline and CI logic
* Correct english
* Some corrections for acme
* Newlines corrections
* Remove changelog
* Use os.devnull instead of /dev/null to be used on Windows
* Uid is a always a number now.
* Correct linting
* PR https://github.com/python/typeshed/pull/2136 has been merge to third-party upstream 6 months ago, so code patch can be removed.
* And so acme coverage should be 100% again.
* More compatible tests Windows+Linux
* Use stable line separator
* Remove unused import
* Do not rely on pytest in certbot tests
* Use json.dumps to another json embedding weird characters
* Change comment
* Add import
* Test rolling builds #1
* Test rolling builds #2
* Correction on json serialization
* It seems that rolling builds are not canceling jobs on PR. Revert back to fail fast code in the pipeline.
* Add and use a compatibility layer to allow certbot to be run on windows.
* Fix path comparison
* Corrections on compat and util for tests
* Less intrusive way to parse prefix in webroot plugin working for both linux and windows.
* Disable pylint import-error for some optional imports in compat.py
* Ensure path is normalized before prefixes are generated in webroot plugin
* Same prefixes in linux and windows, in fact root path is not needed in webroot plugin
* Check that user has administrative rights before continuing on windows (necessary for symlink creation)
* More straightforward way to test administrative rights on windows
* Try to resolve import error in travis ci
* OK. We go for full introspection to trick the ci.
* Move the administrative rights control to the certbot entrypoint
* Add comment for a really non trivial code.
* Allow some commands to be run on a shell without admin rights
* Avoid races conditions on windows for lock files
* Add sphinx doc to the compat functions.
* Remove irrelevant Windows error in the lock mechanism.
* Some corrections on compat
* Revert "Revert "switch signature verification to use pure cryptography (#6000)" (#6074)"
This reverts commit 3cffe1449c.
* Fixes#6073.
This silences the deprecation warnings from cryptography. I looked into only
silencing the cryptography warning specifically in the function, however,
CryptographyDeprecationWarning doesn't seem to be publicly documented, so we
probably shouldn't depend on it.
* switch signature verification to use pure cryptography
On systems that prevent write/execute pages this prevents a segfault
that is caused by pyopenssl creating a dynamic callback in the
verification helper.
* switch to using a verifier for older cryptography releases
also add ec support, test vectors, and a test
* update order object with returned authorizations
* major structure of finalize_order shim refactor
* util methods and imports for finalize_order shim refactor
* update certbot.tests.client_test.py
* extraneous client_test imports
* remove correct import
* update renewal call
* add test for acme.dump_pyopenssl_chain
* Add test for certbot.crypto_util.cert_and_chain_from_fullchain
* add tests for acme.client and change to fetch chain failure to TimeoutError
* s/rytpe/rtype
* remove ClientV1 passthrough
* dump the wrapped cert
* remove dead code
* remove the correct dead code
* support earlier mock
* switch new_order to use crypto_util._pyopenssl_cert_or_req_san
* move certbot.crypto_util._get_names_from_loaded_cert_or_req functionality to acme.crypto_util._pyopenssl_cert_or_req_all_names
* Revert "Don't save keys/csr on dry run (#4380)"
This reverts commit e034b50363.
* Don't save CSRs and keys during dry run
* Factor out _test_obtain_certificate_common
* Add test_obtain_certificate_dry_run
* Wrap key from make_key in util.Key
* Wrap result from make_csr in util.CSR
* cert signature validation for certificates subcommand + a test
* refactoring validation + adding in a check for making sure that the private key matches the certificate
* adding testing certs
* assertIsNone(x) -> assertEqual(None,x) to unbreak the py2.6 tests
* modifying test_verifu_renewable_cert_failure to hopefully appease python 3 test timeouts
* updating cryptography to be >=1.2 so that we can use verify
* removing unused, old testing certificate
* adding better error handling/logging
* adding test for IOError
* switching to a 2048 bit rsa key
Fixes#4559.
* Update options-ssl-nginx.conf in prepare, if it hasn't been modified.
* add previous options-ssl-nginx.conf hashes
* InstallSslOptionsConfTest
* remove .new file and only print warning once
* save digest to /etc/letsencrypt
* add comment reminding devs to update hashes
* add comment and test for sha256sum
* treat hash file as text file because python3
* move constants and rename hidden digest file
It's relatively finicky to make a CSR appropriate to pass to poll_and_request_issuance. I think most users want to be able to give a list of domains and a private key, and get back a CSR. This branch adds that functionality to crypto_util.
Note that the two new functions take arguments, and return values, as PEM-encoded buffers. This is a departure from some existing ACME interfaces that take PyOpenSSL types. I've discussed with the Certbot team, and we agree that this is broadly the direction the ACME API should take, so that users of the module don't need to import PyOpenSSL themselves, or use its primitives.
* Add make_csr.
* accept privkey
* Tweak API.
* Remove make_csr from certbot package.
* Skip test in older Pythons.
* Move get_Extensions call under protection.
* Remove assertIn because not backwards-compatible.
* Fix encoding, and use PEM.
* Fix test
* Fix tests on py35.
* Fix error in test.
* Make import_csr_file always return PEM.
Also delete get_sans_from_csr (unused) and get_names_from_csr (newly unused).
* Fix function doc.
* Fix indent
* Fix call of obtain_certificate_from_Csr
* lint
* Handle review feedback.
* Fix test.
* Standardize arguments name for mode and chmod in the util API
* Handle OpenSSL pem as bytes objects only for Python3 compatibility
* Handle OpenSSL pem as bytes objects only (remaining bits)
* Manipulate bytes objects only when testing PEM-related functions
* Fix argument order when calling util.unique_file
Erica Portnoy