mirror of
https://github.com/certbot/certbot.git
synced 2026-08-03 08:03:10 +02:00
So here we are: after #6361 has been merged, time is to provide an environment to execute the automated testing on Windows. Here are the assertions used to build the CI on Windows: every test running on Linux should ultimately be runnable on Windows, in a cross-platform compatible manner (there is one or two exception, when a test does not have any meaning for Windows), currently some tests are not runnable on Windows: theses tests are ignored by default when the environment is Windows using a custom decorator: @broken_on_windows, test environment should have functionalities similar to Travis, in particular an execution test matrix against various versions of Python and Windows, so test execution is done through AppVeyor, as it supports the requirements: it add a CI step along Travis and Codecov for each PR, all of this ensuring that Certbot is entirely functional on both Linux and Windows, code in tests can be changed, but code in Certbot should be changed as little as possible, to avoid regression risks. So far in this PR, I focused on the tests on Certbot core and ACME library. Concerning the plugins, it will be done later, for plugins which have an interest on Windows. Test are executed against Python 3.4, 3.5, 3.6 and 3.7, for Windows Server 2012 R2 and Windows Server 2016. I succeeded at making 258/259 of acme tests to work, and 828/868 of certbot core tests to work. Most of the errors where not because of Certbot itself, but because of how the tests are written. After redesigning some test utilitaries, and things like file path handling, or CRLF/LF, a lot of the errors vanished. I needed also to ignore a lot of IO errors typically occurring when a tearDown test process tries to delete a file before it has been closed: this kind of behavior is acceptable for Linux, but not for Windows. As a consequence, and until the tearDown process is improved, a lot of temporary files are not cleared on Windows after a test campaign. Remaining broken tests requires a more subtile approach to solve the errors, I will correct them progressively in future PR. Last words about tox. I did not used the existing tox.ini for now. It is just to far from what is supported on Windows: lot of bash scripts that should be rewritten completely, and that contain test logic not ready/relevant for Windows (plugin tests, Docker compilation/test, GNU distribution versatility handling and so on). So I use an independent file tox-win.ini for now, with the goal to merge it ultimately with the existing logic. * Define a tox configuration for windows, to execute tests against Python 3.4, 3.5, 3.6 and 3.7 + code coverage on Codecov.io * Correct windows compatibility on certbot codebase * Correct windows compatibility on certbot display functionalities * Correct windows compatibility on certbot plugins * Correct test utils to run tests on windows. Add decorator to skip (permanently) or mark broken (temporarily) tests on windows * Correct tests on certbot core to run them both on windows and linux. Mark some of them as broken on windows for now. * Lock tests are completely skipped on windows. Planned to be replace in next PR. * Correct tests on certbot display to run them both on windows and linux. Mark some of them as broken on windows for now. * Correct test utils for acme on windows. Add decorator to skip (permanently) or mark broken (temporarily) tests on windows. * Correct acme tests to run them both on windows and linux. Allow a reduction of code coverage of 1% on acme code base. * Create AppVeyor CI for Certbot on Windows, to run the test matrix (py34,35,36,37+coverage) on Windows Server 2012 R2 and Windows Server 2016. * Update changelog with Windows compatibility of Certbot. * Corrections about tox, pyreadline and CI logic * Correct english * Some corrections for acme * Newlines corrections * Remove changelog * Use os.devnull instead of /dev/null to be used on Windows * Uid is a always a number now. * Correct linting * PR https://github.com/python/typeshed/pull/2136 has been merge to third-party upstream 6 months ago, so code patch can be removed. * And so acme coverage should be 100% again. * More compatible tests Windows+Linux * Use stable line separator * Remove unused import * Do not rely on pytest in certbot tests * Use json.dumps to another json embedding weird characters * Change comment * Add import * Test rolling builds #1 * Test rolling builds #2 * Correction on json serialization * It seems that rolling builds are not canceling jobs on PR. Revert back to fail fast code in the pipeline.
151 lines
5.0 KiB
Python
151 lines
5.0 KiB
Python
"""
|
|
Compatibility layer to run certbot both on Linux and Windows.
|
|
|
|
The approach used here is similar to Modernizr for Web browsers.
|
|
We do not check the platform type to determine if a particular logic is supported.
|
|
Instead, we apply a logic, and then fallback to another logic if first logic
|
|
is not supported at runtime.
|
|
|
|
Then logic chains are abstracted into single functions to be exposed to certbot.
|
|
"""
|
|
import os
|
|
import select
|
|
import sys
|
|
import errno
|
|
import ctypes
|
|
import stat
|
|
|
|
from certbot import errors
|
|
|
|
try:
|
|
# Linux specific
|
|
import fcntl # pylint: disable=import-error
|
|
except ImportError:
|
|
# Windows specific
|
|
import msvcrt # pylint: disable=import-error
|
|
|
|
UNPRIVILEGED_SUBCOMMANDS_ALLOWED = [
|
|
'certificates', 'enhance', 'revoke', 'delete',
|
|
'register', 'unregister', 'config_changes', 'plugins']
|
|
def raise_for_non_administrative_windows_rights(subcommand):
|
|
"""
|
|
On Windows, raise if current shell does not have the administrative rights.
|
|
Do nothing on Linux.
|
|
|
|
:param str subcommand: The subcommand (like 'certonly') passed to the certbot client.
|
|
|
|
:raises .errors.Error: If the provided subcommand must be run on a shell with
|
|
administrative rights, and current shell does not have these rights.
|
|
|
|
"""
|
|
# Why not simply try ctypes.windll.shell32.IsUserAnAdmin() and catch AttributeError ?
|
|
# Because windll exists only on a Windows runtime, and static code analysis engines
|
|
# do not like at all non existent objects when run from Linux (even if we handle properly
|
|
# all the cases in the code).
|
|
# So we access windll only by reflection to trick theses engines.
|
|
if hasattr(ctypes, 'windll') and subcommand not in UNPRIVILEGED_SUBCOMMANDS_ALLOWED:
|
|
windll = getattr(ctypes, 'windll')
|
|
if windll.shell32.IsUserAnAdmin() == 0:
|
|
raise errors.Error(
|
|
'Error, "{0}" subcommand must be run on a shell with administrative rights.'
|
|
.format(subcommand))
|
|
|
|
def os_geteuid():
|
|
"""
|
|
Get current user uid
|
|
|
|
:returns: The current user uid.
|
|
:rtype: int
|
|
|
|
"""
|
|
try:
|
|
# Linux specific
|
|
return os.geteuid()
|
|
except AttributeError:
|
|
# Windows specific
|
|
return 0
|
|
|
|
def readline_with_timeout(timeout, prompt):
|
|
"""
|
|
Read user input to return the first line entered, or raise after specified timeout.
|
|
|
|
:param float timeout: The timeout in seconds given to the user.
|
|
:param str prompt: The prompt message to display to the user.
|
|
|
|
:returns: The first line entered by the user.
|
|
:rtype: str
|
|
|
|
"""
|
|
try:
|
|
# Linux specific
|
|
#
|
|
# Call to select can only be done like this on UNIX
|
|
rlist, _, _ = select.select([sys.stdin], [], [], timeout)
|
|
if not rlist:
|
|
raise errors.Error(
|
|
"Timed out waiting for answer to prompt '{0}'".format(prompt))
|
|
return rlist[0].readline()
|
|
except OSError:
|
|
# Windows specific
|
|
#
|
|
# No way with select to make a timeout to the user input on Windows,
|
|
# as select only supports socket in this case.
|
|
# So no timeout on Windows for now.
|
|
return sys.stdin.readline()
|
|
|
|
def lock_file(fd):
|
|
"""
|
|
Lock the file linked to the specified file descriptor.
|
|
|
|
:param int fd: The file descriptor of the file to lock.
|
|
|
|
"""
|
|
if 'fcntl' in sys.modules:
|
|
# Linux specific
|
|
fcntl.lockf(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
|
else:
|
|
# Windows specific
|
|
msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
|
|
|
|
def release_locked_file(fd, path):
|
|
"""
|
|
Remove, close, and release a lock file specified by its file descriptor and its path.
|
|
|
|
:param int fd: The file descriptor of the lock file.
|
|
:param str path: The path of the lock file.
|
|
|
|
"""
|
|
# Linux specific
|
|
#
|
|
# It is important the lock file is removed before it's released,
|
|
# otherwise:
|
|
#
|
|
# process A: open lock file
|
|
# process B: release lock file
|
|
# process A: lock file
|
|
# process A: check device and inode
|
|
# process B: delete file
|
|
# process C: open and lock a different file at the same path
|
|
try:
|
|
os.remove(path)
|
|
except OSError as err:
|
|
if err.errno == errno.EACCES:
|
|
# Windows specific
|
|
# We will not be able to remove a file before closing it.
|
|
# To avoid race conditions described for Linux, we will not delete the lockfile,
|
|
# just close it to be reused on the next Certbot call.
|
|
pass
|
|
else:
|
|
raise
|
|
finally:
|
|
os.close(fd)
|
|
|
|
def compare_file_modes(mode1, mode2):
|
|
"""Return true if the two modes can be considered as equals for this platform"""
|
|
if 'fcntl' in sys.modules:
|
|
# Linux specific: standard compare
|
|
return oct(stat.S_IMODE(mode1)) == oct(stat.S_IMODE(mode2))
|
|
# Windows specific: most of mode bits are ignored on Windows. Only check user R/W rights.
|
|
return (stat.S_IMODE(mode1) & stat.S_IREAD == stat.S_IMODE(mode2) & stat.S_IREAD
|
|
and stat.S_IMODE(mode1) & stat.S_IWRITE == stat.S_IMODE(mode2) & stat.S_IWRITE)
|