Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544)

Summary of changes in this PR:
- Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below).
- Make repo root README symlink to `certbot` README.
- Pull tests outside of the distributed module.
- Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery.
- Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments.
- Replace mentions of `.` with `certbot` when referring to packages to install, usually editably.
- Clean up some unused code around executing tests in a different directory.
- Create public shim around main and make that the entry point.

New directory structure summary:
```
repo root ("certbot", probably, but for clarity all files I mention are relative to here)
├── certbot
│   ├── setup.py
│   ├── certbot
│   │   ├── __init__.py
│   │   ├── achallenges.py
│   │   ├── _internal
│   │   │   ├── __init__.py
│   │   │   ├── account.py
│   │   │   ├── ...
│   │   ├── ...
│   ├── tests
│   │   ├── account_test.py
│   │   ├── display
│   │   │   ├── __init__.py
│   │   │   ├── ...
│   │   ├── ... # note no __init__.py at this level
│   ├── ...
├── acme
│   ├── ...
├── certbot-apache
│   ├── ...
├── ...
```

* refactor certbot/ and certbot/tests/ to use the same structure as the other packages

* git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g"

* git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g"

* git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g"

* Remove replacement of certbot into . in install_and_test.py

* copy license back out to main folder

* remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/

* Move README back into main folder, and make the version inside certbot/ a symlink

* symlink certbot READMEs the other way around

* move testdata into the public api certbot zone

* update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests

* certbot version has been bumped down a directory level

* make certbot tests directory not a package and import sibling as module

* Remove unused script cruft

* change . to certbot in test_sdists

* remove outdated comment referencing a command that doesn't work

* Install instructions should reference an existing file

* update file paths in Dockerfile

* some package named in tox.ini were manually specified, change those to certbot

* new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code

* remove other instance of pyargs

* fix up some references in _release.sh by searching for ' . ' and manual check

* another stray . in tox.ini

* fix paths in tools/_release.sh

* Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper

* Create public shim around main and make that the entry point

* without pyargs, tests cannot be run from an empty directory

* Remove cruft for running certbot directly from main

* Have main shim take real arg

* add docs/api file for main, and fix up main comment

* Update certbot/docs/install.rst

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Fix comments in readthedocs requirements files to refer to current package

* Update .[docs] reference in contributing.rst

* Move plugins tests to certbot tests directory

* add certbot tests to MANIFEST.in so packagers can run python setup.py test

* move examples directory inside certbot/

* Move CHANGELOG into certbot, and create a top-level symlink

* Remove unused sys and logging from main shim

* nginx http01 test no longer relies on certbot plugins common test
This commit is contained in:
ohemorange
2019-11-25 14:28:05 -08:00
committed by Brad Warren
co-authored by Brad Warren
parent d56cd4ef01
commit 4abd81e218
190 changed files with 2334 additions and 2136 deletions
+175
View File
@@ -0,0 +1,175 @@
import codecs
import os
import re
import sys
from distutils.version import StrictVersion
from setuptools import find_packages, setup, __version__ as setuptools_version
from setuptools.command.test import test as TestCommand
# Workaround for http://bugs.python.org/issue8876, see
# http://bugs.python.org/issue8876#msg208792
# This can be removed when using Python 2.7.9 or later:
# https://hg.python.org/cpython/raw-file/v2.7.9/Misc/NEWS
if os.path.abspath(__file__).split(os.path.sep)[1] == 'vagrant':
del os.link
def read_file(filename, encoding='utf8'):
"""Read unicode from given file."""
with codecs.open(filename, encoding=encoding) as fd:
return fd.read()
here = os.path.abspath(os.path.dirname(__file__))
# read version number (and other metadata) from package init
init_fn = os.path.join(here, 'certbot', '__init__.py')
meta = dict(re.findall(r"""__([a-z]+)__ = '([^']+)""", read_file(init_fn)))
readme = read_file(os.path.join(here, 'README.rst'))
version = meta['version']
# This package relies on PyOpenSSL, requests, and six, however, it isn't
# specified here to avoid masking the more specific request requirements in
# acme. See https://github.com/pypa/pip/issues/988 for more info.
install_requires = [
'acme>=0.40.0',
# We technically need ConfigArgParse 0.10.0 for Python 2.6 support, but
# saying so here causes a runtime error against our temporary fork of 0.9.3
# in which we added 2.6 support (see #2243), so we relax the requirement.
'ConfigArgParse>=0.9.3',
'configobj',
'cryptography>=1.2.3', # load_pem_x509_certificate
'distro>=1.0.1',
# 1.1.0+ is required to avoid the warnings described at
# https://github.com/certbot/josepy/issues/13.
'josepy>=1.1.0',
'mock',
'parsedatetime>=1.3', # Calendar.parseDT
'pyrfc3339',
'pytz',
'setuptools',
'zope.component',
'zope.interface',
]
# Add pywin32 on Windows platforms to handle low-level system calls.
# This dependency needs to be added using environment markers to avoid its installation on Linux.
# However environment markers are supported only with setuptools >= 36.2.
# So this dependency is not added for old Linux distributions with old setuptools,
# in order to allow these systems to build certbot from sources.
pywin32_req = 'pywin32>=225' # do not forget to edit pywin32 dependency accordingly in windows-installer/construct.py
if StrictVersion(setuptools_version) >= StrictVersion('36.2'):
install_requires.append(pywin32_req + " ; sys_platform == 'win32'")
elif 'bdist_wheel' in sys.argv[1:]:
raise RuntimeError('Error, you are trying to build certbot wheels using an old version '
'of setuptools. Version 36.2+ of setuptools is required.')
elif os.name == 'nt':
# This branch exists to improve this package's behavior on Windows. Without
# it, if the sdist is installed on Windows with an old version of
# setuptools, pywin32 will not be specified as a dependency.
install_requires.append(pywin32_req)
dev_extras = [
'astroid==1.6.5',
'coverage',
'ipdb',
'pytest',
'pytest-cov',
'pytest-xdist',
'pylint==1.9.4',
'tox',
'twine',
'wheel',
]
dev3_extras = [
'mypy',
'typing', # for python3.4
]
docs_extras = [
# If you have Sphinx<1.5.1, you need docutils<0.13.1
# https://github.com/sphinx-doc/sphinx/issues/3212
'repoze.sphinx.autointerface',
'Sphinx>=1.2', # Annotation support
'sphinx_rtd_theme',
]
class PyTest(TestCommand):
user_options = []
def initialize_options(self):
TestCommand.initialize_options(self)
self.pytest_args = ''
def run_tests(self):
import shlex
# import here, cause outside the eggs aren't loaded
import pytest
errno = pytest.main(shlex.split(self.pytest_args))
sys.exit(errno)
setup(
name='certbot',
version=version,
description="ACME client",
long_description=readme,
url='https://github.com/letsencrypt/letsencrypt',
author="Certbot Project",
author_email='client-dev@letsencrypt.org',
license='Apache License 2.0',
python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*',
classifiers=[
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Environment :: Console :: Curses',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: Apache Software License',
'Operating System :: POSIX :: Linux',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Topic :: Internet :: WWW/HTTP',
'Topic :: Security',
'Topic :: System :: Installation/Setup',
'Topic :: System :: Networking',
'Topic :: System :: Systems Administration',
'Topic :: Utilities',
],
packages=find_packages(exclude=['docs', 'examples', 'tests', 'venv']),
include_package_data=True,
install_requires=install_requires,
extras_require={
'dev': dev_extras,
'dev3': dev3_extras,
'docs': docs_extras,
},
test_suite='certbot',
tests_require=["pytest"],
cmdclass={"test": PyTest},
entry_points={
'console_scripts': [
'certbot = certbot.main:main',
],
'certbot.plugins': [
'manual = certbot._internal.plugins.manual:Authenticator',
'null = certbot._internal.plugins.null:Installer',
'standalone = certbot._internal.plugins.standalone:Authenticator',
'webroot = certbot._internal.plugins.webroot:Authenticator',
],
},
)