Merge remote-tracking branch 'origin/master' into multi-topic-help

2026-07-30 18:04:31 +02:00 · 2016-08-12 17:40:49 -07:00
parent fbadf7550c d05f53bd53
commit 7da7d47e8a
66 changed files with 1208 additions and 379 deletions
@@ -17,9 +17,11 @@ letsencrypt-auto-source/letsencrypt-auto.sig.lzma.base64
 /.vagrant
 tags
 # editor temporary files
 *~
-*.swp
+*.sw?
 \#*#
 .idea
@@ -34,6 +34,8 @@ matrix:
    - python: "2.7"
      env: TOXENV=apacheconftest
      sudo: required
    - python: "2.7"
      env: TOXENV=nginxroundtrip
    - python: "2.7"
      env: TOXENV=py27 BOULDER_INTEGRATION=1
      sudo: true
@@ -53,6 +55,16 @@ matrix:
      services: docker
      before_install:
      addons:
    - sudo: required
      env: TOXENV=apache_compat
      services: docker
      before_install:
      addons:
    - sudo: required
      env: TOXENV=nginx_compat
      services: docker
      before_install:
      addons:
    - python: "2.7"
      env: TOXENV=cover
    - python: "3.3"
@@ -1,169 +1,19 @@
-.. notice for github users
+.. This file contains of a series of comments that are used to include sections of this README in other files. Do not modify these comments unless you know what you are doing. tag:intro-begin
-Disclaimer
+Certbot is part of EFF’s effort to encrypt the entire Internet. Secure communication over the Web relies on HTTPS, which requires the use of a digital certificate that lets browsers verify the identify of web servers (e.g., is that really google.com?). Web servers obtain their certificates from trusted third parties called certificate authorities (CAs). Certbot is an easy-to-use client that fetches a certificate from Let’s Encrypt—an open certificate authority launched by the EFF, Mozilla, and others—and deploys it to a web server.
 ==========
-Certbot (previously, the Let's Encrypt client) is **BETA SOFTWARE**. It
+Anyone who has gone through the trouble of setting up a secure website knows what a hassle getting and maintaining a certificate is. Certbot and Let’s Encrypt can automate away the pain and let you turn on and manage HTTPS with simple commands. Using Certbot and Let's Encrypt is free, so there’s no need to arrange payment.
 contains plenty of bugs and rough edges, and should be tested thoroughly in
 staging environments before use on production systems.
-For more information regarding the status of the project, please see
+How you use Certbot depends on the configuration of your web server. The best way to get started is to use our `interactive guide <https://certbot.eff.org>`_. It generates instructions based on your configuration settings. In most cases, you’ll need `root or administrator access <https://certbot.eff.org/faq/#does-certbot-require-root-privileges>`_ to your web server to run Certbot.
 https://letsencrypt.org. Be sure to checkout the
 `Frequently Asked Questions (FAQ) <https://community.letsencrypt.org/t/frequently-asked-questions-faq/26#topic-title>`_.
-About Certbot
+If you’re using a hosted service and don’t have direct access to your web server, you might not be able to use Certbot. Check with your hosting provider for documentation about uploading certificates or using certificates issues by Let’s Encrypt.
 ==============================
-Certbot is a fully-featured, extensible client for the Let's
+.. Do not modify this comment unless you know what you're doing. tag:intro-end
 Encrypt CA (or any other CA that speaks the `ACME
 <https://github.com/ietf-wg-acme/acme/blob/master/draft-ietf-acme-acme.md>`_
 protocol) that can automate the tasks of obtaining certificates and
 configuring webservers to use them. This client runs on Unix-based operating
 systems.
 Until May 2016, Certbot was named simply ``letsencrypt`` or ``letsencrypt-auto``,
 depending on install method. Instructions on the Internet, and some pieces of the
 software, may still refer to this older name.
 Contributing
 ------------
 If you'd like to contribute to this project please read `Developer Guide
 <https://certbot.eff.org/docs/contributing.html>`_.
 .. _installation:
 Installation
 ------------
 If ``certbot`` (or ``letsencrypt``) is packaged for your Unix OS (visit
 certbot.eff.org_ to find out), you can install it
 from there, and run it by typing ``certbot`` (or ``letsencrypt``).  Because
 not all operating systems have packages yet, we provide a temporary solution
 via the ``certbot-auto`` wrapper script, which obtains some dependencies from
 your OS and puts others in a python virtual environment::
  user@webserver:~$ wget https://dl.eff.org/certbot-auto
  user@webserver:~$ chmod a+x ./certbot-auto
  user@webserver:~$ ./certbot-auto --help
 .. hint:: The certbot-auto download is protected by HTTPS, which is pretty good, but if you'd like to
          double check the integrity of the ``certbot-auto`` script, you can use these steps for verification before running it::
            user@server:~$ wget -N https://dl.eff.org/certbot-auto.asc
            user@server:~$ gpg2 --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2
            user@server:~$ gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc certbot-auto
 And for full command line help, you can type::
  ./certbot-auto --help all
 ``certbot-auto`` updates to the latest client release automatically.  And
 since ``certbot-auto`` is a wrapper to ``certbot``, it accepts exactly
 the same command line flags and arguments.  More details about this script and
 other installation methods can be found `in the User Guide
 <https://certbot.eff.org/docs/using.html#installation>`_.
 How to run the client
 ---------------------
 In many cases, you can just run ``certbot-auto`` or ``certbot``, and the
 client will guide you through the process of obtaining and installing certs
 interactively.
 You can also tell it exactly what you want it to do from the command line.
 For instance, if you want to obtain a cert for ``example.com``,
 ``www.example.com``, and ``other.example.net``, using the Apache plugin to both
 obtain and install the certs, you could do this::
  ./certbot-auto --apache -d example.com -d www.example.com -d other.example.net
 (The first time you run the command, it will make an account, and ask for an
 email and agreement to the Let's Encrypt Subscriber Agreement; you can
 automate those with ``--email`` and ``--agree-tos``)
 If you want to use a webserver that doesn't have full plugin support yet, you
 can still use "standalone" or "webroot" plugins to obtain a certificate::
  ./certbot-auto certonly --standalone --email admin@example.com -d example.com -d www.example.com -d other.example.net
 Understanding the client in more depth
 --------------------------------------
 To understand what the client is doing in detail, it's important to
 understand the way it uses plugins.  Please see the `explanation of
 plugins <https://certbot.eff.org/docs/using.html#plugins>`_ in
 the User Guide.
 Links
 =====
 Documentation: https://certbot.eff.org/docs
 Software project: https://github.com/certbot/certbot
 Notes for developers: https://certbot.eff.org/docs/contributing.html
 Main Website: https://letsencrypt.org/
 IRC Channel: #letsencrypt on `Freenode`_ or #certbot on `OFTC`_
 Community: https://community.letsencrypt.org
 ACME spec: http://ietf-wg-acme.github.io/acme/
 ACME working area in github: https://github.com/ietf-wg-acme/acme
 Mailing list: `client-dev`_ (to subscribe without a Google account, send an
 email to client-dev+subscribe@letsencrypt.org)
 |build-status| |coverage| |docs| |container|
 .. |build-status| image:: https://travis-ci.org/certbot/certbot.svg?branch=master
   :target: https://travis-ci.org/certbot/certbot
   :alt: Travis CI status
 .. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
   :target: https://coveralls.io/r/certbot/certbot
   :alt: Coverage status
 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
   :target: https://readthedocs.org/projects/letsencrypt/
   :alt: Documentation status
 .. |container| image:: https://quay.io/repository/letsencrypt/letsencrypt/status
   :target: https://quay.io/repository/letsencrypt/letsencrypt
   :alt: Docker Repository on Quay.io
 .. _`installation instructions`:
   https://letsencrypt.readthedocs.org/en/latest/using.html
 .. _watch demo video: https://www.youtube.com/watch?v=Gas_sSB-5SU
 System Requirements
 ===================
 The Let's Encrypt Client presently only runs on Unix-ish OSes that include
 Python 2.6 or 2.7; Python 3.x support will hopefully be added in the future. The
 client requires root access in order to write to ``/etc/letsencrypt``,
 ``/var/log/letsencrypt``, ``/var/lib/letsencrypt``; to bind to ports 80 and 443
 (if you use the ``standalone`` plugin) and to read and modify webserver
 configurations (if you use the ``apache`` or ``nginx`` plugins).  If none of
 these apply to you, it is theoretically possible to run without root privileges,
 but for most users who want to avoid running an ACME client as root, either
 `letsencrypt-nosudo <https://github.com/diafygi/letsencrypt-nosudo>`_ or
 `simp_le <https://github.com/kuba/simp_le>`_ are more appropriate choices.
 The Apache plugin currently requires a Debian-based OS with augeas version
 1.0; this includes Ubuntu 12.04+ and Debian 7+.
 .. Do not modify this comment unless you know what you're doing. tag:features-begin
 Current Features
-================
+=====================
 * Supports multiple web servers:
@@ -187,8 +37,6 @@ Current Features
  command line.
 * Free and Open Source Software, made with Python.
 .. Do not modify this comment unless you know what you're doing. tag:features-end
-.. _Freenode: https://webchat.freenode.net?channels=%23letsencrypt
+For extensive documentation on using and contributing to Certbot, go to https://certbot.eff.org/docs. If you would like to contribute to the project or run the latest code from git, you should read our `developer guide <https://certbot.eff.org/docs/contributing.html>`_.
 .. _OFTC: https://webchat.oftc.net?channels=%23certbot
 .. _client-dev: https://groups.google.com/a/letsencrypt.org/forum/#!forum/client-dev
 .. _certbot.eff.org: https://certbot.eff.org/
@@ -14,7 +14,6 @@ from acme import crypto_util
 from acme import fields
 from acme import jose
 logger = logging.getLogger(__name__)
@@ -206,6 +205,74 @@ class KeyAuthorizationChallenge(_TokenChallenge):
                self.validation(account_key, *args, **kwargs))
@ChallengeResponse.register
 class DNS01Response(KeyAuthorizationChallengeResponse):
    """ACME dns-01 challenge response."""
    typ = "dns-01"
    def simple_verify(self, chall, domain, account_public_key):
        """Simple verify.
        :param challenges.DNS01 chall: Corresponding challenge.
        :param unicode domain: Domain name being verified.
        :param JWK account_public_key: Public key for the key pair
            being authorized.
        :returns: ``True`` iff validation with the TXT records resolved from a
            DNS server is successful.
        :rtype: bool
        """
        if not self.verify(chall, account_public_key):
            logger.debug("Verification of key authorization in response failed")
            return False
        validation_domain_name = chall.validation_domain_name(domain)
        validation = chall.validation(account_public_key)
        logger.debug("Verifying %s at %s...", chall.typ, validation_domain_name)
        try:
            from acme import dns_resolver
        except ImportError:  # pragma: no cover
            raise errors.Error("Local validation for 'dns-01' challenges "
                               "requires 'dnspython'")
        txt_records = dns_resolver.txt_records_for_name(validation_domain_name)
        exists = validation in txt_records
        if not exists:
            logger.debug("Key authorization from response (%r) doesn't match "
                         "any DNS response in %r", self.key_authorization,
                         txt_records)
        return exists
@Challenge.register  # pylint: disable=too-many-ancestors
 class DNS01(KeyAuthorizationChallenge):
    """ACME dns-01 challenge."""
    response_cls = DNS01Response
    typ = response_cls.typ
    LABEL = "_acme-challenge"
    """Label clients prepend to the domain name being validated."""
    def validation(self, account_key, **unused_kwargs):
        """Generate validation.
        :param JWK account_key:
        :rtype: unicode
        """
        return jose.b64encode(hashlib.sha256(self.key_authorization(
            account_key).encode("utf-8")).digest()).decode()
    def validation_domain_name(self, name):
        """Domain name for TXT validation record.
        :param unicode name: Domain name being validated.
        """
        return "{0}.{1}".format(self.LABEL, name)
@ChallengeResponse.register
 class HTTP01Response(KeyAuthorizationChallengeResponse):
    """ACME http-01 challenge response."""
@@ -231,8 +298,8 @@ class HTTP01Response(KeyAuthorizationChallengeResponse):
            being authorized.
        :param int port: Port used in the validation.
-        :returns: ``True`` iff validation is successful, ``False``
+        :returns: ``True`` iff validation with the files currently served by the
-            otherwise.
+            HTTP server is successful.
        :rtype: bool
        """
@@ -410,7 +477,7 @@ class TLSSNI01Response(KeyAuthorizationChallengeResponse):
        :returns: ``True`` iff client's control of the domain has been
-            verified, ``False`` otherwise.
+            verified.
        :rtype: bool
        """
@@ -77,6 +77,93 @@ class KeyAuthorizationChallengeResponseTest(unittest.TestCase):
        self.assertFalse(response.verify(self.chall, KEY.public_key()))
 class DNS01ResponseTest(unittest.TestCase):
    # pylint: disable=too-many-instance-attributes
    def setUp(self):
        from acme.challenges import DNS01Response
        self.msg = DNS01Response(key_authorization=u'foo')
        self.jmsg = {
            'resource': 'challenge',
            'type': 'dns-01',
            'keyAuthorization': u'foo',
        }
        from acme.challenges import DNS01
        self.chall = DNS01(token=(b'x' * 16))
        self.response = self.chall.response(KEY)
    def test_to_partial_json(self):
        self.assertEqual(self.jmsg, self.msg.to_partial_json())
    def test_from_json(self):
        from acme.challenges import DNS01Response
        self.assertEqual(self.msg, DNS01Response.from_json(self.jmsg))
    def test_from_json_hashable(self):
        from acme.challenges import DNS01Response
        hash(DNS01Response.from_json(self.jmsg))
    def test_simple_verify_bad_key_authorization(self):
        key2 = jose.JWKRSA.load(test_util.load_vector('rsa256_key.pem'))
        self.response.simple_verify(self.chall, "local", key2.public_key())
    @mock.patch("acme.dns_resolver.txt_records_for_name")
    def test_simple_verify_good_validation(self, mock_resolver):
        mock_resolver.return_value = [self.chall.validation(KEY.public_key())]
        self.assertTrue(self.response.simple_verify(
            self.chall, "local", KEY.public_key()))
        mock_resolver.assert_called_once_with(
            self.chall.validation_domain_name("local"))
    @mock.patch("acme.dns_resolver.txt_records_for_name")
    def test_simple_verify_good_validation_multiple_txts(self, mock_resolver):
        mock_resolver.return_value = [
            "!", self.chall.validation(KEY.public_key())]
        self.assertTrue(self.response.simple_verify(
            self.chall, "local", KEY.public_key()))
        mock_resolver.assert_called_once_with(
            self.chall.validation_domain_name("local"))
    @mock.patch("acme.dns_resolver.txt_records_for_name")
    def test_simple_verify_bad_validation(self, mock_dns):
        mock_dns.return_value = ["!"]
        self.assertFalse(self.response.simple_verify(
            self.chall, "local", KEY.public_key()))
 class DNS01Test(unittest.TestCase):
    def setUp(self):
        from acme.challenges import DNS01
        self.msg = DNS01(token=jose.decode_b64jose(
            'evaGxfADs6pSRb2LAv9IZf17Dt3juxGJ+PCt92wr+oA'))
        self.jmsg = {
            'type': 'dns-01',
            'token': 'evaGxfADs6pSRb2LAv9IZf17Dt3juxGJ-PCt92wr-oA',
        }
    def test_validation_domain_name(self):
        self.assertEqual('_acme-challenge.www.example.com',
                         self.msg.validation_domain_name('www.example.com'))
    def test_validation(self):
        self.assertEqual(
            "rAa7iIg4K2y63fvUhCfy8dP1Xl7wEhmQq0oChTcE3Zk",
            self.msg.validation(KEY))
    def test_to_partial_json(self):
        self.assertEqual(self.jmsg, self.msg.to_partial_json())
    def test_from_json(self):
        from acme.challenges import DNS01
        self.assertEqual(self.msg, DNS01.from_json(self.jmsg))
    def test_from_json_hashable(self):
        from acme.challenges import DNS01
        hash(DNS01.from_json(self.jmsg))
 class HTTP01ResponseTest(unittest.TestCase):
    # pylint: disable=too-many-instance-attributes
@@ -0,0 +1,30 @@
 """DNS Resolver for ACME client.
 Required only for local validation of 'dns-01' challenges.
 """
 import logging
 import dns.resolver
 import dns.exception
 logger = logging.getLogger(__name__)
 def txt_records_for_name(name):
    """Resolve the name and return the TXT records.
    :param unicode name: Domain name being verified.
    :returns: A list of txt records, if empty the name could not be resolved
    :rtype: list of unicode
    """
    try:
        dns_response = dns.resolver.query(name, 'TXT')
    except dns.resolver.NXDOMAIN as error:
        return []
    except dns.exception.DNSException as error:
        logger.error("Error resolving %s: %s", name, str(error))
        return []
    return [txt_rec.decode("utf-8") for rdata in dns_response
            for txt_rec in rdata.strings]
@@ -0,0 +1,53 @@
 """Tests for acme.dns_resolver."""
 import unittest
 import mock
 from acme import dns_resolver
 try:
    import dns
 except ImportError:  # pragma: no cover
    dns = None
 def create_txt_response(name, txt_records):
    """
    Returns an RRSet containing the 'txt_records' as the result of a DNS
    query for 'name'.
    This takes advantage of the fact that an Answer object mostly behaves
    like an RRset.
    """
    return dns.rrset.from_text_list(name, 60, "IN", "TXT", txt_records)
 class TxtRecordsForNameTest(unittest.TestCase):
    @mock.patch("acme.dns_resolver.dns.resolver.query")
    def test_txt_records_for_name_with_single_response(self, mock_dns):
        mock_dns.return_value = create_txt_response('name', ['response'])
        self.assertEqual(['response'],
                         dns_resolver.txt_records_for_name('name'))
    @mock.patch("acme.dns_resolver.dns.resolver.query")
    def test_txt_records_for_name_with_multiple_responses(self, mock_dns):
        mock_dns.return_value = create_txt_response(
            'name', ['response1', 'response2'])
        self.assertEqual(['response1', 'response2'],
                         dns_resolver.txt_records_for_name('name'))
    @mock.patch("acme.dns_resolver.dns.resolver.query")
    def test_txt_records_for_name_domain_not_found(self, mock_dns):
        mock_dns.side_effect = dns.resolver.NXDOMAIN
        self.assertEquals([], dns_resolver.txt_records_for_name('name'))
    @mock.patch("acme.dns_resolver.dns.resolver.query")
    def test_txt_records_for_name_domain_other_error(self, mock_dns):
        mock_dns.side_effect = dns.exception.DNSException
        self.assertEquals([], dns_resolver.txt_records_for_name('name'))
    def run(self, result=None):
        if dns is None:  # pragma: no cover
            print(self, "... SKIPPING, no dnspython available")
            return
        super(TxtRecordsForNameTest, self).run(result)
@@ -24,7 +24,7 @@ class Fixed(jose.Field):
    def encode(self, value):
        if value != self.value:
-            logger.warn(
+            logger.warning(
                'Overriding fixed field (%s) with %r', self.json_name, value)
        return value
@@ -35,6 +35,11 @@ if sys.version_info < (2, 7):
 else:
    install_requires.append('mock')
 # dnspython 1.12 is required to support both Python 2 and Python 3.
 dns_extras = [
    'dnspython>=1.12',
 ]
 dev_extras = [
    'nose',
    'pep8',
@@ -76,6 +81,7 @@ setup(
    include_package_data=True,
    install_requires=install_requires,
    extras_require={
        'dns': dns_extras,
        'dev': dev_extras,
        'docs': docs_extras,
    },
@@ -244,7 +244,7 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator):
        if not path["cert_path"] or not path["cert_key"]:
            # Throw some can't find all of the directives error"
-            logger.warn(
+            logger.warning(
                "Cannot find a cert or key directive in %s. "
                "VirtualHost was not modified", vhost.path)
            # Presumably break here so that the virtualhost is not modified
@@ -522,7 +522,7 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator):
        try:
            args = self.aug.match(path + "/arg")
        except RuntimeError:
-            logger.warn("Encountered a problem while parsing file: %s, skipping", path)
+            logger.warning("Encountered a problem while parsing file: %s, skipping", path)
            return None
        for arg in args:
            addrs.add(obj.Addr.fromstring(self.parser.get_arg(arg)))
@@ -538,6 +538,9 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator):
                is_ssl = True
        filename = get_file_path(self.aug.get("/augeas/files%s/path" % get_file_path(path)))
        if filename is None:
            return None
        if self.conf("handle-sites"):
            is_enabled = self.is_site_enabled(filename)
        else:
@@ -1089,7 +1092,7 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator):
        try:
            func(self.choose_vhost(domain), options)
        except errors.PluginError:
-            logger.warn("Failed %s for %s", enhancement, domain)
+            logger.warning("Failed %s for %s", enhancement, domain)
            raise
    def _enable_ocsp_stapling(self, ssl_vhost, unused_options):
@@ -1276,9 +1279,9 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator):
            # but redirect loops are possible in very obscure cases; see #1620
            # for reasoning.
            if self._is_rewrite_exists(general_vh):
-                logger.warn("Added an HTTP->HTTPS rewrite in addition to "
+                logger.warning("Added an HTTP->HTTPS rewrite in addition to "
-                            "other RewriteRules; you may wish to check for "
+                               "other RewriteRules; you may wish to check for "
-                            "overall consistency.")
+                               "overall consistency.")
            # Add directives to server
            # Note: These are not immediately searchable in sites-enabled
@@ -1801,25 +1804,25 @@ def get_file_path(vhost_path):
    :rtype: str
    """
-    # Strip off /files
+    # Strip off /files/
-    avail_fp = vhost_path[6:]
+    try:
-    # This can be optimized...
+        if vhost_path.startswith("/files/"):
-    while True:
+            avail_fp = vhost_path[7:].split("/")
-        # Cast all to lowercase to be case insensitive
+        else:
-        find_if = avail_fp.lower().find("/ifmodule")
+            return None
-        if find_if != -1:
+    except AttributeError:
-            avail_fp = avail_fp[:find_if]
+        # If we recieved a None path
-            continue
+        return None
-        find_vh = avail_fp.lower().find("/virtualhost")
+
-        if find_vh != -1:
+    last_good = ""
-            avail_fp = avail_fp[:find_vh]
+    # Loop through the path parts and validate after every addition
-            continue
+    for p in avail_fp:
-        find_macro = avail_fp.lower().find("/macro")
+        cur_path = last_good+"/"+p
-        if find_macro != -1:
+        if os.path.exists(cur_path):
-            avail_fp = avail_fp[:find_macro]
+            last_good = cur_path
-            continue
+        else:
-        break
+            break
-    return avail_fp
+    return last_good
 def install_ssl_options_conf(options_ssl):
@@ -2,7 +2,23 @@
 import pkg_resources
 from certbot import util
-
+CLI_DEFAULTS_DEFAULT = dict(
    server_root="/etc/apache2",
    vhost_root="/etc/apache2/sites-available",
    vhost_files="*",
    version_cmd=['apache2ctl', '-v'],
    define_cmd=['apache2ctl', '-t', '-D', 'DUMP_RUN_CFG'],
    restart_cmd=['apache2ctl', 'graceful'],
    conftest_cmd=['apache2ctl', 'configtest'],
    enmod=None,
    dismod=None,
    le_vhost_ext="-le-ssl.conf",
    handle_mods=False,
    handle_sites=False,
    challenge_location="/etc/apache2",
    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
        "certbot_apache", "options-ssl-apache.conf")
 )
 CLI_DEFAULTS_DEBIAN = dict(
    server_root="/etc/apache2",
    vhost_root="/etc/apache2/sites-available",
@@ -71,7 +87,25 @@ CLI_DEFAULTS_DARWIN = dict(
    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
        "certbot_apache", "options-ssl-apache.conf")
 )
 CLI_DEFAULTS_SUSE = dict(
    server_root="/etc/apache2",
    vhost_root="/etc/apache2/vhosts.d",
    vhost_files="*.conf",
    version_cmd=['apache2ctl', '-v'],
    define_cmd=['apache2ctl', '-t', '-D', 'DUMP_RUN_CFG'],
    restart_cmd=['apache2ctl', 'graceful'],
    conftest_cmd=['apache2ctl', 'configtest'],
    enmod="a2enmod",
    dismod="a2dismod",
    le_vhost_ext="-le-ssl.conf",
    handle_mods=False,
    handle_sites=False,
    challenge_location="/etc/apache2/vhosts.d",
    MOD_SSL_CONF_SRC=pkg_resources.resource_filename(
        "certbot_apache", "options-ssl-apache.conf")
 )
 CLI_DEFAULTS = {
    "default": CLI_DEFAULTS_DEFAULT,
    "debian": CLI_DEFAULTS_DEBIAN,
    "ubuntu": CLI_DEFAULTS_DEBIAN,
    "centos": CLI_DEFAULTS_CENTOS,
@@ -83,6 +117,8 @@ CLI_DEFAULTS = {
    "gentoo": CLI_DEFAULTS_GENTOO,
    "gentoo base system": CLI_DEFAULTS_GENTOO,
    "darwin": CLI_DEFAULTS_DARWIN,
    "opensuse": CLI_DEFAULTS_SUSE,
    "suse": CLI_DEFAULTS_SUSE,
 }
 """CLI defaults."""
@@ -115,13 +151,36 @@ HEADER_ARGS = {"Strict-Transport-Security": HSTS_ARGS,
 def os_constant(key):
-    """Get a constant value for operating system
+    """
    Get a constant value for operating system
    :param key: name of cli constant
    :return: value of constant for active os
    """
    os_info = util.get_os_info()
    try:
        constants = CLI_DEFAULTS[os_info[0].lower()]
    except KeyError:
-        constants = CLI_DEFAULTS["debian"]
+        constants = os_like_constants()
        if not constants:
            constants = CLI_DEFAULTS["default"]
    return constants[key]
 def os_like_constants():
    """
    Try to get constants for distribution with
    similar layout and configuration, indicated by
    /etc/os-release variable "LIKE"
    :returns: Constants dictionary
    :rtype: `dict`
    """
    os_like = util.get_systemd_os_like()
    if os_like:
        for os_name in os_like:
            if os_name in CLI_DEFAULTS.keys():
                return CLI_DEFAULTS[os_name]
    return {}
@@ -91,7 +91,7 @@ def _vhost_menu(domain, vhosts):
               "non-interactive mode. Currently Certbot needs each vhost to be "
               "in its own conf file, and may need vhosts to be explicitly "
               "labelled with ServerName or ServerAlias directories.")
-        logger.warn(msg)
+        logger.warning(msg)
        raise errors.MissingCommandlineFlag(msg)
    return code, tag
@@ -146,7 +146,7 @@ class ApacheParser(object):
                constants.os_constant("define_cmd"))
        # Small errors that do not impede
        if proc.returncode != 0:
-            logger.warn("Error in checking parameter list: %s", stderr)
+            logger.warning("Error in checking parameter list: %s", stderr)
            raise errors.MisconfigurationError(
                "Apache is unable to check whether or not the module is "
                "loaded because Apache is misconfigured.")
@@ -56,7 +56,7 @@ class MultipleVhostsTest(util.ApacheTest):
        mock_surgery.return_value = False
        with mock.patch.dict('os.environ', silly_path):
            self.assertRaises(errors.NoInstallationError, self.config.prepare)
-            self.assertEquals(mock_surgery.call_count, 1)
+            self.assertEqual(mock_surgery.call_count, 1)
    @mock.patch("certbot_apache.augeas_configurator.AugeasConfigurator.init_augeas")
    def test_prepare_no_augeas(self, mock_init_augeas):
@@ -125,6 +125,12 @@ class MultipleVhostsTest(util.ApacheTest):
        self.assertTrue("google.com" in names)
        self.assertTrue("certbot.demo" in names)
    def test_get_bad_path(self):
        from certbot_apache.configurator import get_file_path
        self.assertEqual(get_file_path(None), None)
        self.assertEqual(get_file_path("nonexistent"), None)
        self.assertEqual(self.config._create_vhost("nonexistent"), None) # pylint: disable=protected-access
    def test_bad_servername_alias(self):
        ssl_vh1 = obj.VirtualHost(
            "fp1", "ap1", set([obj.Addr(("*", "443"))]),
@@ -1242,8 +1248,8 @@ class MultipleVhostsTest(util.ApacheTest):
        mock_match = mock.Mock(return_value=["something"])
        self.config.aug.match = mock_match
        # pylint: disable=protected-access
-        self.assertEquals(self.config._check_aug_version(),
+        self.assertEqual(self.config._check_aug_version(),
-                          ["something"])
+                         ["something"])
        self.config.aug.match.side_effect = RuntimeError
        self.assertFalse(self.config._check_aug_version())
@@ -25,3 +25,20 @@ class ConstantsTest(unittest.TestCase):
        os_info.return_value = ('Nonexistent Linux', '', '')
        self.assertEqual(constants.os_constant("vhost_root"),
                         "/etc/apache2/sites-available")
    @mock.patch("certbot.util.get_os_info")
    def test_get_default_constants(self, os_info):
        os_info.return_value = ('Nonexistent Linux', '', '')
        with mock.patch("certbot.util.get_systemd_os_like") as os_like:
            # Get defaults
            os_like.return_value = False
            c_hm = constants.os_constant("handle_mods")
            c_sr = constants.os_constant("server_root")
            self.assertFalse(c_hm)
            self.assertEqual(c_sr, "/etc/apache2")
            # Use darwin as like test target
            os_like.return_value = ["something", "nonexistent", "darwin"]
            d_vr = constants.os_constant("vhost_root")
            d_em = constants.os_constant("enmod")
            self.assertFalse(d_em)
            self.assertEqual(d_vr, "/etc/apache2/other")
@@ -129,7 +129,7 @@ class ApacheTlsSni01(common.TLSSNI01):
            # because it's a new vhost that's not configured yet (GH #677),
            # or perhaps because there were multiple <VirtualHost> sections
            # in the config file (GH #1042).  See also GH #2600.
-            logger.warn("Falling back to default vhost %s...", default_addr)
+            logger.warning("Falling back to default vhost %s...", default_addr)
            addrs.add(default_addr)
            return addrs
@@ -0,0 +1,51 @@
 FROM debian:jessie
 MAINTAINER Brad Warren <bmw@eff.org>
 # no need to mkdir anything:
 # https://docs.docker.com/reference/builder/#copy
 # If <dest> doesn't exist, it is created along with all missing
 # directories in its path.
 # TODO: Install non-default Python versions for tox.
 # TODO: Install Apache/Nginx for plugin development.
 COPY certbot-auto /opt/certbot/src/certbot-auto
 RUN /opt/certbot/src/certbot-auto -n --os-packages-only
 # the above is not likely to change, so by putting it further up the
 # Dockerfile we make sure we cache as much as possible
 COPY setup.py README.rst CHANGES.rst MANIFEST.in linter_plugin.py tox.cover.sh tox.ini pep8.travis.sh .pep8 .pylintrc /opt/certbot/src/
 # all above files are necessary for setup.py, however, package source
 # code directory has to be copied separately to a subdirectory...
 # https://docs.docker.com/reference/builder/#copy: "If <src> is a
 # directory, the entire contents of the directory are copied,
 # including filesystem metadata. Note: The directory itself is not
 # copied, just its contents." Order again matters, three files are far
 # more likely to be cached than the whole project directory
 COPY certbot /opt/certbot/src/certbot/
 COPY acme /opt/certbot/src/acme/
 COPY certbot-apache /opt/certbot/src/certbot-apache/
 COPY certbot-nginx /opt/certbot/src/certbot-nginx/
 COPY certbot-compatibility-test /opt/certbot/src/certbot-compatibility-test/
 RUN virtualenv --no-site-packages -p python2 /opt/certbot/venv && \
    /opt/certbot/venv/bin/pip install -U setuptools && \
    /opt/certbot/venv/bin/pip install -U pip && \
    /opt/certbot/venv/bin/pip install \
    -e /opt/certbot/src/acme \
    -e /opt/certbot/src \
    -e /opt/certbot/src/certbot-apache \
    -e /opt/certbot/src/certbot-nginx \
    -e /opt/certbot/src/certbot-compatibility-test \
    -e /opt/certbot/src[dev,docs]
 # install in editable mode (-e) to save space: it's not possible to
 # "rm -rf /opt/certbot/src" (it's stays in the underlaying image);
 # this might also help in debugging: you can "docker run --entrypoint
 # bash" and investigate, apply patches, etc.
 WORKDIR /opt/certbot/src/certbot-compatibility-test/certbot_compatibility_test/testdata
 ENV PATH /opt/certbot/venv/bin:$PATH
@@ -0,0 +1,6 @@
 FROM certbot-compatibility-test
 MAINTAINER Brad Warren <bmw@eff.org>
 RUN apt-get install apache2 -y
 ENTRYPOINT [ "certbot-compatibility-test", "-p", "apache" ]
@@ -0,0 +1,6 @@
 FROM certbot-compatibility-test
 MAINTAINER Brad Warren <bmw@eff.org>
 RUN apt-get install nginx -y
 ENTRYPOINT [ "certbot-compatibility-test", "-p", "nginx" ]
@@ -1,20 +0,0 @@
 FROM httpd
 MAINTAINER Brad Warren <bradmw@umich.edu>
 RUN mkdir /var/run/apache2
 ENV APACHE_RUN_USER=daemon \
    APACHE_RUN_GROUP=daemon \
    APACHE_PID_FILE=/usr/local/apache2/logs/httpd.pid \
    APACHE_RUN_DIR=/var/run/apache2 \
    APACHE_LOCK_DIR=/var/lock \
    APACHE_LOG_DIR=/usr/local/apache2/logs 
 COPY certbot-compatibility-test/certbot_compatibility_test/configurators/apache/a2enmod.sh /usr/local/bin/
 COPY certbot-compatibility-test/certbot_compatibility_test/configurators/apache/a2dismod.sh /usr/local/bin/
 COPY certbot-compatibility-test/certbot_compatibility_test/testdata/rsa1024_key2.pem /usr/local/apache2/conf/
 COPY certbot-compatibility-test/certbot_compatibility_test/testdata/empty_cert.pem /usr/local/apache2/conf/
 # Note: this only exposes the port to other docker containers. You
 # still have to bind to 443@host at runtime.
 EXPOSE 443
@@ -1,5 +1,4 @@
 """Provides a common base for Apache proxies"""
 import re
 import os
 import shutil
 import subprocess
@@ -17,10 +16,6 @@ from certbot_compatibility_test import util
 from certbot_compatibility_test.configurators import common as configurators_common
 APACHE_VERSION_REGEX = re.compile(r"Apache/([0-9\.]*)", re.IGNORECASE)
 APACHE_COMMANDS = ["apachectl", "a2enmod", "a2dismod"]
@zope.interface.implementer(interfaces.IConfiguratorProxy)
 class Proxy(configurators_common.Proxy):
    # pylint: disable=too-many-instance-attributes
@@ -32,37 +27,25 @@ class Proxy(configurators_common.Proxy):
        self.le_config.apache_le_vhost_ext = "-le-ssl.conf"
        self.modules = self.server_root = self.test_conf = self.version = None
        self._apache_configurator = self._all_names = self._test_names = None
        patch = mock.patch(
            "certbot_apache.configurator.display_ops.select_vhost")
        mock_display = patch.start()
        mock_display.side_effect = le_errors.PluginError(
            "Unable to determine vhost")
    def __getattr__(self, name):
        """Wraps the Apache Configurator methods"""
        method = getattr(self._apache_configurator, name, None)
        if callable(method):
            return method
        else:
            raise AttributeError()
    def load_config(self):
        """Loads the next configuration for the plugin to test"""
        config = super(Proxy, self).load_config()
        self._all_names, self._test_names = _get_names(config)
        server_root = _get_server_root(config)
        # with open(os.path.join(config, "config_file")) as f:
        #    config_file = os.path.join(server_root, f.readline().rstrip())
        shutil.rmtree("/etc/apache2")
        shutil.copytree(server_root, "/etc/apache2", symlinks=True)
        self._prepare_configurator()
        try:
-            subprocess.check_call("apachectl -k start".split())
+            subprocess.check_call("apachectl -k restart".split())
        except errors.Error:
            raise errors.Error(
                "Apache failed to load {0} before tests started".format(
@@ -78,38 +61,16 @@ class Proxy(configurators_common.Proxy):
        # An alias
        self.le_config.apache_handle_modules = self.le_config.apache_handle_mods
-        self._apache_configurator = configurator.ApacheConfigurator(
+        self._configurator = configurator.ApacheConfigurator(
            config=configuration.NamespaceConfig(self.le_config),
            name="apache")
-        self._apache_configurator.prepare()
+        self._configurator.prepare()
    def cleanup_from_tests(self):
        """Performs any necessary cleanup from running plugin tests"""
        super(Proxy, self).cleanup_from_tests()
        mock.patch.stopall()
    def get_all_names_answer(self):
        """Returns the set of domain names that the plugin should find"""
        if self._all_names:
            return self._all_names
        else:
            raise errors.Error("No configuration file loaded")
    def get_testable_domain_names(self):
        """Returns the set of domain names that can be tested against"""
        if self._test_names:
            return self._test_names
        else:
            return {"example.com"}
    def deploy_cert(self, domain, cert_path, key_path, chain_path=None,
                    fullchain_path=None):
        """Installs cert"""
        cert_path, key_path, chain_path = self.copy_certs_and_keys(
            cert_path, key_path, chain_path)
        self._apache_configurator.deploy_cert(
            domain, cert_path, key_path, chain_path, fullchain_path)
 def _get_server_root(config):
    """Returns the server root directory in config"""
@@ -5,6 +5,7 @@ import shutil
 import tempfile
 from certbot import constants
 from certbot_compatibility_test import errors
 from certbot_compatibility_test import util
@@ -31,6 +32,18 @@ class Proxy(object):
        self.args = args
        self.http_port = 80
        self.https_port = 443
        self._configurator = self._all_names = self._test_names = None
    def __getattr__(self, name):
        """Wraps the configurator methods"""
        if self._configurator is None:
            raise AttributeError()
        method = getattr(self._configurator, name, None)
        if callable(method):
            return method
        else:
            raise AttributeError()
    def has_more_configs(self):
        """Returns true if there are more configs to test"""
@@ -63,3 +76,25 @@ class Proxy(object):
            chain = None
        return cert, key, chain
    def get_all_names_answer(self):
        """Returns the set of domain names that the plugin should find"""
        if self._all_names:
            return self._all_names
        else:
            raise errors.Error("No configuration file loaded")
    def get_testable_domain_names(self):
        """Returns the set of domain names that can be tested against"""
        if self._test_names:
            return self._test_names
        else:
            return {"example.com"}
    def deploy_cert(self, domain, cert_path, key_path, chain_path=None,
                    fullchain_path=None):
        """Installs cert"""
        cert_path, key_path, chain_path = self.copy_certs_and_keys(
            cert_path, key_path, chain_path)
        self._configurator.deploy_cert(
            domain, cert_path, key_path, chain_path, fullchain_path)
@@ -0,0 +1 @@
 """Certbot compatibility test Nginx configurators"""
@@ -0,0 +1,84 @@
 """Provides a common base for Nginx proxies"""
 import os
 import shutil
 import subprocess
 import zope.interface
 from certbot import configuration
 from certbot_nginx import configurator
 from certbot_nginx import constants
 from certbot_compatibility_test import errors
 from certbot_compatibility_test import interfaces
 from certbot_compatibility_test import util
 from certbot_compatibility_test.configurators import common as configurators_common
@zope.interface.implementer(interfaces.IConfiguratorProxy)
 class Proxy(configurators_common.Proxy):
    # pylint: disable=too-many-instance-attributes
    """A common base for Nginx test configurators"""
    def __init__(self, args):
        """Initializes the plugin with the given command line args"""
        super(Proxy, self).__init__(args)
    def load_config(self):
        """Loads the next configuration for the plugin to test"""
        config = super(Proxy, self).load_config()
        self._all_names, self._test_names = _get_names(config)
        server_root = _get_server_root(config)
        # XXX: Deleting all of this is kind of scary unless the test
        #      instances really each have a complete configuration!
        shutil.rmtree("/etc/nginx")
        shutil.copytree(server_root, "/etc/nginx", symlinks=True)
        self._prepare_configurator()
        try:
            subprocess.check_call("service nginx reload".split())
        except errors.Error:
            raise errors.Error(
                "Nginx failed to load {0} before tests started".format(
                    config))
        return config
    def _prepare_configurator(self):
        """Prepares the Nginx plugin for testing"""
        for k in constants.CLI_DEFAULTS.keys():
            setattr(self.le_config, "nginx_" + k, constants.os_constant(k))
        conf = configuration.NamespaceConfig(self.le_config)
        zope.component.provideUtility(conf)
        self._configurator = configurator.NginxConfigurator(
            config=conf, name="nginx")
        self._configurator.prepare()
 def _get_server_root(config):
    """Returns the server root directory in config"""
    subdirs = [
        name for name in os.listdir(config)
        if os.path.isdir(os.path.join(config, name))]
    if len(subdirs) != 1:
        raise errors.Error("Malformed configuration directory {0}".format(config))
    return os.path.join(config, subdirs[0].rstrip())
 def _get_names(config):
    """Returns all and testable domain names in config"""
    all_names = set()
    for root, _dirs, files in os.walk(config):
        for this_file in files:
            for line in open(os.path.join(root, this_file)):
                if line.strip().startswith("server_name"):
                    names = line.partition("server_name")[2].rpartition(";")[0]
                    for n in names.split():
                        all_names.add(n)
    non_ip_names = set(n for n in all_names if not util.IP_REGEX.match(n))
    return all_names, non_ip_names
@@ -22,7 +22,8 @@ from certbot_compatibility_test import errors
 from certbot_compatibility_test import util
 from certbot_compatibility_test import validator
-from certbot_compatibility_test.configurators.apache import common
+from certbot_compatibility_test.configurators.apache import common as a_common
 from certbot_compatibility_test.configurators.nginx import common as n_common
 DESCRIPTION = """
@@ -32,7 +33,7 @@ tests that the plugin supports are performed.
 """
-PLUGINS = {"apache": common.Proxy}
+PLUGINS = {"apache": a_common.Proxy, "nginx": n_common.Proxy}
 logger = logging.getLogger(__name__)
@@ -143,7 +144,7 @@ def test_deploy_cert(plugin, temp_dir, domains):
    for domain in domains:
        try:
-            plugin.deploy_cert(domain, cert_path, util.KEY_PATH, cert_path)
+            plugin.deploy_cert(domain, cert_path, util.KEY_PATH, cert_path, cert_path)
            plugin.save()  # Needed by the Apache plugin
        except le_errors.Error as error:
            logger.error("Plugin failed to deploy ceritificate for %s:", domain)
@@ -369,10 +370,10 @@ def main():
        plugin.cleanup_from_tests()
    if overall_success:
-        logger.warn("All compatibility tests succeeded")
+        logger.warning("All compatibility tests succeeded")
        sys.exit(0)
    else:
-        logger.warn("One or more compatibility tests failed")
+        logger.warning("One or more compatibility tests failed")
        sys.exit(1)
@@ -160,9 +160,9 @@ class NginxConfigurator(common.Plugin):
        stapling_directives = []
        if self.version >= (1, 3, 7):
            stapling_directives = [
-                ['\n', 'ssl_trusted_certificate', ' ', chain_path],
+                ['\n    ', 'ssl_trusted_certificate', ' ', chain_path],
-                ['\n', 'ssl_stapling', ' ', 'on'],
+                ['\n    ', 'ssl_stapling', ' ', 'on'],
-                ['\n', 'ssl_stapling_verify', ' ', 'on'], ['\n']]
+                ['\n    ', 'ssl_stapling_verify', ' ', 'on'], ['\n']]
        if len(stapling_directives) != 0 and not chain_path:
            raise errors.PluginError(
@@ -179,7 +179,7 @@ class NginxConfigurator(common.Plugin):
                        vhost.filep, vhost.names)
        except errors.MisconfigurationError as error:
            logger.debug(error)
-            logger.warn(
+            logger.warning(
                "Cannot find a cert or key directive in %s for %s. "
                "VirtualHost was not modified.", vhost.filep, vhost.names)
            # Presumably break here so that the virtualhost is not modified
@@ -337,10 +337,10 @@ class NginxConfigurator(common.Plugin):
        """
        snakeoil_cert, snakeoil_key = self._get_snakeoil_paths()
-        ssl_block = [['\n', 'listen', ' ', '{0} ssl'.format(self.config.tls_sni_01_port)],
+        ssl_block = [['\n    ', 'listen', ' ', '{0} ssl'.format(self.config.tls_sni_01_port)],
-                     ['\n', 'ssl_certificate', ' ', snakeoil_cert],
+                     ['\n    ', 'ssl_certificate', ' ', snakeoil_cert],
-                     ['\n', 'ssl_certificate_key', ' ', snakeoil_key],
+                     ['\n    ', 'ssl_certificate_key', ' ', snakeoil_key],
-                     ['\n', 'include', ' ', self.parser.loc["ssl_options"]]]
+                     ['\n    ', 'include', ' ', self.parser.loc["ssl_options"]]]
        self.parser.add_server_directives(
            vhost.filep, vhost.names, ssl_block, replace=False)
        vhost.ssl = True
@@ -385,7 +385,7 @@ class NginxConfigurator(common.Plugin):
            raise errors.PluginError(
                "Unsupported enhancement: {0}".format(enhancement))
        except errors.PluginError:
-            logger.warn("Failed %s for %s", enhancement, domain)
+            logger.warning("Failed %s for %s", enhancement, domain)
    def _enable_redirect(self, vhost, unused_options):
        """Redirect all equivalent HTTP traffic to ssl_vhost.
@@ -401,9 +401,10 @@ class NginxConfigurator(common.Plugin):
        :type unused_options: Not Available
        """
        redirect_block = [[
-            ['if', '($scheme != "https")'],
+            ['\n    ', 'if', ' ', '($scheme != "https") '],
-            [['return', '301 https://$host$request_uri']]
+            [['\n        ', 'return', ' ', '301 https://$host$request_uri'],
-        ]]
+             '\n    ']
        ], ['\n']]
        self.parser.add_server_directives(
            vhost.filep, vhost.names, redirect_block, replace=False)
        logger.info("Redirecting all traffic to ssl in %s", vhost.filep)
@@ -16,3 +16,17 @@ MOD_SSL_CONF_SRC = pkg_resources.resource_filename(
    "certbot_nginx", "options-ssl-nginx.conf")
 """Path to the nginx mod_ssl config file found in the Certbot
 distribution."""
 def os_constant(key):
    # XXX TODO: In the future, this could return different constants
    #           based on what OS we are running under.  To see an
    #           approach to how to handle different OSes, see the
    #           apache version of this file.  Currently, we do not
    #           actually have any OS-specific constants on Nginx.
    """
    Get a constant value for operating system
    :param key: name of cli constant
    :return: value of constant for active os
    """
    return CLI_DEFAULTS[key]
@@ -25,11 +25,19 @@ class RawNginxParser(object):
    key = Word(alphanums + "_/+-.")
    dollar_var = Combine(Literal('$') + Regex(r"[^\{\};,\s]+"))
    condition = Regex(r"\(.+\)")
-    # Matches anything that is not a special character AND any chars in single
+    # Matches anything that is not a special character, and ${SHELL_VARS}, AND
-    # or double quotes
+    # any chars in single or double quotes
    # All of these COULD be upgraded to something like
    # https://stackoverflow.com/a/16130746
-    value = Regex(r"((\".*\")?(\'.*\')?[^\{\};,]?)+")
+    dquoted = Regex(r'(\".*\")')
    squoted = Regex(r"(\'.*\')")
    nonspecial = Regex(r"[^\{\};,]")
    varsub = Regex(r"(\$\{\w+\})")
    # nonspecial nibbles one character at a time, but the other objects take
    # precedence.  We use ZeroOrMore to allow entries like "break ;" to be
    # parsed as assignments
    value = Combine(ZeroOrMore(dquoted | squoted | varsub | nonspecial))
    location = CharsNotIn("{};," + string.whitespace)
    # modifier for location uri [ = | ~ | ~* | ^~ ]
    modifier = Literal("=") | Literal("~*") | Literal("~") | Literal("^~")
@@ -40,6 +48,7 @@ class RawNginxParser(object):
    assignment = space + key + Optional(space + value, default=None) + semicolon
    location_statement = space + Optional(modifier) + Optional(space + location + space)
    if_statement = space + Literal("if") + space + condition + space
    charset_map_statement = space + Literal("charset_map") + space + value + space + value
    map_statement = space + Literal("map") + space + nonspace + space + dollar_var + space
    # This is NOT an accurate way to parse nginx map entries; it's almost
@@ -52,24 +61,27 @@ class RawNginxParser(object):
    map_pattern = Regex(r'".*"') | Regex(r"'.*'") | nonspace
    map_entry = space + map_pattern + space + value + space + semicolon
    map_block = Group(
        # key could for instance be "server" or "http", or "location" (in which case
        # location_statement needs to have a non-empty location)
        Group(map_statement).leaveWhitespace() +
        left_bracket +
        Group(ZeroOrMore(Group(comment | map_entry)) + space).leaveWhitespace() +
        right_bracket)
    block = Forward()
-    block << Group(
+
-        # key could for instance be "server" or "http", or "location" (in which case
+    # key could for instance be "server" or "http", or "location" (in which case
-        # location_statement needs to have a non-empty location)
+    # location_statement needs to have a non-empty location)
-        (Group(space + key + location_statement) ^ Group(if_statement)).leaveWhitespace() +
+
-        left_bracket +
+    block_begin = (Group(space + key + location_statement) ^
-        Group(ZeroOrMore(Group(comment | assignment) | block | map_block) + space).leaveWhitespace()
+                   Group(if_statement) ^
-              + right_bracket)
+                   Group(charset_map_statement)).leaveWhitespace()
    block_innards = Group(ZeroOrMore(Group(comment | assignment) | block | map_block)
                          + space).leaveWhitespace()
    block << Group(block_begin + left_bracket + block_innards + right_bracket)
    script = OneOrMore(Group(comment | assignment) ^ block ^ map_block) + space + stringEnd
-    script.parseWithTabs()
+    script.parseWithTabs().leaveWhitespace()
    def __init__(self, source):
        self.source = source
@@ -168,7 +168,7 @@ class NginxParser(object):
                    self.parsed[item] = parsed
                    trees.append(parsed)
            except IOError:
-                logger.warn("Could not open file: %s", item)
+                logger.warning("Could not open file: %s", item)
            except pyparsing.ParseException:
                logger.debug("Could not parse file: %s", item)
        return trees
@@ -37,8 +37,8 @@ class NginxConfiguratorTest(util.NginxTest):
            errors.NoInstallationError, self.config.prepare)
    def test_prepare(self):
-        self.assertEquals((1, 6, 2), self.config.version)
+        self.assertEqual((1, 6, 2), self.config.version)
-        self.assertEquals(5, len(self.config.parser.parsed))
+        self.assertEqual(5, len(self.config.parser.parsed))
    @mock.patch("certbot_nginx.configurator.util.exe_exists")
    @mock.patch("certbot_nginx.configurator.subprocess.Popen")
@@ -56,7 +56,7 @@ class NginxConfiguratorTest(util.NginxTest):
        self.config.version = None
        self.config.config_test = mock.Mock()
        self.config.prepare()
-        self.assertEquals((1, 6, 2), self.config.version)
+        self.assertEqual((1, 6, 2), self.config.version)
    @mock.patch("certbot_nginx.configurator.socket.gethostbyaddr")
    def test_get_all_names(self, mock_gethostbyaddr):
@@ -415,7 +415,7 @@ class NginxConfiguratorTest(util.NginxTest):
    def test_redirect_enhance(self):
        expected = [
-            ['if', '($scheme != "https")'],
+            ['if', '($scheme != "https") '],
            [['return', '301 https://$host$request_uri']]
        ]
@@ -134,8 +134,8 @@ class TestRawNginxParser(unittest.TestCase):
            parsed_new = load(handle)
        try:
            self.maxDiff = None
-            self.assertEquals(parsed[0], parsed_new[0])
+            self.assertEqual(parsed[0], parsed_new[0])
-            self.assertEquals(parsed[1:], parsed_new[1:])
+            self.assertEqual(parsed[1:], parsed_new[1:])
        finally:
            os.unlink(util.get_data_filename('nginx.new.conf'))
@@ -150,7 +150,7 @@ class TestRawNginxParser(unittest.TestCase):
            parsed_new = load(handle)
        try:
-            self.assertEquals(parsed, parsed_new)
+            self.assertEqual(parsed, parsed_new)
            self.assertEqual(parsed_new, [
                ['#', " Use bar.conf when it's a full moon!"],
@@ -117,9 +117,9 @@ class NginxParserTest(util.NginxTest):
        fooconf = [x for x in vhosts if 'foo.conf' in x.filep][0]
        self.assertEqual(vhost5, fooconf)
        localhost = [x for x in vhosts if 'localhost' in x.names][0]
-        self.assertEquals(vhost1, localhost)
+        self.assertEqual(vhost1, localhost)
        somename = [x for x in vhosts if 'somename' in x.names][0]
-        self.assertEquals(vhost2, somename)
+        self.assertEqual(vhost2, somename)
    def test_add_server_directives(self):
        nparser = parser.NginxParser(self.config_path, self.ssl_options)
@@ -91,10 +91,10 @@ class NginxTlsSni01(common.TLSSNI01):
        # Add the 'include' statement for the challenges if it doesn't exist
        # already in the main config
        included = False
-        include_directive = ['include', ' ', self.challenge_conf]
+        include_directive = ['\n', 'include', ' ', self.challenge_conf]
        root = self.configurator.parser.loc["root"]
-        bucket_directive = ['server_names_hash_bucket_size', ' ', '128']
+        bucket_directive = ['\n', 'server_names_hash_bucket_size', ' ', '128']
        main = self.configurator.parser.parsed[root]
        for key, body in main:
@@ -1,8 +1,8 @@
 """ACME AuthHandler."""
 import itertools
 import logging
 import time
 import six
 import zope.component
 from acme import challenges
@@ -141,7 +141,7 @@ class AuthHandler(object):
        """
        active_achalls = []
-        for achall, resp in itertools.izip(achalls, resps):
+        for achall, resp in six.moves.zip(achalls, resps):
            # This line needs to be outside of the if block below to
            # ensure failed challenges are cleaned up correctly
            active_achalls.append(achall)
@@ -472,7 +472,7 @@ def _report_failed_challs(failed_achalls):
            problems.setdefault(achall.error.typ, []).append(achall)
    reporter = zope.component.getUtility(interfaces.IReporter)
-    for achalls in problems.itervalues():
+    for achalls in six.itervalues(problems):
        reporter.add_message(
            _generate_failed_chall_msg(achalls), reporter.MEDIUM_PRIORITY)
@@ -158,7 +158,7 @@ def possible_deprecation_warning(config):
        # need warnings
        return
    if "CERTBOT_AUTO" not in os.environ:
-        logger.warn("You are running with an old copy of letsencrypt-auto that does "
+        logger.warning("You are running with an old copy of letsencrypt-auto that does "
            "not receive updates, and is less reliable than more recent versions. "
            "We recommend upgrading to the latest certbot-auto script, or using native "
            "OS packages.")
@@ -343,8 +343,10 @@ class HelpfulArgumentParser(object):
        self.determine_verb()
        help1 = self.prescan_for_flag("-h", self.help_topics)
        help2 = self.prescan_for_flag("--help", self.help_topics)
-        assert max(True, "a") == "a", "Gravity changed direction"
+        if isinstance(help1, bool) and isinstance(help2, bool):
-        self.help_arg = max(help1, help2)
+            self.help_arg = help1 or help2
        else:
            self.help_arg = help1 if isinstance(help1, str) else help2
        if self.help_arg is True:
            # just --help with no topic; avoid argparse altogether
            print(usage)
@@ -104,10 +104,10 @@ def register(config, account_storage, tos_cb=None):
        if not config.register_unsafely_without_email:
            msg = ("No email was provided and "
                   "--register-unsafely-without-email was not present.")
-            logger.warn(msg)
+            logger.warning(msg)
            raise errors.Error(msg)
        if not config.dry_run:
-            logger.warn("Registering without email!")
+            logger.warning("Registering without email!")
    # Each new registration shall use a fresh new key
    key = jose.JWKRSA(key=jose.ComparableRSAKey(
@@ -453,10 +453,10 @@ class Client(object):
                try:
                    self.installer.enhance(dom, enhancement, options)
                except errors.PluginEnhancementAlreadyPresent:
-                    logger.warn("Enhancement %s was already set.",
+                    logger.warning("Enhancement %s was already set.",
                            enhancement)
                except errors.PluginError:
-                    logger.warn("Unable to set enhancement %s for %s",
+                    logger.warning("Unable to set enhancement %s for %s",
                            enhancement, dom)
                    raise
@@ -18,7 +18,7 @@ CLI_DEFAULTS = dict(
        os.path.join(os.environ.get("XDG_CONFIG_HOME", "~/.config"),
                     "letsencrypt", "cli.ini"),
    ],
-    verbose_count=-(logging.INFO / 10),
+    verbose_count=-int(logging.INFO / 10),
    server="https://acme-v01.api.letsencrypt.org/directory",
    rsa_key_size=2048,
    rollback_checkpoints=1,
@@ -53,7 +53,7 @@ def post_hook(config, final=False):
        if not pre_hook.already:
            logger.info("No renewals attempted, so not running post-hook")
            if config.verb != "renew":
-                logger.warn("Sanity failure in renewal hooks")
+                logger.warning("Sanity failure in renewal hooks")
            return
        if final or config.verb != "renew":
            logger.info("Running post-hook command: %s", config.post_hook)
@@ -773,5 +773,5 @@ def main(cli_args=sys.argv[1:]):
 if __name__ == "__main__":
    err_string = main()
    if err_string:
-        logger.warn("Exiting with message %s", err_string)
+        logger.warning("Exiting with message %s", err_string)
    sys.exit(err_string)  # pragma: no cover
@@ -3,6 +3,7 @@ import collections
 import itertools
 import logging
 import pkg_resources
 import six
 import zope.interface
 import zope.interface.verify
@@ -194,12 +195,12 @@ class PluginsRegistry(collections.Mapping):
    def init(self, config):
        """Initialize all plugins in the registry."""
        return [plugin_ep.init(config) for plugin_ep
-                in self._plugins.itervalues()]
+                in six.itervalues(self._plugins)]
    def filter(self, pred):
        """Filter plugins based on predicate."""
        return type(self)(dict((name, plugin_ep) for name, plugin_ep
-                               in self._plugins.iteritems() if pred(plugin_ep)))
+                               in six.iteritems(self._plugins) if pred(plugin_ep)))
    def visible(self):
        """Filter plugins based on visibility."""
@@ -216,7 +217,7 @@ class PluginsRegistry(collections.Mapping):
    def prepare(self):
        """Prepare all plugins in the registry."""
-        return [plugin_ep.prepare() for plugin_ep in self._plugins.itervalues()]
+        return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
    def available(self):
        """Filter plugins based on availability."""
@@ -238,7 +239,7 @@ class PluginsRegistry(collections.Mapping):
        """
        # use list instead of set because PluginEntryPoint is not hashable
-        candidates = [plugin_ep for plugin_ep in self._plugins.itervalues()
+        candidates = [plugin_ep for plugin_ep in six.itervalues(self._plugins)
                      if plugin_ep.initialized and plugin_ep.init() is plugin]
        assert len(candidates) <= 1
        if candidates:
@@ -249,7 +250,7 @@ class PluginsRegistry(collections.Mapping):
    def __repr__(self):
        return "{0}({1})".format(
            self.__class__.__name__, ','.join(
-                repr(p_ep) for p_ep in self._plugins.itervalues()))
+                repr(p_ep) for p_ep in six.itervalues(self._plugins)))
    def __str__(self):
        if not self._plugins:
@@ -10,6 +10,7 @@ import sys
 import tempfile
 import time
 import six
 import zope.component
 import zope.interface
@@ -187,7 +188,7 @@ s.serve_forever()" """
        #answer = zope.component.getUtility(interfaces.IDisplay).notification(
        #    message=message, height=25, pause=True)
        sys.stdout.write(message)
-        raw_input("Press ENTER to continue")
+        six.moves.input("Press ENTER to continue")
    def cleanup(self, achalls):
        # pylint: disable=missing-docstring,no-self-use,unused-argument
@@ -84,7 +84,7 @@ def pick_plugin(config, default, plugins, question, ifaces):
        else:
            return plugin_ep.init()
    elif len(prepared) == 1:
-        plugin_ep = prepared.values()[0]
+        plugin_ep = list(prepared.values())[0]
        logger.debug("Single candidate plugin: %s", plugin_ep)
        if plugin_ep.misconfigured:
            return None
@@ -174,7 +174,7 @@ def choose_configurator_plugins(config, plugins, verb):
    if verb == "install":
        need_inst = True
        if config.authenticator:
-            logger.warn("Specifying an authenticator doesn't make sense in install mode")
+            logger.warning("Specifying an authenticator doesn't make sense in install mode")
    # Try to meet the user's request and/or ask them to pick plugins
    authenticator = installer = None
@@ -3,15 +3,27 @@ import logging
 import os
 import socket
 import psutil
 import zope.component
 from certbot import interfaces
 from certbot import util
 try:
    import psutil
    USE_PSUTIL = True
 except ImportError:
    USE_PSUTIL = False
 logger = logging.getLogger(__name__)
 RENEWER_EXTRA_MSG = (
    " For automated renewal, you may want to use a script that stops"
    " and starts your webserver. You can find an example at"
    " https://certbot.eff.org/docs/using.html#renewal ."
    " Alternatively you can use the webroot plugin to renew without"
    " needing to stop and start your webserver.")
 def path_surgery(restart_cmd):
    """Attempt to perform PATH surgery to find restart_cmd
@@ -38,9 +50,11 @@ def path_surgery(restart_cmd):
        return True
    else:
        expanded = " expanded" if any(added) else ""
-        logger.warn("Failed to find %s in%s PATH: %s", restart_cmd, expanded, path)
+        logger.warning("Failed to find %s in%s PATH: %s", restart_cmd,
                       expanded, path)
        return False
 def already_listening(port, renewer=False):
    """Check if a process is already listening on the port.
@@ -53,6 +67,50 @@ def already_listening(port, renewer=False):
    :param int port: The TCP port in question.
    :returns: True or False.
    """
    if USE_PSUTIL:
        return already_listening_psutil(port, renewer=renewer)
    else:
        logger.debug("Psutil not found, using simple socket check.")
        return already_listening_socket(port, renewer=renewer)
 def already_listening_socket(port, renewer=False):
    """Simple socket based check to find out if port is already in use
    :param int port: The TCP port in question.
    :returns: True or False
    """
    try:
        testsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
        try:
            testsocket.bind(("", port))
        except socket.error:
            display = zope.component.getUtility(interfaces.IDisplay)
            extra = ""
            if renewer:
                extra = RENEWER_EXTRA_MSG
            display.notification(
                "Port {0} is already in use by another process. This will "
                "prevent us from binding to that port. Please stop the "
                "process that is populating the port in question and try "
                "again. {1}".format(port, extra), height=13)
            return True
        finally:
            testsocket.close()
    except socket.error:
        pass
    return False
 def already_listening_psutil(port, renewer=False):
    """Psutil variant of the open port check
    :param int port: The TCP port in question.
    :returns: True or False.
    """
    try:
        net_connections = psutil.net_connections()
@@ -81,12 +139,7 @@ def already_listening(port, renewer=False):
            display = zope.component.getUtility(interfaces.IDisplay)
            extra = ""
            if renewer:
-                extra = (
+                extra = RENEWER_EXTRA_MSG
                    " For automated renewal, you may want to use a script that stops"
                    " and starts your webserver. You can find an example at"
                    " https://letsencrypt.org/howitworks/#writing-your-own-renewal-script"
                    ". Alternatively you can use the webroot plugin to renew without"
                    " needing to stop and start your webserver.")
            display.notification(
                "The program {0} (process ID {1}) is already listening "
                "on TCP port {2}. This will prevent us from binding to "
@@ -1,14 +1,22 @@
 """Tests for certbot.plugins.util."""
 import os
 import unittest
 import sys
 import mock
-import psutil
+
 try:
    # Python 3.5+
    from importlib import reload as refresh  # pylint: disable=no-name-in-module
 except ImportError:
    # Python 2-3.4
    from imp import reload as refresh
 class PathSurgeryTest(unittest.TestCase):
    """Tests for certbot.plugins.path_surgery."""
-    @mock.patch("certbot.plugins.util.logger.warn")
+    @mock.patch("certbot.plugins.util.logger.warning")
    @mock.patch("certbot.plugins.util.logger.debug")
    def test_path_surgery(self, mock_debug, mock_warn):
        from certbot.plugins.util import path_surgery
@@ -16,20 +24,64 @@ class PathSurgeryTest(unittest.TestCase):
        with mock.patch.dict('os.environ', all_path):
            with mock.patch('certbot.util.exe_exists') as mock_exists:
                mock_exists.return_value = True
-                self.assertEquals(path_surgery("eg"), True)
+                self.assertEqual(path_surgery("eg"), True)
-                self.assertEquals(mock_debug.call_count, 0)
+                self.assertEqual(mock_debug.call_count, 0)
-                self.assertEquals(mock_warn.call_count, 0)
+                self.assertEqual(mock_warn.call_count, 0)
-                self.assertEquals(os.environ["PATH"], all_path["PATH"])
+                self.assertEqual(os.environ["PATH"], all_path["PATH"])
        no_path = {"PATH": "/tmp/"}
        with mock.patch.dict('os.environ', no_path):
            path_surgery("thingy")
-            self.assertEquals(mock_debug.call_count, 1)
+            self.assertEqual(mock_debug.call_count, 1)
-            self.assertEquals(mock_warn.call_count, 1)
+            self.assertEqual(mock_warn.call_count, 1)
            self.assertTrue("Failed to find" in mock_warn.call_args[0][0])
            self.assertTrue("/usr/local/bin" in os.environ["PATH"])
            self.assertTrue("/tmp" in os.environ["PATH"])
-class AlreadyListeningTest(unittest.TestCase):
+
 class AlreadyListeningTestNoPsutil(unittest.TestCase):
    """Tests for certbot.plugins.already_listening when
    psutil is not available"""
    def setUp(self):
        import certbot.plugins.util
        # Ensure we get importerror
        self.psutil = None
        if "psutil" in sys.modules:
            self.psutil = sys.modules['psutil']
        sys.modules['psutil'] = None
        # Reload hackery to ensure getting non-psutil version
        # loaded to memory
        refresh(certbot.plugins.util)
    def tearDown(self):
        # Need to reload the module to ensure
        # getting back to normal
        import certbot.plugins.util
        sys.modules["psutil"] = self.psutil
        refresh(certbot.plugins.util)
    @mock.patch("certbot.plugins.util.zope.component.getUtility")
    def test_ports_available(self, mock_getutil):
        import certbot.plugins.util as plugins_util
        # Ensure we don't get error
        with mock.patch("socket._socketobject.bind"):
            self.assertFalse(plugins_util.already_listening(80))
            self.assertFalse(plugins_util.already_listening(80, True))
            self.assertEqual(mock_getutil.call_count, 0)
    @mock.patch("certbot.plugins.util.zope.component.getUtility")
    def test_ports_blocked(self, mock_getutil):
        sys.modules["psutil"] = None
        import certbot.plugins.util as plugins_util
        import socket
        with mock.patch("socket._socketobject.bind", side_effect=socket.error):
            self.assertTrue(plugins_util.already_listening(80))
            self.assertTrue(plugins_util.already_listening(80, True))
        with mock.patch("socket.socket", side_effect=socket.error):
            self.assertFalse(plugins_util.already_listening(80))
        self.assertEqual(mock_getutil.call_count, 2)
 class AlreadyListeningTestPsutil(unittest.TestCase):
    """Tests for certbot.plugins.already_listening."""
    def _call(self, *args, **kwargs):
        from certbot.plugins.util import already_listening
@@ -42,6 +94,7 @@ class AlreadyListeningTest(unittest.TestCase):
        # This tests a race condition, or permission problem, or OS
        # incompatibility in which, for some reason, no process name can be
        # found to match the identified listening PID.
        import psutil
        from psutil._common import sconn
        conns = [
            sconn(fd=-1, family=2, type=1, laddr=("0.0.0.0", 30),
@@ -94,7 +147,7 @@ class AlreadyListeningTest(unittest.TestCase):
                  raddr=(), status="LISTEN", pid=4416)]
        mock_net.return_value = conns
        mock_process.name.return_value = "inetd"
-        result = self._call(17)
+        result = self._call(17, True)
        self.assertTrue(result)
        self.assertEqual(mock_get_utility.call_count, 1)
        mock_process.assert_called_once_with(4416)
@@ -124,6 +177,7 @@ class AlreadyListeningTest(unittest.TestCase):
    @mock.patch("certbot.plugins.util.psutil.net_connections")
    def test_access_denied_exception(self, mock_net):
        import psutil
        mock_net.side_effect = psutil.AccessDenied("")
        self.assertFalse(self._call(12345))
@@ -552,7 +552,7 @@ class Reverter(object):
        others.sort()
        if others[-1] != timestamp:
            timetravel = str(float(others[-1]) + 1)
-            logger.warn("Current timestamp %s does not correspond to newest reverter "
+            logger.warning("Current timestamp %s does not correspond to newest reverter "
                "checkpoint; your clock probably jumped. Time travelling to %s",
                timestamp, timetravel)
            timestamp = timetravel
@@ -82,7 +82,7 @@ class RegisterTest(unittest.TestCase):
                self.config.register_unsafely_without_email = True
                self.config.dry_run = False
                self._call()
-                mock_logger.warn.assert_called_once_with(mock.ANY)
+                mock_logger.warning.assert_called_once_with(mock.ANY)
    def test_unsupported_error(self):
        from acme import messages
@@ -362,8 +362,8 @@ class TestFullCheckpointsReverter(unittest.TestCase):
        self.assertEqual(mock_logger.warning.call_count, 1)
        # Test Generic warning
        mock_logger.warning.call_count = 0
        self._setup_three_checkpoints()
        mock_logger.warning.call_count = 0
        self.reverter.rollback_checkpoints(4)
        self.assertEqual(mock_logger.warning.call_count, 1)
@@ -1,7 +1,7 @@
 NAME="SystemdOS"
 VERSION="42.42.42 LTS, Unreal"
 ID=systemdos
-ID_LIKE=debian
+ID_LIKE="something nonexistent debian"
 VERSION_ID="42"
 HOME_URL="http://www.example.com/"
 SUPPORT_URL="http://help.example.com/"
@@ -359,6 +359,15 @@ class OsInfoTest(unittest.TestCase):
        with mock.patch('os.path.isfile', return_value=False):
            self.assertEqual(get_systemd_os_info(), ("", ""))
    def test_systemd_os_release_like(self):
        from certbot.util import get_systemd_os_like
        with mock.patch('os.path.isfile', return_value=True):
            id_likes = get_systemd_os_like(test_util.vector_path(
                "os-release"))
            self.assertEqual(len(id_likes), 3)
            self.assertTrue("debian" in id_likes)
    @mock.patch("certbot.util.subprocess.Popen")
    def test_non_systemd_os_info(self, popen_mock):
        from certbot.util import (get_os_info, get_python_os_info,
@@ -268,6 +268,19 @@ def get_systemd_os_info(filepath="/etc/os-release"):
    return (os_name, os_version)
 def get_systemd_os_like(filepath="/etc/os-release"):
    """
    Get a list of strings that indicate the distribution likeness to
    other distributions.
    :param str filepath: File path of os-release file
    :returns: List of distribution acronyms
    :rtype: `list` of `str`
    """
    return _get_systemd_os_release_var("ID_LIKE", filepath).split(" ")
 def _get_systemd_os_release_var(varname, filepath="/etc/os-release"):
    """
    Get single value from systemd /etc/os-release
@@ -349,7 +362,7 @@ def safe_email(email):
    if EMAIL_REGEX.match(email) is not None:
        return not email.startswith(".") and ".." not in email
    else:
-        logger.warn("Invalid email address: %s.", email)
+        logger.warning("Invalid email address: %s.", email)
        return False
@@ -409,6 +422,9 @@ def enforce_domain_sanity(domain):
        else:
            raise errors.ConfigurationError(str(error_fmt).format(domain))
    if six.PY3:
        domain = domain.decode('ascii')
    # Remove trailing dot
    domain = domain[:-1] if domain.endswith('.') else domain
@@ -151,9 +151,7 @@ Resources for recommendations
 In the course of considering how to handle this issue, we received
 recommendations with sources of expert guidance on ciphersuites and other
 cryptographic parameters. We're grateful to everyone who contributed
-suggestions. The recommendations we received are available at
+suggestions. The recommendations we received are available under Feedback_.
 https://github.com/certbot/certbot/wiki/Ciphersuite-guidance
 Certbot users are welcome to review these authorities to
 better inform their own cryptographic parameter choices. We also
@@ -205,3 +203,137 @@ so far is redirecting HTTP requests to HTTPS in web servers, the
 "redirect" enhancement). The changes here would probably be either a new
 "ciphersuite" enhancement in each plugin that provides an installer,
 or a family of enhancements, one per selectable ciphersuite configuration.
 Feedback
 ========
 We recieve lots of feedback on the type of ciphersuites that Let's Encrypt supports and list some coallated feedback below. This section aims to track suggestions and references that people have offered or identified to improve the ciphersuites that Let's Encrypt enables when configuring TLS on servers.
 Because of the Chatham House Rule applicable to some of the discussions, people are *not* individually credited for their suggestions, but most suggestions here were made or found by other people, and I thank them for their contributions.
 Some people provided rationale information mostly having to do with compatibility of particular user-agents (especially UAs that don't support ECC, or that don't support DH groups > 1024 bits).  Some ciphersuite configurations have been chosen to try to increase compatibility with older UAs while allowing newer UAs to negotiate stronger crypto.  For example, some configurations forego forward secrecy entirely for connections from old UAs, like by offering ECDHE and RSA key exchange, but no DHE at all.  (There are UAs that can fail the negotiation completely if a DHE ciphersuite with prime > 1024 bits is offered.)
 References
 ----------
 RFC 7575
 ~~~~~~~~
 IETF has published a BCP document, RFC 7525, "Recommendations for Secure Use of Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS)"
 https://datatracker.ietf.org/doc/rfc7525/
 BetterCrypto.org
 ~~~~~~~~~~~~~~~~
 BetterCrypto.org, a collaboration of mostly European IT security experts, has published a draft paper, "Applied Crypto Hardening"
 https://bettercrypto.org/static/applied-crypto-hardening.pdf
 FF-DHE Internet-Draft
 ~~~~~~~~~~~~~~~~~~~~~
 Gillmor's Internet-Draft "Negotiated Discrete Log Diffie-Hellman Ephemeral Parameters for TLS" is being developed at the IETF TLS WG.  It advocates using *standardized* DH groups in all cases, not individually-chosen ones (mostly because of the Triple Handshake attack which can involve maliciously choosing invalid DH groups).  The draft provides a list of recommended groups, with primes beginning at 2048 bits and going up from there.  It also has a new protocol mechanism for agreeing to use these groups, with the possibility of backwards compatibility (and use of weaker DH groups) for older clients and servers that don't know about this mechanism.
 https://tools.ietf.org/html/draft-ietf-tls-negotiated-ff-dhe-10
 Mozilla
 ~~~~~~~
 Mozilla's general server configuration guidance is available at https://wiki.mozilla.org/Security/Server_Side_TLS
 Mozilla has also produced a configuration generator: https://mozilla.github.io/server-side-tls/ssl-config-generator/
 Dutch National Cyber Security Centre
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The Dutch National Cyber Security Centre has published guidance on "ICT-beveiligingsrichtlijnen voor Transport Layer Security (TLS)" ("IT Security Guidelines for Transport Layer Security (TLS)").  These are available only in Dutch at
 https://www.ncsc.nl/dienstverlening/expertise-advies/kennisdeling/whitepapers/ict-beveiligingsrichtlijnen-voor-transport-layer-security-tls.html
 I have access to an English-language summary of the recommendations.
 Keylength.com
 ~~~~~~~~~~~~~
 Damien Giry collects recommendations by academic researchers and standards organizations about keylengths for particular cryptoperiods, years, or security levels.  The keylength recommendations of the various sources are summarized in a chart.  This site has been updated over time and includes expert guidance from eight sources published between 2000 and 2015.
 http://www.keylength.com/
 NIST
 ~~~~
 NISA published its "NIST Special Publication 800-52 Revision 1: Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations"
 http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-52r1.pdf
 and its "NIST Special Publication 800-57: Recommendation for Key Management – Part 1: General (Revision 3)"
 http://csrc.nist.gov/publications/nistpubs/800-57/sp800-57_part1_rev3_general.pdf
 ENISA
 ~~~~~
 ENISA published its "Algorithms, Key Sizes and Parameters Report - 2013"
 https://www.enisa.europa.eu/activities/identity-and-trust/library/deliverables/algorithms-key-sizes-and-parameters-report
 WeakDH/Logjam
 -------------
 The WeakDH/Logjam research has thrown into question the safety of some existing practice using DH ciphersuites, especially the use of standardized groups with a prime ≤ 1024 bits.  The authors provided detailed guidance, including ciphersuite lists, at
 https://weakdh.org/sysadmin.html
 These lists may have been derived from Mozilla's recommendations.
 One of the authors clarified his view of the priorities for various changes as a result of the research at
 https://www.ietf.org/mail-archive/web/tls/current/msg16496.html
 In particular, he supports ECDHE and also supports the use of the standardized groups in the FF-DHE Internet-Draft mentioned above (which isn't clear from the group's original recommendations).
 Particular sites' opinions or configurations
 --------------------------------------------
 Amazon ELB
 ~~~~~~~~~~
 Amazon ELB explains its current ciphersuite choices at
 https://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-security-policy-table.html
 U.S. Government 18F
 ~~~~~~~~~~~~~~~~~~~
 The 18F site (https://18f.gsa.gov/) is using 
 ::
    ssl_ciphers 'kEECDH+ECDSA+AES128 kEECDH+ECDSA+AES256 kEECDH+AES128 kEECDH+AES256 kEDH+AES128 kEDH+AES256 DES-CBC3-SHA +SHA !aNULL !eNULL !LOW !MD5 !EXP !DSS !PSK !SRP !kECDH !CAMELLIA !RC4 !SEED';
 Duraconf
 ~~~~~~~~
 The Duraconf project collects particular configuration files, with an apparent focus on avoiding the use of obsolete symmetric ciphers and hash functions, and favoring forward secrecy while not requiring it.
 https://github.com/ioerror/duraconf
 Site scanning or rating tools
 -----------------------------
 Qualys SSL Labs
 ~~~~~~~~~~~~~~~
 Qualys offers the best-known TLS security scanner, maintained by Ivan Ristić.
 https://www.ssllabs.com/
 Dutch NCSC
 ~~~~~~~~~~
 The Dutch NCSC, mentioned above, has also made available its own site security scanner which indicates how well sites comply with the recommendations.
 https://en.internet.nl/
 Java compatibility issue
 ------------------------
 A lot of backward-compatibility concerns have to do with Java hard-coding DHE primes to a 1024-bit limit, accepting DHE ciphersuites in negotiation, and then aborting the connection entirely if a prime > 1024 bits is presented.  The simple summary is that servers offering a Java-compatible DHE ciphersuite in preference to other Java-compatible ciphersuites, and then presenting a DH group with a prime > 1024 bits, will be completely incompatible with clients running some versions of Java.  (This may also be the case with very old MSIE versions...?)  There are various strategies for dealing with this, and maybe we can document the options here.
@@ -6,9 +6,9 @@ Developer Guide
   :local:
-.. _hacking:
+.. _getting_started:
-Hacking
+Getting Started
 =======
 Running a local copy of the client
@@ -323,11 +323,7 @@ Steps:
   should run the integration tests, see `integration`_. See `Known Issues`_
   for some common failures that have nothing to do with your code.
 7. Submit the PR.
-8. Did your tests pass on Travis? If they didn't, it might not be your fault!
+8. Did your tests pass on Travis? If they didn't, fix any errors.
   See `Known Issues`_. If it's not a known issue, fix any errors.
 .. _Known Issues:
  https://github.com/certbot/certbot/wiki/Known-issues
 Updating the documentation
 ==========================
@@ -5,9 +5,11 @@ Welcome to the Certbot documentation!
   :maxdepth: 2
   intro
   install
   using
   contributing
   packaging
   resources
 .. toctree::
   :maxdepth: 1
@@ -0,0 +1,33 @@
 =====================
 Quick Installation
 =====================
 If ``certbot`` (or ``letsencrypt``) is packaged for your Unix OS (visit
 certbot.eff.org_ to find out), you can install it
 from there, and run it by typing ``certbot`` (or ``letsencrypt``).  Because
 not all operating systems have packages yet, we provide a temporary solution
 via the ``certbot-auto`` wrapper script, which obtains some dependencies from
 your OS and puts others in a python virtual environment::
  user@webserver:~$ wget https://dl.eff.org/certbot-auto
  user@webserver:~$ chmod a+x ./certbot-auto
  user@webserver:~$ ./certbot-auto --help
 .. hint:: The certbot-auto download is protected by HTTPS, which is pretty good, but if you'd like to
          double check the integrity of the ``certbot-auto`` script, you can use these steps for verification before running it::
            user@server:~$ wget -N https://dl.eff.org/certbot-auto.asc
            user@server:~$ gpg2 --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2
            user@server:~$ gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc certbot-auto
 And for full command line help, you can type::
  ./certbot-auto --help all
 ``certbot-auto`` updates to the latest client release automatically.  And
 since ``certbot-auto`` is a wrapper to ``certbot``, it accepts exactly
 the same command line flags and arguments.  More details about this script and
 other installation methods can be found `in the User Guide
 <https://certbot.eff.org/docs/using.html#installation>`_.
 .. _certbot.eff.org: https://certbot.eff.org/
@@ -1,6 +1,7 @@
 =====================
-README / Introduction
+Introduction
 =====================
 .. include:: ../README.rst
-.. include:: ../CHANGES.rst
+    :start-after: tag:intro-begin
    :end-before: tag:intro-end
@@ -2,5 +2,83 @@
 Packaging Guide
 ===============
-Documentation can be found at
+Releases
-https://github.com/certbot/certbot/wiki/Packaging.
+========
 We release packages and upload them to PyPI (wheels and source tarballs).
 - https://pypi.python.org/pypi/acme
 - https://pypi.python.org/pypi/certbot
 - https://pypi.python.org/pypi/certbot-apache
 - https://pypi.python.org/pypi/certbot-nginx
 The following scripts are used in the process:
 - https://github.com/letsencrypt/letsencrypt/blob/master/tools/release.sh
 We currently version with the following scheme:
 - ``0.1.0``
 - ``0.2.0dev`` for developement in ``master``
 - ``0.2.0`` (only temporarily in ``master``)
 - ...
 Notes for package maintainers
 =============================
 0. Please use our releases, not ``master``!
 1. Do not package ``certbot-compatibility-test`` or ``letshelp-certbot`` - it's only used internally.
 2. If you'd like to include automated renewal in your package ``certbot renew -q`` should be added to crontab or systemd timer.
 3. ``jws`` is an internal script for ``acme`` module and it doesn't have to be packaged - it's mostly for debugging: you can use it as ``echo foo | jws sign | jws verify``.
 4. Do get in touch with us. We are happy to make any changes that will make packaging easier. If you need to apply some patches don't do it downstream - make a PR here.
 Already ongoing efforts
 =======================
 Arch
 ----
 From our official releases:
 - https://www.archlinux.org/packages/community/any/python2-acme
 - https://www.archlinux.org/packages/community/any/certbot
 - https://www.archlinux.org/packages/community/any/certbot-apache
 - https://www.archlinux.org/packages/community/any/certbot-nginx
 - https://www.archlinux.org/packages/community/any/letshelp-certbot
 From ``master``: https://aur.archlinux.org/packages/certbot-git
 Debian (and its derivatives, including Ubuntu)
 ------
 https://packages.debian.org/sid/certbot
 https://packages.debian.org/sid/python-certbot
 https://packages.debian.org/sid/python-certbot-apache
 Fedora
 ------
 In Fedora 23+.
 - https://admin.fedoraproject.org/pkgdb/package/letsencrypt/
 - https://admin.fedoraproject.org/pkgdb/package/certbot/
 - https://admin.fedoraproject.org/pkgdb/package/python-acme/
 FreeBSD
 -------
 https://svnweb.freebsd.org/ports/head/security/py-certbot/
 GNU Guix
 --------
 - https://www.gnu.org/software/guix/package-list.html#certbot
 OpenBSD
 -------
 - http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/security/letsencrypt/client/
@@ -0,0 +1,54 @@
 =====================
 Resources
 =====================
 Documentation: https://certbot.eff.org/docs
 Software project: https://github.com/certbot/certbot
 Notes for developers: https://certbot.eff.org/docs/contributing.html
 Main Website: https://letsencrypt.org/
 Let's Encrypt FAQ: https://community.letsencrypt.org/t/frequently-asked-questions-faq/26#topic-title
 IRC Channel: #letsencrypt on `Freenode`_ or #certbot on `OFTC`_
 Community: https://community.letsencrypt.org
 ACME spec: http://ietf-wg-acme.github.io/acme/
 ACME working area in github: https://github.com/ietf-wg-acme/acme
 Mailing list: `client-dev`_ (to subscribe without a Google account, send an
 email to client-dev+subscribe@letsencrypt.org)
 |build-status| |coverage| |docs| |container|
 .. |build-status| image:: https://travis-ci.org/certbot/certbot.svg?branch=master
   :target: https://travis-ci.org/certbot/certbot
   :alt: Travis CI status
 .. |coverage| image:: https://coveralls.io/repos/certbot/certbot/badge.svg?branch=master
   :target: https://coveralls.io/r/certbot/certbot
   :alt: Coverage status
 .. |docs| image:: https://readthedocs.org/projects/letsencrypt/badge/
   :target: https://readthedocs.org/projects/letsencrypt/
   :alt: Documentation status
 .. |container| image:: https://quay.io/repository/letsencrypt/letsencrypt/status
   :target: https://quay.io/repository/letsencrypt/letsencrypt
   :alt: Docker Repository on Quay.io
 .. _`installation instructions`:
   https://letsencrypt.readthedocs.org/en/latest/using.html
 .. _watch demo video: https://www.youtube.com/watch?v=Gas_sSB-5SU
 .. _Freenode: https://webchat.freenode.net?channels=%23letsencrypt
 .. _OFTC: https://webchat.oftc.net?channels=%23certbot
 .. _client-dev: https://groups.google.com/a/letsencrypt.org/forum/#!forum/client-dev
@@ -5,6 +5,27 @@ User Guide
 .. contents:: Table of Contents
   :local:
 System Requirements
 ===================
 The Let's Encrypt Client presently only runs on Unix-ish OSes that include
 Python 2.6 or 2.7; Python 3.x support will hopefully be added in the future. The
 client requires root access in order to write to ``/etc/letsencrypt``,
 ``/var/log/letsencrypt``, ``/var/lib/letsencrypt``; to bind to ports 80 and 443
 (if you use the ``standalone`` plugin) and to read and modify webserver
 configurations (if you use the ``apache`` or ``nginx`` plugins).  If none of
 these apply to you, it is theoretically possible to run without root privileges,
 but for most users who want to avoid running an ACME client as root, either
 `letsencrypt-nosudo <https://github.com/diafygi/letsencrypt-nosudo>`_ or
 `simp_le <https://github.com/kuba/simp_le>`_ are more appropriate choices.
 The Apache plugin currently requires OS with augeas version 1.0; currently `it
 supports
 <https://github.com/certbot/certbot/blob/master/certbot-apache/certbot_apache/constants.py>`_
 modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin.
 Getting Certbot
 ===============
@@ -13,7 +34,7 @@ visit certbot.eff.org_. This is the easiest way to learn how to get
 Certbot up and running on your system. 
 If you're offline, you can find some general
-instructions `in the README / Introduction <intro.html#installation>`__
+instructions under `Quick Installation <install.html>`__.
 __ installation_
 .. _certbot.eff.org: https://certbot.eff.org
@@ -406,7 +427,7 @@ good reason to do so.
 You should definitely read the :ref:`where-certs` section, in order to
 know how to manage the certs
-manually. https://github.com/certbot/certbot/wiki/Ciphersuite-guidance
+manually. `Our ciphersuites page <ciphers.html>`__
 provides some information about recommended ciphersuites. If none of
 these make much sense to you, you should definitely use the
 certbot-auto_ method, which enables you to use installer plugins
@@ -564,3 +585,8 @@ Beyond the methods discussed here, other methods may be possible, such as
 installing Certbot directly with pip from PyPI or downloading a ZIP
 archive from GitHub may be technically possible but are not presently
 recommended or supported.
 .. include:: ../README.rst
    :start-after: tag:features-begin
    :end-before: tag:features-end
 .. include:: ../CHANGES.rst
@@ -126,7 +126,7 @@ ExperimentalBootstrap() {
      $2
    fi
  else
-    echo "WARNING: $1 support is very experimental at present..."
+    echo "FATAL: $1 support is very experimental at present..."
    echo "if you would like to work on improving it, please ensure you have backups"
    echo "and then run this script again with the --debug flag!"
    exit 1
@@ -126,7 +126,7 @@ ExperimentalBootstrap() {
      $2
    fi
  else
-    echo "WARNING: $1 support is very experimental at present..."
+    echo "FATAL: $1 support is very experimental at present..."
    echo "if you would like to work on improving it, please ensure you have backups"
    echo "and then run this script again with the --debug flag!"
    exit 1
@@ -40,10 +40,8 @@ install_requires = [
    'configobj',
    'cryptography>=0.7',  # load_pem_x509_certificate
    'parsedatetime>=1.3',  # Calendar.parseDT
    'psutil>=2.2.1',  # 2.1.0 for net_connections and 2.2.1 resolves #1080
    'PyOpenSSL',
    'pyrfc3339',
    'python2-pythondialog>=3.2.2rc1',  # Debian squeeze support, cf. #280
    'pytz',
    # For pkg_resources. >=1.0 so pip resolves it to a version cryptography
    # will tolerate; see #2599:
@@ -53,6 +51,12 @@ install_requires = [
    'zope.interface',
 ]
 # Debian squeeze support, cf. #280
 if sys.version_info[0] == 2:
    install_requires.append('python2-pythondialog>=3.2.2rc1')
 else:
    install_requires.append('pythondialog>=3.2.2rc1')
 # env markers in extras_require cause problems with older pip: #517
 # Keep in sync with conditional_requirements.py.
 if sys.version_info < (2, 7):
@@ -70,6 +74,7 @@ dev_extras = [
    'coverage',
    'nose',
    'pep8',
    'psutil>=2.2.1',  # for tests, optional
    'pylint==1.4.2',  # upstream #248
    'tox',
    'twine',
@@ -5,6 +5,6 @@ set -xe
 # Check out special branch until latest docker changes land in Boulder master.
 git clone -b docker-integration https://github.com/letsencrypt/boulder $BOULDERPATH
 cd $BOULDERPATH
-sed -i 's/FAKE_DNS: .*/FAKE_DNS: 172.17.42.1/' docker-compose.yml
+FAKE_DNS=$(ifconfig docker0 | grep "inet addr:" | cut -d: -f2 | awk '{ print $1}')
 sed -i "s/FAKE_DNS: .*/FAKE_DNS: $FAKE_DNS/" docker-compose.yml
 docker-compose up -d
@@ -5,5 +5,6 @@
 # Check out special branch until latest docker changes land in Boulder master.
 git clone -b docker-integration https://github.com/letsencrypt/boulder $BOULDERPATH
 cd $BOULDERPATH
-sed -i 's/FAKE_DNS: .*/FAKE_DNS: 172.17.42.1/' docker-compose.yml
+FAKE_DNS=$(ifconfig docker0 | grep "inet addr:" | cut -d: -f2 | awk '{ print $1}')
 sed -i "s/FAKE_DNS: .*/FAKE_DNS: $FAKE_DNS/" docker-compose.yml
 docker-compose up -d
@@ -13,7 +13,7 @@ envlist = py{26,33,34,35},cover,lint
 # packages installed separately to ensure that downstream deps problems
 # are detected, c.f. #1002
 commands =
-    pip install -e acme[dev]
+    pip install -e acme[dns,dev]
    nosetests -v acme
    pip install -e .[dev]
    nosetests -v certbot
@@ -35,26 +35,27 @@ deps =
    py{26,27}-oldest: psutil==2.1.0
    py{26,27}-oldest: PyOpenSSL==0.13
    py{26,27}-oldest: python2-pythondialog==3.2.2rc1
    py{26,27}-oldest: dnspython>=1.12
 [testenv:py33]
 commands =
-    pip install -e acme[dev]
+    pip install -e acme[dns,dev]
    nosetests -v acme
 [testenv:py34]
 commands =
-    pip install -e acme[dev]
+    pip install -e acme[dns,dev]
    nosetests -v acme
 [testenv:py35]
 commands =
-    pip install -e acme[dev]
+    pip install -e acme[dns,dev]
    nosetests -v acme
 [testenv:cover]
 basepython = python2.7
 commands =
-    pip install -e acme[dev] -e .[dev] -e certbot-apache -e certbot-nginx -e letshelp-certbot
+    pip install -e acme[dns,dev] -e .[dev] -e certbot-apache -e certbot-nginx -e letshelp-certbot
    ./tox.cover.sh
 [testenv:lint]
@@ -64,7 +65,7 @@ basepython = python2.7
 # duplicate code checking; if one of the commands fails, others will
 # continue, but tox return code will reflect previous error
 commands =
-    pip install -q -e acme[dev] -e .[dev] -e certbot-apache -e certbot-nginx -e certbot-compatibility-test -e letshelp-certbot
+    pip install -q -e acme[dns,dev] -e .[dev] -e certbot-apache -e certbot-nginx -e certbot-compatibility-test -e letshelp-certbot
    ./pep8.travis.sh
    pylint --reports=n --rcfile=.pylintrc certbot
    pylint --reports=n --rcfile=acme/.pylintrc acme/acme
@@ -79,6 +80,10 @@ commands =
    pip install -e acme -e .[dev] -e certbot-apache -e certbot-nginx -e certbot-compatibility-test -e letshelp-certbot
    {toxinidir}/certbot-apache/certbot_apache/tests/apache-conf-files/apache-conf-test --debian-modules
 [testenv:nginxroundtrip]
 commands =
    pip install -e acme[dev] -e .[dev] -e certbot-nginx
    python certbot-compatibility-test/nginx/roundtrip.py certbot-compatibility-test/nginx/nginx-roundtrip-testdata
 [testenv:le_auto]
 # At the moment, this tests under Python 2.7 only, as only that version is
@@ -89,3 +94,21 @@ commands =
 whitelist_externals =
    docker
 passenv = DOCKER_*
 [testenv:apache_compat]
 commands =
    docker build -t certbot-compatibility-test -f certbot-compatibility-test/Dockerfile .
    docker build -t apache-compat -f certbot-compatibility-test/Dockerfile-apache .
    docker run --rm -it apache-compat -c apache.tar.gz -vvvv
 whitelist_externals =
    docker
 passenv = DOCKER_*
 [testenv:nginx_compat]
 commands =
    docker build -t certbot-compatibility-test -f certbot-compatibility-test/Dockerfile .
    docker build -t nginx-compat -f certbot-compatibility-test/Dockerfile-nginx .
    docker run --rm -it nginx-compat -c nginx.tar.gz -vvvv
 whitelist_externals =
    docker
 passenv = DOCKER_*
		`@@ -0,0 +1 @@`
							`"""Certbot compatibility test Nginx configurators"""`