Refactor certbot snap wrapper (#8313)

Partial fix for #8280

This PR refactors the bash script wrapper for snap (`/certbot.wrapper`) into certbot python codebase. Here are the keypoints of this refactoring:
* the wrapping is applied when `main` function from `certbot._internal.main` is called if environment variable `CERTBOT_SNAPPED` is `True`, which is set during the snap build
* the initial bash script wrapper  is removed, simplifying `snap/snapcraft.yaml` by removing the `certbot.wrapper` part
* the dependency to `curl` and `jq` binaries are removed
* the failure during requesting the snapd socket is correctly handled, and displays an informative message in order to correct the situation, as required by #8280

One side note about the modifications done to `app.certbot.command` in `snapcraft.yaml`. Normally calling `bin/certbot` should be sufficient and it is effectively under a normal situation (`core` snap up-to-date). However in the same situation than when the problem occurs in #8280, using `bin/certbot` makes the snap raise an exception about `certbot.main` module that cannot be found.

It seems that when `core` snap is not up-to-date (in Debian for instance with default `snapd` installation), the shebang `/usr/bin/env python3` in the `bin/certbot` wrapper is wrongly resolved to the host Python, instead of the snap Python. It is working as expected if `core` snap is up-to-date. One way to fix that is to keep a bash script wrapper, because in this case, it is the `PATH` value that matters to resolve the Python interpreter, and `PATH` is correctly set up to resolve it from the snap first.

However to keep the simplification provided by the wrapper removal, I prefered to use `bin/python3 $SNAP/bin/certbot` as `command` to explicitly target the correct Python interpreter. Again normally it is not needed because everything is working correctly with a `core` snap up-to-date, but since the root purpose of all of this is to target bad situations, well, it is better to have a snap that is effectively able to start to display the informative message...

* Refactor the bash wrapper for snap execution as Python code into certbot

* Remove wrapper, finalize the python logic

* Organize code

* Improve error handling

* Update command

* Setup basic certbot logging before running the snap prepare logic

* Improve instructions

* Use logging facility

* Handle properly an exception in snap_config

* Use the python script call approach

* Update instructions to keep sync with https://github.com/certbot/website/pull/650

certbot.wrapper

@@ -1,38 +0,0 @@
-#!/bin/sh
-#
-# TODO: We may want to consider rewriting this script in Python. See
-# https://github.com/certbot/certbot/issues/8251 for more info.
-set -e
-
-# This code is based on snapcraft's own patch to work around this problem at
-# https://github.com/snapcore/snapcraft/blob/a97fb5c7ea553a1bd20f4887a7c3393e75761890/patches/ctypes_init.diff.
-# We may not build the Certbot snap for all of these architectures (and as of
-# writing this we do not), but we keep the code for them to avoid having to
-# solve this problem again in the future if we add support for new
-# architectures.
-case "${SNAP_ARCH}" in
-    'arm64')
-        ARCH_TRIPLET='aarch64-linux-gnu';;
-    'armhf')
-        ARCH_TRIPLET='arm-linux-gnueabihf';;
-    'i386')
-        ARCH_TRIPLET='i386-linux-gnu';;
-    'ppc64el')
-        ARCH_TRIPLET='powerpc64le-linux-gnu';;
-    'powerpc')
-        ARCH_TRIPLET='powerpc-linux-gnu';;
-    'amd64')
-        ARCH_TRIPLET='x86_64-linux-gnu';;
-    's390x')
-        ARCH_TRIPLET='s390x-linux-gnu';;
-    *)
-        echo "Unrecongized value of SNAP_ARCH: ${SNAP_ARCH}" >&2
-        exit 1
-esac
-
-export CERTBOT_AUGEAS_PATH="${SNAP}/usr/lib/${ARCH_TRIPLET}/libaugeas.so.0"
-
-CERTBOT_PLUGIN_PATH="$(curl -s --unix-socket /run/snapd.socket "http://localhost/v2/connections?snap=certbot&interface=content" | jq -r  '.result.established | map(select(.plug.plug == "plugin" and ."plug-attrs".content == "certbot-1") | "/snap/"+.slot.snap+"/current/lib/python3.8/site-packages/" ) | join(":")')"
-export CERTBOT_PLUGIN_PATH
-
-exec certbot "$@" --preconfigured-renewal

certbot/certbot/_internal/main.py

@@ -28,6 +28,7 @@ from certbot._internal import hooks
 from certbot._internal import log
 from certbot._internal import renewal
 from certbot._internal import reporter
+from certbot._internal import snap_config
 from certbot._internal import storage
 from certbot._internal import updater
 from certbot._internal.plugins import disco as plugins_disco
@@ -1325,6 +1326,9 @@ def main(cli_args=None):
 
     log.pre_arg_parse_setup()
 
+    if os.environ.get('CERTBOT_SNAPPED') == 'True':
+        cli_args = snap_config.prepare_env(cli_args)
+
     plugins = plugins_disco.PluginsRegistry.find_all()
     logger.debug("certbot version: %s", certbot.__version__)
     # do not log `config`, as it contains sensitive data (e.g. revoke --key)!

certbot/certbot/_internal/snap_config.py

@@ -0,0 +1,102 @@
+"""Module configuring Certbot in a snap environment"""
+import logging
+import socket
+
+from requests import Session
+from requests.adapters import HTTPAdapter
+from requests.exceptions import HTTPError
+from requests.exceptions import RequestException
+
+from acme.magic_typing import List
+from certbot.compat import os
+from certbot.errors import Error
+
+try:
+    from urllib3.connection import HTTPConnection
+    from urllib3.connectionpool import HTTPConnectionPool
+except ImportError:
+    # Stub imports for oldest requirements, that will never be used in snaps.
+    HTTPConnection = object
+    HTTPConnectionPool = object
+
+
+_ARCH_TRIPLET_MAP = {
+    'arm64': 'aarch64-linux-gnu',
+    'armhf': 'arm-linux-gnueabihf',
+    'i386': 'i386-linux-gnu',
+    'ppc64el': 'powerpc64le-linux-gnu',
+    'powerpc': 'powerpc-linux-gnu',
+    'amd64': 'x86_64-linux-gnu',
+    's390x': 's390x-linux-gnu',
+}
+
+LOGGER = logging.getLogger(__name__)
+
+
+def prepare_env(cli_args):
+    # type: (List[str]) -> List[str]
+    """
+    Prepare runtime environment for a certbot execution in snap.
+    :param list cli_args: List of command line arguments
+    :return: Update list of command line arguments
+    :rtype: list
+    """
+    snap_arch = os.environ.get('SNAP_ARCH')
+
+    if snap_arch not in _ARCH_TRIPLET_MAP:
+        raise Error('Unrecognized value of SNAP_ARCH: {0}'.format(snap_arch))
+
+    os.environ['CERTBOT_AUGEAS_PATH'] = '{0}/usr/lib/{1}/libaugeas.so.0'.format(
+        os.environ.get('SNAP'), _ARCH_TRIPLET_MAP[snap_arch])
+
+    session = Session()
+    session.mount('http://snapd/', _SnapdAdapter())
+
+    try:
+        response = session.get('http://snapd/v2/connections?snap=certbot&interface=content')
+        response.raise_for_status()
+    except RequestException as e:
+        if isinstance(e, HTTPError) and e.response.status_code == 404:
+            LOGGER.error('An error occurred while fetching Certbot snap plugins: '
+                         'your version of snapd is outdated.')
+            LOGGER.error('Please run "sudo snap install core; sudo snap refresh core" '
+                         'in your terminal and try again.')
+        else:
+            LOGGER.error('An error occurred while fetching Certbot snap plugins: '
+                         'make sure the snapd service is running.')
+        raise e
+
+    data = response.json()
+    connections = ['/snap/{0}/current/lib/python3.8/site-packages/'.format(item['slot']['snap'])
+                   for item in data.get('result', {}).get('established', [])
+                   if item.get('plug', {}).get('plug') == 'plugin'
+                   and item.get('plug-attrs', {}).get('content') == 'certbot-1']
+
+    os.environ['CERTBOT_PLUGIN_PATH'] = ':'.join(connections)
+
+    cli_args.append('--preconfigured-renewal')
+
+    return cli_args
+
+
+class _SnapdConnection(HTTPConnection):
+    def __init__(self):
+        super(_SnapdConnection, self).__init__("localhost")
+        self.sock = None
+
+    def connect(self):
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        self.sock.connect("/run/snapd.socket")
+
+
+class _SnapdConnectionPool(HTTPConnectionPool):
+    def __init__(self):
+        super(_SnapdConnectionPool, self).__init__("localhost")
+
+    def _new_conn(self):
+        return _SnapdConnection()
+
+
+class _SnapdAdapter(HTTPAdapter):
+    def get_connection(self, url, proxies=None):
+        return _SnapdConnectionPool()

snap/snapcraft.yaml

@@ -20,13 +20,13 @@ adopt-info: certbot
 
 apps:
   certbot:
-    command: certbot.wrapper
+    command: bin/python3 $SNAP/bin/certbot
     environment:
       PATH: "$SNAP/bin:$SNAP/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
       AUGEAS_LENS_LIB: "$SNAP/usr/share/augeas/lenses/dist"
       CERTBOT_SNAPPED: "True"
   renew:
-    command: certbot.wrapper -q renew
+    command: bin/python3 $SNAP/bin/certbot -q renew
     daemon: oneshot
     environment:
       PATH: "$SNAP/bin:$SNAP/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
@@ -71,9 +71,6 @@ parts:
       - python3-distutils
       - python3-pkg-resources
       - python3.8-minimal
-      # added for certbot.wrapper script:
-      - curl
-      - jq
     # To build cryptography and cffi if needed
     build-packages: [gcc, libffi-dev, libssl-dev, git, libaugeas-dev, python3-dev]
     build-environment:
@@ -84,10 +81,6 @@ parts:
         cd $SNAPCRAFT_PART_SRC
         python3 tools/strip_hashes.py letsencrypt-auto-source/pieces/dependency-requirements.txt | grep -v python-augeas > snap-constraints.txt
         snapcraftctl set-version `git describe|sed s/^v//`
-  wrappers:
-    plugin: dump
-    source: .
-    stage: [certbot.wrapper]
   shared-metadata:
     plugin: dump
     source: .