Migrate certbot-auto users on CentOS 6 to Python 3.6 (#7268)

Fixes #7007

Python 3.4 is [EOL](https://www.python.org/dev/peps/pep-0429/), and only Python 3.x version available for CentOS 6 through EPEL is this version, and so is used by `certbot-auto`, the only official way to install Certbot on this platform.

This unpleasant situation becomes a little more uncomfortable, considering that the newest `pip` version (19.2) [just dropped Python 3.4 support](https://github.com/pypa/pip/issues/6685) and will refuse to start on this Python version. We can expect a lot of dependencies to follow this path now.

One direct result of this situation is that a fix to support correctly the ARM platforms requires to upgrade `pip` to 19.2 for `certbot-auto`. So this is not possible right now.

Then, let's upgrade Certbot instances on CentOS 6 to a supported version of Python 3.

This PR proposes a new bootstrap approach for CentOS 6 platform, `BootstrapRpmPython3Legacy`, that will install Python 3.6 from [SCL](https://www.softwarecollections.org) (the latest one available for now on CentOS 6). In term of Python 3 specific bootstrap methods, I take the occasion here to completely separate the bootstrap of CentOS 6 as a legacy system, from the RPM-based newest systems (like Fedora 29+) that are simply dropping support for Python 2.x. This is in prevision of future migration for all systems on Python 3.x, that is a different problematic than supporting old systems.

* Add logic

* Rebuilt letsencrypt-auto

* Fix logic

* Focus on specific packages

* Maintain PATH for further invocations of letsencrypt-auto after bootstrap.

* Various corrections

* Fix farm test for RHEL6

* Working centos6 letsencrypt-auto self tests

* Fix test_sdist for CentOS 6

* Corrections

* Work in progress

* Working configuration

* Fix typo

* Remove EPEL. Add a test.

* Update letsencrypt-auto-source/letsencrypt-auto.template

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Improvements after review

* Improvements

* Add a comment

* Add a test

* Update a test

* Corrections

* Update function return

* Work in progress

* Correct behavior on oracle linux 6.

* Corrections

* Rebuild script

* Add letsencrypt-auto tests for oraclelinux6

* Update tox.ini

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto.template

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto.template

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Remove specific code for scientific linux

* Change some variables names

* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Various corrections

* Fix tests

* Add a comment

* Update message

* Fix test message

* Update letsencrypt-auto-source/letsencrypt-auto.template

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update scripts

* More focused assertion

* Add back a test

* Update script

* Update letsencrypt-auto-source/letsencrypt-auto.template

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Update letsencrypt-auto-source/letsencrypt-auto.template

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>

* Check quiet mode

* Add changelog

* Update letsencrypt-auto-source/tests/oraclelinux6_tests.sh

Co-Authored-By: Brad Warren <bmw@users.noreply.github.com>
This commit is contained in:
Adrien Ferrand
2019-10-30 10:39:45 -07:00
committed by Brad Warren
co-authored by Brad Warren
parent 2dbe47f3a7
commit e19b2e04c7
13 changed files with 540 additions and 168 deletions
@@ -256,20 +256,28 @@ DeprecationBootstrap() {
fi
}
MIN_PYTHON_VERSION="2.7"
MIN_PYVER=$(echo "$MIN_PYTHON_VERSION" | sed 's/\.//')
MIN_PYTHON_2_VERSION="2.7"
MIN_PYVER2=$(echo "$MIN_PYTHON_2_VERSION" | sed 's/\.//')
MIN_PYTHON_3_VERSION="3.5"
MIN_PYVER3=$(echo "$MIN_PYTHON_3_VERSION" | sed 's/\.//')
# Sets LE_PYTHON to Python version string and PYVER to the first two
# digits of the python version
# digits of the python version.
# MIN_PYVER and MIN_PYTHON_VERSION are also set by this function, and their
# values depend on if we try to use Python 3 or Python 2.
DeterminePythonVersion() {
# Arguments: "NOCRASH" if we shouldn't crash if we don't find a good python
#
# If no Python is found, PYVER is set to 0.
if [ "$USE_PYTHON_3" = 1 ]; then
MIN_PYVER=$MIN_PYVER3
MIN_PYTHON_VERSION=$MIN_PYTHON_3_VERSION
for LE_PYTHON in "$LE_PYTHON" python3; do
# Break (while keeping the LE_PYTHON value) if found.
$EXISTS "$LE_PYTHON" > /dev/null && break
done
else
MIN_PYVER=$MIN_PYVER2
MIN_PYTHON_VERSION=$MIN_PYTHON_2_VERSION
for LE_PYTHON in "$LE_PYTHON" python2.7 python27 python2 python; do
# Break (while keeping the LE_PYTHON value) if found.
$EXISTS "$LE_PYTHON" > /dev/null && break
@@ -285,7 +293,7 @@ DeterminePythonVersion() {
fi
fi
PYVER=`"$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//'`
PYVER=$("$LE_PYTHON" -V 2>&1 | cut -d" " -f 2 | cut -d. -f1,2 | sed 's/\.//')
if [ "$PYVER" -lt "$MIN_PYVER" ]; then
if [ "$1" != "NOCRASH" ]; then
error "You have an ancient version of Python entombed in your operating system..."
@@ -298,6 +306,7 @@ DeterminePythonVersion() {
{{ bootstrappers/deb_common.sh }}
{{ bootstrappers/rpm_common_base.sh }}
{{ bootstrappers/rpm_common.sh }}
{{ bootstrappers/rpm_python3_legacy.sh }}
{{ bootstrappers/rpm_python3.sh }}
{{ bootstrappers/suse_common.sh }}
{{ bootstrappers/arch_common.sh }}
@@ -344,31 +353,50 @@ elif [ -f /etc/redhat-release ]; then
RPM_DIST_VERSION=0
fi
# Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
# RHEL 8 also uses python3 by default.
if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 -o "$PYVER" -eq 26 ]; then
RPM_USE_PYTHON_3=1
elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
RPM_USE_PYTHON_3=1
elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
RPM_USE_PYTHON_3=1
else
RPM_USE_PYTHON_3=0
fi
# Handle legacy RPM distributions
if [ "$PYVER" -eq 26 ]; then
# Check if an automated bootstrap can be achieved on this system.
if ! Python36SclIsAvailable; then
INTERACTIVE_BOOTSTRAP=1
fi
if [ "$RPM_USE_PYTHON_3" = 1 ]; then
Bootstrap() {
BootstrapMessage "RedHat-based OSes that will use Python3"
BootstrapRpmPython3
BootstrapMessage "Legacy RedHat-based OSes that will use Python3"
BootstrapRpmPython3Legacy
}
USE_PYTHON_3=1
BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
BOOTSTRAP_VERSION="BootstrapRpmPython3Legacy $BOOTSTRAP_RPM_PYTHON3_LEGACY_VERSION"
# Try now to enable SCL rh-python36 for systems already bootstrapped
# NB: EnablePython36SCL has been defined along with BootstrapRpmPython3Legacy in certbot-auto
EnablePython36SCL
else
Bootstrap() {
BootstrapMessage "RedHat-based OSes"
BootstrapRpmCommon
}
BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
# Starting to Fedora 29, python2 is on a deprecation path. Let's move to python3 then.
# RHEL 8 also uses python3 by default.
if [ "$RPM_DIST_NAME" = "fedora" -a "$RPM_DIST_VERSION" -ge 29 ]; then
RPM_USE_PYTHON_3=1
elif [ "$RPM_DIST_NAME" = "rhel" -a "$RPM_DIST_VERSION" -ge 8 ]; then
RPM_USE_PYTHON_3=1
elif [ "$RPM_DIST_NAME" = "centos" -a "$RPM_DIST_VERSION" -ge 8 ]; then
RPM_USE_PYTHON_3=1
else
RPM_USE_PYTHON_3=0
fi
if [ "$RPM_USE_PYTHON_3" = 1 ]; then
Bootstrap() {
BootstrapMessage "RedHat-based OSes that will use Python3"
BootstrapRpmPython3
}
USE_PYTHON_3=1
BOOTSTRAP_VERSION="BootstrapRpmPython3 $BOOTSTRAP_RPM_PYTHON3_VERSION"
else
Bootstrap() {
BootstrapMessage "RedHat-based OSes"
BootstrapRpmCommon
}
BOOTSTRAP_VERSION="BootstrapRpmCommon $BOOTSTRAP_RPM_COMMON_VERSION"
fi
fi
LE_PYTHON="$prev_le_python"
@@ -545,8 +573,15 @@ if [ "$1" = "--le-auto-phase2" ]; then
# If the selected Bootstrap function isn't a noop and it differs from the
# previously used version
if [ -n "$BOOTSTRAP_VERSION" -a "$BOOTSTRAP_VERSION" != "$PREV_BOOTSTRAP_VERSION" ]; then
# if non-interactive mode or stdin and stdout are connected to a terminal
if [ \( "$NONINTERACTIVE" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
# Check if we can rebootstrap without manual user intervention: this requires that
# certbot-auto is in non-interactive mode AND selected bootstrap does not claim to
# require a manual user intervention.
if [ "$NONINTERACTIVE" = 1 -a "$INTERACTIVE_BOOTSTRAP" != 1 ]; then
CAN_REBOOTSTRAP=1
fi
# Check if rebootstrap can be done non-interactively and current shell is non-interactive
# (true if stdin and stdout are not attached to a terminal).
if [ \( "$CAN_REBOOTSTRAP" = 1 \) -o \( \( -t 0 \) -a \( -t 1 \) \) ]; then
if [ -d "$VENV_PATH" ]; then
rm -rf "$VENV_PATH"
fi
@@ -557,12 +592,21 @@ if [ "$1" = "--le-auto-phase2" ]; then
ln -s "$VENV_PATH" "$OLD_VENV_PATH"
fi
RerunWithArgs "$@"
# Otherwise bootstrap needs to be done manually by the user.
else
error "Skipping upgrade because new OS dependencies may need to be installed."
error
error "To upgrade to a newer version, please run this script again manually so you can"
error "approve changes or with --non-interactive on the command line to automatically"
error "install any required packages."
# If it is because bootstrapping is interactive, --non-interactive will be of no use.
if [ "$INTERACTIVE_BOOTSTRAP" = 1 ]; then
error "Skipping upgrade because new OS dependencies may need to be installed."
error "This requires manual user intervention: please run this script again manually."
# If this is because of the environment (eg. non interactive shell without
# --non-interactive flag set), help the user in that direction.
else
error "Skipping upgrade because new OS dependencies may need to be installed."
error
error "To upgrade to a newer version, please run this script again manually so you can"
error "approve changes or with --non-interactive on the command line to automatically"
error "install any required packages."
fi
# Set INSTALLED_VERSION to be the same so we don't update the venv
INSTALLED_VERSION="$LE_AUTO_VERSION"
# Continue to use OLD_VENV_PATH if the new venv doesn't exist