starred/certbot
1
0
Fork 0
mirror of https://github.com/certbot/certbot.git synced 2026-07-31 10:16:52 +02:00
Code Issues Packages Projects Releases Wiki Activity

Migrate extended/full/test-* tests from azure pipelines to github actions (#10631)

Browse Source 
Related to https://github.com/certbot/certbot/issues/10581

Following up on #10622, this PR converts the `full-test-suite`
[pipeline](https://dev.azure.com/certbot/certbot/_build?definitionId=4)
from Azure to Github Actions.

Nightly test changes for context not included in this PR are available
[here](https://github.com/certbot/certbot/compare/test-convert-full-pipeline...convert-all-pipelines).

Since this branch is named `test-convert-full-pipeline`, these tests
will show up in the checks section of this PR.

The major changes I made here are splitting the docker and snaps tests
for a better github actions UX, and removing the intermediate "stage"
file, since stages are not a concept in GHA. This means that we get the
nice dropdowns for the different categories on the left bar of the [test
run
page](https://github.com/certbot/certbot/actions/runs/25139155528/job/73684692548)
so it's easier to see each type of test. The very slight drawback is
that the four jobs listed in `.github/workflows/full_test_suite.yml` do
need to be duplicated in `nightly.yml`, but that's a reasonable tradeoff
to me.

Also, we now test our certbot and dns plugin snaps on all architectures
for the first time (using `dpkg --add-architecture` to run armhf tests
on an arm64 machine), which is very nice and in my opinion worth the
very slightly extra time and code.

In this PR, we build arm64 and amd64 snaps directly on github's runners.
armhf snaps are built using launchpad as before. This makes the workflow
file a little long. There are perhaps some micro-optimizations for code
deduplication I could make, like creating an action to install
dependencies based on the architecture, but I don't think it's super
worth it, especially since the dependencies vary enough that we'd still
need some code (for example, even between installing deps for certbot
and dns runs, we'd still need to additionally install `nginx-light`).

A very slight potential time improvement we could make here would be to
optionally depend on the different architectures before running their
respective tests. I'm not sure if this can be done without writing
different jobs, and since once those jobs start they run in parallel, it
didn't really seem worth looking into for me. I am of course open to
alternate points of view here and in general.

Another potential change to bring the two build strategies more in line
would be to stop using the python script to send off all the launchpad
builds, and instead put each in a separate, matrixed job like the github
jobs. We could even continue retrying the builds within each job. This
would mean that if one dns plugin build happens to fail three times, all
the builds wouldn't have to be retried. While I think that's not the
worst idea, I personally think that belongs in a separate PR, as this PR
is already quite long.

Speaking of the PR length, I can undo the changes made here to build
arm64 and amd64 snaps on github actions, to have a simpler
conversion-only PR to review. Some of the choices I made here,
particularly around UX, were based on the fact that the jobs would look
like this, so it might not be as clear why I made those choices, but if
it's easier to review it's no problem to put it back. I could also
remove the code that tests the other snaps since it's new, but I figured
it'd be nice to show that they are in fact being built correctly, since
otherwise the built snaps wouldn't be consumed anywhere.

---------

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
This commit is contained in:
ohemorange
2026-05-08 18:03:26 -07:00
committed by GitHub
co-authored by Brad Warren
parent 208ff18405
commit 9339d23aa1
6 changed files with 446 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/docker_packaging_jobs.yml
+82
View File
@@ -0,0 +1,82 @@
name: Docker packaging
on:
workflow_call:
inputs:
dockerTag:
description: 'tag to assign docker images'
type: string
permissions:
contents: read
env:
DOCKER_TAG: ${{ inputs.dockerTag }}
jobs:
docker_build:
name: Build ${{ matrix.DOCKER_ARCH }}
runs-on:
- ${{ matrix.run-on }}
strategy:
fail-fast: false
matrix:
include:
- DOCKER_ARCH: arm64v8
run-on: ubuntu-24.04-arm
- DOCKER_ARCH: amd64
run-on: ubuntu-24.04
- DOCKER_ARCH: arm32v6
run-on: ubuntu-24.04
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- name: Build the Docker images
# We don't filter for the Docker Hub organization to continue to allow
# easy testing of these scripts on forks.
run: tools/docker/build.sh "$DOCKER_TAG" ${{ matrix.DOCKER_ARCH }}
shell: bash
- name: Save the Docker images
run: |-
DOCKER_IMAGES=$(docker images --filter reference="*/certbot" --filter reference="*/dns-*" --format "{{.Repository}}")
docker save --output images.tar $DOCKER_IMAGES
shell: bash
# If the name of the tar file or artifact changes, the deploy stage will
# also need to be updated.
- name: Store Docker artifact
uses: actions/upload-artifact@v7.0.0
with:
name: docker_${{ matrix.DOCKER_ARCH }}
path: "${{ github.workspace }}/images.tar"
docker_test:
name: Test ${{ matrix.DOCKER_ARCH }}
needs:
- docker_build
runs-on:
- ${{ matrix.run-on }}
strategy:
fail-fast: false
matrix:
include:
- DOCKER_ARCH: arm64v8
run-on: ubuntu-24.04-arm
- DOCKER_ARCH: amd64
run-on: ubuntu-24.04
- DOCKER_ARCH: arm32v6
run-on: ubuntu-24.04-arm
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- name: Retrieve Docker images
uses: actions/download-artifact@v8.0.1
with:
name: docker_${{ matrix.DOCKER_ARCH }}
path: "${{ github.workspace }}"
- name: Load Docker images
run: docker load --input ${{ github.workspace }}/images.tar
shell: bash
- name: Run integration tests for Docker images
run: tools/docker/test.sh "$DOCKER_TAG" ${{ matrix.DOCKER_ARCH }}
shell: bash
.github/workflows/extended_tests_jobs.yml
+58
View File
@@ -0,0 +1,58 @@
name: Extended tests jobs
on:
workflow_call:
secrets:
AWS_TEST_FARM_PEM:
required: false
AWS_ACCESS_KEY_ID:
required: false
AWS_SECRET_ACCESS_KEY:
required: false
permissions:
contents: read
jobs:
test:
name: '-' # Makes results viewing cleaner
strategy:
fail-fast: false
matrix:
PYTHON_VERSION: ['3.14']
TOXENV:
- isolated-acme,isolated-certbot,isolated-apache,isolated-cloudflare,isolated-digitalocean,isolated-dnsimple,isolated-dnsmadeeasy,isolated-gehirn,isolated-google,isolated-linode,isolated-luadns,isolated-nsone,isolated-ovh,isolated-rfc2136,isolated-route53,isolated-sakuracloud,isolated-nginx
- nginx_compat
- modification
include:
- PYTHON_VERSION: '3.11'
TOXENV: py311
- PYTHON_VERSION: '3.12'
TOXENV: py312
- PYTHON_VERSION: '3.13'
TOXENV: py313
- PYTHON_VERSION: '3.10'
TOXENV: integration-certbot-oldest
- PYTHON_VERSION: '3.10'
TOXENV: integration-nginx-oldest
- PYTHON_VERSION: '3.10'
TOXENV: integration
- PYTHON_VERSION: '3.11'
TOXENV: integration
- PYTHON_VERSION: '3.12'
TOXENV: integration
- PYTHON_VERSION: '3.13'
TOXENV: integration
# python 3.14 integration tests are not run here because they're run as
# part of the standard test suite
- PYTHON_VERSION: '3.12'
TOXENV: integration-dns-rfc2136
- PYTHON_VERSION: '3.12'
TOXENV: test-farm-apache2
uses: "./.github/workflows/tox_steps.yml"
with:
PYTHON_VERSION: "${{ matrix.PYTHON_VERSION }}"
TOXENV: "${{ matrix.TOXENV }}"
IMAGE_NAME: ubuntu-22.04
secrets:
AWS_TEST_FARM_PEM: "${{ secrets.AWS_TEST_FARM_PEM }}"
AWS_ACCESS_KEY_ID: "${{ secrets.AWS_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.AWS_SECRET_ACCESS_KEY }}"
.github/workflows/full_test_suite.yml
+39
View File
@@ -0,0 +1,39 @@
# Advanced pipeline for running our full test suite on demand.
name: Full test suite
on:
push:
branches:
# When changing these triggers, please ensure the documentation under
# "Running tests in CI" is still correct.
- test-*
workflow_dispatch:
permissions:
contents: read
jobs:
# These four jobs could be grouped in a separate workflow, but the github actions UI
# is much nicer if they are instead listed explicitly here.
standard_tests_jobs:
name: Standard tests
uses: "./.github/workflows/standard_tests_jobs.yml"
extended_tests_jobs:
name: Extended tests
uses: "./.github/workflows/extended_tests_jobs.yml"
secrets:
AWS_TEST_FARM_PEM: "${{ secrets.AWS_TEST_FARM_PEM }}"
AWS_ACCESS_KEY_ID: "${{ secrets.AWS_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.AWS_SECRET_ACCESS_KEY }}"
docker_packaging_jobs:
name: Docker packaging
uses: "./.github/workflows/docker_packaging_jobs.yml"
with:
# We don't publish our Docker images in this pipeline, but when building them
# for testing, let's use the nightly tag.
dockerTag: nightly
snap_packaging_jobs:
name: Snap packaging
uses: "./.github/workflows/snap_packaging_jobs.yml"
with:
snapBuildTimeout: 5400
secrets:
LAUNCHPAD_CREDENTIALS: "${{ secrets.LAUNCHPAD_CREDENTIALS }}"
.github/workflows/snap_packaging_jobs.yml
+248
View File
@@ -0,0 +1,248 @@
name: Snap packaging
on:
workflow_call:
inputs:
snapBuildTimeout:
description: 'timeout for snap builds in seconds'
type: number
secrets:
LAUNCHPAD_CREDENTIALS:
required: true
permissions:
contents: read
env:
SNAP_BUILD_TIMEOUT: ${{ inputs.snapBuildTimeout }}
jobs:
gha_build_snap:
name: Build certbot ${{ matrix.build-for }}
runs-on: ${{ matrix.build-on }}
strategy:
fail-fast: false
matrix:
include:
- build-for: arm64
build-on: ubuntu-24.04-arm
- build-for: amd64
build-on: ubuntu-24.04
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- uses: snapcore/action-build@v1.3.0
id: build
with:
snapcraft-args: "--build-for=${{ matrix.build-for }}"
- name: Store snap artifact
uses: actions/upload-artifact@v7.0.0
with:
name: snap-certbot-${{ matrix.build-for }}
path: ${{ steps.build.outputs.snap }}
generate_dns_list_matrix:
name: List DNS package names
runs-on: ubuntu-latest
outputs:
dns-dirs: ${{ steps.set-dns-dirs.outputs.dns-dirs }}
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- id: set-dns-dirs
run: |
DNS_NAMES="$(echo certbot-dns-* | jq -R -s -c 'split(" ")[:-1]')"
echo "dns-dirs=$DNS_NAMES" >> "$GITHUB_OUTPUT"
gha_build_dns_snaps:
name: Build ${{ matrix.dns-dir }} ${{ matrix.build-for }}
needs: generate_dns_list_matrix
runs-on: ${{ matrix.build-on }}
strategy:
fail-fast: false
matrix:
dns-dir: ${{ fromJSON(needs.generate_dns_list_matrix.outputs.dns-dirs) }}
build-for: [arm64, amd64]
include:
- build-for: arm64
build-on: ubuntu-24.04-arm
- build-for: amd64
build-on: ubuntu-24.04
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- name: generate dns snapcraft.yamls
run: |
tools/snap/generate_dnsplugins_all.sh # unnecessarily doing every file
- uses: snapcore/action-build@v1.3.0
id: build
with:
snapcraft-args: "--build-for=${{ matrix.build-for }}"
path: ${{ matrix.dns-dir }}
- name: Store snap artifact
uses: actions/upload-artifact@v7.0.0
with:
name: snap-${{ matrix.dns-dir }}-${{ matrix.build-for }}
path: ${{ steps.build.outputs.snap }}
launchpad_build_all:
name: Build armhf snaps
runs-on:
- ubuntu-24.04
env:
SNAP_ARCH: "armhf"
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0 # need a non-shallow clone for launchpad
persist-credentials: false
- name: Install dependencies
run: |-
sudo apt-get update
sudo apt-get install -y --no-install-recommends snapd
sudo snap install --classic snapcraft
- uses: actions/setup-python@v5.0.0
with:
python-version: '3.12'
- name: Build snaps
env:
LAUNCHPAD_CREDS: "${{ secrets.LAUNCHPAD_CREDENTIALS }}"
run: |-
git config --global user.email "github-actions[bot]"
git config --global user.name "41898282+github-actions[bot]@users.noreply.github.com"
mkdir -p ~/.local/share/snapcraft/
echo "$LAUNCHPAD_CREDS" > ~/.local/share/snapcraft/launchpad-credentials
python3 tools/snap/build_remote.py ALL --archs "$SNAP_ARCH" --timeout "$SNAP_BUILD_TIMEOUT"
- name: Flatten snaps layout
run: |-
mv *.snap ${{ runner.temp }}
mv certbot-dns-*/*.snap ${{ runner.temp }}
- name: Store snaps artifacts
uses: actions/upload-artifact@v7.0.0
with:
name: snaps_${{ env.SNAP_ARCH }}
path: "${{ runner.temp }}/*.snap"
snap_run:
name: Test certbot ${{ matrix.arch-name }}
needs:
- gha_build_snap
- launchpad_build_all
runs-on:
- ${{ matrix.run-on }}
strategy:
fail-fast: false
matrix:
include:
- arch-name: arm64
run-on: ubuntu-24.04-arm
- arch-name: amd64
run-on: ubuntu-24.04
- arch-name: armhf
run-on: ubuntu-24.04-arm
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- uses: actions/setup-python@v5.0.0
with:
python-version: '3.12'
- name: Install armhf dependencies
if: ${{ matrix.arch-name == 'armhf' }}
run: |-
sudo dpkg --add-architecture armhf
sudo apt-get update
# apparmor will conflict with snapd:armhf dependency if not removed first
sudo apt-get remove -y apparmor
sudo apt-get install -y --no-install-recommends snapd:armhf nginx-light
- name: Install non-armhf depdencies
if: ${{ matrix.arch-name != 'armhf' }}
run: |-
sudo apt-get update
sudo apt-get install -y --no-install-recommends nginx-light snapd
- name: Setup venv
run: |-
python3 -m venv venv
venv/bin/python tools/pip_install.py -U tox
- name: Retrieve Certbot snaps armhf
if: ${{ matrix.arch-name == 'armhf' }}
uses: actions/download-artifact@v8.0.1
with:
name: snaps_${{ matrix.arch-name }}
path: "${{ github.workspace }}/snap"
- name: Retrieve Certbot snaps non-armhf
if: ${{ matrix.arch-name != 'armhf' }}
uses: actions/download-artifact@v8.0.1
with:
pattern: snap-certbot-${{ matrix.arch-name }}
path: "${{ github.workspace }}/snap"
- name: Install Certbot snap
run: |-
sudo snap install --dangerous --classic snap/certbot_*.snap
- name: Run tox
run: |-
venv/bin/python -m tox run -e integration-external,apacheconftest-external-with-pebble
snap_dns_run:
name: Test DNS ${{ matrix.arch-name }}
needs:
- gha_build_dns_snaps
- gha_build_snap
- launchpad_build_all
runs-on:
- ${{ matrix.run-on }}
strategy:
fail-fast: false
matrix:
include:
- arch-name: arm64
run-on: ubuntu-24.04-arm
- arch-name: amd64
run-on: ubuntu-24.04
- arch-name: armhf
run-on: ubuntu-24.04-arm
steps:
- name: checkout
uses: actions/checkout@v6.0.2
with:
persist-credentials: false
- name: Install armhf dependencies
if: ${{ matrix.arch-name == 'armhf' }}
run: |-
sudo dpkg --add-architecture armhf
sudo apt-get update
# apparmor will conflict with snapd:armhf dependency if not removed first
sudo apt-get remove -y apparmor
sudo apt-get install -y --no-install-recommends snapd:armhf
- name: Install non-armhf depdencies
if: ${{ matrix.arch-name != 'armhf' }}
run: |-
sudo apt-get update
sudo apt-get install -y --no-install-recommends snapd
- uses: actions/setup-python@v5.0.0
with:
python-version: '3.12'
- name: Retrieve Certbot snaps armhf
if: ${{ matrix.arch-name == 'armhf' }}
uses: actions/download-artifact@v8.0.1
with:
name: snaps_${{ matrix.arch-name }}
path: "${{ github.workspace }}/snap"
- name: Retrieve Certbot snaps non-armhf
if: ${{ matrix.arch-name != 'armhf' }}
uses: actions/download-artifact@v8.0.1
with:
pattern: snap-*-${{ matrix.arch-name }}
merge-multiple: true
path: "${{ github.workspace }}/snap"
- name: Display structure of downloaded files
run: ls -R "${{ github.workspace }}/snap"
- name: Prepare Certbot-CI
run: |-
python3 -m venv venv
venv/bin/python tools/pip_install.py -e certbot-ci
- name: Test DNS plugins snaps
run: |-
sudo -E venv/bin/pytest certbot-ci/src/snap_integration_tests/dns_tests --allow-persistent-changes --snap-folder ${{ github.workspace }}/snap --snap-arch ${{ matrix.arch-name }}
.github/workflows/tox_steps.yml
+19
View File
@@ -10,6 +10,14 @@ on:
type: string
PIP_USE_PEP517:
type: string
secrets:
AWS_TEST_FARM_PEM:
required: false
AWS_ACCESS_KEY_ID:
required: false
AWS_SECRET_ACCESS_KEY:
required: false
permissions:
contents: read
@@ -18,6 +26,8 @@ jobs:
name: ${{ inputs.TOXENV }} ${{ inputs.IMAGE_NAME }}
runs-on:
- "${{ inputs.IMAGE_NAME }}"
env:
AWS_EC2_PEM_FILE: ${{ github.workspace }}/GHAKeyPair.pem
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
@@ -58,8 +68,17 @@ jobs:
run: |-
python3 tools/pip_install.py tox
shell: bash
- name: Create test farm pem file
if: contains(inputs.TOXENV, 'test-farm')
env:
PEM_CONTENTS: "${{ secrets.AWS_TEST_FARM_PEM }}"
run: |-
echo "${PEM_CONTENTS}" >> $AWS_EC2_PEM_FILE
shell: bash
- name: Run tox
env:
AWS_ACCESS_KEY_ID: "${{ secrets.AWS_ACCESS_KEY_ID }}"
AWS_SECRET_ACCESS_KEY: "${{ secrets.AWS_SECRET_ACCESS_KEY }}"
PIP_USE_PEP517: "${{ inputs.PIP_USE_PEP517 }}"
TOXENV: "${{ inputs.TOXENV }}"
run: |-