Add towncrier for automatic changelog generation (#10379)

blast from the past! resurrects
https://github.com/certbot/certbot/pull/9803 with all of @bmw's changes.
i figured instead of force-pushing a basically brand new branch and
obliterating the old review, i'd just start from a clean slate

fixes #8272

---------

Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
Co-authored-by: Brad Warren <bmw@eff.org>

.azure-pipelines/templates/jobs/standard-tests-jobs.yml

@@ -41,6 +41,9 @@ jobs:
         nginxroundtrip:
           IMAGE_NAME: ubuntu-22.04
           TOXENV: nginxroundtrip
+        validate-changelog:
+          IMAGE_NAME: ubuntu-22.04
+          TOXENV: validate-changelog
     pool:
       vmImage: $(IMAGE_NAME)
     steps:

.github/pull_request_template.md

@@ -1,6 +1,7 @@
 ## Pull Request Checklist
 
 - [ ] The Certbot team has recently expressed interest in reviewing a PR for this. If not, this PR may be closed due our limited resources and need to prioritize how we spend them.
-- [ ] If the change being made is to a [distributed component](https://certbot.eff.org/docs/contributing.html#code-components-and-layout), edit the `main` section of `certbot/CHANGELOG.md` to include a description of the change being made.
+- [ ] If the change being made is to a [distributed component](https://certbot.eff.org/docs/contributing.html#code-components-and-layout), add a description of your change to the `newsfragments` directory. This should be a file called `<title>.<type>`, where `<title>` is either a GitHub issue number or some other unique name starting with `+`, and `<type>` is either `changed`, `fixed`, or `added`.
+  * For example, if you fixed a bug for issue number 42, create a file called `42.fixed` and put a description of your change in that file.
 - [ ] Add or update any documentation as needed to support the changes in this PR.
 - [ ] Include your name in `AUTHORS.md` if you like.

certbot/CHANGELOG.md

@@ -2,29 +2,7 @@
 
 Certbot adheres to [Semantic Versioning](https://semver.org/).
 
-## 4.2.0 - main
+<!-- towncrier release notes start -->
-
-### Added
-
-* Added `--eab-hmac-alg` parameter to support custom HMAC algorithm for External Account Binding.
-
-### Changed
-
-* Catches and ignores errors during the directory fetch for ARI checking so that these
-  errors do not hinder the actual certificate issuance
-* Removed the dependency on `pytz`.
-
-### Fixed
-
-* Certbot now always uses the server value from the renewal configuration file
-  for ARI checks instead of the server value from the current invocation of
-  Certbot. This helps prevent ARI requests from going to the wrong server if
-  the user changes CAs.
-* Previously, we claimed to set FAILED_DOMAINS and RENEWED_DOMAINS env variables for use by
-  post-hooks when certificate renewals fail, but we were not actually setting them. Now, we are.
-
-More details about these changes can be found on our GitHub repo.
-
 ## 4.1.1 - 2025-06-12
 
 ### Fixed

certbot/setup.py

@@ -50,6 +50,7 @@ dev_extras = [
     'poetry>=1.2.0',
     # allows us to use newer urllib3 https://github.com/python-poetry/poetry-plugin-export/issues/183
     'poetry-plugin-export>=1.9.0',
+    'towncrier',
     'twine',
 ]
 

newsfragments/.gitignore

@@ -0,0 +1 @@
+!.gitignore

newsfragments/10259.fixed

@@ -0,0 +1 @@
+Previously, we claimed to set FAILED_DOMAINS and RENEWED_DOMAINS env variables for use by post-hooks when certificate renewals fail, but we were not actually setting them. Now, we are.

newsfragments/10281.fixed

@@ -0,0 +1 @@
+Added `--eab-hmac-alg` parameter to support custom HMAC algorithm for External Account Binding.

newsfragments/10339.fixed

@@ -0,0 +1 @@
+Certbot now always uses the server value from the renewal configuration file for ARI checks instead of the server value from the current invocation of Certbot. This helps prevent ARI requests from going to the wrong server if the user changes CAs.

newsfragments/10342.changed

@@ -0,0 +1 @@
+Catches and ignores errors during the directory fetch for ARI checking so that these errors do not hinder the actual certificate issuance.

newsfragments/10350.changed

@@ -0,0 +1 @@
+Removed the dependency on `pytz`.

tools/_changelog_top.txt

@@ -1,15 +0,0 @@
-## nextversion - main
-
-### Added
-
-*
-
-### Changed
-
-*
-
-### Fixed
-
-*
-
-More details about these changes can be found on our GitHub repo.

tools/_release.sh

@@ -82,7 +82,7 @@ tmpvenv=$(mktemp -d)
 python3 -m venv "$tmpvenv"
 . $tmpvenv/bin/activate
 # update packaging tools to their pinned versions
-tools/pip_install.py virtualenv
+tools/pip_install.py towncrier virtualenv
 
 root_without_le="$version.$$"
 root="$RELEASE_DIR/le.$root_without_le"
@@ -96,9 +96,9 @@ if [ "$RELEASE_BRANCH" != "candidate-$version" ] ; then
 fi
 git checkout "$RELEASE_BRANCH"
 
-# Update changelog
+# Update changelog. `--yes` automatically clears out older newsfragments,
-sed -i "0,/main/ s/main/$(date +'%Y-%m-%d')/" certbot/CHANGELOG.md
+# and all of towncrier's changes are staged for commit when it's done
-git add certbot/CHANGELOG.md
+towncrier build --version "$version" --yes
 git commit -m "Update changelog for $version release"
 
 for pkg_dir in $SUBPKGS certbot-compatibility-test
@@ -205,18 +205,6 @@ git tag --local-user "$RELEASE_GPG_KEY" --sign --message "Release $version" "$ta
 git rm --cached -r "$built_package_dir"
 git commit -m "Remove built packages from git"
 
-# Add main section to CHANGELOG.md
-header=$(head -n 4 certbot/CHANGELOG.md)
-body=$(sed s/nextversion/$nextversion/ tools/_changelog_top.txt)
-footer=$(tail -n +5 certbot/CHANGELOG.md)
-echo "$header
-
-$body
-
-$footer" > certbot/CHANGELOG.md
-git add certbot/CHANGELOG.md
-git commit -m "Add contents to certbot/CHANGELOG.md for next version"
-
 if [ "$RELEASE_BRANCH" = candidate-"$version" ] ; then
     SetVersion "$nextversion".dev0
     git commit -m "Bump version to $nextversion"

tools/requirements.txt

@@ -18,8 +18,8 @@ babel==2.17.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 backports-tarfile==1.2.0 ; python_full_version >= "3.9.2" and python_version < "3.12"
 bcrypt==4.3.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 beautifulsoup4==4.13.4 ; python_full_version >= "3.9.2" and python_version < "4.0"
-boto3==1.39.4 ; python_full_version >= "3.9.2" and python_version < "4.0"
+boto3==1.39.16 ; python_full_version >= "3.9.2" and python_version < "4.0"
-botocore==1.39.4 ; python_full_version >= "3.9.2" and python_version < "4.0"
+botocore==1.39.16 ; python_full_version >= "3.9.2" and python_version < "4.0"
 build==1.2.2.post1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 cachecontrol==0.14.3 ; python_full_version >= "3.9.2" and python_version < "4.0"
 cachetools==5.5.2 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -28,18 +28,20 @@ cffi==1.17.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 chardet==5.2.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 charset-normalizer==3.4.2 ; python_full_version >= "3.9.2" and python_version < "4.0"
 cleo==2.1.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
+click==8.1.8 ; python_full_version >= "3.9.2" and python_version < "3.10"
+click==8.2.1 ; python_version >= "3.10" and python_version < "4.0"
 cloudflare==2.19.4 ; python_full_version >= "3.9.2" and python_version < "4.0"
 colorama==0.4.6 ; python_full_version >= "3.9.2" and python_version < "4.0"
 configargparse==1.7.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 configobj==5.0.9 ; python_full_version >= "3.9.2" and python_version < "4.0"
-coverage==7.9.2 ; python_full_version >= "3.9.2" and python_version < "4.0"
+coverage==7.10.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 crashtest==0.4.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 cryptography==45.0.5 ; python_full_version >= "3.9.2" and python_version < "4.0"
 cython==0.29.37 ; python_full_version >= "3.9.2" and python_version < "4.0"
 decorator==5.2.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 deprecated==1.2.18 ; python_full_version >= "3.9.2" and python_version < "4.0"
 dill==0.4.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
-distlib==0.3.9 ; python_full_version >= "3.9.2" and python_version < "4.0"
+distlib==0.4.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 distro==1.9.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 dns-lexicon==3.21.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 dnspython==2.7.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -53,7 +55,7 @@ fastjsonschema==2.21.1 ; python_full_version >= "3.9.2" and python_version < "4.
 filelock==3.18.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 findpython==0.6.3 ; python_full_version >= "3.9.2" and python_version < "4.0"
 google-api-core==2.25.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
-google-api-python-client==2.176.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
+google-api-python-client==2.177.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 google-auth-httplib2==0.2.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 google-auth==2.40.3 ; python_full_version >= "3.9.2" and python_version < "4.0"
 googleapis-common-protos==1.70.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -66,6 +68,7 @@ idna==3.10 ; python_full_version >= "3.9.2" and python_version < "4.0"
 imagesize==1.4.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 importlib-metadata==8.6.1 ; python_full_version >= "3.9.2" and python_version < "3.10"
 importlib-metadata==8.7.0 ; python_version >= "3.10" and python_version < "3.12"
+importlib-resources==6.5.2 ; python_full_version >= "3.9.2" and python_version < "3.10"
 iniconfig==2.1.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 installer==0.7.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 invoke==2.2.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -97,13 +100,13 @@ msgpack==1.1.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 msrest==0.7.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 mypy-extensions==1.1.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 mypy==1.9.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
-nh3==0.2.22 ; python_full_version >= "3.9.2" and python_version < "4.0"
+nh3==0.3.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 oauthlib==3.3.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 packaging==25.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 paramiko==3.5.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 parsedatetime==2.6 ; python_full_version >= "3.9.2" and python_version < "4.0"
 parso==0.8.4 ; python_full_version >= "3.9.2" and python_version < "4.0"
-pbs-installer==2025.7.12 ; python_full_version >= "3.9.2" and python_version < "4.0"
+pbs-installer==2025.7.23 ; python_full_version >= "3.9.2" and python_version < "4.0"
 pexpect==4.9.0 ; python_full_version >= "3.9.2" and python_version < "4.0" and (sys_platform != "win32" and sys_platform != "emscripten" or python_version < "3.10") and sys_platform != "win32"
 pip==25.1.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 pkginfo==1.12.1.2 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -146,10 +149,10 @@ requests-oauthlib==2.0.0 ; python_full_version >= "3.9.2" and python_version < "
 requests-toolbelt==1.0.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 requests==2.32.4 ; python_full_version >= "3.9.2" and python_version < "4.0"
 rfc3986==2.0.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
-rich==14.0.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
+rich==14.1.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 roman-numerals-py==3.1.0 ; python_version >= "3.11" and python_version < "4.0"
 rsa==4.9.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
-s3transfer==0.13.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
+s3transfer==0.13.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 secretstorage==3.3.3 ; python_full_version >= "3.9.2" and python_version < "4.0" and sys_platform == "linux"
 semantic-version==2.10.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 setuptools-rust==1.11.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -174,6 +177,7 @@ stack-data==0.6.3 ; python_full_version >= "3.9.2" and python_version < "4.0"
 tldextract==5.3.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 tomli==2.2.1 ; python_full_version >= "3.9.2" and python_version <= "3.10"
 tomlkit==0.13.3 ; python_full_version >= "3.9.2" and python_version < "4.0"
+towncrier==24.8.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 tox==4.27.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 traitlets==5.14.3 ; python_full_version >= "3.9.2" and python_version < "4.0"
 trove-classifiers==2025.5.9.12 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -181,7 +185,7 @@ twine==6.1.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 types-httplib2==0.22.0.20250622 ; python_full_version >= "3.9.2" and python_version < "4.0"
 types-pyrfc3339==2.0.1.20241107 ; python_full_version >= "3.9.2" and python_version < "4.0"
 types-python-dateutil==2.9.0.20250708 ; python_full_version >= "3.9.2" and python_version < "4.0"
-types-pywin32==310.0.0.20250516 ; python_full_version >= "3.9.2" and python_version < "4.0"
+types-pywin32==311.0.0.20250728 ; python_full_version >= "3.9.2" and python_version < "4.0"
 types-requests==2.31.0.6 ; python_full_version >= "3.9.2" and python_version < "3.10"
 types-requests==2.32.4.20250611 ; python_version >= "3.10" and python_version < "4.0"
 types-setuptools==80.9.0.20250529 ; python_full_version >= "3.9.2" and python_version < "4.0"
@@ -190,7 +194,7 @@ typing-extensions==4.14.1 ; python_full_version >= "3.9.2" and python_version <
 uritemplate==4.2.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 urllib3==1.26.20 ; python_full_version >= "3.9.2" and python_version < "3.10"
 urllib3==2.5.0 ; python_version >= "3.10" and python_version < "4.0"
-virtualenv==20.31.2 ; python_full_version >= "3.9.2" and python_version < "4.0"
+virtualenv==20.32.0 ; python_full_version >= "3.9.2" and python_version < "4.0"
 wcwidth==0.2.13 ; python_full_version >= "3.9.2" and python_version < "4.0"
 wheel==0.45.1 ; python_full_version >= "3.9.2" and python_version < "4.0"
 wrapt==1.17.2 ; python_full_version >= "3.9.2" and python_version < "4.0"

towncrier.toml

@@ -0,0 +1,26 @@
+[tool.towncrier]
+package = "certbot"
+directory = "newsfragments"
+filename = "certbot/CHANGELOG.md"
+title_format = "## {version} - {project_date}"
+issue_format = "[#{issue}](https://github.com/certbot/certbot/issues/{issue})"
+issue_pattern = "\\d+"
+wrap = true
+ignore = []
+
+# Our three custom news fragment types, declared in order of how we'd like them
+# to appear in the changelog
+[[tool.towncrier.type]]
+directory = "added"
+name = "Added"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "changed"
+name = "Changed"
+showcontent = true
+
+[[tool.towncrier.type]]
+directory = "fixed"
+name = "Fixed"
+showcontent = true

tox.ini

@@ -303,3 +303,9 @@ deps = -e {toxinidir}/letstest
 commands = {toxinidir}/tools/retry.sh letstest targets/targets.yaml {env:AWS_EC2_PEM_FILE} SET_BY_ENV scripts/test_apache2.sh --repo {toxinidir}
 allowlist_externals =
     {toxinidir}/tools/retry.sh
+
+[testenv:validate-changelog]
+deps = towncrier
+# we set --version here so towncrier doesn't try to guess the certbot version which requires certbot
+# to be installed
+commands = towncrier build --draft --version 99.99.0