Add --use-pep517 flag to pip to silence warning in tools/venv.py, and switch codebase to src-layout (#10249)

Fixes #10252.

See further discussion here: https://github.com/pypa/pip/issues/11457

We are doing option:

> Alternatively, enable the --use-pep517 pip option, possibly with
--no-build-isolation. The --use-pip517 flag will force pip to use the
modern mechanism for editable installs. --no-build-isolation may be
needed if your project has build-time requirements beyond setuptools and
wheel. By passing this flag, you are responsible for making sure your
environment already has the required dependencies to build your package.
Once the legacy mechanism is removed, --use-pep517 will have no effect
and will essentially be enabled by default in this context.

Major changes made here include:
- Add `--use-pep517` to use the modern mechanism, which will be the only
mechanism in future pip releases
- Change to `/src` layout to appease mypy, and because for editable
installs that really is the normal way these days.
  - `cd acme && mkdir src && mv acme src/` etc.
- add `where='src'` argument to `find_packages` and add
`package_dir={'': 'src'},` in `setup.py`s
  - update `MANIFEST.in` files with new path locations 
- Update our many hardcoded filepaths
- Update `importlib-metadata` requirement to fix
double-plugin-entry-point problem in oldest tests

certbot/docs/conf.py

@@ -22,7 +22,7 @@ import sphinx
 here = os.path.abspath(os.path.dirname(__file__))
 
 # read version number (and other metadata) from package init
-init_fn = os.path.join(here, '..', 'certbot', '__init__.py')
+init_fn = os.path.join(here, '..', 'src', 'certbot', '__init__.py')
 with codecs.open(init_fn, encoding='utf8') as fd:
     meta = dict(re.findall(r"""__([a-z]+)__ = '([^']+)""", fd.read()))
 

certbot/docs/contributing.rst

@@ -140,13 +140,13 @@ If you're working on a specific test and would like to run just that one:
 
 .. code-block:: shell
 
-    pytest acme/acme/_internal/tests/messages_test.py # Use the test file you're working on
+    pytest acme/src/acme/_internal/tests/messages_test.py # Use the test file you're working on
 
 To run a specific test case within a file:
 
 .. code-block:: shell
 
-    pytest acme/acme/_internal/tests/messages_test.py -k test_to_partial_json
+    pytest acme/src/acme/_internal/tests/messages_test.py -k test_to_partial_json
 
 For debugging, we recommend putting
 ``import ipdb; ipdb.set_trace()`` statements inside the source code, which will require
@@ -261,8 +261,8 @@ certificate once it is issued. Some plugins, like the built-in Apache and Nginx
 plugins, implement both interfaces and perform both tasks. Others, like the
 built-in Standalone authenticator, implement just one interface.
 
-.. _interfaces.py: https://github.com/certbot/certbot/blob/main/certbot/certbot/interfaces.py
-.. _plugins/common.py: https://github.com/certbot/certbot/blob/main/certbot/certbot/plugins/common.py#L45
+.. _interfaces.py: https://github.com/certbot/certbot/blob/main/certbot/src/certbot/interfaces.py
+.. _plugins/common.py: https://github.com/certbot/certbot/blob/main/certbot/src/certbot/plugins/common.py#L45
 
 
 Authenticators

certbot/docs/using.rst

@@ -99,7 +99,7 @@ Apache
 ------
 
 The Apache plugin currently `supports
-<https://github.com/certbot/certbot/blob/main/certbot-apache/certbot_apache/_internal/entrypoint.py>`_
+<https://github.com/certbot/certbot/blob/main/certbot-apache/src/certbot_apache/_internal/entrypoint.py>`_
 modern OSes based on Debian, Fedora, SUSE, Gentoo, CentOS and Darwin.
 This automates both obtaining *and* installing certificates on an Apache
 webserver. To specify this plugin on the command line, simply include