This prevents the integration tests from getting run before the boulder
processes have finished booting in most cases. There's still some small
races with debug ports going up before RPC ports, but this flushes the
big ones (specifically, the WFE ports), and the boulder devs going to
fix the rest in integration-test.py over time.
This also makes boulder-start.sh a blocking operation. Now the TravisCI
integration tests no longer requires boulder-start.sh, we can let the
other priority of being easier for users to control (that is, basically,
make it easy to Ctrl-C) take over. That plus the idea that
self-daemonizing code is tricky to get right, especially over multiple
platforms led me to not trying to get start.py to make itself
asynchronous.
Most of this change is code movement in order to allow developers to run
boulder-start.sh once and boulder-integration.sh many times while also
not duplicating that code in order to run the tests in TravisCI.
I'm not a huge fan of both the letsencrypt's shell scripts and boulder's
integration-test.py having hard-coded file dependencies in the other's
repo.
This, however, seemed like the smallest path to code that would
spuriously break less. All the designs I was able to come up that were
maybe smaller changes either had the "starts tests before the servers
are up" problem or with a "each repo uses another repo's test code file"
problem. Those problem on top of the "it's a bigger change" problem led
me here.
Rather than fetching from bitbucket and building. Bitbucket is often down, and
building from scratch is slow. Github is sometimes down, but at least now we
have our eggs in one basket.
Candidate issues to be addressed:
- race condition: start.sh might fail to start Boulder WFE before
./tests/boulder-integration.sh is run (unlikely, but possible,
leading to flaky tests)
- intertwined build logs (./start.sh boostraps in the background,
integration test debug logs on server and client side)
Jeff Hodges