Commit Graph
100 Commits
Author SHA1 Message Date
Matt Clay 759e037edc Fix integration tests with custom inventory.
All inventory hosts to which a connection is actually established
should declare ansible_python_interpreter to avoid interpreter
auto-discovery, as that may select the wrong interpreter.
2019-03-11 21:43:58 -07:00
Matt ClayandGitHub 8fa294a33a Fix ansible-test coverage stub memory usage. (#53652)
Split coverage stub files into multiple parts based on the number
of lines in files included in each stub. This will keep memory usage
down to avoid hanging on Shippable.
2019-03-11 15:14:09 -07:00
Matt ClayandGitHub 887ab35656 Add support for ansible-test --coverage-check. (#53573) 2019-03-11 15:13:09 -07:00
Matt ClayandGitHub 5134ba95ca Fix conditionals integration test. (#53583)
* Fix inventory for conditionals test.
* Remove extra groups from conditionals test.
2019-03-08 22:59:00 -08:00
Matt Clay 6588df505c Improve display of test timeout message. 2019-03-08 22:43:06 -08:00
Matt Clay 2c59d88154 Fix yamllint issue. 2019-03-08 22:42:19 -08:00
Matt Clay 0c4785720d Add extensions to changelog fragments. 2019-03-08 22:42:19 -08:00
Matt Clay 40318b49b2 Check extensions on changelog fragments. 2019-03-08 22:42:19 -08:00
Matt Clay 37bfa92076 Disable DevTest Labs tests. 2019-03-08 22:22:41 -08:00
Matt Clay 5c866fc17e Move fragment to correct location. 2019-03-08 21:38:53 -08:00
Matt Clay 9546761407 Fix coverage stub generation in CI. 2019-03-08 16:29:14 -08:00
Matt Clay 8fe9eb7b92 Show invalid coverage paths with verbosity > 1. 2019-03-08 15:47:44 -08:00
Matt Clay 7c0bc9e4ac Fix no-assert code-smell regex. 2019-03-08 13:20:20 -08:00
Matt ClayandGitHub 36baf7d83a Make ansible-test less noisy for change detection. (#53497)
* Make ansible-test less noisy for change detection.
* Set verbosity=2 for dependent file expansion.
2019-03-07 22:50:10 -08:00
Matt Clay 8535c05b29 Add ansible-test finally block after tests run.
This will record data from passing tests even when tests fail.
2019-03-07 18:31:02 -08:00
Matt Clay 3907ea31f5 Rebalance docker_* integration tests. 2019-03-07 10:59:40 -08:00
Matt Clay 1c780e79c6 Fix ansible-test junit timeout xml output. 2019-03-07 09:34:46 -08:00
Matt Clay 9ed9cfb40c Split Windows tests into 5 groups. 2019-03-06 20:52:17 -08:00
Matt ClayandGitHub 3d2c0f4973 Fix ansible-test timeout junit output. (#53427)
Fix ansible-test timeout junit output.

Implemented a hack to work around not having junit-xml when running on Shippable outside of our test containers.
2019-03-06 19:46:41 -08:00
Matt Clay 156613f025 Fix coverage handling for non_local tests. 2019-03-06 16:30:37 -08:00
Matt Clay 77a03af394 Use sed instead of perl in assert test. 2019-03-06 15:11:47 -08:00
Matt ClayandGitHub a8d829d9c3 Add timeout support to ansible-test. (#53302)
* Add timeout support to ansible-test.
* Fix ansible-test tar filename filter bug.
* Update timeouts used on Shippable.
* Kill subprocesses when parent process terminates.
* Require explicit use of env --show option.
2019-03-05 11:58:13 -08:00
Matt Clay 29c9b4001d Remove invalid validate-modules ignore entry. 2019-03-01 09:04:42 -08:00
Matt ClayandGitHub eeaff731de Overhaul ansible-test cloud test plugins. (#53044) 2019-02-28 18:25:49 -08:00
Matt Clay e4d0be39c3 Remove skip/rhel8.0 from osx_defaults test.
The test already has `skip/rhel` which is enough.
2019-02-27 17:48:16 -08:00
Matt Clay 861446b2a6 Run network tests when paramiko plugin changes. 2019-02-26 20:47:17 -08:00
Matt Clay b713246f57 Reduce vcenter integration test boilerplate. 2019-02-26 11:29:21 -08:00
Matt Clay c9f549e9c1 Fix integration test script coverage reporting.
Code coverage reporting was ignoring scripts executed during integration
tests when those scripts resided in the temporary working directory used
during an integration test run.
2019-02-26 10:21:02 -08:00
Matt Clay 52a8957b6e Revert "foreman: Use generic python in test run (#52544)"
Cannot use `python.py` with `-c`.

This reverts commit 2991c5d47c.
2019-02-25 12:16:23 -08:00
Matt Clay 4ea96c151b Rebalance AWS integration tests in CI. 2019-02-22 21:56:29 -08:00
Matt Clay 05c303a39f Rebalance Azure tests in CI. 2019-02-22 20:59:18 -08:00
Matt Clay b2f354ce29 Ignore errors during container cleanup in CI. 2019-02-21 21:20:48 -08:00
Matt Clay e8a31a12be Use a consistent RLIMIT_NOFILE for ansible-test. 2019-02-18 23:43:37 -08:00
Matt ClayandGitHub a1fec259c1 Reduce console noise at start of CI session. (#52387)
* Reduce console noise at start of CI session.
* Use allowed shebang.
2019-02-18 13:41:45 -08:00
Matt ClayandSam Doran 687279c7bd Set user expires on FreeBSD using UTC. (#52276) 2019-02-18 09:59:13 -05:00
Matt Clay f86c24a53c Update the default container to version 1.5.2.
ci_complete
2019-02-15 19:07:43 -08:00
Matt ClayandGitHub f5c92f6bc1 Allow setting resource.RLIMIT_NOFILE in modules (#51989) 2019-02-15 17:52:35 -08:00
Matt Clay 1edee04bc5 Update pytest constraint for Python 2.7. 2019-02-15 16:36:04 -08:00
Matt Clay bbad5d1d16 Unfreeze six in test constraints.
This avoids re-installation during integration test runs.

Pinning this requirement shouldn't be needed for consistent test
results when running pylint.
2019-02-15 14:44:04 -08:00
Matt Clay 6ac22416af Fix tests to use ANSIBLE_TEST_PYTHON_INTERPRETER. 2019-02-15 11:24:07 -08:00
Matt Clay 02c9f70898 Fix sanity test failures (pep8). 2019-02-15 09:25:20 -08:00
Matt Clay 6a82827ddd Add Python 3.8 to ansible-test and CI. 2019-02-14 23:24:24 -08:00
Matt Clay d2c5fc8cac Update pytest and pytest-forked constraints. 2019-02-14 11:30:49 -08:00
Matt Clay aea4902822 Add constraint for pyone to keep tests passing. 2019-02-14 10:20:18 -08:00
Matt ClayandGitHub 9bb1c59686 Add FreeBSD 12.0 to CI. (#52195) 2019-02-13 23:36:07 -08:00
Matt Clay 3e778d3f8f Fix pycodestyle E117 issues. 2019-02-13 23:35:39 -08:00
Matt Clay 7016559b1a Fix openssl_privatekey test on FreeBSD 12.0. 2019-02-13 18:34:45 -08:00
Matt Clay 1a4163cb88 Fix misuse of is not testing a string. 2019-02-13 15:10:01 -08:00
Matt Clay 3f387f225c Fix safe_eval on Python 3.8. 2019-02-13 11:54:03 -08:00
Matt Clay 467d2ebecd Fix unit tests on Python 3.8. 2019-02-13 01:03:06 -08:00
Matt ClayandGitHub 1c352dfea2 Update test constraints for Python 2.6. (#52055) 2019-02-13 00:18:15 -08:00
Matt Clay 04112d47a7 Fix more incorrect uses of is operator. 2019-02-12 20:04:20 -08:00
Matt ClayandGitHub 1f3a74c0c8 Improve import sanity test output handling. (#52136)
Handling of unexpected output for the following is improved:

- stdout
- stderr
- warnings
2019-02-12 17:32:57 -08:00
Matt Clay 0a461380a3 Fix incorrect use of is for comparisons.
See https://bugs.python.org/issue34850 for details.
2019-02-12 16:33:13 -08:00
Matt Clay d033884465 Fix scaleway_lb test failures. 2019-02-11 11:20:33 -08:00
Matt Clay 6cba471958 Remove unnecessary validate-modules ignore entry. 2019-02-11 11:12:41 -08:00
Matt Clay 06d83bae05 Support ignore_changes in code-smell tests. 2019-02-05 15:21:01 -08:00
Matt Clayandansibot f6e129046a Fix performance issue with large task retries. (#51581) 2019-01-31 18:44:57 -05:00
Matt Clay a8116497ba More test fixes for split controller/remote tests. 2019-01-25 20:46:53 -08:00
Matt Clay 315434f676 Enable force_handlers for integration tests. 2019-01-25 08:51:57 -08:00
Matt Clay 35761a99c6 Make sure force_handlers is off for handlers test. 2019-01-25 00:26:09 -08:00
Matt Clay 48bdb61870 Fix path encoding bug in config manager. 2019-01-24 22:29:08 -08:00
Matt Clay 215b27eb10 Enable paramiko connection tests on RHEL 8.0. 2019-01-24 19:57:35 -08:00
Matt Clay d2a7cc0b9f Fix integration tests to support remote hosts. 2019-01-24 19:57:04 -08:00
Matt ClayandGitHub a20d0d1afb Fix dnf integration test issues. (#51251)
* update/enable rhel8 beta integration tests for yum and dnf

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix typo in conditional for dnf/tasks/main.yml

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix var scoping for environment checks

Signed-off-by: Adam Miller <admiller@redhat.com>

* Disable install via path on RHEL.
2019-01-23 13:52:03 -08:00
Matt Clay 37767849ee Disable failing azure_rm_postgresqlserver test. 2019-01-22 10:17:29 -08:00
Matt Clay ab70172651 Remove obsolete alias from nios_txt_record test. 2019-01-21 09:31:36 -08:00
Matt ClayandGitHub d4dbc7f2e0 Fix subversion integration test on Fedora 29. (#51089)
* Fix subversion integration test on Fedora 29.

This upgrades the sqlite-libs and subversion packages to make sure
that the version of sqlite expected by subversion is installed.

* Fix compatibility with RHEL and CentOS.
2019-01-18 16:07:17 -08:00
Matt Clay b3601435c4 Enable yum integration tests on RHEL 8.0. 2019-01-18 14:54:36 -08:00
Matt Clay c7b9ef740c Enable seboolean integration test on RHEL 8.0. 2019-01-18 14:07:36 -08:00
Matt Clay 85689a3eec Enable pip integration test on RHEL 8.0. 2019-01-18 13:50:48 -08:00
Matt Clay b834b29e43 Run integration tests from temporary directory.
ci_complete
2019-01-17 17:22:51 -08:00
Matt Clay d95c716249 Fix azure_rm_rediscache test resource naming. 2019-01-17 10:59:47 -08:00
Matt ClayandBrian Coca a864247bd5 Encoding fixes for plugin loader and vault. (#51002) 2019-01-17 12:18:01 -05:00
Matt Clay f06474ae4f Remove unnecessary absolute paths from tests. 2019-01-17 00:23:25 -08:00
Matt Clay d2a9b16f7e Make Shippable timing script more resilient.
This will permit the script to pass through content which could
not be previously decoded or encoded. This could occur when running
some tests on macOS using file paths with non-ASCII characters.
2019-01-17 00:03:38 -08:00
Matt Clay b971ebd343 More path encoding fixes. 2019-01-16 15:44:50 -08:00
Matt ClayandBrian Coca c46a09dcf4 Fix UnicodeEncodeError for ansible --version. (#50997)
The error occurs on Python 2.x when running the following commands:

```
mkdir /tmp/Δ.cfg
ANSIBLE_CONFIG=/tmp/Δ.cfg ansible --version | tee
```
2019-01-16 16:22:07 -05:00
Matt Clay 54867b4add Add comment explaining to_native usage. 2019-01-16 13:13:22 -08:00
Matt Clay 53a640f2cc Use to_bytes on open and to_native on load_source. 2019-01-16 13:13:22 -08:00
Matt Clay a8e6577403 Fix path encoding issue loading plugins. 2019-01-16 13:13:22 -08:00
Matt Clay 1d51c425ba Add missing file deps to vmware_inventory test. 2019-01-16 00:49:18 -08:00
Matt Clay 6842c0467f Fix ansible-test handling of empty test targets.
Previously empty test targets were ignored by ansible-test.
This would prevent them from participating in dependency analysis.
These targets are actually empty roles, and should be processed as such.
2019-01-16 00:03:43 -08:00
Matt Clay eee3dc888b Pass ansible-test inventory path to test funcs.
This avoids declaring the inventory path/names in multiple places.
Also expose the inventory path using the INVENTORY_PATH env var.
2019-01-15 18:07:13 -08:00
Matt ClayandGitHub 4d546398fd Further cleanup of integration test inventory. (#50919)
* Further cleanup of integration test inventory.
* Preserve aci and msc inventory in template.
* Update ansible-test inventory template handling.
* Fix classification of inventory file.
2019-01-15 16:42:49 -08:00
Matt Clay 634533208e Fix integration test shebangs and update tests.
* Integration tests now have their own list of allowed shebangs.
* Use `#!/usr/bin/env bash` instead of `#!/bin/bash`
  since the location is different on various platforms.
2019-01-15 14:08:26 -08:00
Matt ClayandAlicia Cozine 560e922440 Add more ansible-test aliases to test docs. (#50921) 2019-01-15 22:02:43 +01:00
Matt Clay 332dc6a93a Fix false positives in no-get-exception test. 2019-01-15 09:37:10 -08:00
Matt Clay bca50b47ed Switch set_fact test to private inventory.
Fixes a test failure when there is no explicit localhost in inventory.
2019-01-15 00:32:27 -08:00
Matt Clay 6a0452559b Handle non-target file deps for integration tests.
Some integration test targets have dependencies on files outside
the `test/integration/targets/` directory tree. Changes to these
dependencies can result in unexpected test failures since they do
not trigger integration tests which depend on them.
2019-01-14 23:58:10 -08:00
Matt ClayandGitHub 7b4bc572de Fix ansible-test target dependency issues. (#50908)
* Log dependencies at verbosity level 4.

This makes it easier to debug target dependency issues.

* Scan symlinks for target dependencies.

Some test targets use symlinks to files in other test targets.
These dependencies were previously undetected. This could result in
changes made to dependencies without triggering the dependent tests.

* Track missing target deps with `needs/target/*`.

Some existing test targets have untracked dependencies on other
test targets. This can result in changes to those dependencies
not triggering their dependent tests, resulting in test failures
after a PR is merged.

This PR adds the appropriate `needs/target/*` aliases to track
those dependencies, along with appropriate processing in
ansible-test to handle the new aliases.

* Scan meta dependencies in script targets.

Script targets are often former role targets which were converted
to allow custom invocations of ansible-playbook. These targets still
have their meta dependencies, but they were not being detected.

This could result in changes to dependencies not triggering the
targets which depend on them.
2019-01-14 19:57:32 -08:00
Matt Clay dca4eb3dcd Delegate ansible-test with LC_ALL=en_US.UTF-8.
This fixes encoding issues in ansible-test when running tests in
delegated environments that do not have LC_ALL or LANG set by default.
2019-01-14 19:13:51 -08:00
Matt Clay 35caebd036 Fix ansible-test dependency processing.
Previously, the following dependencies:

A used by B
B used by C

Would have been converted to:

A used by C
B used by C

Intead of being expanded to:

A used by B
A used by C
B used by C

This change preserves the existing dependency when expanding it.
2019-01-14 15:31:59 -08:00
Matt ClayandGitHub 465df0ef8d Fix encoding issues with file paths. (#50830)
* Fix encoding issues with file paths.

Discovered while testing with ANSIBLE_CONFIG env var set to a path
that contained unicode characters while LC_ALL=C.

* Fix unit tests.

* Fix another path encoding issue.
2019-01-14 13:06:47 -08:00
Matt Clay 15b1a31aa8 Use different test package in pip test.
This avoids conflicts with other tests such as firewalld,
which use the decorator package installed using OS packages.
2019-01-14 13:02:51 -08:00
Matt Clay d59bc9ae48 Move unrelated connection test to separate target. 2019-01-11 15:27:06 -08:00
Matt Clay fd56788a14 Fix dpkg_selections integration test.
Download package from S3 since the previous location no longer exists.
2019-01-11 09:39:27 -08:00
Matt Clay 6c7003127f Compare output as JSON to avoid sort issues. 2019-01-10 17:42:58 -08:00
Matt ClayandGitHub e5094e8071 Cleanup integration test inventory. (#50753)
* Move var_blending test inventory into test.
* Remove Amazon specific inventory entry for tests.
* Remove Azure specific inventory entry for tests.
* Move var_precedence test inventory into test.
* Move unicode test inventory into test.
* Remove unused inventory entry.
* Move gathering_facts test inventory into test.
* Move delegate_to test inventory into test.
* Clean up inventory for binary_modules test.
* Clean up integration test inventory.
2019-01-10 14:45:12 -08:00
Matt Clay 4915920d71 Use relative inventory paths instead of symlinks. 2019-01-09 19:10:34 -08:00