100 Commits
Author SHA1 Message Date
Ryan BrownandSloane Hertel a51eca364f New module: AWS EC2 Launch Template (#46972)
* Add launch template integration tests
2018-11-29 13:59:10 -05:00
Ryan BrownandSloane Hertel af6427bf40 VPC peer module unhandled exception on bad peering ID (#45306)
* Improve error handling for VPC peer module when a bad peering-id is specified

* changelog
2018-11-29 08:54:16 -05:00
Ryan BrownandSloane Hertel 7f7c059a04 ec2_asg: Retry Autoscaling Group delete calls when scaling activity is in progress (#46124)
* Retry delete calls when scaling activity is in progress

* changelog
2018-11-29 08:42:55 -05:00
Ryan BrownandSloane Hertel 405d6d605f [AWS] Enable Route53 integration tests (#46228) 2018-11-14 11:40:38 -05:00
Ryan BrownandGitHub 18a088c64e GCP MagicModules bug fixes (#47285)
Closes: #46930 #46929 #46928 #46927 #46926 #46925 #46924 #46923 #46922 #46921 #46920 #46919 #46918 #46917 #46916 #46915 #46914 #46913 #46912 #46911 #46910 #46909 #46908 #46907 #46906 #46905 #46903 #46902 #46901 #46900 #46899 #46898 #46897 #46896 #46895 #46894 #46893 #46892 #46891 #46890 #46889 #46888 #46887 #46886 #46885 #46884 #46883 #46882 #46881 #46880 #46879 #46878 #46877 #46876 #46875 #46874 #46873 #46872 #46871 #46870 #46869 #46868 #46867 #46866 #46865 #46864 #46863 #46862 #46861 #46860 #46859 #46858 #46857 #46856 #46855 #46854 #46853 #46852
2018-10-18 10:55:42 -04:00
Ryan S. Brown 05e3b5d632 [docs] Add changelog for ec2_asg launch template feature 2018-09-25 16:42:12 -04:00
Ryan BrownandSloane Hertel 4d7e261074 [aws] Remove deprecated ec2_vpc_dhcp_options_facts alias of ec2_vpc_dhcp_option_facts (#45322) 2018-09-07 09:09:28 -04:00
Ryan BrownandSloane Hertel 8d363f7693 [aws] Remove deprecated ec2_vpc_dhcp_options alias of ec2_vpc_dhcp_option (#45321) 2018-09-07 09:08:09 -04:00
Ryan BrownandGitHub 079299db4d [aws] ec2_group multi-account and peered VPC bugfix (#45296)
* Add tests to replicate bug #44788 

* Handle when userId is same account due to in-account peering

* Module defaults for main.yml

* Turn off VPC peering tests in CI
2018-09-06 15:06:03 -04:00
Ryan BrownandGitHub 20f21779d3 Fix ec2_group for numbered protocols (GRE) (#42765)
* Fix spurious `changed=True` when int is passed as tag

* Fix for all AWS module using compare_aws_tags

* Handle improperly stringified protocols and allow inconsistency between None/-1 on non-tcp protocols

* Add integration test that reproduces the same bug

* Return false if the comparsison is not equal
2018-09-05 13:34:26 -04:00
Ryan BrownandGitHub 9d1bf32825 [aws] Add suboptions to aws_application_scaling_policy (#43042)
* [aws] Add suboptions to aws_application_scaling_policy

Fixes #42917

* Remove any suboptions that are None
2018-09-05 12:35:26 -04:00
Ryan S. Brown c04f5a1d65 Codename: How Many More Times 2018-09-04 12:38:29 -04:00
Ryan S. Brown 617b8df5e6 Add rds_instance to module_defaults groups 2018-09-04 08:17:58 -04:00
Ryan S. Brown 990e95eb57 Fix ec2_instance synopsis. Fixes #44641 2018-08-24 15:44:28 -04:00
Ryan BrownandGitHub 4c8808ec9d Extend module_defaults by adding default groups for cloud modules (#44127)
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
2018-08-22 21:33:27 -04:00
Ryan BrownandWill Thames 91a16990d8 New module: Python Requirements Facts (#43108)
* New boto3_facts module

boto3_facts aims to help users see whether their python and module
versions are as expected.

* Rename to `assert_python_requirements`

* Update integration tests

* Document options

* fix imports

* boilerplate

* fix docs

* reorder import

* Make distutils optional and fail gracefully when it is not available

* fix example doc

* fix docs on requirements_facts
2018-08-21 09:19:10 +10:00
Ryan BrownandSloane Hertel 6d52afeed6 New module cloudformation_stack_set (#41669)
* [AWS] new module cloudformation_stack_set with integration tests
2018-08-20 14:38:14 -04:00
Ryan Brownandansibot b9df8564a2 Fix spelling of ansbile to ansible (#43898) 2018-08-09 18:32:42 -04:00
Ryan BrownandGitHub 43c508b3c3 [AWS] iam_role->AnsibleAWSModule and support IAM permission boundaries (#43270)
* Convert to AnsibleAWSModule and support IAM permission boundaries

* Handle adding boundary to existing role that lacks one

* Properly clean up role boundary associations on delete

* Handle case when policy boundary is `""` but does not exist
2018-07-30 15:06:23 -04:00
Ryan BrownandGitHub 1c082e93ef [aws] Catch errors raised when deleting objects remaining in an s3_bucket (#43358)
* Catch errors raised when deleting objects remaining in a bucket

* Also remove VersionId when it is not needed
2018-07-30 11:22:00 -04:00
Ryan BrownandSloane Hertel b6f8de9204 [aws] Enable SQS events for lambda_event module (#43019)
* Enable SQS events for `lambda_event` module

* Handle SQS event type properly

* PEP8
2018-07-20 10:52:21 -04:00
Ryan S. Brown 859ef3d134 Add CFN Stackset roadmap link 2018-07-20 09:48:34 -04:00
Ryan BrownandGitHub 07927edfac [AWS] Fix SSM Parameter Store module when description is missing (#43020) 2018-07-20 09:32:13 -04:00
Ryan S. Brown a2e7869362 Add supertom to GCP namespace ignore list 2018-07-18 08:25:13 -04:00
Ryan BrownandSloane Hertel 45f5964fed Fix patching of wrong boto3_conn in API Gateway tests (#42700) 2018-07-12 13:06:31 -04:00
Ryan BrownandWill Thames e4c28571d0 Update to AnsibleAWSModule and remove now-duplicate fail_json_aws function (#42629) 2018-07-12 22:52:28 +10:00
Ryan BrownandSloane Hertel d76e9008ee [aws] Default state of iam_managed_policy to present (#42069)
This default matches with the other IAM modules, such as iam_role.
2018-06-28 16:03:40 -04:00
Ryan BrownandSam Doran c7a5e9d4eb Remove unused imports from aws_direct_connect_connection module (#41802)
* Remove unused imports from aws_direct_connect_connection module

* Fix tuple notation
2018-06-21 14:50:09 -04:00
Ryan Brownandansibot 5abb63ea56 Remove unused imports from aws_config_aggregator (#41803) 2018-06-21 13:36:07 -04:00
Ryan BrownandSloane Hertel 84612577ba [aws] Remove unused return in ec2_vpc_dhcp_option_facts module (#41709) 2018-06-19 16:36:45 -04:00
Ryan BrownandSloane Hertel 531cdddeed [aws] Remove unused return in ec2_vpc_igw_facts module (#41708) 2018-06-19 16:31:18 -04:00
Ryan BrownandSloane Hertel a01a17c772 [aws] Remove extra variable definition in ec2_vpc_nacl module (#41707) 2018-06-19 16:29:56 -04:00
Ryan BrownandSloane Hertel 22f1b71be0 [aws] Remove unused variable in ec2_asg module (#41706) 2018-06-19 16:23:41 -04:00
Ryan BrownandSloane Hertel 956fe7362d [aws] Fix unused return value in ec2_vpc_vgw_facts module (#41697) 2018-06-19 14:46:24 -04:00
Ryan BrownandSloane Hertel 67d6e8177e [aws] remove unused variable in iam_user module (#41695) 2018-06-19 14:32:56 -04:00
Ryan BrownandSloane Hertel 1118e441e9 [aws] lambda_policy: Remove spurious definition of policy variable (#41693) 2018-06-19 14:30:04 -04:00
Ryan BrownandSloane Hertel fd8cc775a7 [aws] Simplify conditional in iam_cert module (#41691) 2018-06-19 14:28:43 -04:00
Ryan S. BrownandJonathan Davila 52acec3455 [aws] Remove double-defined variable from iam_policy module 2018-06-19 11:50:16 -04:00
Ryan S. Brown a5ec29ed49 Fix exception message in AWS lambda module 2018-06-19 10:11:14 -04:00
Ryan BrownandGitHub 7dcaef8115 Fix aws_config_aggregator argument count error (#41686) 2018-06-19 08:48:46 -04:00
Ryan BrownandSloane Hertel 7f8654d586 Add minimum botocore and boto3 checking to AnsibleAWSModule (#41005)
* Add minimum botocore and boto3 checking to AnsibleAWSModule
2018-06-01 18:45:22 -04:00
Ryan BrownandGitHub 52b79a87ee Terraform: Pass module targets to terraform plan as well as apply (#40958)
* Pass module targets to terraform plan as well as apply

* Fix undefined variable

* Add target for plan-only operation
2018-05-31 14:35:40 -04:00
Ryan BrownandSloane Hertel 046561bbb0 Split AWS Config modules (#40111)
* Adding module for AWS Config service

* adding integration tests

* Split resource types into their own modules

* Properly use resource_prefix and retry on IAM "eventual consistency"

* Add config aggregator module

* AWS config aggregator integration test fixes

* AWS config recorder module

* Config aggregation auth rule

* Use resource_prefix in IAM role name

* Disable config tests
2018-05-24 15:52:41 -04:00
Ryan BrownandSloane Hertel 858a1b09bb EC2_group module refactor (formerly pr/37255) (#38678)
* Refactor ec2_group

Replace nested for loops with list comprehensions

Purge rules before adding new ones in case sg has maximum permitted rules

* Add check mode tests for ec2_group

* add tests

* Remove dead code

* Fix integration test assertions for old boto versions

* Add waiter for security group that is autocreated

* Add support for in-account group rules

* Add common util to get AWS account ID

Fixes #31383

* Fix protocol number and add separate tests for egress rule handling

* Return egress rule treatment to be backwards compatible

* Remove functions that were obsoleted by `Rule` namedtuple

* IP tests

* Move description updates to a function

* Fix string formatting missing index

* Add tests for auto-creation of the same group in quick succession

* Resolve use of brand-new group in a rule without a description

* Clean up duplicated get-security-group function

* Add reverse cleanup in case of dependency issues

* Add crossaccount ELB group support

* Deal with non-STS calls to account API

* Add filtering of owner IDs that match the current account
2018-05-24 11:53:21 -04:00
Ryan BrownandSloane Hertel ae49f4fd35 Coerce VPC tags from to string types to prevent spurious changed (#40422)
* Coerce VPC tags from to string types to prevent spurious `changed`

* Remove dict comprehension for 2.6 compatibility
2018-05-23 10:17:34 -04:00
Ryan BrownandGitHub ed7d6d9c19 Add changelog for new GCP modules (#40542) 2018-05-22 11:35:29 -04:00
Ryan BrownandGitHub 858b1c277b [docs] Update ec2 module to use wait_for_connection instead of wait_for (#40171) 2018-05-22 11:21:04 -04:00
Ryan BrownandGitHub dda7d9e704 [AWS] Add Ansible Version to botocore user agent string (#39993)
Pull `config` out if it was provided to boto3_conn and add the useragent string
2018-05-17 11:41:14 -04:00
Ryan BrownandSloane Hertel ede47910f8 Add better handling for errors causing integration test instability (#40184)
* Add better handling for errors causing integration test instability
2018-05-15 15:31:19 -04:00
Ryan S. Brown 9a607283aa Terraform module: pass plan_file argument correctly
Per #39611 the `terraform` module wasn't properly building plan-only
files.
2018-05-02 13:45:24 -04:00
Ryan BrownandJordan Borean 269d682f70 Move camel2snake and snake2camel to common/dict_transformations.py (#39498)
Since it will be used outside just AWS modules, this commit moves
`camel_dict_to_snake_dict` and `snake_dict_to_camel_dict` functions into
a new module_utils file under common/ to match their wider usage.
2018-05-01 06:50:32 +10:00
Ryan BrownandWill Thames 86db62c0e9 [AWS] report boto3/botocore versions during fail_json_aws (#39298)
* [AWS] report boto3/botocore versions during `fail_json_aws`

When modules call `fail_json_aws` and boto3 is installed, gather the
boto3 and botocore versions so that any new AWS module issues will
include the user's boto3 installation info. This will make debugging
issues where features aren't available yet easier.

* PEP8

* Switch to `dict` rather than tuple returns
2018-04-27 10:05:41 +10:00
Ryan BrownandGitHub 33f358383a [aws][docs] Add AnsibleAWSModule features to AWS developer guidelines (#38978) 2018-04-26 08:26:03 -04:00
Ryan S. Brown cd39b8263d [aws] fix misspelling in ec2_instance docs 2018-04-26 08:16:37 -04:00
Ryan BrownandSloane Hertel 4dfef45449 [s3_bucket] Increase timeout on versioning check, as it seems to take longer to propagate than other properties (#39325) 2018-04-25 15:24:15 -04:00
Ryan S. Brown d13eccb7bc [aws] fix excepted/expected misspelling 2018-04-25 12:48:05 -04:00
Ryan BrownandSloane Hertel 9c9a70b168 Add retry to ec2_metadata_facts query (#38957)
* Add retry to ec2_metadata_facts query

* Fix conditional to allow for 404 errors
2018-04-23 15:58:35 -04:00
Ryan BrownandSloane Hertel 7544c59a23 Remove ResponseMetadata key from aws_caller_facts (#39102) 2018-04-20 18:31:35 -04:00
Ryan BrownandGitHub f428108845 [terraform] Disable input prompts during terraform init/plan/apply (#38842)
Per Hashicorp's [guidelines][1] for automated use of terraform CLI, this PR
adds the `-input=false` option to all the commands executed in the
module. If input is required, this causes a hard failure that will
become a module failure.

[1]: https://www.terraform.io/guides/running-terraform-in-automation.html
2018-04-20 12:28:49 -04:00
Ryan BrownandGitHub 44d06f8858 [cloud] Add support for updating IAM role with ec2_instance module (#38812)
* [cloud] Add support for updating IAM role with ec2_instance module

* Add test for updating IAM role
2018-04-17 15:02:46 -04:00
Ryan BrownandSloane Hertel 40b7a63d1a Add docs to AnsibleAWSModule.client retry settings (#38689) 2018-04-16 08:48:10 -04:00
Ryan S. Brown 53a3b8b602 [cloud] Use correct string types for comparisons in ec2_instance 2018-04-13 10:29:53 -04:00
Ryan S. Brown d4e5e385ca [docs][cloud] Include volume_size in ec2 module docs 2018-04-10 16:57:04 -04:00
Ryan BrownandSloane Hertel 56cd8f2d48 Retry creation of tags when DHCP optionset is not found (#38528) 2018-04-10 09:08:41 -04:00
Ryan BrownandGitHub 546a406a14 [docs] Add examples to ec2_instance docs (#38479) 2018-04-09 09:14:42 -04:00
Ryan BrownandWill Thames 1c7b9e66b4 AWS WAF module custom waiter (#37026)
Consolidate waiters to a single file

* Add waiter message with token ID
* Add waiter

Add waiter for WAF change tokens

Working waiter for waf_condition module

Add support for waiters to waf_rule

* WAF data model refactor

* Fix ref to self.client

* Add custom waiters to aws_waf_web_acl

* Allow add/remove rule tasks to operate in parallel, then wait for their change tokens to complete

* Move waiter into run_func_with_change_token_backoff since it is generic to all WAF update operations

* Wait for deletes on waf_web_acl

* Remove always-wait

* Remove waiter retry catch
2018-04-05 11:30:57 +10:00
Ryan Brownandansibot 87bd169ca9 Allow subnets with names formatted like subnet-1234 (#37740)
* Allow subnets with names formatted like `subnet-1234`

* Deduplicate IDs, in case a subnet is specified both by name and ID
2018-04-02 17:13:44 -04:00
Ryan BrownandSloane Hertel 93946ba627 Use NormalizedOperationMethod to catch ClientErrors so the waiter can handle them properly (#37356) 2018-03-13 16:02:39 -04:00
Ryan BrownandWill Thames c8ef07e015 Route custom waiter (#36922)
This creates a way for us to use boto3's data-driven waiter support to use custom waiters where Boto3 hasn't implemented them yet.

The only waiter implemented so far is for VPC Route Tables to check that they exist, and this replaces some custom retry code.
2018-03-06 08:22:16 +10:00
Ryan BrownandSloane Hertel 2f72d082c0 Use Tower v2 API when available, falling back to v1 where necessary (#36663)
* Use Tower v2 API when available, falling back to v1 where necessary
2018-03-02 11:31:53 -05:00
Ryan BrownandSloane Hertel 66c38dd6c3 Change to longer interval jittered backoffs for efs facts (#36266)
* Change to longer interval jittered backoffs for security group fact gathering

* Add the ECS throttling error code to the list of things to retry on
2018-02-20 10:17:31 -05:00
Ryan BrownandSloane Hertel e9c57e732f Add AWSRetry when describing VPCs to help stabilize integration tests (#36264)
* Add AWSRetry when describing VPCs to help stabilize integration tests

* Add retry on create_tags because it is possible to reach this API call before the VPC is finished creating
2018-02-20 08:56:34 -05:00
Ryan BrownandWill Thames 53a314f767 Fix incorrect docs in ec2_tag per #35738 (#36268) 2018-02-16 13:01:02 +10:00
Ryan BrownandGitHub 2678508d42 [cloud] Fix inconsistent case of Allocation_id in NAT Gateway outputs (#36058) 2018-02-15 10:51:16 -05:00
Ryan BrownandSloane Hertel f4d896c716 Tower callback fixes (#35927)
* Fix tower_callback script return code

* Add ec2_key parameter to integration tests
2018-02-09 14:28:52 -05:00
Ryan BrownandGitHub 0036cff493 [cloud] Add key_name parameter to ec2_instance (#35901) 2018-02-08 09:27:02 -05:00
Ryan BrownandSloane Hertel e71c6d8e17 New module: ec2_instance (#35749)
New module: ec2_instance

Integration tests for ec2_instance
2018-02-08 01:50:41 -05:00
Ryan S. Brown dff5fa3ec2 Fix os_project_access metadata and argspec import 2018-02-07 19:11:20 -05:00
Ryan S. Brown e24bd29e02 Add ec2_vpc_vpn to CHANGELOG 2018-02-07 17:40:12 -05:00
Ryan S. Brown f91f6e469f Add elb_target to CHANGELOG 2018-02-07 07:46:34 -05:00
Ryan S. Brown 7d721c1ec5 Add gcp_dns_managed_zone to CHANGELOG 2018-02-06 11:51:07 -05:00
Ryan S. Brown 415ed50876 Add aws_waf_rule to CHANGELOG 2018-02-02 17:55:15 -05:00
Ryan S. Brown 8621a80a82 Add aws_batch_job_queue and aws_batch_job_definition to CHANGELOG 2018-01-30 17:31:10 -05:00
Ryan S. Brown 05f765bda0 New module: aws_batch_job_definition 2018-01-30 17:28:45 -05:00
Ryan S. Brown eb14eac1c6 Add aws_batch_compute_environment to CHANGELOG 2018-01-30 13:35:26 -05:00
Ryan S. Brown 8dc0048698 Add AWS Direct Connect VIF (aws_direct_connect_virtual_interface) to CHANGELOG 2018-01-29 17:25:56 -05:00
Ryan S. Brown 13cbe115e1 Add new module iam_user to CHANGELOG 2018-01-26 14:56:27 -05:00
Ryan S. Brown a5d22e3781 Add aws_az_facts to CHANGELOG 2018-01-26 14:53:29 -05:00
Ryan S. Brown 94bb2e3a08 Add aws_region_facts to CHANGELOG 2018-01-26 14:49:48 -05:00
Ryan S. Brown f910d5d16d Add ec2_vpc_egress_igw to CHANGELOG 2018-01-26 09:23:08 -05:00
Ryan BrownandWill Thames b60c67d329 Remove double-specified EC2 ASG test parameters (#34714)
In the ec2_asg integration tests there are a couple places where
parameters are specified twice (as bot `tag.Name` and `name:`) and
others where those parameters aren't needed for the state requested.
2018-01-11 08:44:40 +10:00
Ryan Brownandansibot 95ff8f1a90 Change cow path selection variable from COWPATH to COW_PATH (#34063)
For consistency with other cow-related options, such as
`ANSIBLE_COW_SELECTION`, add an underscore to the new `ANSIBLE_COWPATH`
option.
2018-01-10 15:54:47 -05:00
Ryan S. Brown 60b3a191e6 Add terraform module to changelog 2018-01-10 14:18:53 -05:00
Ryan BrownandGitHub 9233520b31 New module: Terraform (#31214)
* First pass at Terraform module

* Support output variables

* Support idempotent plan checks and external plan files

* Add check mode support

* Support custom statefile location & remove color-coding from shell output

* Docs

* Let missing statefile hard-fail

* Update to support present/planned/absent states

* PEP8

* Add docs & fix python 2.6 incompatibility

* PEP8

* PEP8
2018-01-10 13:52:28 -05:00
Ryan BrownandSloane Hertel 2f66755849 New Lookup Plugin: aws_service_ip_ranges (#32749)
* added ranges lookup

* Add docs

* Use `open_url` from module_utils

* Fix error handling on open_url

* Fix ConnectionError exception import
2018-01-09 17:16:45 -05:00
Ryan S. Brown f0dbbfae84 Add Alex (rambleraptor) and Eric (erjohnso) as GCP module namespace maintainers 2018-01-04 07:45:29 -05:00
Ryan S. Brown b811c907d8 Add aws_application_scaling_policy to changelog 2017-12-21 19:47:55 -05:00
Ryan S. BrownandMatt Clay ae5c370737 Fail hard when tests pass that are expected to fail
`@pytest.mark.xfail()` can be used to decorate tests that don't yet
pass. By default, if I wrote a test like this:

```
@pytest.mark.xfail()
def test_naughty():
    assert 1 == 1
```

Then Pytest would allow this test to pass without notifying that an
expected-fail *didn't* fail. To make that happen, you have to set
xfail_strict=True so that the above test would cause the tests to fail.
2017-12-21 10:27:38 -08:00
Ryan S. Brown 339278fa15 Add new module aws_direct_connect_gateway to CHANGELOG 2017-12-20 09:57:55 -05:00
Ryan S. Brown bedb864bcb Add aws_account_attribute lookup to changelog 2017-12-19 12:21:44 -05:00