Terraform Weekly - Issue #5

Perhaps Terraform's greatest strength is its ability to display the changes it will make before it makes them. That strength has its limits. The tool will show you how its model of the world will change, but that model must still be translated into that of the cloud providers under management and this translation is in the hands of provider plugin authors, who don't have great tools to do their job. That seems to be getting better with upcoming improvements to the plugin SDK (and having a separate plugin SDK at all).

We are still in the beginning days here and hopefully things will continue to improve.

Hey Terraform Plugin Authors, the plugin SDK team appears to be gearing up for a v2 release. Like all major releases this contains a lot of new things, including a handful of breaking changes (maybe too many):

  • Dropping support For Terraform 0.11

  • Removing partial state

  • Panic on ResourceData.Set errors

  • Deprecate Exists

  • Context-aware functions

  • A cleaner API surface area

  • Diagnostics support

  • More accurate test runs

Here's the thing– There are some nice things here, but as a plugin author myself there is only one that gets me excited: the ability to return warnings (not just errors) via the new diagnostics support. Terraform doesn't do a great job of providing feedback to users. Opening up a channel to issue warnings from provider plugins is a great step forward.

Side note– does anyone read https://discuss.hashicorp.com ? I don't understand the strategy from Hashicorp here. Do enterprise customers want this? Based on the speed of responses, I don't think even Hashicorp employees even read it.

Here we have Muffy Barkocy with another installment of her series on RDS.

Sidenote– how many people have written blog posts that were the "first of a series" on a topic and never manage to write a second? I am happy to see someone following through.

This installment covers the challenges of managing RDS parameter groups in Terraform. Again, she highlights the impedance mismatch between the declarative model of Terraform and the imperative model of the AWS API, with a bit of stateful resource management thrown in. Who doesn't love an unplanned reboot of their production database?

Here's the thing– Terraform works hard to show you want is going to to happen before it takes action. Having a separate planning stage with comprehensible output makes for a much better developer experience than other operational tools. However, this is entirely dependent on provider plugin authors creating useful abstractions, because the plans are expressed at the level of those abstractions, not the resources created and managed in the cloud provider. Maybe the warnings feature in v2 of the plugin API will make it easer to give warnings in these cases?

Notable Releases

Depending on your workflow, tfenv may be critical for installing the proper version of Terraform. Version 2.0.0 of tfenv appears to be a significant rewrite. Changelog –

* New logging and debugging library

* Massive testing, logging and loading refactoring

* Fix to 'use' logic: don't overwrite .terraform-version files

* Fix #167 - Never invoke use automatically on install - multiple code and testing changes for new logic

* Fix to not use 0.12.22 during testing which reports its version incorrectly

* Introduce tfenv-resolve-version to deduplicate translation of requested version into actual version

* README.md updates

* Fix #176 - New parameter TFENV_AUTO_INSTALL to handle the version specified by use or a .terraform-version file not being installed

Lots of updates for Terraform's most popular provider. Changelog–

NOTES:

* provider: Region validation now automatically supports the new af-south-1 (Africa (Cape Town)) region. For AWS operations to work in the new region, the region must be explicitly enabled as outlined in the AWS Documentation. When the region is not enabled, the Terraform AWS Provider will return errors during credential validation (e.g. error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid) or AWS operations will throw their own errors (e.g. data.aws_availability_zones.current: Error fetching Availability Zones: AuthFailure: AWS was not able to validate the provided access credentials). (#12715)

* resource/aws_iam_user: The additional force_destroy behavior for handling signing certificates requires two additional IAM permissions (iam:ListSigningCertificates and iam:DeleteSigningCertificate). Restrictive IAM permissions for Terraform runs may require updates. (#10542)

* resource/aws_rds_cluster: Due to recent API support for Aurora MySQL 5.7 and PostgreSQL Global Clusters which implemented the engine mode as provisioned instead of the previous global for Aurora MySQL 5.6, the resource now requires the DescribeGlobalClusters API call. Restrictive IAM permissions may require updates. (#12867)

FEATURES:

* New Resource: aws_apigatewayv2_api_mapping (#9461)

* New Resource: aws_apigatewayv2_vpc_link (#12577)

ENHANCEMENTS:

* data_source/aws_acm_certificate: Add tags output (#11659)

* data-source/aws_cloudtrail_service_account: Support af-south-1 region (#12967)

* data-source/aws_elastic_beanstalk_hosted_zone: Support af-south-1 region (#12967)

* data-source/aws_elb_hosted_zone_id: Support af-south-1 region (#12967)

* data-source/aws_elb_service_account: Support af-south-1 region (#12967)

* data-source/aws_s3_bucket: Support af-south-1 region for hosted_zone_id attribute (#12967)

* provider: Support automatic region validation for af-south-1 (#12715)

* resource/aws_apigatewayv2_api: Add cors_configuration, credentials_arn, route_key and target attributes (#12452)

* resource/aws_appsync_graphql_api: Add log_config configuration block exclude_verbose_content argument (#12884)

* resource/aws_config_configuration_recorder: Prevent error during deletion operation when resource is missing (#12734)

* resource/aws_default_network_acl: Support import (#12924)

* resource/aws_lambda_alias: Suppress differences for equivalent function_name argument values of name versus ARN (#12902)

* resource/aws_network_acl_rule: Support import (#12921)

* resource/aws_route: Add plan-time validation for destination_cidr_block and destination_ipv6_cidr_block arguments (#12890)

* resource/aws_s3_bucket: Support af-south-1 region for hosted_zone_id attribute (#12967)

* resource/aws_service_discovery_private_dns_namespace: Support import (#12929)

* resource/aws_ssm_activation: Support import (#12933)

* resource/aws_ssm_maintenance_window_target: Add plan-time validation to resource_type argument (#11783)

* resource/aws_ssm_maintenance_window_target: Support import (#12935)

* resource/aws_volume_attachment: Support import (#12948)

* resource/aws_waf_ipset: Add plan-time validation for ip_set_descriptors configuration block arguments (#12775)

* resource/aws_waf_sql_injection_match_set: Support import (#11657)

* resource/aws_waf_xss_match_set: Add plan-time validation for xss_match_tuples configuration block arguments (#12777)

* resource/aws_wafregional_web_acl: Add plan-time validation to various arguments (#12793)

BUG FIXES:

* data-source/aws_launch_template: Prevent type error with network_interfaces associate_public_ip_address attribute (#12936)

* resource/aws_glue_security_configuration: Prevent empty string KMS Key ARN in S3 Encryption settings (#12898)

* resource/aws_iam_user: Ensure force_destroy argument removes signing certificates when enabled (#10542)

* resource/aws_rds_cluster: Prevent unexpected global_cluster_identifier differences and deletion error with aurora-mysql and aurora-postgresql Global Cluster members (#12867)

* resource/aws_route: Prevent not found after creation error with destination_ipv6_cidr_block set to ::0/0 (#12890)

Here's the thing about writing your own newsletter: you can put anything in in you want. In this case I am writing about a release I did of a provider I work on (with my team at CZI). Changelog:

* release version 0.12.0

* Bump github.com/stretchr/testify from 1.4.0 to 1.5.1 (#175)

* [fix] reduce spurious diffs to snowflake_view statement (#171)

* Bump github.com/pkg/errors from 0.8.1 to 0.9.1 (#176)

* unvendor our go dependencies (#170)

* [fix] formatting for pr template checklist (#172)

* [feature] future grants (#151)

First release in a long time for Atlantis. Good to see that things are still happening here. Changelog:

This release contains one much-awaited GitHub-only feature: the ability to hide previous plan comments with the --hide-prev-plan-comments flag. It also contains a host of other small features and flags.

Features

* GitHub: Add --hide-prev-plan-comments flag. When set, previous plan comments will be marked as outdated in GitHub's UI. This collapses them making a PR with lots of plan comments easier to read. (#994 by @goodspark)

* GitHub: Ignore draft PRs until they're changed to "ready for review". (#977 by @cket)

* Upgrade default Terraform version in Docker image to 0.12.24.

* Set as_user param when sending slack notifications so the message is decorated appropriately (#907 by @tmcevoy14)

* Add Git LFS support (#872 by @remilapeyre)

* Add --silence-vcs-status-no-plans flag that silences VCS commit status when autoplan finds no projects to plan. When set, Atlantis won't create any VCS statuses if there no projects to plan. (#959 by @cket)

* Add --disable-markdown-folding flag that disables folding for long plan/apply outputs. (#960 by @mhumeSF)

* Ignore casing when setting log levels, e.g. --log-level=INFO now works. (#976 by @jpreese)

* Azure DevOps: Add policy checking. (#984 by @jpreese)

* Upgrade boltdb to latest maintained version. (#992 by @amasover)

Bugfixes

* Azure DevOps: Prevent pull request updated events from triggering autoplan when the event was caused by a change in approvals. (Fixes #946 by @mcdafydd)

Backwards Incompatibilities / Notes:

* GitHub draft PRs are now ignored until they're marked "ready for review" and opened as regular PRs.

* If you're using the Atlantis Docker image and aren't setting the --default-tf-version flag then the default version of Terraform will now be 0.12.24. Simply set the above flag to your desired default version to avoid any issues.

BUG FIXES:

* r/tfe_workspace: Running a plan/apply when a workspace has been deleted outside of terraform no longer causes a panic. (#162)

I can't find the changelog for this release. So....