I recently implemented a series of AWS Batch jobs for a client. While most of these were for implementing, well, batch jobs, in the form of reporting functions, I decided to give a go to deploying Cloud Custodian using the same framework, as it basically involved creating an additional CloudFormation nested stack, building Custodian policies and baking them into a container to deploy through Batch. Getting everyhting up and going was a fair bit of work, so I wanted to encapsulate my learnings into something else others could use.
There are several drastic HCL improvements forthcoming in version 0.12 of Terraform, but after an issue I encountered this week while creating some infrastructure with an 0.11.x version of the runtime, I wanted to cover the issue, how to remedy it in versions < 0.12, and talk about how (I believe) the issue will be remedied thanks to some of the 0.12 improvements. Basically, this type of issue will manifest itself as an error during the plan phase with this form of error message: module.
Beginning with the 0.10.x version tree of Terraform, HashiCorp decided to decouple providers from the main Terraform runtime (see the 0.10.0 CHANGELOG for details). For a lot of users, this is a seamless win as Terraform will pull down whatever providers it deduces that it needs to execute your build assuming you have network connectivity to reach their servers and grab the files. This flexible architecture allows new providers to be released, bugfixes and features to be introduced in existing providers without requiring a version increment and download of a monolithic Terraform package.
Remote state is (IMO) one of Terraform’s most powerful and unsung features. It’s also a feature that I notice a lot of first-time users (and unfortunately sometimes people who’ve even been using it for some time) tend to gloss over and ignore. For the first-time user, the light bulb usually goes off for the need of a solution that remote state provides when a scenario like this comes to pass:
Background In my last post, I talked about creating lean providers for maximum flexibility. As I closed out the post, I mentioned the potential peril of performing operations against the wrong account by virtue of having the AWS_PROFILE variable set for a profile matching an account other than the one you’re intending to work with (and believe me, if you work with more than a handful of accounts, this is a very easy mistake to make).
In the last 15 months, I have become a daily user of Terraform. In that time, several things have happened in my “relationship” to the tool: I have used it to build out some complex (as well as some not-so-complex) deployments (and learned a lot of lessons the hard way) I have been particularly impressed with the growth in the tool’s capability and HashiCorp’s commitment to pushing the tool forward by expanding its capability set as well as improving core functionality.