Introduction

In the world of Infrastructure as Code (IaC), Terraform stands out as a powerful tool for defining, provisioning, and managing cloud resources. However, one critical aspect often overlooked is state management. The Terraform state file keeps track of the resources you’ve created, their current state, and their relationships. In this comprehensive guide, we’ll explore different approaches to handling Terraform state files, focusing on remote storage options like Amazon S3 and Terraform Cloud.

Local State

When you run terraform apply, Terraform stores the state of your infrastructure in a local state file named terraform.tfstate. This file contains a JSON representation of the resources you’ve created, their current state, and their relationships. By default, Terraform stores the state file in the working directory, but you can specify a different location using the -state flag. While local state files are convenient for getting started with Terraform, they have several limitations:

Remote State

Remote state management is a best practice for Terraform, especially in a team environment. It allows multiple team members to work on the same infrastructure without conflicts. Terraform supports several remote state backends, including Amazon S3, Azure Blob Storage, Google Cloud Storage, and Terraform Cloud. In this section, we’ll focus on Amazon S3 and Terraform Cloud as remote state backends.

Amazon S3

Amazon S3 is a popular choice for storing Terraform state files. It provides a durable, scalable, and secure object storage service that can be used to store and retrieve any amount of data at any time. To use Amazon S3 as a remote state backend, you need to create an S3 bucket and a DynamoDB table to store the state lock. The state lock prevents concurrent operations on the state file, ensuring that only one user can modify the state at a time. Here’s an example of how to configure Terraform to use Amazon S3 as a remote state backend:

                    terraform {
                        backend "s3" {
                            bucket = "my-terraform-state"
                            key    = "terraform.tfstate"
                            region = "us-east-1"
                            dynamodb_table = "terraform-state-lock"
                        }
                    }  
                    

With this configuration, Terraform will store the state file in the "my-terraform-state" S3 bucket, using the "terraform.tfstate" key. It will also use the "terraform-state-lock" DynamoDB table to manage the state lock. This ensures that only one user can modify the state at a time, preventing conflicts and data corruption.

Amazon S3 - Multi-account configuration

When you have a multi-account sctructure in aws, where you deploy the same terraform code to various aws accounts, you will need a way to tell terraform to use the correct state file for each account, and also if you use just one IAM user, and create a separate role for each aws account (recommended), you will need to include the role in your backend configuration and use the -backend-config parameter in the terraform init command. So you actually have to run the terraform init command for each account, and then you can run the terraform apply command for each account. Here is an example of a .conf file for the backend configuration:

                        bucket = "my-terraform-state"
                        key    = "terraform.tfstate"
                        region = "us-east-1"
                        dynamodb_table = "terraform-state-lock"
                        role_arn = "arn:aws:iam::123456789012:role/terraform"
                        

And here is an example of the terraform init command:

                        terraform init -backend-config="backend.conf"
                        

With this configuration, Terraform will store the state file in the "my-terraform-state" S3 bucket, using the "terraform.tfstate" key. It will also use the "terraform-state-lock" DynamoDB table to manage the state lock.

                    region = "us-east-1"
                    environment = "dev"
                    instance_type = "t2.micro"
                    

And here is an example of the terraform apply command:

                    terraform apply -var-file="dev.tfvars"
                    

Using this approach you can use the same terraform repo to deploy the same infrastructure to different aws accounts, you just need to use the -backend-config parameter in the terraform init command, and also use the -var-file parameter in the terraform apply command.

Terraform Cloud

Terraform Cloud is a managed service that provides remote state storage, collaboration, and governance features for Terraform. It allows you to store your state files securely in the cloud, and provides a web-based interface for managing your infrastructure. To use Terraform Cloud as a remote state backend, you need to create an account on the Terraform Cloud website and configure your Terraform project to use it. Here’s an example of how to configure Terraform to use Terraform Cloud as a remote state backend:

                    terraform {
                        backend "remote" {
                            organization = "my-organization"
                            workspaces {
                                name = "my-workspace"
                            }
                        }
                    }
                    

With this configuration, Terraform will store the state file in Terraform Cloud, using the "my-organization" organization and the "my-workspace" workspace. This allows you to manage your infrastructure using the Terraform Cloud web interface, and provides collaboration and governance features for your team.

Azure Blob Storage and Google Cloud Storage

In addition to Amazon S3 and Terraform Cloud, Terraform supports other remote state backends like Azure Blob Storage and Google Cloud Storage. These services provide similar features to Amazon S3, allowing you to store your state files securely in the cloud and manage your infrastructure using a web-based interface. To use Azure Blob Storage or Google Cloud Storage as a remote state backend, you need to create an account on the respective cloud provider’s website and configure your Terraform project to use it.

Conclusion

Remote state management is a best practice for Terraform, especially in a team environment. It allows multiple team members to work on the same infrastructure without conflicts, and provides collaboration and governance features for your team. In this guide, we’ve explored different approaches to handling Terraform state files, focusing on remote storage options like Amazon S3 and Terraform Cloud. By following these best practices, you can ensure that your Terraform infrastructure is secure, scalable, and easy to manage.