Remote State

Local state vs. remote state#

As stated at the start of this chapter, the default storage for a Terraform state is in a local file called terraform.tfstate. This is fine when working on a local project or for small proof of concepts, but it does not practically scale beyond that. If you want to work on a Terraform project on more than one machine or with more than one person, then the local state won’t cut it anymore.

The reason local state storage isn’t practical is that (as we have learned in this chapter) Terraform uses its state file to store a record of what it has created. If you run Terraform from one machine and create a number of resources then run the same project from another machine, Terraform will try to create all of those resources again on the second machine as the second machine will not have a state file. This means that the second run will most likely fail since most resources cannot exist more than once. Even if it does succeed, you will still end up with two of each resource.

Remote state#

To get around this issue, the state needs to be stored in a remote location. Thankfully, Terraform has different built-in storage backends for the state. These built-in remote backends include AWS S3, AzureRM, and Consul plus many more. Consult the Hashicorp Terraform website for a full list. Let’s walk through an example of setting up a simple project and use AWS S3 to store our state rather than the default terraform.tfstate file.

Project example#

• Start the example by logging in to the AWS console and create an S3 bucket. You can use any name you want; I have used kevholditch-terraform-state.

  You can click through and create the bucket with the default permissions since, by default, the bucket will only be accessed privately.

• Next, we’ll create a file called main.tf:

There is not much to the code; we are simply setting up the AWS provider and creating a queue.

• To specify that Terraform should use the S3 state backend, we need to create another file called state.tf:

📝Note: Please do remember the bucket first and replacing the bucket name in the state.tf file, with the bucket name you used to create the bucket.

This block configures the state backend. The s3 in quotes after the word backend tells Terraform that we want to use the AWS S3 backend instead of the default local file backend that we have been using up until now. The bucket parameter is the name of the bucket we want to use. You will need to change this to match the bucket that you created. Next is the key we want to use, which is essentially what Terraform will call the state file in S3. Lastly, there is the region where we created the S3 bucket.

Output#

After you click the RUN button, the terraform init command will be executed. You should notice this message:

Terraform tells you this time that you have configured the backend s3. This means that the state will be stored in the AWS S3 bucket and not in your local file.

Go ahead and run the terraform apply command to create the AWS sqs queue. Log into the AWS console, browse to the S3 bucket, and you will see that there will be a file there called myproject.state. This was the value we gave Terraform for the key parameter in the state.tf file.

Project configuration#

By configuring the project in this way, we can now collaborate on a project with multiple people effectively. We could check this project into source control and anyone working on the project would have access to the AWS S3 bucket where the state is stored.

Working with multiple people on a single Terraform project#

If you want to work on a project with multiple people, you need to ensure that the state remote backend that you choose supports locking (as the AWS S3 state backend does). When the remote state backend supports locking, Terraform will lock the state while it is running to prevent anyone else from changing the state at the same time. You will not see a message that Terraform is locking the state. Terraform will just silently do this automatically.

This is an important feature to have since if the state is not locked, multiple people could overwrite it and it could become corrupted. Another tip is that if you are using AWS S3, turn on bucket versioning. This will give you every version of your state file. Although you should avoid manually changing the state file unless you are in an extreme situation, having bucket versioning gives you an extra safety net.

Conclusion#

One last note before we end this chapter: we put the remote backend state configuration in a file called state.tf, which is just a convention used by the community. Terraform actually does not care where you put that block of code. You can technically put it in any file ending .tf including inside the main.tf. It is normal practice, however, to use a file called state.tf so that it is easy to find the remote state configuration.