Terraform State lock file -S3 bucket

  1. How Store Terraform state to file remotely on S3? Before we implement the Dynamo DB locking feature first we need to store the Terrafor...

 

1. How Store Terraform state to file remotely on S3?

Before we implement the Dynamo DB locking feature first we need to store the Terraform state file(terraform.tfstate) file remotely on AWS S3 bucket.

I am gonna take a very simple example in which we are going to provision an AWS EC2 machine and store the terraform state file remotely.

Let's start by creating main.tf and we will add the following resource blocks to it -

1. Provider Block
2. AWS Instance resource block(aws_instance) for EC2
3. Backend S3 block
4. Execute terraform script
5. Verify the remote state file

1.1 Provider Block

As we are working on the AWS environment so we will be using AWS provider. So add the following block to your main. tf -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

BASH

1.2 AWS Instance resource block(aws_instance) for EC2

After adding the provider block let's add the aws_instance resource block in which we are going to set up the EC2 the machine of type t2.micro -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

BASH

1.3 Backend S3 block

(*Note - I have already created an S3 bucket with the name jhooq-terraform-s3-bucket, so make sure to create one for you as well.)

Now after adding the provider and aws_instance block let's add the backend S3 block to my main.tf -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

resource "aws_dynamodb_table" "state_locking" {
  hash_key = "LockID"
  name     = "dynamodb-state-locking"
  attribute {
    name = "LockID"
    type = "S"
  }
  billing_mode = "PAY_PER_REQUEST"
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

terraform {
    backend "s3" {
        bucket = "test-terraform-s3-bucket"
        key    = "test/terraform/remote/s3/terraform.tfstate"
        region     = "eu-central-1"
    }
}

BASH

2. Create DynamoDB table on AWS

Now for implementing the state locking we need to create a DynamoDB table.

1. Goto your AWS management console and search for DynamoDB onto the search bar.

Terraform dynamoDB

2. Click on the DynamoDB

3. From the left navigation panel click on Tables

Terraform dynamoDB table creation

4. Click on Create Table

Terraform create table

5. Enter the Table name - "dynamodb-state-locking" and Partition Key - "LockID"

dynamoDB table name and Partition key LockID

6. Click on Create Table and you can verify the table after the creation

dynamoDB verify the table

3. Add AWS DynamoDB Table reference to Backend S3 remote state?

After creating the DynamoDB table in the previous step, let's add the reference of DynamoDB table name (dynamodb-state-locking) to backend S3 sate.

terraform {
    backend "s3" {
        bucket = "test-terraform-s3-bucket"
        key    = "test/terraform/remote/s3/terraform.tfstate"
        region     = "eu-central-1"
   dynamodb_table  = "dynamodb-state-locking"
    }
}

BASH

Your final Terraform main.tf should look like this -

provider "aws" {
   region     = "eu-central-1"
  
}

resource "aws_dynamodb_table" "state_locking" {
  hash_key = "LockID"
  name     = "dynamodb-state-locking"
  attribute {
    name = "LockID"
    type = "S"
  }
  billing_mode = "PAY_PER_REQUEST"
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

terraform {
    backend "s3" {
        bucket = "jhooq-terraform-s3-bucket"
        key    = "jhooq/terraform/remote/s3/terraform.tfstate"
        region     = "eu-central-1"
       dynamodb_table  = "dynamodb-state-locking"
    }
} 

BASH

3.1 Apply the above terraform configuration with DynamoDB table

1. The first command we are gonna run is terraform init

terraform init for state locking

2. Now the run the terraform plan command

terraform plan for state locking

3. Finally, the terraform apply command

terraform apply for state locking

terraform apply for state locking

4. Verify the DynamoDB LockID by going into the AWS management console -

verify the DynamoDB locking for remote state

4. Spin one more EC2 instance with the same Terraform state file

(*Note- To simulate the locking scenario I am creating another main.tf with the same configuration. I would encourage you to create one main.tf and save the file in some other directory)

To test terraform state locking I will provision one more EC2 machine using the same Terraform state file (jhooq/terraform/remote/s3/terraform.tfstate) stored in my S3 bucket along with the same DynamoDB table (dynamodb-state-locking).

Keep in mind we are still using following two components from previous main.tf

1. S3 Bucket - jhooq-terraform-s3-bucket
2. DynamoDB Table - dynamodb-state-locking
3. Terraform state file - jhooq/terraform/remote/s3/terraform.tfstate

Here is my another main.tf file -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

terraform {
  backend "s3" {
    bucket = "test-terraform-s3-bucket"
    key    = "test/terraform/remote/s3/terraform.tfstate"
    encrypt        = true
    region         = "eu-central-1"
    dynamodb_table = "dynamodb-state-locking"
  }
}

BASH

4.1 Run both the terraform files at the same time to simulate the Locking on terraforming state file

On the left side of the screen, you will see the first terraform file(main.tf) which I have created in the Step-1 and on the right-hand side, you will see the terraform file(main.tf) from the Step-4.

**How did I simulate the remote state locking scenario? **

1. I have executed terraform apply on terraform file present on the right-hand side but did not let it finish. While executing terraform apply command I did not type yes when it asks for Do you want to perform these actions? so basically terraform apply command is still running and holding a lock on the remote state file.

2. At the same time I executed the terraform apply on main.tf from Step-4 which you can see on the right side of the screenshot. Since the second main.tf file also referring the same remote state as well as same dynamo db table it will throw en error - Error: Error acquiring the state lock Error message: ConditionalCheckFailedException: The conditional request failed Lock Info ID: 8f014160-8894-868e-529d-0f16e42af405

Error: Error acquiring the state lock Error message: ConditionalCheckFailedException: The conditional request failed Lock Info

5. Conclusion

Terraform state file locking is one of the most valuable features offered by terraform for managing the Terraform state file. If you are using the AWS S3 and Dynamo DB then terraform state locking can improve your state management and save your time from unforeseen issues.