Followers

Terraform State lock file -S3 bucket

  1. How Store Terraform state to file remotely on S3? Before we implement the Dynamo DB locking feature first we need to store the Terrafor...

 

Before we implement the Dynamo DB locking feature first we need to store the Terraform state file(terraform.tfstate) file remotely on AWS S3 bucket.

I am gonna take a very simple example in which we are going to provision an AWS EC2 machine and store the terraform state file remotely.

Let's start by creating main.tf and we will add the following resource blocks to it -

  1. Provider Block
  2. AWS Instance resource block(aws_instance) for EC2
  3. Backend S3 block
  4. Execute terraform script
  5. Verify the remote state file

As we are working on the AWS environment so we will be using AWS provider. So add the following block to your main. tf -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}
BASH


After adding the provider block let's add the aws_instance resource block in which we are going to set up the EC2 the machine of type t2.micro -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}
BASH

(*Note - I have already created an S3 bucket with the name jhooq-terraform-s3-bucket, so make sure to create one for you as well.)

Now after adding the provider and aws_instance block let's add the backend S3 block to my main.tf -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

resource "aws_dynamodb_table" "state_locking" {
  hash_key = "LockID"
  name     = "dynamodb-state-locking"
  attribute {
    name = "LockID"
    type = "S"
  }
  billing_mode = "PAY_PER_REQUEST"
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

terraform {
    backend "s3" {
        bucket = "test-terraform-s3-bucket"
        key    = "test/terraform/remote/s3/terraform.tfstate"
        region     = "eu-central-1"
    }
}
BASH


Now for implementing the state locking we need to create a DynamoDB table.

  1. Goto your AWS management console and search for DynamoDB onto the search bar.

Terraform dynamoDB

  1. Click on the DynamoDB

  2. From the left navigation panel click on Tables

Terraform dynamoDB table creation

  1. Click on Create Table

Terraform create table

  1. Enter the Table name - "dynamodb-state-locking" and Partition Key - "LockID"

dynamoDB table name and Partition key LockID

  1. Click on Create Table and you can verify the table after the creation

dynamoDB verify the table



After creating the DynamoDB table in the previous step, let's add the reference of DynamoDB table name (dynamodb-state-locking) to backend S3 sate.

terraform {
    backend "s3" {
        bucket = "test-terraform-s3-bucket"
        key    = "test/terraform/remote/s3/terraform.tfstate"
        region     = "eu-central-1"
   dynamodb_table  = "dynamodb-state-locking"
    }
}
BASH

Your final Terraform main.tf should look like this -

provider "aws" {
   region     = "eu-central-1"
  
}

resource "aws_dynamodb_table" "state_locking" {
  hash_key = "LockID"
  name     = "dynamodb-state-locking"
  attribute {
    name = "LockID"
    type = "S"
  }
  billing_mode = "PAY_PER_REQUEST"
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

terraform {
    backend "s3" {
        bucket = "jhooq-terraform-s3-bucket"
        key    = "jhooq/terraform/remote/s3/terraform.tfstate"
        region     = "eu-central-1"
       dynamodb_table  = "dynamodb-state-locking"
    }
} 
BASH


  1. The first command we are gonna run is terraform init

terraform init for state locking

  1. Now the run the terraform plan command

terraform plan for state locking

  1. Finally, the terraform apply command

terraform apply for state locking

terraform apply for state locking

  1. Verify the DynamoDB LockID by going into the AWS management console -

verify the DynamoDB locking for remote state

(*Note- To simulate the locking scenario I am creating another main.tf with the same configuration. I would encourage you to create one main.tf and save the file in some other directory)

To test terraform state locking I will provision one more EC2 machine using the same Terraform state file (jhooq/terraform/remote/s3/terraform.tfstate) stored in my S3 bucket along with the same DynamoDB table (dynamodb-state-locking).

Keep in mind we are still using following two components from previous main.tf

  1. S3 Bucket - jhooq-terraform-s3-bucket
  2. DynamoDB Table - dynamodb-state-locking
  3. Terraform state file - jhooq/terraform/remote/s3/terraform.tfstate

Here is my another main.tf file -

provider "aws" {
   region     = "eu-central-1"
   access_key = var.access_key
   secret_key = var.secret_key
}

resource "aws_instance" "ec2_example" {
    ami = "ami-0767046d1677be5a0"
    instance_type = "t2.micro"
    tags = {
      Name = "EC2 Instance with remote state"
    }
}

terraform {
  backend "s3" {
    bucket = "test-terraform-s3-bucket"
    key    = "test/terraform/remote/s3/terraform.tfstate"
    encrypt        = true
    region         = "eu-central-1"
    dynamodb_table = "dynamodb-state-locking"
  }
}
BASH

On the left side of the screen, you will see the first terraform file(main.tf) which I have created in the Step-1 and on the right-hand side, you will see the terraform file(main.tf) from the Step-4.

**How did I simulate the remote state locking scenario? **

  1. I have executed terraform apply on terraform file present on the right-hand side but did not let it finish. While executing terraform apply command I did not type yes when it asks for Do you want to perform these actions? so basically terraform apply command is still running and holding a lock on the remote state file.

  2. At the same time I executed the terraform apply on main.tf from Step-4 which you can see on the right side of the screenshot. Since the second main.tf file also referring the same remote state as well as same dynamo db table it will throw en error - Error: Error acquiring the state lock Error message: ConditionalCheckFailedException: The conditional request failed Lock Info ID: 8f014160-8894-868e-529d-0f16e42af405


Error: Error acquiring the state lock Error message: ConditionalCheckFailedException: The conditional request failed Lock Info

Terraform state file locking is one of the most valuable features offered by terraform for managing the Terraform state file. If you are using the AWS S3 and Dynamo DB then terraform state locking can improve your state management and save your time from unforeseen issues.

COMMENTS

Name

Ansible,6,AWS,1,Azure DevOps,1,Containerization with docker,2,DevOps,2,Docker Quiz,1,Docker Swarm,1,DockerCompose,1,ELK,2,git,2,Jira,1,Kubernetes,1,Kubernetes Quiz,5,SAST DAST Security Testing,1,SDLC Quiz,3,SonarQube,3,Splunk,2,vagrant kubernetes,1,YAML Basics,1,
ltr
item
DevOpsWorld: Terraform State lock file -S3 bucket
Terraform State lock file -S3 bucket
https://jhooq.com/wp-content/uploads/terraform/terraform-state-lock/dynamodb-search.webp
DevOpsWorld
https://www.devopsworld.co.in/2022/09/terraform-state-lock-file-s3-bucket.html
https://www.devopsworld.co.in/
https://www.devopsworld.co.in/
https://www.devopsworld.co.in/2022/09/terraform-state-lock-file-s3-bucket.html
true
5997357714110665304
UTF-8
Loaded All Posts Not found any posts VIEW ALL Readmore Reply Cancel reply Delete By Home PAGES POSTS View All RECOMMENDED FOR YOU LABEL ARCHIVE SEARCH ALL POSTS Not found any post match with your request Back Home Sunday Monday Tuesday Wednesday Thursday Friday Saturday Sun Mon Tue Wed Thu Fri Sat January February March April May June July August September October November December Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec just now 1 minute ago $$1$$ minutes ago 1 hour ago $$1$$ hours ago Yesterday $$1$$ days ago $$1$$ weeks ago more than 5 weeks ago Followers Follow THIS PREMIUM CONTENT IS LOCKED STEP 1: Share to a social network STEP 2: Click the link on your social network Copy All Code Select All Code All codes were copied to your clipboard Can not copy the codes / texts, please press [CTRL]+[C] (or CMD+C with Mac) to copy Table of Content