Tag: terraform backend

Terraform: error configuring S3 Backend: no valid credential sources for S3 Backend found.

admin

Terraform backend configuration for remote storage may be quite challenging if the correct parameters are not passed.

We can get multiple errors while executing the terraform init command, depending upon the configuration arguments we miss or based upon the permissions defined for our AWS profile or AWS User/Role.

Terraform Backend State Configuration

Terraform Backend Configuration Document

Here, we will focus on valid credential errors.

Error: Error configuring S3 Backend: no valid credential sources for S3 Backend found.

Terraform backend configuration code: The backend was defined as below, we specified the bucket, key, and region.

terraform {
  required_providers {
    aws = {
        source = "hashicorp/aws"
    }
  }
  backend "s3" {
    bucket = "terraform-backend"
    key = "misc-infra-terraform-state"
    region = "us-east-1"
  }
}

When we ran the terraform init command for the above configuration and got the below error,

terraform s3 backend error


E:\terraform>terraform init

Initializing the backend...
╷
│ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
│
│ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
│ for more information about providing credentials.
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│       For verbose messaging see aws.Config.CredentialsChainVerboseErrors

In our case, the fix was pretty easy, the above terraform backend block had missing AWS credentials, we just passed the credential details, and it worked like charm!

Updated S3 Backend configuration

terraform {
  required_providers {
    aws = {
        source = "hashicorp/aws"
    }
  }
  backend "s3" {
    bucket = "terraform-backend"
    key = "misc-infra-terraform-state"
    region = "us-east-1"
    profile = "myaws_profile"
  }
}

Dynamic Terraform State Configuration

admin Leave a comment

Terraform doesn’t support passing dynamic variables for state files or backend blocks.

For example, let us consider we have a backend defined like the below,

Now, we have a scenario where we do not want to pass the values in the backend block in a static format, instead, we want the value to be dynamic.

We will rewrite the code and create two variables to pass the bucket name and bucket object (key) and pass these values in the backend block as variables instead of static values.

The above terraform configuration when we run the terraform init command will throw errors, as variables not allowed in the backend block.

E:\Projects\terraform>terraform init -migrate-state

Initializing the backend...
Backend configuration changed!

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now check for existing state in the backends.

╷
│ Error: Variables not allowed
│
│   on main.tf line 9, in terraform:
│    9:     bucket = var.backend_bucket #"s3-backend-demo"
│
│ Variables may not be used here.
╵

╷
│ Error: Variables not allowed
│
│   on main.tf line 10, in terraform:
│   10:     key = var.backend_key #"backend"
│
│ Variables may not be used here.
╵
Error while using dynamic variables for backend block

To resolve the issue, terraform allows us to pass the backend values dynamically through backend-config arguments via command prompt.

In your backend configuration block, remove the details which you want to pass dynamically. Like in our example we are using s3 as the backend, passing bucket name and object (key) values.

We will remove those attributes from the backend configuration block and instead we will pass them through backend-config arguments.

Passing backend configuration arguments values through a command prompt,

terraform init -backend-config="bucket=s3-backend-demo" -backend-config="key=backend"

In case we have an existing state file and we want to update the new backend arguments, you will need to run the init command with -migrate-state arguments.

terraform init -migrate-state -backend-config="bucket=s3-backend-demo" -backend-config="key=backend"