This is a kind of documentation or walks thought of my work, which can be called a DevSecOps diary.

Production Grade Terraform Project Structure.

I have read the “Terraform: Up and Running” book and do other research on production-grade terraform project structure. However, end up with a project structure that works for me. Grunt Work(gruntwork.io) released two repositories a few months ago

https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example

https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example

They have noted down some points on Monorepo vs. polyrepo etc. Anyone can read that to make a choice for him/her context.

I do end up with one shared module repository and infrastructure modules repositories for each project.

A couple of approaches I took that did not include these basic repository posted by Grunt Work. Here are some modification tips that will helps to make it production grade project.

I have started with a basic example creating an amazon EKS Kubernetes cluster with terraform.

To get started clone the repository from GitHub

git clone https://github.com/nahidupa/k8s-eks-with-terraform.git
git checkout k8s-eks-with-terraform-basic

eks-basic-repo

Refactoring to shared modules

Refactoring steps no make a separate shared module repository is easy, copy “eks-cluster” and  “eks-security-groups” folders to the new repository.

https://github.com/nahidupa/terraform-shared-modules.git

  1. Copy the tf-modules folders to terraform-shared-modules repo.

  2. Delete tf-modules folder.

  3. Change the following code.

Before terragrunt.hcl

terraform {
  source = "../../../tf-modules/eks-cluster"
}

After

terraform {
 source = "git::https://github.com/nahidupa/terraform-shared-modules.git//modules/eks-cluster"
}

Check everything all right or not by the following command

terragrunt plan-all
  1. You should tag your shared repo and referred a specific tag in the repository that using the shared module because the shared module will be keep changed frequently and as that is also pointed by other projects that may break your current repo.

Let’s tag the share module repository with v0.0.1

git tag v0.0.1
git push origin --tags
  1. Now need to change all source in terragrunt.hcl
terraform {
  source = "git::https://github.com/nahidupa/terraform-shared-modules.git//modules/eks-security-groups?ref=v0.0.1"
}

This is help this repository not to become broken by other changes of the shared repository.

The final result is pushed to git branch “shared_modules”

git clone https://github.com/nahidupa/k8s-eks-with-terraform.git
git checkout shared_modules

eks-shared_modules-repo

shared_modules-repo

How to handle dependency outputs.

If you notice in this example “eks-cluster” module is dependent on “eks-security-groups”. Using dependency output made easy by terragrunt . See the following example.

dependencies {
  paths = ["../eks-security-groups"]
}

dependency "eks-security-groups" {
  config_path = "../eks-security-groups"
}

It is possible to use one module outputs to another module using the following way.

dependency.eks-security-groups.outputs.additional_security_group_ids

additional_security_group_ids = [dependency.eks-security-groups.outputs.additional_security_group_ids]

Test Code with plan-all with mock input.

This will be very common in any project that will end up with multiple modules depending on each other, at the very starting point it is a good idea to check the code is working without creating any infrastructure in AWS. However, if terragrunt apply does not apply to any dependent module terragrunt will give an error as it cannot find the dependency outputs.

Here mocks input’s come handy.

dependency "eks-security-groups" {
  config_path = "../eks-security-groups"
  mock_outputs = {
    "additional_security_group_ids" = ""
  }
  skip_outputs = false
}

In the dependency block, you can set skip_outputs as false and put a mock value that is an output of original dependency. By using this trick’s you can run terragrunt apply-all to check the code without creating any real infrastructure.

Inputs that are not sensitive data, but network information you may not want to push in a public repository.

Gruntworks proposed repository we see root “terragrunt.hcl” they add an allowed_account_ids that input taken from local account_id. If you want to push your code any public repository you may not want to ask your account id or sometime you may also not want to disclose vpc_id, subnets, etc.

Original

# Generate an AWS provider block
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
  provider "aws" {
    region = "${local.aws_region}"

    # Only these AWS Account IDs may be operated on by this template
     allowed_account_ids = ["${local.account_id}"]
  }
  EOF
  }

Adjust

# Generate an AWS provider block
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
  provider "aws" {
    region = "${local.aws_region}"

    # Only these AWS Account IDs may be operated on by this template
    # allowed_account_ids = ["${local.account_id}"]
  }
  EOF
  }

You can just block allowed_account_ids and for the sensitive data, you can take that from a external “sensitive-vars.yaml” file that can be ignored from the git repo.

.gitignore
sensitive-vars.yaml

yaml file can be read with following code.

  sensitive_vars = yamldecode(file("${find_in_parent_folders("sensitive-vars.yaml")}"))

Used of yaml file inputs.

  vpc_id = local.sensitive_vars.vpc_id

locals {
  # Automatically load account-level variables
  account_vars = read_terragrunt_config(find_in_parent_folders("account.hcl"))

  # Automatically load environment-level variables
  environment_vars = read_terragrunt_config(find_in_parent_folders("env.hcl"))
  
  # Automatically load region-level variables
  region_vars = read_terragrunt_config(find_in_parent_folders("region.hcl"))

  # Extract out common variables for reuse
  env = local.environment_vars.locals.environment
  account_name = local.account_vars.locals.account_name
  account_id   = local.account_vars.locals.aws_account_id
  aws_region   = local.region_vars.locals.aws_region

  sensitive_vars = yamldecode(file("${find_in_parent_folders("sensitive-vars.yaml")}"))
}

Using both local folder path and git repo with tag for shared module.

I had mentioned using git repo as shared modules source, However, when you are coding and testing it is also possible to take to modules from direct absolute or relative path. So if you changing frequently in a shared module while coding not need to push again and again in git repo just used local path.

terraform {
 source = "git::https://github.com/nahidupa/terraform-shared-modules.git//modules/eks-cluster?ref=v0.0.1"
}

terraform {
 source = ".../modules/eks-cluster"
}

terraform {
 source = "/root/modules/eks-cluster"
}

Clean the cache.

While toggle local path or source and git source or some reason you want to clear your local catch following code snippet is handy.

clear-cache.sh

find . -type d -name ".terragrunt-cache" -prune -exec rm -rf {} \;