Terraform setup to spin up basics in AWS
Description of files and directories within this directory.
| Path | Description |
|---|---|
charts |
Helm charts deployed along with the infrastructure |
env |
Holding variable files for environments (test, prod etc) |
modules |
Holding the local modules created |
remote-state-create |
Configuration of S3 bucket with state lock provided by DynamoDB |
.terraform.lock.hcl |
Terraform version lock file |
backend.tf |
Configuration of Terraform state backend |
eks.tf |
Configuration of EKS cluster |
modules.tf |
Configuration of applications to deploy |
network.tf |
Configuration of network resources to deploy |
policies.tf |
Configuring the necessary policies to use |
providers.tf |
Configuration of AWS Terraform providers |
variables.tf |
Declaration of top level project variables |
versions.tf |
Declaration of required minimum Terraform version to use |
Amazon Web Services (AWS) IAM Admin user with both:
- Programmatic access
- AWS Management Console access
- This will only need to be performed one time, as you can reuse the same profile later.
Create a profile using the aws command line, filling in values for AWS Access Key ID, AWS Secret Access Key, Default region name and Default output format.
- Note that by omitting the
--profile, the profile will be created asdefault, and you will not need to specify--profilefor subsequent commands. However, you can set the AWS_PROFILE environment variable to a certain profile, and all commands will be executed as that profile:export AWS_PROFILE="<profile_name>". - Also note that the default AWS output format is JSON. If omitted, JSON will be used. The credentials file later used in this example relies on JSON output.
aws configure [--profile <profile_name>]- This will only be done one time, as the
terraformuser credentials must be shared with the users creating the Terraform scripts, as well as the CI/CD.
Create an IAM group, and the IAM user that we are using in order to run the Terraform scripts. Add the user to the group.
aws iam create-group --group-name IaC
aws iam create-user --user-name terraform
aws iam add-user-to-group --user-name terraform --group-name IaCVerify that the user has been added to the group by running the following command:
aws iam get-group --group-name IaCProvide the group with an IAM managed policy with sufficient rights. Terraform will create and destroy items all over the place, and will require extensive rights. In this example, the AdministratorAccess right is used:
aws iam attach-group-policy --group-name IaC --policy-arn $(aws iam list-policies --query 'Policies[?PolicyName==`AdministratorAccess`].{ARN:Arn}' --output text)Verify that the policy has been attached by executing the command:
aws iam list-attached-group-policies --group-name IaCCreate the terraform user access key, and pipe the result to a credentials file:
aws iam create-access-key --user-name terraform > credentials.jsonMove the .json file somewhere safe and set the following environment variables:
- Note that in this example, the region eu-west-1 is used. You may, of course, use a different region, but keep in mind to use the same region everywhere.
export AWS_ACCESS_KEY_ID=$(cat /path/to/credentials/file.json | jq -r '.AccessKey.AccessKeyId')
export AWS_SECRET_ACCESS_KEY=$(cat /path/to/credentials/file.json | jq -r '.AccessKey.SecretAccessKey')
export AWS_DEFAULT_REGION="eu-west-1"
export TERRAFORM_ENVIRONMENT=testSee the
remote-state-createfolder for how to set it up. Please remember to export your environment variables for the backend:
export TERRAFORM_STATE_AWS_BUCKET=<Your bucket name>
export TERRAFORM_STATE_DYNAMODB_TABLE=<Your table name>- Navigate back to the main Terraform script in order to set the remote state location.
terraform init -reconfigure -backend-config="dynamodb_table=${TERRAFORM_STATE_DYNAMODB_TABLE}" -backend-config="bucket=${TERRAFORM_STATE_AWS_BUCKET}"
terraform workspace new ${TERRAFORM_ENVIRONMENT} #This only needs to be done the first timeEnsure that you are using the correct workspace when running plan (and apply) by setting the workspace to the Environment Variable. This command only needs to be set when changing between environments or Terraform scripts.
terraform workspace select ${TERRAFORM_ENVIRONMENT}terraform plan -var-file=env/${TERRAFORM_ENVIRONMENT}.tfvarsThe idea behind using an -out file, is that the apply will not need to run the the plan section one more time. It can, instead, read and execute upon the saved .tfplan file. If you create a .tfplan file, please use the Terraform Apply with Out file in order to apply changes.
terraform plan -var-file=env/${TERRAFORM_ENVIRONMENT}.tfvars -out ${TERRAFORM_ENVIRONMENT}.tfplanterraform apply -var-file=env/${TERRAFORM_ENVIRONMENT}.tfvarsOnly valid if you have created a
.tfplanfile for it to apply. You will NOT be asked to accept the changes when using this.
terraform apply ${TERRAFORM_ENVIRONMENT}.tfplanRun the following command in order to fetch the credentials for the new Kubernetes cluster:
aws eks --region region update-kubeconfig --name cluster_nameYou can now list the running pods with the kubectl command:
kubectl get pods --all-namespaces