Skip to content

42-cloud/Cloud

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

90 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Overview

Automated deployment of Wordpress related services using Terraform and Ansible

CI/CD Pipeline

CI/CD Pipeline

Tasks list

network and security

  • only 80 and 443 should be accessible (replace 8080 and 8443)
  • usage of TLS
  • network isolation of DB from the internet

roles

  • Add observability role
  • Wordpress role
    • Readme for wordpress role
    • Molecule tests
    • PHP My Admin configuration
    • handler to reload angie
    • handler to reload wordpress apache

resilience

  • dynamic DNS
  • auto-restart if server is rebooted with data preserved

scalability

  • parallel deploy to multiple servers

flexibility

  • provider-agnostic configuration
  • can create various users with admin rights on EC2, app admin rights on wordpress

Setup

Prerequisites

All commands are compatible with Ubuntu

# Ensure local bin (or .local/bin) directory exists
mkdir -p $HOME/bin

# add taskfile
sh -c "$(curl -sSL https://taskfile.dev/install.sh)" -- -d -b $HOME/bin

# add terraform
TERRAFORM_VERSION="1.11.0"
curl -sSL "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" -o /tmp/terraform.zip
unzip -q /tmp/terraform.zip -d $HOME/bin/
rm /tmp/terraform.zip

# add ansible
python3 -m venv $HOME/.ansible_venv
source $HOME/.ansible_venv/bin/activate
pip install --upgrade pip
pip install ansible-core checkov argcomplete
ln -sf $HOME/.ansible_venv/bin/ansible $HOME/bin/ansible
ln -sf $HOME/.ansible_venv/bin/ansible-playbook $HOME/bin/ansible-playbook
deactivate

# add AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install -i $HOME/.local/aws-cli -b $HOME/bin

# GET key details for AWS IAM profile
# AWS Management Console > IAM > Users > Create access key

# Configure AWS CLI
aws configure
# fill details from IAM

Tasks

# create terraform/terraform.tfvars
task tfvars

# download providers
task init

# check terraform project correctness
task plan

# deploy infrastructure
task apply

# deploy configuration
task ansible:play

Usage

Access the app on https://cloud1.duckdns.org

Stack

Ubuntu as base OS

package installation method

  • DEB822 (from RFC 822) is the new APT norm to declare package repositories on Debian and Ubuntu. Successor to .list.
    • it relies on key: value pairs for suites, components, architecture
    • every repo has its gpg key

Automation and Orchestration

Go-Task

Task runner and build tool : an alternative to Makefile

  • uses YAML
  • handles cache more efficiently than Makefile : fingerprinting vs date of last modification
  • multiplatform

Bash script

tfvars.sh to check variables and generate terraform.tfvars

Containerization

Wolfi OS

A secure Linux distribution

  • used as a base layer for Angie
  • quickly updated in case of CVE
  • compatible with packages compiled for glibc

Melange

Declarative APK builder : compiles packages from source code

Apko

Image assembler : assembles packages into a distroless image

  • secure : images don't have shell, reducing attack surface
  • idempotent : images are identical
  • lightweight

APKO generates:

  • SPDX with all components and licences for each package
  • SBOM (software bill of materials) which can be used to audit supply chain

UID and GID are 65532 : conventional ID for non-root

Infrastructure as Code

We define a basic network architecture :

  • VPC (virtual private cloud) with subnet to isolate the cloud environment
  • Internet Gateway to bridge VPC with public internet
  • Security group acting as a firewall

Best practices

  • Restrict SSH to specific IP

Terraform

An infrastructure as code tool that defines cloud resources

Architecture proposal

monoserver

  • aws_eip Elastic IP

=> We choose a monoserver architecture for the sake of simplicity

multi-tier (possible evolution)

  • load balancer ALB with public IP
  • private subnet EC2 Wordpress with private IP
  • private subnet RDS Database with private IP

NB : Ansible gets access to remote machine

  • with a bastion instance in a public subnet
  • or with AWS Systems Manager Session Manager

Which AWS resources are we declaring ?

name description terraform AWS
AMI Pre-configured Amazon Machine Image providing the base Ubuntu OS template. data.aws_ami AWS AMI Docs
EC2 Instance Amazon Elastic Compute Cloud is a virtual compute server hosting the application, containers, and services. aws_instance AWS EC2 Docs
KMS Key Centralized key managenent aws_kms_key AWS KMS
EBS Block Store Persistent cloud storage volume attached to the instance acting as its root hard drive (gp3). root_block_device AWS EBS Docs
Elastic IP (EIP) Static, persistent public IPv4 address assigned to ensure a fixed endpoint. aws_eip AWS EIP Docs
VPC Virtual Private Cloud : Isolated virtual private network space providing network boundary control. aws_vpc AWS VPC Docs
Subnet A segmented logical partition inside the VPC network to group resources. aws_subnet AWS Subnet Docs
Internet Gateway VPC component enabling bidirectional communication between the network and the public internet. aws_internet_gateway AWS IGW Docs
Route Table Set of routing rules determining where network traffic from the subnets is directed. aws_route_table AWS Route Table Docs
Security Group Virtual stateful firewall controlling permitted inbound and outbound traffic. aws_security_group AWS SG Docs
VPC Flow Logs Feature that captures IP traffic information flowing to and from network interfaces. aws_flow_log AWS Flow Logs Docs
CloudWatch Log Group Centralized log management and storage service repository for system monitoring data. aws_cloudwatch_log_group AWS CloudWatch Logs Docs
IAM Role Identity with specific permission policies determining what AWS resources can do. aws_iam_role AWS IAM Roles Docs

Security rules

Some Checkov lints that we fixed:

code category should should not
CKV_AWS_8 security encrypt storage volumes by default -
CKV2_AWS_11 auditability capture in/out IP trafic from VPC with a aws_flow_log instance -
CKV_AWS_12 networking default security group should explicitely block by default -
CKV_AWS_23 observability description field for rule of security group undocumented rules
CKV2_AWS_41 security instance should use an IAM profile should not store AWS access keys
CKV_AWS_79 security metadata should be secured against SSRF with a token => IMDSv2 no INDSv1 which use classic HTTP GET
CKV_AWS_130 networking assign private IP by default. Necessity of public IP can be mitigated in different ways (see architecture) no public IP by default
CKV_AWS_135 performance instance should have a dedicated bandwidth to EBS volumes do not share disk traffic with standard network traffic
CKV_AWS_158 security encrypt logs -
CKV_AWS_382 networking outgoing trafic rules should be restricted to necessary protocols no -1 (all protocols) on 0.0.0.0
CKV2_ANSIBLE_1 use HTTPS in module calls (ex uri) -

Some others that we ignored:

code category should should not tradeoff
CKV_AWS_126 observability should rely on frequent checks (every 1 mn) billed by cloud provider

Ansible

An agentless configuration management tool

Variables

  • 3 levels of variables
    • group_vars
    • vars (within a role)
    • default (within a role) : allow to reuse group_vars as they have a lower priority than them, contrary to vars
  • Variables can be overloaded for tests in molecule.yaml

caveats

  • best practice (ansible-lint) want us to use role-prefixed variables. We need however to take care not to multiplicate symbols referring to a same value (global > vars > defaults > test in molecule.yaml or in individual test files) or to set up fallback values in multiple places.

TDD with molecules

  • a test_sequence can have many steps, among which main ones:
    • create
    • converge
    • verify
    • destroy

adapting tests to dockerized test environment

  • molecule runs tests in docker container. For docker role, we have distinct tasks whether the environment is prod or test:
    • in test env, we are within a container with no systemd. Therefore, we use ansible.builtin.shell to look for / launch dockerd process
    • prod relies on ansible.builtin.service module
  • similarly, test assertion rely on command docker info

Roles

Role Responsibilities
bootstrap OS preparation, SSH config, UFW config, create directories for data persistence with appropriate permissions
docker generic role to install daemon and docker compose
wordpress centralized role for stack management : would be too difficult to manage 2 different roles for wordpress and db

Ansible Vault

ansible-vault encrypt_string --vault-password-file .vault_pass_cloudone --name <name> <password>

Security

Checkov

Static analysis for Infra As Code

  • compares code against security policies

Services

Angie

Reverse proxy. Fork of nginx with extended features

Wordpress

An open source CMS

NB : There is no official module for Wordpress management. Partly because cli evolves too fast and should be compatible with many php versions

MariaDB

An open sourve fork of MySQL

PHPMyAdmin

An administration tool for DB

Network

Duck DNS

  • we should reestabish mapping every time the infrastructure is redeployed (AWS generates a new public Elastic IP)

Resources

Url Kind Notes
Terraform doc πŸ“”
Ansible doc πŸ“”
Taskfile doc πŸ“”
Chainguard doc πŸ“” for Melange and Apko
Automating IT with Ansible πŸ“˜
Infra as Code using Terraform πŸ“˜
Stephane Robert πŸ“˜ Excellent tutorials
Installing WP with Ansible πŸ“˜ Tutorial using MySQL setup, PHP-FPM, Nginx, wp-cli. ⚠️ Not the same containerized approach yet useful for wp cli steps

Resource type

  • πŸ“” official doc
  • πŸ“˜ course
  • πŸ—’οΈ cheatsheet, synthesis
  • 🌐 web, article
  • πŸ“½οΈ video

AI Usage

  • guide setup with a prompt asking to proofcheck our approach and suggest alternatives with pros and cons -> we remain in charge of choosing the next step
  • fix and improve terraform variables collection script
  • debugging help
  • PR review

About

Infrastructure as Code for a self-hosted WordPress stack: Terraform, Ansible, Docker with distroless OCI images built via apko/melange with automated CVE scanning.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors