Skip to content
Snippets Groups Projects
Commit 445da85d authored by marcoemi.poleggi's avatar marcoemi.poleggi
Browse files

WIP: lab refactoring on OpenStack

parent a6985e36
Branches bedran_exercise-list
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 912 additions and 533 deletions
.gitattributes
+ 1
− 2
View file @ 445da85d
AWS/main.tf.advnc filter=git-crypt diff=git-crypt
OpenStack/main.tf.advnc filter=git-crypt diff=git-crypt
OpenStack/outputs.tf filter=git-crypt diff=git-crypt
SwitchEngines/main.tf.advnc filter=git-crypt diff=git-crypt
AWS/README.md 0 → 100644
+ 541
− 0
View file @ 445da85d
# Lab: Cloud provisioning/orchestration - Terraform and AWS
Lab template for a Cloud provisioning/orchestration exercise with Terraform
(TF) and AWS.
## Pedagogical objectives ##
* Become familiar with a Cloud provisioning/orchestration tool
* Provision Cloud resources in an automated fashion
## Tasks ##
In this lab you will perform a number of tasks and document your progress in a
lab report. Each task specifies one or more deliverables to be
produced. Collect all the deliverables in your lab report.
**N.B.** Some tasks require interacting with your local machine's OS: any
related commands are supposed to be run into a terminal with the following
conventions about the *command line prompt*:
* `#`: execution with super user's (root) privileges
* `$`: execution with normal user's privileges
* `lcl`: your local machine
* `ins`: your VM instance
TF CLI commands' output follow a diff-style convention of prefixing lines with
special marks to explain what is (or would be) going on:
* `+`: something is added
* `-`: something is removed/destroyed
* `-/+`: a resource is destroyed and recreated
* `~`: something is modified in place
### Task #1: install Terraform CLI and AWS CLI ###
**Goal:** install the Terraform CLI and AWS CLI on your local machine.
Please, refer to your OS documentation for the proper way to do so
1. [Terraform
CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli)
v1.1.4. Skip the TF "Quick start tutorial" (Docker).
1. [AWS CLI](https://aws.amazon.com/cli/) v1.22..
1. Create a new [AWS Access
Key](https://console.aws.amazon.com/iam/home?#/security_credentials).
1. Configure the AWS CLI:
``` shell
lcl$ aws configure
```
The following tasks are based on TF's tutorial [build an example
infrastructure on
AWS](https://learn.hashicorp.com/tutorials/terraform/aws-build).
### Task #2: configure TF for AWS ###
**Goal:** instruct TF to handle a single AWS EC2 instance.
<a name="AMI-query"></a>Find out which AMI to use for a recent Ubuntu server.
You can query the AMI Catalog via
a. [AWS dashboard](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#AMICatalog:), searching "Ubuntu", selecting "Quickstart AMIs" and filtering by "Free tier",
b. or with CLI, and get the 2 most recent AMIs (it will not list free-tier ones):
``` shell
lcl$ aws ec2 describe-images --region us-east-1 \
--filters "Name=root-device-type,Values=ebs" \
"Name=name,Values=*ubuntu-jammy-22.04-amd64-server*" \
"Name=architecture,Values=x86_64" \
--query "reverse(sort_by(Images, &ImageId))[:2].[ImageId]" --output text
ami-0f65ab0fd913bc7be
ami-0eb45246cba02871b
```
:bulb: In the following task(s), we use the first AMI found for the
placeholder `<your-EC2-AMI-ID>`.
Create a "sandbox" directory on your local machine `~/terraform/AWS/`. Inside
it, create a file called `main.tf` (written in HCL language), the
infrastructure *definition* file, with the following content:
``` hcl
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.27"
}
}
required_version = ">= 0.14.9"
}
provider "aws" {
profile = "default"
region = "us-east-1"
}
resource "aws_instance" "app_server" {
ami = "<your-EC2-AMI-ID>"
instance_type = "t2.micro"
tags = {
Name = "ExampleAppServerInstance"
}
}
```
Initialize your sandbox with:
``` shell
lcl$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.27"...
- Installing hashicorp/aws v3.27.0...
- Installed hashicorp/aws v3.27.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository so
that Terraform can guarantee to make the same selections by default when you run
"terraform init" in the future.
Terraform has been successfully initialized!
...
```
Have a look inside the newly created sub-directory
`~/terraform/AWS/.terraform/`, you'll find the required `aws` provider module
that has been downloaded during the initialization.
It's good practice to format and validate your configuration:
``` shell
lcl$ terraform fmt
main.tf
lcl$ terraform validate
Success! The configuration is valid.
```
### Task #3: deploy your AWS infrastructure ###
**Goal:** provision your AWS EC2 instance via TF.
Run the following command, confirm by typing "yes" and observe its output:
``` shell
lcl$ terraform apply
Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_instance.app_server will be created
+ resource "aws_instance" "app_server" {
+ ami = "ami-0fa37863afb290840"
+ arn = (known after apply)
...
+ instance_type = "t2.micro"
...
+ tags = {
+ "Name" = "ExampleAppServerInstance"
}
...
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_instance.app_server: Creating...
...
aws_instance.app_server: Creation complete after 38s [id=i-0155ba9d77ee0a854]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```
The information shown above before the prompt for action is the `execution
plan`: the `+` prefix mark things to add. Of course, many details are unknown
until the corresponding resource is instantiated.
:question: How many resources were created?
You can verify via the AWS dashboard that one EC2 instance has been created.
The state of the resources acted upon is locally stored. Let's see what's in
our sandbox:
``` shell
lcl$ tree -a
.
├── .terraform
│ └── providers
...
├── .terraform.lock.hcl
├── main.tf
└── terraform.tfstate
```
:question: What's in file `terraform.tfstate`? The answer comes from the
following commands:
``` shell
lcl$ terraform state list
aws_instance.app_server
```
It confirms that we're tracking one AWS instance. Let's dig a bit more:
``` shell
lcl$ terraform show
# aws_instance.app_server:
resource "aws_instance" "app_server" {
ami = "ami-0fa37863afb290840"
arn = "arn:aws:ec2:us-east-1:768034348959:instance/i-0155ba9d77ee0a854"
associate_public_ip_address = true
availability_zone = "us-east-1e"
...
id = "i-0155ba9d77ee0a854"
instance_initiated_shutdown_behavior = "stop"
instance_state = "running"
instance_type = "t2.micro"
...
private_dns = "ip-172-31-94-207.ec2.internal"
private_ip = "172.31.94.207"
public_dns = "ec2-3-94-184-169.compute-1.amazonaws.com"
public_ip = "3.94.184.169"
...
tags = {
"Name" = "ExampleAppServerInstance"
}
...
vpc_security_group_ids = [
"sg-0c420780b4f729d3e",
]
...
}
```
The above command output provides useful runtime (state) information, like the
instance IP's address. Indeed, there is a kind of *digital twin* stored inside
the file `terraform.tfstate`.
:question: Is that instance accessible via SSH? Give it a try. If not, why?
Now, stop the running instance (its ID is shown above ;-):
``` shell
lcl$ aws ec2 stop-instances --instance-ids i-0155ba9d77ee0a854
```
wait some seconds and test again:
``` shell
lcl$ terraform show | grep instance_state
instance_state = "running"
```
How come? We just discovered that TF read only the local status of a
resource. So, let's refresh it, and check again:
``` shell
lcl$ terraform refresh
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
lcl$ terraform show | grep instance_state
instance_state = "stopped"
```
Ah-ha!
Hold on a second: our TF plan does not specify the desired status of a
resource. What happens if we reapply the plan? Lets' try:
``` shell
lcl$ terraform apply -auto-approve
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
lcl$ terraform show | grep instance_state
instance_state = "stopped"
```
:warning: Apply was run in `-auto-approve` (non interactive) mode which
assumes "yes" to all questions. Use with care!
From the above commands' output we see that
* the local state is refreshed before doing anything,
* no changes are applied, and
* huh?... the resource is still stopped.
:bulb: Concerning the last point above, there is acutally no way with TF to
specify an AWS instances's desired *runtime* state ([a bug is
filed](https://github.com/hashicorp/terraform-provider-aws/issues/22)). Of
course, this issue can be solved with a configuration management tool.
### Task #4: change your infrastructure ###
**Goal:** modify the resource created before, and learn how to apply changes
to a Terraform project.
Restart your managed instance:
``` shell
lcl$ aws ec2 start-instances --instance-ids i-0155ba9d77ee0a854
```
Refresh TF's view of the world:
``` shell
lcl$ terraform refresh
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
lcl$ terraform show | grep instance_state
instance_state = "running"
```
Replace the resource's `ami` in `main.tf` with the second one found from the
[catalog query done above](#AMI-query) (or another one available with your
account). Before applying our new plan, let's see what TF thinks of it:
``` shell
lcl$ terraform plan -out=change-AMI.tfplan
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
Terraform will perform the following actions:
# aws_instance.app_server must be replaced
-/+ resource "aws_instance" "app_server" {
~ ami = "ami-0fa37863afb290840" -> "ami-0e2512bd9da751ea8" # forces replacement
...
Plan: 1 to add, 0 to change, 1 to destroy.
Saved the plan to: change-AMI.tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "change-AMI.tfplan"
```
:bulb: Remarks:
* The change we want to apply is destructive!
* We saved our plan. :question: Why? It is not really necessary in a simple
scenario like ours, however a more complex IaC workflow might require plan
artifacts to be programmatically validated and versioned.
Apply the saved plan:
``` shell
lcl$ terraform apply change-AMI.tfplan
aws_instance.app_server: Destroying... [id=i-0155ba9d77ee0a854]
...
aws_instance.app_server: Destruction complete after 33s
aws_instance.app_server: Creating...
...
aws_instance.app_server: Creation complete after 48s [id=i-0470db35749548101]
```
:bulb: What? Not asking for confirmation? Indeed, a saved plan is intended for
automated workflows! Moreover, a saved plan will come handy for rolling back a
broken infrastructure to the last working setup.
:question: What if we did not save our plan, and called a plain apply command?
Would the result be the same?
### Task #5: input variables ###
**Goal:** make a TF plan more flexible via input variables.
Our original plan has all its content hard-coded. Let's make it more flexible
with some input variables stored in a separate `variables.tf` file inside your
TF sandbox:
``` hcl
variable "instance_name" {
description = "Value of the Name tag for the EC2 instance"
type = string
default = "AnotherAppServerInstance"
}
```
Then modify the `main.tf` as follows:
``` hcl
resource "aws_instance" "app_server" {
ami = "ami-0e2512bd9da751ea8"
instance_type = "t2.micro"
tags = {
- Name = "ExampleAppServerInstance"
+ Name = var.instance_name
}
}
```
Apply the changes:
``` shell
lcl$ terraform apply -auto-approve
aws_instance.app_server: Refreshing state... [id=i-0470db35749548101]
...
~ update in-place
Terraform will perform the following actions:
# aws_instance.app_server will be updated in-place
~ resource "aws_instance" "app_server" {
id = "i-0470db35749548101"
~ tags = {
~ "Name" = "ExampleAppServerInstance" -> "AnotherAppServerInstance"
}
~ tags_all = {
~ "Name" = "ExampleAppServerInstance" -> "AnotherAppServerInstance"
}
...
}
Plan: 0 to add, 1 to change, 0 to destroy.
...
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
```
:bulb: **Exercise:** input variables can also be passed on the `apply` command
line. Find how to do that with another different value for the variable
`instance_name`. :question: Would this last change be persistent if we rerun a
plain `terraform apply`?
### Task #6: queries with outputs ###
**Goal:** use output values to query a provisioned infrastructure.
We have seen in the previous tasks that the infrastructure's status can be
displayed via `terraform show`: a rather clumsy way, if you just want to
extract some specific information. A better programmatic way of querying your
infrastructure makes use of "outputs". Put the following in a file called
`~/terraform/AWS/outputs.tf`:
``` hcl
output "instance_id" {
description = "ID of the EC2 instance"
value = aws_instance.app_server.id
}
output "instance_public_ip" {
description = "Public IP address of the EC2 instance"
value = aws_instance.app_server.public_ip
}
```
We have declared two outputs. As usual with TF, before querying their
associated values, we need to apply the changes:
``` shell
lcl$ terraform apply -auto-approve
...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
instance_id = "i-0470db35749548101"
instance_public_ip = "34.201.252.63"
instance_tags = tomap({
"Name" = "AnotherAppServerInstance"
})
```
So, we already got the needed information, but, within a workflow, it is more
practical to do something like:
``` shell
lcl$ terraform output -json instance_tags
{"Name":"AnotherAppServerInstance"}
```
:question: What if the `Name` tag is changed outside TF? Try it.
:question: What must be done to have TF respect that external change?
:question: How to revert an external change via TF?
### Task #7: SSH provisioning with Cloud-Init ###
**Goal:** use Cloud-Init to provision an SSH access to your TF-managed
instance.
Did you try to SSH into the instance you created via TF? It cannot work,
because we did not instructed TF about networks, users, keys or anything
else. **This is left entirely to you as an exercise.** You need to:
1. Destroy your infrastructure. There's a special TF command for that.
1. Create an SSH key pair `tf-cloud-init`.
1. Create a new cloud-init file
`~/terraform/AWS/scripts/add-ssh.yaml` with the following content:
``` yaml
#cloud-config
#^^^^^^^^^^^^
# DO NOT TOUCH the first line!
---
groups:
- ubuntu: [root, sys]
- terraform
users:
- default
- name: terraform
gecos: terraform
primary_group: terraform
groups: users, admin
ssh_authorized_keys:
- <your-SSH-pub-key-on-one-line>
```
:warning: **Mind that the first line of this file must spell exactly
`#cloud-config`**!
1. Modify the `main.tf` as follows:
1. add a `resource` block of type `"aws_security_group"` allowing ingress
ports 22 and 80 from any address, with any egress port open;
1. add a `data` block referencing the above authorization file as a
`"template_file"` type;
1. extend the `"aws_instance"` resource to:
1. associate a public IP address,
1. link the `data` block to a user data attribute;
1. add an output `"public_ip"`.
When done, *init* your new plan, *validate* and *apply* it. Verify that you
can SSH as user `terraform` (not the customary `ubuntu`) into your instance:
``` shell
lcl$ ssh terraform@$(terraform output -raw instance_public_ip) -i ../tf-cloud-init
```
OpenStack/main.tf deleted 120000 → 0
+ 0
− 1
View file @ a6985e36
main.tf.advnc
\ No newline at end of file
README.md
+ 7
− 530
View file @ 445da85d
# Lab: Cloud provisioning/orchestration - Terraform and AWS
# Lab: Cloud provisioning/orchestration - Terraform
Lab with Terraform and any Cloud
......@@ -10,534 +10,11 @@ Lab template for a Cloud provisioning/orchestration exercise with Terraform
* Become familiar with a Cloud provisioning/orchestration tool
* Provision Cloud resources in an automated fashion
## Tasks ##
## Cloud providers ##
In this lab you will perform a number of tasks and document your progress in a
lab report. Each task specifies one or more deliverables to be
produced. Collect all the deliverables in your lab report.
This lab has been tested with the following cloud providers:
**N.B.** Some tasks require interacting with your local machine's OS: any
related commands are supposed to be run into a terminal with the following
conventions about the *command line prompt*:
* `#`: execution with super user's (root) privileges
* `$`: execution with normal user's privileges
* `lcl`: your local machine
* `ins`: your VM instance
TF CLI commands' output follow a diff-style convention of prefixing lines with
special marks to explain what is (or would be) going on:
* `+`: something is added
* `-`: something is removed/destroyed
* `-/+`: a resource is destroyed and recreated
* `~`: something is modified in place
### Task #1: install Terraform CLI and AWS CLI ###
**Goal:** install the Terraform CLI and AWS CLI on your local machine.
Please, refer to your OS documentation for the proper way to do so
1. [Terraform
CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli)
v1.1.4. Skip the TF "Quick start tutorial" (Docker).
1. [AWS CLI](https://aws.amazon.com/cli/) v1.22..
1. Create a new [AWS Access
Key](https://console.aws.amazon.com/iam/home?#/security_credentials).
1. Configure the AWS CLI:
``` shell
lcl$ aws configure
```
The following tasks are based on TF's tutorial [build an example
infrastructure on
AWS](https://learn.hashicorp.com/tutorials/terraform/aws-build).
### Task #2: configure TF for AWS ###
**Goal:** instruct TF to handle a single AWS EC2 instance.
<a name="AMI-query"></a>Find out which AMI to use for a recent Ubuntu server.
You can query the AMI Catalog via
a. [AWS dashboard](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#AMICatalog:), searching "Ubuntu", selecting "Quickstart AMIs" and filtering by "Free tier",
b. or with CLI, and get the 2 most recent AMIs (it will not list free-tier ones):
``` shell
lcl$ aws ec2 describe-images --region us-east-1 \
--filters "Name=root-device-type,Values=ebs" \
"Name=name,Values=*ubuntu-jammy-22.04-amd64-server*" \
"Name=architecture,Values=x86_64" \
--query "reverse(sort_by(Images, &ImageId))[:2].[ImageId]" --output text
ami-0f65ab0fd913bc7be
ami-0eb45246cba02871b
```
:bulb: In the following task(s), we use the first AMI found for the
placeholder `<your-EC2-AMI-ID>`.
Create a "sandbox" directory on your local machine `~/terraform/AWS/`. Inside
it, create a file called `main.tf` (written in HCL language), the
infrastructure *definition* file, with the following content:
``` hcl
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.27"
}
}
required_version = ">= 0.14.9"
}
provider "aws" {
profile = "default"
region = "us-east-1"
}
resource "aws_instance" "app_server" {
ami = "<your-EC2-AMI-ID>"
instance_type = "t2.micro"
tags = {
Name = "ExampleAppServerInstance"
}
}
```
Initialize your sandbox with:
``` shell
lcl$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.27"...
- Installing hashicorp/aws v3.27.0...
- Installed hashicorp/aws v3.27.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository so
that Terraform can guarantee to make the same selections by default when you run
"terraform init" in the future.
Terraform has been successfully initialized!
...
```
Have a look inside the newly created sub-directory
`~/terraform/AWS/.terraform/`, you'll find the required `aws` provider module
that has been downloaded during the initialization.
It's good practice to format and validate your configuration:
``` shell
lcl$ terraform fmt
main.tf
lcl$ terraform validate
Success! The configuration is valid.
```
### Task #3: deploy your AWS infrastructure ###
**Goal:** provision your AWS EC2 instance via TF.
Run the following command, confirm by typing "yes" and observe it's output:
``` shell
lcl$ terraform apply
Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_instance.app_server will be created
+ resource "aws_instance" "app_server" {
+ ami = "ami-0fa37863afb290840"
+ arn = (known after apply)
...
+ instance_type = "t2.micro"
...
+ tags = {
+ "Name" = "ExampleAppServerInstance"
}
...
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_instance.app_server: Creating...
...
aws_instance.app_server: Creation complete after 38s [id=i-0155ba9d77ee0a854]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```
The information shown above before the prompt for action is the `execution
plan`: the `+` prefix mark things to add. Of course, many details are unknown
until the corresponding resource is instantiated.
:question: How many resources were created?
You can verify via the AWS dashboard that one EC2 instance has been created.
The state of the resources acted upon is locally stored. Let's see what's in
our sandbox:
``` shell
lcl$ tree -a
.
├── .terraform
│ └── providers
...
├── .terraform.lock.hcl
├── main.tf
└── terraform.tfstate
```
:question: What's in file `terraform.tfstate`? The answer comes from the
following commands:
``` shell
lcl$ terraform state list
aws_instance.app_server
```
It confirms that we're tracking one AWS instance. Let's dig a bit more:
``` shell
lcl$ terraform show
# aws_instance.app_server:
resource "aws_instance" "app_server" {
ami = "ami-0fa37863afb290840"
arn = "arn:aws:ec2:us-east-1:768034348959:instance/i-0155ba9d77ee0a854"
associate_public_ip_address = true
availability_zone = "us-east-1e"
...
id = "i-0155ba9d77ee0a854"
instance_initiated_shutdown_behavior = "stop"
instance_state = "running"
instance_type = "t2.micro"
...
private_dns = "ip-172-31-94-207.ec2.internal"
private_ip = "172.31.94.207"
public_dns = "ec2-3-94-184-169.compute-1.amazonaws.com"
public_ip = "3.94.184.169"
...
tags = {
"Name" = "ExampleAppServerInstance"
}
...
vpc_security_group_ids = [
"sg-0c420780b4f729d3e",
]
...
}
```
The above command output provides useful runtime (state) information, like the
instance IP's address. Indeed, there is a kind of *digital twin* stored inside
the file `terraform.tfstate`.
:question: Is that instance accessible via SSH? Give it a try. If not, why?
Now, stop the running instance (its ID is shown above ;-):
``` shell
lcl$ aws ec2 stop-instances --instance-ids i-0155ba9d77ee0a854
```
wait some seconds and test again:
``` shell
lcl$ terraform show | grep instance_state
instance_state = "running"
```
How come? We just discovered that TF read only the local status of a
resource. So, let's refresh it, and check again:
``` shell
lcl$ terraform refresh
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
lcl$ terraform show | grep instance_state
instance_state = "stopped"
```
Ah-ha!
Hold on a second: our TF plan does not specify the desired status of a
resource. What happens if we reapply the plan? Lets' try:
``` shell
lcl$ terraform apply -auto-approve
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
lcl$ terraform show | grep instance_state
instance_state = "stopped"
```
:warning: Apply was run in `-auto-approve` (non interactive) mode which
assumes "yes" to all questions. Use with care!
From the above commands' output we see that
* the local state is refreshed before doing anything,
* no changes are applied, and
* huh?... the resource is still stopped.
:bulb: Concerning the last point above, there is acutally no way with TF to
specify an AWS instances's desired *runtime* state ([a bug is
filed](https://github.com/hashicorp/terraform-provider-aws/issues/22)). Of
course, this issue can be solved with a configuration management tool.
### Task #4: change your infrastructure ###
**Goal:** modify the resource created before, and learn how to apply changes
to a Terraform project.
Restart your managed instance:
``` shell
lcl$ aws ec2 start-instances --instance-ids i-0155ba9d77ee0a854
```
Refresh TF's view of the world:
``` shell
lcl$ terraform refresh
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
lcl$ terraform show | grep instance_state
instance_state = "running"
```
Replace the resource's `ami` in `main.tf` with the second one found from the
[catalog query done above](#AMI-query) (or another one available with your
account). Before applying our new plan, let's see what TF thinks of it:
``` shell
lcl$ terraform plan -out=change-AMI.tfplan
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
Terraform will perform the following actions:
# aws_instance.app_server must be replaced
-/+ resource "aws_instance" "app_server" {
~ ami = "ami-0fa37863afb290840" -> "ami-0e2512bd9da751ea8" # forces replacement
...
Plan: 1 to add, 0 to change, 1 to destroy.
Saved the plan to: change-AMI.tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "change-AMI.tfplan"
```
:bulb: Remarks:
* The change we want to apply is destructive!
* We saved our plan. :question: Why? It is not really necessary in a simple
scenario like ours, however a more complex IaC workflow might require plan
artifacts to be programmatically validated and versioned.
Apply the saved plan:
``` shell
lcl$ terraform apply change-AMI.tfplan
aws_instance.app_server: Destroying... [id=i-0155ba9d77ee0a854]
...
aws_instance.app_server: Destruction complete after 33s
aws_instance.app_server: Creating...
...
aws_instance.app_server: Creation complete after 48s [id=i-0470db35749548101]
```
:bulb: What? Not asking for confirmation? Indeed, a saved plan is intended for
automated workflows! Moreover, a saved plan will come handy for rolling back a
broken infrastructure to the last working setup.
:question: What if we did not save our plan, and called a plain apply command?
Would the result be the same?
### Task #5: input variables ###
**Goal:** make a TF plan more flexible via input variables.
Our original plan has all its content hard-coded. Let's make it more flexible
with some input variables stored in a separate `variables.tf` file inside your
TF sandbox:
``` hcl
variable "instance_name" {
description = "Value of the Name tag for the EC2 instance"
type = string
default = "AnotherAppServerInstance"
}
```
Then modify the `main.tf` as follows:
``` hcl
resource "aws_instance" "app_server" {
ami = "ami-0e2512bd9da751ea8"
instance_type = "t2.micro"
tags = {
- Name = "ExampleAppServerInstance"
+ Name = var.instance_name
}
}
```
Apply the changes:
``` shell
lcl$ terraform apply -auto-approve
aws_instance.app_server: Refreshing state... [id=i-0470db35749548101]
...
~ update in-place
Terraform will perform the following actions:
# aws_instance.app_server will be updated in-place
~ resource "aws_instance" "app_server" {
id = "i-0470db35749548101"
~ tags = {
~ "Name" = "ExampleAppServerInstance" -> "AnotherAppServerInstance"
}
~ tags_all = {
~ "Name" = "ExampleAppServerInstance" -> "AnotherAppServerInstance"
}
...
}
Plan: 0 to add, 1 to change, 0 to destroy.
...
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
```
:bulb: **Exercise:** input variables can also be passed on the `apply` command
line. Find how to do that with another different value for the variable
`instance_name`. :question: Would this last change be persistent if we rerun a
plain `terraform apply`?
### Task #6: queries with outputs ###
**Goal:** use output values to query a provisioned infrastructure.
We have seen in the previous tasks that the infrastructure's status can be
displayed via `terraform show`: a rather clumsy way, if you just want to
extract some specific information. A better programmatic way of querying your
infrastructure makes use of "outputs". Put the following in a file called
`~/terraform/AWS/outputs.tf`:
``` hcl
output "instance_id" {
description = "ID of the EC2 instance"
value = aws_instance.app_server.id
}
output "instance_public_ip" {
description = "Public IP address of the EC2 instance"
value = aws_instance.app_server.public_ip
}
```
We have declared two outputs. As usual with TF, before querying their
associated values, we need to apply the changes:
``` shell
lcl$ terraform apply -auto-approve
...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
instance_id = "i-0470db35749548101"
instance_public_ip = "34.201.252.63"
instance_tags = tomap({
"Name" = "AnotherAppServerInstance"
})
```
So, we already got the needed information, but, within a workflow, it is more
practical to do something like:
``` shell
lcl$ terraform output -json instance_tags
{"Name":"AnotherAppServerInstance"}
```
:question: What if the `Name` tag is changed outside TF? Try it.
:question: What must be done to have TF respect that external change?
:question: How to revert an external change via TF?
### Task #7: SSH provisioning with Cloud-Init ###
**Goal:** use Cloud-Init to provision an SSH access to your TF-managed
instance.
Did you try to SSH into the instance you created via TF? It cannot work,
because we did not instructed TF about networks, users, keys or anything
else. **This is left entirely to you as an exercise.** You need to:
1. Destroy your infrastructure. There's a special TF command for that.
1. Create an SSH key pair `tf-cloud-init`.
1. Create a new cloud-init file
`~/terraform/AWS/scripts/add-ssh.yaml` with the following content:
``` yaml
#cloud-config
#^^^^^^^^^^^^
# DO NOT TOUCH the first line!
---
groups:
- ubuntu: [root, sys]
- terraform
users:
- default
- name: terraform
gecos: terraform
primary_group: terraform
groups: users, admin
ssh_authorized_keys:
- <your-SSH-pub-key-on-one-line>
```
:warning: **Mind that the first line of this file must spell exactly
`#cloud-config`**!
1. Modify the `main.tf` as follows:
1. add a `resource` block of type `"aws_security_group"` allowing ingress
ports 22 and 80 from any address, with any egress port open;
1. add a `data` block referencing the above authorization file as a
`"template_file"` type;
1. extend the `"aws_instance"` resource to:
1. associate a public IP address,
1. link the `data` block to a user data attribute;
1. add an output `"public_ip"`.
When done, *init* your new plan, *validate* and *apply* it. Verify that you
can SSH as user `terraform` (not the customary `ubuntu`) into your instance:
``` shell
lcl$ ssh terraform@$(terraform output -raw instance_public_ip) -i ../tf-cloud-init
```
* [Amazon AWS](AWS/README.md "README-AWS.md")
* [Switch Engines (OpenStack)](SwitchEngines/README.md
"README-SwitchEngines.md") :bulb: **this is the reference provider for
2022/2023**
README-OpenStack.md → SwitchEngines/README.md
+ 310
− 0
View file @ 445da85d
## Lab: Cloud provisioning/orchestration - Terraform and OpenStack
## Lab: Cloud provisioning/orchestration - Terraform and SwitchEngines (OpenStack)
Lab template for a Cloud provisioning/orchestration exercise with Terraform
(TF) and OpenStack/SwitchEngines.
(TF) and SwitchEngines (OpenStack).
## Pedagogical objectives ##
......@@ -39,19 +39,29 @@ Please, refer to your OS documentation for the proper way to do so
1. [Terraform
CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli)
v1.1.4. Skip the TF "Quick start tutorial" (Docker).
1. [OpenStack CLI](https://docs.openstack.org/newton/user-guide/common/cli-install-openstack-command-line-clients.html) v5.7.0.
1. [OpenStack
CLI](https://docs.openstack.org/newton/user-guide/common/cli-install-openstack-command-line-clients.html)
version >= 5.7.0. It is recommended to install your original distribution
package named like `python-openstackclient`.
1. Grab or create a new project in your OpenStack account. The placeholder
used in this exercise is `<your-project-name>` with value `Cloud-MA-IaC`.
1. Create a new [OpenStack Access
1. On the OpenStack Horizon dashboard,
1. create new [Application
Credentials](https://engines.switch.ch/horizon/identity/application_credentials/)
and save as `~/.config/openstack/clouds.yaml`. With SwitchEngine, the
cloud name to use in this is `engines`. :warning: App credentials might
not work for some commands; [API
and save it as `~/.config/openstack/clouds.yaml`. With SwitchEngines, the
cloud name to use for this is `engines`. :warning: App credentials might
not work for some commands. A template is also available in repo file
[`conf/clouds.yaml.api_cred`](conf/clouds.yaml.api_cred). Once created,
1. alternatively, you can use your [API
credentials](https://engines.switch.ch/horizon/project/api_access/clouds.yaml)
with explicit project name/ID setup might be better.
1. Verify that your credentials are OK:
with explicit project name/ID -- you'll have to add your `password` from
your profile page's ["Credentials"
tab](https://engines.admin.switch.ch/users/profile). A template is
available in repo file
[`conf/clouds.yaml.app_cred`](conf/clouds.yaml.app_cred);
1. Verify that your credentials are OK (:warning: it will reveal secrets!):
``` shell
lcl$ $ openstack --os-cloud=Cloud-MA-IaC [application] credential list
lcl$ $ openstack --os-cloud=engines [application] credential list
```
### Task #2: configure TF for OpenStack ###
......@@ -61,11 +71,11 @@ Please, refer to your OS documentation for the proper way to do so
<a name="image-query"></a>Find out the smallest image to use for a Debian server:
``` shell
lcl$ openstack --os-cloud=Cloud-MA-IaC image list --limit=10 --public --status=active --sort-column=Size -c ID -c Name -c Size --long
lcl$ openstack --os-cloud=engines image list --public --status=active --sort-column=Size -c ID -c Name -c Size --long
+--------------------------------------+-------------------------------------+-------------+
| ID | Name | Size |
+--------------------------------------+-------------------------------------+-------------+
| c6596c8a-b074-4a72-9c8c-411d1cb11113 | Debian Buster 10 (SWITCHengines) | 1537409024 |
| 8674f1a5-f7d9-4975-af0b-d2e9e33c9152 | Debian Buster 10 (SWITCHengines) | 1537409024 |
...
```
......@@ -75,7 +85,7 @@ SwitchEngines this is 1.5GB.
Find out the smallest instance *flavor* that acommodates our Debian image.
``` shell
lcl$ openstack --os-cloud=engines flavor list --sort-column=RAM --sort-column=Disk --min-disk=5 --min-ram=1024 --limit=1 --public
lcl$ openstack --os-cloud=Cloud-MA-IaC flavor list --sort-column=RAM --sort-column=Disk --min-disk=5 --min-ram=1024 --limit=1 --public
+----+----------+------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+----------+------+------+-----------+-------+-----------+
......@@ -85,23 +95,66 @@ Find out the smallest instance *flavor* that acommodates our Debian image.
:bulb: Our flavor will be `m1.small` for the placeholder `<your-flavor>`.
Create a "sandbox" directory on your local machine `~/terraform/OpenStack/`. Inside
it, create a file called `main.tf` (written in HCL language), the
infrastructure *definition* file, with the appropriate content to handle:
1. an instance,
1. a security group to allow SSH access,
1. a floating IP in the public pool associated to your instance.
Create a "sandbox" directory on your local machine
`~/terraform/OpenStack/`. Inside it, create a file called `main.tf` (written
in HCL language), the infrastructure *definition* file, with the following
content (replace all `<...>` tokens with actual values):
Also, prepare an `~/terraform/OpenStack/outputs.tf` file with the appropriate
contents to get the instance's public IP address (via `instance_public_ip`).
``` hcl
terraform {
required_version = ">= 0.14.9"
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
version = "~> 1.48.0"
}
}
}
provider "openstack" {
auth_url = "https://keystone.cloud.switch.ch:5000/v3"
project_domain_id = "<your-project-ID>"
application_credential_id = "<your-app-cred-ID>"
application_credential_secret = "<your-app-cred-secret>"
region = "ZH"
}
resource "openstack_compute_instance_v2" "app_server" {
name = "TF-managed"
image_id = "<your-image-ID>"
flavor_name = "<your-flavor>"
}
```
Initialize your sandbox with:
``` shell
lcl$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding terraform-provider-openstack/openstack versions matching "~> 1.48.0"...
- Installing terraform-provider-openstack/openstack v1.48.0...
- Installed terraform-provider-openstack/openstack v1.48.0 (self-signed, key ID 4F80527A391BEFD2)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
...
```
Format and validate your configuration:
Have a look inside the newly created sub-directory
`~/terraform/OpenStack/.terraform/`, you'll find the required `openstack`
provider module that has been downloaded during the initialization.
It's good practice to format and validate your configuration:
``` shell
lcl$ terraform fmt
......@@ -110,6 +163,140 @@ lcl$ terraform validate
Success! The configuration is valid.
```
### Task #3: deploy your OpenStack infrastructure ###
**Goal:** provision your OpenStack instance via TF.
So far, our plan has been validated but not executed yet. You can either
prepare your plan for later execution or apply it directly. Let's see what would be done:
``` shell
lcl$ terraform plan
Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# openstack_compute_instance_v2.app_server will be created
+ resource "openstack_compute_instance_v2" "app_server" {
+ access_ip_v4 = (known after apply)
...
+ flavor_name = "m1.small"
+ force_delete = false
+ id = (known after apply)
+ image_id = "8674f1a5-f7d9-4975-af0b-d2e9e33c9152"
+ image_name = (known after apply)
+ name = "TF-managed"
...
+ network {
+ access_network = (known after apply)
+ fixed_ip_v4 = (known after apply)
...
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
──────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now
```
In the command output above, notice the `+` prefix that marks things to
*add*. Of course, many details are unknown until the corresponding resource is
instantiated.
:question: How many resources would be created?
If it looks OK, then run the following command, confirm by typing "yes":
``` shell
lcl$ terraform apply
...
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
openstack_compute_instance_v2.app_server: Creating...
openstack_compute_instance_v2.app_server: Still creating... [10s elapsed]
openstack_compute_instance_v2.app_server: Creation complete after 18s [id=f70aef53-51f9-4d12-a4a9-fe9a6cc5a58f]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```
You can verify via the SwitchEngine dashboard that one instance named
"TF-managed" has indeed been created.
The state of the resources acted upon is locally stored. Let's see what's in
our sandbox:
``` shell
lcl$ tree -a
.
├── .terraform
│ └── providers
...
├── .terraform.lock.hcl
├── main.tf
└── terraform.tfstate
```
:question: What's in file `terraform.tfstate`? The answer comes from the
following commands:
``` shell
lcl$ terraform state list
openstack_compute_instance_v2.app_server
```
It confirms that we're tracking one OpenStack instance. Let's dig a bit more:
``` shell
lcl$ terraform show
# openstack_compute_instance_v2.app_server:
resource "openstack_compute_instance_v2" "app_server" {
access_ip_v4 = "10.0.8.116"
...
flavor_name = "m1.small"
...
id = "f70aef53-51f9-4d12-a4a9-fe9a6cc5a58f"
image_id = "8674f1a5-f7d9-4975-af0b-d2e9e33c9152"
image_name = "Debian Buster 10 (SWITCHengines)"
name = "TF-managed"
power_state = "active"
region = "ZH"
security_groups = [
"default",
]
stop_before_destroy = false
network {
...
}
}
```
The above command output provides useful runtime (state) information, like the
instance IP's address. Indeed, there is a kind of *digital twin* stored inside
the file `terraform.tfstate`.
:question: Is that instance accessible from the internet? Give it a try. If
not, why?
Indeed, the instance doesn't have a (floating) *public* IP address, so we need
to associate one:
@@@@@ OLD part below@@@@
Also, prepare a `~/terraform/OpenStack/outputs.tf` file with the appropriate
contents to get the instance's public IP address (via `instance_public_ip`).
Then apply!
``` shell
......
SwitchEngines/main.tf 0 → 120000
+ 1
− 0
View file @ 445da85d
main.tf.basic
\ No newline at end of file
OpenStack/main.tf.advnc → SwitchEngines/main.tf.advnc
+ 0
− 0
View file @ 445da85d
File moved
SwitchEngines/main.tf.basic 0 → 100644
+ 52
− 0
View file @ 445da85d
terraform {
required_version = ">= 0.14.9"
required_providers {
openstack = {
source = "terraform-provider-openstack/openstack"
# version = "~> 1.35.0"
version = "~> 1.48.0"
}
}
}
# Configure the OpenStack Provider
provider "openstack" {
auth_url = "https://keystone.cloud.switch.ch:5000/v3"
project_domain_id = "<your-project-ID>"
application_credential_id = "<your-app-cred-ID>"
application_credential_secret = "<your-app-cred-secret>"
region = "LS"
}
resource "openstack_compute_instance_v2" "app_server" {
name = "TF-managed"
image_id = "<your-image-ID>"
flavor_name = "<your-flavor>"
key_pair = "TF_lab"
security_groups = ["default", "secgroup_tf"]
}
resource "openstack_networking_secgroup_v2" "secgroup_tf" {
name = "secgroup_tf"
}
resource "openstack_networking_secgroup_rule_v2" "secgroup_rule_1" {
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "0.0.0.0/0"
security_group_id = "${openstack_networking_secgroup_v2.secgroup_tf.id}"
}
resource "openstack_networking_floatingip_v2" "fip_1" {
pool = "public"
}
resource "openstack_compute_floatingip_associate_v2" "fip_1" {
floating_ip = "${openstack_networking_floatingip_v2.fip_1.address}"
instance_id = "${openstack_compute_instance_v2.app_server.id}"
# to avoid getting "Resource not found"
wait_until_associated = true
}
OpenStack/outputs.tf → SwitchEngines/outputs.tf
+ 0
− 0
View file @ 445da85d
File moved
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment