Skip to content
Snippets Groups Projects
Commit c8f35b43 authored by marcoemi.poleggi's avatar marcoemi.poleggi
Browse files

OpenStack: added TP-style readme (no solution) and bits

parent 75976e23
No related branches found
No related tags found
No related merge requests found
AWS/main.tf.advnc filter=git-crypt diff=git-crypt
OpenStack/main.tf.advnc filter=git-crypt diff=git-crypt
......@@ -9,10 +9,10 @@ output "instance_public_ip" {
}
output "public_ip" {
description = "Public IP address of the EC2 instance"
value = aws_instance.app_server.public_ip
}
output "instance_tags" {
description = "Public IP address of the EC2 instance"
value = aws_instance.app_server.tags_all
}
# Project-agnostic -- API credentials
clouds:
engines:
auth:
auth_url: https://keystone.cloud.switch.ch:5000/v3
username: "<your-user-name>"
password: "<your-password"
user_domain_name: "Default"
interface: "public"
identity_api_version: 3
# Project-specific -- application credentials.
clouds:
Cloud-MA-IaC:
auth:
auth_url: https://keystone.cloud.switch.ch:5000/v3
application_credential_id: "<your-app-cred-ID>"
application_credential_secret: "<your-app-cred-secret>"
user_domain_name: Default
project_domain_name: Default
interface: "public"
identity_api_version: 3
auth_type: "v3applicationcredential"
main.tf.advnc
\ No newline at end of file
File added
output "instance_id" {
description = "ID of the instance"
value = openstack_compute_instance_v2.app_server.id
}
output "instance_public_ip" {
description = "Public IP address of instance"
value = openstack_networking_floatingip_v2.fip_1.address
}
## Lab: Cloud provisioning/orchestration - Terraform and AWS
Lab with Terraform and any Cloud
## Lab: Cloud provisioning/orchestration - Terraform and OpenStack
Lab template for a Cloud provisioning/orchestration exercise with Terraform
(TF) and OpenStack/SwitchEngines.
......@@ -42,13 +40,18 @@ Please, refer to your OS documentation for the proper way to do so
CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli)
v1.1.4. Skip the TF "Quick start tutorial" (Docker).
1. [OpenStack CLI](https://docs.openstack.org/newton/user-guide/common/cli-install-openstack-command-line-clients.html) v5.7.0.
1. Grab or create a new project in your OpenStack account. The placeholder
used in this exercise is `<your-project-name>` with value `Cloud-MA-IaC`.
1. Create a new [OpenStack Access
Credentials](https://engines.switch.ch/horizon/identity/application_credentials/)
and save as `~/.config/openstack/clouds.yaml`. With SwitchEngine, the
cloud name to use in this lab should be `engines`.
cloud name to use in this is `engines`. :warning: App credentials might
not work for some commands; [API
credentials](https://engines.switch.ch/horizon/project/api_access/clouds.yaml)
with explicit project name/ID setup might be better.
1. Verify that your credentials are OK:
``` shell
lcl$ $ openstack --os-cloud=engines [application] credential list
lcl$ $ openstack --os-cloud=Cloud-MA-IaC [application] credential list
```
### Task #2: configure TF for OpenStack ###
......@@ -58,7 +61,7 @@ Please, refer to your OS documentation for the proper way to do so
<a name="image-query"></a>Find out the smallest image to use for a Debian server:
``` shell
lcl$ openstack --os-cloud=engines image list --limit=20 --public --status=active --sort-column=Size -c ID -c Name -c Size --long
lcl$ openstack --os-cloud=Cloud-MA-IaC image list --limit=10 --public --status=active --sort-column=Size -c ID -c Name -c Size --long
+--------------------------------------+-------------------------------------+-------------+
| ID | Name | Size |
+--------------------------------------+-------------------------------------+-------------+
......@@ -82,66 +85,23 @@ Find out the smallest instance *flavor* that acommodates our Debian image.
:bulb: Our flavor will be `m1.small` for the placeholder `<your-flavor>`.
**@@@ RESTART FROM HERE @@@**
Create a "sandbox" directory on your local machine `~/terraform/AWS/`. Inside
Create a "sandbox" directory on your local machine `~/terraform/OpenStack/`. Inside
it, create a file called `main.tf` (written in HCL language), the
infrastructure *definition* file, with the following content:
``` hcl
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.27"
}
}
required_version = ">= 0.14.9"
}
infrastructure *definition* file, with the appropriate content to handle:
1. an instance,
1. a security group to allow SSH access,
1. a floating IP in the public pool associated to your instance.
provider "aws" {
profile = "default"
region = "us-east-1"
}
resource "aws_instance" "app_server" {
ami = "<your-image-ID>"
instance_type = "t2.micro"
tags = {
Name = "ExampleAppServerInstance"
}
}
```
Also, prepare an `~/terraform/OpenStack/outputs.tf` file with the appropriate
contents to get the instance's public IP address (via `instance_public_ip`).
Initialize your sandbox with:
``` shell
lcl$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 3.27"...
- Installing hashicorp/aws v3.27.0...
- Installed hashicorp/aws v3.27.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository so
that Terraform can guarantee to make the same selections by default when you run
"terraform init" in the future.
Terraform has been successfully initialized!
...
```
Have a look inside the newly created sub-directory
`~/terraform/AWS/.terraform/`, you'll find the required `aws` provider module
that has been downloaded during the initialization.
It's good practice to format and validate your configuration:
Format and validate your configuration:
``` shell
lcl$ terraform fmt
......@@ -150,406 +110,14 @@ lcl$ terraform validate
Success! The configuration is valid.
```
### Task #3: deploy your AWS infrastructure ###
**Goal:** provision your AWS EC2 instance via TF.
Run the following command, confirm by typing "yes" and observe it's output:
Then apply!
``` shell
lcl$ terraform apply
Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_instance.app_server will be created
+ resource "aws_instance" "app_server" {
+ ami = "ami-0fa37863afb290840"
+ arn = (known after apply)
...
+ instance_type = "t2.micro"
...
+ tags = {
+ "Name" = "ExampleAppServerInstance"
}
...
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_instance.app_server: Creating...
...
aws_instance.app_server: Creation complete after 38s [id=i-0155ba9d77ee0a854]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```
The information shown above before the prompt for action is the `execution
plan`: the `+` prefix mark things to add. Of course, many details are unknown
until the corresponding resource is instantiated.
:question: How many resources were created?
You can verify via the AWS dashboard that one EC2 instance has been created.
The state of the resources acted upon is locally stored. Let's see what's in
our sandbox:
``` shell
lcl$ tree -a
.
├── .terraform
│ └── providers
...
├── .terraform.lock.hcl
├── main.tf
└── terraform.tfstate
```
:question: What's in file `terraform.tfstate`? The answer comes from the
following commands:
``` shell
lcl$ terraform state list
aws_instance.app_server
```
It confirms that we're tracking one AWS instance. Let's dig a bit more:
``` shell
lcl$ terraform show
# aws_instance.app_server:
resource "aws_instance" "app_server" {
ami = "ami-0fa37863afb290840"
arn = "arn:aws:ec2:us-east-1:768034348959:instance/i-0155ba9d77ee0a854"
associate_public_ip_address = true
availability_zone = "us-east-1e"
...
id = "i-0155ba9d77ee0a854"
instance_initiated_shutdown_behavior = "stop"
instance_state = "running"
instance_type = "t2.micro"
...
private_dns = "ip-172-31-94-207.ec2.internal"
private_ip = "172.31.94.207"
public_dns = "ec2-3-94-184-169.compute-1.amazonaws.com"
public_ip = "3.94.184.169"
...
tags = {
"Name" = "ExampleAppServerInstance"
}
...
vpc_security_group_ids = [
"sg-0c420780b4f729d3e",
]
...
}
```
The above command output provides useful runtime (state) information, like the
instance IP's address. Indeed, there is a kind of *digital twin* stored inside
the file `terraform.tfstate`.
:question: Is that instance accessible via SSH? Give it a try. If not, why?
Now, stop the running instance (its ID is shown above ;-):
``` shell
lcl$ aws ec2 stop-instances --instance-ids i-0155ba9d77ee0a854
```
wait some seconds and test again:
``` shell
lcl$ terraform show | grep instance_state
instance_state = "running"
```
How come? We just discovered that TF read only the local status of a
resource. So, let's refresh it, and check again:
``` shell
lcl$ terraform refresh
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
lcl$ terraform show | grep instance_state
instance_state = "stopped"
```
Ah-ha!
Hold on a second: our TF plan does not specify the desired status of a
resource. What happens if we reapply the plan? Lets' try:
``` shell
lcl$ terraform apply -auto-approve
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
No changes. Your infrastructure matches the configuration.
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
lcl$ terraform show | grep instance_state
instance_state = "stopped"
```
:warning: Apply was run in `-auto-approve` (non interactive) mode which
assumes "yes" to all questions. Use with care!
From the above commands' output we see that
* the local state is refreshed before doing anything,
* no changes are applied, and
* huh?... the resource is still stopped.
Concerning the last point above, think about the basic objectives of TF: as a
provisioning tool it is concerned with the *existence* of a resource, not with
its *runtime* state. This latter is the business of configuration management
tools. :bulb: There is no way with TF to specify a resource's desired runtime
state.
### Task #4: change your infrastructure ###
**Goal:** modify the resource created before, and learn how to apply changes
to a Terraform project.
Restart your managed instance:
``` shell
lcl$ aws ec2 start-instances --instance-ids i-0155ba9d77ee0a854
```
Refresh TF's view of the world:
``` shell
lcl$ terraform refresh
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
lcl$ terraform show | grep instance_state
instance_state = "running"
```
Replace the resource's `ami` in `main.tf` with the second one found from the
[catalog query done above](#image-query) (or another one available with your
account). Before applying our new plan, let's see what TF thinks of it:
``` shell
lcl$ terraform plan -out=change-AMI.tfplan
aws_instance.app_server: Refreshing state... [id=i-0155ba9d77ee0a854]
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
Terraform will perform the following actions:
# aws_instance.app_server must be replaced
-/+ resource "aws_instance" "app_server" {
~ ami = "ami-0fa37863afb290840" -> "ami-0e2512bd9da751ea8" # forces replacement
...
Plan: 1 to add, 0 to change, 1 to destroy.
Saved the plan to: change-AMI.tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "change-AMI.tfplan"
```
:bulb: Remarks:
* The change we want to apply is destructive!
* We saved our plan. :question: Why? It is not really necessary in a simple
scenario like ours, however a more complex IaC workflow might require plan
artifacts to be programmatically validated and versioned.
Apply the saved plan:
``` shell
lcl$ terraform apply change-AMI.tfplan
aws_instance.app_server: Destroying... [id=i-0155ba9d77ee0a854]
...
aws_instance.app_server: Destruction complete after 33s
aws_instance.app_server: Creating...
...
aws_instance.app_server: Creation complete after 48s [id=i-0470db35749548101]
```
:bulb: What? Not asking for confirmation? Indeed, a saved plan is intended for
automated workflows! Moreover, a saved plan will come handy for rolling back a
broken infrastructure to the last working setup.
:question: What if we did not save our plan, and called a plain apply command?
Would the result be the same?
### Task #5: input variables ###
**Goal:** make a TF plan more flexible via input variables.
Our original plan has all its content hard-coded. Let's make it more flexible
with some input variables stored in a separate `variables.tf` file inside your
TF sandbox:
``` hcl
variable "instance_name" {
description = "Value of the Name tag for the EC2 instance"
type = string
default = "AnotherAppServerInstance"
}
```
Then modify the `main.tf` as follows:
``` hcl
resource "aws_instance" "app_server" {
ami = "ami-0e2512bd9da751ea8"
instance_type = "t2.micro"
tags = {
- Name = "ExampleAppServerInstance"
+ Name = var.instance_name
}
}
```
Apply the changes:
``` shell
lcl$ terraform apply -auto-approve
aws_instance.app_server: Refreshing state... [id=i-0470db35749548101]
...
~ update in-place
Terraform will perform the following actions:
# aws_instance.app_server will be updated in-place
~ resource "aws_instance" "app_server" {
id = "i-0470db35749548101"
~ tags = {
~ "Name" = "ExampleAppServerInstance" -> "AnotherAppServerInstance"
}
~ tags_all = {
~ "Name" = "ExampleAppServerInstance" -> "AnotherAppServerInstance"
}
...
}
Plan: 0 to add, 1 to change, 0 to destroy.
...
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
```
:bulb: **Exercise:** input variables can also be passed on the `apply` command
line. Find how to do that with another different value for the variable
`instance_name`. :question: Would this last change be persistent if we rerun a
plain `terraform apply`?
### Task #6: queries with outputs ###
**Goal:** use output values to query a provisioned infrastructure.
We have seen in the previous tasks that the infrastructure's status can be
displayed via `terraform show`: a rather clumsy way, if you just want to
extract some specific information. A better programmatic way of querying your
infrastructure makes use of "outputs". Put the following in a file called
`~/terraform/AWS/outputs.tf`:
``` hcl
output "instance_id" {
description = "ID of the EC2 instance"
value = aws_instance.app_server.id
}
output "instance_public_ip" {
description = "Public IP address of the EC2 instance"
value = aws_instance.app_server.public_ip
}
```
We have declared two outputs. As usual with TF, before querying their
associated values, we need to apply the changes:
``` shell
lcl$ terraform apply -auto-approve
...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
instance_id = "i-0470db35749548101"
instance_public_ip = "34.201.252.63"
instance_tags = tomap({
"Name" = "AnotherAppServerInstance"
})
```
So, we already got the needed information, but, within a workflow, it is more
practical to do something like:
``` shell
lcl$ terraform output -json instance_tags
{"Name":"AnotherAppServerInstance"}
```
:question: What if the `Name` tag is changed outside TF? Try it.
:question: What must be done to have TF respect that external change?
:question: How to revert an external change via TF?
### Task #7: SSH provisioning with Cloud-Init ###
**Goal:** use Cloud-Init to provision an SSH access to your TF-managed
instance.
Did you try to SSH into the instance you created via TF? It cannot work,
because we did not instructed TF about networks, users, keys or anything
else. **This is left entirely to you as an exercise.** You need to:
1. Destroy your infrastructure. There's a special TF command for that.
1. Create an SSH key pair `tf-cloud-init`.
1. Create a new cloud-init file
`~/terraform/AWS/scripts/add-ssh.yaml` with the following content:
``` yaml
#cloud-config
#^^^^^^^^^^^^
# DO NOT TOUCH the first line!
---
groups:
- ubuntu: [root, sys]
- terraform
users:
- default
- name: terraform
gecos: terraform
primary_group: terraform
groups: users, admin
ssh_authorized_keys:
- <your-SSH-pub-key-on-one-line>
```
:warning: **Mind that the first line of this file must spell exactly
`#cloud-config`**!
1. Modify the `main.tf` as follows:
1. add a `resource` block of type `"aws_security_group"` allowing ingress
ports 22 and 80 from any address, with any egress port open;
1. add a `data` block referencing the above authorization file as a
`"template_file"` type;
1. extend the `"aws_instance"` resource to:
1. associate a public IP address,
1. link the `data` block to a user data attribute;
1. add an output `"public_ip"`.
When done, *init* your new plan, *validate* and *apply* it. Verify that you
can SSH as user `terraform` (not the customary `ubuntu`) into your instance:
Verify that you can SSH as user `debian` into your instance:
``` shell
lcl$ ssh terraform@$(terraform output -raw public_ip) -i ../tf-cloud-init
lcl$ ssh debian@$(terraform output -raw instance_public_ip) -i ../tf-cloud-init
```
......@@ -541,5 +541,5 @@ When done, *init* your new plan, *validate* and *apply* it. Verify that you
can SSH as user `terraform` (not the customary `ubuntu`) into your instance:
``` shell
lcl$ ssh terraform@$(terraform output -raw public_ip) -i ../tf-cloud-init
lcl$ ssh terraform@$(terraform output -raw instance_public_ip) -i ../tf-cloud-init
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment