Reviewed for 2024/2025

Signed-off-by: Marco Emilio "sphakka" Poleggi <marcoep@ieee.org>

SwitchEngines/README.md

@@ -34,10 +34,10 @@ Please, refer to your OS documentation for the proper way to do so.
 
   1. [Terraform
      CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli)
-     v1.5. Skip the TF "Quick start tutorial" (Docker).
+     version >= 1.9.0. Skip the TF "Quick start tutorial" (Docker).
   1. [OpenStack
      CLI](https://docs.openstack.org/newton/user-guide/common/cli-install-openstack-command-line-clients.html)
-     version >= 6.0.0. It is recommended to install your original distribution
+     version >= 7.0.0. It is recommended to install your original distribution
      package named like `python-openstackclient`.
   1. :bulb: **Skip this step if you already have a project assigned in SwitchEngines.**
      On the OpenStack "Horizon" dashboard, create a new project. The placeholder used in
@@ -86,69 +86,85 @@ clouds:
     ...
 ```
 
-Find out the smallest image to use for a Debian server:
+Find out the smallest image to use for a Debian 11 server:
 
 ``` shell
 lcl$ openstack --os-cloud=engines image list --public --status=active --sort-column=Size -c ID -c Name -c Size --long
 +--------------------------------------+-------------------------------------+-------------+
 | ID                                   | Name                                |        Size |
 +--------------------------------------+-------------------------------------+-------------+
-| 8674f1a5-f7d9-4975-af0b-d2e9e33c9152 | Debian Buster 10 (SWITCHengines)    |  1537409024 |
+| 06c5a538-8123-49d3-83f4-0a8fc5a07967 | Debian Bullseye 11 (SWITCHengines)  |  1670840320 |
 ...
 ```
 
 :bulb: We use the first ID found for the placeholder `<your-image-ID>`. In
-SwitchEngines, that has a size of ~1.5GB.
+SwitchEngines, that has a size of ~1.7GB.
 
 Find out the smallest instance *flavor* that acommodates our Debian image.
 
-  ``` shell
-  lcl$ openstack --os-cloud=engines flavor list --sort-column=RAM --sort-column=Disk --min-disk=5 --min-ram=1024 --limit=10 --public
+``` shell
+lcl$ openstack --os-cloud=engines flavor list --sort-column=RAM --sort-column=Disk --min-disk=5 --min-ram=1024 --limit=10 --public
 +----+----------+------+------+-----------+-------+-----------+
 | ID | Name     |  RAM | Disk | Ephemeral | VCPUs | Is Public |
 +----+----------+------+------+-----------+-------+-----------+
 | 2  | m1.small | 2048 |   20 |         0 |     1 | True      |
 +----+----------+------+------+-----------+-------+-----------+
-  ```
+```
 
 :bulb: Our flavor will be `m1.small` for the placeholder `<your-flavor>`.
 
-Create a "sandbox" directory on your local machine
-`~/terraform/OpenStack/`. Inside it, create a file called `main.tf` (written
-in HCL language), the infrastructure *definition* file, with the following
-content (replace all `<...>` tokens with actual values):
+Clone or, if you want to track your progress, fork and clone this GIT repo.
+
+It's not advised to run terraform inside your repo clone because sensitive
+information like SSH keys might get committed in Git. Thus, create a "sandbox"
+directory on your local machine:
+``` shell
+lcl$ mkdir -p ~/terraform/OpenStack/
+```
 
-:bulb: Application credentials shall match those in file
-`~/.config/openstack/clouds.yaml`.
+Copy your repo clone's content over to your sandbox (you might need to install `rsync`):
+``` shell
+lcl$ rsync -Cahv YOUR_REPOS/lab-terraform/ ~/terraform/OpenStack/
+...
+```
 
+Have a look at the boilerplate `main.tf` which is written in the HCL
+language. That's the infrastructure *definition* file. For a one-instance
+recipe, there must be at least three blocks:
 ``` hcl
+# The mandatory header block that specifies what providers have to be used
 terraform {
-  required_version = ">= 0.14.9"
   required_providers {
     openstack = {
-      source  = "terraform-provider-openstack/openstack"
-      version = "~> 1.48.0"
+      ...
     }
   }
+  ...
 }
 
+# A "provider" block
 provider "openstack" {
-  auth_url          = "https://keystone.cloud.switch.ch:5000/v3"
-  project_domain_id = "<your-project-ID>"
-  application_credential_id     = "<your-app-cred-ID>"
-  application_credential_secret = "<your-app-cred-secret>"
-  region = "<your-region>"
+  ...
 }
 
+# A "resource" block with a name
 resource "openstack_compute_instance_v2" "app_server" {
-  name            = "TF-managed"
-  image_id        = "<your-image-ID>"
-  flavor_name     = "<your-flavor>"
+  ...
 }
 ```
-Initialize your sandbox with:
 
+Indeed, all TF files inside a directory constitute a "module". In our case,
+that's the "root" module.
+
+You shall replace all `<...>` tokens with actual values.
+
+:bulb: If you want use explicit provider resource configuration, the values
+shall match those in file `~/.config/openstack/clouds.yaml`.
+
+
+Initialize your sandbox with:
 ``` shell
+lcl$ cd ~/terraform/OpenStack/
 lcl$ terraform init
 
 Initializing the backend...
@@ -157,11 +173,7 @@ Initializing provider plugins...
 - Finding terraform-provider-openstack/openstack versions matching "~> 1.48.0"...
 - Installing terraform-provider-openstack/openstack v1.48.0...
 - Installed terraform-provider-openstack/openstack v1.48.0 (self-signed, key ID 4F80527A391BEFD2)
-
-Partner and community providers are signed by their developers.
-If you'd like to know more about provider signing, you can read about it here:
-https://www.terraform.io/docs/cli/plugins/signing.html
-
+...
 Terraform has created a lock file .terraform.lock.hcl to record the provider
 selections it made above. Include this file in your version control repository
 so that Terraform can guarantee to make the same selections by default when
@@ -171,9 +183,9 @@ Terraform has been successfully initialized!
 ...
 ```
 
-Have a look inside the newly created sub-directory
-`~/terraform/OpenStack/.terraform/`, you'll find the required `openstack`
-provider module that has been downloaded during the initialization.
+Have a look inside the newly created sub-directory `.terraform/`, you'll find
+the required `openstack` provider module that has been downloaded during the
+initialization.
 
 It's good practice to format and validate your configuration:
 
@@ -243,7 +255,6 @@ Do you want to perform these actions?
   Only 'yes' will be accepted to approve.
 
   Enter a value: yes
-
 ...
 openstack_compute_instance_v2.app_server: Creation complete after 18s [id=f70aef53-51f9-4d12-a4a9-fe9a6cc5a58f]
 
@@ -254,7 +265,7 @@ You can verify via the SwitchEngine dashboard that one instance named
 "TF-managed" has indeed been created.
 
 The state of the resources acted upon is locally stored. Let's see what's in
-our sandbox:
+our sandbox (you might need to install `tree`):
 ``` shell
 lcl$ tree -a
 .
@@ -262,7 +273,7 @@ lcl$ tree -a
 │   └── providers
 ...
 ├── .terraform.lock.hcl
-├── main.tf
+...
 └── terraform.tfstate
 ```
 
@@ -273,6 +284,7 @@ following commands:
 lcl$ terraform state list
 openstack_compute_instance_v2.app_server
 ```
+
 It confirms that we're tracking one OpenStack instance. Let's dig a bit more:
 ``` shell
 lcl$ terraform show
@@ -304,12 +316,12 @@ instance IP's address. Indeed, there is a kind of *digital twin* stored inside
 the file `terraform.tfstate`.
 
 :question: Is that instance accessible from the internet? Give it a try. If
-not, why? We'll deal with that in Task #4 below.
+not, why? We'll deal with that in Task #7 below.
 
 Now, stop the running instance (its ID is shown above ;-):
 
 ``` shell
-lcl$ openstack --os-cloud=engines server stop f70aef53-51f9-4d12-a4a9-fe9a6cc5a58f
+lcl$ openstack --os-cloud=engines server stop YOUR_INSTANCE_ID
 ```
 
 wait some seconds and test again:
@@ -319,14 +331,15 @@ lcl$ terraform show | grep power_state
     power_state         = "active"
 ```
 
-How come? We just discovered that TF read only the local status of a
-resource. So, let's refresh it, and check again:
+How come? We just discovered that TF reads only the local status of a
+resource. So, let's refresh it, and check again -- this time with JSON output
+and `jq` (you might need to install it):
 
 ``` shell
 lcl$ terraform refresh
 ...
-lcl$ terraform show | grep power_state
-    power_state         = "shutoff"
+lcl$ terraform show -json | jq .values.root_module.resources[0].values.power_state
+"shutoff"
 ```
 
 Ah-ha!
@@ -363,7 +376,7 @@ From the above commands' output we see that
 to a Terraform project.
 
 Replace the resource's `image_id` in `main.tf` with the second one found from
-the catalog query done above -- it should be a "Debian Bullseye 11
+the catalog query done above -- it should be a "Debian Bullseye 12
 (SWITCHengines)". Before applying our new plan, let's see what TF thinks of
 it:
 
@@ -393,10 +406,12 @@ To perform exactly these actions, run the following command to apply:
 ```
 
 :bulb: Remarks:
-  * The change we want to apply is *not* destructive from terraform's perspective, but, of course, the instance has been reinstalled with a different OS!
+  * The change we want to apply is *not* destructive from terraform's
+    perspective, but, of course, the instance must be reinstalled with a
+    different OS!
   * We saved our plan. :question: Why? It is not really necessary in a simple
     scenario like ours, however a more complex IaC workflow might require plan
-    artifacts to be programmatically validated and versioned.
+    artifacts to be stored, validated and versioned.
 
 Apply the saved plan:
 ``` shell
@@ -503,7 +518,7 @@ lcl$ terraform output instance_id
 :question: What if the `my_instance_name` tag is changed outside TF? Try f.i.:
 ``` shell
 lcl$ openstack --os-cloud=engines server set \
-       --property my_instance_name="Foo-Bar" 93914f14-e521-4cc1-acfe-046bc3fa31be
+       --property my_instance_name="Foo-Bar" $(terraform output instance_id)
 ```
 
 :question: What must be done to have TF respect that external change?
@@ -513,8 +528,9 @@ lcl$ openstack --os-cloud=engines server set \
 
 ### Task #7: networking and SSH provisioning with Cloud-Init ###
 
-**Goal:** use Cloud-Init to provision an SSH access to your TF-managed
-instance.
+**Goal:** complete the network configuration and use
+[Cloud-init](https://cloudinit.readthedocs.io/) to provision an SSH access to
+your TF-managed instance.
 
 Did you try to SSH into the instance you created via TF? It cannot work,
 because we did not instructed TF about networking, login, users, keys or
@@ -523,9 +539,13 @@ to the official TF documentation for the [OpenStack
 provider](https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs/), you shall:
 
   1. Destroy your infrastructure. Guess how ;-)
-  1. Create an SSH key pair named `tf-cloud-init` either via the Horizon GUI or locally (must be RSA type).
+  1. Create **locally** in your sandbox an SSH key pair named `tf-cloud-init`:
+      ``` shell
+      lcl$ mkdir -p keys
+      lcl$ yes | ssh-keygen -t ed25519 -f keys/tf-cloud-init -C terraform@TF-lab
+      ```
   1. Create a new cloud-init file
-     `~/terraform/OpenStack/scripts/add-ssh.yaml` with the following content:
+     `~/terraform/OpenStack/conf/cloud-init.add-ssh.yaml` with the following content:
      ``` yaml
      #cloud-config
      #^^^^^^^^^^^^
@@ -553,25 +573,25 @@ provider](https://registry.terraform.io/providers/terraform-provider-openstack/o
         resources shall reference your security group;
      1. add a resource block for a v2 *floating IP* from the public pool;
      1. add a resource block for a v2 *floating IP association* referencing
-        your floating IP resource and your compute instance resource;
+        your floating IP resource and your compute instance resource via a
+        'data' port;
      1. extend your `"app_server"` resource block to:
         1. reference your custom security group as well as the default one,
-        1. associate a public IP address,
         1. include the above cloud-init file,
-     1. add a TF output `"instance_public_ip"`.
+     1. add two TF outputs: `"instance_public_ip"` and `"ssh_user"`.
 
-:bulb: In the above instruction, "reference" means that the keys shall take
-values dynamically from variable expansion, like:
+:bulb: In the above instruction, "reference" means that the configuration keys shall take
+values dynamically from variable expansion, like (`${}` is optional in simple cases):
 ``` hcl
   key = ${<object.attribute...>}
 ```
 
 When done, *validate* your new plan and *apply* it. Verify that you can ping
-and connect via SSH as user `terraform` into your instance:
+and connect via SSH into your instance (notice the two output queries):
 
 ``` shell
 lcl$ ping $(terraform output -raw instance_public_ip)
 ...
-lcl$ ssh terraform@$(terraform output -raw instance_public_ip) -i /path/to/private/key/tf-cloud-init
+lcl$ ssh -i keys/tf-cloud-init -o IdentitiesOnly=yes $(terraform output -raw ssh_user)@$(terraform output -raw instance_public_ip)
 ...
 ```

SwitchEngines/main.tf

-main.tf.basic
\ No newline at end of file

SwitchEngines/main.tf

+# -*- mode: terraform -*-
+terraform {
+  required_providers {
+    openstack = {
+      source  = "terraform-provider-openstack/openstack"
+      version = "~> 3.0.0"
+    }
+  }
+  required_version = ">= 1.9.0"
+}
+
+# Configure the OpenStack Provider
+provider "openstack" {
+  # Either declare explicitly...
+  # auth_url                      = "https://keystone.cloud.switch.ch:5000/v3"
+  # project_domain_id             = "<your-project-ID>"
+  # application_credential_id     = "<your-app-cred-ID>"
+  # application_credential_secret = "<your-app-cred-secret>"
+  # region                        = "<your-region>"
+  #...or use the system's clouds.yaml configuration
+  cloud                           = "engines"
+}
+
+resource "openstack_compute_instance_v2" "app_server" {
+  name            = "TF-managed"
+  image_id        = "<your-image-ID>"
+  flavor_name     = "<your-flavor>"
+
+  # @Task #7.
+
+  # @@ SECGROUPS CONFIGURATION @@
+
+  # SSH access to your instance: 3 methods are possible.
+
+  # Option #1. Key pair configured manually in the Horizon dashboard. Access
+  # with _default_ image's user
+
+  # Option #2. Use a resource. Access with _default_ image's user
+
+  # Option #3 (*preferred*). Cloud-init with user data. Access with user
+  # specified in the cloud-init YAML file. Key pair is not visible in Horizon
+
+  # @@ KEY PAIR CONFIGURATION @@
+}
+
+# @Task #7. Network configuration.
+
+# @@ SEC GROUP CONFIGURATION @@
+
+# @@ SEC GROUP RULES CONFIGURATION: ICMP, TCP/22 @@
+
+# @@ FIP + PORT + ASSOCIATION CONFIGURATION @@

SwitchEngines/main.tf.basic

-# -*- mode: terraform -*-
-terraform {
-  required_providers {
-    openstack = {
-      source  = "terraform-provider-openstack/openstack"
-      version = "~> 1.48.0"
-    }
-  }
-  required_version = ">= 0.15.0"
-}
-
-# Configure the OpenStack Provider
-provider "openstack" {
-  auth_url          = "https://keystone.cloud.switch.ch:5000/v3"
-  project_domain_id = "<your-project-ID>"
-  application_credential_id     = "<your-app-cred-ID>"
-  application_credential_secret = "<your-app-cred-secret>"
-  region = "<your-region>"
-}
-
-resource "openstack_compute_instance_v2" "app_server" {
-  name            = "TF-managed"
-  image_id        = "<your-image-ID>"
-  flavor_name     = "<your-flavor>"
-  key_pair        = "TF_lab"
-}

SwitchEngines/scripts/export_keypair.sh

+#!/usr/bin/env sh
+################################################################################
+# export_keypair: export a TF-generated key pair (SSH Option #2)
+################################################################################
+
+instance_user_def=${SANDBOX_TF_INSTANCE_USER:-'INSTANCE_USER'}
+instance_user=${1:-${instance_user_def}}
+
+key_store_def=${SANDBOX_TF_KEY_STORE:-'keys'}
+key_store=${2:-${key_store_def}}
+
+keyname=$(terraform output --raw ssh_keypair_name) || exit 1
+
+mkdir -p $key_store || exit 1
+
+ssh_priv_key_file=${key_store}/id_RSA.${keyname}
+ssh_pub_key_file=${key_store}/id_RSA.${keyname}.pub
+
+terraform output --raw ssh_keypair_pub  > $ssh_pub_key_file
+terraform output --raw ssh_keypair_priv  > $ssh_priv_key_file
+
+chmod 0600 $ssh_priv_key_file
+
+echo >&2 "Key '${keyname}' exported as '${ssh_priv_key_file}[.pub]'"
+echo >&2 "To connect to your instance use something like:"
+cat <<EOF
+
+  ssh -i ${ssh_priv_key_file} -o "IdentitiesOnly=yes" ${instance_user}@\$(terraform output -raw instance_public_ip)
+
+EOF