diff --git a/SwitchEngines/README.md b/SwitchEngines/README.md index 0261ec35735156f2500e09c542b58ddf0e300cec..3ddccb3c8200ff928974c128db8d92e8c36a5cef 100644 --- a/SwitchEngines/README.md +++ b/SwitchEngines/README.md @@ -526,7 +526,7 @@ lcl$ openstack --os-cloud=engines server set \ :question: How to revert an external change via TF? -### Task #7: networking and SSH provisioning with Cloud-Init ### +### Task #7: networking and SSH provisioning with Cloud-init ### **Goal:** complete the network configuration and use [Cloud-init](https://cloudinit.readthedocs.io/) to provision an SSH access to @@ -534,9 +534,13 @@ your TF-managed instance. Did you try to SSH into the instance you created via TF? It cannot work, because we did not instructed TF about networking, login, users, keys or -anything else. **This is left entirely to you as an exercise.** With reference -to the official TF documentation for the [OpenStack -provider](https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs/), you shall: +anything else. + +**This is left to you as an exercise.** + +With reference to the official TF documentation for the [OpenStack +provider](https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs/), +you shall: 1. Destroy your infrastructure. Guess how ;-) 1. Create **locally** in your sandbox an SSH key pair named `tf-cloud-init`: @@ -545,14 +549,13 @@ provider](https://registry.terraform.io/providers/terraform-provider-openstack/o lcl$ yes | ssh-keygen -t ed25519 -f keys/tf-cloud-init -C terraform@TF-lab ``` 1. Create a new cloud-init file - `~/terraform/OpenStack/conf/cloud-init.add-ssh.yaml` with the following content: + `conf/cloud-init.add-ssh.yaml` with the following content: ``` yaml #cloud-config #^^^^^^^^^^^^ # DO NOT TOUCH the first line! --- groups: - - debian: [root, sys] - terraform users: @@ -560,7 +563,7 @@ provider](https://registry.terraform.io/providers/terraform-provider-openstack/o - name: terraform gecos: terraform primary_group: terraform - groups: users, admin + groups: [users, admin] ssh_authorized_keys: - <your-SSH-pub-key-on-one-line> ``` @@ -577,8 +580,8 @@ provider](https://registry.terraform.io/providers/terraform-provider-openstack/o 'data' port; 1. extend your `"app_server"` resource block to: 1. reference your custom security group as well as the default one, - 1. include the above cloud-init file, - 1. add two TF outputs: `"instance_public_ip"` and `"ssh_user"`. + 1. include the above cloud-init file via a [`user_data` attribute](https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs/resources/compute_instance_v2), + 1. add two TF outputs: `"instance_public_ip"` (dynamic) and `"ssh_user"` (static). :bulb: In the above instruction, "reference" means that the configuration keys shall take values dynamically from variable expansion, like (`${}` is optional in simple cases): @@ -595,3 +598,153 @@ lcl$ ping $(terraform output -raw instance_public_ip) lcl$ ssh -i keys/tf-cloud-init -o IdentitiesOnly=yes $(terraform output -raw ssh_user)@$(terraform output -raw instance_public_ip) ... ``` + +### Task #8: further practice with Cloud-init ### + +**Goal:** get more acquainted with Cloud-init: change the instance's default +configuration, handle packages, integrate more with Terraform. + +You may already have realized that Cloud-init is a declarative configuration +management tool for the Cloud ;-) + +As for the provisioning side, the mechanism to pass extra instance +configuration, scripts, etc. is the ["user +data"](https://cloudinit.readthedocs.io/en/latest/explanation/format.html). Of +course, most if not all cloud images come with Cloud-init installed. + +The provisioning tool/platform (here Terraform) just does some proper +formatting/marshaling of user data before sending them to the cloud. + +:bulb: Before trying more, here are some tips for [debugging your +cloud-init](https://cloudinit.readthedocs.io/en/latest/howto/debugging.html#how-to-debug) +user data (of course your instance must be accessible via SSH): + +**Q.** Where are the instance's Cloud-init logs? A. They're in files +`/var/log/cloud-init*.log`. + +**Q.** Where are user-data files stored *in the instance*? A. Here they are: +- Received file: `/var/lib/cloud/instance/user-data.txt ` +- Rendered file: `/var/lib/cloud/instance/cloud-config.txt` + +**Q.** [How to rerun Cloud-init](https://cloudinit.readthedocs.io/en/latest/howto/rerun_cloud_init.html) on the instance without reinstalling it? +A. Run the command: `sudo cloud-init clean --logs --reboot`. + + +**The following tasks are left to you as exercises.** + + +#### Task #8.1: default user configuration, etc. #### + +In Task #7, you added the extra user `terraform` while leaving untouched the +default one. Actually, we might know nothing about this latter (it depends on +the distribution), so why not impose *our* default user? + +Easy! You just need to modify the `conf/cloud-init.add-ssh.yaml` above and +replace the `users` section with a suitable `default_user` dictionary under +section +[`system_info`](https://cloudinit.readthedocs.io/en/latest/reference/base_config_reference.html#system-info-keys): + +``` yaml +system_info: + default_user: + name: terraform + ... +``` + +You may also have a look at the distro's original Cloud-init configuration +`/etc/cloud/cloud.cfg` in your provisioned instance. + +Since our image is rather light, let's add some handy packages (names must be +the official distro's) to our cloud-init YAML: +``` yaml +packages: + - curl + - nano + - ripgrep + - <add your favorite here> + - ... +``` + +See how that's agnostic of the package manager? Of course, since we're on +Debian/Ubuntu, `apt` is actually called behind the scenes. + +Want to install a package out of the distro's tree? Quickly done with the +generic shell command support: +``` yaml +runcmd: + - mkdir -p /run/tmp + - curl -Lo /run/tmp/kind https://kind.sigs.k8s.io/dl/v0.24.0/kind-linux-amd64 + - install -o root -g root -m 0755 /run/tmp/kind /usr/local/bin/kind +``` + +Once done, rerun terraform apply as in Task #7. You should be able to connect +as before with user `terraform`. Verify that the extra packages have been +installed. + + +#### Task #8.2: integrating Cloud-init with Terraform #### + +So far Terraform didn't interfere at all with the user data: it just sent some +kind of blob down the pipe. Now, suppose you have *several* user data files +(configuration, scripts, etc.) to be provisioned: how to do that? Several +`user_data` attributes? Nah, that wouldn't work. + +Enter the [`cloudinit-config` data source](https://registry.terraform.io/providers/hashicorp/cloudinit/latest/docs/data-sources/config)! + +[Data +sources](https://developer.hashicorp.com/terraform/language/data-sources) +provide special blocks to store information that can be used by resource +blocks. You already used a data block for the FIP port association. + +The cloudinit-config data source supports the declaration of several parts +which are *merged* together into a single Multi-part MIME piece, so that the +result can be ingested by the instance resource's `user_data` attribute. + +You shall: + +1. Split your cloud-init YAML file above into 2 files: + - `conf/cloud-init.users.yaml` with basic user-related configuration, and + - `conf/cloud-init.packages.yaml` with package-related configuration. +1. Modify your `main.tf`: + 1. Add a cloudinit-config data block with two parts declaring each one of + the above YAML conf file. + 1. Change, in the `app_server` resource block, the `user_data` assignment to + get the "rendered" output of the cloudinit-config data block. + +Then, rerun terraform apply, connect to your instance and verify that +everything is there. + +:question: What if one needs to modify or override in the second cloud-init +YAML file something already declared in the first one? E.g., adding a special +package-provided group to the default user. There's risk of breaking things, +unless the correct [user-data merge strategy is +specified](https://cloudinit.readthedocs.io/en/latest/reference/merging.html). + + +### Task #9: Provisioning KinD with with Terraform ### + +**Please, complete the [K8s Lab +tutorial](https://gitedu.hesge.ch/lsds/teaching/bachelor/cloud-and-deployment/lab-k8s) +before starting this exercise.** + +Now you have all the skills to provision your micro cloud infrastructure based +on KinD. + +You shall: + +1. Modify your `main.tf` to use a more capable flavor instance for your + `app_server`, like a `m1.medium` or `c1.large`. +2. Extend your YAML files to + - install all the required packages. Tips: add the distro's `docker.io` + package, install `kubectl` and `kind` via explicit `runcmd` scripts; + - add any needed extra group for the default user. + +Run again terraform apply, wait a couple of minutes to let the installation of +all extra packages to complete, then connect to your instance. + +Since KinD is not designed for remote use, you'll have to create your cluster +on the instance as done during the K8s Lab: + +1. Write a minimal KinD configuration file for 2 worker nodes. +1. Provision the cluster. +1. Deploy the matallb load balancer service.