From 111120a927b7dbcf282f442621630d6b8a08cee8 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 16:29:11 +0000 Subject: [PATCH 01/28] feat: new section for procedures Got started on how to init a new server --- _git.tar.gz | 4 +- src/procedures/skynet/new_server.md | 67 +++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 2 deletions(-) create mode 100644 src/procedures/skynet/new_server.md diff --git a/_git.tar.gz b/_git.tar.gz index 1bdc667..21dcb3b 100644 --- a/_git.tar.gz +++ b/_git.tar.gz @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:607910832377319c9b05ae51ff9cfdc1eee8de4b53d422f99473720c392eaf59 -size 397421 +oid sha256:c3f36679e432c85bbd2f13f1ba064a5e7ceb8dbb2e20f9a469d9bb534b79f628 +size 401139 diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md new file mode 100644 index 0000000..fb3ae6d --- /dev/null +++ b/src/procedures/skynet/new_server.md @@ -0,0 +1,67 @@ +# New Server (LXC) + +This is the instruction guide for setting up a new LXC server. + +Rough steps are as follows: +1. Plan the servers config +2. Login to Proxmox +3. Create Container using the base LXC image +4. Grab server details +5. Push new configuration + +## Plan server Configuration +To allocate he correct resources there are a few questions that need to be asked and answered. + +1. What will this server be **For**? +2. What will its **Name** be? +3. What will its **IP** be? +4. Fill the details into the tracking sheet. + +### What is it **For**? +What a server is for dictates what hardware resources need to be allocated. +Is there one already existing that you can copy the configuration of? +Do you have prior experience with what will be hosted on it? +Is there documentation that you can use as a foundation? + +When you know these, write it down and save for later. + +### What is its **Name**? +In our cluster we have a very definite naming scheme for the servers we have. +There are two rules: + +1. The login server (where folks have their home dirs and websites) is called Skynet. + * [This can be traced back to 2007][server_name_skynet] + * > By popular demand, the skynet name was retained for the login server +2. All other servers are also named after AI's + + +In the current cluster we tend to use groups of AI names for particular functions. +For example: + +* Vigil/Vendetta were AI's in the Mass effect series, pointing Shepherd forward, so they are our DNS servers +* Glados/Wheatly from Portal, that game runs on the Source Engine, so they are our Source control servers +* Optimus/Bumblebee from Transformers, their origin is a line of toys, so fittingly our games servers. +* Neuromancer/Wintermute, from Neuromancer, each with multiple minds, thus became our backup (redundancy) servers. + +If at all possible try to get the name to match its task. +Some past names and ideas for others can be found on [the nixos wiki][server_names] + +[server_name_skynet]: https://2009.skynet.ie/history.html +[server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md + + + + +### What is its **IP** address? +We have a ``/26`` allocation, so about 60 IP's we can make use of. +Like with the names above several servers are grouped together IP wise. +Check the [Server Inventory][server_inventory] to see what addresses are available. + +[server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv + +### Tracking sheet +Now that ye have all the details about the build it is time to add them to the [Server Inventory][server_inventory]. +Add a new one, incrementing the index, and fill in the rest of the information. + +For the IP address if the last segment (ABC in this: 193.1.99.ABC) is less than 100 then add a leading ``0`` to it. +This is so that it can be easily sorted. \ No newline at end of file From 76a87c8d9504c17f62e1caa5f2009214212ccafd Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 17:09:40 +0000 Subject: [PATCH 02/28] feat: abandon the use of the git information in the repo itself --- .githooks/post-checkout | 3 --- .githooks/post-commit | 3 --- .githooks/post-merge | 3 --- .githooks/pre-commit | 4 ---- .githooks/pre-push | 3 --- _git.tar.gz | 3 --- flake.nix | 4 +--- mkdocs.yml | 8 ++++---- 8 files changed, 5 insertions(+), 26 deletions(-) delete mode 100755 .githooks/post-checkout delete mode 100755 .githooks/post-commit delete mode 100755 .githooks/post-merge delete mode 100755 .githooks/pre-commit delete mode 100755 .githooks/pre-push delete mode 100644 _git.tar.gz diff --git a/.githooks/post-checkout b/.githooks/post-checkout deleted file mode 100755 index ca7fcb4..0000000 --- a/.githooks/post-checkout +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-checkout' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs post-checkout "$@" diff --git a/.githooks/post-commit b/.githooks/post-commit deleted file mode 100755 index 52b339c..0000000 --- a/.githooks/post-commit +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-commit' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs post-commit "$@" diff --git a/.githooks/post-merge b/.githooks/post-merge deleted file mode 100755 index a912e66..0000000 --- a/.githooks/post-merge +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-merge' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs post-merge "$@" diff --git a/.githooks/pre-commit b/.githooks/pre-commit deleted file mode 100755 index 99c93f8..0000000 --- a/.githooks/pre-commit +++ /dev/null @@ -1,4 +0,0 @@ -#!/usr/bin/env bash - -tar czf _git.tar.gz --exclude .git/lfs .git -git add _git.tar.gz diff --git a/.githooks/pre-push b/.githooks/pre-push deleted file mode 100755 index 0f0089b..0000000 --- a/.githooks/pre-push +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'pre-push' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs pre-push "$@" diff --git a/_git.tar.gz b/_git.tar.gz deleted file mode 100644 index 21dcb3b..0000000 --- a/_git.tar.gz +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:c3f36679e432c85bbd2f13f1ba064a5e7ceb8dbb2e20f9a469d9bb534b79f628 -size 401139 diff --git a/flake.nix b/flake.nix index 31415bf..b6401ae 100644 --- a/flake.nix +++ b/flake.nix @@ -19,8 +19,6 @@ packages = with pkgs.python3Packages; [ mkdocs mkdocs-material - mkdocs-git-authors-plugin - mkdocs-git-revision-date-localized-plugin ]; in { formatter = alejandra.defaultPackage.${system}; @@ -28,7 +26,7 @@ name = "skynet-wiki"; src = self; buildInputs = packages; - buildPhase = "tar -zxf _git.tar.gz && mkdocs build"; + buildPhase = "mkdocs build"; installPhase = "mkdir -p $out; cp -R site/* $out;"; }; diff --git a/mkdocs.yml b/mkdocs.yml index 0fd35f5..35c0d7a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,10 +25,10 @@ theme: view: material/eye plugins: - search - - git-authors: - show_email_address: false - - git-revision-date-localized: - enable_creation_date: true +# - git-authors: +# show_email_address: false +# - git-revision-date-localized: +# enable_creation_date: true markdown_extensions: - tables - pymdownx.caret From 742c93f69ce937cafe2b1703d5c876d698d68817 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 18:23:40 +0000 Subject: [PATCH 03/28] doc: added details on how to actually set up the container --- src/procedures/skynet/new_server.md | 71 ++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index fb3ae6d..d926e43 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -64,4 +64,73 @@ Now that ye have all the details about the build it is time to add them to the [ Add a new one, incrementing the index, and fill in the rest of the information. For the IP address if the last segment (ABC in this: 193.1.99.ABC) is less than 100 then add a leading ``0`` to it. -This is so that it can be easily sorted. \ No newline at end of file +This is so that it can be easily sorted. + +## Login to Proxmox + +Login to Proxmox (TODO: insert link to accessing Proxmox here) + +## Create Container Using the Base LXC Image +The Proxmox documentation for LXC's is available [here][proxmox_lxc]. + +Top right there is a button [Create CT], that brings up a window. +Each section below is one of the tabs in the window. + +### General +#### Hostname +This is the **Name** of the server, lowercase. +#### Unprivilaged +Ensure this is ticked. +#### Nesting +Ensure this is ticked. +#### SSH Public Keys +Enter the ``root`` pub ssh key. +This is used to login to teh container later. + +### Template +Select the container image, most likely ``nixos-system-x86_64-linux.tar.xz``. + +### Disks +#### Storage +Most likely it is ``main_pool``, it should have a significant amount of storage available. +**DO NOT** use ``local-zfs``, this is on Proxmox's own drive and not suited for container data. + +#### Disk Size +Self-explanatory, how much space you want to give teh container. +A minimum of 30Gb is suggested. + +### CPU +One core minimum, larger servers will require up to 6 or so. + +### Memory +Nixos will happily run on 512Mb if its load is not too intensive + +### Network +#### Bridge +The main bridge we use is ``vmbr0`` which is for most servers as it connects to the normal Skynet DMZ. + +We also have ``vmbr1`` which is for ``skynet.skynet.ie`` and connects to Skynet-EXT DMZ. +This is due to our users needing ssh access. + +#### IPv4 +##### IPv5/CIDR +This is the **IP** followed by ``/26``. +For example ``193.1.99.75/26`` + +##### Gateway +The main gateway we use is ``193.1.99.65``. +There is a secondary one for ``skynet.skynet.ie`` which is ``193.1.96.161``. + +### DNS +You can either use ``use host settings`` or fill in ``193.1.99.120`` and ``193.1.99.109``. + +### Confirm +Use this as a chance to review all the options. +There are a few gotcha's outlined above that ye do have to look over. + +#### Start after created +Tick this box if you want it to boot up immediately after being installed. + + + +[proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container \ No newline at end of file From 57d8b3daeed92ea7831482c66e0b4bb3f5550e00 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 18:24:32 +0000 Subject: [PATCH 04/28] fmt: little bit of re-organisation --- src/procedures/skynet/new_server.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index d926e43..38dee07 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -46,19 +46,12 @@ For example: If at all possible try to get the name to match its task. Some past names and ideas for others can be found on [the nixos wiki][server_names] -[server_name_skynet]: https://2009.skynet.ie/history.html -[server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md - - - ### What is its **IP** address? We have a ``/26`` allocation, so about 60 IP's we can make use of. Like with the names above several servers are grouped together IP wise. Check the [Server Inventory][server_inventory] to see what addresses are available. -[server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv - ### Tracking sheet Now that ye have all the details about the build it is time to add them to the [Server Inventory][server_inventory]. Add a new one, incrementing the index, and fill in the rest of the information. @@ -79,7 +72,7 @@ Each section below is one of the tabs in the window. ### General #### Hostname This is the **Name** of the server, lowercase. -#### Unprivilaged +#### Unprivileged Ensure this is ticked. #### Nesting Ensure this is ticked. @@ -133,4 +126,9 @@ Tick this box if you want it to boot up immediately after being installed. + + +[server_name_skynet]: https://2009.skynet.ie/history.html +[server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md +[server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv [proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container \ No newline at end of file From a26204fe38c128218c01ade32ffaa47d03a84dd4 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 15:43:21 +0000 Subject: [PATCH 05/28] feat: login to teh server step completed --- src/procedures/skynet/new_server.md | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index 38dee07..b347178 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -6,7 +6,7 @@ Rough steps are as follows: 1. Plan the servers config 2. Login to Proxmox 3. Create Container using the base LXC image -4. Grab server details +4. Login to Server 5. Push new configuration ## Plan server Configuration @@ -124,11 +124,29 @@ There are a few gotcha's outlined above that ye do have to look over. #### Start after created Tick this box if you want it to boot up immediately after being installed. +## Login to Server +Now that the server is up and running it is time to login to it. +Assuming you have your SSH configured like (TODO: admin ssh config). +You just have to use ``ssh root@IP`` (for example ``ssh root@192.99.1.111``) +### Getting the server ssh key +We are logging in because we need to get the servers own ssh key. +You can find it in ``/etc/ssh``. +You have a choice between ``ssh_host_ed25519_key.pub`` and ``ssh_host_rsa_key.pub``. +``ssh_host_ed25519_key.pub`` is the recommended one. + +### Using the server ssh key +This key is used to decrypt secrets stored in our Nixos repo and as such needs to be added there. +1. Add the key with the other system keys [here][nixos_secrets]. +2. Add it to the systems array underneath that. +3. In the ``secrets`` folder run ``cd secrets && agenix -r`` to rekey the secrets. + * This is to give the new server access. +4. Commit all the changed files [server_name_skynet]: https://2009.skynet.ie/history.html [server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md [server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv -[proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container \ No newline at end of file +[proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container +[nixos_secrets]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/secrets/secrets.nix#L35 \ No newline at end of file From 5c73c5a7e2ffdf2c696965bf04b876f6102fc75c Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 19:42:43 +0000 Subject: [PATCH 06/28] doc: added segments on how to deploy the config --- src/procedures/skynet/new_server.md | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index b347178..7ec7a89 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -144,9 +144,35 @@ This key is used to decrypt secrets stored in our Nixos repo and as such needs t 4. Commit all the changed files +## Push new configuration +### Create config +An example server config for nixos is [available here][nixos_template]. +Copy it to a new file and name it ``name.nix``. +Fill in all relevant details (name/ip/name details/its purpose/...). + +Finally add it to the bottom of the [``flake.nix``][nixos_flake]. +Save and commit the files. + +### Push Config +There are two ways to test out the config, manual and pipeline. +Each has pros and cons. + +#### Manually +This requires your key to be added the the keys for the [root account][nixos_root] and your ``~/.ssh/config`` to be set up properly (TODO: SSH tutorial) + +1. Build it with ``colmena build --on name`` +2. If it builds ye can test deployment +3. Deploy it using ``colmena apply --on name`` + +#### Pipeline +Just push it to the repo and the pipeline will handle building and deployment. +The disadvantage of this is you cannot fix any mistakes before they are pushed. [server_name_skynet]: https://2009.skynet.ie/history.html [server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md [server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv [proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container -[nixos_secrets]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/secrets/secrets.nix#L35 \ No newline at end of file +[nixos_secrets]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/secrets/secrets.nix#L35 +[nixos_template]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/machines/_template.nix +[nixos_flake]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/flake.nix#L156 +[nixos_root]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/machines/_base.nix#L75 \ No newline at end of file From ae2b53a868b7292b11b4858c023cf814d30a2edd Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 19:47:14 +0000 Subject: [PATCH 07/28] fmt: rename --- src/procedures/skynet/{new_server.md => server_new.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/procedures/skynet/{new_server.md => server_new.md} (100%) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/server_new.md similarity index 100% rename from src/procedures/skynet/new_server.md rename to src/procedures/skynet/server_new.md From 8d3ae44c16ce2ba7e12147f7ce2d0d71e1c29048 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 19:49:40 +0000 Subject: [PATCH 08/28] doc: planned documentation --- src/procedures/skynet/admin_add.md | 0 src/procedures/skynet/admin_remove.md | 0 src/procedures/skynet/admin_ssh.md | 0 src/procedures/skynet/almnui_verify.md | 0 src/procedures/skynet/committee_modify.md | 0 src/procedures/skynet/proxmox_access.md | 0 src/procedures/skynet/proxmox_reboot.md | 0 src/procedures/skynet/server_update.md | 0 src/procedures/skynet/vaultwarden.md | 0 9 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 src/procedures/skynet/admin_add.md create mode 100644 src/procedures/skynet/admin_remove.md create mode 100644 src/procedures/skynet/admin_ssh.md create mode 100644 src/procedures/skynet/almnui_verify.md create mode 100644 src/procedures/skynet/committee_modify.md create mode 100644 src/procedures/skynet/proxmox_access.md create mode 100644 src/procedures/skynet/proxmox_reboot.md create mode 100644 src/procedures/skynet/server_update.md create mode 100644 src/procedures/skynet/vaultwarden.md diff --git a/src/procedures/skynet/admin_add.md b/src/procedures/skynet/admin_add.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/admin_remove.md b/src/procedures/skynet/admin_remove.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/admin_ssh.md b/src/procedures/skynet/admin_ssh.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/almnui_verify.md b/src/procedures/skynet/almnui_verify.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/committee_modify.md b/src/procedures/skynet/committee_modify.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/proxmox_access.md b/src/procedures/skynet/proxmox_access.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/proxmox_reboot.md b/src/procedures/skynet/proxmox_reboot.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/server_update.md b/src/procedures/skynet/server_update.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/vaultwarden.md b/src/procedures/skynet/vaultwarden.md new file mode 100644 index 0000000..e69de29 From fb94234e40853b22b3f39967f29fe672573c1c3b Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Wed, 8 Jan 2025 23:58:50 +0000 Subject: [PATCH 09/28] doc: add about how to verify alumni --- src/procedures/skynet/almnui_verify.md | 0 src/procedures/skynet/alumni_verify.md | 95 +++++++++++++++++++ .../skynet/alumni_verify/plus_icon.png | 3 + .../skynet/alumni_verify/private_thread.png | 3 + 4 files changed, 101 insertions(+) delete mode 100644 src/procedures/skynet/almnui_verify.md create mode 100644 src/procedures/skynet/alumni_verify.md create mode 100644 src/procedures/skynet/alumni_verify/plus_icon.png create mode 100644 src/procedures/skynet/alumni_verify/private_thread.png diff --git a/src/procedures/skynet/almnui_verify.md b/src/procedures/skynet/almnui_verify.md deleted file mode 100644 index e69de29..0000000 diff --git a/src/procedures/skynet/alumni_verify.md b/src/procedures/skynet/alumni_verify.md new file mode 100644 index 0000000..e5ede7f --- /dev/null +++ b/src/procedures/skynet/alumni_verify.md @@ -0,0 +1,95 @@ +# Alumni Verification +We are an old Society (1992-now) with a long history of members staying members long after they leave UL. +In 2023 we had an outage ([see here for more information][outage_2023]) which lead to us loosing contact with many of these since their contact details were their Skynet email. +We are also required to ensure that all active Skynet accounts are linked to a UL Computer Society membership. +This put us in the catch22, where folks needed their Skynet account in order to gain access to their Skynet account. + +Many of these people want to regain access to their accounts and will make a request for help. + +## How to process Requests +These requests to gain access to an old Skynet account may come in through one of our Email addresses (``contact@skynet.ie``) or from our [Discord][discord] + +### Email +Keep an eye on the inboxes ye have access to. +Ensure that you have set up the aliases correctly (TODO: Email Aliases). +Aside from that be polite. + +### Discord +Generally users will bring up their query in ``help-and-support``. +It is recommended that you create a Private thread and ping them to bring them into it. +This ensures that no private information will leak out. + +#### How to create a Private Thread + +1. Bottom left there is a Plus icon, click on that and select the ``Thread`` option. + ![plus_icon.png](alumni_verify/plus_icon.png) +2. In teh segment that popped up tick the Private Thread checkbox. + ![private_thread.png](alumni_verify/private_thread.png) + +## The user recovers their own account. +The easiest way is that the user can recover their own account. +This works if they know their own username/password. +See [Account Recovery here][account_recovery] for more details. + +If the user's account is old enough that the password is hashed with CRYPT then tehy will be unable to reset it themselves (even if they know teh password). +This is due to CRYPT being specific to the *machine* it is on, and the LDAP server has moved several times since in the last two decades. +In which case check out the following methods. + +## Check if there is ``.forward`` +A ``.forward`` file was placed in a users home directory to forward all incoming mail to that address. + +```shell +USER="username" + +# Output a list of paths if one exists for this user +find /skynet_old/{mailconfig,home}/$USER-name '.forward' + +# use cat to output the contents to teh terminal +cat /skynet_old/mailconfig/$USER/.forward +``` + +### Usernames +One possible contents of the file is one or more Skynet usernames. + +These are not too useful for our use case. +``` +usera +userb +``` + +### Email(s) +What we are looking for is an email (identity) to tie the Skynet account to. +In which case you can check with the user that they still have access to teh email. +Get them to send you a mail from that account and verify the headers (TODO: Email headers) +``` +username@example.com +``` + +## Find a link to Personal Email +Nothing in the ``.forwards`` or they no longer have access to the email its time to see if you can tie their current identity to skynet. +This is mostly possible since many members used to send mail between their Skynet account and their private account. + +SSH into ``skynet.skynet.ie`` and run: +```shell +EMAIL="email@example.ie" +USER="username" +grep -r "$EMAIL" /skynet_old/{home,mail,mailconfig}/$USER +``` +It may take a while to run depending on the contents of these folders. +in some cases they may have proof of their ID, in which case you can search for that. + +Honestly the output of this is vibes based. +If there are forwarded email headers its a strong indication. +If a file in their ``public_html`` shows up that is also a strong indication. +The easiest one is if nothing pops up. + +Remember you are the guardian of data on Skynet, if a user cannot prove a connection you are not obligated to grant them access. + +[outage_2023]: ../../support/recovery.md +[discord]: https://discord.skynet.ie +[account_recovery]: ../../support/recovery#account-recovery + + \ No newline at end of file diff --git a/src/procedures/skynet/alumni_verify/plus_icon.png b/src/procedures/skynet/alumni_verify/plus_icon.png new file mode 100644 index 0000000..6b3616c --- /dev/null +++ b/src/procedures/skynet/alumni_verify/plus_icon.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9c8b9c84d2e446106fd7b0dea7cbc174427560889c7434a3bd78fab972f2775 +size 1602 diff --git a/src/procedures/skynet/alumni_verify/private_thread.png b/src/procedures/skynet/alumni_verify/private_thread.png new file mode 100644 index 0000000..1f9e821 --- /dev/null +++ b/src/procedures/skynet/alumni_verify/private_thread.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2b8a766d63908e1f684734f210d9bf982c23a63b281336ff468f96cae79acf9 +size 5904 From 6f8221e12df6b60e5d060823ee806ec6f32cd829 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Fri, 10 Jan 2025 23:12:56 +0000 Subject: [PATCH 10/28] doc: add how to update Nixos based servers --- src/procedures/skynet/server_update_nixos.md | 84 ++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 src/procedures/skynet/server_update_nixos.md diff --git a/src/procedures/skynet/server_update_nixos.md b/src/procedures/skynet/server_update_nixos.md new file mode 100644 index 0000000..23605fc --- /dev/null +++ b/src/procedures/skynet/server_update_nixos.md @@ -0,0 +1,84 @@ +# Updating Servers (NixOS) +A short guide on how to update NixOS servers. +This is required at least once a semester in order to keep the systems up to date. + +These are some of the easiest servers we have to update. +This is how you update it locally, and then deploy. + +## Update +Invoke the devshell for the NixOS repo locally on your computer (TODO: Setup nix and into to devshell). +This will give you all the tools you need to update the NixOS servers. + +In that shell you can run these commands. + +```shell +# this will update every input, sometimes that is not desired +nix flake update + +# This will update a single input, nixpkgs which will update the OS's +nix flake update nixpkgs colmena + +# Update multiple at once, these are the core tools and utilities +# Recommended command +nix flake update nixpkgs flake-utils agenix arion alejandra colmena +``` + +## Build +Now that the inputs have been updated it is time to build it locally to ensure no errors pop up. +For any errors that pop up you can use ``--key-result`` in order to explore the system output. + +```shell +# This will build all the servers locally +colmena build + +# Build a single one +colmena build --on glados + +# Build a logical group of them +colmena build --on @active-dns + +# Build all but keep the outputs in ``.gcroots`` +# This is useful if you need to explore the config locally +colmena build --keep-result +``` + + +## Deploy +It is now time to commit and deploy the updates. + + +### Manually +Most of the flags from the build step above apply for teh deploy step. +You dont need to have the files committed to do this, so it cna be useful for testing. + +```shell +# This will build all the servers locally +colmena deploy + +# Build a single one +colmena deploy --on glados + +# Build a logical group of them +colmena deploy --on @active-dns +``` + + +### Pipeline +To deploy via the pipeline simply commit and push to Forgejo. +The pipeline will build and deploy to almost all the servers. +Downside of this is slow iteration for testing, but it is useful if you are pretty sure of teh changes. + +#### Git Server/Runner +The CI/CD pipelines we have run on Glados and Wheatly, as such the pipelines are not able to update these two servers on their own. +These need to be updated manually using any of these commands: + +```shell +# This will deploy all the servers +colmena deploy + +# Deploy a single one +colmena deploy --on glados wheatly + +# Deploy a logical group of them +colmena deploy --on @active-git +``` \ No newline at end of file From c9980c891a43564c1a613b9f145355524b65b7dc Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Fri, 10 Jan 2025 23:13:08 +0000 Subject: [PATCH 11/28] doc: new topics to cover --- src/procedures/skynet/{server_update.md => domains_renew.md} | 0 src/procedures/skynet/server_update_procmox.md | 3 +++ 2 files changed, 3 insertions(+) rename src/procedures/skynet/{server_update.md => domains_renew.md} (100%) create mode 100644 src/procedures/skynet/server_update_procmox.md diff --git a/src/procedures/skynet/server_update.md b/src/procedures/skynet/domains_renew.md similarity index 100% rename from src/procedures/skynet/server_update.md rename to src/procedures/skynet/domains_renew.md diff --git a/src/procedures/skynet/server_update_procmox.md b/src/procedures/skynet/server_update_procmox.md new file mode 100644 index 0000000..cd5f5d6 --- /dev/null +++ b/src/procedures/skynet/server_update_procmox.md @@ -0,0 +1,3 @@ +# Updating Servers (Proxmox) +A short guide on how to update servers + From 424a7b0c2f19a8065c5d1b2c2ab9bf572551bee5 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sat, 11 Jan 2025 00:14:08 +0000 Subject: [PATCH 12/28] doc: added procedure to modify committee members --- src/procedures/skynet/committee_modify.md | 48 +++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/src/procedures/skynet/committee_modify.md b/src/procedures/skynet/committee_modify.md index e69de29..8983cac 100644 --- a/src/procedures/skynet/committee_modify.md +++ b/src/procedures/skynet/committee_modify.md @@ -0,0 +1,48 @@ +# Change Committee Members +This page covers the adding and removing of UL Computer Society committee members. + +## Adding +Before starting the committee member in question *must* have a Skynet Account. +This is because we have several services that require authentication to access. +Details on how to create one can be found [here][skynet_account_creation]. + +Once they have an account add their username to [``users.nix``][users.nix]. +Then commit and push. + +Once everything is deployed they should have access to all resources within 15 min. +If you require it sooner than that see the [Force Update](#force-update) section below. + +Next checkout the page on [VaultWarden](./vaultwarden.md) to add the user to the password manager. + +## Removing +Removing is essentially the same as adding. + +Remove their username to [``users.nix``][users.nix]. +Then commit and push. + +## Force Update +If you need to hasten an update you can log into teh server to give it a (virtual) kick. + +### SSH into Kitt +Kit is teh home of the user accounts. +```shell +ssh root@kitt.skynet.ie +``` +### Once attached run these commands +```shell +# reboot the main process +systemctl reboot skynet_ldap_backend.service + +# Update the data (this ensures that folks are current members) +systemctl reboot skynet_ldap_backend-update_data.service + +# Apply the new group roles +systemctl reboot skynet_ldap_backend-update_groups.service +``` +### Exit the terminal. +```shell +quit +``` + +[skynet_account_creation]: ../../tutorials/skynet/create_account.md +[users.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/config/users.nix#L52 \ No newline at end of file From 87f85dd314ca1dbe95f8db7e346cad600e5bb1fb Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sat, 11 Jan 2025 01:54:20 +0000 Subject: [PATCH 13/28] doc: added page on how to add/remove admins --- src/procedures/skynet/admin_add.md | 0 src/procedures/skynet/admin_modify.md | 78 +++++++++++++++++++++++ src/procedures/skynet/admin_remove.md | 0 src/procedures/skynet/committee_modify.md | 4 +- 4 files changed, 80 insertions(+), 2 deletions(-) delete mode 100644 src/procedures/skynet/admin_add.md create mode 100644 src/procedures/skynet/admin_modify.md delete mode 100644 src/procedures/skynet/admin_remove.md diff --git a/src/procedures/skynet/admin_add.md b/src/procedures/skynet/admin_add.md deleted file mode 100644 index e69de29..0000000 diff --git a/src/procedures/skynet/admin_modify.md b/src/procedures/skynet/admin_modify.md new file mode 100644 index 0000000..f7927b5 --- /dev/null +++ b/src/procedures/skynet/admin_modify.md @@ -0,0 +1,78 @@ +# Change Admins +A Skynet Admin is a user with teh highest level of access to the cluster, having been trained up over a year. +This guide will cover adding and removing an Admin. + +The normal Committee Member process applies to them as well, so check [here](./committee_modify.md#adding) for teh specific details. + +A user cannot add or remove themselves from being an admin, an existing admin must do so (one who will still be admin after the procedure). + +## Add +Steps to add a new Admin + +### NixOS +There are several steps related to the NixOS repo on Forgejo. + +#### Username Added +Like a normal committee member the users username must be added to the [``config/users.nix``][users.nix], specifically the admin section. +This gives teh user teh correct Skynet roles on our services. +This also enables them to use their personal account to login to all the servers. + +#### SSH Key +##### Root Account +There is a ``root`` account that admins can use. +To be able to login as root you have to add you key to the keys for ``root`` + +You can add it in the [``machines/_base.nix``][base.nix]. + +##### Secrets Management +We store our secrets encrypted in the repo using a tool called ``agenix``, a nix implementation of [age][age]. +In order to create/edit any secret your key has to be added to [``secrets/secrets.nix``][secrets.nix]. + +Add the key mimicking the format for the other admins. +Then add it to the users list/array. +Finally re-key the secrets + + +###### Re-key Secrets +In order to re-key the secrets your key must have been previously added (this just not work if you have just added your key, an existing admin must do this). + + +```shell +# enter the devshell +nix develop + +# Move into teh secrets folder +cd secrets + +# Re-key the secrets +agenix -R +``` + +Then commit the changes. + +### VPN +The VPN is what allows admins to get access to the internal parts of teh cluster when not on campus. +It is provided by ITD. + +1. Add an entry to the bottom of [``ITD/VPN_Admins.csv``][VPN_Admins.csv] +2. Add an entry to [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv] with ``Pending`` for teh Action. +3. Open up a TopDesk request with ITD to add the new Admin (TODO: TopDesk procedure) +4. Add the ticket reference to [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv] +5. When complete mark the entry ``Added`` on [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv]. + +## Remove +For teh most part the removal/retirement of an admin is just the opposite of the adding. +The main difference is contacting ITD to remove them from the VPN. + +### VPN +1. Open up a TopDesk request with ITD to remove the old Admin. +2. Add the ticket reference to [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv] +3. When complete mark the entry ``Removed`` on [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv]. + + +[users.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/config/users.nix#L76 +[base.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/machines/_base.nix#L75 +[secrets.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/secrets/secrets.nix#L3-L18 +[VPN_Admins.csv]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/ITD/VPN_Admins.csv +[VPN_Admins_changes.csv]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/ITD/VPN_Admins_changes.csv +[age]: https://github.com/FiloSottile/age diff --git a/src/procedures/skynet/admin_remove.md b/src/procedures/skynet/admin_remove.md deleted file mode 100644 index e69de29..0000000 diff --git a/src/procedures/skynet/committee_modify.md b/src/procedures/skynet/committee_modify.md index 8983cac..a903127 100644 --- a/src/procedures/skynet/committee_modify.md +++ b/src/procedures/skynet/committee_modify.md @@ -6,7 +6,7 @@ Before starting the committee member in question *must* have a Skynet Account. This is because we have several services that require authentication to access. Details on how to create one can be found [here][skynet_account_creation]. -Once they have an account add their username to [``users.nix``][users.nix]. +Once they have an account add their username to [``config/users.nix``][users.nix]. Then commit and push. Once everything is deployed they should have access to all resources within 15 min. @@ -17,7 +17,7 @@ Next checkout the page on [VaultWarden](./vaultwarden.md) to add the user to the ## Removing Removing is essentially the same as adding. -Remove their username to [``users.nix``][users.nix]. +Remove their username to [``config/users.nix``][users.nix]. Then commit and push. ## Force Update From c9093fdcf1fa70901551f5562bf8ca1a00fb0fd3 Mon Sep 17 00:00:00 2001 From: evanc Date: Sat, 11 Jan 2025 17:50:49 +0000 Subject: [PATCH 14/28] Explicity mention imap/smtp settings and include tutorial for sieve --- src/skynet/services/email.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/src/skynet/services/email.md b/src/skynet/services/email.md index 0db8597..0110e40 100644 --- a/src/skynet/services/email.md +++ b/src/skynet/services/email.md @@ -23,21 +23,49 @@ This work is being done with Rust. Use your Skynet ``username@skynet.ie`` and ``password`` to login. +### IMAP / SMTP settings + +Use your full Skynet email ``username@skynet.ie`` and ``password``. + +#### SMTP +Server: ``mail.skynet.ie`` +Port: ``993`` +Authentication Method: ``Normal Password`` +Connection Security: ``SSL/TLS`` + +#### IMAP +Server: ``mail.skynet.ie`` +Port: ``465`` +Authentication Method: ``Normal Password`` +Connection Security: ``SSL/TLS`` + ## Sieve Scripts Sieve scripts allow you to sort and manage yer email in a programmatic way. Recommended tool: -Small tutorial will follow later. + +* Download and open the tool. +* Enter settings for skynet. + - Hostname: ``mail.skynet.ie`` + - Port: ``4190`` + - Username: ``username@skynet.ie`` +* Hit Create. +* Hit Connect and enter your ``password``. ### ``.forwards`` replacement Since we no longer support ``.forwards`` this is the sieve script counterpart. It copies all incoming mail to the target address. +* Continuing from above +* Hit Create new script, enter a name and press edit. +* Enter code below. (You may also use the gui drag and drop builder to create the script) ```sieve require "copy"; redirect :copy "sending_mail_to@example.ie"; ``` +* Save the script. +* Go back to the home tab and press activate on your newly created script. This will enable the forwarding ### Committee/Admins There is an inbuilt sieve script that passes mail from committee/admin addresses into a folder in their Skynet inbox. From 2858674956a240a4a3c420b3e5ae19ebf4060975 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 16:29:11 +0000 Subject: [PATCH 15/28] feat: new section for procedures Got started on how to init a new server --- _git.tar.gz | 4 +- src/procedures/skynet/new_server.md | 67 +++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 2 deletions(-) create mode 100644 src/procedures/skynet/new_server.md diff --git a/_git.tar.gz b/_git.tar.gz index 1bdc667..21dcb3b 100644 --- a/_git.tar.gz +++ b/_git.tar.gz @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:607910832377319c9b05ae51ff9cfdc1eee8de4b53d422f99473720c392eaf59 -size 397421 +oid sha256:c3f36679e432c85bbd2f13f1ba064a5e7ceb8dbb2e20f9a469d9bb534b79f628 +size 401139 diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md new file mode 100644 index 0000000..fb3ae6d --- /dev/null +++ b/src/procedures/skynet/new_server.md @@ -0,0 +1,67 @@ +# New Server (LXC) + +This is the instruction guide for setting up a new LXC server. + +Rough steps are as follows: +1. Plan the servers config +2. Login to Proxmox +3. Create Container using the base LXC image +4. Grab server details +5. Push new configuration + +## Plan server Configuration +To allocate he correct resources there are a few questions that need to be asked and answered. + +1. What will this server be **For**? +2. What will its **Name** be? +3. What will its **IP** be? +4. Fill the details into the tracking sheet. + +### What is it **For**? +What a server is for dictates what hardware resources need to be allocated. +Is there one already existing that you can copy the configuration of? +Do you have prior experience with what will be hosted on it? +Is there documentation that you can use as a foundation? + +When you know these, write it down and save for later. + +### What is its **Name**? +In our cluster we have a very definite naming scheme for the servers we have. +There are two rules: + +1. The login server (where folks have their home dirs and websites) is called Skynet. + * [This can be traced back to 2007][server_name_skynet] + * > By popular demand, the skynet name was retained for the login server +2. All other servers are also named after AI's + + +In the current cluster we tend to use groups of AI names for particular functions. +For example: + +* Vigil/Vendetta were AI's in the Mass effect series, pointing Shepherd forward, so they are our DNS servers +* Glados/Wheatly from Portal, that game runs on the Source Engine, so they are our Source control servers +* Optimus/Bumblebee from Transformers, their origin is a line of toys, so fittingly our games servers. +* Neuromancer/Wintermute, from Neuromancer, each with multiple minds, thus became our backup (redundancy) servers. + +If at all possible try to get the name to match its task. +Some past names and ideas for others can be found on [the nixos wiki][server_names] + +[server_name_skynet]: https://2009.skynet.ie/history.html +[server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md + + + + +### What is its **IP** address? +We have a ``/26`` allocation, so about 60 IP's we can make use of. +Like with the names above several servers are grouped together IP wise. +Check the [Server Inventory][server_inventory] to see what addresses are available. + +[server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv + +### Tracking sheet +Now that ye have all the details about the build it is time to add them to the [Server Inventory][server_inventory]. +Add a new one, incrementing the index, and fill in the rest of the information. + +For the IP address if the last segment (ABC in this: 193.1.99.ABC) is less than 100 then add a leading ``0`` to it. +This is so that it can be easily sorted. \ No newline at end of file From f63b72a0a7f61ddfc57172825f40db23a8aae9f2 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 17:09:40 +0000 Subject: [PATCH 16/28] feat: abandon the use of the git information in the repo itself --- .githooks/post-checkout | 3 --- .githooks/post-commit | 3 --- .githooks/post-merge | 3 --- .githooks/pre-commit | 4 ---- .githooks/pre-push | 3 --- _git.tar.gz | 3 --- flake.nix | 4 +--- mkdocs.yml | 8 ++++---- 8 files changed, 5 insertions(+), 26 deletions(-) delete mode 100755 .githooks/post-checkout delete mode 100755 .githooks/post-commit delete mode 100755 .githooks/post-merge delete mode 100755 .githooks/pre-commit delete mode 100755 .githooks/pre-push delete mode 100644 _git.tar.gz diff --git a/.githooks/post-checkout b/.githooks/post-checkout deleted file mode 100755 index ca7fcb4..0000000 --- a/.githooks/post-checkout +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-checkout' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs post-checkout "$@" diff --git a/.githooks/post-commit b/.githooks/post-commit deleted file mode 100755 index 52b339c..0000000 --- a/.githooks/post-commit +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-commit' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs post-commit "$@" diff --git a/.githooks/post-merge b/.githooks/post-merge deleted file mode 100755 index a912e66..0000000 --- a/.githooks/post-merge +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'post-merge' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs post-merge "$@" diff --git a/.githooks/pre-commit b/.githooks/pre-commit deleted file mode 100755 index 99c93f8..0000000 --- a/.githooks/pre-commit +++ /dev/null @@ -1,4 +0,0 @@ -#!/usr/bin/env bash - -tar czf _git.tar.gz --exclude .git/lfs .git -git add _git.tar.gz diff --git a/.githooks/pre-push b/.githooks/pre-push deleted file mode 100755 index 0f0089b..0000000 --- a/.githooks/pre-push +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -command -v git-lfs >/dev/null 2>&1 || { echo >&2 "\nThis repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting the 'pre-push' file in the hooks directory (set by 'core.hookspath'; usually '.git/hooks').\n"; exit 2; } -git lfs pre-push "$@" diff --git a/_git.tar.gz b/_git.tar.gz deleted file mode 100644 index 21dcb3b..0000000 --- a/_git.tar.gz +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:c3f36679e432c85bbd2f13f1ba064a5e7ceb8dbb2e20f9a469d9bb534b79f628 -size 401139 diff --git a/flake.nix b/flake.nix index 31415bf..b6401ae 100644 --- a/flake.nix +++ b/flake.nix @@ -19,8 +19,6 @@ packages = with pkgs.python3Packages; [ mkdocs mkdocs-material - mkdocs-git-authors-plugin - mkdocs-git-revision-date-localized-plugin ]; in { formatter = alejandra.defaultPackage.${system}; @@ -28,7 +26,7 @@ name = "skynet-wiki"; src = self; buildInputs = packages; - buildPhase = "tar -zxf _git.tar.gz && mkdocs build"; + buildPhase = "mkdocs build"; installPhase = "mkdir -p $out; cp -R site/* $out;"; }; diff --git a/mkdocs.yml b/mkdocs.yml index 0fd35f5..35c0d7a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,10 +25,10 @@ theme: view: material/eye plugins: - search - - git-authors: - show_email_address: false - - git-revision-date-localized: - enable_creation_date: true +# - git-authors: +# show_email_address: false +# - git-revision-date-localized: +# enable_creation_date: true markdown_extensions: - tables - pymdownx.caret From 427e154332b442bba7a11d6826b95e3778259506 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 18:23:40 +0000 Subject: [PATCH 17/28] doc: added details on how to actually set up the container --- src/procedures/skynet/new_server.md | 71 ++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index fb3ae6d..d926e43 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -64,4 +64,73 @@ Now that ye have all the details about the build it is time to add them to the [ Add a new one, incrementing the index, and fill in the rest of the information. For the IP address if the last segment (ABC in this: 193.1.99.ABC) is less than 100 then add a leading ``0`` to it. -This is so that it can be easily sorted. \ No newline at end of file +This is so that it can be easily sorted. + +## Login to Proxmox + +Login to Proxmox (TODO: insert link to accessing Proxmox here) + +## Create Container Using the Base LXC Image +The Proxmox documentation for LXC's is available [here][proxmox_lxc]. + +Top right there is a button [Create CT], that brings up a window. +Each section below is one of the tabs in the window. + +### General +#### Hostname +This is the **Name** of the server, lowercase. +#### Unprivilaged +Ensure this is ticked. +#### Nesting +Ensure this is ticked. +#### SSH Public Keys +Enter the ``root`` pub ssh key. +This is used to login to teh container later. + +### Template +Select the container image, most likely ``nixos-system-x86_64-linux.tar.xz``. + +### Disks +#### Storage +Most likely it is ``main_pool``, it should have a significant amount of storage available. +**DO NOT** use ``local-zfs``, this is on Proxmox's own drive and not suited for container data. + +#### Disk Size +Self-explanatory, how much space you want to give teh container. +A minimum of 30Gb is suggested. + +### CPU +One core minimum, larger servers will require up to 6 or so. + +### Memory +Nixos will happily run on 512Mb if its load is not too intensive + +### Network +#### Bridge +The main bridge we use is ``vmbr0`` which is for most servers as it connects to the normal Skynet DMZ. + +We also have ``vmbr1`` which is for ``skynet.skynet.ie`` and connects to Skynet-EXT DMZ. +This is due to our users needing ssh access. + +#### IPv4 +##### IPv5/CIDR +This is the **IP** followed by ``/26``. +For example ``193.1.99.75/26`` + +##### Gateway +The main gateway we use is ``193.1.99.65``. +There is a secondary one for ``skynet.skynet.ie`` which is ``193.1.96.161``. + +### DNS +You can either use ``use host settings`` or fill in ``193.1.99.120`` and ``193.1.99.109``. + +### Confirm +Use this as a chance to review all the options. +There are a few gotcha's outlined above that ye do have to look over. + +#### Start after created +Tick this box if you want it to boot up immediately after being installed. + + + +[proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container \ No newline at end of file From 68875cace9e4080b82d0a76e20dea4dbae1a9ecf Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Thu, 2 Jan 2025 18:24:32 +0000 Subject: [PATCH 18/28] fmt: little bit of re-organisation --- src/procedures/skynet/new_server.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index d926e43..38dee07 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -46,19 +46,12 @@ For example: If at all possible try to get the name to match its task. Some past names and ideas for others can be found on [the nixos wiki][server_names] -[server_name_skynet]: https://2009.skynet.ie/history.html -[server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md - - - ### What is its **IP** address? We have a ``/26`` allocation, so about 60 IP's we can make use of. Like with the names above several servers are grouped together IP wise. Check the [Server Inventory][server_inventory] to see what addresses are available. -[server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv - ### Tracking sheet Now that ye have all the details about the build it is time to add them to the [Server Inventory][server_inventory]. Add a new one, incrementing the index, and fill in the rest of the information. @@ -79,7 +72,7 @@ Each section below is one of the tabs in the window. ### General #### Hostname This is the **Name** of the server, lowercase. -#### Unprivilaged +#### Unprivileged Ensure this is ticked. #### Nesting Ensure this is ticked. @@ -133,4 +126,9 @@ Tick this box if you want it to boot up immediately after being installed. + + +[server_name_skynet]: https://2009.skynet.ie/history.html +[server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md +[server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv [proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container \ No newline at end of file From 32a4086ff64ebfaa7bbc3613b4786da5c0d6b6c0 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 15:43:21 +0000 Subject: [PATCH 19/28] feat: login to teh server step completed --- src/procedures/skynet/new_server.md | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index 38dee07..b347178 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -6,7 +6,7 @@ Rough steps are as follows: 1. Plan the servers config 2. Login to Proxmox 3. Create Container using the base LXC image -4. Grab server details +4. Login to Server 5. Push new configuration ## Plan server Configuration @@ -124,11 +124,29 @@ There are a few gotcha's outlined above that ye do have to look over. #### Start after created Tick this box if you want it to boot up immediately after being installed. +## Login to Server +Now that the server is up and running it is time to login to it. +Assuming you have your SSH configured like (TODO: admin ssh config). +You just have to use ``ssh root@IP`` (for example ``ssh root@192.99.1.111``) +### Getting the server ssh key +We are logging in because we need to get the servers own ssh key. +You can find it in ``/etc/ssh``. +You have a choice between ``ssh_host_ed25519_key.pub`` and ``ssh_host_rsa_key.pub``. +``ssh_host_ed25519_key.pub`` is the recommended one. + +### Using the server ssh key +This key is used to decrypt secrets stored in our Nixos repo and as such needs to be added there. +1. Add the key with the other system keys [here][nixos_secrets]. +2. Add it to the systems array underneath that. +3. In the ``secrets`` folder run ``cd secrets && agenix -r`` to rekey the secrets. + * This is to give the new server access. +4. Commit all the changed files [server_name_skynet]: https://2009.skynet.ie/history.html [server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md [server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv -[proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container \ No newline at end of file +[proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container +[nixos_secrets]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/secrets/secrets.nix#L35 \ No newline at end of file From 9e1677dbc2dd258fabb7534db0ed26971bcdd098 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 19:42:43 +0000 Subject: [PATCH 20/28] doc: added segments on how to deploy the config --- src/procedures/skynet/new_server.md | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/new_server.md index b347178..7ec7a89 100644 --- a/src/procedures/skynet/new_server.md +++ b/src/procedures/skynet/new_server.md @@ -144,9 +144,35 @@ This key is used to decrypt secrets stored in our Nixos repo and as such needs t 4. Commit all the changed files +## Push new configuration +### Create config +An example server config for nixos is [available here][nixos_template]. +Copy it to a new file and name it ``name.nix``. +Fill in all relevant details (name/ip/name details/its purpose/...). + +Finally add it to the bottom of the [``flake.nix``][nixos_flake]. +Save and commit the files. + +### Push Config +There are two ways to test out the config, manual and pipeline. +Each has pros and cons. + +#### Manually +This requires your key to be added the the keys for the [root account][nixos_root] and your ``~/.ssh/config`` to be set up properly (TODO: SSH tutorial) + +1. Build it with ``colmena build --on name`` +2. If it builds ye can test deployment +3. Deploy it using ``colmena apply --on name`` + +#### Pipeline +Just push it to the repo and the pipeline will handle building and deployment. +The disadvantage of this is you cannot fix any mistakes before they are pushed. [server_name_skynet]: https://2009.skynet.ie/history.html [server_names]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/Possible_Server_Names.md [server_inventory]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/ITD/Server_Inventory.csv [proxmox_lxc]: https://pve.proxmox.com/wiki/Linux_Container -[nixos_secrets]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/secrets/secrets.nix#L35 \ No newline at end of file +[nixos_secrets]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/secrets/secrets.nix#L35 +[nixos_template]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/machines/_template.nix +[nixos_flake]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/flake.nix#L156 +[nixos_root]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/machines/_base.nix#L75 \ No newline at end of file From 29c72f8c92e3f751a14fa1a787ae1975caff931c Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 19:47:14 +0000 Subject: [PATCH 21/28] fmt: rename --- src/procedures/skynet/{new_server.md => server_new.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/procedures/skynet/{new_server.md => server_new.md} (100%) diff --git a/src/procedures/skynet/new_server.md b/src/procedures/skynet/server_new.md similarity index 100% rename from src/procedures/skynet/new_server.md rename to src/procedures/skynet/server_new.md From 6df7d4221b71b6de70214201504b7217e273427a Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sun, 5 Jan 2025 19:49:40 +0000 Subject: [PATCH 22/28] doc: planned documentation --- src/procedures/skynet/admin_add.md | 0 src/procedures/skynet/admin_remove.md | 0 src/procedures/skynet/admin_ssh.md | 0 src/procedures/skynet/almnui_verify.md | 0 src/procedures/skynet/committee_modify.md | 0 src/procedures/skynet/proxmox_access.md | 0 src/procedures/skynet/proxmox_reboot.md | 0 src/procedures/skynet/server_update.md | 0 src/procedures/skynet/vaultwarden.md | 0 9 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 src/procedures/skynet/admin_add.md create mode 100644 src/procedures/skynet/admin_remove.md create mode 100644 src/procedures/skynet/admin_ssh.md create mode 100644 src/procedures/skynet/almnui_verify.md create mode 100644 src/procedures/skynet/committee_modify.md create mode 100644 src/procedures/skynet/proxmox_access.md create mode 100644 src/procedures/skynet/proxmox_reboot.md create mode 100644 src/procedures/skynet/server_update.md create mode 100644 src/procedures/skynet/vaultwarden.md diff --git a/src/procedures/skynet/admin_add.md b/src/procedures/skynet/admin_add.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/admin_remove.md b/src/procedures/skynet/admin_remove.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/admin_ssh.md b/src/procedures/skynet/admin_ssh.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/almnui_verify.md b/src/procedures/skynet/almnui_verify.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/committee_modify.md b/src/procedures/skynet/committee_modify.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/proxmox_access.md b/src/procedures/skynet/proxmox_access.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/proxmox_reboot.md b/src/procedures/skynet/proxmox_reboot.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/server_update.md b/src/procedures/skynet/server_update.md new file mode 100644 index 0000000..e69de29 diff --git a/src/procedures/skynet/vaultwarden.md b/src/procedures/skynet/vaultwarden.md new file mode 100644 index 0000000..e69de29 From 9dd4362a18bb66177a0fc923a22e76a73c188caa Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Wed, 8 Jan 2025 23:58:50 +0000 Subject: [PATCH 23/28] doc: add about how to verify alumni --- src/procedures/skynet/almnui_verify.md | 0 src/procedures/skynet/alumni_verify.md | 95 +++++++++++++++++++ .../skynet/alumni_verify/plus_icon.png | 3 + .../skynet/alumni_verify/private_thread.png | 3 + 4 files changed, 101 insertions(+) delete mode 100644 src/procedures/skynet/almnui_verify.md create mode 100644 src/procedures/skynet/alumni_verify.md create mode 100644 src/procedures/skynet/alumni_verify/plus_icon.png create mode 100644 src/procedures/skynet/alumni_verify/private_thread.png diff --git a/src/procedures/skynet/almnui_verify.md b/src/procedures/skynet/almnui_verify.md deleted file mode 100644 index e69de29..0000000 diff --git a/src/procedures/skynet/alumni_verify.md b/src/procedures/skynet/alumni_verify.md new file mode 100644 index 0000000..e5ede7f --- /dev/null +++ b/src/procedures/skynet/alumni_verify.md @@ -0,0 +1,95 @@ +# Alumni Verification +We are an old Society (1992-now) with a long history of members staying members long after they leave UL. +In 2023 we had an outage ([see here for more information][outage_2023]) which lead to us loosing contact with many of these since their contact details were their Skynet email. +We are also required to ensure that all active Skynet accounts are linked to a UL Computer Society membership. +This put us in the catch22, where folks needed their Skynet account in order to gain access to their Skynet account. + +Many of these people want to regain access to their accounts and will make a request for help. + +## How to process Requests +These requests to gain access to an old Skynet account may come in through one of our Email addresses (``contact@skynet.ie``) or from our [Discord][discord] + +### Email +Keep an eye on the inboxes ye have access to. +Ensure that you have set up the aliases correctly (TODO: Email Aliases). +Aside from that be polite. + +### Discord +Generally users will bring up their query in ``help-and-support``. +It is recommended that you create a Private thread and ping them to bring them into it. +This ensures that no private information will leak out. + +#### How to create a Private Thread + +1. Bottom left there is a Plus icon, click on that and select the ``Thread`` option. + ![plus_icon.png](alumni_verify/plus_icon.png) +2. In teh segment that popped up tick the Private Thread checkbox. + ![private_thread.png](alumni_verify/private_thread.png) + +## The user recovers their own account. +The easiest way is that the user can recover their own account. +This works if they know their own username/password. +See [Account Recovery here][account_recovery] for more details. + +If the user's account is old enough that the password is hashed with CRYPT then tehy will be unable to reset it themselves (even if they know teh password). +This is due to CRYPT being specific to the *machine* it is on, and the LDAP server has moved several times since in the last two decades. +In which case check out the following methods. + +## Check if there is ``.forward`` +A ``.forward`` file was placed in a users home directory to forward all incoming mail to that address. + +```shell +USER="username" + +# Output a list of paths if one exists for this user +find /skynet_old/{mailconfig,home}/$USER-name '.forward' + +# use cat to output the contents to teh terminal +cat /skynet_old/mailconfig/$USER/.forward +``` + +### Usernames +One possible contents of the file is one or more Skynet usernames. + +These are not too useful for our use case. +``` +usera +userb +``` + +### Email(s) +What we are looking for is an email (identity) to tie the Skynet account to. +In which case you can check with the user that they still have access to teh email. +Get them to send you a mail from that account and verify the headers (TODO: Email headers) +``` +username@example.com +``` + +## Find a link to Personal Email +Nothing in the ``.forwards`` or they no longer have access to the email its time to see if you can tie their current identity to skynet. +This is mostly possible since many members used to send mail between their Skynet account and their private account. + +SSH into ``skynet.skynet.ie`` and run: +```shell +EMAIL="email@example.ie" +USER="username" +grep -r "$EMAIL" /skynet_old/{home,mail,mailconfig}/$USER +``` +It may take a while to run depending on the contents of these folders. +in some cases they may have proof of their ID, in which case you can search for that. + +Honestly the output of this is vibes based. +If there are forwarded email headers its a strong indication. +If a file in their ``public_html`` shows up that is also a strong indication. +The easiest one is if nothing pops up. + +Remember you are the guardian of data on Skynet, if a user cannot prove a connection you are not obligated to grant them access. + +[outage_2023]: ../../support/recovery.md +[discord]: https://discord.skynet.ie +[account_recovery]: ../../support/recovery#account-recovery + + \ No newline at end of file diff --git a/src/procedures/skynet/alumni_verify/plus_icon.png b/src/procedures/skynet/alumni_verify/plus_icon.png new file mode 100644 index 0000000..6b3616c --- /dev/null +++ b/src/procedures/skynet/alumni_verify/plus_icon.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9c8b9c84d2e446106fd7b0dea7cbc174427560889c7434a3bd78fab972f2775 +size 1602 diff --git a/src/procedures/skynet/alumni_verify/private_thread.png b/src/procedures/skynet/alumni_verify/private_thread.png new file mode 100644 index 0000000..1f9e821 --- /dev/null +++ b/src/procedures/skynet/alumni_verify/private_thread.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2b8a766d63908e1f684734f210d9bf982c23a63b281336ff468f96cae79acf9 +size 5904 From 4e3ca7c89640c59a05e4c8f08c97b1cac8da708a Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Fri, 10 Jan 2025 23:12:56 +0000 Subject: [PATCH 24/28] doc: add how to update Nixos based servers --- src/procedures/skynet/server_update_nixos.md | 84 ++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 src/procedures/skynet/server_update_nixos.md diff --git a/src/procedures/skynet/server_update_nixos.md b/src/procedures/skynet/server_update_nixos.md new file mode 100644 index 0000000..23605fc --- /dev/null +++ b/src/procedures/skynet/server_update_nixos.md @@ -0,0 +1,84 @@ +# Updating Servers (NixOS) +A short guide on how to update NixOS servers. +This is required at least once a semester in order to keep the systems up to date. + +These are some of the easiest servers we have to update. +This is how you update it locally, and then deploy. + +## Update +Invoke the devshell for the NixOS repo locally on your computer (TODO: Setup nix and into to devshell). +This will give you all the tools you need to update the NixOS servers. + +In that shell you can run these commands. + +```shell +# this will update every input, sometimes that is not desired +nix flake update + +# This will update a single input, nixpkgs which will update the OS's +nix flake update nixpkgs colmena + +# Update multiple at once, these are the core tools and utilities +# Recommended command +nix flake update nixpkgs flake-utils agenix arion alejandra colmena +``` + +## Build +Now that the inputs have been updated it is time to build it locally to ensure no errors pop up. +For any errors that pop up you can use ``--key-result`` in order to explore the system output. + +```shell +# This will build all the servers locally +colmena build + +# Build a single one +colmena build --on glados + +# Build a logical group of them +colmena build --on @active-dns + +# Build all but keep the outputs in ``.gcroots`` +# This is useful if you need to explore the config locally +colmena build --keep-result +``` + + +## Deploy +It is now time to commit and deploy the updates. + + +### Manually +Most of the flags from the build step above apply for teh deploy step. +You dont need to have the files committed to do this, so it cna be useful for testing. + +```shell +# This will build all the servers locally +colmena deploy + +# Build a single one +colmena deploy --on glados + +# Build a logical group of them +colmena deploy --on @active-dns +``` + + +### Pipeline +To deploy via the pipeline simply commit and push to Forgejo. +The pipeline will build and deploy to almost all the servers. +Downside of this is slow iteration for testing, but it is useful if you are pretty sure of teh changes. + +#### Git Server/Runner +The CI/CD pipelines we have run on Glados and Wheatly, as such the pipelines are not able to update these two servers on their own. +These need to be updated manually using any of these commands: + +```shell +# This will deploy all the servers +colmena deploy + +# Deploy a single one +colmena deploy --on glados wheatly + +# Deploy a logical group of them +colmena deploy --on @active-git +``` \ No newline at end of file From 9240938c3bcfa4dfca4450c3ca6e3dc678643346 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Fri, 10 Jan 2025 23:13:08 +0000 Subject: [PATCH 25/28] doc: new topics to cover --- src/procedures/skynet/{server_update.md => domains_renew.md} | 0 src/procedures/skynet/server_update_procmox.md | 3 +++ 2 files changed, 3 insertions(+) rename src/procedures/skynet/{server_update.md => domains_renew.md} (100%) create mode 100644 src/procedures/skynet/server_update_procmox.md diff --git a/src/procedures/skynet/server_update.md b/src/procedures/skynet/domains_renew.md similarity index 100% rename from src/procedures/skynet/server_update.md rename to src/procedures/skynet/domains_renew.md diff --git a/src/procedures/skynet/server_update_procmox.md b/src/procedures/skynet/server_update_procmox.md new file mode 100644 index 0000000..cd5f5d6 --- /dev/null +++ b/src/procedures/skynet/server_update_procmox.md @@ -0,0 +1,3 @@ +# Updating Servers (Proxmox) +A short guide on how to update servers + From 976907c5d4308559b0555a62722224badc15f559 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sat, 11 Jan 2025 00:14:08 +0000 Subject: [PATCH 26/28] doc: added procedure to modify committee members --- src/procedures/skynet/committee_modify.md | 48 +++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/src/procedures/skynet/committee_modify.md b/src/procedures/skynet/committee_modify.md index e69de29..8983cac 100644 --- a/src/procedures/skynet/committee_modify.md +++ b/src/procedures/skynet/committee_modify.md @@ -0,0 +1,48 @@ +# Change Committee Members +This page covers the adding and removing of UL Computer Society committee members. + +## Adding +Before starting the committee member in question *must* have a Skynet Account. +This is because we have several services that require authentication to access. +Details on how to create one can be found [here][skynet_account_creation]. + +Once they have an account add their username to [``users.nix``][users.nix]. +Then commit and push. + +Once everything is deployed they should have access to all resources within 15 min. +If you require it sooner than that see the [Force Update](#force-update) section below. + +Next checkout the page on [VaultWarden](./vaultwarden.md) to add the user to the password manager. + +## Removing +Removing is essentially the same as adding. + +Remove their username to [``users.nix``][users.nix]. +Then commit and push. + +## Force Update +If you need to hasten an update you can log into teh server to give it a (virtual) kick. + +### SSH into Kitt +Kit is teh home of the user accounts. +```shell +ssh root@kitt.skynet.ie +``` +### Once attached run these commands +```shell +# reboot the main process +systemctl reboot skynet_ldap_backend.service + +# Update the data (this ensures that folks are current members) +systemctl reboot skynet_ldap_backend-update_data.service + +# Apply the new group roles +systemctl reboot skynet_ldap_backend-update_groups.service +``` +### Exit the terminal. +```shell +quit +``` + +[skynet_account_creation]: ../../tutorials/skynet/create_account.md +[users.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/branch/main/config/users.nix#L52 \ No newline at end of file From 8f84904ba81ebb6a55a2aad1fdd274c7c3211f03 Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Sat, 11 Jan 2025 01:54:20 +0000 Subject: [PATCH 27/28] doc: added page on how to add/remove admins --- src/procedures/skynet/admin_add.md | 0 src/procedures/skynet/admin_modify.md | 78 +++++++++++++++++++++++ src/procedures/skynet/admin_remove.md | 0 src/procedures/skynet/committee_modify.md | 4 +- 4 files changed, 80 insertions(+), 2 deletions(-) delete mode 100644 src/procedures/skynet/admin_add.md create mode 100644 src/procedures/skynet/admin_modify.md delete mode 100644 src/procedures/skynet/admin_remove.md diff --git a/src/procedures/skynet/admin_add.md b/src/procedures/skynet/admin_add.md deleted file mode 100644 index e69de29..0000000 diff --git a/src/procedures/skynet/admin_modify.md b/src/procedures/skynet/admin_modify.md new file mode 100644 index 0000000..f7927b5 --- /dev/null +++ b/src/procedures/skynet/admin_modify.md @@ -0,0 +1,78 @@ +# Change Admins +A Skynet Admin is a user with teh highest level of access to the cluster, having been trained up over a year. +This guide will cover adding and removing an Admin. + +The normal Committee Member process applies to them as well, so check [here](./committee_modify.md#adding) for teh specific details. + +A user cannot add or remove themselves from being an admin, an existing admin must do so (one who will still be admin after the procedure). + +## Add +Steps to add a new Admin + +### NixOS +There are several steps related to the NixOS repo on Forgejo. + +#### Username Added +Like a normal committee member the users username must be added to the [``config/users.nix``][users.nix], specifically the admin section. +This gives teh user teh correct Skynet roles on our services. +This also enables them to use their personal account to login to all the servers. + +#### SSH Key +##### Root Account +There is a ``root`` account that admins can use. +To be able to login as root you have to add you key to the keys for ``root`` + +You can add it in the [``machines/_base.nix``][base.nix]. + +##### Secrets Management +We store our secrets encrypted in the repo using a tool called ``agenix``, a nix implementation of [age][age]. +In order to create/edit any secret your key has to be added to [``secrets/secrets.nix``][secrets.nix]. + +Add the key mimicking the format for the other admins. +Then add it to the users list/array. +Finally re-key the secrets + + +###### Re-key Secrets +In order to re-key the secrets your key must have been previously added (this just not work if you have just added your key, an existing admin must do this). + + +```shell +# enter the devshell +nix develop + +# Move into teh secrets folder +cd secrets + +# Re-key the secrets +agenix -R +``` + +Then commit the changes. + +### VPN +The VPN is what allows admins to get access to the internal parts of teh cluster when not on campus. +It is provided by ITD. + +1. Add an entry to the bottom of [``ITD/VPN_Admins.csv``][VPN_Admins.csv] +2. Add an entry to [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv] with ``Pending`` for teh Action. +3. Open up a TopDesk request with ITD to add the new Admin (TODO: TopDesk procedure) +4. Add the ticket reference to [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv] +5. When complete mark the entry ``Added`` on [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv]. + +## Remove +For teh most part the removal/retirement of an admin is just the opposite of the adding. +The main difference is contacting ITD to remove them from the VPN. + +### VPN +1. Open up a TopDesk request with ITD to remove the old Admin. +2. Add the ticket reference to [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv] +3. When complete mark the entry ``Removed`` on [``ITD/VPN_Admins_changes.csv``][VPN_Admins_changes.csv]. + + +[users.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/config/users.nix#L76 +[base.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/machines/_base.nix#L75 +[secrets.nix]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/secrets/secrets.nix#L3-L18 +[VPN_Admins.csv]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/ITD/VPN_Admins.csv +[VPN_Admins_changes.csv]: https://forgejo.skynet.ie/Skynet/nixos/src/commit/b46eca16b0b207d14e173d4e40286160749c5c07/ITD/VPN_Admins_changes.csv +[age]: https://github.com/FiloSottile/age diff --git a/src/procedures/skynet/admin_remove.md b/src/procedures/skynet/admin_remove.md deleted file mode 100644 index e69de29..0000000 diff --git a/src/procedures/skynet/committee_modify.md b/src/procedures/skynet/committee_modify.md index 8983cac..a903127 100644 --- a/src/procedures/skynet/committee_modify.md +++ b/src/procedures/skynet/committee_modify.md @@ -6,7 +6,7 @@ Before starting the committee member in question *must* have a Skynet Account. This is because we have several services that require authentication to access. Details on how to create one can be found [here][skynet_account_creation]. -Once they have an account add their username to [``users.nix``][users.nix]. +Once they have an account add their username to [``config/users.nix``][users.nix]. Then commit and push. Once everything is deployed they should have access to all resources within 15 min. @@ -17,7 +17,7 @@ Next checkout the page on [VaultWarden](./vaultwarden.md) to add the user to the ## Removing Removing is essentially the same as adding. -Remove their username to [``users.nix``][users.nix]. +Remove their username to [``config/users.nix``][users.nix]. Then commit and push. ## Force Update From f932fbe504f70ad292e96ac16b175e0264a9054f Mon Sep 17 00:00:00 2001 From: Brendan Golden Date: Mon, 13 Jan 2025 13:38:57 +0000 Subject: [PATCH 28/28] doc: added how to find information on updating proxmox --- src/procedures/skynet/server_update_procmox.md | 3 --- src/procedures/skynet/server_update_proxmox.md | 16 ++++++++++++++++ 2 files changed, 16 insertions(+), 3 deletions(-) delete mode 100644 src/procedures/skynet/server_update_procmox.md create mode 100644 src/procedures/skynet/server_update_proxmox.md diff --git a/src/procedures/skynet/server_update_procmox.md b/src/procedures/skynet/server_update_procmox.md deleted file mode 100644 index cd5f5d6..0000000 --- a/src/procedures/skynet/server_update_procmox.md +++ /dev/null @@ -1,3 +0,0 @@ -# Updating Servers (Proxmox) -A short guide on how to update servers - diff --git a/src/procedures/skynet/server_update_proxmox.md b/src/procedures/skynet/server_update_proxmox.md new file mode 100644 index 0000000..bf1ecab --- /dev/null +++ b/src/procedures/skynet/server_update_proxmox.md @@ -0,0 +1,16 @@ +# Updating Servers (Proxmox) +A short guide on how to update servers running Proxmox VM Host + +There is documentation for Proxmox available on teh [web][docs_web] and locally on teh server [itself][docs_inbuilt]. +The inbuilt ones are preferred since they match teh version of Proxmox we are using. +In both there is a chapter called [``System Software Updates``][docs_inbuilt_update] which tells you how to update. + +When there is a major version released there will be a new link on [this page][docs_wiki] in teh format of: +``Upgrade from $VersionCurrent to $VersionNext`` + + + +[docs_inbuilt]: https://193.1.99.73:8006/pve-docs/pve-admin-guide.html +[docs_inbuilt_update]: https://193.1.99.73:8006/pve-docs/pve-admin-guide.html#system_software_updates +[docs_web]: https://pve.proxmox.com/pve-docs/pve-admin-guide.html +[docs_wiki]: https://pve.proxmox.com/wiki/Category:Upgrade \ No newline at end of file