hscloud

VMs are coming any day now ~q3k, A.D. 2018

Our new internal highly-available Infrastructure/Platform-as-a-Service.

This runs in our datacenter (dcr01 on netbox). This is different from our ISP services or internal machines.

Currently hscloud is made up of a Kubernetes cluster named k0.hswaw.net. It runs on the following machines: bc01n01, bc01n02, bc01n03, dcr01s22, dcr01s24. In total we have 328GB of RAM and 144 x86 cores. We also have a half PB of storage on old SAS drives (most of it currently cold), accessible via Ceph (radosgw or Kubernets PersistentVolumes).

We are moving services from our old machines into Kubernetes. Amongst other, currently running on the cluster is:

• https://matrix.hackerspace.pl
• https://gerrit.hackerspace.pl
• https://profile.hackerspace.pl
• https://cfp.cebula.camp

With more to come.

You are also free to host your own personal stuff there within reason. See below for access.

Here's a list of services that currently live on Boston Packets, but we'd like to migrate to hscloud. Ask on #infra on how to contribute.

• https://kasownik.hackerspace.pl
• https://owncloud.hackerspace.pl
• https://wiki.hackerspace.pl - dokuwiki, as of 2023-10, on noisersup's todo list
• https://blog.hackerspace.pl - wordpress, not actively maintained
• https://gallery.hackerspace.pl - unmaintained for ages
• https://webchat.hackerspace.pl - just a redirect?
• https://piwik.hackerspace.pl - kill?
• https://static.hackerspace.pl - ???
• https://api.hackerspace.pl - dead? kill?
• https://tickers.hackerspace.pl - dead? kill?
• https://mail.hackerspace.pl - roundcube, fairly easy, but boston nginx would still serve as proxy (analogous to hackerspace.pl site)
• mailman (https://lists.hackerspace.pl) - the web service is already on k8s, but also proxied via boston. Mailman-core and the database (used by both web and core, postgres) are still on boston.
• ldap/kerberos - (hard)
• email services (exim, dovecot) - (hard)

https://monitoring-global-dashboard.k0.hswaw.net/

Self-documenting in hackdoc (hscloud documentation stored within hscloud): https://hackdoc.hackerspace.pl/doc/codelabs/index.md

TBD, WIP: You need to describe a deployment of your docker image in the same way as it's done for https://cs.hackerspace.pl/hscloud/-/tree/hswaw/paperless .

Build your docker image by running following command:

docker build --tag registry.k0.hswaw.net/$YOUR_USERNAME/$APP_NAME-$APP_VERSION

Get your login credentials by going to this site https://registry.k0.hswaw.net/ and authenticating via SSO. You should get a command by which you'll login to our docker registry.

Afterwards, push your image by using

docker push registry.k0.hswaw.net/$YOUR_USERNAME/$YOUR_IMAGE_TAG

docker build --tag registry.k0.hswaw.net/palid/walne-generator:1.0-alpha
docker push registry.k0.hswaw.net/palid/walne-generator:1.0-alpha

Here is a list of common external/internal services used by apps hosted in hscloud, with guidelines on how to get access to those:

• Persistent storage/Block storage
  • Use PersistentVolumeClaim in waw-hdd-redundant-3 storage class
• S3/Object storage
  • Add user object in //cluster/kube/k0.libsonnet:k0.ceph.clients
  • Ask hscloud ops to update
• CockroachDB
  • Add user object in //cluster/kube/k0.libsonnet:k0.cockroach.clients
  • Ask hscloud ops to update
• Docker Container Registry
  • Use https://registry.k0.hswaw.net to authenticate
  • registry.k0.hswaw.net/USERNAME/… is your personal container namespace
• DNS
  • *.hackerspace.pl, *.hswaw.net: Ask hscloud ops to create/update relevant DNS entries in ns{1,2}.bytesexual.net, adjust admitomatic config (see below)
  • *: create your own DNS CNAME record pointing at ingress.k0.hswaw.net
  • Adjust //cluster/kube/k0.libsonnet:k0.admitomatic.cfg.proto.allow_domain if you want your domain to be secured against hijacking by other cluster users (*.hackerspace.pl is one such domain)
• Postgres
  • Use //kube/postgres.libsonnet to create local deployment in app namespace
  • Alternative: ask hscloud ops for a database on blessed high-performance ssd node
• Redis
  • Use //kube/redis.libsonnet to create local deployment in app namespace
• SSO (OAuth2/OpenID Connect for HSWAW members authentication)
  • Self-service - create your own app on https://sso.hackerspace.pl
• LDAP (only very specific cases, when user/group listing is required - otherwise use SSO)
  • Ask ops to create an LDAP service account
    • Create cn=…,ou=Services,dc=hackerspace,dc=pl in LDAP
    • Add relevant ACL in /etc/openldap/slapd.conf on boston-packets.hackerspace.pl
• Beyondspace (access to *.waw.hackerspace.pl services from WAN/hscloud)
  • Ask ops to add specific internal domain to beyondspace (//hswaw/machines/customs.hackerspace.pl/beyondspace.nix) and create a service authentication token on customs
• Mailing (SMTP/IMAP)
  • Ask ops to create local mailing user account
    • Create local service user on boston-packets.hackerspace.pl: useradd -rm SERVICE; passwd SERVICE
    • Create mailbox on boston-packets.hackerspace.pl: mkdir /var/spool/mail/SERVICE; chown SERVICE:mail /var/spool/mail/SERVICE
    • Optionally: add aliases (and/or incoming exec hook) in /etc/mail/aliases